Files
seaweedfs/weed/s3api/s3_constants/header_test.go
T
ad89f003a5 s3: end a catalog-vended session with the access key it was vended for (#11945)
* s3: end a catalog-vended session with the access key it was vended for

A table credential the Iceberg catalog vends is a self-contained STS session:
it stays valid until it expires (an hour by default) even after the access key
that loaded the table is deleted or deactivated, or its identity is removed
or disabled. An operator revoking a leaked key cannot cut off the table
credentials that key already obtained.

Sessions are stateless JWTs, so there is no revocation list to consult. This
change follows the same pattern as role sessions, which are already refused
once their role's ID no longer matches (checkSessionRoleBinding): a vended
session carries the access key it was vended for (src_ak, recorded by the
previous change), and validateSTSSessionToken checks it against the identity
table held in memory. A session of provider iceberg-catalog that carries a
source key is refused (InvalidAccessKeyId, as for an unknown key) once its
identity is gone or disabled, or the key is no longer one of that identity's
active, unexpired credentials.

Scope:
- Only catalog-vended sessions are checked; web-identity, AssumeRole and
  every other STS session are validated as before.
- The vendor binds a session to the caller's key only when the caller is a
  stored identity holding that key. A caller that signed with another STS
  session's key, an OIDC user, or a request that recorded no key gets an
  unbound session, as today; binding it would refuse every request made with
  it. A session minted before this change carries no key and is unbound.
- The check is one map lookup under the read lock and a scan of one
  identity's keys (about 11 ns, 0 allocations, with 1,000 identities).

Tests (s3api_iceberg_vended_session_test.go) mint real sessions through the
catalog vendor and validate them the way a data-path request is validated:
deactivating, expiring or deleting key A ends A's session while the session
vended through key B on the same identity keeps working; disabling or
deleting the identity ends it; callers that are not a stored identity holding
the key get unbound sessions that keep working; other STS sessions are not
checked. The first two tests and the third fail without this change.

* s3: keep the vended-session key binding when the key dies before vend

vendedSessionKeyLive gated the binding, so a key revoked between the
request's authentication and the vend skipped it and minted an unbound
session that stayed usable until it expired. Every catalog auth path
pairs the identity name in context with the key that authenticated, so
a stored-identity principal always means the request signed with that
identity's own key. Bind whenever the principal is a stored identity: a
session bound to a now-dead key is refused on its first request, which
is what the binding is for.

* s3: gate the vended-session key binding on the authenticated caller type

lookupByIdentityName(principal) named only that some identity exists, not
that the request signed with a stored key: an STS session or OIDC principal
that collides with an identity name got a session bound to its temporary
key, which vendedSessionKeyLive then refused on every request. Verifiers
now record the verified key as a stored credential, and the vendor binds
only when that marker is present — keeping the binding even when the key
was revoked between authentication and the vend.

* s3: enforce the vended key binding under a dedicated claim

Sessions minted before the binding existed carry the caller's key in the
audit-only SourceAccessKeyId — including temporary STS keys that can never
resolve in the identity table. Enforcing on that field would end live table
credentials at upgrade; the binding now travels under BoundAccessKeyId,
leaving legacy sessions unbound as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: bind vended sessions to a credential fingerprint

The key id alone cannot identify a credential: an operator can delete a
key and re-create it under the same id with a different secret, reviving
sessions minted for the original. Verifiers now record a fingerprint of
the secret at authentication; the session dies unless it still matches.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: keep recording the caller's key on every catalog session

SourceAccessKeyId was unconditional audit metadata before the binding
existed; gating it on the stored-credential marker dropped the record
for callers that authenticated another way. With the binding under its
own claim there is no ambiguity left, so the field records the caller's
key again regardless of how it authenticated.

* s3: key the vended-session credential fingerprint with the signing key

The bound-credential fingerprint travels inside the vended session
token. As a plain SHA-256 of the secret it gave any token holder an
offline guessing oracle for weak secrets, and CodeQL flags it twice.

Store the verified secret in the request marker instead and HMAC it
with the STS signing key at vend and at validation. The claim can no
longer be mined, and the per-request auth path stops hashing secrets.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 03:58:06 -07:00

333 lines
9.4 KiB
Go

package s3_constants
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestNormalizeObjectKey(t *testing.T) {
tests := []struct {
name string
input string
expected string
}{
{
name: "simple key",
input: "file.txt",
expected: "file.txt",
},
{
name: "key with leading slash",
input: "/file.txt",
expected: "file.txt",
},
{
name: "key with directory",
input: "folder/file.txt",
expected: "folder/file.txt",
},
{
name: "key with leading slash and directory",
input: "/folder/file.txt",
expected: "folder/file.txt",
},
{
name: "key with duplicate slashes",
input: "folder//subfolder///file.txt",
expected: "folder/subfolder/file.txt",
},
{
name: "Windows backslash - simple",
input: "folder\\file.txt",
expected: "folder/file.txt",
},
{
name: "Windows backslash - nested",
input: "folder\\subfolder\\file.txt",
expected: "folder/subfolder/file.txt",
},
{
name: "Windows backslash - with leading slash",
input: "/folder\\subfolder\\file.txt",
expected: "folder/subfolder/file.txt",
},
{
name: "mixed slashes",
input: "folder\\subfolder/another\\file.txt",
expected: "folder/subfolder/another/file.txt",
},
{
name: "Windows full path style (edge case)",
input: "C:\\Users\\test\\file.txt",
expected: "C:/Users/test/file.txt",
},
{
name: "empty string",
input: "",
expected: "",
},
{
name: "just a slash",
input: "/",
expected: "",
},
{
name: "just a backslash",
input: "\\",
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := NormalizeObjectKey(tt.input)
if result != tt.expected {
t.Errorf("NormalizeObjectKey(%q) = %q, want %q", tt.input, result, tt.expected)
}
})
}
}
func TestIsValidObjectKey(t *testing.T) {
tests := []struct {
name string
input string
want bool
}{
{"empty", "", true},
{"plain", "folder/file.txt", true},
{"leading slash", "/folder/file.txt", true},
{"trailing slash", "folder/", true},
{"hidden file ok", ".hidden", true},
{"dotdot in name ok", "..hidden", true},
{"double dots inside name", "foo..bar/baz", true},
{"bare dotdot", "..", false},
{"bare dot", ".", false},
{"leading dotdot segment", "../evil-bucket/test.txt", false},
{"leading dot-slash", "./evil/test.txt", false},
{"nested dotdot segment", "good/../evil/test.txt", false},
{"trailing dotdot segment", "good/..", false},
{"backslash dotdot", "..\\evil\\test.txt", false},
{"mixed-slash dotdot", "good\\..\\evil/test.txt", false},
{"dotdot after duplicate slash", "good//../evil", false},
{"nul byte", "foo\x00bar", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsValidObjectKey(tt.input); got != tt.want {
t.Errorf("IsValidObjectKey(%q) = %v, want %v", tt.input, got, tt.want)
}
})
}
}
func TestNormalizeObjectKeyAllowBackslash(t *testing.T) {
SetAllowBackslashInObjectKey(true)
defer SetAllowBackslashInObjectKey(false)
normalizeTests := []struct {
name string
input string
expected string
}{
{"backslash kept verbatim", "folder\\file.txt", "folder\\file.txt"},
{"nested backslash kept", "folder\\sub\\file.txt", "folder\\sub\\file.txt"},
{"trailing backslash kept", "folder\\", "folder\\"},
{"lone backslash kept", "\\", "\\"},
{"double backslash kept", "a\\\\b", "a\\\\b"},
{"mixed separators", "folder\\sub/another\\file.txt", "folder\\sub/another\\file.txt"},
{"duplicate slashes still collapse", "folder//sub///file.txt", "folder/sub/file.txt"},
{"leading slash still trimmed", "/folder\\file.txt", "folder\\file.txt"},
{"trailing slash preserved", "folder/", "folder/"},
{"windows path style", "C:\\Users\\test\\file.txt", "C:\\Users\\test\\file.txt"},
}
for _, tt := range normalizeTests {
t.Run(tt.name, func(t *testing.T) {
if got := NormalizeObjectKey(tt.input); got != tt.expected {
t.Errorf("NormalizeObjectKey(%q) = %q, want %q", tt.input, got, tt.expected)
}
})
}
validTests := []struct {
name string
input string
want bool
}{
{"backslash dotdot is one segment", "..\\evil\\test.txt", true},
{"backslash dotdot inside", "good\\..\\evil/test.txt", true},
{"nested dotdot still rejected", "good/../evil/test.txt", false},
{"leading dotdot still rejected", "../evil/test.txt", false},
{"plain backslash key", "folder\\file.txt", true},
}
for _, tt := range validTests {
t.Run(tt.name, func(t *testing.T) {
if got := IsValidObjectKey(tt.input); got != tt.want {
t.Errorf("IsValidObjectKey(%q) = %v, want %v", tt.input, got, tt.want)
}
})
}
}
func TestIsValidBucketName(t *testing.T) {
tests := []struct {
name string
input string
want bool
}{
{"empty ok", "", true},
{"plain", "my-bucket", true},
{"name containing dots", "my.bucket.name", true},
{"bare dot", ".", false},
{"bare dotdot", "..", false},
{"with slash", "evil/bucket", false},
{"with backslash", "evil\\bucket", false},
{"with nul", "evil\x00bucket", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsValidBucketName(tt.input); got != tt.want {
t.Errorf("IsValidBucketName(%q) = %v, want %v", tt.input, got, tt.want)
}
})
}
}
func TestIsValidPathSegment(t *testing.T) {
tests := []struct {
input string
want bool
}{
{"opaque-id_123", true},
{"..hidden", true},
{"", false},
{".", false},
{"..", false},
{"dir/value", false},
{"dir\\value", false},
{"nul\x00value", false},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
if got := IsValidPathSegment(tt.input); got != tt.want {
t.Errorf("IsValidPathSegment(%q) = %v, want %v", tt.input, got, tt.want)
}
})
}
}
func TestRemoveDuplicateSlashes(t *testing.T) {
tests := []struct {
name string
input string
expected string
}{
{
name: "no duplicates",
input: "/folder/file.txt",
expected: "/folder/file.txt",
},
{
name: "double slash",
input: "/folder//file.txt",
expected: "/folder/file.txt",
},
{
name: "triple slash",
input: "/folder///file.txt",
expected: "/folder/file.txt",
},
{
name: "multiple duplicate locations",
input: "//folder//subfolder///file.txt",
expected: "/folder/subfolder/file.txt",
},
{
name: "empty string",
input: "",
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := removeDuplicateSlashes(tt.input)
if result != tt.expected {
t.Errorf("removeDuplicateSlashes(%q) = %q, want %q", tt.input, result, tt.expected)
}
})
}
}
// TestIdentityHolderPropagation reproduces the audit-log middleware chain: an
// outer handler installs the holder, an inner handler authenticates and records
// the identity on a request copy, and the outer handler must still recover the
// requester from its own (earlier) copy of the request.
func TestIdentityHolderPropagation(t *testing.T) {
outer := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
// Before the holder is installed there is no identity to recover.
if got := GetIdentityNameFromContext(outer); got != "" {
t.Fatalf("unauthenticated request reports requester %q, want empty", got)
}
outer = EnsureIdentityHolder(outer)
// The inner handler sets the identity on a copy, mirroring how auth wraps
// the request with r.WithContext before invoking the next handler.
innerCtx := SetIdentityNameInContext(outer.Context(), "admin")
inner := outer.WithContext(innerCtx)
if got := GetIdentityNameFromContext(inner); got != "admin" {
t.Errorf("inner request requester = %q, want admin", got)
}
if got := GetIdentityNameFromContext(outer); got != "admin" {
t.Errorf("outer request requester = %q, want admin (must propagate via holder)", got)
}
}
func TestEnsureIdentityHolderIdempotent(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
first := EnsureIdentityHolder(req)
if first == req {
t.Fatal("EnsureIdentityHolder should return a new request when no holder is present")
}
again := EnsureIdentityHolder(first)
if again != first {
t.Error("EnsureIdentityHolder should be idempotent when a holder is already present")
}
}
func TestGetIdentityNameFromContextWithoutHolder(t *testing.T) {
// Without a holder, the per-request context value is still honored so paths
// that set identity without installing a holder keep working.
req := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
req = req.WithContext(SetIdentityNameInContext(req.Context(), "admin"))
if got := GetIdentityNameFromContext(req); got != "admin" {
t.Errorf("requester = %q, want admin", got)
}
}
func TestStoredCredentialFingerprintIsKeyed(t *testing.T) {
key := []byte("test-signing-key-32-characters!!")
fp := StoredCredentialFingerprint(key, "AKIAEXAMPLE", "secret")
if fp == "" {
t.Fatal("fingerprint is empty with a key")
}
if again := StoredCredentialFingerprint(key, "AKIAEXAMPLE", "secret"); again != fp {
t.Error("fingerprint is not deterministic")
}
if other := StoredCredentialFingerprint([]byte("another-signing-key-padded-out"), "AKIAEXAMPLE", "secret"); other == fp {
t.Error("fingerprint ignores the server key, leaving the secret guessable")
}
if noKey := StoredCredentialFingerprint(nil, "AKIAEXAMPLE", "secret"); noKey != "" {
t.Error("an empty key must yield an empty fingerprint")
}
}