mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-24 15:00:43 +02:00
* s3: end a catalog-vended session with the access key it was vended for A table credential the Iceberg catalog vends is a self-contained STS session: it stays valid until it expires (an hour by default) even after the access key that loaded the table is deleted or deactivated, or its identity is removed or disabled. An operator revoking a leaked key cannot cut off the table credentials that key already obtained. Sessions are stateless JWTs, so there is no revocation list to consult. This change follows the same pattern as role sessions, which are already refused once their role's ID no longer matches (checkSessionRoleBinding): a vended session carries the access key it was vended for (src_ak, recorded by the previous change), and validateSTSSessionToken checks it against the identity table held in memory. A session of provider iceberg-catalog that carries a source key is refused (InvalidAccessKeyId, as for an unknown key) once its identity is gone or disabled, or the key is no longer one of that identity's active, unexpired credentials. Scope: - Only catalog-vended sessions are checked; web-identity, AssumeRole and every other STS session are validated as before. - The vendor binds a session to the caller's key only when the caller is a stored identity holding that key. A caller that signed with another STS session's key, an OIDC user, or a request that recorded no key gets an unbound session, as today; binding it would refuse every request made with it. A session minted before this change carries no key and is unbound. - The check is one map lookup under the read lock and a scan of one identity's keys (about 11 ns, 0 allocations, with 1,000 identities). Tests (s3api_iceberg_vended_session_test.go) mint real sessions through the catalog vendor and validate them the way a data-path request is validated: deactivating, expiring or deleting key A ends A's session while the session vended through key B on the same identity keeps working; disabling or deleting the identity ends it; callers that are not a stored identity holding the key get unbound sessions that keep working; other STS sessions are not checked. The first two tests and the third fail without this change. * s3: keep the vended-session key binding when the key dies before vend vendedSessionKeyLive gated the binding, so a key revoked between the request's authentication and the vend skipped it and minted an unbound session that stayed usable until it expired. Every catalog auth path pairs the identity name in context with the key that authenticated, so a stored-identity principal always means the request signed with that identity's own key. Bind whenever the principal is a stored identity: a session bound to a now-dead key is refused on its first request, which is what the binding is for. * s3: gate the vended-session key binding on the authenticated caller type lookupByIdentityName(principal) named only that some identity exists, not that the request signed with a stored key: an STS session or OIDC principal that collides with an identity name got a session bound to its temporary key, which vendedSessionKeyLive then refused on every request. Verifiers now record the verified key as a stored credential, and the vendor binds only when that marker is present — keeping the binding even when the key was revoked between authentication and the vend. * s3: enforce the vended key binding under a dedicated claim Sessions minted before the binding existed carry the caller's key in the audit-only SourceAccessKeyId — including temporary STS keys that can never resolve in the identity table. Enforcing on that field would end live table credentials at upgrade; the binding now travels under BoundAccessKeyId, leaving legacy sessions unbound as before. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3: bind vended sessions to a credential fingerprint The key id alone cannot identify a credential: an operator can delete a key and re-create it under the same id with a different secret, reviving sessions minted for the original. Verifiers now record a fingerprint of the secret at authentication; the session dies unless it still matches. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3: keep recording the caller's key on every catalog session SourceAccessKeyId was unconditional audit metadata before the binding existed; gating it on the stored-credential marker dropped the record for callers that authenticated another way. With the binding under its own claim there is no ambiguity left, so the field records the caller's key again regardless of how it authenticated. * s3: key the vended-session credential fingerprint with the signing key The bound-credential fingerprint travels inside the vended session token. As a plain SHA-256 of the secret it gave any token holder an offline guessing oracle for weak secrets, and CodeQL flags it twice. Store the verified secret in the request marker instead and HMAC it with the STS signing key at vend and at validation. The claim can no longer be mined, and the per-request auth path stops hashing secrets. --------- Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
333 lines
9.4 KiB
Go
333 lines
9.4 KiB
Go
package s3_constants
|
|
|
|
import (
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"testing"
|
|
)
|
|
|
|
func TestNormalizeObjectKey(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
input string
|
|
expected string
|
|
}{
|
|
{
|
|
name: "simple key",
|
|
input: "file.txt",
|
|
expected: "file.txt",
|
|
},
|
|
{
|
|
name: "key with leading slash",
|
|
input: "/file.txt",
|
|
expected: "file.txt",
|
|
},
|
|
{
|
|
name: "key with directory",
|
|
input: "folder/file.txt",
|
|
expected: "folder/file.txt",
|
|
},
|
|
{
|
|
name: "key with leading slash and directory",
|
|
input: "/folder/file.txt",
|
|
expected: "folder/file.txt",
|
|
},
|
|
{
|
|
name: "key with duplicate slashes",
|
|
input: "folder//subfolder///file.txt",
|
|
expected: "folder/subfolder/file.txt",
|
|
},
|
|
{
|
|
name: "Windows backslash - simple",
|
|
input: "folder\\file.txt",
|
|
expected: "folder/file.txt",
|
|
},
|
|
{
|
|
name: "Windows backslash - nested",
|
|
input: "folder\\subfolder\\file.txt",
|
|
expected: "folder/subfolder/file.txt",
|
|
},
|
|
{
|
|
name: "Windows backslash - with leading slash",
|
|
input: "/folder\\subfolder\\file.txt",
|
|
expected: "folder/subfolder/file.txt",
|
|
},
|
|
{
|
|
name: "mixed slashes",
|
|
input: "folder\\subfolder/another\\file.txt",
|
|
expected: "folder/subfolder/another/file.txt",
|
|
},
|
|
{
|
|
name: "Windows full path style (edge case)",
|
|
input: "C:\\Users\\test\\file.txt",
|
|
expected: "C:/Users/test/file.txt",
|
|
},
|
|
{
|
|
name: "empty string",
|
|
input: "",
|
|
expected: "",
|
|
},
|
|
{
|
|
name: "just a slash",
|
|
input: "/",
|
|
expected: "",
|
|
},
|
|
{
|
|
name: "just a backslash",
|
|
input: "\\",
|
|
expected: "",
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
result := NormalizeObjectKey(tt.input)
|
|
if result != tt.expected {
|
|
t.Errorf("NormalizeObjectKey(%q) = %q, want %q", tt.input, result, tt.expected)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestIsValidObjectKey(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
input string
|
|
want bool
|
|
}{
|
|
{"empty", "", true},
|
|
{"plain", "folder/file.txt", true},
|
|
{"leading slash", "/folder/file.txt", true},
|
|
{"trailing slash", "folder/", true},
|
|
{"hidden file ok", ".hidden", true},
|
|
{"dotdot in name ok", "..hidden", true},
|
|
{"double dots inside name", "foo..bar/baz", true},
|
|
|
|
{"bare dotdot", "..", false},
|
|
{"bare dot", ".", false},
|
|
{"leading dotdot segment", "../evil-bucket/test.txt", false},
|
|
{"leading dot-slash", "./evil/test.txt", false},
|
|
{"nested dotdot segment", "good/../evil/test.txt", false},
|
|
{"trailing dotdot segment", "good/..", false},
|
|
{"backslash dotdot", "..\\evil\\test.txt", false},
|
|
{"mixed-slash dotdot", "good\\..\\evil/test.txt", false},
|
|
{"dotdot after duplicate slash", "good//../evil", false},
|
|
{"nul byte", "foo\x00bar", false},
|
|
}
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
if got := IsValidObjectKey(tt.input); got != tt.want {
|
|
t.Errorf("IsValidObjectKey(%q) = %v, want %v", tt.input, got, tt.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestNormalizeObjectKeyAllowBackslash(t *testing.T) {
|
|
SetAllowBackslashInObjectKey(true)
|
|
defer SetAllowBackslashInObjectKey(false)
|
|
|
|
normalizeTests := []struct {
|
|
name string
|
|
input string
|
|
expected string
|
|
}{
|
|
{"backslash kept verbatim", "folder\\file.txt", "folder\\file.txt"},
|
|
{"nested backslash kept", "folder\\sub\\file.txt", "folder\\sub\\file.txt"},
|
|
{"trailing backslash kept", "folder\\", "folder\\"},
|
|
{"lone backslash kept", "\\", "\\"},
|
|
{"double backslash kept", "a\\\\b", "a\\\\b"},
|
|
{"mixed separators", "folder\\sub/another\\file.txt", "folder\\sub/another\\file.txt"},
|
|
{"duplicate slashes still collapse", "folder//sub///file.txt", "folder/sub/file.txt"},
|
|
{"leading slash still trimmed", "/folder\\file.txt", "folder\\file.txt"},
|
|
{"trailing slash preserved", "folder/", "folder/"},
|
|
{"windows path style", "C:\\Users\\test\\file.txt", "C:\\Users\\test\\file.txt"},
|
|
}
|
|
for _, tt := range normalizeTests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
if got := NormalizeObjectKey(tt.input); got != tt.expected {
|
|
t.Errorf("NormalizeObjectKey(%q) = %q, want %q", tt.input, got, tt.expected)
|
|
}
|
|
})
|
|
}
|
|
|
|
validTests := []struct {
|
|
name string
|
|
input string
|
|
want bool
|
|
}{
|
|
{"backslash dotdot is one segment", "..\\evil\\test.txt", true},
|
|
{"backslash dotdot inside", "good\\..\\evil/test.txt", true},
|
|
{"nested dotdot still rejected", "good/../evil/test.txt", false},
|
|
{"leading dotdot still rejected", "../evil/test.txt", false},
|
|
{"plain backslash key", "folder\\file.txt", true},
|
|
}
|
|
for _, tt := range validTests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
if got := IsValidObjectKey(tt.input); got != tt.want {
|
|
t.Errorf("IsValidObjectKey(%q) = %v, want %v", tt.input, got, tt.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestIsValidBucketName(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
input string
|
|
want bool
|
|
}{
|
|
{"empty ok", "", true},
|
|
{"plain", "my-bucket", true},
|
|
{"name containing dots", "my.bucket.name", true},
|
|
|
|
{"bare dot", ".", false},
|
|
{"bare dotdot", "..", false},
|
|
{"with slash", "evil/bucket", false},
|
|
{"with backslash", "evil\\bucket", false},
|
|
{"with nul", "evil\x00bucket", false},
|
|
}
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
if got := IsValidBucketName(tt.input); got != tt.want {
|
|
t.Errorf("IsValidBucketName(%q) = %v, want %v", tt.input, got, tt.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestIsValidPathSegment(t *testing.T) {
|
|
tests := []struct {
|
|
input string
|
|
want bool
|
|
}{
|
|
{"opaque-id_123", true},
|
|
{"..hidden", true},
|
|
{"", false},
|
|
{".", false},
|
|
{"..", false},
|
|
{"dir/value", false},
|
|
{"dir\\value", false},
|
|
{"nul\x00value", false},
|
|
}
|
|
for _, tt := range tests {
|
|
t.Run(tt.input, func(t *testing.T) {
|
|
if got := IsValidPathSegment(tt.input); got != tt.want {
|
|
t.Errorf("IsValidPathSegment(%q) = %v, want %v", tt.input, got, tt.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestRemoveDuplicateSlashes(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
input string
|
|
expected string
|
|
}{
|
|
{
|
|
name: "no duplicates",
|
|
input: "/folder/file.txt",
|
|
expected: "/folder/file.txt",
|
|
},
|
|
{
|
|
name: "double slash",
|
|
input: "/folder//file.txt",
|
|
expected: "/folder/file.txt",
|
|
},
|
|
{
|
|
name: "triple slash",
|
|
input: "/folder///file.txt",
|
|
expected: "/folder/file.txt",
|
|
},
|
|
{
|
|
name: "multiple duplicate locations",
|
|
input: "//folder//subfolder///file.txt",
|
|
expected: "/folder/subfolder/file.txt",
|
|
},
|
|
{
|
|
name: "empty string",
|
|
input: "",
|
|
expected: "",
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
result := removeDuplicateSlashes(tt.input)
|
|
if result != tt.expected {
|
|
t.Errorf("removeDuplicateSlashes(%q) = %q, want %q", tt.input, result, tt.expected)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestIdentityHolderPropagation reproduces the audit-log middleware chain: an
|
|
// outer handler installs the holder, an inner handler authenticates and records
|
|
// the identity on a request copy, and the outer handler must still recover the
|
|
// requester from its own (earlier) copy of the request.
|
|
func TestIdentityHolderPropagation(t *testing.T) {
|
|
outer := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
|
|
|
|
// Before the holder is installed there is no identity to recover.
|
|
if got := GetIdentityNameFromContext(outer); got != "" {
|
|
t.Fatalf("unauthenticated request reports requester %q, want empty", got)
|
|
}
|
|
|
|
outer = EnsureIdentityHolder(outer)
|
|
|
|
// The inner handler sets the identity on a copy, mirroring how auth wraps
|
|
// the request with r.WithContext before invoking the next handler.
|
|
innerCtx := SetIdentityNameInContext(outer.Context(), "admin")
|
|
inner := outer.WithContext(innerCtx)
|
|
|
|
if got := GetIdentityNameFromContext(inner); got != "admin" {
|
|
t.Errorf("inner request requester = %q, want admin", got)
|
|
}
|
|
if got := GetIdentityNameFromContext(outer); got != "admin" {
|
|
t.Errorf("outer request requester = %q, want admin (must propagate via holder)", got)
|
|
}
|
|
}
|
|
|
|
func TestEnsureIdentityHolderIdempotent(t *testing.T) {
|
|
req := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
|
|
|
|
first := EnsureIdentityHolder(req)
|
|
if first == req {
|
|
t.Fatal("EnsureIdentityHolder should return a new request when no holder is present")
|
|
}
|
|
|
|
again := EnsureIdentityHolder(first)
|
|
if again != first {
|
|
t.Error("EnsureIdentityHolder should be idempotent when a holder is already present")
|
|
}
|
|
}
|
|
|
|
func TestGetIdentityNameFromContextWithoutHolder(t *testing.T) {
|
|
// Without a holder, the per-request context value is still honored so paths
|
|
// that set identity without installing a holder keep working.
|
|
req := httptest.NewRequest(http.MethodGet, "/bucket/object", nil)
|
|
req = req.WithContext(SetIdentityNameInContext(req.Context(), "admin"))
|
|
|
|
if got := GetIdentityNameFromContext(req); got != "admin" {
|
|
t.Errorf("requester = %q, want admin", got)
|
|
}
|
|
}
|
|
|
|
func TestStoredCredentialFingerprintIsKeyed(t *testing.T) {
|
|
key := []byte("test-signing-key-32-characters!!")
|
|
fp := StoredCredentialFingerprint(key, "AKIAEXAMPLE", "secret")
|
|
if fp == "" {
|
|
t.Fatal("fingerprint is empty with a key")
|
|
}
|
|
if again := StoredCredentialFingerprint(key, "AKIAEXAMPLE", "secret"); again != fp {
|
|
t.Error("fingerprint is not deterministic")
|
|
}
|
|
if other := StoredCredentialFingerprint([]byte("another-signing-key-padded-out"), "AKIAEXAMPLE", "secret"); other == fp {
|
|
t.Error("fingerprint ignores the server key, leaving the secret guessable")
|
|
}
|
|
if noKey := StoredCredentialFingerprint(nil, "AKIAEXAMPLE", "secret"); noKey != "" {
|
|
t.Error("an empty key must yield an empty fingerprint")
|
|
}
|
|
}
|