Files
seaweedfs/weed/s3api/auth_signature_v4_unsigned_headers_test.go
T
Chris LuandDevin 56fbc3deb5 s3api: align S3/IAM error responses with AWS (#11632)
* s3err: add InvalidArgument and AuthorizationHeaderMalformed codes

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: answer unrecognized bucket PUT sub-resources with 501

A PUT on a bucket carrying an unrecognized query (logging, metrics,
intelligent-tiering, ...) fell through to the bare CreateBucket route
and returned BucketAlreadyOwnedByYou or re-created the bucket. AWS
answers these with NotImplemented.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: reject malformed copy-source and multipart PUT parameters

A malformed X-Amz-Copy-Source or a non-numeric partNumber fell through
to the plain PutObject route and stored the body as a regular object.
Answer them with InvalidArgument-class errors instead of writing data.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: verify x-amz-content-sha256 against the streamed body

A PUT carrying a hex or base64 payload hash now streams through a
verifier that reports a mismatch once the stream is exhausted, instead
of storing an object that does not match its declared hash. The error
is deferred so intermediate reads that drop (n>0, err) results cannot
silently swallow it. Sentinel values (unsigned/streaming payloads)
remain exempt and malformed values fail fast.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: map truncated SigV4 headers to the error for the missing field

AWS answers an Authorization header missing Credential= with
InvalidArgument and one missing or malformed Signature= with
AuthorizationHeaderMalformed, instead of a generic MissingFields.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: answer IAM/STS failures in the query-protocol envelope

Embedded IAM and STS routes now report authentication, form-parse and
authorization failures with the IAM ErrorResponse body instead of the
S3 Error envelope, so IAM SDK clients can parse them. Requests signed
for s3 keep the S3 envelope, keyed off the credential scope.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: return 403 AccessDenied when the request has no Date

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: answer throttling rejections as SlowDown

ErrTooManyRequest and ErrRequestBytesExceed reported made-up codes;
AWS serves these throttling rejections as SlowDown.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: reject versionId requests on buckets that never had versioning

GET, HEAD and DELETE carrying a non-empty versionId on an unversioned
bucket now fail with InvalidArgument instead of being answered as a
plain object request.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: answer DeleteObjects over 1000 keys with MalformedXML

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: InvalidArgument for non-numeric or out-of-range part numbers

partNumber=abc, 0 and >10000 all resolve to InvalidArgument, matching
AWS, instead of InvalidPart.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iam: correct LimitExceeded, InvalidAction and ServiceFailure mappings

LimitExceeded is a conflict (409), an unknown Action is InvalidAction
(404) rather than NotImplemented, and internal failures report the IAM
receiver fault type. Applies to both the embedded IAM endpoint and the
standalone iamapi server.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iam: refuse DeleteUser while access keys remain

Deleting a user with live credentials orphaned its access keys; AWS
answers DeleteConflict until they are removed first.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/s3: add S3/IAM error-response compatibility harness

* s3: return InvalidArgument for malformed x-amz-content-sha256

A header value that decodes to neither 32-byte hex nor base64 is a
malformed argument, not a hash mismatch.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: stop spawned mini when readiness times out

A slow-starting server otherwise survives the failure path and keeps
the S3 port occupied for the next run.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: register atexit cleanup before setup

A failed setup previously skipped cleanup, leaking the bucket and IAM
user on persistent servers.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: reject unrouted subresources on the DELETE bucket catch-all

PutBucketHandler gained the same guard when the route-level check moved
into the handlers; DeleteBucketHandler was missed, so an authorized
DELETE /bucket?logging could delete the bucket instead of answering
NotImplemented.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: tolerate unset fixture variables in cleanup and cover DELETE ?logging

Cleanup now runs its IAM/multipart steps only when setup reached them, so
an early setup failure still removes the bucket. Added a DELETE
bucket-subresource case asserting NotImplemented and bucket survival.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: fail a case when its side-effect check reports a regression

A non-empty check note now fails the case, so a deleted bucket or an
object created by a malformed request cannot slip through behind a
passing status check.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 21:02:38 +08:00

389 lines
14 KiB
Go

package s3api
import (
"fmt"
"net/http"
"sort"
"strings"
"testing"
"time"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
)
// TestVerifySignedHeadersCoverage_Unit exercises the helper directly.
func TestVerifySignedHeadersCoverage_Unit(t *testing.T) {
tests := []struct {
name string
headers map[string]string
signedHeaders []string
isPresigned bool
want s3err.ErrorCode
}{
{
name: "no x-amz headers is fine",
headers: map[string]string{"Content-Type": "text/plain"},
signedHeaders: []string{"host"},
want: s3err.ErrNone,
},
{
name: "signed x-amz header is accepted",
headers: map[string]string{
"X-Amz-Date": "20250101T000000Z",
"X-Amz-Tagging": "a=b",
},
signedHeaders: []string{"host", "x-amz-date", "x-amz-tagging"},
want: s3err.ErrNone,
},
{
name: "unsigned x-amz-tagging is rejected (header-based)",
headers: map[string]string{
"X-Amz-Date": "20250101T000000Z",
"X-Amz-Tagging": "secret=pwn",
},
signedHeaders: []string{"host", "x-amz-date"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-meta-* is rejected",
headers: map[string]string{
"X-Amz-Meta-Owner": "attacker",
},
signedHeaders: []string{"host"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-acl is rejected",
headers: map[string]string{
"X-Amz-Acl": "public-read",
},
signedHeaders: []string{"host"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-storage-class is rejected",
headers: map[string]string{
"X-Amz-Storage-Class": "GLACIER",
},
signedHeaders: []string{"host"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-server-side-encryption is rejected",
headers: map[string]string{
"X-Amz-Server-Side-Encryption": "AES256",
},
signedHeaders: []string{"host"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-object-lock-retain-until-date is rejected",
headers: map[string]string{
"X-Amz-Object-Lock-Retain-Until-Date": "2099-01-01T00:00:00Z",
},
signedHeaders: []string{"host"},
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "unsigned x-amz-security-token is rejected (even presigned)",
headers: map[string]string{
"X-Amz-Security-Token": "attacker-token",
},
signedHeaders: []string{"host"},
isPresigned: true,
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "x-amz-content-sha256 is always exempt (presigned)",
headers: map[string]string{
"X-Amz-Content-Sha256": "UNSIGNED-PAYLOAD",
},
signedHeaders: []string{"host"},
isPresigned: true,
want: s3err.ErrNone,
},
{
name: "x-amz-content-sha256 is always exempt (header-based)",
headers: map[string]string{
"X-Amz-Content-Sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
},
signedHeaders: []string{"host"},
isPresigned: false,
want: s3err.ErrNone,
},
{
name: "presigned exempts sigv4 query params echoed as headers",
headers: map[string]string{
"X-Amz-Algorithm": "AWS4-HMAC-SHA256",
"X-Amz-Credential": "AKIA/20250101/us-east-1/s3/aws4_request",
"X-Amz-Date": "20250101T000000Z",
"X-Amz-Expires": "3600",
"X-Amz-SignedHeaders": "host",
"X-Amz-Signature": "deadbeef",
},
signedHeaders: []string{"host"},
isPresigned: true,
want: s3err.ErrNone,
},
{
name: "header-based does NOT exempt x-amz-date when unsigned",
headers: map[string]string{
"X-Amz-Date": "20250101T000000Z",
},
signedHeaders: []string{"host"},
isPresigned: false,
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "signed headers match is case-insensitive",
headers: map[string]string{
"X-Amz-Tagging": "a=b",
},
signedHeaders: []string{"HOST", "X-Amz-Tagging"},
want: s3err.ErrNone,
},
{
name: "presigned exempts unsigned SSE-C key headers",
headers: map[string]string{
"X-Amz-Server-Side-Encryption-Customer-Algorithm": "AES256",
"X-Amz-Server-Side-Encryption-Customer-Key": "key",
"X-Amz-Server-Side-Encryption-Customer-Key-MD5": "md5",
},
signedHeaders: []string{"host", "x-amz-server-side-encryption-customer-algorithm"},
isPresigned: true,
want: s3err.ErrNone,
},
{
name: "presigned exempts unsigned SSE-C copy-source key headers",
headers: map[string]string{
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Algorithm": "AES256",
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key": "key",
"X-Amz-Copy-Source-Server-Side-Encryption-Customer-Key-MD5": "md5",
},
signedHeaders: []string{"host", "x-amz-copy-source-server-side-encryption-customer-algorithm"},
isPresigned: true,
want: s3err.ErrNone,
},
{
name: "presigned still requires SSE-C algorithm to be signed",
headers: map[string]string{
"X-Amz-Server-Side-Encryption-Customer-Algorithm": "AES256",
"X-Amz-Server-Side-Encryption-Customer-Key": "key",
"X-Amz-Server-Side-Encryption-Customer-Key-MD5": "md5",
},
signedHeaders: []string{"host"},
isPresigned: true,
want: s3err.ErrSignatureDoesNotMatch,
},
{
name: "header-based does NOT exempt unsigned SSE-C key headers",
headers: map[string]string{
"X-Amz-Server-Side-Encryption-Customer-Algorithm": "AES256",
"X-Amz-Server-Side-Encryption-Customer-Key": "key",
"X-Amz-Server-Side-Encryption-Customer-Key-MD5": "md5",
},
signedHeaders: []string{"host", "x-amz-server-side-encryption-customer-algorithm"},
isPresigned: false,
want: s3err.ErrSignatureDoesNotMatch,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, err := http.NewRequest(http.MethodPut, "http://example.com/bucket/key", nil)
if err != nil {
t.Fatalf("NewRequest: %v", err)
}
for k, v := range tt.headers {
req.Header.Set(k, v)
}
got := verifySignedHeadersCoverage(req, tt.signedHeaders, tt.isPresigned)
if got != tt.want {
t.Fatalf("verifySignedHeadersCoverage = %v, want %v", got, tt.want)
}
})
}
}
// TestPresignedPutRejectsUnsignedTagging exercises the full verification path
// through reqSignatureV4Verify for a presigned URL that signs only `host` but
// whose caller attached an x-amz-tagging header after-the-fact.
func TestPresignedPutRejectsUnsignedTagging(t *testing.T) {
iam := newTestIAM()
req, err := newTestRequest(http.MethodPut, "http://127.0.0.1:9000/bucket/key", 0, nil)
if err != nil {
t.Fatalf("newTestRequest: %v", err)
}
if err := preSignV4(iam, req, "AKIAIOSFODNN7EXAMPLE", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", 600); err != nil {
t.Fatalf("preSignV4: %v", err)
}
// Attacker appends an unsigned header after the URL was signed.
req.Header.Set("X-Amz-Tagging", "classification=public")
_, errCode := iam.reqSignatureV4Verify(req)
if errCode != s3err.ErrSignatureDoesNotMatch {
t.Fatalf("expected ErrSignatureDoesNotMatch for unsigned x-amz-tagging, got %v", errCode)
}
}
// TestPresignedPutAcceptsSignedTagging confirms that a presigned URL whose
// SignedHeaders list covers x-amz-tagging still validates successfully.
func TestPresignedPutAcceptsSignedTagging(t *testing.T) {
iam := newTestIAM()
req, err := newTestRequest(http.MethodPut, "http://127.0.0.1:9000/bucket/key", 0, nil)
if err != nil {
t.Fatalf("newTestRequest: %v", err)
}
req.Header.Set("X-Amz-Tagging", "classification=public")
if err := preSignV4WithHeaders(iam, req, "AKIAIOSFODNN7EXAMPLE", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", 600, []string{"host", "x-amz-tagging"}); err != nil {
t.Fatalf("preSignV4WithHeaders: %v", err)
}
_, errCode := iam.reqSignatureV4Verify(req)
if errCode != s3err.ErrNone {
t.Fatalf("expected ErrNone for signed x-amz-tagging, got %v", errCode)
}
}
// TestPresignedGetAcceptsUnsignedSSECKeyHeaders mirrors the AWS SDK presign
// flow for SSE-C objects: only the algorithm header is signed while the key
// and key-MD5 headers are attached to the request unsigned.
func TestPresignedGetAcceptsUnsignedSSECKeyHeaders(t *testing.T) {
iam := newTestIAM()
req, err := newTestRequest(http.MethodGet, "http://127.0.0.1:9000/bucket/key", 0, nil)
if err != nil {
t.Fatalf("newTestRequest: %v", err)
}
req.Header.Set("X-Amz-Server-Side-Encryption-Customer-Algorithm", "AES256")
if err := preSignV4WithHeaders(iam, req, "AKIAIOSFODNN7EXAMPLE", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", 600, []string{"host", "x-amz-server-side-encryption-customer-algorithm"}); err != nil {
t.Fatalf("preSignV4WithHeaders: %v", err)
}
req.Header.Set("X-Amz-Server-Side-Encryption-Customer-Key", "MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTIzNDU2Nzg5MDE=")
req.Header.Set("X-Amz-Server-Side-Encryption-Customer-Key-MD5", "md5")
_, errCode := iam.reqSignatureV4Verify(req)
if errCode != s3err.ErrNone {
t.Fatalf("expected ErrNone for presigned SSE-C with unsigned key headers, got %v", errCode)
}
}
// preSignV4WithHeaders is a test helper that builds a presigned URL whose
// SignedHeaders list covers the specified headers (which must already be set
// on the request), then computes the signature over those headers and
// attaches X-Amz-Signature.
func preSignV4WithHeaders(_ *IdentityAccessManagement, req *http.Request, accessKey, secretKey string, expires int64, signedHeaders []string) error {
now := time.Now().UTC()
dateStr := now.Format(iso8601Format)
scope := fmt.Sprintf("%s/%s/%s/%s", now.Format(yyyymmdd), "us-east-1", "s3", "aws4_request")
credential := fmt.Sprintf("%s/%s", accessKey, scope)
normalized := make([]string, 0, len(signedHeaders))
for _, h := range signedHeaders {
normalized = append(normalized, strings.ToLower(h))
}
sort.Strings(normalized)
query := req.URL.Query()
query.Set("X-Amz-Algorithm", signV4Algorithm)
query.Set("X-Amz-Credential", credential)
query.Set("X-Amz-Date", dateStr)
query.Set("X-Amz-Expires", fmt.Sprintf("%d", expires))
query.Set("X-Amz-SignedHeaders", strings.Join(normalized, ";"))
req.URL.RawQuery = query.Encode()
hashedPayload := query.Get("X-Amz-Content-Sha256")
if hashedPayload == "" {
hashedPayload = unsignedPayload
}
extracted := make(http.Header)
for _, h := range normalized {
if h == "host" {
extracted[h] = []string{req.Host}
continue
}
if values, ok := req.Header[http.CanonicalHeaderKey(h)]; ok {
extracted[h] = values
}
}
canonicalRequest := getCanonicalRequest(extracted, hashedPayload, req.URL.RawQuery, req.URL.Path, req.Method)
stringToSign := getStringToSign(canonicalRequest, now, scope)
signingKey := getSigningKey(secretKey, now.Format(yyyymmdd), "us-east-1", "s3")
signature := getSignature(signingKey, stringToSign)
query.Set("X-Amz-Signature", signature)
req.URL.RawQuery = query.Encode()
return nil
}
// TestPresignedPutRejectsUnsignedMetadataHeaders checks a spread of
// security-relevant headers that the PUT handler persists.
func TestPresignedPutRejectsUnsignedMetadataHeaders(t *testing.T) {
dangerous := []struct {
name string
header string
value string
}{
{"acl", "X-Amz-Acl", "public-read"},
{"user-metadata", "X-Amz-Meta-Owner", "attacker"},
{"storage-class", "X-Amz-Storage-Class", "GLACIER"},
{"server-side-encryption", "X-Amz-Server-Side-Encryption", "AES256"},
{"sse-kms-key-id", "X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id", "kms-key"},
{"object-lock-mode", "X-Amz-Object-Lock-Mode", "GOVERNANCE"},
{"object-lock-retain-until", "X-Amz-Object-Lock-Retain-Until-Date", "2099-01-01T00:00:00Z"},
{"website-redirect-location", "X-Amz-Website-Redirect-Location", "https://attacker.example/"},
{"grant-full-control", "X-Amz-Grant-Full-Control", "id=attacker"},
}
for _, c := range dangerous {
t.Run(c.name, func(t *testing.T) {
iam := newTestIAM()
req, err := newTestRequest(http.MethodPut, "http://127.0.0.1:9000/bucket/key", 0, nil)
if err != nil {
t.Fatalf("newTestRequest: %v", err)
}
if err := preSignV4(iam, req, "AKIAIOSFODNN7EXAMPLE", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", 600); err != nil {
t.Fatalf("preSignV4: %v", err)
}
req.Header.Set(c.header, c.value)
_, errCode := iam.reqSignatureV4Verify(req)
if errCode != s3err.ErrSignatureDoesNotMatch {
t.Fatalf("expected rejection for unsigned %s, got %v", c.header, errCode)
}
})
}
}
// TestParseSignV4MissingFields exercises how a truncated Authorization header
// maps to the AWS error code for the absent field.
func TestParseSignV4MissingFields(t *testing.T) {
valid := "AWS4-HMAC-SHA256 Credential=AKIA/20260101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=0123456789abcdef"
tests := []struct {
name string
auth string
want s3err.ErrorCode
}{
{"valid", valid, s3err.ErrNone},
{"no signature field", "AWS4-HMAC-SHA256 Credential=AKIA/20260101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date", s3err.ErrAuthorizationHeaderMalformed},
{"empty signature field", "AWS4-HMAC-SHA256 Credential=AKIA/20260101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, ", s3err.ErrAuthorizationHeaderMalformed},
{"no credential field", "AWS4-HMAC-SHA256 SignedHeaders=host;x-amz-date, Signature=0123456789abcdef", s3err.ErrInvalidArgument},
{"leading comma after dropping credential", "AWS4-HMAC-SHA256 , SignedHeaders=host;x-amz-date, Signature=0123456789abcdef", s3err.ErrInvalidArgument},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, got := parseSignV4(tt.auth)
if got != tt.want {
t.Fatalf("parseSignV4() = %v, want %v", got, tt.want)
}
})
}
}