Files
seaweedfs/weed/s3api/s3api_bucket_handlers_misc.go
T
Chris LuandClaude Opus 5.5 53bc62c773 s3api: accept the accelerate and notification configurations the bucket already reports (#12747)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI

Runs the suite, pinned to a commit, against weed mini through a small
provider patch, and fails on any test that is not in
expected_failures.txt or on a listed test that starts passing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* test/s3/tigris_compat: prepare the suite again when the patch changes

The checkout was reused whenever it sat at the pinned commit, so an
edited seaweedfs-provider.patch, or a checkout whose patch failed to
apply, kept being built as it was. Record the commit and the patch's
hash once the patch applies, and prepare the checkout again when either
differs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: keep the pid file of a server already running

start-server removed weed-server.pid before it found the port taken, so
a second start lost the pid of the server holding it and stop-server
could no longer stop that one. Remove the file only once the port is
free, and have test-with-server stop only a server it started.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: bound the S3 port probes

A listener that accepts but never answers held curl, and with it the
readiness loop and the cleanup after it, for good. Give each probe two
seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: accept PutBucketAccelerateConfiguration with Status=Suspended

GetBucketAccelerateConfiguration always reports Suspended, but the PUT
answered 501 for every request. Suspended is now accepted, Enabled
stays NotImplemented, and the accelerate subresource resolves to
s3:GetAccelerateConfiguration / s3:PutAccelerateConfiguration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* s3api: accept an empty PutBucketNotificationConfiguration

GetBucketNotificationConfiguration always reports an empty
configuration, but the PUT answered 501, including for the empty
configuration clients send to clear notifications. An empty one is now
accepted; a topic, queue, Lambda or EventBridge destination stays
NotImplemented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* s3api: reject accelerate and notification bodies with the wrong root

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* test/s3/tigris_compat: expect a listed test to end with its listed status

A listed test passed the check with any non-pass status, so a setup error
that skipped the call under test (RestoreObject's PutObject, say) went
unnoticed. Entries now expect fail unless they name another status; only
GetBucketAccelerateConfiguration, whose setup PUT is unsupported, expects
error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: refuse to start while the recorded server still runs

A second start-server before the first server answered on the S3 port
passed the port probe and dropped the pid file of the live server, so
stop-server could no longer stop it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: recognize the test server by its command line

start-server and stop-server took any process named weed at the recorded
pid for the test server, so a weed process that reused the pid would be
reported as running and killed by stop-server. Match the command line
start-server launches instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: recognize the test server by its start time

A weed mini started from another checkout has the same command line, so
a pid it reused still matched. Record the process start time next to the
pid at launch and require both before treating the process as the test
server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: PutObjectTagging_VersionId passes now

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: keep the server records until the server exits

stop-server waited a fixed two seconds after SIGTERM and then removed
weed-server.pid and weed-server.started even if weed was still shutting
down, so a later stop-server could not find it and clean could delete
data it still used. Wait up to 30 seconds for the recorded server to exit
and keep the records, failing, while it still runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test/s3/tigris_compat: start the server on an empty data directory

A data directory left by an earlier run on other ports made the master
freeze volume growth (topology id mismatch), and every PutObject failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 14:55:18 -08:00

279 lines
9.9 KiB
Go

package s3api
import (
"encoding/xml"
"errors"
"net/http"
"strings"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
)
// putBucketMiscConfigMaxBodyBytes caps the request body of the configuration
// PUTs in this file; a valid payload is a few hundred bytes.
const putBucketMiscConfigMaxBodyBytes = 64 * 1024
type policyStatusResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ PolicyStatus"`
IsPublic bool `xml:"IsPublic"`
}
type accelerateConfigurationResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ AccelerateConfiguration"`
Status string `xml:"Status"`
}
type bucketLoggingStatusResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ BucketLoggingStatus"`
}
type notificationConfigurationResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ NotificationConfiguration"`
}
type accelerateConfigurationRequest struct {
XMLName xml.Name `xml:"AccelerateConfiguration"`
Status string `xml:"Status"`
}
type notificationConfigurationRequest struct {
XMLName xml.Name `xml:"NotificationConfiguration"`
Topics []struct{} `xml:"TopicConfiguration"`
Queues []struct{} `xml:"QueueConfiguration"`
Functions []struct{} `xml:"CloudFunctionConfiguration"`
EventBridge []struct{} `xml:"EventBridgeConfiguration"`
}
// GetBucketPolicyStatusHandler reports whether the bucket policy grants public access.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicyStatus.html
func (s3a *S3ApiServer) GetBucketPolicyStatusHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
policyDoc, err := s3a.getBucketPolicy(bucket)
if err != nil {
if errors.Is(err, ErrPolicyNotFound) {
s3err.WriteErrorResponse(w, r, s3err.ErrNoSuchBucketPolicy)
} else if errors.Is(err, ErrBucketNotFound) {
s3err.WriteErrorResponse(w, r, s3err.ErrNoSuchBucket)
} else {
glog.Errorf("GetBucketPolicyStatusHandler load policy %s: %v", bucket, err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
}
return
}
writeSuccessResponseXML(w, r, policyStatusResponse{IsPublic: isPolicyPublic(policyDoc)})
}
// isPolicyPublic returns true if any Allow statement grants access to "*" without restricting conditions.
func isPolicyPublic(doc *policy_engine.PolicyDocument) bool {
if doc == nil {
return false
}
for _, st := range doc.Statement {
if st.Effect != policy_engine.PolicyEffectAllow {
continue
}
if len(st.Condition) > 0 {
continue
}
for _, p := range st.Principal.Strings() {
if p == "*" {
return true
}
}
}
return false
}
// PutBucketRequestPaymentHandler accepts only Payer=BucketOwner; Requester is rejected.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketRequestPayment.html
func (s3a *S3ApiServer) PutBucketRequestPaymentHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
defer r.Body.Close()
body, err := readRequestBody(r, putBucketMiscConfigMaxBodyBytes)
if err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
var cfg RequestPaymentConfiguration
if err := xml.Unmarshal(body, &cfg); err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
if cfg.Payer != "BucketOwner" {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
writeSuccessResponseEmpty(w, r)
}
// GetBucketAccelerateConfigurationHandler returns a static Suspended status.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketAccelerateConfiguration.html
func (s3a *S3ApiServer) GetBucketAccelerateConfigurationHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
writeSuccessResponseXML(w, r, accelerateConfigurationResponse{Status: "Suspended"})
}
// PutBucketAccelerateConfigurationHandler accepts Status=Suspended, the state
// GetBucketAccelerateConfigurationHandler always reports; Enabled is not implemented.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketAccelerateConfiguration.html
func (s3a *S3ApiServer) PutBucketAccelerateConfigurationHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
defer r.Body.Close()
body, err := readRequestBody(r, putBucketMiscConfigMaxBodyBytes)
if err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
var cfg accelerateConfigurationRequest
if err := xml.Unmarshal(body, &cfg); err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
switch cfg.Status {
case "Suspended":
writeSuccessResponseEmpty(w, r)
case "Enabled":
s3err.WriteErrorResponse(w, r, s3err.ErrNotImplemented)
default:
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
}
}
// PutBucketNotificationConfigurationHandler accepts an empty configuration, the
// one GetBucketNotificationConfigurationHandler always reports; any destination
// is not implemented.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketNotificationConfiguration.html
func (s3a *S3ApiServer) PutBucketNotificationConfigurationHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
defer r.Body.Close()
body, err := readRequestBody(r, putBucketMiscConfigMaxBodyBytes)
if err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
var cfg notificationConfigurationRequest
if err := xml.Unmarshal(body, &cfg); err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrMalformedXML)
return
}
if len(cfg.Topics)+len(cfg.Queues)+len(cfg.Functions)+len(cfg.EventBridge) > 0 {
s3err.WriteErrorResponse(w, r, s3err.ErrNotImplemented)
return
}
writeSuccessResponseEmpty(w, r)
}
// GetBucketLoggingHandler returns an empty BucketLoggingStatus (logging disabled).
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLogging.html
func (s3a *S3ApiServer) GetBucketLoggingHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
writeSuccessResponseXML(w, r, bucketLoggingStatusResponse{})
}
// GetBucketNotificationConfigurationHandler returns an empty configuration; SeaweedFS
// has no bucket event notifications, and AWS answers an unconfigured bucket the same way.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketNotificationConfiguration.html
func (s3a *S3ApiServer) GetBucketNotificationConfigurationHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
writeSuccessResponseXML(w, r, notificationConfigurationResponse{})
}
// GetBucketReplicationHandler reports that no replication configuration exists.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketReplication.html
func (s3a *S3ApiServer) GetBucketReplicationHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
s3err.WriteErrorResponse(w, r, s3err.ErrReplicationConfigurationNotFound)
}
// GetBucketWebsiteHandler reports that no website configuration exists.
// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketWebsite.html
func (s3a *S3ApiServer) GetBucketWebsiteHandler(w http.ResponseWriter, r *http.Request) {
bucket, _ := s3_constants.GetBucketAndObject(r)
if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, err)
return
}
s3err.WriteErrorResponse(w, r, s3err.ErrNoSuchWebsiteConfiguration)
}
// listObjectsQueryParams are the query keys a ListObjects/ListObjectsV2 request may
// carry. x-id is stamped by the AWS SDKs and carries no meaning for the server; the
// rest of the non-listing keys are what a presigned URL signs into the query string.
var listObjectsQueryParams = map[string]bool{
"prefix": true, "delimiter": true, "marker": true, "max-keys": true,
"encoding-type": true, "list-type": true, "continuation-token": true,
"start-after": true, "fetch-owner": true, "expected-bucket-owner": true,
"allow-unordered": true, "x-id": true,
// SigV2 presigned URLs.
"AWSAccessKeyId": true, "Signature": true, "Expires": true,
}
// unroutedBucketSubresource names a query key on a bucket GET that no route claimed.
// Everything SeaweedFS implements is matched by its own route before the ListObjects
// catch-all, so anything left is a subresource it does not implement - and answering
// it with a bucket listing is worse than saying so.
func unroutedBucketSubresource(r *http.Request) (string, bool) {
return unroutedSubresource(r, listObjectsQueryParams)
}
// unroutedSubresource names a query key that no route claimed, outside the
// request's own parameters, SigV2 presign keys, and SigV4 presign keys
// (all "X-Amz-" prefixed).
func unroutedSubresource(r *http.Request, allowed map[string]bool) (string, bool) {
for key := range r.URL.Query() {
switch {
case allowed[key], strings.HasPrefix(key, "X-Amz-"):
case key == "x-id", key == "AWSAccessKeyId", key == "Signature", key == "Expires":
default:
return key, true
}
}
return "", false
}