* s3: add option to disable bucket auto-creation on upload
* command: expose -autoCreateBucket in s3, filer, server, and mini
* s3: apply the bucket auto-create policy to directory marker uploads
* s3: validate the bucket name before the auto-create disabled check
* s3: cover the disabled auto-create gate at all three upload entry points
* s3: report the effective ownership when a bucket has none stored
GetBucketOwnershipControls read Seaweed-X-Amz-Ownership straight out of the
bucket entry, so a bucket that never had one written reported an empty
ObjectOwnership. The object write path defaults the same missing attribute to
BucketOwnerEnforced, so the API contradicted the behavior it describes.
Resolve the stored value through one helper both readers share, and let
PutBucketOwnershipControls persist unconditionally so setting the default
value still gives DeleteBucketOwnershipControls something to remove.
* test: cover the bucket ownership controls round trip
Pins the behaviors the ownership default fix depends on: a bucket that never
had ownership controls written reports BucketOwnerEnforced, and putting that
same value on such a bucket still persists it, so the delete that follows has
something to remove. The put-then-delete case gets its own bucket -- run after
an ObjectWriter put, it would pass against an implementation that skips only
the initial write.
The acl workflow already runs this package against a live weed mini, so it
needs no wiring.
* s3: stop warming every bucket's config at startup
Listing all buckets in BucketRegistry.init() made S3 gateway startup
O(buckets) and pinned every bucket's metadata and config resident, which
does not scale past a few hundred thousand buckets. Both caches already
have lazy miss paths, so load on first access instead and let the
metadata subscription refresh only entries already resident; cold
buckets cost one filer round-trip on their first request.
* s3: bound the per-bucket caches with LRU eviction
The bucket config cache, bucket registry, and their negative caches were
plain maps that only ever grew: the config cache TTL made Get miss but
never evicted the entry, and the not-found sets grew on every probe of a
nonexistent bucket name. Cap all four at 65536 entries with LRU eviction
so a gateway keeps its hot working set and evicted buckets reload from
the filer on next access.
* s3: cache parsed bucket config instead of the full filer entry
Each cached BucketConfig retained the whole bucket entry (extended
attribute map plus raw content bytes) alongside the fields parsed from
it, roughly doubling per-bucket cache cost and keeping data the read
path never looks at. Parse everything up front in
newBucketConfigFromEntry - now also the creator identity, tags,
encryption config, and stored lifecycle XML - and drop the entry.
updateBucketConfig now reads the entry fresh from the filer and diffs
the mapped extended attributes against it, so the patch is computed
against current state instead of a cached copy; the config clone
helpers that existed for that path go away.
* s3: dedup cold bucket-registry loads per bucket
The registry's notFound lock doubled as the load serializer, holding one
global mutex across the filer round-trip so first-touch requests for
different buckets queued behind each other; the cache fill also happened
after the lock was released, so two concurrent misses for the same
bucket could both reach the filer. Replace it with a singleflight per
bucket that fills the cache inside the flight: different buckets load
concurrently, the same bucket loads once.
* s3: return BucketAlreadyOwnedByYou when recreating your own bucket
PutBucket returned BucketAlreadyExists for every existing bucket, even
when the caller already owns it, so idempotent re-creation (e.g. a
container that creates its bucket on startup) couldn't tell "someone
else took the name" from "it's already mine".
Recreating a bucket you own now returns BucketAlreadyOwnedByYou, unless
the request conflicts with the existing bucket: a different Object Lock
setting, or an ACL on the request or the existing bucket. To detect the
latter, a requested non-default canned/grant ACL is now persisted on
creation instead of being dropped.
* s3: fail PutBucket when the existing bucket's config can't be read
When a bucket already exists, an unreadable config left the recreate
defaulting to BucketAlreadyOwnedByYou, masking the backend error and
possibly accepting a conflicting recreate (Object Lock / ACL unknown).
Surface the read error instead.
* s3: return the stored bucket ACL from GetBucketAcl
GetBucketAcl always returned the owner's default full-control grant and
ignored any stored ACL, so a bucket created with a canned ACL or one set
via PutBucketAcl never read back correctly. Decode the stored grants
instead, sharing one grants-to-XML helper with the object ACL handler.
The shared helper also emits each grantee's real xsi:type (e.g. Group for
public-read) instead of a hardcoded CanonicalUser, so group grants read
back correctly for both bucket and object ACLs.
* s3: resolve the right already-exists error on the concurrent-create race
When two requests create the same bucket at once, the loser's mkdir
fails and the handler fell back to a flat BucketAlreadyExists, bypassing
the same-owner idempotency check. Route both the pre-check and the race
fallback through one existingBucketError helper so a same-owner recreate
still gets BucketAlreadyOwnedByYou.
* s3: record the bucket owner's account id at creation
setBucketOwner only stored the creating identity name, so the canonical
account id wasn't available later. Persist it under ExtAmzOwnerKey too,
the same field PutBucketAcl writes, so the bucket owner can be reported
independently of whoever reads it.
* s3: report the bucket owner from GetBucketAcl, not the caller
GetBucketAcl built the ACL Owner from the caller's account header, so an
admin or cross-account read returned the wrong owner. Use the owner
persisted on the bucket, falling back to the caller only when none is
recorded.
* feat(s3): add four bucket configuration handlers
- GetBucketPolicyStatus: computes IsPublic from the existing bucket policy
- PutBucketRequestPayment: companion writer to the existing GET; accepts
only BucketOwner
- GetBucketAccelerateConfiguration: returns <Status>Suspended</Status>
- GetBucketLogging: returns an empty BucketLoggingStatus
Lets AWS SDK probes succeed instead of returning MethodNotAllowed.
* review: route GetBucketPolicyStatus through checkBucket
Mirrors the existence/auth gating used by other bucket handlers and
drops the bespoke filer_pb lookup so NoSuchBucket precedence is
consistent across the API surface.
* review: cap PutBucketRequestPayment body with MaxBytesReader
The body is unmarshalled as RequestPaymentConfiguration, which is a
handful of bytes; reject excessively large payloads up front and
defer Close immediately after wrapping.
* review: gate static getters on checkBucket
GetBucketAccelerateConfiguration and GetBucketLogging now run the
standard bucket existence check before returning the static
Suspended / empty-status response so a missing bucket cannot appear
to have valid configuration.
* review: share cache helper across misc tests; check io.ReadAll error
Accelerate and Logging tests now run through newMiscTestServer like
the others so the checkBucket guard sees a cached bucket; the
ReadAll error is explicitly checked.
Chris Lu