4 Commits
Author SHA1 Message Date
Chris Lu 6536fac9bd s3: answer NoSuchBucket for encryption and policy writes to a missing bucket (#12133)
* s3: answer NoSuchBucket for encryption and policy writes to a missing bucket

PutBucketEncryption, DeleteBucketEncryption, and PutBucketPolicy surfaced
the filer's bucket-miss as InternalError; GetBucketEncryption masked it as
ServerSideEncryptionConfigurationNotFoundError. Map the miss to
NoSuchBucket instead, matching AWS. Internal readers still treat a missing
bucket as "no encryption configured" so uploads keep their behavior.

* s3: wrap ErrBucketNotFound only for verified misses in bucket policy ops

The filer lookup helpers in get/set/deleteBucketPolicy marked every
LookupDirectoryEntry failure as bucket-not-found, so a transient filer
outage answered NoSuchBucket for a live bucket. Route them through
filer_pb.LookupEntry, which normalizes only true misses to ErrNotFound.

* s3: answer NoSuchBucket when the policy delete's own write loses the bucket

* s3: classify a filer NotFound on the policy UpdateEntry as ErrBucketNotFound

* s3: require the bucket entry to exist on config patch transactions

* s3: answer NoSuchBucket when tagging and CORS writes lose the bucket mid-request

* s3: keep the existence check on no-op bucket config patches

An unchanged config write produced an empty patch that early-returned
before the transaction, so a bucket deleted between the read and the
write still reported success. Ship the condition-only transaction so
IF_EXISTS runs on its own.

* s3: guard the iam step and cover failure-vs-miss classification

Nil-guard s3a.iam in the bucket policy handlers so the handlers do not
panic when no IAM server is wired (unit tests, embedded servers), treat
a "precondition failed" response string the same as the error code when
mapping a failed patch to ErrNotFound, and add handler-level tests that
pin the miss/failure distinction: missing buckets and vanished mid-write
buckets answer 404 NoSuchBucket while transient filer failures stay 500
InternalError.
2026-10-27 13:07:05 -07:00
Chris Lu 8f6121694a test: fix HEAD bucket assertions and FUSE failover startup timing (#11967)
test: fix HEAD bucket assertions and warm up failover clusters
2026-10-22 14:34:52 -07:00
Chris LuandDevin 5d80ac39b8 s3api: verify under the write lock before re-committing an ambiguous routed PUT (#11649)
* s3api: verify under the write lock before re-committing an ambiguous routed PUT

A routed PUT whose response was lost after the owner committed, or whose
transaction returned a store error, fell straight into the lock path and
re-sent the same entry. A concurrent PUT that had superseded the commit in
the meantime had already deleted this entry's chunks as old, so the
re-commit restored metadata pointing at dead needles and the object read
back 404 permanently.

The lock path now resolves the routed attempt's outcome inside the write
lock first: a stored entry with the uploaded chunks means the route
landed; a different stored entry or an unresolved lookup refuses the
re-commit with ServiceUnavailable so the client retries with a fresh
upload; only a proven-absent entry falls through to the normal create.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: tighten ambiguous routed PUT recovery

- a match found only after an unanswered filer is not authoritative;
  the unreachable filer may hold a newer entry, so refuse instead of
  finalizing on it (both recovery paths)
- a failed post-recovery finalization now rolls the recovered entry
  back, matching the create path's undo
- a proven-absent entry is only re-committed after probing that the
  uploaded chunks' needles are still alive; a committed-and-deleted
  PUT would otherwise write back an entry pointing at reclaimed
  needles
- the .versions latest pointer no longer flips back to an older
  version when a newer one committed while the write was uncertain

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: stamp late versions noncurrent when a newer latest pointer wins

The keep-newer early return in updateLatestVersionInDirectory left the
version just stored without ExtNoncurrentSinceNsKey, so the lifecycle
engine could never age it out.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: only a failure past mutation 0 makes a routed PUT ambiguous

A deterministic refusal at the PUT mutation (e.g. "existing entry is a
directory") applied nothing, but marking it ambiguous sent the lock-path
fallback through conservative recovery, which found the directory entry
and refused with 503 instead of the correct 409.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: keep all transaction errors ambiguous; route directory conflicts through the lock path

* s3: verify uploaded chunks before re-committing over a directory

A stored directory does not prove the routed PUT never committed: the
route can land, a delete can remove the entry and its chunks, and a
nested write can recreate the directory before recovery takes the lock.
Re-committing then stores an entry pointing at dead needles. Probe the
uploaded chunks first and refuse with ServiceUnavailable when they can
no longer be verified.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-10 08:52:34 +08:00
Chris Lu d850f36513 s3: distinguish a failed bucket lookup from a missing bucket on HEAD (#11000)
HeadBucket treated any lookup error as ErrNoSuchBucket, so a transient
filer failure answered 404 instead of 500 and clients stopped retrying.
Split the two cases the way the bucket policy handlers already do.

Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU
2026-08-27 22:26:15 -07:00