2 Commits

Include renames

Author	SHA1	Message	Date
Chris Lu	e6a0534e3b	s3api: make ambiguous routed-PUT recovery atomic and stop transaction replay (#11787) ... * filer_pb: map PRECONDITION_FAILED to a sentinel error A conditional CreateEntry that the owner's path lock refuses comes back as FilerError_PRECONDITION_FAILED; mapping it to ErrPreconditionFailed lets the S3 gateway tell a rejected precondition apart from a transport failure and report 412 instead of retrying the write. * s3api: make the ambiguous-PUT recovery create conditional under the lock createAfterAmbiguousRoute verified state and then issued an unconditional create, so a write landing between the lookup and the create was still clobbered by the stale re-commit (issue #11631). The recovery create now carries a WriteCondition — IF_NOT_EXISTS for a proven-absent entry, IF_ENTRY_EQUAL for a directory it may need to write onto — which the owner filer evaluates under its per-path lock, making the check and the write atomic. A refused precondition loops back to re-read the state instead of overwriting whatever won the race. * s3api: do not fail over a routed transaction past a possibly-sent request withFilerClientFailover retried the callback on the next filer on any transport error, so an ObjectTransaction whose response was lost could be re-applied over a write that committed in between (issue #11631). A gRPC error cannot distinguish "never sent" from "committed", but channel readiness can: a non-replayable callback now runs only on a connection that just reached READY, and failover continues only over channels that never came up — the one failure mode that provably sent nothing. * s3api: run ambiguous-route recovery for empty PUTs too The recovery was gated on a nonempty chunk list, so a zero-byte PUT whose routed transaction lost its response still fell back to an unconditional create that could overwrite a newer write and delete its chunks. Chunk-count mismatch on an empty upload list is enough to distinguish a superseding entry from this PUT's — there is nothing to save by skipping the check. * s3api: keep conditional recovery creates off the replay path The conditional create in the ambiguous-PUT fallback rode the replayable WithFilerClient failover, so a committed create whose response was lost could be re-sent to another filer — where a delete in between would let IF_NOT_EXISTS pass again and resurrect an entry pointing at reclaimed chunks. Conditional creates now run through the non-replayable failover; an ambiguous result falls through to confirmCreateLanded, which resolves the committed entry rather than resending. * s3api: do not mark a filer unhealthy for an answered request A reply that reached the callback — including an application-level failure like a refused write precondition — proves the filer is alive. Checking received before the health bookkeeping keeps definitive application errors from tripping the failure circuit or flagging a live routed owner as unreachable. * s3api: do not claim an empty stored entry as an ambiguous PUT's own Two chunkless entries compare equal under sameFileChunks, so an older empty object satisfies "the route landed" even though it predates this write — recovery would return success while leaving the request's Content-Type and metadata unapplied. A landed PUT stores the entry exactly as built, so require mime/size/md5/content and the extended attributes to match before claiming it; anything else stays uncertain and the client retries. * s3api: prove ambiguous empty PUTs by write id and pin recovery to the owner - Conditional recovery creates are sent to the routed owner alone instead of failing over: the condition only serializes with routed writes under the owner's per-path lock, and another filer's forwarding is only as correct as its ring view. An unreachable owner now yields an uncertain result rather than a create under a different serialization boundary. - A proven-absent path no longer satisfies an empty, chunkless PUT: the routed commit may have landed and been deleted, and there are no chunks to verify, so IF_NOT_EXISTS could resurrect a removed object. Recovery returns uncertain for the client to retry. - Chunkless identity moves from metadata equality to a per-write id (x-seaweedfs-write-id) stamped into Extended: an identical older object can match mime/md5/content but not the id. Entries without this write's id are not claimed, and confirmCreateLanded can now positively confirm a committed empty object instead of always reporting failure.	2026-10-17 21:36:39 -07:00
Chris LuandDevin	5d80ac39b8	s3api: verify under the write lock before re-committing an ambiguous routed PUT (#11649) ... * s3api: verify under the write lock before re-committing an ambiguous routed PUT A routed PUT whose response was lost after the owner committed, or whose transaction returned a store error, fell straight into the lock path and re-sent the same entry. A concurrent PUT that had superseded the commit in the meantime had already deleted this entry's chunks as old, so the re-commit restored metadata pointing at dead needles and the object read back 404 permanently. The lock path now resolves the routed attempt's outcome inside the write lock first: a stored entry with the uploaded chunks means the route landed; a different stored entry or an unresolved lookup refuses the re-commit with ServiceUnavailable so the client retries with a fresh upload; only a proven-absent entry falls through to the normal create. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3api: tighten ambiguous routed PUT recovery - a match found only after an unanswered filer is not authoritative; the unreachable filer may hold a newer entry, so refuse instead of finalizing on it (both recovery paths) - a failed post-recovery finalization now rolls the recovered entry back, matching the create path's undo - a proven-absent entry is only re-committed after probing that the uploaded chunks' needles are still alive; a committed-and-deleted PUT would otherwise write back an entry pointing at reclaimed needles - the .versions latest pointer no longer flips back to an older version when a newer one committed while the write was uncertain Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3api: stamp late versions noncurrent when a newer latest pointer wins The keep-newer early return in updateLatestVersionInDirectory left the version just stored without ExtNoncurrentSinceNsKey, so the lifecycle engine could never age it out. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3api: only a failure past mutation 0 makes a routed PUT ambiguous A deterministic refusal at the PUT mutation (e.g. "existing entry is a directory") applied nothing, but marking it ambiguous sent the lock-path fallback through conservative recovery, which found the directory entry and refused with 503 instead of the correct 409. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3: keep all transaction errors ambiguous; route directory conflicts through the lock path * s3: verify uploaded chunks before re-committing over a directory A stored directory does not prove the routed PUT never committed: the route can land, a delete can remove the entry and its chunks, and a nested write can recreate the directory before recovery takes the lock. Re-committing then stores an entry pointing at dead needles. Probe the uploaded chunks first and refuse with ServiceUnavailable when they can no longer be verified. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-10 08:52:34 +08:00