8 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandClaude Opus 5.5	45cd6033a0	s3: make conditional DeleteObject atomic on versioned buckets (#12624) ... * s3: re-check a versioned DeleteObject If-Match under the object lock A versioned or suspended DeleteObject without a versionId routed to the owner filer after checking If-Match outside any lock, so concurrent conditional deletes could all pass the check and each add a delete marker. Keep conditional deletes on the lock path, which re-checks the precondition under the object write lock, as conditional versioned PUTs already do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: guard a conditional version delete on the owner filer DeleteObject with a versionId checked If-Match before routing the delete to the owner filer, so concurrent conditional deletes of one version could all report success. Carry the If-Match clause into the owner's transaction as a guard on the version (the null entry for versionId=null), evaluated under the object's path lock before anything applies, and map that guard failing to 412 instead of the object-lock 403. An If-Match the filer cannot evaluate stays on the lock path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: carry an If-Match ETag list into the routed delete A DeleteObject whose If-Match listed several ETags fell back to the gateway's object write lock, while a single-ETag delete of the same key ran under the owner filer's path lock. The two locks do not exclude each other, so a list delete and a single-ETag delete could both report success. Send every candidate to the filer, whose strong comparison matches checkDeleteIfMatch, so all conditional deletes with an owner are decided under the owner's lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: fail a conditional null delete on an owner transport error The owner may have committed the delete before the error, and the gateway lock path it fell back to does not exclude routed deletes, so a second If-Match delete could pass there too. Answer 500 as the specific-version path does; unconditional deletes still fall back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: fail a conditional null delete on any owner error but the nested refusal Only the owner's non-empty-folder refusal is known to apply nothing. After any other owner error the delete may have been partly applied, and a lock-path re-check does not exclude a concurrent routed delete of the same version, so a conditional null-version delete answers 500 instead of falling back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: strip a nested conditional null delete on the owner A conditional versionId=null delete that the owner refuses because other keys are nested under the entry used to fall back to the gateway lock and re-check If-Match there. That lock does not exclude routed deletes: once the last nested key is removed, a concurrent routed delete can pass the same check on the owner, and the fallback then treats the missing entry as deleted, so both requests answer 204. Demote the entry on the owner instead, in a transaction guarded by the If-Match clauses and by the entry it was read as, with the object-lock guards on the request. A missing or already demoted entry answers 412. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: add an IF_OBJECT_EXISTS write condition IF_EXISTS also holds for a plain directory, which is what is left of a key once the object on it is stripped while other keys stay nested under it. IF_OBJECT_EXISTS holds only for an entry that carries an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: require an object for a conditional null delete with If-Match: * Stripping the null version off an entry other keys are nested under leaves a plain directory. Once the last nested key is gone, a routed delete with If-Match: * still found an entry there and removed the empty directory, so a second conditional delete could also answer 204. The owner now checks that the entry carries an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: settle a wildcard null delete on the entry when the filer predates IF_OBJECT_EXISTS A filer older than IF_OBJECT_EXISTS fails it closed, so with the gateway upgraded first every If-Match: * delete of a null version answered 412. When the narrowed guard fails, the gateway now reads the entry on the owner: absent or a plain directory is a 412, otherwise the file is deleted, or the directory stripped, under IF_EXISTS and IF_ENTRY_EQUAL on the entry it read. The demote keeps the caller's If-Match clauses, since the entry guard already requires an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-13 12:47:22 -08:00
Chris Lu	2dbf9c9108	s3: evaluate all conditional write headers under the object lock (#12211) ... * filer_pb,filer: per-mutation preconditions on ObjectTransaction A transaction's request-level condition evaluates one entry before any mutation runs, so a commit guarding two paths — the object it publishes and the upload directory it assembles — could not be expressed. Give each ObjectMutation an optional condition (evaluated against condition_key, else the mutation's target, under the same lock just before the mutation applies) and report which mutation failed through failed_mutation on the response, so callers can tell a mutation guard's rejection apart from the request-level precondition's. * s3: evaluate multipart completion conditions under the object lock CompleteMultipartUpload checked conditional headers gateway-side before committing, and the routed commit transaction guarded only the upload directory — nothing made If-None-Match: * atomic with the publish, so concurrent conditional completions of the same absent key could all report success. The request's reducible write condition (If-None-Match/If-Match, star or single strong ETag) now rides the commit as the transaction's request condition on the owner filer: on the object path for unversioned buckets, falling back to the .versions directory for suspended and enabled versioning. The upload-exists guard moves onto the PUT mutation, keeping its check-before-publish ordering. Conditions the reducer cannot express stay on the distributed write lock, where check and commit already serialize. * filer,s3: resolve the latest version first for versioned conditional writes * filer: keep directory entries holding content out of the absent-slot rule * s3: keep a gateway-side upload-exists check for filers without mutation conditions * filer: honor null-version-is-latest resolving prefer-alt conditions * drop scratch multipart-race repro programs * s3: note the write lock does not cover routed writes on non-reducible conditions * filer: mirror the gateway's remote-etag precedence in condition evaluation storedEntryETag only knew the extended attribute and the chunk fallback, so a synced remote entry compared chunk-derived ETags while the gateway compares the remote ETag. Follow the same precedence: extended attribute, remote ETag while the entry still holds the remote bytes, Md5, then the chunk hash — and no ETag for an entry holding none. * s3: reduce all conditional write headers into owner-lock clauses Time-based headers, combined conditions, ETag lists, and weak ETags stayed on the distributed write lock because buildWriteCondition could not express them, leaving their gateway check racing owner-routed writes. Reduce every parseable header form to WriteCondition clauses — multiple clauses compose AND, ETag candidates carry the raw list for the filer's canonicalization, and RFC 7232 precedence keeps If-Unmodified-Since hidden by If-Match and If-Modified-Since hidden by If-None-Match — so the owner filer evaluates them under the same object-path lock as the commit. Only unparseable headers still fall back, where the lock path reports the same parse error.	2026-10-29 11:49:48 -07:00
Chris LuandDevin	a901c1a5e2	filer: compare IF_ENTRY_EQUAL chunks by fid, not file_id (#11514) ... The stored entry came through FindEntry, which restores chunk file ids from their fid form, while an expected entry built from a metadata-log event still carries the serialized form (file_id moved into fid). The proto.Equal saw file_id "" against the restored id and refused every stamp, so remote.sync re-uploaded each entry and the RemoteEntry stamp never landed. Clone both sides and run BeforeEntrySerialization before comparing, so chunks match on their fid and the file_id spelling is ignored; the stored entry and the request's ExpectedEntry are left untouched. Generated with [Devin](https://devin.ai) Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-09-29 11:34:07 +08:00
jsasandChris Lu	5b79f51e3c	filer.remote.sync: stamp entries with IF_CHUNKS_EQUAL so a stale write-back cannot delete live chunks (#11435) ... * filer.remote.sync: stamp entries with IF_CHUNKS_EQUAL so a stale write-back cannot delete live chunks updateLocalEntry records the RemoteEntry stamp after an upload by writing the event's entry back with UpdateEntry. The filer deletes every stored chunk absent from an updated entry, so when the file was rewritten while its upload was in flight (or the event is a replay), the stale snapshot deletes the rewrite's chunks: the entry then points at the new fid with no needle behind it, and the rewrite's own upload fails and is skipped as superseded. The stamp write now carries WriteCondition IF_CHUNKS_EQUAL over the event's chunk fids, evaluated by the filer under the path lock. A refused stamp means the filer moved past this event; the superseding event follows in the log and stamps the current entry, so the refusal is logged and skipped like a superseded upload. Reproduction: weed server -filer plus a weed server -s3 remote, remote.mount, filer.remote.sync; hold the remote (docker pause) so one upload stays in flight, rewrite the file through the filer, unpause. Before: the entry's chunk is 404 on every volume server. After: the stale stamp is refused, the rewrite's chunk stays live and reads back after a vacuum. * filer.remote.sync: stamp entries with IF_ENTRY_EQUAL so stale inline content or metadata cannot be restored The IF_CHUNKS_EQUAL guard compared only the chunk fid multiset, so a rewrite that touched inline content or metadata alone still compared equal and the stale snapshot overwrote the live entry. The new clause compares the whole stored entry against the event's entry under the same path lock. * filer: route conditional UpdateEntry to the entry's owner filer Two filers locking the same path locally could still pass a stale condition on the non-owner while the owner's entry had moved on. When a condition or expected_extended precondition is set, forward the request to the entry's owner the same way conditional CreateEntry does, with is_moved bounding the hop. * filer: compare IF_ENTRY_EQUAL against the normalized expected entry FindEntry grows FileSize to the chunk extent, so a raw event entry with FileSize still zero failed the condition on an unchanged file and the stamp was skipped, letting a replay upload the object again. * filer.remote.sync: classify refused stamps by gRPC status only A FailedPrecondition substring in an unrelated error would have been swallowed as a skipped stamp; status.FromError already unwraps. * remote sync: keep the event entry intact for IF_ENTRY_EQUAL --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	2026-09-25 09:02:34 +08:00
Chris Lu	cdb60069a6	filer: conditional UpdateEntry with a chunk-set write condition (#10382) ... * filer: accept a WriteCondition on UpdateEntry, under the per-path lock UpdateEntry was a bare read-modify-write: the precondition check, the chunk garbage diff, and the store write could interleave with a concurrent update to the same path. Take the per-path lock CreateEntry already holds, and evaluate an optional CreateEntry-style WriteCondition under it, failing with FailedPrecondition like expected_extended. * filer: IF_CHUNKS_EQUAL write condition compares the stored chunk fid set A chunk-preserving read-modify-write (tagging, setattr, copy-in-place) races UpdateEntry's garbage diff: if a concurrent update empties the chunk list first, the stale writer's commit resurrects fids that are already queued for deletion, stranding the entry on a dead needle once vacuum reclaims it. The reverse also holds: a writer that read an empty chunk list can wipe chunks a concurrent update just added. IF_CHUNKS_EQUAL guards both: the stored chunk fid multiset must still equal what the caller read, order-independent, with an empty fids list expecting no chunks. Absent entry counts as no chunks for CreateEntry overwrites and transactions. * filer: delete and append serialize on the entry path lock DeleteEntry queues the entry's chunks for deletion and AppendToEntry rewrites the chunk list, but neither held the per-path lock, so either could interleave with a conditional update between its precondition check and its write — a passed IF_CHUNKS_EQUAL would then resurrect fids already on the deletion queue, or clobber a freshly appended chunk. AppendToEntry keeps the cluster lock for cross-filer append serialization; the path lock covers the local read-modify-write. * filer: reuse lockPath in UpdateEntry lookup	2026-07-21 00:19:15 -07:00
Chris Lu	fa7056dc6f	s3: route object-lock version-specific deletes off the distributed lock (#9657) ... A version-specific DELETE (real version or the null version, including object-lock WORM-checked ones and governance-bypass) now runs as one routed transaction on the object's owner instead of holding the distributed lock. For a real version: recompute the .versions pointer excluding the version (repoint-before-delete, so a crash leaves a recoverable orphan rather than a dangling pointer), then delete the version file, under the object's per-path lock. The null version is the regular object entry, deleted directly (no pointer). Object-lock buckets gate the delete on the version's WORM guards evaluated on the owner: legal hold (always) + retention (while not elapsed). Governance bypass scopes the retention guard to COMPLIANCE mode, so the filer allows a governance-mode delete while still denying compliance and legal hold — the gateway never reads the version. Three primitives make this expressible: - ObjectTransaction.condition_key: evaluate the condition against a named entry (the version) while the lock stays on lock_key (the object). - Recompute.exclude_name: omit a child from the scan, to repoint before delete. - WriteCondition.Clause gate_key/gate_value: scope IF_EXTENDED_TIME_ELAPSED to a mode, expressing governance bypass without a gateway-side read.	2026-05-24 11:41:08 -07:00
Chris Lu	e2203b2a0b	filer: add extended-attribute guard clauses for object-lock (#9648) ... Routing object-lock buckets off the distributed lock needs the retention and legal-hold check to run atomically with the write, under the per-path lock. Move just the comparison into the filer, not the S3 semantics: two generic clause kinds on an extended attribute. IF_EXTENDED_NOT_EQUAL blocks while extended[ext_key] equals ext_value (a legal hold). IF_EXTENDED_TIME_ELAPSED blocks while extended[ext_key], read as a unix- second deadline, is in the future against the filer's clock (retention); a malformed deadline fails safe. The caller composes these from the object-lock state and, for a governance bypass, simply omits the retention clause once the bypass is authorized -- the filer makes no authorization decision and keeps no S3 knowledge.	2026-05-23 19:38:08 -07:00
Chris Lu	b18d3dc96c	filer: evaluate a write precondition in CreateEntry (#9650) ... Add an optional WriteCondition to CreateEntryRequest. When set, the filer evaluates it against the current entry while holding the per-path lock, so the check and the write are atomic on this filer, and returns PRECONDITION_FAILED when it does not hold. The caller must route the key's writes to the owner filer for the check to be authoritative. A condition is a list of clauses that all must hold (logical AND). One clause is the common case; several express what a single comparison cannot: an ETag set (If-Match / If-None-Match with multiple values), weak-ETag comparison, and compound conditions. ETag comparison mirrors the S3 gateway's precedence (stored Seaweed ETag attribute, then the Md5/chunk fallback) and follows RFC 7232 strong/weak rules, so results match without coupling the filer to S3 handling. Condition parsing and evaluation live in filer_grpc_server_condition.go.	2026-05-23 16:29:14 -07:00