mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-04 11:07:35 +01:00
cd82e663ad3c1f1e40ea7a69e7fb34cddf1f8a90
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris LuandClaude Opus 5.5
|
ca15fdbff6 |
remote.uncache, remote.meta.sync, remote.cache: keep local writes that land while they run (#12382)
* remote.uncache: skip a file overwritten since the walk read it The uncache cleared the chunks of the entry its directory walk returned and wrote it back unconditionally. A file overwritten after the walk read it was reset to the remote pointer, and its new chunks were dropped, so an acknowledged write was lost. Write back only while the filer still holds the entry as read, and report the file as skipped otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: keep files written locally after the directory listing meta.sync lists each local directory once and works from that listing. A changed remote file was written back over the listed entry unconditionally, and a name missing from the listing was created without O_EXCL, so a file written locally in between was replaced by the remote pointer either way. Update only while the filer still holds the listed entry, create exclusively, and report such files as skipped. The create now also surfaces errors the filer returns in its response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.cache: keep files written locally after the lookup The metadata step looks each file up and then either rewrites it from the remote unconditionally or creates it without O_EXCL, so a local write landing between the lookup and the write was replaced by the remote pointer. Reuse doSaveRemoteEntry, which updates only while the filer still holds the looked-up entry, create exclusively, and report such files as skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: do not descend into a name a local file took When the exclusive create of a remote directory finds the name taken, the sync still descended into it, and creating the remote children under a local file would turn that file into a directory. Descend only when the name now holds a directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: delete only the file version it listed The type-change and -deleteStale paths removed a remote-backed file unconditionally, so a local write landing after the directory listing was deleted with it. Condition the delete on the listed mtime, as fs.verify and volume.fsck do; the filer then keeps a file written since. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: condition directory and mtime-less deletes too The listed-mtime cutoff was only sent for files, so a remote-backed directory replaced locally by a file between the emptiness check and the delete was removed unconditionally, and so was a file listed without an mtime. Send the cutoff for directories as well, and for an entry listed without an mtime keep it once a write has set one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test that DeleteEntry applies the request's mtime cutoff Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: keep the listed nanoseconds as the cutoff of an epoch mtime An entry listed at the epoch with a sub-second mtime was sent a 1 ns cutoff, so the filer kept it unchanged although the old code deleted it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * remote.meta.sync: report an entry the filer kept instead of deleting it The filer answers a delete it skipped under the mtime cutoff with success, so the sync printed (delete) for an entry a local write kept. deleteLocalEntry now looks the name up after the delete and prints the outcome, and a replaced file that was kept is not recreated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chris Lu
|
2dbf9c9108 |
s3: evaluate all conditional write headers under the object lock (#12211)
* filer_pb,filer: per-mutation preconditions on ObjectTransaction A transaction's request-level condition evaluates one entry before any mutation runs, so a commit guarding two paths — the object it publishes and the upload directory it assembles — could not be expressed. Give each ObjectMutation an optional condition (evaluated against condition_key, else the mutation's target, under the same lock just before the mutation applies) and report which mutation failed through failed_mutation on the response, so callers can tell a mutation guard's rejection apart from the request-level precondition's. * s3: evaluate multipart completion conditions under the object lock CompleteMultipartUpload checked conditional headers gateway-side before committing, and the routed commit transaction guarded only the upload directory — nothing made If-None-Match: * atomic with the publish, so concurrent conditional completions of the same absent key could all report success. The request's reducible write condition (If-None-Match/If-Match, star or single strong ETag) now rides the commit as the transaction's request condition on the owner filer: on the object path for unversioned buckets, falling back to the .versions directory for suspended and enabled versioning. The upload-exists guard moves onto the PUT mutation, keeping its check-before-publish ordering. Conditions the reducer cannot express stay on the distributed write lock, where check and commit already serialize. * filer,s3: resolve the latest version first for versioned conditional writes * filer: keep directory entries holding content out of the absent-slot rule * s3: keep a gateway-side upload-exists check for filers without mutation conditions * filer: honor null-version-is-latest resolving prefer-alt conditions * drop scratch multipart-race repro programs * s3: note the write lock does not cover routed writes on non-reducible conditions * filer: mirror the gateway's remote-etag precedence in condition evaluation storedEntryETag only knew the extended attribute and the chunk fallback, so a synced remote entry compared chunk-derived ETags while the gateway compares the remote ETag. Follow the same precedence: extended attribute, remote ETag while the entry still holds the remote bytes, Md5, then the chunk hash — and no ETag for an entry holding none. * s3: reduce all conditional write headers into owner-lock clauses Time-based headers, combined conditions, ETag lists, and weak ETags stayed on the distributed write lock because buildWriteCondition could not express them, leaving their gateway check racing owner-routed writes. Reduce every parseable header form to WriteCondition clauses — multiple clauses compose AND, ETag candidates carry the raw list for the filer's canonicalization, and RFC 7232 precedence keeps If-Unmodified-Since hidden by If-Match and If-Modified-Since hidden by If-None-Match — so the owner filer evaluates them under the same object-path lock as the commit. Only unparseable headers still fall back, where the lock path reports the same parse error. |
||
|
|
a6be3c89c2 |
filer: support nanosecond timestamps for conditional delete (#11339) (#12071)
* filer: support nanosecond timestamps for conditional delete (#11339) * shell: fs.verify prune carries the entry mtime in nanoseconds IfNotModifiedAfter now resolves to sub-second precision on the filer, so pass the looked-up mtime_ns along with mtime; the re-lookup check catches same-second rewrites that share the second, and the test mock mirrors the filer's seconds/nanoseconds cutoff. * shell: volume.fsck guards empty-directory deletes with mtime nanoseconds The looked-up mtime_ns joins the seconds in IfNotModifiedAfter so a directory rewritten within the same second is not dropped; the stub filer mirrors the filer's scale heuristic and the race fixtures use a realistic mtime so the nanosecond branch engages. * filer: cover the seconds guard and equality boundary Test second-resolution cutoffs on both sides of the entry's second and the exact-equality nanosecond guard that still deletes. * filer: compare the nanosecond guard as a time, not UnixNano UnixNano overflows for mtimes outside 1678-2262, letting a far-future entry slip past the guard; time.After compares exactly at every epoch. The seconds branch keeps Unix() so a same-second cutoff still deletes. * filer_pb: add explicit if_not_modified_after_ns to DeleteEntryRequest * filer: prefer explicit nanoseconds guard, keep if_not_modified_after as seconds * shell: send seconds and nanoseconds delete cutoffs, skip unexpressible mtimes * filer: merge conditional-delete guards into one nanosecond cutoff DeleteEntryMetaAndData carried both the seconds and nanoseconds guards even though the precedence between them is fixed, so every caller passed a dead parameter. The gRPC handler now folds if_not_modified_after into nanoseconds at the boundary, where (sec+1)*1e9-1 keeps the whole-second cutoff exact, and the filer takes a single ifNotModifiedAfterNs. * filer: hold the conditional-delete cutoff as a time, not int64 nanoseconds A seconds-only guard at or past year 2262 saturated to MaxInt64 nanoseconds (~2262), so a delete whose cutoff matched a far-future mtime was skipped while the request still reported success, and any entry between 2262 and the real cutoff survived pruning. The merged cutoff is now a time.Time, where a whole-second guard maps to the last nanosecond of its second for any int64 second. --------- Co-authored-by: user.email <chrislusf@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Chris LuandDevin
|
a901c1a5e2 |
filer: compare IF_ENTRY_EQUAL chunks by fid, not file_id (#11514)
The stored entry came through FindEntry, which restores chunk file ids from their fid form, while an expected entry built from a metadata-log event still carries the serialized form (file_id moved into fid). The proto.Equal saw file_id "" against the restored id and refused every stamp, so remote.sync re-uploaded each entry and the RemoteEntry stamp never landed. Clone both sides and run BeforeEntrySerialization before comparing, so chunks match on their fid and the file_id spelling is ignored; the stored entry and the request's ExpectedEntry are left untouched. Generated with [Devin](https://devin.ai) Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
jsasandChris Lu
|
5b79f51e3c |
filer.remote.sync: stamp entries with IF_CHUNKS_EQUAL so a stale write-back cannot delete live chunks (#11435)
* filer.remote.sync: stamp entries with IF_CHUNKS_EQUAL so a stale write-back cannot delete live chunks updateLocalEntry records the RemoteEntry stamp after an upload by writing the event's entry back with UpdateEntry. The filer deletes every stored chunk absent from an updated entry, so when the file was rewritten while its upload was in flight (or the event is a replay), the stale snapshot deletes the rewrite's chunks: the entry then points at the new fid with no needle behind it, and the rewrite's own upload fails and is skipped as superseded. The stamp write now carries WriteCondition IF_CHUNKS_EQUAL over the event's chunk fids, evaluated by the filer under the path lock. A refused stamp means the filer moved past this event; the superseding event follows in the log and stamps the current entry, so the refusal is logged and skipped like a superseded upload. Reproduction: weed server -filer plus a weed server -s3 remote, remote.mount, filer.remote.sync; hold the remote (docker pause) so one upload stays in flight, rewrite the file through the filer, unpause. Before: the entry's chunk is 404 on every volume server. After: the stale stamp is refused, the rewrite's chunk stays live and reads back after a vacuum. * filer.remote.sync: stamp entries with IF_ENTRY_EQUAL so stale inline content or metadata cannot be restored The IF_CHUNKS_EQUAL guard compared only the chunk fid multiset, so a rewrite that touched inline content or metadata alone still compared equal and the stale snapshot overwrote the live entry. The new clause compares the whole stored entry against the event's entry under the same path lock. * filer: route conditional UpdateEntry to the entry's owner filer Two filers locking the same path locally could still pass a stale condition on the non-owner while the owner's entry had moved on. When a condition or expected_extended precondition is set, forward the request to the entry's owner the same way conditional CreateEntry does, with is_moved bounding the hop. * filer: compare IF_ENTRY_EQUAL against the normalized expected entry FindEntry grows FileSize to the chunk extent, so a raw event entry with FileSize still zero failed the condition on an unchanged file and the stamp was skipped, letting a replay upload the object again. * filer.remote.sync: classify refused stamps by gRPC status only A FailedPrecondition substring in an unrelated error would have been swallowed as a skipped stamp; status.FromError already unwraps. * remote sync: keep the event entry intact for IF_ENTRY_EQUAL --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
Chris Lu
|
f46b2a1925 |
Stop the filer test helpers from pinning gigabytes of log buffers (#10560)
* log buffer: wake the interval loop on shutdown instead of sleeping through it loopInterval parked in time.Sleep(flushInterval) and only re-checked IsStopping when it woke, so a buffer shut down early kept both loop goroutines - and the PreviousBufferCount+1 slabs of BufferSize they reach - alive for up to a full interval afterwards. Select on shutdownCh against a ticker instead, and give the loops a WaitGroup so a test can observe that they exit. * test: release the filers the server tests build Every helper here left its filer's meta log buffer running, so each test pinned PreviousBufferCount+1 buffers of BufferSize for the rest of the run: ~3.5GB of live heap across the package, which overruns the address space on linux/386 and kills the 32-bit job with an out-of-memory throw. Thread the test through the helpers so the buffer is shut down on cleanup, and shut the subscribe harness's filer down outright - its deletion loop keeps the whole filer reachable otherwise. That harness quiesces its flush path first, since Filer.Shutdown closes the store a flush still in flight would write through. |
||
|
Chris Lu
|
cdb60069a6 |
filer: conditional UpdateEntry with a chunk-set write condition (#10382)
* filer: accept a WriteCondition on UpdateEntry, under the per-path lock UpdateEntry was a bare read-modify-write: the precondition check, the chunk garbage diff, and the store write could interleave with a concurrent update to the same path. Take the per-path lock CreateEntry already holds, and evaluate an optional CreateEntry-style WriteCondition under it, failing with FailedPrecondition like expected_extended. * filer: IF_CHUNKS_EQUAL write condition compares the stored chunk fid set A chunk-preserving read-modify-write (tagging, setattr, copy-in-place) races UpdateEntry's garbage diff: if a concurrent update empties the chunk list first, the stale writer's commit resurrects fids that are already queued for deletion, stranding the entry on a dead needle once vacuum reclaims it. The reverse also holds: a writer that read an empty chunk list can wipe chunks a concurrent update just added. IF_CHUNKS_EQUAL guards both: the stored chunk fid multiset must still equal what the caller read, order-independent, with an empty fids list expecting no chunks. Absent entry counts as no chunks for CreateEntry overwrites and transactions. * filer: delete and append serialize on the entry path lock DeleteEntry queues the entry's chunks for deletion and AppendToEntry rewrites the chunk list, but neither held the per-path lock, so either could interleave with a conditional update between its precondition check and its write — a passed IF_CHUNKS_EQUAL would then resurrect fids already on the deletion queue, or clobber a freshly appended chunk. AppendToEntry keeps the cluster lock for cross-filer append serialization; the path lock covers the local read-modify-write. * filer: reuse lockPath in UpdateEntry lookup |
||
|
Chris Lu
|
fa7056dc6f |
s3: route object-lock version-specific deletes off the distributed lock (#9657)
A version-specific DELETE (real version or the null version, including object-lock WORM-checked ones and governance-bypass) now runs as one routed transaction on the object's owner instead of holding the distributed lock. For a real version: recompute the .versions pointer excluding the version (repoint-before-delete, so a crash leaves a recoverable orphan rather than a dangling pointer), then delete the version file, under the object's per-path lock. The null version is the regular object entry, deleted directly (no pointer). Object-lock buckets gate the delete on the version's WORM guards evaluated on the owner: legal hold (always) + retention (while not elapsed). Governance bypass scopes the retention guard to COMPLIANCE mode, so the filer allows a governance-mode delete while still denying compliance and legal hold — the gateway never reads the version. Three primitives make this expressible: - ObjectTransaction.condition_key: evaluate the condition against a named entry (the version) while the lock stays on lock_key (the object). - Recompute.exclude_name: omit a child from the scan, to repoint before delete. - WriteCondition.Clause gate_key/gate_value: scope IF_EXTENDED_TIME_ELAPSED to a mode, expressing governance bypass without a gateway-side read. |
||
|
Chris Lu
|
83195fc111 |
filer: reuse the caller's fetched entry in CreateEntry (#9645)
CreateEntry starts with a FindEntry to load the current entry. A conditional CreateEntry already fetched that entry to evaluate the precondition under the per-path lock, so the create repeated the lookup. Add an existing *Entry parameter: when non-nil it is used as the current entry and the internal lookup is skipped; nil keeps the lookup. The gRPC CreateEntry handler passes the entry it fetched for the precondition, removing the redundant read while the lock is held. All other callers pass nil. |
||
|
Chris Lu
|
e2203b2a0b |
filer: add extended-attribute guard clauses for object-lock (#9648)
Routing object-lock buckets off the distributed lock needs the retention and legal-hold check to run atomically with the write, under the per-path lock. Move just the comparison into the filer, not the S3 semantics: two generic clause kinds on an extended attribute. IF_EXTENDED_NOT_EQUAL blocks while extended[ext_key] equals ext_value (a legal hold). IF_EXTENDED_TIME_ELAPSED blocks while extended[ext_key], read as a unix- second deadline, is in the future against the filer's clock (retention); a malformed deadline fails safe. The caller composes these from the object-lock state and, for a governance bypass, simply omits the retention clause once the bypass is authorized -- the filer makes no authorization decision and keeps no S3 knowledge. |
||
|
Chris Lu
|
b18d3dc96c |
filer: evaluate a write precondition in CreateEntry (#9650)
Add an optional WriteCondition to CreateEntryRequest. When set, the filer evaluates it against the current entry while holding the per-path lock, so the check and the write are atomic on this filer, and returns PRECONDITION_FAILED when it does not hold. The caller must route the key's writes to the owner filer for the check to be authoritative. A condition is a list of clauses that all must hold (logical AND). One clause is the common case; several express what a single comparison cannot: an ETag set (If-Match / If-None-Match with multiple values), weak-ETag comparison, and compound conditions. ETag comparison mirrors the S3 gateway's precedence (stored Seaweed ETag attribute, then the Md5/chunk fallback) and follows RFC 7232 strong/weak rules, so results match without coupling the filer to S3 handling. Condition parsing and evaluation live in filer_grpc_server_condition.go. |