mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-19 02:15:02 +01:00
94de65b3b5d79b85e1112321199e81ccc1273cc1
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b50d4a9cab |
mount: don't spend the close()/fsync commit deadline on uploads (#12641)
* mount: start the close()/fsync commit deadline after the data upload Flush, Fsync and the truncate path of SetAttr create the 30 s metadataFlushTimeout context on entry and pass it to doFlush, so the clock also runs through FlushData, the upload of the dirty pages. That upload has its own timeouts and retries. When an assign through a stalled filer or a slow volume server takes most of the 30 s, CreateEntry then fails at once with "context deadline exceeded" and close() returns EIO for data that was already stored. The deadline is meant for the commit alone (#10102: "a healthy CreateEntry completes in well under a second"). doFlush now takes the commit timeout and starts the deadline once the data is uploaded. close() still cannot hang on a stuck filer: the commit keeps its 30 s bound. Release and the flush before a rename pass 0, as before (no overall bound, only the per-call filer timeouts). Seen in production on a training cluster: a 30 s stall of one replica's disk, and a filer that stalled for about 30 s, each failed close() with EIO although the upload had succeeded. Tests: close(), fsync() and a truncate below dirty data, each after an assign slower than the deadline (all three fail before this change), and a filer that never answers CreateEntry still fails close() at the deadline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: charge the flush commit deadline only for its filer calls Starting the 30 s deadline after the data upload is not enough. flushMetadataToFiler does more than call the filer before it commits: it waits for the handle's locks, reads the file back and uploads it again when its chunks are bloated (maybeMergeChunks), and uploads a manifest once enough chunks have built up (MaybeManifestize). None of that takes the context. When it outlasts the deadline, CreateEntry again fails at once with "context deadline exceeded", and close() or fsync() returns an error for data that was stored. The deadline becomes a budget, commitBudget, charged only while a filer call of the commit runs: CreateEntry, and the read of a changed hard link before it. The attempts of one flush share it, and a flush is not retried once it is spent, so a stuck filer still fails close() after 30 s. Release, the flush before a rename and the async flush pass no budget, as before. Seen in production: the merge of a 1.8 MiB sqlite file with 183 chunks took 32 s because one chunk's read first tried a volume server that was down, and the commit failed 32 microseconds after the merge finished. Tests: a flush whose manifest upload is slower than the deadline commits (it fails without this change), and the filer calls of one flush share the budget. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: give the stalled-lookup refusal test the new doFlush argument TestContentFlushRefusalWithStalledLookup came in from master after this branch changed doFlush's signature, so the merged package did not build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
652648d9c4 |
mount: merge concurrent content changes from another mount instead of failing or dropping them (#12596)
* mount: merge a refused content flush onto the stored entry A content flush conditioned on the handle's base failed with EIO whenever a peer had changed the file since, so two mounts appending to one file could not both close it cleanly. On a refused condition, reread the entry and replay this handle's changes since its base onto it: the chunks it added or dropped, and the attributes, extended keys and inline content it changed. A local truncation keeps its size and clips the peer's chunks to it; otherwise the larger size wins. The merged entry is published against the stored one, up to three times, and installed in the handle so its next flush keeps the peer's chunks. A replaced or deleted file is still refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: merge a peer's content change into a dirty handle A subscription event carrying a peer's content change replaced a dirty handle's entry and discarded its unflushed pages, so the handle's writes were lost and its close still returned OK. Fold the peer's entry into the handle with the same merge a refused flush uses, keep the dirty pages, and take the peer's entry as the new base. A replaced file still invalidates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: stop a flush merge once a filer accepts the write withFilerClient can call the callback on several filers. A refusal from one left its stored entry behind, so a later filer's success went on to another merge round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: refuse a merge that puts inline content next to chunks A read inside the inline bytes returns them without looking at chunks, so one side's inline content merged with the other side's chunks hid data after a successful flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: refuse to merge a whole-file rewrite over a peer's chunk change maybeMergeChunks re-uploads the handle's whole view as unstamped chunks. Merged after a peer's change, they tie with a peer's own rewrite or bring back a range the peer truncated, so that flush fails as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: clip peer chunks at the lowest truncate, not the final size A handle that truncated and then wrote past the old end ends up larger than its base, so the merge did not see the truncate and kept the peer's chunks inside the range the truncate cleared. SetAttr now records the lowest size truncated to over the current base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a peer's truncate when the handle left the size alone The merge took the larger of the handle's and the stored size, so a peer truncate came back as a zero-filled tail when the handle only overwrote bytes inside it. The stored size now stands unless the handle changed its size or wrote past it, and rebasing keeps only the growth writes made after the flush was taken. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: retry a read view whose merged manifests did not resolve A failed SetChunks leaves the chunk group answering every read with the resolution error, and nothing re-ran it while the entry stayed the same. The handle now marks such a view (on rebase and on SetEntry) and the next read or lseek resolves the entry's chunks again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a dirty handle's writes when a peer change cannot merge The new merge refusals sent a peer event on the same file down the old path, which discarded the dirty pages and adopted the peer's entry, so close returned OK without the writes. The handle now keeps its writes and base, and the flush meets the conflict and fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep an unpublished truncate across a peer's base change The truncate mark was tied to the base pointer, so a peer event that moved a dirty handle's base (a merged content change or a metadata-only change) dropped it, and a later merge on a regrown file kept peer chunks inside the truncated range. Those base moves now carry the mark; a flush ack or a discarded handle state still clears it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: wait for the flush before merging a shrink over buffered writes A write inside the file leaves its bytes in dirty pages without moving the size. Merging a peer's truncate then adopted the smaller size, and the chunk uploaded later sat past it, unreadable through the handle. The event path and the periodic metadata flush now refuse a merge that shrinks the handle; the close flush uploads the pages first and merges then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: tell a shortened base chunk from a whole-file rewrite A truncate shortens a base chunk in place, so it gets a new reference key, and an old chunk can carry no timestamp. The rewrite refusal now applies only to unstamped chunks whose file id is new to the base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the creation time when the base has no inode An entry without an inode or hard link id matched any replacement at the same path, so a stale handle could merge into a different file. Without an inode the merge now needs the same creation time. The filer keeps it across updates; only a synthesized base has neither and still merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a dirty handle's writes when a peer replaces the file A peer entry with another inode at the handle's path went down the fallback that destroys the dirty pages and installs the replacement, so close returned OK without the writes. The handle now keeps its writes and base like it does for an unmergeable change, and its flush refuses to update the replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: build the content flush tests on freebsd The invalidation tests, which every platform compiles, now use the content flush helpers, so the freebsd cross-compile vet failed with undefined: flushConditionEntry. Nothing in the file is freebsd-specific. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a dirty handle's base when a same-content file replaces it A replacement with the base's chunks but another inode counted as a metadata-only change, so the dirty handle adopted it as its base and the next conditional flush overwrote the new file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: bring a peer's metadata change into a dirty handle A metadata-only event moved a dirty handle's base but left its live entry, so the next flush read the peer's chmod as a local edit and published the old mode back. Replay the handle's own changes onto the peer's entry instead, as for a content change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: wait for the flush before merging peer inline content over buffered writes A buffered write is not a chunk yet, so a merge could install a peer's inline content that later reads return ahead of the write's chunk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: track a whole-file chunk rewrite on the handle The merge told a rewrite by its unstamped chunks, which a manifest fold hides: manifests are unstamped either way. Record the rewrite against the base it was made from, carried across peer base changes like a pending truncate, and refuse the merge from that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: refuse a merge that shortens a manifest at the truncate Clipping a manifest reference leaves its members whole, so peer bytes past the handle's truncate could reappear once the file grew again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: flush a dirty handle after a peer replaces its file The replacement test checked only the retained handle state. It now flushes against a filer holding the replacement, including one with the base's content, and checks the flush fails and leaves it unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test an append fsync after a peer's attribute-only change A second mount's acknowledged chmod, chown, utimes or setxattr leaves the content alone, so the writer's conditional content flush must still publish its append and keep that change. On master the refused flush surfaces as EIO from Fsync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c2b2ed769b |
fix(mount): preserve appended chunks during stale attribute updates (#12593)
* fix(mount): preserve acknowledged chunks during attribute updates * mount: keep the WFS field block aligned Adding the publication-lock fields inside the aligned block made gofmt re-indent 25 unrelated lines. Put them in their own group after dirIdleEvict, matching the enterprise layout, so syncs do not conflict there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: finish attribute saves through a FUSE interrupt saveEntryAttributes derived its RPC context from the syscall's cancel channel, so an interrupt while the conditional UpdateEntry was at the owner returned EIO for an update the owner then applied, and skipped the local install and retention. Keep the metadata deadline but drop the cancellation, like the other metadata mutations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: map attribute CAS ids in place instead of cloning the entry saveEntryAttributes cloned the whole entry three times per attempt only to rewrite uid/gid for the filer, and each clone is as large as the chunk list. Switch the ids in place and restore the exact local values after the call, keeping one clone. The test now checks the reply carries the local uid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: compare attribute updates against the filer's own uid/gid With -map.uid/-map.gid, a filer id that is also a mapped local id reads back unchanged and maps forward to another filer id. The IF_ENTRY_EQUAL expected entry then never matched, so chmod, utimes and xattr changes on such files failed with EAGAIN after three attempts, and an update would have rewritten the owner. Keep the filer ids read with the entry for the ids the syscall leaves alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mount): align attribute retention with read lock order * mount: take the handle lock before entryLock in the attribute snapshot attributeEntrySnapshot held entryLock while it queued for the shared handle lock. A Read holding that lock waits on entryLock, the snapshot waits behind a queued exclusive (an invalidation or a Write), and the exclusive waits on the Read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mount): complete handle lock order and preserve explicit ID intent * mount: re-read a refused attribute update from the file's owner filer The filer checks the IF_ENTRY_EQUAL condition on the owner of the path's write route key. The retry read any filer, so behind one that had not applied the owner's latest update, every attempt sent the same stale base and the syscall failed with EAGAIN. Read from the owner once the lock ring is known, like lookupFilerEntry, but keep the filer's raw ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |