mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-19 02:15:02 +01:00
c7d8b106d50f72da1ab10df8aee3cd2fa080caa4
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4f70ad015e |
mount: fail the write when the swap file chunk cannot be created (#12655)
* mount: fail the write when the swap file chunk cannot be created #7957 added a nil check after NewSwapFileChunk so a failed swap file creation returns an error instead of panicking. The result was assigned to pageChunk, a PageChunk interface, before the check, and an interface holding a nil *SwapFileChunk is not nil, so the check never fired: SaveDataAt stored the nil chunk and panicked in WriteDataAt. Check the *SwapFileChunk before converting it. The new test points the swap directory below a regular file; on master it panics in SwapFileChunk.WriteDataAt. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * mount: grow the file only after its write is buffered Write set the new size, times, pending append offset and uncommitted bytes before AddPage. When AddPage failed (now that a swap file chunk that cannot be created returns an error), the write was reported as failed but the larger size stayed on the handle's entry, so the next flush published it with a hole where the data should be. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report a partly buffered write as a short write PageWriter.AddPage fills one chunk at a time, so when a later chunk cannot be created the earlier chunks already hold their bytes. Return how much was buffered, and let Write grow the size and times over that prefix and report it, instead of failing the whole write while the prefix still reaches the next flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report the bytes a failed copy_file_range already buffered A streaming copy whose page fails partway returned an error even though earlier pages, or the start of the failing one, were already buffered and would reach the next flush with the destination size left short. Count the buffered bytes, grow the destination over them, and return them as a short copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: free the copy test's destination pages The buffered memory chunk counts toward the process-wide memory chunk ceiling, so repeated runs of the test pushed later ones onto the unusable swap directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: count only the bytes a swap chunk stored A swap chunk whose file write fails stores a prefix or nothing, but SaveDataAt returned no error and AddPage counted the whole span as buffered. SaveDataAt now reports the short write, and AddPage passes the stored count up so a write is never acknowledged past it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail a partly buffered writeback write Kernel writeback ignores the count in a WRITE reply, so a short write there drops the unbuffered rest without an error. A write flagged as page-cache writeback now keeps its buffered prefix but returns the error, which the kernel reports on fsync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b50d4a9cab |
mount: don't spend the close()/fsync commit deadline on uploads (#12641)
* mount: start the close()/fsync commit deadline after the data upload Flush, Fsync and the truncate path of SetAttr create the 30 s metadataFlushTimeout context on entry and pass it to doFlush, so the clock also runs through FlushData, the upload of the dirty pages. That upload has its own timeouts and retries. When an assign through a stalled filer or a slow volume server takes most of the 30 s, CreateEntry then fails at once with "context deadline exceeded" and close() returns EIO for data that was already stored. The deadline is meant for the commit alone (#10102: "a healthy CreateEntry completes in well under a second"). doFlush now takes the commit timeout and starts the deadline once the data is uploaded. close() still cannot hang on a stuck filer: the commit keeps its 30 s bound. Release and the flush before a rename pass 0, as before (no overall bound, only the per-call filer timeouts). Seen in production on a training cluster: a 30 s stall of one replica's disk, and a filer that stalled for about 30 s, each failed close() with EIO although the upload had succeeded. Tests: close(), fsync() and a truncate below dirty data, each after an assign slower than the deadline (all three fail before this change), and a filer that never answers CreateEntry still fails close() at the deadline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: charge the flush commit deadline only for its filer calls Starting the 30 s deadline after the data upload is not enough. flushMetadataToFiler does more than call the filer before it commits: it waits for the handle's locks, reads the file back and uploads it again when its chunks are bloated (maybeMergeChunks), and uploads a manifest once enough chunks have built up (MaybeManifestize). None of that takes the context. When it outlasts the deadline, CreateEntry again fails at once with "context deadline exceeded", and close() or fsync() returns an error for data that was stored. The deadline becomes a budget, commitBudget, charged only while a filer call of the commit runs: CreateEntry, and the read of a changed hard link before it. The attempts of one flush share it, and a flush is not retried once it is spent, so a stuck filer still fails close() after 30 s. Release, the flush before a rename and the async flush pass no budget, as before. Seen in production: the merge of a 1.8 MiB sqlite file with 183 chunks took 32 s because one chunk's read first tried a volume server that was down, and the commit failed 32 microseconds after the merge finished. Tests: a flush whose manifest upload is slower than the deadline commits (it fails without this change), and the filer calls of one flush share the budget. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: give the stalled-lookup refusal test the new doFlush argument TestContentFlushRefusalWithStalledLookup came in from master after this branch changed doFlush's signature, so the merged package did not build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |