mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-07 14:57:48 +02:00
675020b342
* fix(filer.sync): validate chunk size in FilerSink to prevent 0-byte propagation
FilerSink.fetchAndWrite previously trusted the source response and the
upload result blindly: a 200 OK / Content-Length: 0 reply from a broken
source volume was happily uploaded as a 0-byte needle to the destination,
and the destination filer metadata was then written with the source
chunk size. The result was permanent silent corruption -- ls shows the
file at its original size but reads fail with EIO.
Add two cheap defenses inside fetchAndWrite:
1. After assembling fullData, compare its length against sourceChunk.Size.
2. After a successful upload, compare uploadResult.Size against
sourceChunk.Size.
Both checks wrap a new sentinel errChunkSizeMismatch that the retry
callback recognizes and refuses to retry -- needle.size=0 on disk is a
persistent state, not a transient network error, so the sync should stop
loudly on the affected entry instead of looping or, worse, silently
propagating it.
Tests:
* TestValidateReplicatedChunkSize -- table-driven coverage of healthy,
legitimately empty, zero-byte read, short read, and truncated upload
cases.
* TestFetchAndWriteRejectsZeroByteSource -- end-to-end: an httptest
source that returns 200 OK with an empty body must cause fetchAndWrite
to return errChunkSizeMismatch after exactly one source hit (fail
fast, no retry storm).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* filer.sync: bubble size-mismatch past CreateEntry/UpdateEntry
Three follow-ups on the chunk-size validation:
- Use %w in replicateOneChunk so the errChunkSizeMismatch sentinel
survives the wrap and reaches errors.Is callers up the stack.
- In FilerSink.CreateEntry/UpdateEntry, surface errChunkSizeMismatch
instead of warning-and-nil. Other errors (deleted source chunk,
transient network) keep the existing swallow so a hiccup doesn't
stall the stream.
- Drop validateReplicatedUploadSize: uploadResult.Size is set
client-side from the same len(fullData) we already validated
pre-upload, so the second check can't fail.
Test: scope the RetryWaitTime override to the one test that needs it,
add a regression that locks in the errors.Is chain through
replicateChunks.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>