mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-30 17:00:48 +01:00
Chris LuandDevin
1a62b1d09f
* s3sink: upload through a concurrent-safe ReaderAt so parts do not thrash
s3manager splits a seekable body into parallel section readers, and every
part's ReadAt landed on the single chunk buffer in ChunkStreamReader: parts
at different offsets kept evicting each other's chunk and refetching it
whole, which produced massive read amplification against the volume
servers and collapsed throughput.
Build the upload body from filer.ChunkReadAt instead, the same
concurrent-safe reader the mount and query paths use: each part's range is
served from a shared reader cache with parallel chunk fetches and
range-reads for clipped views, so every byte is fetched once per pass.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* filer: buffer recently fetched chunks for concurrent readers
* Revert "s3sink: upload through a concurrent-safe ReaderAt so parts do not thrash"
This reverts commit be1ec2d7c7.
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>