Commit Graph
10 Commits
Author SHA1 Message Date
Chris Lu d4548376a1 fix(ec): off-by-one in nLargeBlockRows causes EC read corruption (#8957)
* fix(ec): off-by-one in nLargeBlockRows causes EC read corruption (#8947)

The nLargeBlockRows formula in locateOffset used (shardDatSize-1)/largeBlockLength,
which produces an off-by-one error when shardDatSize is an exact multiple of
largeBlockLength (e.g. a 30GB volume with 10 data shards = 3GB per shard).
This causes needles in the last large block row to be mislocated as small blocks,
reading from completely wrong shard positions and returning garbage data.

Fix: remove the -1 from locateOffset and only apply it in the ecdFileSize fallback
path (old volumes without datFileSize in .vif), where it's needed to handle the
ambiguous case conservatively.

Also fix ReadEcShardNeedle to pass offset=0 to ReadBytes, consistent with the
scrub path, since the bytes buffer already starts at position 0.

* fix: add volume context to EC read errors, remove contextless glog

The glog.Errorf in ReadBytes logged "entry not found" without any volume
ID, making it impossible to identify which volume was affected. Remove
this contextless log and instead add volume ID, needle ID, offset, and
size to the error returned from the EC read path.

The EC scrub callers already wrap errors with volume context.
2026-04-07 12:02:51 -07:00
Chris Lu f23e09f58b fix: skip exhausted blocks before creating an interval (#8180)
* fix: skip exhausted blocks before creating an interval

* refactor: optimize interval creation and fix logic duplication

* docs: add docstring for LocateData

* refactor: extract moveToNextBlock helper to deduplicate logic

* fix: use int64 for block index comparison to prevent overflow

* test: add unit test for LocateData boundary crossing (issue #8179)

* fix: skip exhausted blocks to prevent negative interval size and panics (issue #8179)

* refactor: apply review suggestions for test maintainability and code style
2026-02-02 11:12:31 -08:00
chrislu ec155022e7 "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
chrislu 07f4998188 add dat file size into vif for EC 2024-08-13 13:56:00 -07:00
chrislu 3a2e21fee7 fix EC reading on nLargeBlockRows
fix https://github.com/seaweedfs/seaweedfs/issues/5465
2024-08-13 12:01:46 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Chris Lu 6a92f0bc7a refactoring to typed Size
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu b4b407e403 add grpc ec shard read 2019-05-27 11:59:03 -07:00
Chris Lu a4f3d82c57 convert needle id to ec intervals to read from 2019-05-27 01:29:46 -07:00
Chris Lu 7c2c60c376 add locating data inside the ec files 2019-05-19 03:01:58 -07:00