Commit Graph
9 Commits
Author SHA1 Message Date
Chris Lu f6a3286b32 fix(volume): derive needle body tail bound from the version layout (#11395)
* fix(volume): derive needle body tail bound from the version layout

The size guard in ReadNeedleBodyBytes computed the tail length as
checksum, plus timestamp only for Version3. Forks and future on-disk
formats whose tail carries more fields would silently under-check and
still panic in readNeedleTail on a truncated body. Derive the tail from
NeedleBodyLength minus data and padding so the bound stays exact for
every version.

Iterate IsSupportedVersion in the new tests instead of hardcoding
v1-v3 so downstream formats get covered automatically, and skip
versions the build cannot write rather than failing on them.

* test: skip needle write only on the unsupported-version error

A blanket skip would hide a real writer regression. Skip the version
subtest only when the writer reports the version is not supported in
this build (the error text differs between builds), and fail on any
other write error.
2026-09-19 12:20:01 -07:00
hsdfat 5769057af3 fix(volume): return an error instead of panicking on a corrupt needle size (#11393)
ReadNeedleBodyBytes sliced the needle body with the size from the needle
header without checking it. A corrupted .dat header carrying size -1 still
gets a positive body length (16 bytes on v3), so vacuum compaction read
that body and panicked with "slice bounds out of range [:-1]".

Writers never put a negative size in a .dat header: a delete appends a
size-0 record, and TombstoneFileSize only lives in the .idx. Reject a size
that is negative or leaves no room for the checksum/timestamp tail with an
error wrapping ErrorCorrupted. ScanVolumeFileFrom already logs body read
errors and moves on, so compaction now skips the record like any other
corrupt needle.

Fixes #6763
2026-09-18 23:00:06 -07:00
chrislu 2bfc8970d2 refactor: move ReadNeedleDataInto into volume_read.go 2022-09-08 18:54:02 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 11f1e7996b fix test 2022-06-05 18:50:41 -07:00
chrislu 056c480eb0 volume: checksum remove the hashing step in Value() 2022-06-05 15:24:02 -07:00
chrislu ecef844dfc stream read large files 2022-06-05 11:54:04 -07:00
chrislu 685643d506 needle read into a writer 2022-06-04 19:10:52 -07:00
chrislu 663bc5dc23 add needle reading in chunks 2022-06-04 18:15:39 -07:00