ec.decode: check the rebuilt .dat is complete before the shards can be deleted (#10768)

A decode ends by deleting the shards it read, and the only thing standing
between that and a bad reconstruction is verifyDecodedVolumeBeforeDelete,
which asks whether .dat and .idx are non-empty. A .dat truncated to a
single byte passes, and the shards -- the only other copy of everything
past the cut -- are deleted on the strength of it.

The server already knows the answer it never checks: FindDatFileSize
returns the extent the EC index references, and WriteDatFile rebuilds to
it. Compare the two once the file is written and fail the decode instead
of reporting a short volume as a good one.

Longer than the extent still verifies -- padding is not missing data --
so only a genuinely short rebuild is rejected.

Needle counts cannot answer this: .idx is written from .ecx, so the count
matches by construction and a truncated .dat still reports every needle.
This commit is contained in:
Chris Lu
2026-08-15 13:28:49 -07:00
committed by GitHub
parent 829064af71
commit fbd85d31b0
3 changed files with 115 additions and 0 deletions
+18
View File
@@ -126,6 +126,24 @@ func FindDatFileSize(shard0FileName, indexBaseFileName string) (datSize int64, e
return
}
// VerifyDecodedDatFile checks that a reconstructed .dat is long enough to hold
// every needle its index references. datFileSize is the extent the EC index
// describes (see FindDatFileSize), so a shorter file cannot serve the needles
// past the cut -- and the caller is about to delete the shards that are their
// only other copy, which turns a short write into data loss rather than a
// failed decode.
func VerifyDecodedDatFile(dataBaseFileName string, datFileSize int64) error {
datPath := dataBaseFileName + ".dat"
stat, err := os.Stat(datPath)
if err != nil {
return fmt.Errorf("stat decoded %s: %w", datPath, err)
}
if stat.Size() < datFileSize {
return fmt.Errorf("decoded %s is %d bytes, short of the %d its ec index references", datPath, stat.Size(), datFileSize)
}
return nil
}
func readEcVolumeVersion(shard0FileName string) (version needle.Version, err error) {
// find volume version