Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-11-11 22:38:53 +01:00
Code Issues Packages Projects Releases Wiki Activity

EC bitrot follow-ups: protect destination sidecar on optional copy; cap sidecar block_size (#9763)

Browse Source 
* fix(ec_bitrot): cap sidecar block_size in ValidateBitrotManifest

A sidecar loaded from disk (or supplied via a backfill/peer RPC) could carry a
huge power-of-two block_size that passed validation, then force a multi-GiB
scratch-buffer allocation in scrub/verify. Add a shared MaxBitrotBlockSize
(64 MiB) constant, enforce it as an upper bound in isPow2MultipleOf1MiB, and
derive the volume flag cap from the same constant so they cannot drift.

* fix(ec_bitrot): don't destroy a valid destination sidecar on an optional copy

writeToFile opened the destination with O_TRUNC before knowing whether the
source had the file, so an optional copy (ignoreSourceFileNotFound) from a source
that lacks the .ecsum truncated and then removed a valid pre-existing destination
sidecar. Stage the optional copy into a temp sibling and commit it with an atomic
rename only when the source actually delivered the file; a missing source is now
a no-op. Mandatory copies keep their in-place behavior.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-05-31 23:42:33 -07:00
1 parent 9658f309d2
commit 80dd3b2621
5 files changed
+113 -31
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

weed/storage/erasure_coding/ec_bitrot_test.go
+7 -6
View File
@@ -108,12 +108,13 @@ func TestValidateBitrotManifest(t *testing.T) {
t.Fatalf("valid manifest rejected: %v", err)
}
bad := map[string]func(*volume_server_pb.EcBitrotProtection){
"missing shard": func(p *volume_server_pb.EcBitrotProtection) { p.Shards = p.Shards[:2] },
"out of range id": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[2].ShardId = 9 },
"duplicate id": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[2].ShardId = 0 },
"zero covered": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[0].CoveredSize = 0 },
"bad crc count": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[0].BlockCrc32C = []byte{1, 2, 3} },
"bad block size": func(p *volume_server_pb.EcBitrotProtection) { p.BlockSize = 3 << 20 }, // not power of two
"missing shard": func(p *volume_server_pb.EcBitrotProtection) { p.Shards = p.Shards[:2] },
"out of range id": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[2].ShardId = 9 },
"duplicate id": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[2].ShardId = 0 },
"zero covered": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[0].CoveredSize = 0 },
"bad crc count": func(p *volume_server_pb.EcBitrotProtection) { p.Shards[0].BlockCrc32C = []byte{1, 2, 3} },
"bad block size": func(p *volume_server_pb.EcBitrotProtection) { p.BlockSize = 3 << 20 }, // not power of two
"oversized block size": func(p *volume_server_pb.EcBitrotProtection) { p.BlockSize = 128 << 20 }, // power of two but > MaxBitrotBlockSize
"bad algorithm": func(p *volume_server_pb.EcBitrotProtection) {
p.Algorithm = volume_server_pb.ChecksumAlgorithm_CHECKSUM_NONE
},