mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-05 03:27:42 +01:00
Chris LuandClaude Opus 5.5
51ee165279
* storage: search corrupt shard sets when an EC needle fails to rebuild A needle whose reassembled bytes fail to parse is rebuilt with its own shards excluded. When other shards are corrupt over the same range, that decode reads them as inputs, and the retry dropped only one input at a time, so two or more extra corrupt shards left the needle unreadable even within parity. After the single pass fails, try every set of up to ParityShards shards that holds a suspect as the corrupt set, smallest first. Suspect ranges inside the set are rebuilt from the shards outside it and the rest keep their bytes, so a clean suspect can still feed the decode. Sets whose decodes read the same inputs are tried once, and the search is bounded by set count and by rebuilt bytes past the single-shard sets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * seaweed-volume: search corrupt shard sets when an EC needle fails to rebuild Mirror the Go read path: after the single pass over the suspects fails, gather each interval from every other shard once and try each set of up to parity_shards shards holding a suspect as the corrupt set, smallest first, with the same dedup and bounds. The search runs on the blocking pool, holding the recovery permit, since it can take many decodes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: charge only multi-shard sets against the rebuild byte budget The budget check already let single-shard sets through, but each one was still charged, so a needle spanning many shards could spend the budget before the first multi-shard set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: bound the corrupt-set gather to the recovery budget The search gathered every other shard per interval, so a large needle spanning several shards held up to Total-1 copies of itself under a permit capped at the budget. Gather only as many as the budget covers, but never fewer than the suspects-excluded gather the fallback always held, so single-suspect recovery keeps every other shard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: gather at least one decode's worth for the corrupt-set search A needle over more shards than parity skips the single pass, and its suspects-excluded width can fall below DataShards, so the bounded gather left even one corrupt shard unrecoverable. Floor the width at DataShards, which the per-suspect rebuild this search replaced read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: rebuild one suspect at a time when the search gather is narrow Flooring the search gather at DataShards made a large needle over more shards than parity hold ten copies of itself, where the per-suspect rebuild it replaced held one interval's fan-out. Keep that rebuild after the search, and skip the search when the budget leaves fewer inputs than one decode. Parity slack still widens the gather to every shard outside the suspects, as the dropped-input retry did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: keep the dropped-input retry and run the set search within budget A narrowed search gather skipped only each interval's own shard, so it could read corrupt suspects in place of healthy shards the old retry kept, and its byte bound could stop before that retry's hypotheses. Restore the dropped-input retry as it was, move the search into its own function after it, and run the search only when every interval's full gather fits the recovery budget. Past the budget a read takes the same paths, with the same footprint, as before the search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>