mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-16 03:20:50 +02:00
Chris Lu
5d8a463b3e
* test/ec: fix EC interruption matrix slot exhaustion The EC integration test cluster (test/erasure_coding/chaos_lifecycle_test.go) configured each disk with -max 4 and the seedAndSpread spread loop fired volume.grow -count 4 every 2 s with no per-server cap. Because the master topology lags the volume.grow writes, the loop re-fired before the prior grow was visible, over-filling disks to capacity. A full disk leaves zero free EC shard slots (failing the cluster-wide capacity check with "no free ec shard slots") and drops the source disk below the encode's FreeVolumeCount >= 2 health check (failing with "no healthy replicas"), which aborted ec.encode before any phase marker printed and made every encode scenario in TestECInterruptionMatrix fail. Three changes to the test cluster: 1. Raise -max from 4 to 8 per disk so the source disk always retains FreeVolumeCount >= 2 for ec.encode's 14-shard generation (2 volume-slot equivalents) even after the spread loop and multiple encodes. 2. Switch the spread loop from -count 4 to -count 1 so each grow lands exactly one volume on the volume server's least-loaded disk, giving deterministic cross-disk spreading instead of relying on a single multi-volume grow to fan out. 3. Cap grows per server at 4 so heartbeat lag cannot run away and over-fill disks before the master registers the prior grow. 4. Pass -minFreeSpace 0 so the test is not falsely gated by the physical disk's free-space percentage on the host running CI (the EC shard slot calculation separately enforces a 90 % disk-usage cap via balancer.DiskTooFullAfter, which already guards against an over-set maxVolumeCount on a physically full disk). Verified locally by running TestECInterruptionMatrix twice (all encode, decode, and balance scenarios pass, including the previously failing encode@Deletingoriginalvolumes). * test/ec: only count successful grows toward the spread cap A failed volume.grow (e.g. a transient collectTopologyInfo or VolumeGrow RPC error) would otherwise consume one of the four permitted attempts without creating any volume, exhausting the retry budget and leaving the loop to only poll until the Eventually timeout. Increment the per-server counter only when commandGrow.Do returns nil.