mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-11 22:38:53 +01:00
fix: three hardware blockers — WAL retention + registry race + shutdown beat
All 43 actions pass on m01/m02 hardware. Auto-failover PASS. dd_write: 30s → 123ms. Post-failover write: 33,621 IOPS. 1. WAL retention: remove keepup retention floor (MinShippedLSN). WAL cannot be pinned during sustained async writes — any pin strategy either fills WAL (blocking writes) or over-recycles (breaking catch-up). Flusher recycles freely. Future LBA map will provide catch-up without WAL retention. MinShippedLSN on ShipperGroup retained as diagnostic surface. 2. Registry stale-cleanup race: add RegisteredAt grace period. Race: master registers volume → next VS heartbeat arrives before VS discovers the volume → stale cleanup deletes the entry → failover finds 0 entries. Fix: skip stale cleanup for entries registered within 30s (> 2 heartbeat intervals). 2 new tests: grace protects new entry, old entry still cleaned. 3. Shutdown heartbeat: VS disconnect heartbeat no longer claims block inventory authority. Previously, the shutdown beat's empty inventory triggered stale cleanup, deleting the entry before failover could use it. Scenario fix: recovery-baseline-failover.yaml now kills the correct node (discovered primary, not hardcoded), connects to the correct new primary for post-failover verification. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
39f1232fe2
commit
e0116fc631
28 files changed
+201
-2971
No files matched your search
@@ -196,6 +196,11 @@ func (ms *MasterServer) failoverBlockVolumes(deadServer string) {
|
||||
}
|
||||
ms.blockRegistry.FailoversTotal.Add(1)
|
||||
entries := ms.blockRegistry.ListByServer(deadServer)
|
||||
glog.V(0).Infof("failover: deadServer=%s entries=%d", deadServer, len(entries))
|
||||
for i, e := range entries {
|
||||
glog.V(0).Infof("failover: entry[%d] name=%q vs=%s role=%d hasReplica=%v epoch=%d",
|
||||
i, e.Name, e.VolumeServer, e.Role, e.HasReplica(), e.Epoch)
|
||||
}
|
||||
now := time.Now()
|
||||
for _, entry := range entries {
|
||||
// Case 1: Dead server is the primary.
|
||||
|
||||
Reference in new issue
Block a user