mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-02 10:17:10 +01:00
filer: resume a rejoining peer's meta subscription at its last position (#12279)
* filer: resume a rejoining peer's meta subscription at its last position OnPeerUpdate starts every peer add at the update's arrival time. For filers sharing one store there is no persisted offset to correct it, so a peer that drops out and rejoins (a graceful restart) resubscribes from "now" and the events it logged while its stream was down never reach the aggregated buffer - filer.backup and other meta subscribers silently skip them. Track each peer's delivered-through position (the same timestamp domain as the stream's events and idle-heartbeat claims) and rewind the start of a rejoining subscription to it. First-seen peers keep the given start, and non-shared peers still resume from their persisted offset. * filer: keep the re-add resume point behind a replay-floored event The session's replay floor already pins the persisted offset and the reconnect resume point so an event that outlived its retry budget is redelivered on resubscribe; cap peerResumeTsNs to the same position on session exit so the new re-add resume cannot skip past it either. * filer: stop test aggregators' log-buffer workers * filer: cover peer rejoin resubscribing at the resume position * filer: drop the unreachable replay-floor cap on the resume point Only a peer with its own store can floor an event, and that peer's subscribe start always comes from its persisted offset, never from the re-add resume point. The cap also ran in the old session's defer, after the re-add had already read the value. * filer: say why a re-add may resume from an idle heartbeat The heartbeat comments promised a reconnect never resumes from one, while the re-add resume point does. Spell out the difference and the fence that makes the local-stream claim safe to resume from. * filer: forget a removed peer's resume point after an hour The map held an entry for every address ever seen; with pod IPs that is one per restart. The removal mark now outlives the watermark grace and takes the resume point with it once the retention passes. * filer: advance the resume point only for tracked peers Same rule as the watermarks: a straggler from a dropped session must not recreate the entry. * filer: set the test aggregator's subscribe loop instead of a nil fallback
This commit is contained in:
1 parent
f5989a21fb
commit
8d3cb2ed2d
4 files changed
+157
-14
No files matched your search
@@ -1385,7 +1385,9 @@ func (fs *FilerServer) maybeSendFlushReport(req *filer_pb.SubscribeMetadataReque
|
||||
// maybeSendIdleHeartbeat emits an empty response carrying the current time when
|
||||
// the subscriber has consumed everything up to the buffer head. The client uses
|
||||
// it to advance freshness signals (e.g. filer.sync's sync_offset) without moving
|
||||
// its resume checkpoint, so a restart still re-reads from the last real event.
|
||||
// its persisted resume checkpoint, so a restart still re-reads from the last
|
||||
// real event. The fenced local-stream claim is the one exception: a peer
|
||||
// aggregator resumes a re-added peer from it (MetaAggregator.peerResumeStartFrom).
|
||||
//
|
||||
// The catch-up floor is the max of two read-progress markers:
|
||||
// - readPositionTsNs: how far the read cursor has advanced. It starts at
|
||||
|
||||
Reference in new issue
Block a user