mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-06 20:17:38 +01:00
mini: lift the post-mint allocation hold on first inventory report (#12061)
* mini: lift the post-mint allocation hold on first inventory report DeferAllocation armed a fixed warmup window whenever the leader minted a fresh topology id, so the first write on a brand-new weed mini blocked for the whole window even though no prior cluster state could exist. The window still has to cover a rebuilt master's veteran servers, which report on their first post-reconnect heartbeat. Mini has exactly one possible reporter — the embedded volume server, whose first heartbeat carries the full inventory — so hold allocation only until that report is applied. The bound raise and topology-id freeze checks run while the heartbeat is processed, so a wiped master state still raises the bound or trips the freeze before the hold lifts. Standalone weed master and weed server keep the fixed window: volume servers outside the process can still be on their way. * master: apply queued topology-id reports before opening the allocation gate SetTopologyInitPending(false) cleared the pending flag before replaying topology ids stashed while the local id was still being minted. A NextVolumeId landing in the drain window saw no gate and no freeze yet. The post-mint allocation hold used to cover the gap by delaying the winner past the freeze; once the hold can release on first inventory, the gate must stay closed until the queued checks have run. * mini: gate the first-report release on single-master mode releaseAllocOnFirstReport assumes the embedded volume server is the only possible reporter of prior cluster state. A mini started with -master.peers joins other masters, whose own volume servers can still be on their way when the first report lands, so keep the regular quiet-pulse hold there. * master: check reported topology id on empty full reports A HasNoVolumes heartbeat carries the same LastSeenTopologyId a populated list does, but the mismatch freeze only ran for non-empty reports. An EC-only veteran could therefore release the post-mint allocation hold through NoteInventoryApplied while its stale topology id never reached CheckReportedTopologyId. * master: drain the topology-id stash inside the init-gate lock CheckReportedTopologyId only held reportedTopologyIdsLock while stashing, so a report in flight during SetTopologyInitPending(false) could land its freeze after topologyInitPending cleared, letting a NextVolumeId slip through the gap. The whole check now runs under the lock, and the gate is released inside the same critical section as the drain. * mini, master: pin and enforce the single volume-server reporter isSingleMasterMode only checks master peers; SendHeartbeat still accepted any volume server, so a late foreign report could carry prior cluster state after the first-report release dropped the hold. Mini now pins the embedded volume server's address and the master refuses registration from any other reporter while ReleaseAllocOnFirstReport is set. * master: admit the volume-server reporter before applying heartbeat state * test: cover SendHeartbeat admission ordering A refused reporter must not freeze growth or raise the id bound, and the pinned reporter's stale topology id still must. * mini: re-derive the single-reporter release on every run The package-level master options survive in-process reruns, so a multi-master mini following a single-master one must not inherit the first-report release or a stale reporter pin. * master: admit addressless first heartbeats under the reporter pin An unregistered heartbeat with no Ip skipped the single-reporter admission check but still reached the seen-bound raise and the topology-id freeze before the delta-heartbeat skip. Admission now runs for every unregistered heartbeat; an empty address can never equal the pin.
This commit is contained in:
1 parent
88b91cccba
commit
53834117a6
7 files changed
+316
-16
No files matched your search
+20
-9
@@ -82,6 +82,15 @@ type MasterOptions struct {
|
||||
// is cancelled. Used by integration tests and by weed mini; nil for
|
||||
// standalone weed master.
|
||||
shutdownCtx context.Context
|
||||
// releaseAllocOnFirstReport lifts the post-mint allocation hold as soon as
|
||||
// one full volume-server inventory has been applied. weed mini sets it in
|
||||
// single-master mode, with singleReporterAddr pinning the embedded volume
|
||||
// server as the only admitted reporter: any other volume server is refused,
|
||||
// so the warmup window after a fresh topology id is dead time.
|
||||
// Standalone weed master / weed server can face late-arriving veterans and
|
||||
// must keep the window.
|
||||
releaseAllocOnFirstReport bool
|
||||
singleReporterAddr string
|
||||
}
|
||||
|
||||
func init() {
|
||||
@@ -488,14 +497,16 @@ func (m *MasterOptions) toMasterOption(whiteList []string) *weed_server.MasterOp
|
||||
VacuumIntervalSeconds: *m.vacuumIntervalSeconds,
|
||||
VacuumDeleteEmptyAfterSeconds: *m.vacuumDeleteEmptyAfterSeconds,
|
||||
// PulseSeconds: *m.pulseSeconds,
|
||||
DefaultReplicaPlacement: *m.defaultReplication,
|
||||
GarbageThreshold: *m.garbageThreshold,
|
||||
WhiteList: whiteList,
|
||||
DisableHttp: *m.disableHttp,
|
||||
MetricsAddress: *m.metricsAddress,
|
||||
MetricsIntervalSec: *m.metricsIntervalSec,
|
||||
TelemetryUrl: *m.telemetryUrl,
|
||||
TelemetryEnabled: *m.telemetryEnabled,
|
||||
SingleMaster: isSingleMasterMode(*m.peers),
|
||||
DefaultReplicaPlacement: *m.defaultReplication,
|
||||
GarbageThreshold: *m.garbageThreshold,
|
||||
WhiteList: whiteList,
|
||||
DisableHttp: *m.disableHttp,
|
||||
MetricsAddress: *m.metricsAddress,
|
||||
MetricsIntervalSec: *m.metricsIntervalSec,
|
||||
TelemetryUrl: *m.telemetryUrl,
|
||||
TelemetryEnabled: *m.telemetryEnabled,
|
||||
SingleMaster: isSingleMasterMode(*m.peers),
|
||||
ReleaseAllocOnFirstReport: m.releaseAllocOnFirstReport,
|
||||
SingleVolumeReporter: m.singleReporterAddr,
|
||||
}
|
||||
}
|
||||
@@ -1564,6 +1564,22 @@ func runMini(cmd *Command, args []string) bool {
|
||||
// tear those services down too. On Ctrl+C they rely on their own
|
||||
// OnInterrupt hooks (see grace's LIFO ordering).
|
||||
miniMasterOptions.shutdownCtx = MiniClusterCtx
|
||||
// In single-master mode mini's embedded volume server is the only reporter
|
||||
// of prior cluster state; its first heartbeat already carries the full
|
||||
// inventory, so the post-mint allocation hold ends there instead of
|
||||
// running the whole warmup window on every fresh start. The master pins
|
||||
// the embedded address and refuses any other volume server, keeping the
|
||||
// single-reporter assumption true. Multi-master minis can field
|
||||
// late-reporting volume servers and keep the regular hold.
|
||||
// Both fields are re-derived on every run: the package-level options
|
||||
// survive in-process reruns, so a multi-master mini after a
|
||||
// single-master one must not inherit the release or the stale pin.
|
||||
singleMaster := isSingleMasterMode(*miniMasterOptions.peers)
|
||||
miniMasterOptions.releaseAllocOnFirstReport = singleMaster
|
||||
miniMasterOptions.singleReporterAddr = ""
|
||||
if singleMaster {
|
||||
miniMasterOptions.singleReporterAddr = util.JoinHostPort(*miniIp, *miniOptions.v.port)
|
||||
}
|
||||
miniOptions.v.shutdownCtx = MiniClusterCtx
|
||||
miniFilerOptions.shutdownCtx = MiniClusterCtx
|
||||
// Mini is a small/dev setup with short-lived RPCs; cap the filer's
|
||||
|
||||
Reference in new issue
Block a user