topology: wake the vacuum dispatcher when a worker frees quota (#11436)

* topology: wake the vacuum dispatcher when a worker frees quota

The dispatch loop slept a fixed 10s whenever every pending volume was
waiting for a per-server quota slot, so a sweep took volumes x 10s
regardless of how fast the compactions were. Workers now signal on a
buffered channel after crediting quota; the dispatcher waits on it with
the 10s sleep kept only as a timeout.

* master: add -vacuumIntervalSeconds to tune the automatic sweep interval

The 14-minute base interval was a literal inside the refresh loop while
every neighbouring vacuum knob was already a flag. Defaults to 840s,
unchanged.

* topology: keep the 14 minute floor on the vacuum interval

A zero-valued MasterOption or a negative -vacuumIntervalSeconds left the
sweep sleeping only its jitter, so treat non-positive intervals as the
previous default.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-09-25 02:14:59 +08:00
1 parent 8c1be63c92
commit 56d2f05ccd
7 files changed
+22 -3

No files matched your search

+3
View File
@@ -57,6 +57,7 @@ type MasterOptions struct {
fileSizeLimitMB *int
volumePreallocate *bool
maxParallelVacuumPerServer *int
vacuumIntervalSeconds *int
// pulseSeconds *int
defaultReplication *string
garbageThreshold *float64
@@ -93,6 +94,7 @@ func init() {
m.fileSizeLimitMB = cmdMaster.Flag.Int("fileSizeLimitMB", 256, "limit the file size accepted by /submit, should match the volume servers' -fileSizeLimitMB (-volume.fileSizeLimitMB under weed server or weed mini, which set this for you)")
m.volumePreallocate = cmdMaster.Flag.Bool("volumePreallocate", false, "Preallocate disk space for volumes.")
m.maxParallelVacuumPerServer = cmdMaster.Flag.Int("maxParallelVacuumPerServer", 1, "maximum number of volumes to vacuum in parallel per volume server")
m.vacuumIntervalSeconds = cmdMaster.Flag.Int("vacuumIntervalSeconds", 840, "seconds between automatic vacuum sweeps")
// m.pulseSeconds = cmdMaster.Flag.Int("pulseSeconds", 5, "number of seconds between heartbeats")
m.defaultReplication = cmdMaster.Flag.String("defaultReplication", "", "Default replication type if not specified.")
m.garbageThreshold = cmdMaster.Flag.Float64("garbageThreshold", 0.3, "threshold to vacuum and reclaim spaces")
@@ -473,6 +475,7 @@ func (m *MasterOptions) toMasterOption(whiteList []string) *weed_server.MasterOp
FileSizeLimitMB: *m.fileSizeLimitMB,
VolumePreallocate: *m.volumePreallocate,
MaxParallelVacuumPerServer: *m.maxParallelVacuumPerServer,
VacuumIntervalSeconds: *m.vacuumIntervalSeconds,
// PulseSeconds: *m.pulseSeconds,
DefaultReplicaPlacement: *m.defaultReplication,
GarbageThreshold: *m.garbageThreshold,