vacuum: report a request dropped because another vacuum is running (#12648)

* vacuum: report a request dropped because another vacuum is running

The master runs one vacuum at a time. A request that arrives while
another one is running, the automatic vacuum or an earlier request, is
dropped: Topology.Vacuum logs "Vacuum is already running" and returns.
The VacuumVolume RPC and /vol/vacuum then answered as if the vacuum had
run, so volume.vacuum printed nothing and exited 0 with no volume
compacted. Where the automatic vacuum is busy for hours, an explicit
volume.vacuum -volumeId could do nothing every time, and nothing said so.

Topology.Vacuum now returns ErrVacuumRunning for a dropped request. The
RPC returns it and /vol/vacuum answers 409 with it. volume.vacuum names
the request that failed, because a list of volumes or collections is
sent as several requests and a vacuum that starts in between fails only
the later ones. Requests are still dropped, not queued, and the
automatic vacuum ignores the error as before.

The integration tests that need their /vol/vacuum request to run ask
again while the master answers 409.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* docs: note the 409 answer from /vol/vacuum while another vacuum runs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-11-14 11:07:44 -08:00
1 parent 512f898600
commit 1167ca8cf5
8 files changed
+153 -16

No files matched your search

+19 -1
View File
@@ -38,6 +38,10 @@ func (c *commandVacuum) Help() string {
Without -volumeId this runs the same sweep as the automatic vacuum, which skips
read-only volumes. Name a read-only volume with -volumeId to vacuum it anyway.
The master runs one vacuum at a time. While the automatic vacuum or another
request is running, this command fails instead of waiting for its turn. Run it
again later, or pause the automatic vacuum first with volume.vacuum.disable.
`
}
@@ -157,13 +161,27 @@ func (c *commandVacuum) Do(args []string, commandEnv *CommandEnv, writer io.Writ
return err
})
if err != nil {
return err
// A list of volumes or collections is several requests, and a vacuum
// that starts in between only fails the ones after it.
return fmt.Errorf("vacuum %s: %w", describeVacuumRequest(req), err)
}
}
return nil
}
func describeVacuumRequest(req *master_pb.VacuumVolumeRequest) string {
switch {
case req.VolumeId != 0 && req.Collection != "":
return fmt.Sprintf("volume %d in collection %s", req.VolumeId, req.Collection)
case req.VolumeId != 0:
return fmt.Sprintf("volume %d", req.VolumeId)
case req.Collection != "":
return "collection " + req.Collection
}
return "all volumes"
}
// vidReplica pairs a volume replica's read-only state with the server that
// reports it, so the unnamed-collection sweep can reproduce the master's
// low-disk-space exception client-side.