* vacuum: size the compaction space check by live bytes, not volume size
ensureCompactVolumeSpace required the volume's current .dat and .idx size as
free space before compacting. That is the size of the garbage, not of what
compaction writes, so on a disk that filled up until its volumes went
read-only every compaction was refused, including all-garbage volumes that
would compact to a superblock and an empty index. The sweep then retried
every volume each cycle and reclaimed nothing (issue #11516).
Estimate the output from what the needle map already tracks: live content
bytes plus a per-needle framing upper bound behind a superblock, and one
index entry per live needle. The estimate never exceeds the current volume
size and preallocate still wins when larger. Volumes whose deleted sizes are
unknown (.sdx converted back to .idx) keep the whole volume as the estimate.
The disk probe moves behind a package variable so the tests can stand in
for a full disk; the tests build real volumes instead of re-implementing
the formula.
* vacuum: space check reserves the index on top of preallocate, checks a separate index disk
Review follow-ups: preallocate only stands in for the new .dat, so the
rebuilt index is added on top of it; with separate index directories the
data disk is checked for the .cpd and the index disk for the .cpx; and the
estimates carry 1/16 headroom because counters rebuilt from an index file
pass through a Bloom filter with a 0.1% false positive rate. Neither
estimate exceeds the current file.
* vacuum: split the space check by filesystem, not by directory name
Two directories can sit on one filesystem and share its free space, so
the data and index estimates are checked separately only when the index
directory is on another device; otherwise the sum must fit. Unknown is
treated as shared.
* vacuum: ask the index directory for its share even when it looks like the same filesystem
A volume mounted under the data directory's drive letter on Windows has
the same volume name, so the identity check calls it shared. Checking the
index directory for the index estimate as well costs one statfs and
catches a full index mount either way.
* vacuum: identify a Windows volume by its GUID, not its path prefix
A volume can be reached through a drive letter and through a folder it is
mounted on, so filepath.VolumeName says nothing about the free-space pool.
Resolve each directory to its mount point and compare the volume GUIDs;
when that fails the two are treated as shared.
* vacuum: keep the framing and disk_space_low coverage the rebase displaced
* rust volume: split the compaction space check across data and index disks
Mirror the Go check: estimate the new .dat and rebuilt .idx separately —
live content plus per-needle framing capped at the current file, with
preallocate standing in for the data file when larger — and check each
directory against its own filesystem's free space. Two directories on one
filesystem are asked for the sum.
* vacuum: tighten comments on the compaction space check
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* storage/topology: keep disk-full read-only volumes vacuumable
The vacuum sweep skipped every read-only replica, so a volume that went
read-only because its disk filled could never reclaim its garbage — the
exact situation compaction exists for. The volume server now reports
disk_space_low in VacuumVolumeCheckResponse, and the sweep skips a
read-only replica only when the flag is clear. An explicit volumeId
vacuum is unaffected: it already bypassed the read-only rule.
The field takes number 4: 2 and 3 are downstream-allocated for tombstone
retention, keeping the wire merge clean.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* storage: measure vacuum free space against live bytes
The pre-compaction space check required the current .dat + .idx size
free, which includes the garbage being reclaimed — on a nearly full disk
that estimate can never fit, so the volume stayed garbage-bound forever.
Measure against the estimated compacted output instead: superblock plus
live index entries plus live content bytes, with the existing ten
percent buffer unchanged. Mirrors the same check in the Rust volume
server.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* vacuum: count per-needle framing in the compacted-size estimate
The live-bytes estimate covered each live needle's content and index
entry but not its .dat framing (header, checksum, timestamp, padding —
~32 bytes on version 3). For small-needle volumes that is more than the
10% headroom, so a disk with space between the estimate and the real
output still ran out mid-compaction. Rust side mirrors the same formula.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* storage: report disk_space_low only when it is the sole read-only cause
Review feedback (ihnokim, greptile, devin): a volume read-only for low
disk space AND an operator mark or I/O quarantine was still eligible for
the automatic sweep, rewriting a copy meant to stay protected. The flag
now reports only the benign sole-cause case in both servers.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* topology: fail closed when the read-only lookup misses in the sweep
A heartbeat can drop the volume from the DataNode cache between the
location-list copy and VacuumVolumeCheck; a lookup error previously
skipped the read-only check entirely. Review feedback (coderabbit).
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
