* volume.fsck: queue per-volume orphan deletes through the shared purge pool
The worker pool added for -reallyDeleteFilerEntries only covered entries
whose volume was missing entirely; the per-volume check that finds filer
entries pointing at absent needles still issued one synchronous HTTP
DELETE per orphan, which is the bulk of the work the report measured.
One pool now serves both purge phases and is drained before the empty-
directory sweep.
* volume.fsck: keep hard link ids in the filer fid file
Orphans queued by the per-volume check must hash into the pool by their
hard link id just like the missing-volume path, so two paths sharing an
id still serialize on one worker. The fid file now stores the id beside
each path and readFilerFileIdFile hands it to the caller.
* volume.fsck: cover orphan purge queuing and hard link id round-trip
* volume.fsck: fail on a truncated hard link id in the fid file
A .fid record cut short inside the hard link id length was emitted with a
nil id, so its purge could hash to a different worker than the entries it
shares the link with. Read the length strictly like the header and path.
* volume.fsck: drain the save consumer before returning a root genFn error
The early return left the save goroutine blocked on outputChan while the
caller ran finishPurge, so a late queuePurge could send on a closed
channel and panic.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* storage: skip EC shard-size encoding tests when disk space is low
TestCalculateExpectedShardSizeWithRealEncoding writes a .dat file plus
14 EC shards per subtest (~1.3GB peak for the 512MB case) into the temp
dir. Shared CI runners can run low on space while other test packages
write to /tmp in parallel, and the encode then dies with ENOSPC
mid-write, surfacing as a test failure. Check free space up front and
skip the subtest instead.
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* volume.fsck: delete missing filer entries through a worker pool
-reallyDeleteFilerEntries issued one synchronous HTTP DELETE per orphan
path inside the BFS consumer, so a million-entry purge ran one request
at a time. Queue the confirmed paths onto a bounded pool (default 16,
-tunable via -purgeConcurrency) so deletes run in parallel while the
traversal continues.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* volume.fsck: serialize command output across purge workers
* volume.fsck: serialize purges that share a hard link id
* volume.fsck: cover the purge worker pool
* volume.fsck: take the purge shard remainder in unsigned arithmetic
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* shell: remove the directories emptied by volume.fsck's filer entry purge
volume.fsck -findMissingChunksInFiler -reallyDeleteFilerEntries deleted the
orphan entries but left their parent directories behind, so a namespace
accumulated empty directories that had to be cleaned up by hand.
Remember the parent of every purged entry and, once the purge is done, walk
up from each one deleting the directories that are now empty. The delete is
non-recursive, so the filer itself rejects a directory that still has
children; a bucket and a directory that is an S3 object of its own are left
alone.
Claude-Session: https://claude.ai/code/session_01BncsNo2RVANCDtdbw96Kfc
* shell: keep a directory volume.fsck saw change under it
The empty-directory sweep read the entry to spot an S3 directory key object
and then deleted unconditionally, so a directory promoted to an object in
between was removed anyway.
Delete with the mtime the lookup returned, leaving the filer to skip a
directory that has changed since.
Claude-Session: https://claude.ai/code/session_01BncsNo2RVANCDtdbw96Kfc
* shell: leave a directory volume.fsck just saw written for the next run
The mtime the delete is conditioned on has second resolution, so a write
landing in the same second as the one already on the directory is
indistinguishable from it and the directory would still be deleted.
Skip a directory modified within the last few seconds. A write after the
lookup then always carries a later second than the one the delete carries,
and the sweep picks the directory up on the next run.
Claude-Session: https://claude.ai/code/session_01BncsNo2RVANCDtdbw96Kfc
* shell: skip a directory volume.fsck cannot condition a delete on
A zero mtime disables the delete's condition at the filer, so a directory
whose entry carries none was removed unconditionally and a concurrent
promotion to an S3 object went with it.
Leave such a directory alone.
Claude-Session: https://claude.ai/code/session_01BncsNo2RVANCDtdbw96Kfc
* shell: hold volume.fsck's quiet period to the cutoff second itself
Mtime keeps whole seconds, so a directory whose mtime lands on the cutoff
second was written up to a second after it. Skip that directory too, so the
quiet period fails closed.
Claude-Session: https://claude.ai/code/session_01BncsNo2RVANCDtdbw96Kfc
