* filer: shut down test filers' meta-log buffers, fixing the 32-bit OOM
Each log_buffer.NewLogBuffer preallocates five 8 MiB windows and spawns
loopFlush/loopInterval goroutines that keep them reachable. The
weed/filer tests built ~70 filers and aggregators that almost never
called ShutdownLogBuffer, pinning ~2.8 GB plus live churn, and the
linux/386 job ran out of its ~4 GiB address space inside
newSealedBuffers.
- MetaAggregator.Shutdown now retires MetaLogBuffer too: the "aggr"
buffer's loops previously outlived the aggregator.
- newTestFiler and newTestFilerWithStubStore register t.Cleanup
shutdowns; the one-off filer/aggregator literals defer theirs.
* filer: wait for the aggregated meta-log buffer to stop in Shutdown
ShutdownLogBuffer only signals the stop; loopFlush and its pooled windows
exit asynchronously. Match Filer.Shutdown's local-buffer sequence so the
aggregator returns with the buffer's memory already released.
* filer: cancel test filers' flush context before buffer shutdown
The stub-store helpers build real NewFiler instances, so the queued
shutdown flush retries appendToFile every ~737ms under flushCtx, which
nothing ever cancelled — the buffer and its pool stayed alive past each
test. Cancelling first lets the flush drop the window and WaitForShutdown
confirms loopFlush exited.
* filer: wait for test log buffers to drain after shutdown
The stub-flush buffers cannot hang, but without WaitForShutdown their
loopFlush goroutines release the pooled windows asynchronously — the
memory this PR means to free is still counted as live while the next
test allocates.
* filer: support nanosecond timestamps for conditional delete (#11339)
* shell: fs.verify prune carries the entry mtime in nanoseconds
IfNotModifiedAfter now resolves to sub-second precision on the filer,
so pass the looked-up mtime_ns along with mtime; the re-lookup check
catches same-second rewrites that share the second, and the test mock
mirrors the filer's seconds/nanoseconds cutoff.
* shell: volume.fsck guards empty-directory deletes with mtime nanoseconds
The looked-up mtime_ns joins the seconds in IfNotModifiedAfter so a
directory rewritten within the same second is not dropped; the stub
filer mirrors the filer's scale heuristic and the race fixtures use a
realistic mtime so the nanosecond branch engages.
* filer: cover the seconds guard and equality boundary
Test second-resolution cutoffs on both sides of the entry's second and
the exact-equality nanosecond guard that still deletes.
* filer: compare the nanosecond guard as a time, not UnixNano
UnixNano overflows for mtimes outside 1678-2262, letting a far-future
entry slip past the guard; time.After compares exactly at every epoch.
The seconds branch keeps Unix() so a same-second cutoff still deletes.
* filer_pb: add explicit if_not_modified_after_ns to DeleteEntryRequest
* filer: prefer explicit nanoseconds guard, keep if_not_modified_after as seconds
* shell: send seconds and nanoseconds delete cutoffs, skip unexpressible mtimes
* filer: merge conditional-delete guards into one nanosecond cutoff
DeleteEntryMetaAndData carried both the seconds and nanoseconds guards
even though the precedence between them is fixed, so every caller passed
a dead parameter. The gRPC handler now folds if_not_modified_after into
nanoseconds at the boundary, where (sec+1)*1e9-1 keeps the whole-second
cutoff exact, and the filer takes a single ifNotModifiedAfterNs.
* filer: hold the conditional-delete cutoff as a time, not int64 nanoseconds
A seconds-only guard at or past year 2262 saturated to MaxInt64
nanoseconds (~2262), so a delete whose cutoff matched a far-future mtime
was skipped while the request still reported success, and any entry
between 2262 and the real cutoff survived pruning. The merged cutoff is
now a time.Time, where a whole-second guard maps to the last nanosecond
of its second for any int64 second.
---------
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* feat(filer): option to store system metadata logs in their own collection
The filer's internal /topics/.system/log chunks are assigned to the
filer's default collection (-collection). In a multi-filer deployment
that default is often empty, so every restart flap, full-sync, or
event-buffered flush grows the default collection with system chunks that
are indistinguishable from user data in collection.list. This is a large
part of what makes the default collection balloon and confuses orphan
analysis.
This keeps the internal log in a dedicated collection when the operator
asks for one, without changing where user data goes:
- New optional override, filer.options.metaLog.collection (and
.replication), read in NewFiler so both `weed filer` and
`weed server -filer` honour it. Default "" => exactly today's
behaviour (log follows the filer default), fully backward compatible.
- Resolution is a small helper: override first, then the filer default,
then a storage rule matched on the log path. Kept separate from the
user write path so the internal log targets itself.
- bucketCollection() is hardened the same way it already protects the
filer's default collection: a bucket that happens to resolve to the
redirected meta-log collection must not drop it on delete, because it
backs internal log volumes.
- Scaffold filer.toml documents the new knobs under [filer.options].
Related to the persisted deletion ledger branch (fix/persist-deletion-queue):
together they cut the two sources of post-flap junk in the default
collection — that PR stops orphaned user-chunk leak on filer crash,
this one stops the internal log from living in default at all. They are
independent: no file overlap, no functional dependency; either can merge
first. They are paired only in the narrative of cleaning up default.
Adds unit tests for the collection/replication resolution chain, the
viper keys, and the bucket-delete guard (run green under -race).
Co-Authored-By: Athena 🏛️ <hermes-agent@local> (custom / Qwen3.8-Flash-Next-ROCmFP4)
* filer: collect bucket chunks when its collection survives the delete
bucketCollection returning "" preserves the collection, but the bucket
path still skipped per-entry chunk collection and could skip listing the
children entirely, so a bucket sharing the meta-log (or any preserved)
collection left its object chunks orphaned with no entry pointing at
them. Only the wholesale drop of a deleted collection skips those now.
Note in filer.toml that the meta-log target should stay stable: chunks
written under an older collection are not migrated.
* filer: exercise the metaLog override wiring through NewFiler
The viper test only echoed back the keys it set, so a wrong key in
NewFiler would still pass. It now asserts the fields NewFiler fills
from those keys.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* filer: tighten comments around the metaLog collection override
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Chris Lu