Commit Graph
2 Commits
Author SHA1 Message Date
Chris Lu f46b2a1925 Stop the filer test helpers from pinning gigabytes of log buffers (#10560)
* log buffer: wake the interval loop on shutdown instead of sleeping through it

loopInterval parked in time.Sleep(flushInterval) and only re-checked
IsStopping when it woke, so a buffer shut down early kept both loop
goroutines - and the PreviousBufferCount+1 slabs of BufferSize they
reach - alive for up to a full interval afterwards. Select on shutdownCh
against a ticker instead, and give the loops a WaitGroup so a test can
observe that they exit.

* test: release the filers the server tests build

Every helper here left its filer's meta log buffer running, so each test
pinned PreviousBufferCount+1 buffers of BufferSize for the rest of the
run: ~3.5GB of live heap across the package, which overruns the address
space on linux/386 and kills the 32-bit job with an out-of-memory throw.

Thread the test through the helpers so the buffer is shut down on
cleanup, and shut the subscribe harness's filer down outright - its
deletion loop keeps the whole filer reachable otherwise. That harness
quiesces its flush path first, since Filer.Shutdown closes the store a
flush still in flight would write through.
2026-08-04 11:38:38 -07:00
Chris Lu cdb60069a6 filer: conditional UpdateEntry with a chunk-set write condition (#10382)
* filer: accept a WriteCondition on UpdateEntry, under the per-path lock

UpdateEntry was a bare read-modify-write: the precondition check, the
chunk garbage diff, and the store write could interleave with a
concurrent update to the same path. Take the per-path lock CreateEntry
already holds, and evaluate an optional CreateEntry-style WriteCondition
under it, failing with FailedPrecondition like expected_extended.

* filer: IF_CHUNKS_EQUAL write condition compares the stored chunk fid set

A chunk-preserving read-modify-write (tagging, setattr, copy-in-place)
races UpdateEntry's garbage diff: if a concurrent update empties the
chunk list first, the stale writer's commit resurrects fids that are
already queued for deletion, stranding the entry on a dead needle once
vacuum reclaims it. The reverse also holds: a writer that read an empty
chunk list can wipe chunks a concurrent update just added.

IF_CHUNKS_EQUAL guards both: the stored chunk fid multiset must still
equal what the caller read, order-independent, with an empty fids list
expecting no chunks. Absent entry counts as no chunks for CreateEntry
overwrites and transactions.

* filer: delete and append serialize on the entry path lock

DeleteEntry queues the entry's chunks for deletion and AppendToEntry
rewrites the chunk list, but neither held the per-path lock, so either
could interleave with a conditional update between its precondition
check and its write — a passed IF_CHUNKS_EQUAL would then resurrect
fids already on the deletion queue, or clobber a freshly appended
chunk. AppendToEntry keeps the cluster lock for cross-filer append
serialization; the path lock covers the local read-modify-write.

* filer: reuse lockPath in UpdateEntry lookup
2026-07-21 00:19:15 -07:00