* cache resolved chunk manifests for Mount
* Address PR review: per-mount cache, singleflight, reuse ResolveOneChunkManifest
- Own the manifest cache per WFS mount instead of a process-global
variable, so manifests from one filer backend are never served to
another (Devin/CodeRabbit major bug).
- Coalesce concurrent cold misses via singleflight so only one fetch
runs during a cold burst (Greptile P2).
- Copy cached data after releasing the mutex so a large copy does not
block concurrent hits, inserts, and evictions (CodeRabbit nitpick).
- Reuse the existing ResolveOneChunkManifest function name instead of
introducing a new resolveOneChunkManifest wrapper.
- Validate (unmarshal) manifest bytes before caching so malformed
manifests do not poison the cache.
- Add TestChunkGroupManifestResolutionCoalescesColdMisses covering
the singleflight cold-miss path.
* Address round 2 review: coalesced-miss cancellation, test overlap
- Use singleflight.DoChan in fetchOrLoad and select on ctx.Done() so a
caller whose context is canceled while waiting for an in-flight fetch
returns ctx.Err() promptly instead of blocking for the leader's
result (Devin BUG).
- Add TestResolveOneChunkManifestCanceledWaiterReturnsDuringCoalescedMiss
covering the canceled-waiter path.
- Delay the cold-miss fixture response so the leader's fetch is still
in flight when concurrent opens join the singleflight, making the
one-fetch assertions reliable (CodeRabbit Minor).
* Address review: keep ResolveOneChunkManifest four-argument
Restore the exported ResolveOneChunkManifest to its original
four-argument signature so external callers keep compiling. Move the
cache-aware resolution into an unexported resolveOneChunkManifest
helper that accepts the per-mount ChunkManifestCache. The exported
function delegates to the helper with a nil cache, preserving the
historical uncached behavior for every non-Mount caller. The Mount
path (ChunkGroup.SetChunks) now calls the unexported helper with the
mount-owned cache. Tests and benchmarks that exercise the cache path
call the unexported helper directly.
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
* fix(filer): bound retained reader cache buffers by bytes
* test(filer): keep in-flight downloads during cache trimming
* feat(mem): expose pooled allocation capacity for byte reservations
* fix(mount): share a configurable reader buffer budget across files
* fix(filer): release failed prefetch slots and memory reservations
* feat(mount): expose a soft Go runtime memory limit
* docs(filer): restore shared-download rationale in startCaching
The one-line comment replacing the original context.Background() explanation was too thin for readChunkAt to cross-reference shared resource semantics. Restore a concise note on why request cancellation must not abort a download shared by concurrent readers.
* test(filer): loosen reader cache test deadlines to 5s
Three tests used 1-second deadlines that can flake on CI under load:
TestReaderCacheBudgetInFlight, TestReaderCacheEvictionDoesNotHoldCacheLock,
and TestReaderCacheFailedPrefetchReleasesBudget. Increase to 5 seconds.
* test(filer): cover re-read after reader cache eviction
Add TestReaderCacheReReadAfterEviction: reads chunk 'a', reads chunk 'b'
(evicting 'a' via budget pressure), then re-reads 'a' and asserts a
fresh download returns correct data. Verifies the core correctness
property that eviction never exposes missing or stale data to readers.
