[Mount] Cache Chunk Manifest Resolution for Repeated File Opens (#11266)

* cache resolved chunk manifests for Mount

* Address PR review: per-mount cache, singleflight, reuse ResolveOneChunkManifest

- Own the manifest cache per WFS mount instead of a process-global
  variable, so manifests from one filer backend are never served to
  another (Devin/CodeRabbit major bug).
- Coalesce concurrent cold misses via singleflight so only one fetch
  runs during a cold burst (Greptile P2).
- Copy cached data after releasing the mutex so a large copy does not
  block concurrent hits, inserts, and evictions (CodeRabbit nitpick).
- Reuse the existing ResolveOneChunkManifest function name instead of
  introducing a new resolveOneChunkManifest wrapper.
- Validate (unmarshal) manifest bytes before caching so malformed
  manifests do not poison the cache.
- Add TestChunkGroupManifestResolutionCoalescesColdMisses covering
  the singleflight cold-miss path.

* Address round 2 review: coalesced-miss cancellation, test overlap

- Use singleflight.DoChan in fetchOrLoad and select on ctx.Done() so a
  caller whose context is canceled while waiting for an in-flight fetch
  returns ctx.Err() promptly instead of blocking for the leader's
  result (Devin BUG).
- Add TestResolveOneChunkManifestCanceledWaiterReturnsDuringCoalescedMiss
  covering the canceled-waiter path.
- Delay the cold-miss fixture response so the leader's fetch is still
  in flight when concurrent opens join the singleflight, making the
  one-fetch assertions reliable (CodeRabbit Minor).

* Address review: keep ResolveOneChunkManifest four-argument

Restore the exported ResolveOneChunkManifest to its original
four-argument signature so external callers keep compiling. Move the
cache-aware resolution into an unexported resolveOneChunkManifest
helper that accepts the per-mount ChunkManifestCache. The exported
function delegates to the helper with a nil cache, preserving the
historical uncached behavior for every non-Mount caller. The Mount
path (ChunkGroup.SetChunks) now calls the unexported helper with the
mount-owned cache. Tests and benchmarks that exercise the cache path
call the unexported helper directly.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-09-12 20:06:25 -07:00
1 parent bd6bcd47e3
commit 8db41d0217
11 files changed
+595 -20

No files matched your search

+1 -1
View File
@@ -15,7 +15,7 @@ func TestChunkGroupReaderCacheMemory(t *testing.T) {
budget := NewReaderCacheBudget(8 << 10)
groups := make([]*ChunkGroup, 32)
for i := range groups {
group, err := NewChunkGroup(func(context.Context, string) ([]string, error) { return []string{"unused"}, nil }, newMockChunkCacheForReaderCache(), nil, 128, nil, budget)
group, err := NewChunkGroup(func(context.Context, string) ([]string, error) { return []string{"unused"}, nil }, newMockChunkCacheForReaderCache(), nil, 128, nil, nil, budget)
if err != nil {
t.Fatal(err)
}