Commit Graph
4 Commits
Author SHA1 Message Date
Chris Lu dd75add62f filer: fetch only the part of a chunk the view covers
Replaces the PinRandomMode size heuristic with a per-chunk coverage rule.
ViewFromVisibleIntervals already clips chunk views to the request window,
so a view that is not IsFullChunk() is one the request only partially
needs; fetch it as a range regardless of the detected read pattern.

This closes the holes a request-size pin left open: ranges larger than
SeqTolerance no longer revert to whole-chunk downloads once their buffers
look sequential, and ranges that fully cover a chunk keep the shared
whole-chunk path instead of fetching 256KiB slices piecemeal. Prefetch
(MaybeCache) skips clipped views so it cannot amplify a range read either.

PinRandomMode is dropped: no caller needs it once coverage drives the
fetch choice. Range fetches route through fetchChunkDataFn so tests
observe them the same way as whole-chunk downloads.
2026-10-03 09:36:59 +08:00
Chris LuandDevin 58aa9acfb0 s3: pin small ranged GETs to range reads
A ranged GET whose first read lands within SeqTolerance of offset 0 is
judged sequential immediately, and even a far-starting range could flip
back mid-request; either way readChunkSliceAt downloads each covered
chunk in full, multiplying disk reads for small ranged reads (measured
~7x). Pin random mode for ranged requests no larger than SeqTolerance so
all of the request's buffer reads stay range fetches. Larger ranges keep
the dynamic pattern, where whole-chunk fetches amortize.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-02 09:33:33 -07:00
Chris LuandDevin 8227ae3f49 filer: keep a ranged read in random mode through its contiguous tail
A far ReadAt on a fresh ReaderPattern left the sequential counter at -1,
so the next buffer of the same ranged request landed on the frontier and
flipped the verdict straight back to sequential — readChunkSliceAt then
paid a whole-chunk fetch for the remainder of the range. Drop the
counter to -ModeChangeLimit when random mode is entered so the verdict
needs sustained sequential evidence to undo, matching the hysteresis an
established sequential stream already gets.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-02 09:32:40 -07:00
Chris Lu 7e13340dcb filer: tolerance-window read pattern detection for concurrent readahead (#9983)
* filer: tolerance-window read pattern detection for concurrent readahead

ReaderPattern detected sequential access via strict contiguity (lastReadStopOffset
== offset). Under concurrent/reordered ReadAt (parallel readahead, which ReadAt
already supports via RLock + atomics), that exact match frequently misses even for
a genuinely sequential stream, drifting toward random mode and disabling prefetch.

Track a read frontier (max offset+size, advanced with a lock-free CAS loop) and
treat a read as sequential when its start is within SeqTolerance (8 MiB) of the
frontier. The window absorbs reordered/concurrent readahead while still rejecting
far random jumps; the existing ModeChangeLimit hysteresis is unchanged. API
(NewReaderPattern/MonitorReadAt/IsRandomMode) is unchanged, so reader_at is
unaffected. Adds reader_pattern_test.go.

* filer: read the read frontier inside the CAS loop

Capture the frontier from the CAS loop's own load rather than a separate
up-front snapshot, so the sequentiality diff is judged against the freshest
pre-image even if a concurrent readahead advances the frontier while we loop.
Also drops the now-redundant load. Mirrors the same fix on the write-side
ReaderPattern twin (review feedback on the companion PR).

* filer: add frontier/boundary/recovery guard tests for ReaderPattern

Mirror the regression guards added to the write-side WriterPattern: assert
the max-frontier never regresses on a backward read (the CAS invariant), pin
both sides of the inclusive SeqTolerance boundary, and verify sustained near
reads recover sequential mode out of the negative floor. Each guards an
invariant the original four tests left uncovered.
2026-06-16 09:15:44 -07:00