scooby87 73949daa98 filer: keep sequential prefetch within prefetchCount chunks (#11674) ...

ChunkReadAt.ReadAt asks the ReaderCache to prefetch after every read,
and MaybeCache counted only the downloads it newly started toward the
window, so each call reached prefetchCount chunks further than the last.
A reader consuming a chunk in several reads, as the S3 GET path does
with a 256 KiB copy buffer over 4 MiB chunks, ran the read-ahead up to
the ReaderCache downloader limit and kept most of a large object in
memory, enough to get an S3 gateway OOM-killed mid-response.

Count every chunk in the window, downloading or not, so the read-ahead
stays at prefetchCount chunks past the reader.


Assisted-by: LLM

Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>

2026-10-12 17:41:01 +08:00

..

abstract_sql

filer/postgres: create filemeta table on startup via createTable config (#11229)

2026-09-08 16:17:02 -07:00

arangodb

filer: use bind variables for request-controlled values in the arangodb store (#10795)

2026-08-17 15:15:26 -07:00

cassandra

cassandra: do not mask KvGet backend errors as not-found (#10361)

2026-07-18 13:57:24 -07:00

cassandra2

cassandra: do not mask KvGet backend errors as not-found (#10361)

2026-07-18 13:57:24 -07:00

elastic/v7

add elastic8 filer store for Elasticsearch 8 (#9916)

2026-06-10 12:10:49 -07:00

empty_folder_cleanup

mount/shell: bucket allow-empty-folders toggle, mount keeps explicit false (#11370)

2026-09-17 20:30:53 -07:00

etcd

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

foundationdb

filer: guard FoundationDB 100KB value limit and pack earlier (#11161)

2026-09-04 23:12:27 -07:00

hbase

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

leveldb

filer: preserve accepted metadata log records on shutdown (#11359)

2026-09-16 12:23:59 -07:00

leveldb2

filer: close the empty-folder race by checking after each mutation (#10799)

2026-08-17 19:57:05 -07:00

leveldb3

filer: keep the internal .system folder out of the per-bucket store path (#10248)

2026-07-06 11:25:50 -07:00

mongodb

fix(mongodb): merge URI auth fields with username/password override (#9889)

2026-06-09 00:18:33 -07:00

mysql

filer/postgres: create filemeta table on startup via createTable config (#11229)

2026-09-08 16:17:02 -07:00

mysql2

filer: a listing over a hard link no longer deadlocks a bounded SQL pool (#11118)

2026-09-02 23:45:45 -07:00

posixlock

filer/posixlock: remove the unused lock-set serde (#9676)

2026-05-25 13:15:19 -07:00

postgres

filer/postgres: create filemeta table on startup via createTable config (#11229)

2026-09-08 16:17:02 -07:00

postgres2

filer/postgres: create filemeta table on startup via createTable config (#11229)

2026-09-08 16:17:02 -07:00

redis

filer: restore a folder that received an entry while it was deleted (#10783)

2026-08-17 00:04:07 -07:00

redis2

filer: restore a folder that received an entry while it was deleted (#10783)

2026-08-17 00:04:07 -07:00

redis3

filer: restore a folder that received an entry while it was deleted (#10783)

2026-08-17 00:04:07 -07:00

redis_conf

build(deps): bump github.com/redis/go-redis/v9 from 9.21.0 to 9.22.0 (#11306)

2026-09-14 16:16:35 -07:00

redis_tls

filer: honor the documented TLS options in every redis store (#10425)

2026-07-24 10:26:38 -07:00

rocksdb

filer: reuse the caller's fetched entry in CreateEntry (#9645)

2026-05-23 21:40:41 -07:00

sqlite

filer: a listing over a hard link no longer deadlocks a bounded SQL pool (#11118)

2026-09-02 23:45:45 -07:00

store_test

fix(weed/filer/store_test): fix dropped errors (#8782)

2026-03-26 12:07:48 -07:00

tarantool

tarantool: fix upsert data corruption and missing context propagation (#10926)

2026-08-24 15:17:08 -07:00

tikv

go fix

2026-02-20 18:42:00 -08:00

ydb

filer: keep the internal .system folder out of the per-bucket store path (#10248)

2026-07-06 11:25:50 -07:00

configuration.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

copy_params.go

Use filer-side copy for mounted whole-file copy_file_range (#8747)

2026-03-23 18:35:15 -07:00

entry_codec_atime_test.go

feat(filer): add atime to FuseAttributes + TouchAccessTime RPC (#9556)

2026-05-19 10:22:17 -07:00

entry_codec_attributes_test.go

filer: list directories without decoding chunk lists (#10616)

2026-08-07 12:03:18 -07:00

entry_codec_attributes.go

filer: list directories without decoding chunk lists (#10616)

2026-08-07 12:03:18 -07:00

entry_codec.go

feat(filer): add atime to FuseAttributes + TouchAccessTime RPC (#9556)

2026-05-19 10:22:17 -07:00

entry_s3_expiry_test.go

[s3] force filer apply s3 expiry metadata (#10469)

2026-08-05 23:55:20 -07:00

entry.go

filer: apply the path's storage rule TTL on every write path (#10963)

2026-08-26 08:49:25 -07:00

filechunk_group_test.go

filer: clean up manifest resolve error propagation and add webdav tes… (#11297)

2026-09-13 18:34:09 -07:00

filechunk_group.go

filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

2026-10-08 22:04:06 +08:00

filechunk_manifest_cache_benchmark_test.go

filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

2026-10-08 22:04:06 +08:00

filechunk_manifest_cache_test.go

filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

2026-10-08 22:04:06 +08:00

filechunk_manifest_cache.go

[Mount] Cache Chunk Manifest Resolution for Repeated File Opens (#11266)

2026-09-12 20:06:25 -07:00

filechunk_manifest_resolve_test.go

s3: separate Object Lock configuration permission (#11361)

2026-09-16 15:27:09 -07:00

filechunk_manifest_rollback_test.go

filer: pack SSE chunks into manifests (#11175)

2026-09-05 10:16:17 -07:00

filechunk_manifest_test.go

filer: fail the request, not the process, when chunks cannot be resolved (#11685)

2026-10-10 23:39:29 +08:00

filechunk_manifest.go

filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

2026-10-08 22:04:06 +08:00

filechunk_section_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

filechunk_section.go

[Mount] Add ChunkGroup seeking tests and fix boundary handling (#11223)

2026-09-08 10:50:41 -07:00

filechunks2_test.go

filer: fail the request, not the process, when chunks cannot be resolved (#11685)

2026-10-10 23:39:29 +08:00

filechunks_cancel_test.go

filer: fail the request, not the process, when chunks cannot be resolved (#11685)

2026-10-10 23:39:29 +08:00

filechunks_read_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

filechunks_read.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

filechunks_test.go

fix(filer): prevent nil pointer dereference in CompactFileChunks when context is cancelled (#11690)

2026-10-11 22:17:26 +08:00

filechunks.go

fix(filer): prevent nil pointer dereference in CompactFileChunks when context is cancelled (#11690)

2026-10-11 22:17:26 +08:00

filer_buckets.go

Prevent bucket renaming in filer, fuse mount, and S3 (#8048)

2026-01-16 19:48:09 -08:00

filer_conf_test.go

filer: skip UpdateEntry when inline content is unchanged (#11580)

2026-10-03 21:00:51 +08:00

filer_conf.go

admin: add bucket lifecycle rule editing (#10860)

2026-08-21 23:42:26 -07:00

filer_delete_collection_test.go

filer: resolve the collection a bucket delete drops (#11439)

2026-09-25 07:30:42 +08:00

filer_delete_entry_test.go

fix: return NoSuchKey and drop stale entries when a remote-mounted object is gone from the remote (#11680)

2026-10-10 11:00:04 +08:00

filer_delete_entry.go

fix: return NoSuchKey and drop stale entries when a remote-mounted object is gone from the remote (#11680)

2026-10-10 11:00:04 +08:00

filer_deletion_persist_test.go

filer: persist pending chunk deletions across restarts (durable deletion ledger) (#11550)

2026-10-02 22:37:31 +08:00

filer_deletion_persist.go

filer: persist pending chunk deletions across restarts (durable deletion ledger) (#11550)

2026-10-02 22:37:31 +08:00

filer_deletion_test.go

filer: persist pending chunk deletions across restarts (durable deletion ledger) (#11550)

2026-10-02 22:37:31 +08:00

filer_deletion.go

filer: persist pending chunk deletions across restarts (durable deletion ledger) (#11550)

2026-10-02 22:37:31 +08:00

filer_hardlink.go

move to https://github.com/seaweedfs/seaweedfs

2022-07-29 00:17:28 -07:00

filer_inode_test.go

filer: fail writes when the existing-entry lookup fails (#11699)

2026-10-10 22:32:23 +08:00

filer_inode.go

filer: keep generated inodes inside the positive signed 64-bit range (#11567)

2026-10-03 09:13:59 +08:00

filer_lazy_remote_listing.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

filer_lazy_remote_test.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

filer_lazy_remote.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

filer_notify_append_test.go

fix(filer): apply -filer.disk default to metadata log assigns (#10080)

2026-06-24 10:47:11 -07:00

filer_notify_append.go

filer: bound metadata log flush retries during shutdown (#11596)

2026-10-06 09:51:02 +08:00

filer_notify_inflight_test.go

fix(filer): bound aggregated metadata reads by peer watermarks (#10803)

2026-08-19 18:38:46 -07:00

filer_notify_read_minute_test.go

fix(filer): stop skipping recent unflushed events on metadata subscription gaps (#10501)

2026-08-02 00:06:22 -07:00

filer_notify_read_testing.go

fix(filer): stop skipping recent unflushed events on metadata subscription gaps (#10501)

2026-08-02 00:06:22 -07:00

filer_notify_read.go

fix(filer): stop skipping recent unflushed events on metadata subscription gaps (#10501)

2026-08-02 00:06:22 -07:00

filer_notify_test.go

filer: stop an oversized metadata log flush from wedging the change feed (#10430)

2026-07-24 17:59:20 -07:00

filer_notify.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

filer_on_meta_event_test.go

Adjust rename events metadata format (#8854)

2026-03-30 18:25:11 -07:00

filer_on_meta_event.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

filer_prefix_object_test.go

filer: demote client-cancelled directory listing log from error (#11495) (#11496)

2026-09-28 14:35:41 +08:00

filer_remote_evict_test.go

filer: evict remote-cached objects under storage pressure (#11515)

2026-09-29 22:05:42 +08:00

filer_remote_evict.go

filer: evict remote-cached objects under storage pressure (#11515)

2026-09-29 22:05:42 +08:00

filer_remote_tombstone_test.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

filer_remote_tombstone.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

filer_rename_test.go

stats: remote-mount bucket cache hit/miss metrics (#10352)

2026-07-16 16:58:45 -07:00

filer_rename.go

Prevent bucket renaming in filer, fuse mount, and S3 (#8048)

2026-01-16 19:48:09 -08:00

filer_search.go

filer: async empty folder cleanup via metadata events (#7614)

2025-12-03 21:12:19 -08:00

filer_shutdown_test.go

filer: bound metadata log flush retries during shutdown (#11596)

2026-10-06 09:51:02 +08:00

filer.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

filerstore_hardlink.go

fix(filer): update hard link ctime when nlink changes on unlink (#9018)

2026-04-10 11:23:52 -07:00

filerstore_translate_path.go

Add error list each entry func (#7485)

2025-11-25 19:35:19 -08:00

filerstore_wrapper_test.go

fix(filer): remove cancellation guard from RollbackTransaction and clean up #8909 (#8916)

2026-04-03 17:55:27 -07:00

filerstore_wrapper.go

fix: return NoSuchKey and drop stale entries when a remote-mounted object is gone from the remote (#11680)

2026-10-10 11:00:04 +08:00

filerstore.go

Add error list each entry func (#7485)

2025-11-25 19:35:19 -08:00

gateway_upload.go

refactor: centralize genUploadUrl in UploadOption (#10164)

2026-06-30 20:45:43 -07:00

interval_list_test.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

interval_list.go

Mount concurrent read (#4400)

2023-04-13 22:32:45 -07:00

manifest_batch_foundationdb.go

filer: guard FoundationDB 100KB value limit and pack earlier (#11161)

2026-09-04 23:12:27 -07:00

manifest_batch.go

filer: guard FoundationDB 100KB value limit and pack earlier (#11161)

2026-09-04 23:12:27 -07:00

meta_aggregator_replay_test.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

meta_aggregator_test.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

meta_aggregator_testing.go

filer: end local-only metadata subscriptions when remote peers appear (#11251)

2026-09-09 20:16:50 -07:00

meta_aggregator_watermark_test.go

filer: stop aggregated metadata subscribers from spinning on a peer watermark hold (#10863)

2026-08-21 15:22:05 -07:00

meta_aggregator.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

meta_log_collection_test.go

filer: option to store system metadata logs in their own collection (#11551)

2026-10-02 08:50:24 +08:00

meta_path_ts.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

meta_replay.go

filer: converge peer metadata replication on restart (#11698)

2026-10-12 17:39:34 +08:00

metadata_event_context.go

Adjust rename events metadata format (#8854)

2026-03-30 18:25:11 -07:00

metadata_event_sink_test.go

reload filer config on local metadata updates (#10622)

2026-08-07 19:51:45 -07:00

metadata_event_sink.go

mount: make metadata cache rebuilds snapshot-consistent (#8531)

2026-03-07 09:19:40 -08:00

mount_peer_registry_test.go

peer chunk sharing 2/8: filer mount registry (#9131)

2026-04-18 20:03:23 -07:00

mount_peer_registry.go

peer chunk sharing 2/8: filer mount registry (#9131)

2026-04-18 20:03:23 -07:00

persisted_log_cache_test.go

Cut per-subscriber replay decode and widen replay concurrency (#9917)

2026-06-10 13:08:34 -07:00

persisted_log_cache.go

filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

2026-10-08 22:04:06 +08:00

placement_overlay.go

filer: add a placement overlay seam for the write path (#10437)

2026-07-25 11:20:41 -07:00

read_remote_test.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

read_remote.go

preserve Content-Encoding for remote-mounted objects (#10340)

2026-07-15 19:20:00 -07:00

read_write_test.go

filer: skip UpdateEntry when inline content is unchanged (#11580)

2026-10-03 21:00:51 +08:00

read_write.go

filer: skip UpdateEntry when inline content is unchanged (#11580)

2026-10-03 21:00:51 +08:00

reader_at_prefetch_window_test.go

filer: keep sequential prefetch within prefetchCount chunks (#11674)

2026-10-12 17:41:01 +08:00

reader_at_shared_test.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

reader_at_test.go

S3: optionally serve remote-mounted objects from remote when the local read fails (#10837)

2026-08-21 11:47:53 -07:00

reader_at.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

reader_cache_budget.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

reader_cache_memory_test.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

reader_cache_test.go

filer: cut ReaderCache mutex contention on small reads (#11677)

2026-10-10 06:52:37 +08:00

reader_cache.go

filer: keep sequential prefetch within prefetchCount chunks (#11674)

2026-10-12 17:41:01 +08:00

reader_pattern_test.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

reader_pattern.go

s3: keep small ranged GETs on range reads, no whole-chunk downloads (#11577)

2026-10-03 15:01:04 +08:00

remote_mapping_test.go

read cold remote objects straight from the origin while caching (#10731)

2026-08-12 23:00:10 -07:00

remote_mapping.go

read cold remote objects straight from the origin while caching (#10731)

2026-08-12 23:00:10 -07:00

remote_storage_test.go

filer: apply SSRF guard to the lazy-remote fetch/list/delete paths (#11294)

2026-09-13 14:43:55 -07:00

remote_storage.go

filer: keep lazy remote reads from resurrecting deleted paths (#11452)

2026-09-26 08:00:39 +08:00

s3iam_conf_test.go

s3: fix configuring IAM for the same user

2022-08-30 09:37:52 -07:00

s3iam_conf.go

convert error fromating to %w everywhere (#6995)

2025-07-16 23:39:27 -07:00

stream_benchmark_test.go

go fmt

2026-04-10 17:31:14 -07:00

stream_jwt_test.go

filer: authorize the chunk proxy and the root listing like the rest of the filer port (#11152)

2026-09-04 16:39:36 -07:00

stream_manifest_error_test.go

filer: clean up manifest resolve error propagation and add webdav tes… (#11297)

2026-09-13 18:34:09 -07:00

stream_prefetch_test.go

feat(s3): add concurrent chunk prefetch for large file downloads (#8917)

2026-04-03 19:57:30 -07:00

stream_prefetch.go

read: try a replica that stopped answering last, and relearn its volume's locations (#11130)

2026-09-03 11:51:48 -07:00

stream_source_error_test.go

s3 sink: report the source read error the SDK hides (#11277)

2026-09-11 19:44:30 -07:00

stream_test.go

filer: stop isSameChunks from sorting caller-owned chunk slices (#11584)

2026-10-04 17:37:26 +08:00

stream.go

filer: fail the request, not the process, when chunks cannot be resolved (#11685)

2026-10-10 23:39:29 +08:00

topics.go

merge current message queue code changes (#6201)

2024-11-04 12:08:25 -08:00