* perf(cache): drop OS page cache after disk cache reads
After reading from the on-disk chunk cache, advise the kernel via
FADV_DONTNEED to release the corresponding page cache pages. This
prevents double-caching the same data in both user-space and kernel
page caches, freeing RAM for other uses on systems with large disk
caches.
* fix(cache): guard dropReadCache against zero length and invalid fd
A zero-length fadvise is interpreted as "to end of file" on Linux,
which would inadvertently drop the page cache for the entire remainder
of the cache volume. Also check fd >= 0 to avoid unnecessary syscalls
when the backend file is closed.
* perf(cache): only apply FADV_DONTNEED for reads >= 1 MiB
For small needle reads the syscall overhead outweighs the memory
savings, and the kernel page cache is more beneficial for warm data.
Restrict fadvise to reads of at least 1 MiB where the freed page
cache is meaningful.
This reverts commit 670cb759f8.
with the pr
weed/storage () - (master) > go test -count=1 ./...
ok github.com/seaweedfs/seaweedfs/weed/storage 18.486s
? github.com/seaweedfs/seaweedfs/weed/storage/backend [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map 0.025s
? github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding 0.864s
? github.com/seaweedfs/seaweedfs/weed/storage/idx [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/needle 0.110s
ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map 24.414s
ok github.com/seaweedfs/seaweedfs/weed/storage/super_block 0.203s
? github.com/seaweedfs/seaweedfs/weed/storage/types [no test files]
? github.com/seaweedfs/seaweedfs/weed/storage/volume_info [no test files]
weed/storage () - (master) >
weed/storage () - (master) >
without the pr
weed/storage () - (master) >
weed/storage () - (master) > go test -count=1 ./...
ok github.com/seaweedfs/seaweedfs/weed/storage 1.617s
? github.com/seaweedfs/seaweedfs/weed/storage/backend [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/backend/memory_map 0.026s
? github.com/seaweedfs/seaweedfs/weed/storage/backend/s3_backend [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding 0.906s
? github.com/seaweedfs/seaweedfs/weed/storage/idx [no test files]
ok github.com/seaweedfs/seaweedfs/weed/storage/needle 0.202s
ok github.com/seaweedfs/seaweedfs/weed/storage/needle_map 24.533s
ok github.com/seaweedfs/seaweedfs/weed/storage/super_block 0.280s
? github.com/seaweedfs/seaweedfs/weed/storage/types [no test files]
? github.com/seaweedfs/seaweedfs/weed/storage/volume_info [no test files]
Chris Lu