filer leveldb2: tunable cache, buffer, block and table sizes (#11927)

* filer leveldb2: read cache and buffer sizes from filer.toml

The leveldb2 store hardcoded its block cache (32MiB) and write buffer
(16MiB) and offered no way to tune the table block size or compaction
table size. Optional block_cache_size_mb, write_buffer_size_mb,
block_size_kb and compaction_table_size_mb keys under [leveldb2] now
override them; zero or negative values keep the previous defaults.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2: document tuning options in scaffold filer.toml

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2: saturate size options instead of overflowing int

The option fields take int, so a MiB config value past math.MaxInt/MiB
(e.g. 2048 on 32-bit) wrapped negative and goleveldb would panic sizing
the memtable. Clamp the product instead of the input.

* filer leveldb2: reject size options too large for int bytes

Clamping a MiB setting at math.MaxInt silently hands goleveldb a
different size than configured; fail Initialize on values that cannot
fit instead.

* filer leveldb3: same tunable cache, buffer, block and table sizes

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2/3: exercise oversized-option rejection on 64-bit too

The tests skipped whenever int was 64-bit, but write_buffer_size_mb
values above MaxInt/1MiB still overflow there, so the rejection path
had no coverage on the primary platform. Use MaxInt/1MiB + 1 so the
rejection triggers on every platform.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Chris LuandDevin authored and GitHub committed 2026-10-21 20:39:06 -07:00
1 parent 1451096c17
commit 0b197867a1
5 files changed
+306 -21

No files matched your search

+10
View File
@@ -33,12 +33,22 @@ recursive_delete = false
# faster than previous leveldb, recommended.
enabled = true
dir = "./filerldb2" # directory to store level db files
# optional tuning, applied to each of the 8 partitioned stores
# block_cache_size_mb = 32 # sorted-table read cache
# write_buffer_size_mb = 16 # memtable size before flushing to disk
# block_size_kb = 4 # uncompressed block ('page') size inside table files
# compaction_table_size_mb = 2 # target size of generated table files
[leveldb3]
# similar to leveldb2.
# each bucket has its own meta store.
enabled = false
dir = "./filerldb3" # directory to store level db files
# optional tuning, applied to the default store and each bucket store
# block_cache_size_mb = 32 # sorted-table read cache
# write_buffer_size_mb = 16 # memtable size before flushing to disk
# block_size_kb = 4 # uncompressed block ('page') size inside table files
# compaction_table_size_mb = 2 # target size of generated table files
[rocksdb]
# local on disk, similar to leveldb