S3 Vector Bucket Performance: distributed querying is in Enterprise

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
Chris LuandClaude Opus 5.5 committed 2026-11-02 03:57:42 -08:00
1 parent 8de89b6267
commit fa4e48fd5e
1 file changed
+4
+4
@@ -56,6 +56,10 @@ What the numbers show:
* **Ingest is steady.** Batch latency did not grow from the first to the last tenth of the load (for example 283 ms → 272 ms at 50k × 384).
* **No consistent difference between the volume servers.** The largest gap, the filtered query at 50k × 384 (82 ms on Go, 45 ms on Rust), goes the other way at 10k × 128 (40 ms against 51 ms).
## Beyond one engine
The open-source gateway sends every query to one `weed-vector` engine at a time, so every number above comes from a single engine. The [SeaweedFS Enterprise](https://seaweedfs.com/) version adds distributed vector querying: each index is spread over a group of engines, a query goes to one group, each engine in it searches its share of the index, and the gateway merges the results. An index can have several groups to serve more queries, and when an engine is lost, queries keep returning the same results from the remaining engines. The distributed mode has not been measured on this page.
## The read cache
`weed-vector` keeps reads of an index's data files in memory, 512 MiB by default (`--read-cache-mb`, `WEED_VECTOR_READ_CACHE_MB`; `0` disables it). The results above ran with the default. The same 50k × 384 index on the Go volume server, with the cache off and on, run alternately twice: