* seaweed-vector: build and extend the vector index
weed_lance_worker::vector_index builds an IVF_FLAT index in the metric the
table was created with, sizes partitions from the row count, and extends an
index with a new segment or a retrain; the core crate re-exports it and adds
the probes a query spends. Tests cover coverage after build, append, retrain
and compaction.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* weed-vector-cli: index build and index optimize
Logs go to stderr so stdout stays the JSON a caller parses.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test/s3vectors: PERF_INDEX builds the index before the timed queries
The cluster fixture hands out its catalog URL, the Makefile builds
weed-vector-cli, and the perf suite builds the index after ingest and
reports the build time and segment count.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* seaweed-vector: test that a build takes the table's metric
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test/s3vectors: make clean removes weed-vector-cli
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* seaweed-vector: optimize only the indices on the vector column
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* seaweed-vector: resolve the query metric with index::distance_metric
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-vector: test the L2 metric under the name CreateIndex stores
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: PERF_CLUSTERS draws the perf suite's vectors around shared centres
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* seaweed-vector: test that compaction drops the deleted key from tied hits
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-vector: test that an append-only optimize serves the new row
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: record PERF_NOISE in the perf report config
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: report recall through an index without asserting it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: count a hit tying the k-th best score toward recall
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: allow float32 rounding at the recall cutoff
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: count a repeated hit key once toward recall
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: credit cutoff ties only up to the slots the top-k leaves
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test/s3vectors: assert every filtered hit passes the filter, indexed or not
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* test(s3vectors): add an opt-in end-to-end perf suite
perf_test.py runs on the same cluster fixture as the integration suite
(Go or Rust volume, single or distributed queries) and times the data
plane through the S3 Vectors API: PutVectors ingest and upsert, QueryVectors
latency with and without metadata and a 10% filter, query throughput
across client threads, GetVectors, segmented ListVectors, and
DeleteVectors. Recall is checked against an exact client-side search and
listings must be complete; timings only report. Sizes come from PERF_*
variables, and results go to a JSON report and a printed summary.
It runs through run_perf.sh or `make perf`. run_test.sh does not collect
it, so CI is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test(s3vectors): check every listed key in the perf suite, not just the count
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test(s3vectors): cap perf samples at PERF_VECTORS so small runs work
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test(s3vectors): warm perf worker clients before starting throughput timers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* test(s3vectors): report the key count of perf GetVectors and DeleteVectors calls
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
