Commit Graph
1 Commits
Author SHA1 Message Date
Chris LuandDevin 7388d0d4fc vector: seaweed-vector workspace + flat/indexed top-K engine (P0 phase 1) (#12096)
* vector: scaffold the seaweed-vector workspace

Standalone cargo workspace for the vector query engine, mirroring the
seaweed-worker layout: own lockfile, same lint policy, path dependencies
across the two trees since there is no common root manifest.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: resolve a table's vector layout

VectorTable::resolve finds the vector column (explicit name, else the
column named vector, else the sole fixed-size float list), the optional
key column, and the metadata columns a query can project.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: flat and indexed top-k, key lookup

query::flat runs an exact scan (use_index(false)); query::ann lets Lance
split the query between index segments and a flat pass over fragments no
index covers yet, so rows written after the last index build stay
eligible. Both honour a metadata filter and return the raw _distance in
the query's metric.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: reject a non-utf8 or non-identifier key column

An explicit --key-column naming a non-UTF8 field made hits() silently
report row ids as keys, and get() interpolated the name into a SQL
filter verbatim. Require Utf8 at resolve and in hits(), and require a
plain identifier before building the IN predicate.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: fall back to row id on a null key

keys.value() on a null slot fabricates an empty-string key; a row with
no key reports its row id instead, matching the no-key-column case.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: quote the key column in the IN filter

A Utf8 key column named e.g. document-key made the predicate parse as
subtraction, and an identifier interpolated unquoted is injectable.
Backtick-quote it and double its inner backticks; double quotes are
string literals in the filter dialect, not identifiers.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* vector: project only the columns a hit needs

The knn projection pulled every metadata column — including other
vector columns — although Hit returns none of them. Filters still
evaluate off the scan path, so restrict the output to the key and the
scoring columns.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 10:13:05 -07:00