mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-27 15:30:45 +01:00
Chris LuandDevin
7388d0d4fc
* vector: scaffold the seaweed-vector workspace Standalone cargo workspace for the vector query engine, mirroring the seaweed-worker layout: own lockfile, same lint policy, path dependencies across the two trees since there is no common root manifest. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: resolve a table's vector layout VectorTable::resolve finds the vector column (explicit name, else the column named vector, else the sole fixed-size float list), the optional key column, and the metadata columns a query can project. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: flat and indexed top-k, key lookup query::flat runs an exact scan (use_index(false)); query::ann lets Lance split the query between index segments and a flat pass over fragments no index covers yet, so rows written after the last index build stay eligible. Both honour a metadata filter and return the raw _distance in the query's metric. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: reject a non-utf8 or non-identifier key column An explicit --key-column naming a non-UTF8 field made hits() silently report row ids as keys, and get() interpolated the name into a SQL filter verbatim. Require Utf8 at resolve and in hits(), and require a plain identifier before building the IN predicate. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: fall back to row id on a null key keys.value() on a null slot fabricates an empty-string key; a row with no key reports its row id instead, matching the no-key-column case. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: quote the key column in the IN filter A Utf8 key column named e.g. document-key made the predicate parse as subtraction, and an identifier interpolated unquoted is injectable. Backtick-quote it and double its inner backticks; double quotes are string literals in the filter dialect, not identifiers. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: project only the columns a hit needs The knn projection pulled every metadata column — including other vector columns — although Hit returns none of them. Filters still evaluate off the scan path, so restrict the output to the key and the scoring columns. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
29 lines
936 B
TOML
29 lines
936 B
TOML
# Vector query engine for SeaweedFS Lance table buckets.
|
|
#
|
|
# `core` is the engine: open a table through the Lance namespace and run
|
|
# top-K over it. The crate layout follows vector_buckets.md; a `server`
|
|
# crate with the VectorService gRPC endpoint lands with the S3 Vectors
|
|
# milestone.
|
|
[workspace]
|
|
resolver = "2"
|
|
members = ["crates/core"]
|
|
|
|
[workspace.package]
|
|
version = "0.1.0"
|
|
edition = "2024"
|
|
# The edition needs 1.85; the lance dependency tree needs more. Same floor as
|
|
# seaweed-worker, which this workspace path-depends on.
|
|
rust-version = "1.94.1"
|
|
|
|
# No common root manifest links this tree to seaweed-worker or
|
|
# seaweed-common; keep the lint policy identical to theirs.
|
|
[workspace.lints.clippy]
|
|
needless_update = "allow"
|
|
undocumented_unsafe_blocks = "warn"
|
|
|
|
[workspace.dependencies]
|
|
anyhow = "1"
|
|
tokio = { version = "1", features = ["full"] }
|
|
tracing = "0.1"
|
|
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|