mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-04 11:07:35 +01:00
5dfda3645ead2003ff85cd2f236bb296af39fc22
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris LuandClaude Opus 5.5
|
5063aa9be2 |
weed-vector: check master certificates against the server allowlists, read -config_dir (#12374)
* weed-vector: look for security.toml in -config_dir first weed processes search the directory -config_dir names before ., so a cluster keeping security.toml there left the engine in plaintext. --config-dir (also -config_dir, WEED_VECTOR_CONFIG_DIR) does the same. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: dial the masters the way weed clients do The master's certificate must chain to grpc.ca and is not checked against the host dialed, as weed clients verify (advancedtls.CertVerification), so masters sharing one workload certificate, or dialed by IP, register as they do from weed. Instead, grpc.client.master_allowed_server_commonNames, grpc.client.allowed_server_commonNames or grpc.allowed_server_commonNames (and the wildcard domain likewise) restrict which server identities are accepted: the common name or a DNS SAN, or an exact URI SAN, as security.Authenticator.AuthenticateServer does. These used to be refused at startup; --tls-server-name is gone with the host check. The TLS runs in a connector that reads the credentials asynchronously at every dial, so the connect timeout still covers the reads, the TCP connect and the handshake together. An IPv6 master's brackets are stripped before the dial. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3vectors: the hardened leg allowlists server identities for grpc.client The gateway and the engine then accept only servers named localhost, which exercises the engine's allowed_server_commonNames check on the master dial. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: parse the refresh interval with time.ParseDuration's arithmetic The interval was summed in floating-point seconds, so 0.9ns rounded up to 1ns where Go truncates it to 0 and falls back to the 5h default, 0.3s was off by a nanosecond, and 3000000h was accepted where Go rejects anything past the longest time.Duration. Each part is now an integer count of nanoseconds plus the fraction Go adds, with Go's overflow bounds, and the value is no longer trimmed, as Go does not trim it either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: test a master dial whose credential read stalls The bounded-dial tests passed no credentials or a prebuilt endpoint, so moving the credential read out of the connect timeout would have failed none of them. The new test points grpc.ca at a FIFO nobody writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: dial a closed port in the stalled credential read test Against a silent listener, a dial that never read the credentials timed out with the same message, so the test passed without the read. Nothing listens on the port now, so only the stalled read can time out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: test that Security::load searches the config dir Only candidates() was tested, so load dropping config_dir on the way would have passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: expand a leading ~ in -config_dir like weed's ResolvePath Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: hold the port in the stalled credential read test A released port can be taken before the dial, and a dial that skipped the read would then time out and pass. A listener that drops every connection keeps the port and still fails such a dial at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Chris LuandClaude Opus 5.5
|
eeb8aefe70 |
weed-vector: gRPC TLS to the masters and for the gateway (#12368)
* weed-vector: read the cluster's security.toml The engine finds security.toml where weed processes do (., ~/.seaweedfs, /usr/local/etc/seaweedfs, /etc/seaweedfs) with WEED_* environment variables over it, and builds gRPC client and server credentials by the rules of security.LoadClientTLS and LoadServerTLS: a dedicated client certificate is preferred, both halves of a pair must be set, and grpc.ca is required once either is. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: gRPC TLS to the masters and from the gateway The masters are dialed with [grpc.client]'s credentials when the cluster has them, the master's certificate checked against the host dialed or --tls-server-name. The engine's own listener takes its identity from [grpc.vector] and then requires a client certificate chaining to grpc.ca, which is how the gateway, dialing with [grpc.client], reaches it. The rustls provider is pinned before the first TLS use, as the other binaries linking both aws-lc-rs and ring do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * scaffold: document [grpc.vector] Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: put every gRPC hop under mutual TLS in the hardened leg VECTOR_TLS=1 now also writes [grpc.master], [grpc.volume], [grpc.filer], [grpc.s3], [grpc.client] and [grpc.vector], and probes the master's and the engine's gRPC ports: plaintext HTTP/2 and a client without a certificate are refused, the client certificate gets a SETTINGS frame. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: give the test volume server 64 volume slots Every collection grows 7 volumes and test_vector_bucket_encryption opens two buckets while the previous test's bucket is still being torn down, so 16 slots were marginal: the Go legs failed CreateIndex now and then with "failed to find writable volumes" and master.log showing "create 7 volume, created 4: Not enough data nodes found!". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: reload the listener's certificates and enforce allowed_commonNames The listener now runs the way weed servers run theirs. The certificate, key and CA files are re-read every WEED_TLS_CERT_REFRESH_INTERVAL (weed's default of 5h) and on SIGHUP, and new handshakes use the changed files without a restart; a file that no longer parses is logged and the working configuration stays. [grpc.vector] allowed_commonNames and grpc.allowed_wildcard_domain restrict which client certificates are served, checked against the subject common name as security.Authenticator and the Rust volume server's CommonNameVerifier do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * scaffold: allowed_commonNames and the refresh cadence for [grpc.vector] Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: the engine's allowlist keeps an outsider certificate out The hardened leg sets [grpc.vector] allowed_commonNames = "localhost" and probes the engine with a second certificate the same CA signed for CN=outsider: the handshake must be refused, while the gateway's certificate still gets an HTTP/2 SETTINGS frame. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: take the master's TLS name from the parsed endpoint A master given without a port (https://master.example) had the whole URL checked against its certificate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: bound the master TLS handshake connect_timeout covers only the TCP connect, so a master that accepted the connection and never answered the handshake held up the other masters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: look up security.toml keys the way viper does An empty WEED_* variable no longer hides the file's value, and keys match regardless of case, as for weed processes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: refuse master allowlists it cannot enforce weed checks a dialed server against allowed_server_commonNames and allowed_server_wildcard_domain on top of grpc.ca. The engine's master dial accepted every certificate the CA signed even when one was set; it now fails at startup and names the key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3vectors: read the whole frame header in the gRPC TLS probe Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3vectors: probe the volume, filer and S3 gRPC listeners too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial credentials only when given masters A standalone engine never dials with grpc.client, so its settings no longer stop it from starting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: run each listener TLS handshake on its own task The accept loop awaited every handshake before accepting the next connection, so a peer that connected and sent nothing stopped every gateway connection after it. Handshakes now run concurrently, each under a 10s timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial's TLS files at every dial The client certificate, key and CA were read once at startup, so after the files rotated a reconnect still presented the old pair. Each dial now reads them, as the listener re-reads its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read every unit Go durations accept in the refresh interval WEED_TLS_CERT_REFRESH_INTERVAL values in ns, us or a leading + fell back to five hours in the engine while weed servers honored them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: fall back on a refresh interval too long to hold An oversized WEED_TLS_CERT_REFRESH_INTERVAL panicked the engine at startup; weed servers reject it and keep the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: refuse allowed_commonNames without the listener's certificate With [grpc.vector] allowed_commonNames set but no cert and key, the listener served plaintext and checked no client at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial's credentials inside its connect timeout The CA, certificate and key were read synchronously before the timeout started, so a stalled credential path held up every other master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: let a set key win over an empty spelling of it With allowed_server_commonNames = "" beside allowed_server_commonnames = "master", the lookup stopped at the empty exact match and the allowlist went unnoticed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Chris LuandClaude Opus 5.5
|
55f1e1efc5 |
vector: cache reads of write-once Lance files across requests (#12303)
* lance worker: let dataset::open take an object store wrapper open_with is open with the dataset's object store wrapped, so a caller such as weed-vector can put a read cache in front of the table's files. open keeps its signature and opens unwrapped, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: cache reads of write-once Lance files across requests Every VectorService call opens the table afresh, and Lance keeps no data pages between opens, so each QueryVectors re-read the whole vector column through the S3 gateway: 83 MB and about 228 range GETs per query at 50k x 384. The gateway and volume server spent their CPU streaming it and throughput stayed at 6-8 queries/s whatever the client concurrency. weed-vector now wraps each table's object store with a bounded in-memory cache (--read-cache-mb, 512 MiB by default, 0 disables). It only serves data files, deletion files and index files: Lance writes each once under a fresh random name, so a cached read cannot go stale, even when an index is dropped and recreated at the same location. Manifests, tags and every other path still go to the store, and each call still describes the table, loads the latest version and takes the query lease, so reads see every write committed before them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: cache an owned copy of each fetched read A range from get_ranges can be a slice of a larger response buffer, such as a coalesced fetch, and caching the slice kept the whole buffer alive while the weigher counted only the slice. The cache now copies each fetched range and whole-object read into its own buffer, so --read-cache-mb bounds what it actually holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chris LuandClaude Opus 5.5
|
4be191173a |
vector: keep ListVectors pages and segments flat as an index grows; read only keys in GetVectors (#12300)
* vector: scan only the fragments a ListVectors page covers A page pushed its offset into the scan, but the scan still loaded every fragment first, reading each one's deletion file: about 105 S3 GETs per page at 50k rows against 25 at 10k. The page's fragments now come from the live row counts in the manifest, so a page reads only its own. Tokens and row order are unchanged; a manifest without the counts keeps the old scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3vectors: list a ListVectors segment in the engine The gateway paged through the whole index for every segment and kept one row in segmentCount, so a segment cost segmentCount engine pages and 16 segments listed slower than one. The engine now takes segment_count and segment_index: it reads keys across the stride, keeps the segment's rows and takes their data and metadata alone, one call per page. A segment is still the rows whose position modulo segmentCount is segmentIndex, and its token still the position to resume from. An engine that predates the fields does not set segmented, and the gateway then pages through it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: read only the key column across the table in GetVectors The key lookup scanned every column. On S3, lance reads a column during the scan when a row of it is under 1000 bytes, so a 128-dimension vector was read for every row of the table to return 100 of them, and data and metadata were read even when the request did not ask for them. The scan now projects the requested columns and materializes everything but the key late, for the matching rows only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3vectors: start the legacy segment fallback from the page already read With an engine that ignores segment_count, the gateway dropped the page it had just read and fetched it again. The fallback now consumes that page first, so a one-row segment page still costs one engine call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: check metadata on segmented ListVectors pages The segment test listed data only. Its rows now carry metadata on two in three keys, and every segmented row, read through take_rows, must equal the same row in a plain listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: lease the version while a segment takes its data A segmented page with data or metadata reads keys first and then takes the other columns from the same version, so maintenance could reclaim its files in between. That path now holds a lease like GetVectors. A keys-only page is one scan and stays unleased, as plain pages are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * vector: leave GetVectors materialization to lance Forcing every column late made a 100-key GetVectors with data slower at 10k x 128 in the perf cluster: 54.7 ms against 46.3 ms. The take issued a read per row, where lance's heuristic reads a narrow vector column in one pass per fragment. The projection keeps the gain on its own: with neither data nor metadata requested, the call went from 38.6 ms to 25.3 ms and from 150 to 89 S3 GETs. At 50k x 384 the vector is already late under the heuristic, and a full lookup is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chris Lu
|
190ab32cfb |
vector: weed-vector registers to master; s3 discovers it there (#12146)
* vector: weed-vector registers with the weed master weed-vector-server --masters holds a KeepConnected stream to the weed master, the same mechanism filer and s3 use via wdclient — the server shows up in ListClusterNodes(client_type=vector). Resends registration and rediscovers on failure. * vector: s3 discovers weed-vector through the master With -vectors.grpc unset the vectors gateway resolves one registered weed-vector via ListClusterNodes on the masters it already derives from filer config — discovery replaces the static dial for the single-worker OSS path. The conformance harness registers the engine to mini's master and drops -vectors.grpc. * vector: default-fill registration fields ..Default::default() keeps the source compiling against proto variants that carry extra fields, so the same file works in either tree. * vector: serve the Vectors API on the main s3 port The vectors router sits on the main -port listener behind a dispatch that checks the SigV4 credential scope for /s3vectors/ — no separate port needed. -port.vectors stays for explicit dedicated-port deployments; the conformance harness now hits vectors through :8334. * vector: drop -port.vectors entirely It was new — no compat to preserve. The main -port listener serves the API whenever a weed-vector backend is reachable. * vector: drop -vectors.grpc; discovery is the only locator weed-vector always registers to master, the gateway always resolves it through ListClusterNodes — a static address just forks membership. * vector: binary is weed-vector Matches the enterprise binary; package and lib names are unchanged. * vector: review fixes — cluster type, grpc port, lazy resolve, leader hint, advertise - cluster: register 'vector' as a node type so ListClusterNodes returns weed-vector members - s3: dial masters via ToGrpcAddress (HTTP port + 10000) - s3: backend resolves lazily through vectorDiscovery — workers joining after gateway startup work on first query, and address changes re-dial - master_client: convert master HTTP addr to grpc +10000, honor http(s) scheme, follow the leader hint in KeepConnectedResponse, and --advertise covers wildcard/unreachable listen addresses * vector: no --advertise flag; derive the outbound address A wildcard listen resolves to the source IP the master sees via a route lookup, so registration always advertises something reachable without another flag. * vector: review fixes — plaintext dispatch, conn leaks, discovery backoff The two plaintext http servers now serve the vectors dispatch too — signed s3vectors calls on the main port reach the right router. discoverVectorWorker closes each master conn and bounds lookups to 5s; a dead or absent worker no longer re-hits master on every RPC — the resolver backs off for 5s between rediscovery attempts. * vector: 1 GiB grpc cap, dead-worker invalidation, CLI rename Valid PutVectors batches exceed the tonic 4 MiB default — match the gateway's 1 GiB cap. A replaced worker no longer sticks: any RPC error resets the cached address and the next call rediscovers through the master. The core CLI bin renames to weed-vector-cli so the two crates do not collide on the same output path. * vector: retry-only-transport reset, periodic re-resolve, cli rename - reset() takes the failed client and runs only on transport errors — an InvalidArgument on a healthy conn no longer interrupts unrelated requests, and a concurrent re-resolve wins the race. - cli() re-resolves once the interval passes even with a live client, so a worker that moves addresses is picked up. - verify.rs spike docs use weed-vector-cli after the bin rename. * vector: only Unavailable counts as a transport failure A request deadline or an application error can fire on a healthy connection — resetting the shared conn on those churned reconnects. |
||
|
Chris LuandDevin
|
e2d899ac25 |
test/vectors: fix the S3 Vectors e2e harness (#12112)
* test/vectors: give the vector server static storage credentials The e2e harness ran weed-vector-server without credentials, so lance's AWS object store fell back to IMDS and CreateIndex died on the runner's 411 from the token endpoint. Point it at the test credentials the s3 gateway now reads from s3.json. * vector: take the list item field from the schema when batching vectors Lance stores the fixed-size list item as nullable even when the write schema declared it non-null, so PutVectors built batches that no longer matched: expected FixedSizeList(4 x Float32) but found FixedSizeList(4 x non-null Float32). * vector: store index metadata as lance jsonb so filters resolve The metadata field was plain Utf8, but the filter functions lance's planner registers (json_get_string, json_get_float, json_exists) take a LargeBinary jsonb column, so QueryVectors with a metadata filter failed to plan. Declare the field with the arrow.json extension — lance encodes it to jsonb on write and back on read — and encode the batch's strings when the stored column is LargeBinary. Also rename the filter's json_get_str call to json_get_string, the name lance registers. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Chris LuandDevin
|
87f15e4d6c |
vector: weed-vector-server, the VectorService gRPC (P1) (#12102)
* vector: VectorService protobuf contract The shared RPC boundary between the Go S3 gateway and the Rust vector engine from the vector design doc: CreateIndex/DropIndex, Put/Get/ List/Delete/Query vectors. A Table reference carries the Lance table id and an optional pinned version, matching the snapshot discipline queries are required to keep. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: generated Go stubs for vector.proto protoc output via weed/pb/Makefile. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: carry return_data through QueryVectors S3 Vectors QueryVectors honours returnData, but the contract had no way to ask for or carry vector data in a query response, so the gateway would need a second GetVectors round-trip per query. Add return_data to the request and data to QueryResult now, while the contract is still new. * vector: pin metadata numbers to double precision in the contract Metadata arrives as JSON, where numbers carry double precision; say so on the field so neither side promises integer fidelity past 2^53. * vector: declare_table on the lance namespace client CreateIndex needs a way to ask the catalog for a new table's location; expose POST /v1/table/{id}/declare. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: cargo fmt coverage.rs Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: weed-vector-server, the VectorService gRPC A tonic server over weed-vector-core: CreateIndex declares the table through the namespace and writes an empty {key, vector, metadata} dataset; PutVectors upserts by key (delete + append until merge_insert is exercised); Get/List/Delete/Query route through the engine's resolved table, honoring pinned versions and the prefilter default. Metadata travels as a Utf8 JSON column, proto Struct on the wire. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: stamp the metric in the schema, tighten put validation CreateIndex now bounds dimension at the S3 Vectors 4096 maximum and records the distance metric in the vector field's Arrow metadata, so a later reader recovers the index's configured metric from the table itself. batch_of checks every vector's data length individually before flattening — equal totals could still let a short vector borrow floats from its neighbor — and writes against the dataset's own schema. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: upsert vectors in a single merge commit PutVectors ran delete-then-append: a failed append left the old keys deleted, and two concurrent puts could both append a replacement. Use lance's merge_insert on the key column — one commit, no delete window, conflicts fail the request instead of duplicating keys. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: read the index's metric; fetch metadata in one take QueryVectors hardcoded L2 — it now reads the metric the index was created with from the vector field's metadata. return_metadata fetched each hit's row one at a time; collect the row ids into a single take_rows call and join metadata back on the hit key. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: honor return_data in query results QueryResult grew a data field but query_vectors never populated it. Fetch the vector column in the same batched take_rows as metadata and return each hit's stored embedding. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: reject a malformed query as invalid_argument A zero top_k silently became one result and a dim-mismatched query vector surfaced as an internal scan error; reject both before querying. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: reject a malformed list next_token A token that failed to parse silently restarted the listing from offset zero; surface it as invalid_argument instead. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: answer an empty put without a commit and cover ragged input An empty vectors list would still run a merge commit; return the current version instead. Pin the ragged-vector check with a test. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: keep non_filterable_metadata_keys on the table schema create_index dropped the declared filter policy. Persist it as JSON on the metadata field so the index carries which keys a filter may touch; enforcing the list belongs to the gateway that translates the request filter into Lance SQL. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: reject duplicate keys in one put A repeated key for a new row committed both copies, and for an existing key the merge failure surfaced as internal. Reject duplicates up front as invalid_argument. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: report open failures as not_found or unavailable A catalog 404 or a missing dataset/version is not a client mistake, and a catalog outage is retryable: mapping every open error to invalid_argument made both indistinguishable from a bad request. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: page list_vectors when max_results is unset A proto3 client that omits max_results sends 0, which used to limit the scan to zero rows and return no token. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * vector: drop stray empty service.rs.tmp * vector: rename datas map to data_map codespell flags datas as a misspelling of data; use a distinct map name. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |