vector: weed-vector registers to master; s3 discovers it there (#12146)

* vector: weed-vector registers with the weed master

weed-vector-server --masters holds a KeepConnected stream to the weed
master, the same mechanism filer and s3 use via wdclient — the server
shows up in ListClusterNodes(client_type=vector). Resends registration
and rediscovers on failure.

* vector: s3 discovers weed-vector through the master

With -vectors.grpc unset the vectors gateway resolves one registered
weed-vector via ListClusterNodes on the masters it already derives
from filer config — discovery replaces the static dial for the
single-worker OSS path. The conformance harness registers the engine
to mini's master and drops -vectors.grpc.

* vector: default-fill registration fields

..Default::default() keeps the source compiling against proto variants
that carry extra fields, so the same file works in either tree.

* vector: serve the Vectors API on the main s3 port

The vectors router sits on the main -port listener behind a dispatch
that checks the SigV4 credential scope for /s3vectors/ — no separate
port needed. -port.vectors stays for explicit dedicated-port
deployments; the conformance harness now hits vectors through :8334.

* vector: drop -port.vectors entirely

It was new — no compat to preserve. The main -port listener serves the
API whenever a weed-vector backend is reachable.

* vector: drop -vectors.grpc; discovery is the only locator

weed-vector always registers to master, the gateway always resolves it
through ListClusterNodes — a static address just forks membership.

* vector: binary is weed-vector

Matches the enterprise binary; package and lib names are unchanged.

* vector: review fixes — cluster type, grpc port, lazy resolve, leader hint, advertise

- cluster: register 'vector' as a node type so ListClusterNodes returns
  weed-vector members
- s3: dial masters via ToGrpcAddress (HTTP port + 10000)
- s3: backend resolves lazily through vectorDiscovery — workers joining
  after gateway startup work on first query, and address changes re-dial
- master_client: convert master HTTP addr to grpc +10000, honor http(s)
  scheme, follow the leader hint in KeepConnectedResponse, and
  --advertise covers wildcard/unreachable listen addresses

* vector: no --advertise flag; derive the outbound address

A wildcard listen resolves to the source IP the master sees via a
route lookup, so registration always advertises something reachable
without another flag.

* vector: review fixes — plaintext dispatch, conn leaks, discovery backoff

The two plaintext http servers now serve the vectors dispatch too —
signed s3vectors calls on the main port reach the right router.
discoverVectorWorker closes each master conn and bounds lookups to 5s;
a dead or absent worker no longer re-hits master on every RPC — the
resolver backs off for 5s between rediscovery attempts.

* vector: 1 GiB grpc cap, dead-worker invalidation, CLI rename

Valid PutVectors batches exceed the tonic 4 MiB default — match the
gateway's 1 GiB cap. A replaced worker no longer sticks: any RPC error
resets the cached address and the next call rediscovers through the
master. The core CLI bin renames to weed-vector-cli so the two crates
do not collide on the same output path.

* vector: retry-only-transport reset, periodic re-resolve, cli rename

- reset() takes the failed client and runs only on transport errors —
  an InvalidArgument on a healthy conn no longer interrupts unrelated
  requests, and a concurrent re-resolve wins the race.
- cli() re-resolves once the interval passes even with a live client,
  so a worker that moves addresses is picked up.
- verify.rs spike docs use weed-vector-cli after the bin rename.

* vector: only Unavailable counts as a transport failure

A request deadline or an application error can fire on a healthy
connection — resetting the shared conn on those churned reconnects.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-10-28 00:58:31 -07:00
1 parent 81879fba7b
commit 190ab32cfb
15 files changed
+423 -116

No files matched your search

+3 -3
View File
@@ -1,8 +1,8 @@
# End-to-end S3 Vectors conformance run:
# weed mini (cluster + lance catalog) + weed s3 (s3vectors port) +
# weed-vector-server (VectorService gRPC), then boto3 s3vectors calls.
# weed-vector (VectorService gRPC), then boto3 s3vectors calls.
WEED := ./weed
VEC := ./weed-vector-server
VEC := ./weed-vector
.PHONY: all test build clean FORCE
@@ -14,7 +14,7 @@ $(WEED): FORCE
$(VEC): FORCE
cargo build --release --manifest-path ../../seaweed-vector/Cargo.toml -p weed-vector-server
cp ../../seaweed-vector/target/release/weed-vector-server $(VEC)
cp ../../seaweed-vector/target/release/weed-vector $(VEC)
FORCE:
+3 -4
View File
@@ -23,13 +23,12 @@ PIDS+=($!)
-ip 127.0.0.1 \
-ip.bind 127.0.0.1 \
-port 8334 \
-port.lance 9102 \
-port.vectors 9201 \
-vectors.grpc 127.0.0.1:9301 &
-port.lance 9102 &
PIDS+=($!)
./weed-vector-server \
./weed-vector \
--namespace http://127.0.0.1:9102 \
--masters 127.0.0.1:9333 \
--storage-option aws_access_key_id=test \
--storage-option aws_secret_access_key=test \
--listen 127.0.0.1:9301 &
+3 -3
View File
@@ -1,13 +1,13 @@
"""Boto3 conformance subset for the S3 Vectors API gateway.
Run via `make test` — a mini cluster, an s3 gateway with -port.vectors, and a
weed-vector-server are up by then.
Run via `make test` — a mini cluster, an s3 gateway serving the vectors API on its main port, and a
weed-vector are up by then.
"""
import boto3
import pytest
ENDPOINT = "http://127.0.0.1:9201"
ENDPOINT = "http://127.0.0.1:8334"
BUCKET = "vectors-bucket"
INDEX = "idx"