test: boto3 s3vectors conformance harness (P1) (#12105)

* vector: boto3 s3vectors conformance harness

test/vectors spins a mini cluster, an s3 gateway with -port.vectors and
-vectors.grpc, and a weed-vector-server, then exercises the AWS client
end to end: bucket and index lifecycle, put/get/list/delete vectors,
and a filtered query asserting distances and metadata.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run the s3vectors conformance subset

An e2e job in rust-vector-tests: builds weed and weed-vector-server,
then runs test/vectors against a mini cluster plus the vectors gateway.
The workflow now also watches vector.proto, its stubs, and test/vectors.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: fix e2e protoc and codespell findings

The e2e job needed the same vendored-protoc step as the unit-test job —
lance's build scripts locate protoc themselves. Codespell flags the AWS
`$nin` operator literal and the `lits` variable: exempt `nin` in
.codespellrc like the existing word exemptions, rename the variable,
and fold the negation into the IN/NOT IN symbol directly.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: classify catalog errors by code, not message text

* s3: drop the implicit vector namespace on vector-bucket delete

* test: point the vector engine at the gateway lance port

* test: bind the vectors gateway listeners to loopback

* test: make the filter and distance assertions load-bearing

* test: rebuild the harness binaries on every run

* ci: fix the vectors end-to-end job

* ci: run the vector tests on gateway and command changes

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

test/vectors/Makefile

@@ -0,0 +1,28 @@
+# End-to-end S3 Vectors conformance run:
+# weed mini (cluster + lance catalog) + weed s3 (s3vectors port) +
+# weed-vector-server (VectorService gRPC), then boto3 s3vectors calls.
+WEED := ./weed
+VEC := ./weed-vector-server
+
+.PHONY: all test build clean FORCE
+
+all: test
+
+# No prerequisites: go/cargo decide for themselves whether a rebuild is needed.
+$(WEED): FORCE
+	go build -o $(WEED) ../../weed
+
+$(VEC): FORCE
+	cargo build --release --manifest-path ../../seaweed-vector/Cargo.toml -p weed-vector-server
+	cp ../../seaweed-vector/target/release/weed-vector-server $(VEC)
+
+FORCE:
+
+build: $(WEED) $(VEC)
+
+test: build
+	./run_test.sh
+
+clean:
+	rm -f $(WEED) $(VEC)
+	rm -rf mini-data

test/vectors/run_test.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Spin a mini cluster plus the vectors gateway and engine, run the boto3
+# conformance subset, tear everything down.
+set -euo pipefail
+cd "$(dirname "$0")"
+
+DATA=mini-data
+rm -rf "$DATA"
+PIDS=()
+cleanup() {
+  kill "${PIDS[@]}" 2>/dev/null || true
+  rm -rf "$DATA"
+}
+trap cleanup EXIT
+
+./weed mini -dir "$DATA" \
+  -s3=false -webdav=false -admin.ui=false &
+PIDS+=($!)
+
+./weed s3 \
+  -filer 127.0.0.1:8888 \
+  -ip 127.0.0.1 \
+  -ip.bind 127.0.0.1 \
+  -port 8334 \
+  -port.lance 9102 \
+  -port.vectors 9201 \
+  -vectors.grpc 127.0.0.1:9301 &
+PIDS+=($!)
+
+./weed-vector-server \
+  --namespace http://127.0.0.1:9102 \
+  --listen 127.0.0.1:9301 &
+PIDS+=($!)
+
+# Any HTTP response (even an auth rejection) means the lance catalog is up.
+for _ in $(seq 1 60); do
+  code=$(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:9102/v1/namespace/root/list || true)
+  [ "$code" != "000" ] && break
+  sleep 1
+done
+[ "${code:-000}" != "000" ] || { echo "lance catalog on :9102 never came up" >&2; exit 1; }
+
+for _ in $(seq 1 60); do
+  (exec 3<>/dev/tcp/127.0.0.1/9301) 2>/dev/null && break
+  sleep 1
+done
+(exec 3<>/dev/tcp/127.0.0.1/9301) 2>/dev/null || { echo "vector server on :9301 never came up" >&2; exit 1; }
+
+python3 -m pytest -xvs s3vectors_test.py

test/vectors/s3vectors_test.py

@@ -0,0 +1,81 @@
+"""Boto3 conformance subset for the S3 Vectors API gateway.
+
+Run via `make test` — a mini cluster, an s3 gateway with -port.vectors, and a
+weed-vector-server are up by then.
+"""
+
+import boto3
+import pytest
+
+ENDPOINT = "http://127.0.0.1:9201"
+BUCKET = "vectors-bucket"
+INDEX = "idx"
+
+
+@pytest.fixture(scope="module")
+def vectors():
+    return boto3.client(
+        "s3vectors",
+        endpoint_url=ENDPOINT,
+        aws_access_key_id="test",
+        aws_secret_access_key="test",
+        region_name="us-east-1",
+    )
+
+
+def test_vector_lifecycle(vectors):
+    vectors.create_vector_bucket(vectorBucketName=BUCKET)
+    buckets = [b["vectorBucketName"] for b in vectors.list_vector_buckets()["vectorBuckets"]]
+    assert BUCKET in buckets
+
+    vectors.create_index(
+        vectorBucketName=BUCKET,
+        indexName=INDEX,
+        dataType="float32",
+        dimension=4,
+        distanceMetric="euclidean",
+    )
+    idx = vectors.get_index(vectorBucketName=BUCKET, indexName=INDEX)["index"]
+    assert idx["dimension"] == 4
+
+    vectors.put_vectors(
+        vectorBucketName=BUCKET,
+        indexName=INDEX,
+        vectors=[
+            {"key": "a", "data": {"float32": [1, 0, 0, 0]}, "metadata": {"color": "red"}},
+            {"key": "b", "data": {"float32": [0.95, 0.05, 0, 0]}, "metadata": {"color": "blue"}},
+            {"key": "c", "data": {"float32": [0.9, 0.1, 0, 0]}, "metadata": {"color": "red"}},
+        ],
+    )
+
+    got = vectors.get_vectors(
+        vectorBucketName=BUCKET, indexName=INDEX, keys=["a"], returnData=True, returnMetadata=True
+    )["vectors"]
+    assert got[0]["key"] == "a" and got[0]["metadata"]["color"] == "red"
+
+    listed = vectors.list_vectors(vectorBucketName=BUCKET, indexName=INDEX)["vectors"]
+    assert sorted(v["key"] for v in listed) == ["a", "b", "c"]
+
+    hits = vectors.query_vectors(
+        vectorBucketName=BUCKET,
+        indexName=INDEX,
+        queryVector={"float32": [1, 0, 0, 0]},
+        topK=2,
+        returnDistance=True,
+        returnMetadata=True,
+        filter={"color": {"$eq": "red"}},
+    )["vectors"]
+    # b is the second-nearest vector but blue, so an ignored filter would
+    # return ["a", "b"] here.
+    assert [h["key"] for h in hits] == ["a", "c"]
+    distances = {h["key"]: h["distance"] for h in hits}
+    assert distances["a"] == pytest.approx(0.0, abs=1e-6)
+    assert distances["c"] == pytest.approx(0.02, abs=1e-4)
+    assert all(h["metadata"]["color"] == "red" for h in hits)
+
+    vectors.delete_vectors(vectorBucketName=BUCKET, indexName=INDEX, keys=["b"])
+    listed = vectors.list_vectors(vectorBucketName=BUCKET, indexName=INDEX)["vectors"]
+    assert sorted(v["key"] for v in listed) == ["a", "c"]
+
+    vectors.delete_index(vectorBucketName=BUCKET, indexName=INDEX)
+    vectors.delete_vector_bucket(vectorBucketName=BUCKET)