S3 Vectors: the engine, worker and CLI keep tables on the filers and volume servers
After seaweedfs#12469 and #12470: weed-vector, weed-worker and
weed-vector-cli take --masters and hold no S3 credentials
(--storage-option, --access-key and --secret-key are refused); the
worker and CLI authenticate to the Lance catalog with the cluster key;
an SSE-S3 bucket needs the gateway's key on the engine too, including
weed mini's generated one. The performance page compares the engine
through the gateway with the direct one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
S3 Vectors: draw how it works with Mermaid; mark the missing operations planned
The diagram shows what the old text art blurred: the S3 Vectors API, the S3
API and the Lance catalog are one S3 server process, the engine reaches the
data through the catalog and the S3 API, and the master only connects the
gateway to the engine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
S3 Vectors: what changed with #12318, #12319, #12320, #12323 and #12324
Verified on master 02d22800aa in a clean container:
- euclidean returns the Euclidean distance ([3, 4] from [0, 0] is 5.0)
- bucket-scoped identity actions, account-based principals and ownership
- request bodies up to 32 MiB; larger ones get ValidationException
- InternalFailure says to look at the gateway log, which logs the cause at the
default level
- weed-vector defaults to the Lance port 9101 and listens on 8090; it works on
a SeaweedFS without identities
- AES256 vector buckets encrypt their index objects and need an SSE-S3 key
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
S3 Vectors: array filters, and building weed-vector without protoc
Filters now match array elements with $eq, a bare value and $in, with
$ne/$nin as their opposites (seaweedfs#12321), checked live on master
54777d7fb9. weed-vector builds with no system protoc from inside
seaweed-vector/ (seaweedfs#12317), checked in a clean rust:1.94.1-bookworm
container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
Requests signed for s3vectors have at most 10 MiB of their body hashed for
the signature check, so a larger body fails authentication. Checked with
19.6, 25.4 and 33.2 MiB PutVectors bodies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
