S3 Vector Bucket: vertical diagram with the three bucket kinds

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

S3-Vector-Bucket.md

@@ -9,45 +9,45 @@ SeaweedFS serves the [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/la
 ## How it works
 
 ```mermaid
-flowchart LR
-    subgraph path ["request and data path"]
-        direction TB
-        client["<b>Clients</b><br/>AWS CLI, boto3, any AWS SDK"]
-        vapi["<b>S3 gateway: S3 Vectors API</b><br/>port 8333, signed for s3vectors"]
-        engines["<b>weed-vector</b><br/>gRPC port 8090"]
-        sapi["<b>S3 gateway: Lance catalog, S3 API</b><br/>ports 9101 and 8333"]
-        filers[("<b>filers</b><br/>bucket, policy and table<br/>records, file metadata")]
+---
+config:
+  flowchart:
+    wrappingWidth: 420
+    nodeSpacing: 50
+    rankSpacing: 50
+---
+flowchart TB
+    client["<b>Clients</b><br/>AWS CLI, boto3, any AWS SDK"]
+    gateway["<b>S3 gateway</b><br/>S3, S3 Tables and S3 Vectors APIs on the S3 port"]
+    subgraph buckets ["bucket kinds"]
+        direction LR
+        normal["<b>normal bucket</b><br/>objects"]
+        table["<b>table bucket</b><br/>Iceberg or Lance tables"]
+        vector["<b>vector bucket</b><br/>a table bucket with format LANCE"]
+    end
+    engine["<b>weed-vector</b><br/>exact search over the index"]
+    subgraph store ["stored in SeaweedFS, through the gateway's S3 API"]
+        direction LR
+        filers[("<b>filers</b><br/>file metadata")]
         volumes[("<b>volume servers</b><br/>file chunks")]
-        client -->|"S3 Vectors requests"| vapi
-        vapi -->|"index and vector operations"| engines
-        engines -->|"index tables and Lance files"| sapi
-        sapi -->|"file metadata"| filers
-        sapi -->|"file chunks"| volumes
     end
-    subgraph ent ["SeaweedFS Enterprise: distributed queries"]
-        direction TB
-        workers["<b>weed-vector-worker</b> engines<br/>gRPC port 18888, one share of each index"]
-    end
-    vapi -.->|"QueryVectors, one request per engine"| workers
-    workers -.->|"file metadata, read directly"| filers
-    workers -.->|"file chunks, read directly"| volumes
-    subgraph side ["cluster services"]
-        master["<b>master</b>, port 9333<br/>engines register here,<br/>the gateway looks them up"]
-    end
-    path -.- side
+    client -->|"S3, S3 Tables and S3 Vectors requests"| gateway
+    gateway --> normal
+    gateway --> table
+    gateway --> vector
+    vector -->|"index and vector operations"| engine
+    engine -->|"table and file metadata"| filers
+    engine -->|"file chunks"| volumes
 
     classDef s3server fill:#dbeafe,stroke:#2563eb,color:#0f172a
     classDef other fill:#f1f5f9,stroke:#64748b,color:#0f172a
-    classDef enterprise fill:#fef3c7,stroke:#d97706,color:#0f172a
-    class vapi,sapi s3server
-    class client,engines,master,filers,volumes other
-    class workers enterprise
-    style path fill:none,stroke:#94a3b8
-    style ent fill:none,stroke:#d97706,stroke-dasharray:4 3
-    style side fill:none,stroke:#94a3b8
+    class gateway s3server
+    class client,normal,table,vector,engine,filers,volumes other
+    style buckets fill:none,stroke:#94a3b8
+    style store fill:none,stroke:#94a3b8
 ```
 
-Both blue boxes run in the S3 server process: `weed-vector` calls back into an S3 server for the index data, and that S3 server reads the file metadata from the filers and the chunks from the volume servers like any other S3 request. Each index is one Lance dataset under `s3://<bucket>/vector/<index>/`, stored as file chunks on the volume servers. The dashed path is SeaweedFS Enterprise: its search engines (`weed-vector-worker`) answer `QueryVectors` and read the same Lance files straight from the filers and volume servers, skipping the gateway; writes and index management still go through `weed-vector`.
+The gateway serves all three bucket kinds on the S3 port. A vector bucket is a table bucket with format `LANCE`; each index is one Lance dataset under `s3://<bucket>/vector/<index>/`. `weed-vector` reads and writes it through the gateway's S3 API, and the gateway, as for any S3 request, keeps the file metadata on the filers and the chunks on the volume servers.
 
 * **The S3 gateway** (`weed s3`, or the S3 server inside `weed server -s3` and `weed mini`) answers S3 Vectors requests on the normal S3 port. It tells them apart by the service name in the SigV4 credential scope: a request signed for `s3vectors` goes to the vectors API, everything else to the S3 API.
 * **The vector engine** is a separate process, `weed-vector`. It registers with the master, and the gateway finds it through the master on the first request that needs it. Bucket and policy operations work without it; index and vector operations need it. Several can register: the open-source gateway sends its requests to one at a time, and SeaweedFS Enterprise spreads each index and its queries over them (see [[S3 Vector Bucket Performance#beyond-one-engine]]).