adding table bucket

Chris Lu committed 2026-01-30 11:33:33 -08:00
1 parent f9907f4326
commit 645c4cf45b
3 files changed
+299

No files matched your search

+277
@@ -0,0 +1,277 @@
SeaweedFS supports the S3 Tables API, providing a way to manage table-based data structures with AWS S3 compatibility. This guide provides commands to test the S3 Tables features in SeaweedFS.
## Configuration
### Prerequisites
For AWS CLI and curl commands, configure your environment variables:
```bash
export S3_ENDPOINT=http://localhost:8333
export AWS_ACCESS_KEY_ID=any
export AWS_SECRET_ACCESS_KEY=any
export AWS_DEFAULT_REGION=us-east-1
```
> [!NOTE]
> The `s3tables` service is a relatively new AWS service. Ensure your AWS CLI is up to date (v2.17+ recommended). If `aws s3tables` is not available, use the `curl` commands provided below.
---
## 1. Table Bucket Operations
### Create Table Bucket
**AWS CLI:**
```bash
aws s3tables create-table-bucket --endpoint-url $S3_ENDPOINT --name my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.CreateTableBucket" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"name": "my-test-bucket"}'
```
---
### List Table Buckets
**AWS CLI:**
```bash
aws s3tables list-table-buckets --endpoint-url $S3_ENDPOINT
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.ListTableBuckets" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{}'
```
---
### Get Table Bucket
**AWS CLI:**
```bash
aws s3tables get-table-bucket --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.GetTableBucket" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}'
```
---
## 2. Namespace Operations
### Create Namespace
**AWS CLI:**
```bash
aws s3tables create-namespace --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.CreateNamespace" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"namespace": ["my_namespace"]
}'
```
---
### List Namespaces
**AWS CLI:**
```bash
aws s3tables list-namespaces --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.ListNamespaces" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}'
```
---
## 3. Table Operations
### Create Table (Iceberg)
**AWS CLI:**
```bash
aws s3tables create-table --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace --name my_table --format ICEBERG
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.CreateTable" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"namespace": ["my_namespace"],
"name": "my_table",
"format": "ICEBERG"
}'
```
---
### List Tables
**AWS CLI:**
```bash
aws s3tables list-tables --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.ListTables" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"namespace": ["my_namespace"]
}'
```
---
## 4. Policy and Tagging
### Put Table Bucket Policy
**AWS CLI:**
```bash
aws s3tables put-table-bucket-policy --endpoint-url $S3_ENDPOINT \
--table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \
--resource-policy "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3tables:*\",\"Resource\":\"*\"}]}"
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.PutTableBucketPolicy" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"resourcePolicy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3tables:*\",\"Resource\":\"*\"}]}"
}'
```
### Get Table Bucket Policy
**AWS CLI:**
```bash
aws s3tables get-table-bucket-policy --endpoint-url $S3_ENDPOINT \
--table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.GetTableBucketPolicy" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}'
```
### Tag Resource
**AWS CLI:**
```bash
aws s3tables tag-resource --endpoint-url $S3_ENDPOINT \
--resource-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \
--tags Environment=test,Owner=admin
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.TagResource" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"resourceArn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"tags": {"Environment": "test", "Owner": "admin"}
}'
```
### List Tags
**AWS CLI:**
```bash
aws s3tables list-tags-for-resource --endpoint-url $S3_ENDPOINT \
--resource-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.ListTagsForResource" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"resourceArn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}'
```
---
## Cleanup
### Delete Table
**AWS CLI:**
```bash
aws s3tables delete-table --endpoint-url $S3_ENDPOINT \
--table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \
--namespace my_namespace \
--name my_table
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.DeleteTable" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"namespace": ["my_namespace"],
"name": "my_table"
}'
```
### Delete Namespace
**AWS CLI:**
```bash
aws s3tables delete-namespace --endpoint-url $S3_ENDPOINT \
--table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \
--namespace my_namespace
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.DeleteNamespace" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{
"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket",
"namespace": ["my_namespace"]
}'
```
### Delete Table Bucket
**AWS CLI:**
```bash
aws s3tables delete-table-bucket --endpoint-url $S3_ENDPOINT \
--table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket
```
**curl:**
```bash
curl -X POST $S3_ENDPOINT/ \
-H "X-Amz-Target: S3Tables.DeleteTableBucket" \
-H "Content-Type: application/x-amz-json-1.1" \
-d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}'
```
+19
@@ -0,0 +1,19 @@
# S3 Tables
> [!IMPORTANT]
> **Work in Progress**
> This feature is currently under development.
## Introduction
SeaweedFS is adding support for Amazon S3 Tables, which provides a way to manage structured datasets using the Apache Iceberg table format. S3 Tables combine the scalability of S3 with the performance and features of dedicated table formats.
## Key Features
- **Apache Iceberg Support**: Native support for Iceberg tables.
- **Table Buckets**: Dedicated buckets for storing your tables.
- **Namespaces**: Logical grouping of tables within a table bucket.
## Documentation
- [S3 Table Bucket Commands](S3-Table-Bucket-Commands) - Examples of how to manage table buckets, namespaces, and tables.
+3
@@ -82,6 +82,9 @@
* [[S3 Nginx Proxy]]
* [[Docker Compose for S3]]
### S3 Table Bucket
* [[S3 Table Bucket]]
### S3 Authentication & IAM
* [[S3 Configuration]] - Start Here
* [[S3 Credentials]] (`-s3.config`)