diff --git a/S3-Table-Bucket-Commands.md b/S3-Table-Bucket-Commands.md new file mode 100644 index 0000000..d86e9c3 --- /dev/null +++ b/S3-Table-Bucket-Commands.md @@ -0,0 +1,277 @@ +SeaweedFS supports the S3 Tables API, providing a way to manage table-based data structures with AWS S3 compatibility. This guide provides commands to test the S3 Tables features in SeaweedFS. + +## Configuration + +### Prerequisites + +For AWS CLI and curl commands, configure your environment variables: + +```bash +export S3_ENDPOINT=http://localhost:8333 +export AWS_ACCESS_KEY_ID=any +export AWS_SECRET_ACCESS_KEY=any +export AWS_DEFAULT_REGION=us-east-1 +``` + +> [!NOTE] +> The `s3tables` service is a relatively new AWS service. Ensure your AWS CLI is up to date (v2.17+ recommended). If `aws s3tables` is not available, use the `curl` commands provided below. + +--- + +## 1. Table Bucket Operations + +### Create Table Bucket +**AWS CLI:** +```bash +aws s3tables create-table-bucket --endpoint-url $S3_ENDPOINT --name my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.CreateTableBucket" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"name": "my-test-bucket"}' +``` + +--- + +### List Table Buckets +**AWS CLI:** +```bash +aws s3tables list-table-buckets --endpoint-url $S3_ENDPOINT +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.ListTableBuckets" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{}' +``` + +--- + +### Get Table Bucket +**AWS CLI:** +```bash +aws s3tables get-table-bucket --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.GetTableBucket" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}' +``` + +--- + +## 2. Namespace Operations + +### Create Namespace +**AWS CLI:** +```bash +aws s3tables create-namespace --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.CreateNamespace" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "namespace": ["my_namespace"] + }' +``` + +--- + +### List Namespaces +**AWS CLI:** +```bash +aws s3tables list-namespaces --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.ListNamespaces" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}' +``` + +--- + +## 3. Table Operations + +### Create Table (Iceberg) +**AWS CLI:** +```bash +aws s3tables create-table --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace --name my_table --format ICEBERG +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.CreateTable" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "namespace": ["my_namespace"], + "name": "my_table", + "format": "ICEBERG" + }' +``` + +--- + +### List Tables +**AWS CLI:** +```bash +aws s3tables list-tables --endpoint-url $S3_ENDPOINT --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket --namespace my_namespace +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.ListTables" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "namespace": ["my_namespace"] + }' +``` + +--- + +## 4. Policy and Tagging + +### Put Table Bucket Policy +**AWS CLI:** +```bash +aws s3tables put-table-bucket-policy --endpoint-url $S3_ENDPOINT \ + --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \ + --resource-policy "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3tables:*\",\"Resource\":\"*\"}]}" +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.PutTableBucketPolicy" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "resourcePolicy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3tables:*\",\"Resource\":\"*\"}]}" + }' +``` + +### Get Table Bucket Policy +**AWS CLI:** +```bash +aws s3tables get-table-bucket-policy --endpoint-url $S3_ENDPOINT \ + --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.GetTableBucketPolicy" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}' +``` + +### Tag Resource +**AWS CLI:** +```bash +aws s3tables tag-resource --endpoint-url $S3_ENDPOINT \ + --resource-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \ + --tags Environment=test,Owner=admin +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.TagResource" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "resourceArn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "tags": {"Environment": "test", "Owner": "admin"} + }' +``` + +### List Tags +**AWS CLI:** +```bash +aws s3tables list-tags-for-resource --endpoint-url $S3_ENDPOINT \ + --resource-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.ListTagsForResource" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"resourceArn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}' +``` + +--- + +## Cleanup + +### Delete Table +**AWS CLI:** +```bash +aws s3tables delete-table --endpoint-url $S3_ENDPOINT \ + --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \ + --namespace my_namespace \ + --name my_table +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.DeleteTable" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "namespace": ["my_namespace"], + "name": "my_table" + }' +``` + +### Delete Namespace +**AWS CLI:** +```bash +aws s3tables delete-namespace --endpoint-url $S3_ENDPOINT \ + --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket \ + --namespace my_namespace +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.DeleteNamespace" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{ + "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket", + "namespace": ["my_namespace"] + }' +``` + +### Delete Table Bucket +**AWS CLI:** +```bash +aws s3tables delete-table-bucket --endpoint-url $S3_ENDPOINT \ + --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket +``` + +**curl:** +```bash +curl -X POST $S3_ENDPOINT/ \ + -H "X-Amz-Target: S3Tables.DeleteTableBucket" \ + -H "Content-Type: application/x-amz-json-1.1" \ + -d '{"tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-test-bucket"}' +``` diff --git a/S3-Table-Bucket.md b/S3-Table-Bucket.md new file mode 100644 index 0000000..3fd6e3c --- /dev/null +++ b/S3-Table-Bucket.md @@ -0,0 +1,19 @@ +# S3 Tables + +> [!IMPORTANT] +> **Work in Progress** +> This feature is currently under development. + +## Introduction + +SeaweedFS is adding support for Amazon S3 Tables, which provides a way to manage structured datasets using the Apache Iceberg table format. S3 Tables combine the scalability of S3 with the performance and features of dedicated table formats. + +## Key Features + +- **Apache Iceberg Support**: Native support for Iceberg tables. +- **Table Buckets**: Dedicated buckets for storing your tables. +- **Namespaces**: Logical grouping of tables within a table bucket. + +## Documentation + +- [S3 Table Bucket Commands](S3-Table-Bucket-Commands) - Examples of how to manage table buckets, namespaces, and tables. diff --git a/_Sidebar.md b/_Sidebar.md index 1c7ff20..d4ba785 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -82,6 +82,9 @@ * [[S3 Nginx Proxy]] * [[Docker Compose for S3]] +### S3 Table Bucket +* [[S3 Table Bucket]] + ### S3 Authentication & IAM * [[S3 Configuration]] - Start Here * [[S3 Credentials]] (`-s3.config`)