From b1a4bb089dac592153c183383ddaa13965bf8dde Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Tue, 23 Jun 2026 11:35:10 -0700 Subject: [PATCH] Add Iceberg REST Catalog API support matrix page --- Iceberg-REST-Catalog-API.md | 73 ++++++++++++++++++++++++++++++++++++ SeaweedFS-Iceberg-Catalog.md | 1 + _Sidebar.md | 1 + 3 files changed, 75 insertions(+) create mode 100644 Iceberg-REST-Catalog-API.md diff --git a/Iceberg-REST-Catalog-API.md b/Iceberg-REST-Catalog-API.md new file mode 100644 index 0000000..fecdfd2 --- /dev/null +++ b/Iceberg-REST-Catalog-API.md @@ -0,0 +1,73 @@ +SeaweedFS implements the [Iceberg REST Catalog API](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml), so any Iceberg REST client (Spark, Trino, PyIceberg, iceberg-go, DuckDB, Dremio, RisingWave, Doris) can manage namespaces, tables, and views directly against SeaweedFS while data and metadata files live in SeaweedFS S3 buckets. + +For setup, authentication, and client integrations, see [[SeaweedFS Iceberg Catalog]]. + +# Endpoint + +The catalog listens on its own port (default `8181`, set with `-s3.port.iceberg`). A catalog is a table bucket; address it either by URL prefix (`/v1/{bucket}/...`) or with `?warehouse=s3://{bucket}/`. With neither, requests default to the bucket named `warehouse`. See [[SeaweedFS Iceberg Catalog]] for the bucket relationship. + +# Supported APIs + +The tables below list the Iceberg REST Catalog operations and their support status, using the operation names from the REST catalog OpenAPI spec. + +## Configuration and OAuth + +| API Operation | Method & Path | Supported | Notes | +|---|---|---|---| +| GetConfig | `GET /v1/config` | Yes | Echoes `overrides.prefix` for a `?warehouse=` query | +| GetToken | `POST /v1/oauth/tokens` | Yes | `client_credentials` grant; S3 access/secret key as client id/secret | + +## Namespace + +| API Operation | Method & Path | Supported | Notes | +|---|---|---|---| +| ListNamespaces | `GET /v1/{prefix}/namespaces` | Yes | `parent` and pagination supported | +| CreateNamespace | `POST /v1/{prefix}/namespaces` | Yes | | +| LoadNamespaceMetadata | `GET /v1/{prefix}/namespaces/{ns}` | Yes | | +| NamespaceExists | `HEAD /v1/{prefix}/namespaces/{ns}` | Yes | | +| DropNamespace | `DELETE /v1/{prefix}/namespaces/{ns}` | Yes | Rejects a non-empty namespace | +| UpdateNamespaceProperties | `POST /v1/{prefix}/namespaces/{ns}/properties` | Yes | Returns the removed/updated/missing summary | + +## Table + +| API Operation | Method & Path | Supported | Notes | +|---|---|---|---| +| ListTables | `GET /v1/{prefix}/namespaces/{ns}/tables` | Yes | | +| CreateTable | `POST /v1/{prefix}/namespaces/{ns}/tables` | Yes | Includes stage-create | +| LoadTable | `GET /v1/{prefix}/namespaces/{ns}/tables/{table}` | Yes | Vends S3 FileIO config so clients read data files directly | +| TableExists | `HEAD /v1/{prefix}/namespaces/{ns}/tables/{table}` | Yes | | +| UpdateTable | `POST /v1/{prefix}/namespaces/{ns}/tables/{table}` | Yes | Commit with assert-* requirements | +| DropTable | `DELETE /v1/{prefix}/namespaces/{ns}/tables/{table}` | Yes | | +| RegisterTable | `POST /v1/{prefix}/namespaces/{ns}/register` | Yes | Registers an existing `metadata.json` location | +| RenameTable | `POST /v1/{prefix}/tables/rename` | Yes | Catalog-only; data and metadata files do not move | +| ReportMetrics | `POST /v1/{prefix}/namespaces/{ns}/tables/{table}/metrics` | No | Scan/commit metrics are not collected | +| LoadCredentials | `GET /v1/{prefix}/namespaces/{ns}/tables/{table}/credentials` | No | Use the vended FileIO config / S3 keys | +| PlanTableScan / FetchScanTasks | `.../tables/{table}/plan` | No | Server-side scan planning | + +## View + +| API Operation | Method & Path | Supported | Notes | +|---|---|---|---| +| ListViews | `GET /v1/{prefix}/namespaces/{ns}/views` | Yes | | +| CreateView | `POST /v1/{prefix}/namespaces/{ns}/views` | Yes | | +| LoadView | `GET /v1/{prefix}/namespaces/{ns}/views/{view}` | Yes | | +| ViewExists | `HEAD /v1/{prefix}/namespaces/{ns}/views/{view}` | Yes | | +| ReplaceView | `POST /v1/{prefix}/namespaces/{ns}/views/{view}` | Yes | Commit with assert-view-uuid requirements | +| DropView | `DELETE /v1/{prefix}/namespaces/{ns}/views/{view}` | Yes | | + +## Transaction + +| API Operation | Method & Path | Supported | Notes | +|---|---|---|---| +| CommitTransaction | `POST /v1/{prefix}/transactions/commit` | Yes | Multi-table commit; requirements validated together, applied best-effort (not crash-atomic) | + +# Notes + +- **Naming**: namespace and table/view names use the S3 Tables charset — lowercase `a-z`, `0-9`, and `_`. Names with other characters (hyphens, uppercase) are rejected with `400 BadRequestException`. +- **Table and view names share a namespace**: a name is unique across both within a namespace. +- **Metadata storage**: the catalog entry is a pointer; the Iceberg `metadata.json` is an object at `s3://{bucket}/{ns}/{table}/metadata/vN.metadata.json`. A commit writes a new metadata file and flips the pointer. + +# See Also + +- [[SeaweedFS Iceberg Catalog]] +- [[Iceberg Table Maintenance]] diff --git a/SeaweedFS-Iceberg-Catalog.md b/SeaweedFS-Iceberg-Catalog.md index 0d26079..57a690f 100644 --- a/SeaweedFS-Iceberg-Catalog.md +++ b/SeaweedFS-Iceberg-Catalog.md @@ -119,6 +119,7 @@ If SeaweedFS is running without IAM configuration (e.g., `weed mini` with no `-s ## See Also +- [[Iceberg REST Catalog API]] - Supported REST Catalog operations - [[S3 Table Bucket]] - Creating and managing table buckets - [[S3 Tables Security]] - IAM policies for table access - [[S3 Table Bucket Commands]] - `weed shell` commands diff --git a/_Sidebar.md b/_Sidebar.md index dbb908d..4d073b0 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -106,6 +106,7 @@ * [[S3 Table Bucket Commands]] * [[S3 Tables Security]] * [[SeaweedFS Iceberg Catalog]] +* [[Iceberg REST Catalog API]] * [[Iceberg Table Maintenance]] ### Iceberg Integrations