Files
seaweedfs/weed/s3api/iceberg/iceberg_issue_9103_test.go
T
Chris Lu 7522e17b6d iceberg: vend table-scoped credentials to clients that ask for delegation (#10777)
* iceberg: vend table-scoped credentials to clients that ask for delegation

The catalog recognised X-Iceberg-Access-Delegation: vended-credentials
and then deliberately said nothing, because it had nothing to vend: it
withheld even the S3 endpoint so the client would keep the credentials it
was configured with. That left every engine expecting the catalog to hand
out access - Snowflake, Databricks, Trino with vending, any multi-tenant
setup - needing static S3 keys distributed out of band.

Mint an STS session per request instead, scoped by a session policy to
the table's own prefix plus the bucket listing needed to resolve it, and
return it in the load response config and storage-credentials. The role
to assume is named by -s3.iceberg.credentialRole; its trust policy is
what decides whether a caller may assume it, and vending stays off until
it is set. A failed mint falls back to the old silence rather than
handing back an endpoint the client cannot sign for.

* iceberg: keep vended credentials inside the table prefix

Review follow-ups on credential vending:

Listing was granted on the bucket ARN with no condition, so a credential
vended for one table could enumerate every other table's object names.
Constrain s3:prefix to the table's own prefix, which the S3 gateway
already populates for list requests.

A table location carrying * or ? would have gone into the policy's
resource pattern unescaped and widened the session to sibling prefixes.
Refuse to vend for such a location rather than escaping it; nothing the
catalog generates contains those characters.

DurationSeconds skipped the 900..43200 bounds the other assume-role paths
enforce, so -s3.iceberg.credentialDurationSeconds could ask for a session
outside them. The check is now shared by all three entry points.

* iceberg: return the vended credentials from buildFileIOConfig itself

buildStorageConfig was a second name for what buildFileIOConfig already
did; it now returns the storage credentials alongside the properties, and
callers that only want the properties drop them.

* iceberg: split the vended bucket grants, and refuse a whole-bucket scope

The prefix condition sat on a statement that also granted
GetBucketLocation and ListBucketMultipartUploads, neither of which carries
an s3:prefix to satisfy it, so both were denied for every vended
credential. GetBucketLocation moves to its own unconditioned statement.
ListBucketMultipartUploads is dropped: Iceberg writers complete and abort
by upload id, and granting it either leaks in-flight keys bucket-wide or
breaks on the same missing prefix.

A table whose location has no prefix - one registered at the bucket root -
would have been vended read and write over every other table in the
bucket. Refuse, the way a location with wildcards is refused.
2026-08-16 12:57:12 -07:00

106 lines
3.2 KiB
Go

package iceberg
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestGetBucketFromPrefix_WarehouseQueryFallback(t *testing.T) {
tests := []struct {
name string
url string
want string
}{
{
name: "warehouse query routes to its bucket when no prefix in path",
url: "/v1/namespaces?warehouse=s3%3A%2F%2Fmyblkt%2F",
want: "myblkt",
},
{
name: "warehouse query with sub-path still picks the bucket",
url: "/v1/namespaces?warehouse=s3%3A%2F%2Fanother%2Fextra",
want: "another",
},
{
name: "bare bucket name is taken as the table bucket",
url: "/v1/namespaces?warehouse=not-a-url",
want: "not-a-url",
},
{
name: "table bucket ARN routes to its bucket",
url: "/v1/namespaces?warehouse=arn%3Aaws%3As3tables%3Aus-east-1%3Aadmin%3Abucket%2Fseaweed-iceberg",
want: "seaweed-iceberg",
},
{
name: "unusable warehouse value falls through to default",
url: "/v1/namespaces?warehouse=file%3A%2F%2F%2Ftmp%2Fwh",
want: "warehouse",
},
{
name: "no warehouse and no prefix returns default",
url: "/v1/namespaces",
want: "warehouse",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest("GET", tc.url, nil)
got := getBucketFromPrefix(r)
if got != tc.want {
t.Fatalf("getBucketFromPrefix(%q) = %q, want %q", tc.url, got, tc.want)
}
})
}
}
func TestBuildFileIOConfig(t *testing.T) {
loadTable := func() *http.Request {
return httptest.NewRequest(http.MethodGet, "/v1/namespaces/ns/tables/t", nil)
}
t.Run("no endpoint configured yields empty config", func(t *testing.T) {
s := &Server{}
got, _ := s.buildFileIOConfig(loadTable(), "s3://warehouse/ns/t")
if len(got) != 0 {
t.Fatalf("buildFileIOConfig() = %v, want empty", got)
}
})
t.Run("endpoint is advertised with path-style-access and region", func(t *testing.T) {
s := &Server{s3Endpoint: "http://seaweed.example:8333"}
got, _ := s.buildFileIOConfig(loadTable(), "s3://warehouse/ns/t")
if got["s3.endpoint"] != "http://seaweed.example:8333" {
t.Fatalf("s3.endpoint = %q, want %q", got["s3.endpoint"], "http://seaweed.example:8333")
}
if got["s3.path-style-access"] != "true" {
t.Fatalf("s3.path-style-access = %q, want %q", got["s3.path-style-access"], "true")
}
if got["s3.region"] == "" {
t.Fatalf("s3.region was empty, want a non-empty default so clients like DuckDB do not require AWS_REGION")
}
})
}
func TestGetBucketFromPrefix_TableBucketEnvFallback(t *testing.T) {
r := httptest.NewRequest("GET", "/v1/namespaces", nil)
t.Setenv("S3_TABLE_BUCKET", " ,analytics, other")
if got := getBucketFromPrefix(r); got != "analytics" {
t.Fatalf("first S3_TABLE_BUCKET entry: got %q, want analytics", got)
}
// The explicit default still wins over the table bucket list.
t.Setenv("S3TABLES_DEFAULT_BUCKET", "explicit")
if got := getBucketFromPrefix(r); got != "explicit" {
t.Fatalf("S3TABLES_DEFAULT_BUCKET override: got %q, want explicit", got)
}
// A prefixless value with neither env set falls through to the default.
t.Setenv("S3TABLES_DEFAULT_BUCKET", "")
t.Setenv("S3_TABLE_BUCKET", " , ")
if got := getBucketFromPrefix(r); got != "warehouse" {
t.Fatalf("empty specs: got %q, want warehouse", got)
}
}