* iceberg: vend table-scoped credentials to clients that ask for delegation
The catalog recognised X-Iceberg-Access-Delegation: vended-credentials
and then deliberately said nothing, because it had nothing to vend: it
withheld even the S3 endpoint so the client would keep the credentials it
was configured with. That left every engine expecting the catalog to hand
out access - Snowflake, Databricks, Trino with vending, any multi-tenant
setup - needing static S3 keys distributed out of band.
Mint an STS session per request instead, scoped by a session policy to
the table's own prefix plus the bucket listing needed to resolve it, and
return it in the load response config and storage-credentials. The role
to assume is named by -s3.iceberg.credentialRole; its trust policy is
what decides whether a caller may assume it, and vending stays off until
it is set. A failed mint falls back to the old silence rather than
handing back an endpoint the client cannot sign for.
* iceberg: keep vended credentials inside the table prefix
Review follow-ups on credential vending:
Listing was granted on the bucket ARN with no condition, so a credential
vended for one table could enumerate every other table's object names.
Constrain s3:prefix to the table's own prefix, which the S3 gateway
already populates for list requests.
A table location carrying * or ? would have gone into the policy's
resource pattern unescaped and widened the session to sibling prefixes.
Refuse to vend for such a location rather than escaping it; nothing the
catalog generates contains those characters.
DurationSeconds skipped the 900..43200 bounds the other assume-role paths
enforce, so -s3.iceberg.credentialDurationSeconds could ask for a session
outside them. The check is now shared by all three entry points.
* iceberg: return the vended credentials from buildFileIOConfig itself
buildStorageConfig was a second name for what buildFileIOConfig already
did; it now returns the storage credentials alongside the properties, and
callers that only want the properties drop them.
* iceberg: split the vended bucket grants, and refuse a whole-bucket scope
The prefix condition sat on a statement that also granted
GetBucketLocation and ListBucketMultipartUploads, neither of which carries
an s3:prefix to satisfy it, so both were denied for every vended
credential. GetBucketLocation moves to its own unconditioned statement.
ListBucketMultipartUploads is dropped: Iceberg writers complete and abort
by upload id, and granting it either leaks in-flight keys bucket-wide or
breaks on the same missing prefix.
A table whose location has no prefix - one registered at the bucket root -
would have been vended read and write over every other table in the
bucket. Refuse, the way a location with wildcards is refused.
* iceberg: withhold the S3 endpoint from credential-vending clients
A client that sends X-Iceberg-Access-Delegation: vended-credentials builds
its storage credential out of the LoadTable config and drops the one it was
configured with. We vend no credentials, so the endpoint we advertised left
DuckDB signing nothing: every metadata and data file came back 403, and its
attempt to refresh the empty credential 404ed on stage-created tables.
Answer those clients with no config at all so they keep their own
credentials. Clients that do not ask for delegation still get the endpoint.
* iceberg: mark load responses as varying on the delegation header
The FileIO config in a table or view load response now depends on whether
the client asked for vended credentials, so a cache between us and the
client must key on that header rather than on the URL alone.
* test: cover the DuckDB vended-credentials access pattern
Runs weed mini with -s3.externalUrl, which is what makes the catalog
advertise an endpoint at all, and checks both halves: a plain LoadTable
still gets the endpoint, while one asking for vended credentials never gets
an endpoint without the credentials to sign with. The DuckDB round trip
creates a table from a query and reads it back, which is the flow that
failed with 403 on every data file.
Chris Lu