* iceberg: make a table commit a compare-and-swap
The catalog validated the caller's version token, ran its authorization
checks, and only then wrote the new metadata xattr. Two engines
committing against the same base both passed that check and both wrote,
so the second silently dropped the first one's snapshot. Both also derive
the same v{N}.metadata.json name and the file write overwrote, leaving
the surviving pointer aimed at the loser's metadata - and the loser's
conflict cleanup then deleted the winner's file.
Write the metadata file with an exclusive create and update the xattr
conditionally on the bytes the handler read, the way the maintenance
worker already commits. A writer that lost the race re-reads and retries,
and reports 409 CommitFailedException once out of attempts.
* iceberg: stage a commit under a unique name when the versioned one is taken
Two follow-ups from review of the commit compare-and-swap:
Refusing to overwrite v{N}.metadata.json also refused to get past a file
left behind by a commit that died between staging and updating the
pointer. Every later commit derived the same name, saw the collision, and
reported a conflict, so the table stayed uncommittable until an orphan
sweep removed the file. Stage under v{N}-{uuid} instead: neither writer's
file is overwritten and the catalog pointer still decides who won, which
is how the maintenance worker has always staged its own metadata.
metadataVersionFromLocation learned to read the version back out of that
name.
The conditional update guarded only the metadata attribute while the
write replaced the whole entry, so a policy or tag written in the same
window was silently reverted. Guard every catalog attribute, which turns
that into a conflict the caller retries on fresh state.
* iceberg: give saveMetadataFile the exclusive flag instead of a second name
saveNewMetadataFile, saveMetadataBlobExclusive and uniqueMetadataFileName
were three new names around one existing helper. The flag now rides on
saveMetadataFile and saveMetadataBlob, and the unique-name construction
sits where it is used.
* iceberg: reuse the filer CAS helpers #10773 added, and stage transactions exclusively
#10773 landed mutateEntryExtended, which already writes an entry back under a
whole-entry precondition and retries. Drop the helper this branch added and
route the table commit through it: the check that the metadata is still the
one this request read now lives in the mutation, where it sees current state.
The policy the request was authorized against is asserted too, so an
administrator restricting it mid-commit sends the caller back through
authorization instead of having a stale decision applied. Bucket and
namespace policies live on other entries and a single-entry precondition
cannot cover them.
Multi-table transactions stage their metadata exclusively for the same
reason single-table commits do, and carry the name they landed on into the
pointer flip.
* s3api/iceberg: report the reason a table schema was rejected
newTableMetadata swallowed the iceberg-go error and returned nil, so every
schema the metadata builder refused came back as a bare 500 "Failed to build
table metadata". A v3-only column type is the common case: creating a table
with a variant field but no format-version 3 property leaves the client with
nothing, while "variant is not supported until v3" sits in the server log.
Return the error instead and classify it. Schema, spec and argument failures
are the caller's input, so they answer 400 with the underlying reason; the
rest stay 500. Paths that build placeholder metadata with no schema keep
their existing 500 via newEmptyTableMetadata.
* s3api/iceberg: fail LoadTable when placeholder metadata cannot be built
buildLoadTableResult dropped a nil from the placeholder path straight into
the response. That serializes as "metadata":null under HTTP 200, which no
Iceberg client can parse -- a worse outcome than the 500 the nil was meant
to signal.
Return an error instead and let the five callers answer 500. The nil-return
convention goes away with it, so the commit and transaction paths check an
error rather than a sentinel.
* s3api/iceberg: route rejected schemas through writeManagerError
The two helpers added here duplicated work the package already does.
writeManagerError is the canonical error-to-response mapper -- it already
downgrades client-input failures to 400 and defaults the rest to 500 -- so
teach it the iceberg-go schema and spec sentinels instead of standing up a
parallel classifier. The placeholder wrapper was a pure alias for
newTableMetadata with nil arguments; call that directly.
No behavior change beyond the 500 message, which now reads err.Error()
like every other manager error rather than carrying its own prefix.
* iceberg: support multi-table transaction commit
Add handleCommitTransaction for POST /v1/transactions/commit. Validation
is atomic across all table-changes (resolve, load, evaluate every
requirement before any write); metadata writes and pointer flips are
best-effort with rollback, so this is not crash-atomic.
* iceberg: route transactions/commit with and without prefix
* iceberg: test transaction commit request decoding
* iceberg: restore full prior table state on transaction rollback
* iceberg: test transaction rollback restores full prior table state
* iceberg: only clean up metadata for rolled-back tables
Chris Lu