2 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandClaude Opus 5.5	4f24b756ba	s3tables, iceberg: give tables created without a location their own directory (#12475) ... * s3tables: give a table created without a location its own directory An Iceberg table created without a location kept its files under its name path, so a rename left them where the next table of that name would land. Such a table now gets s3://<bucket>/<namespace>/<name>-<uuid>, stored as the record's warehouseLocation and returned by GetTable; its first commit writes there without moving the root. Records without one keep resolving to their name path, so existing tables need no rewrite. Lance datasets stay at their name path, where Lance clients and the vector engine look for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: create tables and views in a directory of their own A table or view created without a location took its name path, or a unique one only when that path was already taken. It now always gets <name>-<uuid>, so a later rename never leaves its files under a name another table can take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: load and commit a table at its warehouse location A table created through S3 Tables without a location has no metadata until its first commit; loading or committing it now uses the directory recorded for it, and only a record without one falls back to its name path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: fail a stage-create whose marker is not saved The marker is how the commit finds the location the stage-create handed out. Without it the commit fell back to the name path while the client's files sat under the returned location. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: expect tables created without a location in their own directory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: rename a table that keeps its files elsewhere before reusing its name Enterprise keeps the storage marker on a renamed table's files, so the name of a table whose files sit under its name path stays taken there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: remove the table directory when a create backs out Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: answer a create that lost the race with the registered table Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: answer a view create that lost the race with the registered view Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: fail a staged commit whose name another create registered Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: remove the directory of a create whose name another table took Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: keep the metadata of a staged commit whose name is taken Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: write the lakekeeper table's data under its warehouse location Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: commit a staged table at the location of its own stage Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: never continue another stage when a commit's own marker is gone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: test filer honors exclusive creates and token-guarded deletes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-07 16:37:31 -08:00
Nguyễn Lộc Phúc	195f8156de	fix(iceberg): keep a losing table create from deleting the winner's metadata (#12474) ... * iceberg: keep a losing table create from deleting the winner's metadata * iceberg: keep a staged file whose published record was renamed away A create answers a conflict when its record was already visible and another request changed it before the create could back it out. The cleanup then looked only at the original name, so a record renamed in between read as absent and the file it still points at was deleted. S3 Tables now reports that case with a conflict of its own, and both create paths keep the staged file on it instead of probing the name. A conflict over a still-hidden record, lost to another create, still removes the file. * iceberg: discard a losing create's metadata only while it is still its own The cleanup deleted the staged file by name. The path can be reused before it runs: the table is dropped and purged, and another create stages the same v1.metadata.json and registers it, which the losing request then deleted. Staged files now carry a per-request token in an internal extended attribute, and the cleanup deletes with that token as the filer's ExpectedExtended precondition, checked under the path lock the delete runs under. A file that took the path since is left in place. * iceberg: classify a typed S3 Tables error by its code, not its message isS3TablesAlreadyExists matched "already exists" in the message before looking at the code, so an InternalError whose text mentioned an existing entry read as the name being taken: CreateTable answered 409 instead of 500, and registrationRefused discarded a staged file the catalog may still point at. A typed error now decides by its code; only an untyped one is matched by message. * iceberg: pin stage-create on in the stage-create conflict test Stage-create is on by default, but a shell that sets ICEBERG_ENABLE_STAGE_CREATE=false turned the expected 409 into a 501. Setting it in the test keeps the result independent of the environment. * iceberg: test two creates overlapping at the canonical metadata name The concurrent-creators test only lines the creators up before they start, so whether two of them ever collide at v1.metadata.json is up to the scheduler. The new test fixes the order: the second create runs while the first has staged v1.metadata.json and not yet registered it. It checks that every staging attempt is exclusive, that the second falls back to v1-<uuid>, that both files hold their own metadata while both are staged, and that the loser removes only its own file. * s3tables: a create whose unmark write never went out stayed hidden confirmChild counted any failure to take a create's mark off as possibly published, the lookup that precedes the write included. That lookup failing leaves the mark on, so the record stayed hidden; if another create then adopted it, the back-out reported a published record that changed, and the Iceberg catalog kept a staged file no record names. The mark-removal calls now go through a wrapper that notes whether an UpdateEntry went out without the filer refusing it. Only then may the record have been seen. A refused compare, a missing entry, or a failure before any write leaves the create's record hidden. * s3tables: pass mayBeVisible into backOutChild Fixes the build after rebasing onto the backOutChild split.	2026-11-07 10:54:04 -08:00