mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-09 21:38:19 +01:00
f39b1abd0395d261663bc6656efb160339bff352
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris LuandClaude Opus 5.5
|
4f24b756ba |
s3tables, iceberg: give tables created without a location their own directory (#12475)
* s3tables: give a table created without a location its own directory An Iceberg table created without a location kept its files under its name path, so a rename left them where the next table of that name would land. Such a table now gets s3://<bucket>/<namespace>/<name>-<uuid>, stored as the record's warehouseLocation and returned by GetTable; its first commit writes there without moving the root. Records without one keep resolving to their name path, so existing tables need no rewrite. Lance datasets stay at their name path, where Lance clients and the vector engine look for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: create tables and views in a directory of their own A table or view created without a location took its name path, or a unique one only when that path was already taken. It now always gets <name>-<uuid>, so a later rename never leaves its files under a name another table can take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: load and commit a table at its warehouse location A table created through S3 Tables without a location has no metadata until its first commit; loading or committing it now uses the directory recorded for it, and only a record without one falls back to its name path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: fail a stage-create whose marker is not saved The marker is how the commit finds the location the stage-create handed out. Without it the commit fell back to the name path while the client's files sat under the returned location. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: expect tables created without a location in their own directory Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: rename a table that keeps its files elsewhere before reusing its name Enterprise keeps the storage marker on a renamed table's files, so the name of a table whose files sit under its name path stays taken there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: remove the table directory when a create backs out Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: answer a create that lost the race with the registered table Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: answer a view create that lost the race with the registered view Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: fail a staged commit whose name another create registered Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: remove the directory of a create whose name another table took Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: keep the metadata of a staged commit whose name is taken Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: write the lakekeeper table's data under its warehouse location Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: commit a staged table at the location of its own stage Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: never continue another stage when a commit's own marker is gone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: test filer honors exclusive creates and token-guarded deletes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chris LuandClaude Opus 5.5
|
5b1450b06b |
s3tables, iceberg: one catalog record per table location (#12473)
* s3tables: refuse a table or view location another record holds A create, register, update or rename could name a location at, under or above another catalog record's files, including the files a renamed table left under its old name. The new record then loaded that table's metadata, had credentials vended for its files, and could purge them on drop. Each of those writes now scans the bucket's records and refuses a root another record claims, and a drop of a record sharing its files is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: check a table's location before writing to it REST create and stage-create write a metadata file, and stage-create vends write credentials, before the catalog registers the table, so the create gate now also validates the location and refuses one another record holds. RegisterTable runs the same gate before it reads the metadata file, instead of reading first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: keep a dropped location another record holds DropTable and DropView purge the dropped record's location after the catalog delete. A view recorded at a table's location took that table's directory, record included. The purge now leaves a location another record still holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: authorize CreateTable before reporting the namespace A caller without CreateTable got 404 for a missing namespace, 403 for an existing one and 500 with decoder text for an unreadable one. The grant is now evaluated against the bucket first, and only a caller that passes learns the namespace is missing or unreadable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg maintenance: skip a table whose files another record shares Orphan removal and snapshot expiry delete what a table's metadata does not reference. For two records over one location that deletes the other's files, so detection leaves such tables alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: count another record's catalog entry as claimed A table keeping its files elsewhere still owns its entry directory, and a view recorded there would take that entry with it on drop. The ownership check now refuses a root at, above or under another record's entry too. Maintenance still compares files only, so a table created over a renamed table's old name does not stop that table being maintained. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: check a claimed location again once the record is written Two creates, or a create and an update moving a root, could both read the same root free and both land. A create now checks again after writing its still-marked record and backs out if another record claims the root, and an update that moved its root checks again after the write and puts the previous metadata back. Of two racing writers at least one sees the other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: enable stage-create in the location ownership test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg maintenance: check for a shared location again before removing orphans A job can run long after its table was scheduled, or come from a detector that never checked, so remove_orphans reads the bucket's records itself and leaves the files alone when another record shares the table's location. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: check a renamed record's new root again once it is written A record without a location moves its root with its name, and was checked only before the rename wrote the destination, so a create landing on that root in between could share it. The rename now checks again with the destination written and backs out, like a create. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: put a moved record back when its location cannot be checked again An update that moved its root kept the move when the check after the write failed to list the bucket, so a racing writer that relied on that check could share the root. The move is now undone as it is when the root is taken. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: hold a moved record's old root until the move lands An update moving a record's root wrote the new metadata first and put the old metadata back when the check that followed found the new root taken. The old root was free in between, so a create racing for it could pass its own check and then share it with the restored record. The update now stores the metadata it is about to write beside the current one, so the record holds both roots, checks the new root with that reservation in place, and only then switches the metadata. A move that backs out drops the reservation without touching the metadata, and an update that keeps its root clears a reservation a crashed move left behind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: test a move racing another update of the record Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: drop a move's reservation when a rename or delete takes the record A rename or a decoupled delete that took a record while an update held its move reservation left the reservation on the old name, where the move could no longer clear it, and later creates of that name were refused as taken. Both now clear it with the record's other attributes, and a failed rename does not put it back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: register a table from metadata under its own location The Spark register test registered a live table's metadata file under a second name, which now fails because both records would own the source's files. It checks that refusal instead, and registers a copy uploaded under the new table's own directory, with its location pointing there, to cover reading and writing a registered table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chris Lu
|
28862c866e |
Authorize an Iceberg table create before it writes (#10991)
* s3tables: share one CreateTable authorization gate CreateTable and RegisterTable each carried their own copy of the name validation, policy load and permission check. Fold them into authorizeCreateTable, and expose it on the Manager for callers that write into a table bucket before the table itself is registered. Claude-Session: https://claude.ai/code/session_01QiJkka1T2NAWDWq4JQ8Vuy * iceberg: authorize a table create before it writes Stage-create returns before the S3Tables registration that authorizes a create, and the plain create writes its metadata file before reaching it, so a caller who may not create the table could still leave a staged template, a marker and a v1.metadata.json in the target bucket - and get vended credentials for a location of their choosing. Run the CreateTable gate as soon as the table is known to be absent. Claude-Session: https://claude.ai/code/session_01QiJkka1T2NAWDWq4JQ8Vuy * iceberg: authorize a create-on-commit the same way A commit against a table that does not exist creates it, writing the metadata file first and only then reaching the registration that checks the caller may create it. Denied callers saw a 500 for what is a 403. Claude-Session: https://claude.ai/code/session_01QiJkka1T2NAWDWq4JQ8Vuy * iceberg: pin that identity actions reach the create gate The manager request is built from the caller's own context, so an identity whose actions carry the permission still passes. Worth a test: a fresh context here would silently deny every such caller. Claude-Session: https://claude.ai/code/session_01QiJkka1T2NAWDWq4JQ8Vuy |
||
|
Chris Lu
|
0403e47ef6 |
iceberg: support views (#10069)
* s3tables: tag table entries and exclude views from table listings * s3tables: add view CRUD operations * iceberg: support view create, load, exists, drop, and list * iceberg: support view update * iceberg: test view error classification and metadata round-trip * iceberg: pre-check existence and write view metadata only after create * iceberg: map view namespace-not-found to 404 * iceberg: test view create namespace-404 and duplicate no-clobber * s3tables: tag view metadata and entry type atomically CreateView wrote ExtendedKeyMetadata and ExtendedKeyEntryType in two UpdateEntry calls, so a partial failure could leave a view directory untagged. Add setExtendedAttributes to set both in one UpdateEntry. * iceberg: roll back view registration when metadata write fails The metadata file is written after the catalog registers the view. If that write fails, drop the just-created view so it doesn't linger pointing at a missing metadata.json. Reuse the DeleteView path via a shared dropView helper. |