mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-07 20:47:46 +01:00
Nguyễn Lộc Phúc
195f8156de
* iceberg: keep a losing table create from deleting the winner's metadata * iceberg: keep a staged file whose published record was renamed away A create answers a conflict when its record was already visible and another request changed it before the create could back it out. The cleanup then looked only at the original name, so a record renamed in between read as absent and the file it still points at was deleted. S3 Tables now reports that case with a conflict of its own, and both create paths keep the staged file on it instead of probing the name. A conflict over a still-hidden record, lost to another create, still removes the file. * iceberg: discard a losing create's metadata only while it is still its own The cleanup deleted the staged file by name. The path can be reused before it runs: the table is dropped and purged, and another create stages the same v1.metadata.json and registers it, which the losing request then deleted. Staged files now carry a per-request token in an internal extended attribute, and the cleanup deletes with that token as the filer's ExpectedExtended precondition, checked under the path lock the delete runs under. A file that took the path since is left in place. * iceberg: classify a typed S3 Tables error by its code, not its message isS3TablesAlreadyExists matched "already exists" in the message before looking at the code, so an InternalError whose text mentioned an existing entry read as the name being taken: CreateTable answered 409 instead of 500, and registrationRefused discarded a staged file the catalog may still point at. A typed error now decides by its code; only an untyped one is matched by message. * iceberg: pin stage-create on in the stage-create conflict test Stage-create is on by default, but a shell that sets ICEBERG_ENABLE_STAGE_CREATE=false turned the expected 409 into a 501. Setting it in the test keeps the result independent of the environment. * iceberg: test two creates overlapping at the canonical metadata name The concurrent-creators test only lines the creators up before they start, so whether two of them ever collide at v1.metadata.json is up to the scheduler. The new test fixes the order: the second create runs while the first has staged v1.metadata.json and not yet registered it. It checks that every staging attempt is exclusive, that the second falls back to v1-<uuid>, that both files hold their own metadata while both are staged, and that the loser removes only its own file. * s3tables: a create whose unmark write never went out stayed hidden confirmChild counted any failure to take a create's mark off as possibly published, the lookup that precedes the write included. That lookup failing leaves the mark on, so the record stayed hidden; if another create then adopted it, the back-out reported a published record that changed, and the Iceberg catalog kept a staged file no record names. The mark-removal calls now go through a wrapper that notes whether an UpdateEntry went out without the filer refusing it. Only then may the record have been seen. A refused compare, a missing entry, or a failure before any write leaves the create's record hidden. * s3tables: pass mayBeVisible into backOutChild Fixes the build after rebasing onto the backOutChild split.
39 lines
1.3 KiB
Go
39 lines
1.3 KiB
Go
package s3tables
|
|
|
|
import (
|
|
"path"
|
|
"testing"
|
|
)
|
|
|
|
func TestTableDataDirFromMetadataLocation(t *testing.T) {
|
|
cases := []struct {
|
|
loc string
|
|
want string
|
|
}{
|
|
{"s3://warehouse/sales/orders/metadata/v1.metadata.json", path.Join(TablesPath, "warehouse/sales/orders")},
|
|
{"s3://warehouse/sales/orders/metadata/00003-9f1c.metadata.json", path.Join(TablesPath, "warehouse/sales/orders")},
|
|
{"s3://warehouse/ns/tbl", path.Join(TablesPath, "warehouse/ns/tbl")},
|
|
{"", ""},
|
|
}
|
|
for _, c := range cases {
|
|
if got := TableDataDirFromMetadataLocation(c.loc); got != c.want {
|
|
t.Errorf("TableDataDirFromMetadataLocation(%q) = %q, want %q", c.loc, got, c.want)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestMetadataVersionFromLocationUniqueSuffix(t *testing.T) {
|
|
cases := map[string]int{
|
|
"s3://bkt/ns/t/metadata/v1-0a1b2c3d-4e5f-6789-abcd-ef0123456789.metadata.json": 1,
|
|
"s3://bkt/ns/t/metadata/v4.metadata.json": 4,
|
|
"s3://bkt/ns/t/metadata/v4-0a1b2c3d-4e5f-6789-abcd-ef0123456789.metadata.json": 4,
|
|
"s3://bkt/ns/t/metadata/00007-0a1b2c3d.metadata.json": 7,
|
|
"s3://bkt/ns/t/metadata/whatever.metadata.json": 1,
|
|
}
|
|
for location, want := range cases {
|
|
if got := metadataVersionFromLocation(location); got != want {
|
|
t.Errorf("metadataVersionFromLocation(%q) = %d, want %d", location, got, want)
|
|
}
|
|
}
|
|
}
|