mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-31 01:10:41 +01:00
Nguyễn Lộc Phúcandchrislusf
eba2cb926d
* s3tables: refuse a create that a namespace delete can lose DeleteNamespace checks the namespace for children and then deletes its directory recursively in a second filer call. The filer locks each path on its own, so a table created under the namespace between the two is not seen by the check and is removed by the delete, while both requests answer 200. The mirror case loses the table too: a create that passed its namespace check before the delete finished re-created the namespace directory through the filer's parent creation and recorded a table in a namespace that no longer exists. Each side now writes before it reads. DeleteNamespace claims the namespace record with a fresh token before it scans, deletes only while the record still holds that claim, and lifts its own claim when it stops. CreateTable, RegisterTable and CreateView re-read the namespace after writing their record and back out when it is claimed or gone: they remove what they wrote and answer 409 or 404 instead of reporting an entry that does not survive, or 500 when the record could not be removed and may remain. An entry the scan missed was written after the claim, so its create sees the claim; a create and a delete can both fail, but no longer both succeed. A create is also refused up front while a claim is held, but only after its authorization check, so a caller that may not create there does not learn of the delete. A create writes its record marked "create" and lifts the mark once the namespace is confirmed open. Until then GetTable, GetView, UpdateTable and a second create treat the record as absent, so no request is told about a table or view that the first create may still back out. * s3tables: keep a create's unfinished record out of reads and listings * s3tables: refuse catalog writes on a record an operation claimed * s3tables: answer 404 when a create's write path loses the namespace * s3tables: confirm the rename destination's namespace like a create * s3tables: cover unfinished-record reads, claimed-record writes, rename claim * s3tables: lift a leftover claim when a no-token delete clears the record * s3tables: answer 409 when a claimed record turns away a write UpdateView, TagResource, and UntagResource surfaced ErrConcurrentUpdate as a 500, so an expected conflict looked like a server fault to the caller. Map it to Conflict the way UpdateTable and the policy handlers already do. * s3tables: name the lost source when a rename finds it deleted * s3tables: report an entry deleted before its update as not found The filer answers an UpdateEntry on an entry removed since the lookup with codes.NotFound. mutateEntryExtended returned that error unchanged instead of filer_pb.ErrNotFound, so a rename whose source was deleted mid-claim answered 500 instead of NoSuchTable or NoSuchView. MemFiler now answers a missing entry the same way, so the rename tests for tables and views take the real filer's path. * s3tables: unpublish the destination when a rename's source clear fails * s3tables: refuse to delete a record another operation claimed * s3tables: release catalog claims on an uncanceled cleanup context * s3tables: answer 404 when a racing delete already removed the table * s3tables: keep the rename destination marked until the source clears * s3tables: guard the rename rollback against a recreated source and an unfinished destination * s3tables: drop the unfinished destination only once the source is restored * s3tables: re-resolve the purge target on the record a delete claims * s3tables: cover a delete that meets a moved metadata location * s3tables: answer 409 when a create's record changed under it * s3tables: adopt a record still marked by its unfinished create * s3tables: skip unfinished creates in the namespace empty check * s3tables: delete only the record the caller was authorized on DeleteView and DeleteTable check permissions on the record they read first, then read the name again to claim it. A record that took the name in between, such as a table or another owner's view created after a concurrent delete, was claimed and removed on the first record's authorization, and DeleteView never compared the caller's version token with it. The second read now repeats the first one's checks: a record of the other kind or an unfinished create answers 404, and a token that no longer matches answers 409. A delete without a token also compares the record's version token with the authorized one and answers 409 when they differ. * s3tables: re-check the parent after a create lifts its mark A create that passed checkParentOpen and paused before lifting its mark could still confirm while a namespace delete held the claim: the delete scans the still-marked record, skips it, and removes it under a create that already answered success. confirmParent now re-reads the same parent record after the mark clears and backs the confirmed record out when the namespace moved on. * s3tables: mark the rename destination with a claim of its own claimCatalogEntry adopted every create-marked record, including a rename's destination: a create landing after the source cleared could replace the destination before the rename published it, and with the source name reused the restore could leave the table on neither name. The destination now marks with the rename's own claim, which readers still treat as unpublished but a create must refuse, and the commit re-checks the destination namespace after the record goes live. The source keeps its rename claim until the commit lands so a namespace delete scanning the cleared record still refuses. * s3tables: compare the parent's createdAt, not its editable metadata A namespace update rewrites ExtendedKeyMetadata in place, so a create or rename whose second parent read landed across an UpdateNamespace saw a changed blob and reported the still-live namespace as gone: creates backed their record out and answered 404, renames undid their destination. createdAt survives the edit but changes on a delete+recreate, so it still identifies the record. * s3tables: retry lifting the rename's claim off the cleared source The release's only failure mode is a transient filer error — nothing else can write the cleared record — yet a single loss left the claim behind: the old name stayed refused to creates and the namespace counted the record as a child. Retry the release before warning. * s3tables: page the namespace scan before deleting it A page of skippable entries — bare directories or unfinished creates — ended the scan with hasChildren false, so a table on a later page was removed with the namespace. Keep listing with StartFromFileName until a child shows or the listing ends. --------- Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
491 lines
19 KiB
Go
491 lines
19 KiB
Go
package s3tables
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"net/http"
|
|
"path"
|
|
"testing"
|
|
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3tables/s3tablestest"
|
|
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
)
|
|
|
|
func runDeleteTable(t *testing.T, m *Manager, fs *s3tablestest.MemFiler, namespace, name string) error {
|
|
t.Helper()
|
|
return m.Execute(context.Background(), NewManagerClient(fs.Client), "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{namespace},
|
|
Name: name,
|
|
}, nil, "")
|
|
}
|
|
|
|
// A write that lands between the delete's read and its conditional commit —
|
|
// an xattr update keeps mtime — must turn a token delete into a conflict, not
|
|
// get erased by it.
|
|
func TestDeleteTableTokenDeleteLosesToConcurrentUpdate(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
token := "tok-ct"
|
|
meta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "ct",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
VersionToken: token,
|
|
})
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "ct", map[string][]byte{
|
|
ExtendedKeyMetadata: meta,
|
|
})
|
|
tablePath := GetTablePath(renameTestBucket, "ns", "ct")
|
|
fs.Put(tablePath, "data", nil)
|
|
|
|
fs.BeforeDelete = func() {
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "ct", map[string][]byte{
|
|
ExtendedKeyMetadata: meta,
|
|
ExtendedKeyTags: []byte(`{"k":"v"}`),
|
|
})
|
|
}
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "ct",
|
|
VersionToken: token,
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
assert.NotNil(t, fs.Get(GetNamespacePath(renameTestBucket, "ns"), "ct"), "the updated record must survive")
|
|
assert.NotNil(t, fs.Get(tablePath, "data"), "the table's data must survive")
|
|
}
|
|
|
|
// A create that checked the name while it was free must not write over a
|
|
// delete's restore: the claim loses instead of layering new metadata over the
|
|
// old tags, policy, and maintenance settings.
|
|
func TestCreateTableLosesToRestoredMarkers(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
fs.Put(nsPath, "newt", nil)
|
|
|
|
oldMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "newt",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
})
|
|
restored := map[string][]byte{
|
|
ExtendedKeyMetadata: oldMeta,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
ExtendedKeyTags: []byte(`{"old":"tag"}`),
|
|
ExtendedKeyMaintenance: []byte(`{"old":"maintenance"}`),
|
|
}
|
|
fs.BeforeUpdate = func() {
|
|
fs.Put(nsPath, "newt", restored)
|
|
}
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "CreateTable", &CreateTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "newt",
|
|
Format: FormatIceberg,
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeTableAlreadyExists, resp.Type)
|
|
|
|
got := fs.Get(nsPath, "newt")
|
|
require.NotNil(t, got)
|
|
assert.Equal(t, restored[ExtendedKeyTags], got.Extended[ExtendedKeyTags], "restored markers must not be overwritten")
|
|
assert.Equal(t, restored[ExtendedKeyMetadata], got.Extended[ExtendedKeyMetadata])
|
|
}
|
|
|
|
// A table whose data was decoupled from its name (created over a leftover from a
|
|
// rename): catalog marker at ns/newt, data at ns/newt-x, and ns/newt still holds
|
|
// another table's leftover data. Dropping it must purge only its own data and
|
|
// clear the marker, never the data under the reused name path.
|
|
func TestDeleteTableDecoupledKeepsReusedNamePath(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
newtMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "newt",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/newt-x/metadata/v1.metadata.json",
|
|
})
|
|
markerKeys := []string{ExtendedKeyMetadata, ExtendedKeyMetadataVersion, ExtendedKeyPolicy, ExtendedKeyTags, ExtendedKeyEntryType}
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "newt", map[string][]byte{
|
|
ExtendedKeyMetadata: newtMeta,
|
|
ExtendedKeyMetadataVersion: []byte("v1"),
|
|
ExtendedKeyPolicy: []byte(`{"Version":"2012-10-17"}`),
|
|
ExtendedKeyTags: []byte(`{"k":"v"}`),
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
fs.Put(GetTablePath(renameTestBucket, "ns", "newt"), "leftover", nil) // another table's data under the name path
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "newt-x", nil) // this table's own (decoupled) data
|
|
fs.Put(GetTablePath(renameTestBucket, "ns", "newt-x"), "metadata", nil)
|
|
|
|
require.NoError(t, runDeleteTable(t, m, fs, "ns", "newt"))
|
|
|
|
assert.Nil(t, fs.Get(GetNamespacePath(renameTestBucket, "ns"), "newt-x"),
|
|
"the table's own data location must be purged")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "newt"), "leftover"),
|
|
"data under the reused name path must survive")
|
|
marker := fs.Get(GetNamespacePath(renameTestBucket, "ns"), "newt")
|
|
require.NotNil(t, marker)
|
|
for _, key := range markerKeys {
|
|
_, present := marker.Extended[key]
|
|
assert.Falsef(t, present, "catalog attribute %s must be cleared", key)
|
|
}
|
|
}
|
|
|
|
// A decoupled record still carrying a claim its operation never lifted — a
|
|
// delete or rename that ended between the claim and the marker clear — must go
|
|
// with the rest of the marker, or the leftover claim blocks every create of
|
|
// the name from then on.
|
|
func TestDeleteTableDecoupledClearsLeftoverClaim(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
newtMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "newt",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/newt-x/metadata/v1.metadata.json",
|
|
})
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
fs.Put(nsPath, "newt", map[string][]byte{
|
|
ExtendedKeyMetadata: newtMeta,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
ExtendedKeyDeleting: []byte("delete"),
|
|
})
|
|
fs.Put(nsPath, "newt-x", nil)
|
|
fs.Put(GetTablePath(renameTestBucket, "ns", "newt-x"), "metadata", nil)
|
|
|
|
require.NoError(t, runDeleteTable(t, m, fs, "ns", "newt"))
|
|
|
|
marker := fs.Get(nsPath, "newt")
|
|
require.NotNil(t, marker)
|
|
assert.NotContains(t, marker.Extended, ExtendedKeyDeleting, "a leftover claim must not survive the delete")
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "CreateTable", &CreateTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "newt",
|
|
Format: FormatIceberg,
|
|
})
|
|
assert.Equal(t, http.StatusOK, status, "the name must take a create once the claim is gone: %+v", resp)
|
|
}
|
|
|
|
// An update that moves a decoupled table's MetadataLocation between the
|
|
// delete's authorized read and the read its claim and purge run on must turn
|
|
// the delete into a conflict: the purge otherwise empties the stale directory
|
|
// and unmarks a record whose data lives somewhere else entirely.
|
|
func TestDeleteTableDecoupledLosesToMovedLocation(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
metaA, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "mv",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/mv-a/metadata/v1.metadata.json",
|
|
})
|
|
metaB, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "mv",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/mv-b/metadata/v1.metadata.json",
|
|
})
|
|
fs.Put(nsPath, "mv", map[string][]byte{
|
|
ExtendedKeyMetadata: metaA,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
fs.Put(GetTablePath(renameTestBucket, "ns", "mv-a"), "data", nil)
|
|
fs.Put(GetTablePath(renameTestBucket, "ns", "mv-b"), "data", nil)
|
|
|
|
// The first read of the marker is the authorized one; land the update's
|
|
// write in front of the read the claim and purge run on.
|
|
lookups := 0
|
|
fs.OnLookup = func(dir, name string) {
|
|
if dir != nsPath || name != "mv" {
|
|
return
|
|
}
|
|
lookups++
|
|
if lookups > 1 {
|
|
fs.OnLookup = nil
|
|
fs.Put(nsPath, "mv", map[string][]byte{
|
|
ExtendedKeyMetadata: metaB,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
}
|
|
}
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "mv",
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "mv-a"), "data"),
|
|
"the stale location's data must survive")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "mv-b"), "data"),
|
|
"the live location's data must survive")
|
|
rec := fs.Get(nsPath, "mv")
|
|
require.NotNil(t, rec)
|
|
assert.Equal(t, metaB, rec.Extended[ExtendedKeyMetadata])
|
|
assert.NotContains(t, rec.Extended, ExtendedKeyDeleting, "a refused delete must not leave a claim")
|
|
}
|
|
|
|
// A table whose MetadataLocation resolves to an ancestor of its own name path
|
|
// (here the namespace root, e.g. from corrupt metadata) must not be deleted: a
|
|
// recursive purge of that data path would wipe sibling tables. The delete is
|
|
// refused and the namespace's other tables survive.
|
|
func TestDeleteTableRefusesAncestorDataPath(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
badMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "badt",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/metadata/v1.metadata.json",
|
|
})
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "badt", map[string][]byte{ExtendedKeyMetadata: badMeta})
|
|
|
|
require.Error(t, runDeleteTable(t, m, fs, "ns", "badt"))
|
|
|
|
// The sibling table seeded by startRenameManager and its data must survive.
|
|
assert.NotNil(t, fs.Get(GetNamespacePath(renameTestBucket, "ns"), "t"),
|
|
"sibling table marker must survive a refused delete")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "data"),
|
|
"sibling table data must survive a refused delete")
|
|
}
|
|
|
|
// A normal colocated table (data under its own name path) is removed wholesale.
|
|
func TestDeleteTableColocatedRemovesData(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
require.NoError(t, runDeleteTable(t, m, fs, "ns", "t"))
|
|
|
|
assert.Nil(t, fs.Get(GetNamespacePath(renameTestBucket, "ns"), "t"), "colocated table entry must be deleted")
|
|
assert.Nil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "metadata"), "colocated table data must be deleted")
|
|
}
|
|
|
|
// A table whose MetadataLocation points at a sibling that is still a live
|
|
// catalog entry must not be deleted: only the named table was authorized, so a
|
|
// recursive purge of that location would destroy another tenant's table.
|
|
func TestDeleteTableRefusesLiveSiblingDataPath(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsMeta, _ := json.Marshal(namespaceMetadata{Namespace: []string{"attackerns"}, OwnerAccountID: DefaultAccountID})
|
|
fs.Put(GetTableBucketPath(renameTestBucket), "attackerns", map[string][]byte{ExtendedKeyMetadata: nsMeta})
|
|
|
|
decoyMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "decoy",
|
|
Namespace: "attackerns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/t/metadata/v3.metadata.json",
|
|
})
|
|
fs.Put(GetNamespacePath(renameTestBucket, "attackerns"), "decoy", map[string][]byte{ExtendedKeyMetadata: decoyMeta})
|
|
|
|
require.Error(t, runDeleteTable(t, m, fs, "attackerns", "decoy"))
|
|
|
|
assert.NotNil(t, fs.Get(GetNamespacePath(renameTestBucket, "ns"), "t"),
|
|
"the targeted table's catalog entry must survive")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "data"),
|
|
"the targeted table's data must survive")
|
|
}
|
|
|
|
// A location inside another table's directory carries no catalog marker of its
|
|
// own — the marker lives on the name path. Dropping a table whose
|
|
// MetadataLocation resolves into nsa/vtab/data must be refused by the ancestor
|
|
// check, or the victim's marker-less data subdirectory is purged wholesale.
|
|
func TestDeleteTableRefusesNestedSiblingDataPath(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsaMeta, _ := json.Marshal(namespaceMetadata{Namespace: []string{"nsa"}, OwnerAccountID: DefaultAccountID})
|
|
fs.Put(GetTableBucketPath(renameTestBucket), "nsa", map[string][]byte{ExtendedKeyMetadata: nsaMeta})
|
|
|
|
victimMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "vtab",
|
|
Namespace: "nsa",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/nsa/vtab/metadata/v1.metadata.json",
|
|
})
|
|
victimTablePath := GetTablePath(renameTestBucket, "nsa", "vtab")
|
|
fs.Put(GetNamespacePath(renameTestBucket, "nsa"), "vtab", map[string][]byte{
|
|
ExtendedKeyMetadata: victimMeta,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
fs.Put(victimTablePath, "data", nil)
|
|
fs.Put(path.Join(victimTablePath, "data"), "important-0001.parquet", nil)
|
|
|
|
evilMeta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "evil",
|
|
Namespace: "attackerns2",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/nsa/vtab/data/metadata/v1.metadata.json",
|
|
})
|
|
nsMeta, _ := json.Marshal(namespaceMetadata{Namespace: []string{"attackerns2"}, OwnerAccountID: DefaultAccountID})
|
|
fs.Put(GetTableBucketPath(renameTestBucket), "attackerns2", map[string][]byte{ExtendedKeyMetadata: nsMeta})
|
|
fs.Put(GetNamespacePath(renameTestBucket, "attackerns2"), "evil", map[string][]byte{
|
|
ExtendedKeyMetadata: evilMeta,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
|
|
require.Error(t, runDeleteTable(t, m, fs, "attackerns2", "evil"))
|
|
|
|
assert.NotNil(t, fs.Get(path.Join(victimTablePath, "data"), "important-0001.parquet"),
|
|
"data inside another table's directory must survive")
|
|
assert.NotNil(t, fs.Get(GetNamespacePath(renameTestBucket, "nsa"), "vtab"),
|
|
"the victim table's catalog entry must survive")
|
|
}
|
|
|
|
// A rename that completes in the token delete's check-to-claim window moves
|
|
// the record to another name: the stale delete must lose on the claim instead
|
|
// of purging the data the moved record still points at.
|
|
func TestDeleteTableTokenDeleteLosesToCompletedRename(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
token := "tok-rn"
|
|
meta, _ := json.Marshal(tableMetadataInternal{
|
|
Name: "rn",
|
|
Namespace: "ns",
|
|
Format: "ICEBERG",
|
|
OwnerAccountID: DefaultAccountID,
|
|
VersionToken: token,
|
|
MetadataLocation: "s3://" + renameTestBucket + "/ns/old-x/metadata/v1.metadata.json",
|
|
})
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "rn", map[string][]byte{
|
|
ExtendedKeyMetadata: meta,
|
|
ExtendedKeyEntryType: []byte(EntryTypeTable),
|
|
})
|
|
dataPath := GetTablePath(renameTestBucket, "ns", "old-x")
|
|
fs.Put(dataPath, "data", nil)
|
|
|
|
fs.BeforeUpdate = func() {
|
|
// The rename's source clear has already run by the time the delete's
|
|
// claim write lands.
|
|
fs.Put(GetNamespacePath(renameTestBucket, "ns"), "rn", nil)
|
|
}
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "rn",
|
|
VersionToken: token,
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
assert.NotNil(t, fs.Get(dataPath, "data"), "the moved table's data must survive the stale delete")
|
|
}
|
|
|
|
// A delete's claim on the source must turn a rename into a conflict before
|
|
// any destination entry exists: the claim is what keeps a losing rename from
|
|
// leaving a destination marker pointing at data the delete went on to purge.
|
|
func TestRenameTableLosesToDeleteClaim(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
fs.BeforeUpdate = func() {
|
|
e := fs.Get(nsPath, "t")
|
|
claimed := map[string][]byte{}
|
|
for k, v := range e.Extended {
|
|
claimed[k] = v
|
|
}
|
|
claimed[ExtendedKeyDeleting] = []byte("delete")
|
|
fs.Put(nsPath, "t", claimed)
|
|
}
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "RenameTable", &RenameTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
SourceNamespace: []string{"ns"},
|
|
SourceName: "t",
|
|
DestNamespace: []string{"ns"},
|
|
DestName: "t2",
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
assert.Nil(t, fs.Get(nsPath, "t2"), "a losing rename must not publish the destination")
|
|
src := fs.Get(nsPath, "t")
|
|
require.NotNil(t, src)
|
|
assert.NotNil(t, src.Extended[ExtendedKeyMetadata], "the claimed source record must survive")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "data"))
|
|
}
|
|
|
|
// A no-token delete that meets a record claimed by a rename loses to it: the
|
|
// claim is what keeps the rename's destination from pointing at purged data.
|
|
func TestDeleteTableLosesToRenameClaim(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
e := fs.Get(nsPath, "t")
|
|
claimed := map[string][]byte{}
|
|
for k, v := range e.Extended {
|
|
claimed[k] = v
|
|
}
|
|
claimed[ExtendedKeyDeleting] = []byte("rename")
|
|
fs.Put(nsPath, "t", claimed)
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "t",
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
src := fs.Get(nsPath, "t")
|
|
require.NotNil(t, src)
|
|
assert.NotNil(t, src.Extended[ExtendedKeyMetadata], "the claimed record must survive")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "data"))
|
|
}
|
|
|
|
// The token delete takes its snapshot after the rename's claim landed; it must
|
|
// still refuse the claimed record rather than deleting it.
|
|
func TestDeleteTableTokenLosesToRenameClaim(t *testing.T) {
|
|
fs, m := startRenameManager(t)
|
|
|
|
nsPath := GetNamespacePath(renameTestBucket, "ns")
|
|
e := fs.Get(nsPath, "t")
|
|
claimed := map[string][]byte{}
|
|
for k, v := range e.Extended {
|
|
claimed[k] = v
|
|
}
|
|
var meta tableMetadataInternal
|
|
require.NoError(t, json.Unmarshal(claimed[ExtendedKeyMetadata], &meta))
|
|
meta.VersionToken = "tok"
|
|
metaBytes, err := json.Marshal(&meta)
|
|
require.NoError(t, err)
|
|
claimed[ExtendedKeyMetadata] = metaBytes
|
|
claimed[ExtendedKeyDeleting] = []byte("rename")
|
|
fs.Put(nsPath, "t", claimed)
|
|
|
|
status, resp := runHandler(t, m, fs.Client, "DeleteTable", &DeleteTableRequest{
|
|
TableBucketARN: mustBucketARN(t),
|
|
Namespace: []string{"ns"},
|
|
Name: "t",
|
|
VersionToken: "tok",
|
|
})
|
|
assert.Equal(t, http.StatusConflict, status)
|
|
assert.Equal(t, ErrCodeConflict, resp.Type)
|
|
|
|
src := fs.Get(nsPath, "t")
|
|
require.NotNil(t, src)
|
|
assert.NotNil(t, src.Extended[ExtendedKeyMetadata], "the claimed record must survive")
|
|
assert.NotNil(t, fs.Get(GetTablePath(renameTestBucket, "ns", "t"), "data"))
|
|
}
|