Files
seaweedfs/weed/s3api/s3tables/handler_policy.go
T
Nguyễn Lộc Phúcandchrislusf eba2cb926d fix(s3tables): refuse a create that a namespace delete can lose (#12239)
* s3tables: refuse a create that a namespace delete can lose

DeleteNamespace checks the namespace for children and then deletes its
directory recursively in a second filer call. The filer locks each path
on its own, so a table created under the namespace between the two is
not seen by the check and is removed by the delete, while both requests
answer 200. The mirror case loses the table too: a create that passed
its namespace check before the delete finished re-created the namespace
directory through the filer's parent creation and recorded a table in a
namespace that no longer exists.

Each side now writes before it reads. DeleteNamespace claims the
namespace record with a fresh token before it scans, deletes only while
the record still holds that claim, and lifts its own claim when it
stops. CreateTable, RegisterTable and CreateView re-read the namespace
after writing their record and back out when it is claimed or gone:
they remove what they wrote and answer 409 or 404 instead of reporting
an entry that does not survive, or 500 when the record could not be
removed and may remain. An entry the scan missed was written after the
claim, so its create sees the claim; a create and a delete can both
fail, but no longer both succeed. A create is also refused up front
while a claim is held, but only after its authorization check, so a
caller that may not create there does not learn of the delete.

A create writes its record marked "create" and lifts the mark once the
namespace is confirmed open. Until then GetTable, GetView, UpdateTable
and a second create treat the record as absent, so no request is told
about a table or view that the first create may still back out.

* s3tables: keep a create's unfinished record out of reads and listings

* s3tables: refuse catalog writes on a record an operation claimed

* s3tables: answer 404 when a create's write path loses the namespace

* s3tables: confirm the rename destination's namespace like a create

* s3tables: cover unfinished-record reads, claimed-record writes, rename claim

* s3tables: lift a leftover claim when a no-token delete clears the record

* s3tables: answer 409 when a claimed record turns away a write

UpdateView, TagResource, and UntagResource surfaced ErrConcurrentUpdate
as a 500, so an expected conflict looked like a server fault to the
caller. Map it to Conflict the way UpdateTable and the policy handlers
already do.

* s3tables: name the lost source when a rename finds it deleted

* s3tables: report an entry deleted before its update as not found

The filer answers an UpdateEntry on an entry removed since the lookup
with codes.NotFound. mutateEntryExtended returned that error unchanged
instead of filer_pb.ErrNotFound, so a rename whose source was deleted
mid-claim answered 500 instead of NoSuchTable or NoSuchView.

MemFiler now answers a missing entry the same way, so the rename tests
for tables and views take the real filer's path.

* s3tables: unpublish the destination when a rename's source clear fails

* s3tables: refuse to delete a record another operation claimed

* s3tables: release catalog claims on an uncanceled cleanup context

* s3tables: answer 404 when a racing delete already removed the table

* s3tables: keep the rename destination marked until the source clears

* s3tables: guard the rename rollback against a recreated source and an unfinished destination

* s3tables: drop the unfinished destination only once the source is restored

* s3tables: re-resolve the purge target on the record a delete claims

* s3tables: cover a delete that meets a moved metadata location

* s3tables: answer 409 when a create's record changed under it

* s3tables: adopt a record still marked by its unfinished create

* s3tables: skip unfinished creates in the namespace empty check

* s3tables: delete only the record the caller was authorized on

DeleteView and DeleteTable check permissions on the record they read
first, then read the name again to claim it. A record that took the name
in between, such as a table or another owner's view created after a
concurrent delete, was claimed and removed on the first record's
authorization, and DeleteView never compared the caller's version token
with it.

The second read now repeats the first one's checks: a record of the
other kind or an unfinished create answers 404, and a token that no
longer matches answers 409. A delete without a token also compares the
record's version token with the authorized one and answers 409 when they
differ.

* s3tables: re-check the parent after a create lifts its mark

A create that passed checkParentOpen and paused before lifting its mark
could still confirm while a namespace delete held the claim: the delete
scans the still-marked record, skips it, and removes it under a create
that already answered success. confirmParent now re-reads the same
parent record after the mark clears and backs the confirmed record out
when the namespace moved on.

* s3tables: mark the rename destination with a claim of its own

claimCatalogEntry adopted every create-marked record, including a
rename's destination: a create landing after the source cleared could
replace the destination before the rename published it, and with the
source name reused the restore could leave the table on neither name.
The destination now marks with the rename's own claim, which readers
still treat as unpublished but a create must refuse, and the commit
re-checks the destination namespace after the record goes live. The
source keeps its rename claim until the commit lands so a namespace
delete scanning the cleared record still refuses.

* s3tables: compare the parent's createdAt, not its editable metadata

A namespace update rewrites ExtendedKeyMetadata in place, so a create or
rename whose second parent read landed across an UpdateNamespace saw a
changed blob and reported the still-live namespace as gone: creates backed
their record out and answered 404, renames undid their destination.
createdAt survives the edit but changes on a delete+recreate, so it still
identifies the record.

* s3tables: retry lifting the rename's claim off the cleared source

The release's only failure mode is a transient filer error — nothing else
can write the cleared record — yet a single loss left the claim behind:
the old name stayed refused to creates and the namespace counted the
record as a child. Retry the release before warning.

* s3tables: page the namespace scan before deleting it

A page of skippable entries — bare directories or unfinished creates —
ended the scan with hasChildren false, so a table on a later page was
removed with the namespace. Keep listing with StartFromFileName until a
child shows or the listing ends.

---------

Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
2026-10-30 03:18:52 -07:00

1103 lines
38 KiB
Go

package s3tables
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
)
// extractResourceOwnerAndBucket extracts ownership info and bucket name from resource metadata.
// This helper consolidates the repeated pattern used in handleTagResource, handleListTagsForResource,
// and handleUntagResource.
func (h *S3TablesHandler) extractResourceOwnerAndBucket(
data []byte,
resourcePath string,
rType ResourceType,
) (ownerAccountID, bucketName string, err error) {
// Extract bucket name from resource path (format: /buckets/{bucket}/... for both tables and buckets)
parts := strings.Split(strings.Trim(resourcePath, "/"), "/")
if len(parts) >= 2 {
bucketName = parts[1]
}
if rType == ResourceTypeTable {
var meta tableMetadataInternal
if err := json.Unmarshal(data, &meta); err != nil {
return "", "", err
}
ownerAccountID = meta.OwnerAccountID
} else {
var meta tableBucketMetadata
if err := json.Unmarshal(data, &meta); err != nil {
return "", "", err
}
ownerAccountID = meta.OwnerAccountID
}
return ownerAccountID, bucketName, nil
}
// handlePutTableBucketPolicy puts a policy on a table bucket
func (h *S3TablesHandler) handlePutTableBucketPolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req PutTableBucketPolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN is required")
return fmt.Errorf("tableBucketARN is required")
}
if req.ResourcePolicy == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "resourcePolicy is required")
return fmt.Errorf("resourcePolicy is required")
}
if err := validatePolicyDocument(req.ResourcePolicy); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
// Check if bucket exists and get metadata for ownership check
bucketPath := GetTableBucketPath(bucketName)
var bucketMetadata tableBucketMetadata
var bucketPolicy string
var bucketTags map[string]string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
data, err := h.getExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &bucketMetadata); err != nil {
return fmt.Errorf("failed to unmarshal bucket metadata: %w", err)
}
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
bucketTags, err = h.readTags(r.Context(), client, bucketPath)
return err
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchBucket, fmt.Sprintf("table bucket %s not found", bucketName))
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to check table bucket: %v", err))
}
return err
}
bucketARN := h.generateTableBucketARN(bucketMetadata.OwnerAccountID, bucketName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
TableBucketTags: bucketTags,
ResourceTags: bucketTags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("PutTableBucketPolicy", principal, bucketPolicy, bucketARN, ctx) ||
!CheckPermissionWithContext("PutTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, "", bucketARN, ctx) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table bucket policy")
return NewAuthError("PutTableBucketPolicy", principal, "not authorized to put table bucket policy")
}
// Conditional on the policy this request was authorized against: a Deny
// landing mid-request sends the caller back through authorization.
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.mutateEntryExtended(r.Context(), client, bucketPath, func(extended map[string][]byte) error {
if !bytes.Equal(extended[ExtendedKeyPolicy], []byte(bucketPolicy)) {
return fmt.Errorf("%w: %s", ErrConcurrentUpdate, ExtendedKeyPolicy)
}
extended[ExtendedKeyPolicy] = []byte(req.ResourcePolicy)
return nil
})
})
if err != nil {
if errors.Is(err, ErrConcurrentUpdate) {
h.writeError(w, http.StatusConflict, ErrCodeConflict, "table bucket policy changed concurrently, retry the request")
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to put table bucket policy")
}
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// handleGetTableBucketPolicy gets the policy of a table bucket
func (h *S3TablesHandler) handleGetTableBucketPolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req GetTableBucketPolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN is required")
return fmt.Errorf("tableBucketARN is required")
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketPath := GetTableBucketPath(bucketName)
var policy []byte
var bucketMetadata tableBucketMetadata
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
// Get metadata for ownership check
data, err := h.getExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &bucketMetadata); err != nil {
return fmt.Errorf("failed to unmarshal bucket metadata: %w", err)
}
// Get policy
policy, err = h.getExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyPolicy)
return err
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchBucket, fmt.Sprintf("table bucket %s not found", bucketName))
return err
}
if errors.Is(err, ErrAttributeNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchPolicy, "table bucket policy not found")
return err
}
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to get table bucket policy: %v", err))
return err
}
bucketARN := h.generateTableBucketARN(bucketMetadata.OwnerAccountID, bucketName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
if !CheckPermissionWithContext("GetTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, string(policy), bucketARN, &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table bucket policy")
return NewAuthError("GetTableBucketPolicy", principal, "not authorized to get table bucket policy")
}
resp := &GetTableBucketPolicyResponse{
ResourcePolicy: string(policy),
}
h.writeJSON(w, http.StatusOK, resp)
return nil
}
// handleDeleteTableBucketPolicy deletes the policy of a table bucket
func (h *S3TablesHandler) handleDeleteTableBucketPolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req DeleteTableBucketPolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN is required")
return fmt.Errorf("tableBucketARN is required")
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketPath := GetTableBucketPath(bucketName)
// Check if bucket exists and get metadata for ownership check
var bucketMetadata tableBucketMetadata
var bucketPolicy string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
data, err := h.getExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &bucketMetadata); err != nil {
return fmt.Errorf("failed to unmarshal bucket metadata: %w", err)
}
// Fetch bucket policy if it exists
policyData, err := h.getExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyPolicy)
if err != nil {
if !errors.Is(err, ErrAttributeNotFound) {
return fmt.Errorf("failed to read bucket policy: %w", err)
}
// Policy not found is not an error; bucketPolicy remains empty
} else {
bucketPolicy = string(policyData)
}
return nil
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchBucket, fmt.Sprintf("table bucket %s not found", bucketName))
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to check table bucket: %v", err))
}
return err
}
bucketARN := h.generateTableBucketARN(bucketMetadata.OwnerAccountID, bucketName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
if !CheckPermissionWithContext("DeleteTableBucketPolicy", principal, bucketMetadata.OwnerAccountID, bucketPolicy, bucketARN, &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table bucket policy")
return NewAuthError("DeleteTableBucketPolicy", principal, "not authorized to delete table bucket policy")
}
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.deleteExtendedAttribute(r.Context(), client, bucketPath, ExtendedKeyPolicy)
})
if err != nil && !errors.Is(err, ErrAttributeNotFound) {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to delete table bucket policy")
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// handlePutTablePolicy puts a policy on a table
func (h *S3TablesHandler) handlePutTablePolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req PutTablePolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" || len(req.Namespace) == 0 || req.Name == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN, namespace, and name are required")
return fmt.Errorf("missing required parameters")
}
namespaceName, err := validateNamespace(req.Namespace)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.ResourcePolicy == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "resourcePolicy is required")
return fmt.Errorf("resourcePolicy is required")
}
if err := validatePolicyDocument(req.ResourcePolicy); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
// Check if table exists
tableName, err := validateTableName(req.Name)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tablePath := GetTablePath(bucketName, namespaceName, tableName)
bucketPath := GetTableBucketPath(bucketName)
var metadata tableMetadataInternal
var bucketPolicy, tablePolicy string
var bucketTags, tableTags map[string]string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
data, err := h.getExtendedAttribute(r.Context(), client, tablePath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &metadata); err != nil {
return fmt.Errorf("failed to unmarshal table metadata: %w", err)
}
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
if bucketTags, err = h.readTags(r.Context(), client, bucketPath); err != nil {
return err
}
if tablePolicy, err = h.readPolicy(r, client, tablePath); err != nil {
return err
}
tableTags, err = h.readTags(r.Context(), client, tablePath)
return err
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchTable, fmt.Sprintf("table %s not found", tableName))
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to check table: %v", err))
}
return err
}
tableARN := h.generateTableARN(metadata.OwnerAccountID, bucketName, namespaceName+"/"+tableName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: namespaceName,
TableName: tableName,
TableBucketTags: bucketTags,
ResourceTags: tableTags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("PutTablePolicy", principal, tablePolicy, tableARN, ctx) ||
!CheckPermissionWithContext("PutTablePolicy", principal, metadata.OwnerAccountID, bucketPolicy, tableARN, ctx) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to put table policy")
return NewAuthError("PutTablePolicy", principal, "not authorized to put table policy")
}
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.mutateCatalogRecord(r.Context(), client, tablePath, func(extended map[string][]byte) error {
if !bytes.Equal(extended[ExtendedKeyPolicy], []byte(tablePolicy)) {
return fmt.Errorf("%w: %s", ErrConcurrentUpdate, ExtendedKeyPolicy)
}
extended[ExtendedKeyPolicy] = []byte(req.ResourcePolicy)
return nil
})
})
if err != nil {
if errors.Is(err, ErrConcurrentUpdate) {
h.writeError(w, http.StatusConflict, ErrCodeConflict, "table policy changed concurrently, retry the request")
} else if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchTable, fmt.Sprintf("table %s not found", tableName))
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to put table policy")
}
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// handleGetTablePolicy gets the policy of a table
func (h *S3TablesHandler) handleGetTablePolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req GetTablePolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" || len(req.Namespace) == 0 || req.Name == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN, namespace, and name are required")
return fmt.Errorf("missing required parameters")
}
namespaceName, err := validateNamespace(req.Namespace)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tableName, err := validateTableName(req.Name)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tablePath := GetTablePath(bucketName, namespaceName, tableName)
bucketPath := GetTableBucketPath(bucketName)
var policy []byte
var metadata tableMetadataInternal
var bucketPolicy string
var bucketTags, tableTags map[string]string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
// Get metadata for ownership check
data, err := h.getExtendedAttribute(r.Context(), client, tablePath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &metadata); err != nil {
return fmt.Errorf("failed to unmarshal table metadata: %w", err)
}
// Get policy
policy, err = h.getExtendedAttribute(r.Context(), client, tablePath, ExtendedKeyPolicy)
if err != nil {
return err
}
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
if bucketTags, err = h.readTags(r.Context(), client, bucketPath); err != nil {
return err
}
tableTags, err = h.readTags(r.Context(), client, tablePath)
return err
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchTable, fmt.Sprintf("table %s not found", tableName))
return err
}
if errors.Is(err, ErrAttributeNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchPolicy, "table policy not found")
return err
}
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to get table policy: %v", err))
return err
}
tableARN := h.generateTableARN(metadata.OwnerAccountID, bucketName, namespaceName+"/"+tableName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: namespaceName,
TableName: tableName,
TableBucketTags: bucketTags,
ResourceTags: tableTags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("GetTablePolicy", principal, string(policy), tableARN, ctx) ||
!CheckPermissionWithContext("GetTablePolicy", principal, metadata.OwnerAccountID, bucketPolicy, tableARN, ctx) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to get table policy")
return NewAuthError("GetTablePolicy", principal, "not authorized to get table policy")
}
resp := &GetTablePolicyResponse{
ResourcePolicy: string(policy),
}
h.writeJSON(w, http.StatusOK, resp)
return nil
}
// handleDeleteTablePolicy deletes the policy of a table
func (h *S3TablesHandler) handleDeleteTablePolicy(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req DeleteTablePolicyRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.TableBucketARN == "" || len(req.Namespace) == 0 || req.Name == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tableBucketARN, namespace, and name are required")
return fmt.Errorf("missing required parameters")
}
namespaceName, err := validateNamespace(req.Namespace)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
bucketName, err := parseBucketNameFromARN(req.TableBucketARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tableName, err := validateTableName(req.Name)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tablePath := GetTablePath(bucketName, namespaceName, tableName)
bucketPath := GetTableBucketPath(bucketName)
// Check if table exists
var metadata tableMetadataInternal
var bucketPolicy, tablePolicy string
var bucketTags, tableTags map[string]string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
data, err := h.getExtendedAttribute(r.Context(), client, tablePath, ExtendedKeyMetadata)
if err != nil {
return err
}
if err := json.Unmarshal(data, &metadata); err != nil {
return fmt.Errorf("failed to unmarshal table metadata: %w", err)
}
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
if bucketTags, err = h.readTags(r.Context(), client, bucketPath); err != nil {
return err
}
if tablePolicy, err = h.readPolicy(r, client, tablePath); err != nil {
return err
}
tableTags, err = h.readTags(r.Context(), client, tablePath)
return err
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
h.writeError(w, http.StatusNotFound, ErrCodeNoSuchTable, fmt.Sprintf("table %s not found", tableName))
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to check table: %v", err))
}
return err
}
tableARN := h.generateTableARN(metadata.OwnerAccountID, bucketName, namespaceName+"/"+tableName)
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: namespaceName,
TableName: tableName,
TableBucketTags: bucketTags,
ResourceTags: tableTags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("DeleteTablePolicy", principal, tablePolicy, tableARN, ctx) ||
!CheckPermissionWithContext("DeleteTablePolicy", principal, metadata.OwnerAccountID, bucketPolicy, tableARN, ctx) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, "not authorized to delete table policy")
return NewAuthError("DeleteTablePolicy", principal, "not authorized to delete table policy")
}
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.removeExtendedAttributesIf(r.Context(), client, tablePath,
map[string][]byte{ExtendedKeyPolicy: []byte(tablePolicy)},
ExtendedKeyPolicy)
})
if err != nil && !errors.Is(err, ErrAttributeNotFound) {
if errors.Is(err, ErrConcurrentUpdate) {
h.writeError(w, http.StatusConflict, ErrCodeConflict, "table policy changed concurrently, retry the request")
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to delete table policy")
}
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// handleTagResource adds tags to a resource
func (h *S3TablesHandler) handleTagResource(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req TagResourceRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.ResourceARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "resourceArn is required")
return fmt.Errorf("resourceArn is required")
}
if len(req.Tags) == 0 {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tags are required")
return fmt.Errorf("tags are required")
}
// Parse resource ARN to determine if it's a bucket or table
resourcePath, extendedKey, rType, err := h.resolveResourcePath(req.ResourceARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
// Read existing tags and merge, AND check permissions based on metadata ownership
existingTags := make(map[string]string)
var bucketPolicy, tablePolicy string
var bucketTags map[string]string
var tableNamespace, tableName string
requestTagKeys := mapKeys(req.Tags)
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
// Read metadata for ownership check
data, err := h.getExtendedAttribute(r.Context(), client, resourcePath, ExtendedKeyMetadata)
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) || errors.Is(err, ErrAttributeNotFound) {
return h.absentTagTargetError(r, client, "TagResource", "not authorized to tag resource", req.ResourceARN, rType)
}
return err
}
ownerAccountID, bucketName, err := h.extractResourceOwnerAndBucket(data, resourcePath, rType)
if err != nil {
return err
}
// Fetch bucket policy if we have a bucket name
if bucketName != "" {
bucketPath := GetTableBucketPath(bucketName)
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
bucketTags, err = h.readTags(r.Context(), client, bucketPath)
if err != nil {
return err
}
}
if rType == ResourceTypeTable {
if _, tableNamespace, tableName, err = parseTableFromARN(req.ResourceARN); err != nil {
return err
}
if tablePolicy, err = h.readPolicy(r, client, resourcePath); err != nil {
return err
}
}
// Read existing tags
data, err = h.getExtendedAttribute(r.Context(), client, resourcePath, extendedKey)
if err != nil {
if !errors.Is(err, ErrAttributeNotFound) {
return err
}
} else if err := json.Unmarshal(data, &existingTags); err != nil {
return err
}
var resourceARN string
if rType == ResourceTypeTable {
resourceARN = h.generateTableARN(ownerAccountID, bucketName, tableNamespace+"/"+tableName)
} else {
resourceARN = h.generateTableBucketARN(ownerAccountID, bucketName)
}
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: tableNamespace,
TableName: tableName,
TableBucketTags: bucketTags,
RequestTags: req.Tags,
TagKeys: requestTagKeys,
ResourceTags: existingTags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("TagResource", principal, tablePolicy, resourceARN, ctx) ||
!CheckPermissionWithContext("TagResource", principal, ownerAccountID, bucketPolicy, resourceARN, ctx) {
return NewAuthError("TagResource", principal, "not authorized to tag resource")
}
return nil
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
errorCode := ErrCodeNoSuchBucket
if rType == ResourceTypeTable {
errorCode = ErrCodeNoSuchTable
}
h.writeError(w, http.StatusNotFound, errorCode, "resource not found")
} else if isAuthError(err) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, err.Error())
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to read existing tags: %v", err))
}
return err
}
// Merge new tags
for k, v := range req.Tags {
existingTags[k] = v
}
// Write merged tags
tagsBytes, err := json.Marshal(existingTags)
if err != nil {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to marshal tags")
return fmt.Errorf("failed to marshal tags: %w", err)
}
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.mutateCatalogRecord(r.Context(), client, resourcePath, func(extended map[string][]byte) error {
extended[extendedKey] = tagsBytes
return nil
})
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
errorCode := ErrCodeNoSuchBucket
if rType == ResourceTypeTable {
errorCode = ErrCodeNoSuchTable
}
h.writeError(w, http.StatusNotFound, errorCode, "resource not found")
return err
}
if errors.Is(err, ErrConcurrentUpdate) {
h.writeError(w, http.StatusConflict, ErrCodeConflict, "resource tags changed concurrently, retry the request")
return err
}
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to tag resource")
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// handleListTagsForResource lists tags for a resource
func (h *S3TablesHandler) handleListTagsForResource(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req ListTagsForResourceRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.ResourceARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "resourceArn is required")
return fmt.Errorf("resourceArn is required")
}
resourcePath, extendedKey, rType, err := h.resolveResourcePath(req.ResourceARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
tags := make(map[string]string)
var bucketPolicy, tablePolicy string
var bucketTags map[string]string
var tableNamespace, tableName string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
// Read metadata for ownership check
data, err := h.getExtendedAttribute(r.Context(), client, resourcePath, ExtendedKeyMetadata)
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) || errors.Is(err, ErrAttributeNotFound) {
return h.absentTagTargetError(r, client, "ListTagsForResource", "not authorized to list tags for resource", req.ResourceARN, rType)
}
return err
}
ownerAccountID, bucketName, err := h.extractResourceOwnerAndBucket(data, resourcePath, rType)
if err != nil {
return err
}
// Fetch bucket policy if we have a bucket name
if bucketName != "" {
bucketPath := GetTableBucketPath(bucketName)
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
bucketTags, err = h.readTags(r.Context(), client, bucketPath)
if err != nil {
return err
}
}
if rType == ResourceTypeTable {
if _, tableNamespace, tableName, err = parseTableFromARN(req.ResourceARN); err != nil {
return err
}
if tablePolicy, err = h.readPolicy(r, client, resourcePath); err != nil {
return err
}
}
data, err = h.getExtendedAttribute(r.Context(), client, resourcePath, extendedKey)
if err != nil {
if !errors.Is(err, ErrAttributeNotFound) {
return err
}
} else if err := json.Unmarshal(data, &tags); err != nil {
return err
}
var resourceARN string
if rType == ResourceTypeTable {
resourceARN = h.generateTableARN(ownerAccountID, bucketName, tableNamespace+"/"+tableName)
} else {
resourceARN = h.generateTableBucketARN(ownerAccountID, bucketName)
}
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: tableNamespace,
TableName: tableName,
TableBucketTags: bucketTags,
ResourceTags: tags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("ListTagsForResource", principal, tablePolicy, resourceARN, ctx) ||
!CheckPermissionWithContext("ListTagsForResource", principal, ownerAccountID, bucketPolicy, resourceARN, ctx) {
return NewAuthError("ListTagsForResource", principal, "not authorized to list tags for resource")
}
return nil
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
errorCode := ErrCodeNoSuchBucket
if rType == ResourceTypeTable {
errorCode = ErrCodeNoSuchTable
}
h.writeError(w, http.StatusNotFound, errorCode, "resource not found")
} else if isAuthError(err) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, err.Error())
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, fmt.Sprintf("failed to list tags: %v", err))
}
return err
}
resp := &ListTagsForResourceResponse{
Tags: tags,
}
h.writeJSON(w, http.StatusOK, resp)
return nil
}
// handleUntagResource removes tags from a resource
func (h *S3TablesHandler) handleUntagResource(w http.ResponseWriter, r *http.Request, filerClient FilerClient) error {
var req UntagResourceRequest
if err := h.readRequestBody(r, &req); err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
if req.ResourceARN == "" {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "resourceArn is required")
return fmt.Errorf("resourceArn is required")
}
if len(req.TagKeys) == 0 {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, "tagKeys are required")
return fmt.Errorf("tagKeys are required")
}
resourcePath, extendedKey, rType, err := h.resolveResourcePath(req.ResourceARN)
if err != nil {
h.writeError(w, http.StatusBadRequest, ErrCodeInvalidRequest, err.Error())
return err
}
// Read existing tags, check permission
tags := make(map[string]string)
var bucketPolicy, tablePolicy string
var bucketTags map[string]string
var tableNamespace, tableName string
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
// Read metadata for ownership check
data, err := h.getExtendedAttribute(r.Context(), client, resourcePath, ExtendedKeyMetadata)
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) || errors.Is(err, ErrAttributeNotFound) {
return h.absentTagTargetError(r, client, "UntagResource", "not authorized to untag resource", req.ResourceARN, rType)
}
return err
}
ownerAccountID, bucketName, err := h.extractResourceOwnerAndBucket(data, resourcePath, rType)
if err != nil {
return err
}
// Fetch bucket policy if we have a bucket name
if bucketName != "" {
bucketPath := GetTableBucketPath(bucketName)
if bucketPolicy, err = h.readPolicy(r, client, bucketPath); err != nil {
return err
}
bucketTags, err = h.readTags(r.Context(), client, bucketPath)
if err != nil {
return err
}
}
if rType == ResourceTypeTable {
if _, tableNamespace, tableName, err = parseTableFromARN(req.ResourceARN); err != nil {
return err
}
if tablePolicy, err = h.readPolicy(r, client, resourcePath); err != nil {
return err
}
}
data, err = h.getExtendedAttribute(r.Context(), client, resourcePath, extendedKey)
if err != nil {
if !errors.Is(err, ErrAttributeNotFound) {
return err
}
} else if err := json.Unmarshal(data, &tags); err != nil {
return err
}
var resourceARN string
if rType == ResourceTypeTable {
resourceARN = h.generateTableARN(ownerAccountID, bucketName, tableNamespace+"/"+tableName)
} else {
resourceARN = h.generateTableBucketARN(ownerAccountID, bucketName)
}
principal := h.getAccountID(r)
identityActions := getIdentityActions(r)
ctx := &PolicyContext{
TrustedIssuers: h.trustedIssuers(r),
IsLocalAccountID: h.localAccountIDs,
TableBucketName: bucketName,
Namespace: tableNamespace,
TableName: tableName,
TableBucketTags: bucketTags,
TagKeys: req.TagKeys,
ResourceTags: tags,
IdentityActions: identityActions,
DefaultAllow: h.defaultAllowFor(r),
}
if deniedByPolicy("UntagResource", principal, tablePolicy, resourceARN, ctx) ||
!CheckPermissionWithContext("UntagResource", principal, ownerAccountID, bucketPolicy, resourceARN, ctx) {
return NewAuthError("UntagResource", principal, "not authorized to untag resource")
}
return nil
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
errorCode := ErrCodeNoSuchBucket
if rType == ResourceTypeTable {
errorCode = ErrCodeNoSuchTable
}
h.writeError(w, http.StatusNotFound, errorCode, "resource not found")
} else if isAuthError(err) {
h.writeError(w, http.StatusForbidden, ErrCodeAccessDenied, err.Error())
} else {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to read existing tags")
}
return err
}
// Remove specified tags
for _, key := range req.TagKeys {
delete(tags, key)
}
// Write updated tags
tagsBytes, err := json.Marshal(tags)
if err != nil {
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to marshal tags")
return fmt.Errorf("failed to marshal tags: %w", err)
}
err = filerClient.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return h.mutateCatalogRecord(r.Context(), client, resourcePath, func(extended map[string][]byte) error {
extended[extendedKey] = tagsBytes
return nil
})
})
if err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
errorCode := ErrCodeNoSuchBucket
if rType == ResourceTypeTable {
errorCode = ErrCodeNoSuchTable
}
h.writeError(w, http.StatusNotFound, errorCode, "resource not found")
return err
}
if errors.Is(err, ErrConcurrentUpdate) {
h.writeError(w, http.StatusConflict, ErrCodeConflict, "resource tags changed concurrently, retry the request")
return err
}
h.writeError(w, http.StatusInternalServerError, ErrCodeInternalError, "failed to untag resource")
return err
}
h.writeJSON(w, http.StatusOK, nil)
return nil
}
// absentTagTargetError resolves a tag operation when the resource's catalog
// metadata did not load: a caller the operation would refuse gets its denial,
// everyone else the same not-found an absent name gets.
func (h *S3TablesHandler) absentTagTargetError(r *http.Request, client filer_pb.SeaweedFilerClient, operation, deniedMessage, resourceARN string, rType ResourceType) error {
var bucketName, namespace, name string
if rType == ResourceTypeTable {
bucketName, namespace, name, _ = parseTableFromARN(resourceARN)
} else {
bucketName, _ = parseBucketNameFromARN(resourceARN)
}
if !h.authorizeAbsentTarget(r, &singleClient{client}, operation, bucketName, namespace, name, resourceARN) {
return NewAuthError(operation, h.getAccountID(r), deniedMessage)
}
return filer_pb.ErrNotFound
}
// resolveResourcePath determines the resource path and extended attribute key from a resource ARN
func (h *S3TablesHandler) resolveResourcePath(resourceARN string) (path string, key string, rType ResourceType, err error) {
// Try parsing as table ARN first
bucketName, namespace, tableName, err := parseTableFromARN(resourceARN)
if err == nil {
return GetTablePath(bucketName, namespace, tableName), ExtendedKeyTags, ResourceTypeTable, nil
}
// Try parsing as bucket ARN
bucketName, err = parseBucketNameFromARN(resourceARN)
if err == nil {
return GetTableBucketPath(bucketName), ExtendedKeyTags, ResourceTypeBucket, nil
}
return "", "", "", fmt.Errorf("invalid resource ARN: %s", resourceARN)
}