mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-30 08:50:48 +01:00
* s3vectors: translate vector bucket policies to the catalog form The document is stored verbatim on the shared table bucket and evaluated against s3tables actions and ARNs, so a policy written for the vector API was a dead rule. Actions now map to the catalog operation the request authorizes against, and an arn:aws:s3vectors: resource becomes the bucket/.../table/vector/... ARN the index's catalog entry answers to. * s3tables: expose PrincipalFor and PolicyContext for façade evaluation A façade authorizing against its own action namespace needs the same identity resolution and evaluation context the catalog uses internally. * s3vectors: keep bucket policies in vector form and evaluate them in the gateway The earlier approach translated the document to catalog actions and ARNs, which lost the caller's original policy for GetVectorBucketPolicy and could not cover data-plane calls that never touch the catalog. The document now stays in s3vectors form under a dedicated bucket attribute and every handler evaluates it through authorizeVector against the action and ARN the request names; a dedicated key also keeps a vector grant from ever authorizing a catalog call. * s3vectors: close policy-mutation races and keep vector grants visible - Authorize Put/DeleteVectorBucketPolicy inside the policy slot's compare-and-swap, so a grant revoked between reads cannot authorize the pending write. - Assert the policy attributes stay absent in the update precondition, so a concurrent policy or tag write trips the retry instead of being silently reverted. - Migrate a raw vector document out of the shared policy slot on replace or delete, keeping its catalog rules. - Pass the unresolved identity to PolicyContext so open-by-default deployments still work. - Filter ListVectorBuckets on the vector policy itself, so buckets a caller was granted under s3vectors actions stay listed. * s3vectors: reuse SnapshotExtended for the vector-policy CAS precondition * s3vectors: skip buckets that vanish mid-list instead of failing the page * s3vectors: reject non-vector bucket formats inside authorizeVector * s3vectors: check bucket format after authorization in authorizeVector An unauthorized caller probing an ICEBERG bucket got NoSuchBucket,\nleaking that the bucket exists. The format check now runs after the\npermission check so denied callers always see AccessDenied. * s3vectors: cover legacy shared-slot policy removal on delete * s3tables: resolve the forwarded identity name like getAccountID in PrincipalFor A synthesized manager request resolves its caller through the identity- name branch of getAccountID, which normalizes the name (ARNs collapse to their trailing segment, whitespace trims) before falling back to the handler's account ID. PrincipalFor returned the raw name, so ARN-shaped identities never matched their own grants. * s3tables: assert the vector policy key in update preconditions s3vectors-policy is stored on table bucket entries but was missing from s3tablesExtendedKeys, so a catalog update that read the entry before the first vector policy was saved could overwrite it without tripping the compare-and-swap. Asserting the key makes such a race fail instead of silently erasing the policy. * s3vectors: reject rules the evaluator cannot honor in vector bucket policies The shared Statement shape has no negated fields, the vector evaluation supplies no condition keys, and a document that fails the typed decode is discarded wholesale at evaluation — including denies the writer believed active. Reject NotPrincipal, Condition, foreign-service actions and resource ARNs, and require the typed PolicyDocument decode. Also cover that an admin-named identity still reaches its own buckets. * s3vectors: read vector policy fields the way the evaluator decodes them Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3tables: match ARN policy principals through caller normalization Caller principals are normalized ids, so a statement naming a full IAM ARN could never match: an ARN-scoped deny was silently dead and an ARN-scoped allow unreachable. Compare arn: specs against the caller under the same normalizePrincipalID the ownership path uses, while an ARN caller keeps verbatim wildcard matching. * s3tables: grant the admin bypass on the caller's actions, not the name A principal that only normalizes to the admin id skipped every policy deny. When the request carries an identity, the bypass now requires the identity's own admin action — the same gate getAccountID applies — while name-only transports with no action list keep it. PolicyContext takes the request so it can carry IdentityActions, and the vector context helpers route through the shared identity keys so the resolved name and actions agree. * s3tables: match ARN policy principals only for deny statements Normalization drops the ARN's account and path, so an allow matched on the trailing name could grant a same-named caller the statement never verified. Denies keep the normalized match so the rule cannot be ducked through the same collapse. * s3tables: scope service wildcards to their own service s3tables:* counted as blanket admin inside the shared permission check, so an identity holding only catalog grants administered s3vectors operations too. hasAdminActionFor now requires the wildcard to name the operation's service; the global admin markers and s3:* stay service-agnostic. --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
201 lines
6.7 KiB
Go
201 lines
6.7 KiB
Go
package s3tables
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"io"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
|
|
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
|
|
)
|
|
|
|
// Manager provides reusable S3 Tables operations for shell/admin without HTTP routing.
|
|
type Manager struct {
|
|
handler *S3TablesHandler
|
|
}
|
|
|
|
// NewManager creates a new Manager.
|
|
func NewManager() *Manager {
|
|
m := &Manager{handler: NewS3TablesHandler()}
|
|
// Default to allowing access when IAM is not configured
|
|
m.handler.SetDefaultAllow(true)
|
|
return m
|
|
}
|
|
|
|
// SetRegion sets the AWS region for ARN generation.
|
|
func (m *Manager) SetRegion(region string) {
|
|
m.handler.SetRegion(region)
|
|
}
|
|
|
|
// SetAccountID sets the AWS account ID for ARN generation.
|
|
func (m *Manager) SetAccountID(accountID string) {
|
|
m.handler.SetAccountID(accountID)
|
|
}
|
|
|
|
// SetDefaultAllow sets whether to allow access by default.
|
|
func (m *Manager) SetDefaultAllow(allow bool) {
|
|
m.handler.SetDefaultAllow(allow)
|
|
}
|
|
|
|
// SetTrusted lets trusted local tooling (shell, admin console) bypass authorization.
|
|
func (m *Manager) SetTrusted(trusted bool) {
|
|
m.handler.SetTrusted(trusted)
|
|
}
|
|
|
|
// PrincipalFor maps a forwarded identity name onto the account the catalog
|
|
// would charge the request to: a synthesized request resolves its identity
|
|
// name through getAccountID's name branch — normalized like any principal —
|
|
// and an empty name falls back to the handler's account ID.
|
|
func (m *Manager) PrincipalFor(identity string) string {
|
|
if principal := normalizePrincipalID(identity); principal != "" {
|
|
return principal
|
|
}
|
|
return m.handler.accountID
|
|
}
|
|
|
|
// PolicyContext builds the evaluation context a Manager call's synthesized
|
|
// request produces: the configured local-account predicate, the caller's own
|
|
// identity actions so admin and action grants resolve on privileges rather
|
|
// than the forwarded name, plus the open-by-default fallback only for a
|
|
// trusted caller or no resolved identity.
|
|
func (m *Manager) PolicyContext(r *http.Request) *PolicyContext {
|
|
ctx := &PolicyContext{IsLocalAccountID: m.handler.localAccountIDs, IdentityActions: getIdentityActions(r)}
|
|
if m.handler.trusted || (m.handler.defaultAllow && s3_constants.GetIdentityNameFromContext(r) == "") {
|
|
ctx.DefaultAllow = true
|
|
}
|
|
return ctx
|
|
}
|
|
|
|
// Execute runs an S3 Tables operation and decodes the response into resp (if provided).
|
|
func (m *Manager) Execute(ctx context.Context, filerClient FilerClient, operation string, req interface{}, resp interface{}, identity string) error {
|
|
body, err := json.Marshal(req)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
httpReq, err := newManagerRequest(ctx, operation, body, identity)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
recorder := httptest.NewRecorder()
|
|
m.handler.HandleRequest(recorder, httpReq, filerClient)
|
|
return decodeS3TablesHTTPResponse(recorder, resp)
|
|
}
|
|
|
|
// AuthorizeCreateTable checks that identity may create the table the request
|
|
// describes, without creating anything. A deferred create (Iceberg
|
|
// stage-create) writes into the table bucket long before it registers the
|
|
// table, so it passes this gate first.
|
|
func (m *Manager) AuthorizeCreateTable(ctx context.Context, filerClient FilerClient, req *CreateTableRequest, identity string) error {
|
|
httpReq, err := newManagerRequest(ctx, "CreateTable", nil, identity)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
recorder := httptest.NewRecorder()
|
|
_, authErr := m.handler.authorizeCreateTable(recorder, httpReq, filerClient, req.TableBucketARN, req.Namespace, req.Name, req.Tags, req.AllowMissingNamespace)
|
|
if authErr == nil {
|
|
return nil
|
|
}
|
|
if decoded := decodeS3TablesHTTPResponse(recorder, nil); decoded != nil {
|
|
return decoded
|
|
}
|
|
return authErr
|
|
}
|
|
|
|
// AuthorizeUpdateTable checks that identity may update the table the request
|
|
// describes, without updating anything. Credential vending gates on the same
|
|
// permission a commit needs.
|
|
func (m *Manager) AuthorizeUpdateTable(ctx context.Context, filerClient FilerClient, req *UpdateTableRequest, identity string) error {
|
|
httpReq, err := newManagerRequest(ctx, "UpdateTable", nil, identity)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
recorder := httptest.NewRecorder()
|
|
_, authErr := m.handler.authorizeUpdateTable(recorder, httpReq, filerClient, req)
|
|
if authErr == nil {
|
|
return nil
|
|
}
|
|
if decoded := decodeS3TablesHTTPResponse(recorder, nil); decoded != nil {
|
|
return decoded
|
|
}
|
|
return authErr
|
|
}
|
|
|
|
// AuthorizeUpdateView checks that identity may update the view the request
|
|
// describes, without updating anything.
|
|
func (m *Manager) AuthorizeUpdateView(ctx context.Context, filerClient FilerClient, req *UpdateViewRequest, identity string) error {
|
|
httpReq, err := newManagerRequest(ctx, "UpdateView", nil, identity)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
recorder := httptest.NewRecorder()
|
|
_, authErr := m.handler.authorizeUpdateView(recorder, httpReq, filerClient, req)
|
|
if authErr == nil {
|
|
return nil
|
|
}
|
|
if decoded := decodeS3TablesHTTPResponse(recorder, nil); decoded != nil {
|
|
return decoded
|
|
}
|
|
return authErr
|
|
}
|
|
|
|
func newManagerRequest(ctx context.Context, operation string, body []byte, identity string) (*http.Request, error) {
|
|
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, "/", bytes.NewReader(body))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
httpReq.Header.Set("Content-Type", "application/x-amz-json-1.1")
|
|
httpReq.Header.Set("X-Amz-Target", "S3Tables."+operation)
|
|
if identity != "" {
|
|
httpReq.Header.Set(s3_constants.AmzAccountId, identity)
|
|
httpReq = httpReq.WithContext(s3_constants.SetIdentityNameInContext(httpReq.Context(), identity))
|
|
}
|
|
return httpReq, nil
|
|
}
|
|
|
|
func decodeS3TablesHTTPResponse(recorder *httptest.ResponseRecorder, resp interface{}) error {
|
|
result := recorder.Result()
|
|
defer result.Body.Close()
|
|
data, err := io.ReadAll(result.Body)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if result.StatusCode >= http.StatusBadRequest {
|
|
var errResp S3TablesError
|
|
if len(data) > 0 {
|
|
if jsonErr := json.Unmarshal(data, &errResp); jsonErr == nil && (errResp.Type != "" || errResp.Message != "") {
|
|
return &errResp
|
|
}
|
|
}
|
|
return &S3TablesError{Type: ErrCodeInternalError, Message: string(bytes.TrimSpace(data))}
|
|
}
|
|
if resp == nil || len(data) == 0 {
|
|
return nil
|
|
}
|
|
if err := json.Unmarshal(data, resp); err != nil {
|
|
return err
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// ManagerClient adapts a SeaweedFilerClient to the FilerClient interface.
|
|
type ManagerClient struct {
|
|
client filer_pb.SeaweedFilerClient
|
|
}
|
|
|
|
// NewManagerClient wraps a filer client.
|
|
func NewManagerClient(client filer_pb.SeaweedFilerClient) *ManagerClient {
|
|
return &ManagerClient{client: client}
|
|
}
|
|
|
|
// WithFilerClient implements FilerClient.
|
|
func (m *ManagerClient) WithFilerClient(streamingMode bool, fn func(client filer_pb.SeaweedFilerClient) error) error {
|
|
if m.client == nil {
|
|
return errors.New("nil filer client")
|
|
}
|
|
return fn(m.client)
|
|
}
|