mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-12 09:17:44 +02:00
* s3/iam: manage roles through the IAM API, with an opt-in persistent role store
Roles could only come from the IAM config file: the S3 server pinned the
role store to memory and the embedded IAM API had no role actions, so a
role could not be created, retrusted or revoked without editing the file
and restarting every gateway.
Role store
- Read the `roleStore` key (the IAMConfig field already existed). With an
IAM config file the default stays memory; with none it is the filer, as
for OIDC providers, so zero-config clusters keep runtime-created roles.
- Roles from the IAM config file never go into a persistent role store,
which outlives the file and may be shared by S3 servers with different
files. They are served from memory beneath the store, as OIDC providers
are: a stored role of the same name takes precedence, and deleting it
restores the file's. A config-file role cannot be changed or deleted
through the API (UnmodifiableEntity), and removing one from the file
removes it at the next start. An in-memory store holds them as records,
as before. They have no creation time, so CreateDate is omitted rather
than reporting when this server started. SetRoleStore installs a store
the same way, so a store set after startup keeps the config-file roles,
as SetOIDCProviderStore does for providers.
- Watch /etc/iam/roles and drop the cached role definitions on change. The
cached filer store otherwise serves a peer's stale role for up to its 5m
TTL, which keeps a revoked trust policy in force on the other gateways.
- Role stores wrap ErrRoleNotFound for a missing role; the filer store
used to report any failed lookup as "role not found". CreateRole proceeds
only on a confirmed absence, so an unreadable store cannot let it write
over an existing role.
IAM actions
- CreateRole, GetRole, ListRoles, DeleteRole, UpdateAssumeRolePolicy,
AttachRolePolicy, DetachRolePolicy, ListAttachedRolePolicies. The reads
are allowed in read-only mode.
- A role defined in the config file is reloaded from it at every start, so
changing or deleting it through the API is refused (UnmodifiableEntity)
rather than silently reverted.
- DeleteRole with policies attached is refused (DeleteConflict), as on AWS.
- Role names follow AWS's rules ([\w+=,.@-]{1,64}); a role is stored as
<name>.json in the filer, so this also keeps a name from leaving the role
store's directory. At most 10 managed policies per role (AWS's default
quota; MaxManagedPoliciesPerUser is 10 too), LimitExceeded beyond.
- DeletePolicy is refused (DeleteConflict) while a role attaches the
policy, as it already is for users and groups: roles attach policies by
name, so a policy created later under the deleted one's name would
otherwise take effect on the role.
- Role paths other than "/" and role tags are not stored, so they are
refused rather than dropped.
Role IDs and sessions
- Roles get a unique RoleId when first stored (random, AWS AROA form),
kept across updates; a config-file role gets a stable ID derived from its
name, since it is created again at every start.
- Sessions issued through AssumeRoleWithWebIdentity, AssumeRoleWithCredentials
and AssumeRole carry the role's ID (claim "rid"), and a request under a role
whose current ID differs is denied. Resolving a session's policies by role
name let a session outlive its role: once a role was deleted, a role later
created under the same name — with a different trust policy and different
policies — revived every unexpired session of the old one with the new
role's permissions. Sessions issued before this change carry no ID and are
unaffected until they expire.
Integration test (test/s3/iam, run with `make start-services`):
TestWebIdentityWithProviderAndRoleManagedThroughIAMAPI configures an OIDC
provider, a managed policy and a role entirely through the IAM API against a
JWKS served by the test, then checks the trusted subject gets credentials
scoped to the attached policy; another subject, a token signed by another
key, an unsigned token and a token for another audience are refused; and UpdateAssumeRolePolicy moves the
trust at once.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* iam: manage OIDC providers and roles over the filer IAM gRPC service
The filer's SeaweedIdentityAccessManagement service covers users, access
keys, policies and service accounts, but not the OIDC providers and roles
that STS web-identity federation needs. A controller that already manages
IAM over this service (seaweedfs-operator's S3OIDCProvider) has no
transport for them; its swadmin client returns ErrOIDCNotWired and names
this as the recommended fix.
- PutOIDCProvider / GetOIDCProvider / DeleteOIDCProvider / ListOIDCProviders
and PutRole / GetRole / DeleteRole / ListRoles.
- They write the filer-backed stores at their default paths, which S3
servers read when configured with a filer-typed "oidcProviderStore" and
"roleStore"; the S3 servers' /etc/iam subscription applies changes
without a restart.
- Put is an upsert, so a controller can reconcile to it. Deleting a
provider or role that does not exist returns NotFound, as DeleteUser does
for a user; clients treat that as already deleted. The provider's account
ID travels in the request, since the filer does not know the STS
accountId.
- PutRole applies the IAM API's rules: AWS role names, at most 10 managed
policies.
- An S3 server serves the roles and providers of its own IAM config file
ahead of the store, so a stored entry with the same name has no effect
on that server.
- PutRole keeps a replaced role's RoleId and gives a role created anew a
fresh one, so sessions of a deleted role do not carry over to a later role
of the same name.
- DeletePolicy returns FailedPrecondition while a role attaches the policy
(see the IAM API's DeleteConflict in the previous change). DeletePolicy on
this service still does not check user attachments, which predates this.
- PutOIDCProvider requires an https issuer (http only for a loopback host):
STS fetches the issuer's signing keys from it, so over plain HTTP anyone
on the network path could substitute their own.
- The OIDC provider and role RPCs refuse to run on an unauthenticated
service (FailedPrecondition until jwt.filer_signing.key is set). Users and
policies keep the service's opt-in auth, but these grant STS access
outright: otherwise anyone who can reach the port could register an issuer
they control, create a role trusting it, and exchange a token for S3
credentials. The filer's unauthenticated notice becomes a warning that says
so.
- A store that cannot be read is Unavailable, never "not found", so a Put
never writes over an entry it could not see.
- Validation is shared with the IAM API through PrepareRoleDefinition and
PrepareOIDCProviderRecord.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* s3/iam: bind every role session to its role, and change roles atomically
Review follow-ups.
Session binding
- The role-ID check ran only when a session carried no policy names, and
AssumeRole embeds the role's attached policies, so those sessions kept
their permissions after the role was deleted or recreated. The check
now runs for every session carrying a role ID, before policy selection.
- A named role that cannot be resolved at issuance gets no session,
instead of one with no role ID (which nothing binds).
- A config-file role's ID is derived from its name and trust policy, not
the name alone: a different role put in the file under the same name
gets a new ID, while an unchanged role keeps its sessions across restarts.
Role writes
- RoleStore gains UpdateRole, a read-modify-write that lands only if the
role is unchanged since the read, and otherwise re-reads and retries. The
filer store uses the filer's write conditions (IF_NOT_EXISTS for a new
role, IF_ENTRY_EQUAL otherwise). CreateRole, UpdateAssumeRolePolicy and
Attach/DetachRolePolicy all go through it, so two gateways no longer
overwrite each other's changes, a change racing a delete no longer
writes the role back, and of two concurrent creates one gets
EntityAlreadyExists.
- The filer store's ListRoles pages past 1,000 entries and fails on a
broken stream instead of returning what arrived, so DeletePolicy's
attachment check sees every role. ListRoles skips a role deleted between
listing and reading it.
- CreateRole validates first; a failed write is ServiceFailure, not
InvalidInput. Any Tags.* parameter is refused, not only the first key.
- ExecuteAction's skipPersist covers the S3ApiConfiguration only; the
comment now says so. Role and OIDC provider actions write their own stores.
Each fix has a test that fails without it. Against a real filer with two
gateways, concurrent AttachRolePolicy calls lost 1-4 of 8 attachments per
run before this change and none after.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* iam: PutRole changes roles atomically and checks its ARN; https issuers' keys stay on https
Review follow-ups on top of the role-store changes.
- PutRole goes through RoleStore.UpdateRole, so the decision to keep an
existing role's ID or mint a new one is made against the role as it is
when written. A PutRole racing a DeleteRole can no longer write the
deleted role back with its old ID, which would revive its sessions. A
failed store read or write is Unavailable.
- PutRole refuses a role_arn that does not name the role: STS resolves a
role by the name in the ARN it is given.
- PutOIDCProvider requires an https issuer, but discovery could still name
a plain-http jwks_uri, and a key fetch could be redirected to http. For
an https issuer, a non-https jwks_uri from discovery is refused (the
issuer's own /.well-known/jwks.json is used instead), and the client
that fetches discovery and keys refuses any https-to-http redirect. An
operator-set jwksUri is left as configured.
Each has a test that fails without its guard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* s3/iam: one role snapshot per decision; DeleteRole is atomic; watch a custom role store path
Review follow-ups.
- Authorization evaluates the policies of the role definition the session's
binding was checked against, instead of reading the role again: a role
replaced in between cannot lend a session its policies.
- AssumeRole and AssumeRoleWithLDAPIdentity issue the session from the
definition whose trust admits the caller (IAMManager.ResolveRoleForPrincipal),
and take its ID, duration cap and embedded policies from that same
definition. A role replaced after the caller's trust check by one that does
not trust the caller now yields AccessDenied, not a session bound to the
replacement.
- A RoleUpdate that returns nil deletes the role, on the same condition as a
write: the filer store deletes with ObjectTransaction on IF_ENTRY_EQUAL,
routed and locked like the conditional CreateEntry. DeleteRole decides
against the role it deletes, so a policy attached meanwhile on another
server is a DeleteConflict, and a delete never removes a role written
after its check.
- S3 servers watch the role store's configured basePath, not only
/etc/iam/roles, so a custom path also drops peers' cached roles on change.
Each has a test that fails without it. Live against a real filer: DeleteRole
refuses while a policy is attached and removes the entry once detached; all
test/s3/iam CI stages pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* s3/iam: state which roles DeletePolicy's attachment check can see
RolesAttachingPolicy sees the stored roles and this server's config-file
roles. A role defined only in another server's IAM config file is invisible
to it, so a config-file role that attaches a managed policy is protected
only on the servers whose file defines it. The doc comment now says so and
how to avoid it: keep such roles in every server's file, or attach only
config-file policies to config-file roles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* iam: note that a role store set after startup is not watched for peer changes
S3 servers build their metadata watch list once, at startup, from the role
store installed then. SetRoleStore's doc now says that a filer-backed store
installed later with a different basePath is not watched, so peers' changes
to it reach this server's cached roles only when the cache expires.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* iam: DeleteRole deletes only the role it saw; issuer URLs are bare
Review follow-ups.
- The filer IAM service's DeleteRole looked the role up, then deleted by
name, so a PutRole landing in between had its new definition deleted. It
now deletes through RoleStore.UpdateRole, conditional on the entry it
read. If the role was replaced meanwhile, it returns Aborted rather than
deleting the replacement, and the caller decides again.
- PutOIDCProvider refuses an issuer URL with userinfo, a query or a
fragment. The provider's ARN comes from host and path alone, while STS
matches a token's iss claim against the stored URL exactly, so such a
provider shared the bare issuer's ARN and matched no token. A loopback
"localhost" is now matched without regard to case.
Both have tests that fail without them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* iam: write OIDC providers atomically over the filer IAM gRPC service
PutOIDCProvider read the record, then stored unconditionally; a racing
DeleteOIDCProvider left the put's stale read merged into the rewritten
record. DeleteOIDCProvider read, then deleted unconditionally; a racing
PutOIDCProvider's newer record could be removed instead. These are the
races the role RPCs closed with UpdateRole.
OIDCProviderStore gains UpdateProvider with the same contract: memory
under its lock, filer as a conditional write (IF_ENTRY_EQUAL /
IF_NOT_EXISTS) or conditional delete retrying a changed entry.
PutOIDCProvider merges the fields the request cannot carry against the
record as it is written; DeleteOIDCProvider aborts rather than delete a
record replaced meanwhile.
isRoleWriteConflict is renamed isEntryWriteConflict — the conditional-
write check is shared by both stores now.
* iam: guard PutRole against a nil credential manager, fix its doc comment
PutRole read attached policies through s.credentialManager without the
nil check its sibling handlers make, so a server built without one
panicked on a PutRole naming a policy. It now fails the call as
FailedPrecondition like the others.
The doc comment also had the store/static precedence backwards: a stored
role shadows a same-named config-file role (as the overlay serves it),
not the other way around.
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
742 lines
24 KiB
Go
742 lines
24 KiB
Go
package integration
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"regexp"
|
|
"strings"
|
|
"sync"
|
|
"time"
|
|
|
|
"github.com/karlseguin/ccache/v2"
|
|
"github.com/seaweedfs/seaweedfs/weed/glog"
|
|
"github.com/seaweedfs/seaweedfs/weed/iam/policy"
|
|
"github.com/seaweedfs/seaweedfs/weed/pb"
|
|
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
|
|
"google.golang.org/grpc"
|
|
"google.golang.org/grpc/codes"
|
|
"google.golang.org/grpc/status"
|
|
)
|
|
|
|
// ErrRoleNotFound is wrapped by every RoleStore's GetRole when the role does
|
|
// not exist, so callers can tell a missing role from a store that could not
|
|
// be read.
|
|
var ErrRoleNotFound = errors.New("role not found")
|
|
|
|
// ErrRoleStatic refuses a change to a role defined in the server's IAM config
|
|
// file: change it there instead.
|
|
var ErrRoleStatic = errors.New("role is defined in the IAM config file")
|
|
|
|
// ErrRoleExists is returned by an UpdateRole whose update refuses a role that
|
|
// already exists (CreateRole).
|
|
var ErrRoleExists = errors.New("role already exists")
|
|
|
|
// RoleUpdate computes a role's new definition from its current one, nil when
|
|
// the role does not exist. It returns nil to delete the role, and an error to
|
|
// leave it unchanged. It may run more than once: it is called again with the
|
|
// fresh definition when another writer changed the role in between.
|
|
type RoleUpdate func(current *RoleDefinition) (*RoleDefinition, error)
|
|
|
|
// maxRoleUpdateAttempts bounds UpdateRole's retries under contention.
|
|
const maxRoleUpdateAttempts = 10
|
|
|
|
// errRoleUpdateContended is returned when the role kept changing under
|
|
// UpdateRole for maxRoleUpdateAttempts reads.
|
|
var errRoleUpdateContended = errors.New("role changed concurrently; retry")
|
|
|
|
// RoleStore defines the interface for storing IAM role definitions
|
|
type RoleStore interface {
|
|
// StoreRole stores a role definition (filerAddress ignored for memory stores)
|
|
StoreRole(ctx context.Context, filerAddress string, roleName string, role *RoleDefinition) error
|
|
|
|
// GetRole retrieves a role definition (filerAddress ignored for memory stores)
|
|
GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error)
|
|
|
|
// ListRoles lists all role names (filerAddress ignored for memory stores)
|
|
ListRoles(ctx context.Context, filerAddress string) ([]string, error)
|
|
|
|
// DeleteRole deletes a role definition (filerAddress ignored for memory stores)
|
|
DeleteRole(ctx context.Context, filerAddress string, roleName string) error
|
|
|
|
// UpdateRole replaces a role with update's result, atomically: the write
|
|
// lands only if the role is still as update saw it, absent included, and
|
|
// update is retried against the current role otherwise. Every change the
|
|
// IAM API and the filer IAM service make to a role goes through it, so
|
|
// writers on different servers neither lose each other's changes nor bring
|
|
// back a role deleted in between.
|
|
UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error
|
|
}
|
|
|
|
// MemoryRoleStore implements RoleStore using in-memory storage
|
|
type MemoryRoleStore struct {
|
|
roles map[string]*RoleDefinition
|
|
mutex sync.RWMutex
|
|
}
|
|
|
|
// NewMemoryRoleStore creates a new memory-based role store
|
|
func NewMemoryRoleStore() *MemoryRoleStore {
|
|
return &MemoryRoleStore{
|
|
roles: make(map[string]*RoleDefinition),
|
|
}
|
|
}
|
|
|
|
// StoreRole stores a role definition in memory (filerAddress ignored for memory store)
|
|
func (m *MemoryRoleStore) StoreRole(ctx context.Context, filerAddress string, roleName string, role *RoleDefinition) error {
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
if role == nil {
|
|
return fmt.Errorf("role cannot be nil")
|
|
}
|
|
|
|
m.mutex.Lock()
|
|
defer m.mutex.Unlock()
|
|
|
|
// Deep copy the role to prevent external modifications
|
|
m.roles[roleName] = copyRoleDefinition(role)
|
|
return nil
|
|
}
|
|
|
|
// GetRole retrieves a role definition from memory (filerAddress ignored for memory store)
|
|
func (m *MemoryRoleStore) GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error) {
|
|
if roleName == "" {
|
|
return nil, fmt.Errorf("role name cannot be empty")
|
|
}
|
|
|
|
m.mutex.RLock()
|
|
defer m.mutex.RUnlock()
|
|
|
|
role, exists := m.roles[roleName]
|
|
if !exists {
|
|
return nil, fmt.Errorf("%w: %s", ErrRoleNotFound, roleName)
|
|
}
|
|
|
|
// Return a copy to prevent external modifications
|
|
return copyRoleDefinition(role), nil
|
|
}
|
|
|
|
// ListRoles lists all role names in memory (filerAddress ignored for memory store)
|
|
func (m *MemoryRoleStore) ListRoles(ctx context.Context, filerAddress string) ([]string, error) {
|
|
m.mutex.RLock()
|
|
defer m.mutex.RUnlock()
|
|
|
|
names := make([]string, 0, len(m.roles))
|
|
for name := range m.roles {
|
|
names = append(names, name)
|
|
}
|
|
|
|
return names, nil
|
|
}
|
|
|
|
// DeleteRole deletes a role definition from memory (filerAddress ignored for memory store)
|
|
func (m *MemoryRoleStore) DeleteRole(ctx context.Context, filerAddress string, roleName string) error {
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
|
|
m.mutex.Lock()
|
|
defer m.mutex.Unlock()
|
|
|
|
delete(m.roles, roleName)
|
|
return nil
|
|
}
|
|
|
|
// UpdateRole applies update under the store's lock (filerAddress ignored for
|
|
// memory store).
|
|
func (m *MemoryRoleStore) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error {
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
m.mutex.Lock()
|
|
defer m.mutex.Unlock()
|
|
next, err := update(copyRoleDefinition(m.roles[roleName]))
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if next == nil {
|
|
delete(m.roles, roleName)
|
|
return nil
|
|
}
|
|
m.roles[roleName] = copyRoleDefinition(next)
|
|
return nil
|
|
}
|
|
|
|
// copyRoleDefinition creates a deep copy of a role definition
|
|
func copyRoleDefinition(original *RoleDefinition) *RoleDefinition {
|
|
if original == nil {
|
|
return nil
|
|
}
|
|
|
|
copied := &RoleDefinition{
|
|
RoleName: original.RoleName,
|
|
RoleArn: original.RoleArn,
|
|
Description: original.Description,
|
|
MaxSessionDuration: original.MaxSessionDuration,
|
|
Source: original.Source,
|
|
CreatedAt: original.CreatedAt,
|
|
RoleId: original.RoleId,
|
|
}
|
|
|
|
// Deep copy trust policy if it exists
|
|
if original.TrustPolicy != nil {
|
|
// Use JSON marshaling for deep copy of the complex policy structure
|
|
trustPolicyData, _ := json.Marshal(original.TrustPolicy)
|
|
var trustPolicyCopy policy.PolicyDocument
|
|
json.Unmarshal(trustPolicyData, &trustPolicyCopy)
|
|
copied.TrustPolicy = &trustPolicyCopy
|
|
}
|
|
|
|
// Copy attached policies slice
|
|
if original.AttachedPolicies != nil {
|
|
copied.AttachedPolicies = make([]string, len(original.AttachedPolicies))
|
|
copy(copied.AttachedPolicies, original.AttachedPolicies)
|
|
}
|
|
|
|
return copied
|
|
}
|
|
|
|
// roleListPageSize is the number of entries FilerRoleStore.ListRoles asks the
|
|
// filer for per page.
|
|
const roleListPageSize = 1000
|
|
|
|
// FilerRoleStore implements RoleStore using SeaweedFS filer
|
|
type FilerRoleStore struct {
|
|
grpcDialOption grpc.DialOption
|
|
basePath string
|
|
filerAddressProvider func() string
|
|
}
|
|
|
|
// NewFilerRoleStore creates a new filer-based role store
|
|
func NewFilerRoleStore(config map[string]interface{}, filerAddressProvider func() string) (*FilerRoleStore, error) {
|
|
store := &FilerRoleStore{
|
|
basePath: "/etc/iam/roles", // Default path for role storage - aligned with /etc/ convention
|
|
filerAddressProvider: filerAddressProvider,
|
|
}
|
|
|
|
// Parse configuration - only basePath and other settings, NOT filerAddress
|
|
if config != nil {
|
|
if basePath, ok := config["basePath"].(string); ok && basePath != "" {
|
|
store.basePath = strings.TrimSuffix(basePath, "/")
|
|
}
|
|
}
|
|
|
|
glog.V(2).Infof("Initialized FilerRoleStore with basePath %s", store.basePath)
|
|
|
|
return store, nil
|
|
}
|
|
|
|
// StoreRole stores a role definition in filer
|
|
func (f *FilerRoleStore) StoreRole(ctx context.Context, filerAddress string, roleName string, role *RoleDefinition) error {
|
|
// Use provider function if filerAddress is not provided
|
|
if filerAddress == "" && f.filerAddressProvider != nil {
|
|
filerAddress = f.filerAddressProvider()
|
|
}
|
|
if filerAddress == "" {
|
|
return fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
if role == nil {
|
|
return fmt.Errorf("role cannot be nil")
|
|
}
|
|
|
|
// Serialize role to JSON
|
|
roleData, err := json.MarshalIndent(role, "", " ")
|
|
if err != nil {
|
|
return fmt.Errorf("failed to serialize role: %v", err)
|
|
}
|
|
|
|
rolePath := f.getRolePath(roleName)
|
|
|
|
// Store in filer
|
|
return f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
|
|
request := &filer_pb.CreateEntryRequest{
|
|
Directory: f.basePath,
|
|
Entry: &filer_pb.Entry{
|
|
Name: f.getRoleFileName(roleName),
|
|
IsDirectory: false,
|
|
Attributes: &filer_pb.FuseAttributes{
|
|
Mtime: time.Now().Unix(),
|
|
Crtime: time.Now().Unix(),
|
|
FileMode: uint32(0600), // Read/write for owner only
|
|
Uid: uint32(0),
|
|
Gid: uint32(0),
|
|
},
|
|
Content: roleData,
|
|
},
|
|
}
|
|
|
|
glog.V(3).Infof("Storing role %s at %s", roleName, rolePath)
|
|
_, err := client.CreateEntry(ctx, request)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to store role %s: %v", roleName, err)
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|
|
|
|
// UpdateRole reads the role's entry, applies update, and writes the result on
|
|
// the condition that the entry is unchanged since the read — absent, when the
|
|
// role did not exist — so the filer refuses a write racing another writer's
|
|
// change or delete, and update is applied again to what that writer left. A
|
|
// delete is made on the same condition, so it removes the role update saw and
|
|
// not one written after it.
|
|
func (f *FilerRoleStore) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error {
|
|
if filerAddress == "" && f.filerAddressProvider != nil {
|
|
filerAddress = f.filerAddressProvider()
|
|
}
|
|
if filerAddress == "" {
|
|
return fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
return f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
|
|
for attempt := 0; attempt < maxRoleUpdateAttempts; attempt++ {
|
|
var entry *filer_pb.Entry
|
|
var current *RoleDefinition
|
|
resp, err := filer_pb.LookupEntry(ctx, client, &filer_pb.LookupDirectoryEntryRequest{
|
|
Directory: f.basePath,
|
|
Name: f.getRoleFileName(roleName),
|
|
})
|
|
switch {
|
|
case errors.Is(err, filer_pb.ErrNotFound):
|
|
case err != nil:
|
|
return fmt.Errorf("lookup role %s: %w", roleName, err)
|
|
case resp.Entry != nil:
|
|
entry = resp.Entry
|
|
current = &RoleDefinition{}
|
|
if err := json.Unmarshal(entry.Content, current); err != nil {
|
|
return fmt.Errorf("failed to deserialize role %s: %v", roleName, err)
|
|
}
|
|
}
|
|
|
|
next, err := update(current)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if next == nil {
|
|
if entry == nil {
|
|
return nil
|
|
}
|
|
deleted, err := f.deleteRoleEntryIfUnchanged(ctx, client, entry)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to delete role %s: %w", roleName, err)
|
|
}
|
|
if !deleted {
|
|
glog.V(3).Infof("Role %s changed before its delete; retrying", roleName)
|
|
continue
|
|
}
|
|
return nil
|
|
}
|
|
roleData, err := json.MarshalIndent(next, "", " ")
|
|
if err != nil {
|
|
return fmt.Errorf("failed to serialize role: %v", err)
|
|
}
|
|
|
|
clause := &filer_pb.WriteCondition_Clause{Kind: filer_pb.WriteCondition_IF_NOT_EXISTS}
|
|
if entry != nil {
|
|
clause = &filer_pb.WriteCondition_Clause{Kind: filer_pb.WriteCondition_IF_ENTRY_EQUAL, ExpectedEntry: entry}
|
|
}
|
|
now := time.Now().Unix()
|
|
created, err := client.CreateEntry(ctx, &filer_pb.CreateEntryRequest{
|
|
Directory: f.basePath,
|
|
Entry: &filer_pb.Entry{
|
|
Name: f.getRoleFileName(roleName),
|
|
Attributes: &filer_pb.FuseAttributes{
|
|
Mtime: now,
|
|
Crtime: now,
|
|
FileMode: uint32(0600),
|
|
},
|
|
Content: roleData,
|
|
},
|
|
Condition: &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{clause}},
|
|
})
|
|
if isEntryWriteConflict(created, err) {
|
|
glog.V(3).Infof("Role %s changed during update; retrying", roleName)
|
|
continue
|
|
}
|
|
if err != nil {
|
|
return fmt.Errorf("failed to store role %s: %v", roleName, err)
|
|
}
|
|
if created.Error != "" {
|
|
return fmt.Errorf("failed to store role %s: %s", roleName, created.Error)
|
|
}
|
|
return nil
|
|
}
|
|
return fmt.Errorf("update role %s: %w", roleName, errRoleUpdateContended)
|
|
})
|
|
}
|
|
|
|
// deleteRoleEntryIfUnchanged deletes the role's entry if it still equals
|
|
// entry, reporting false when it changed. The delete is routed and locked as
|
|
// the conditional CreateEntry of the same path is, so the two serialize.
|
|
func (f *FilerRoleStore) deleteRoleEntryIfUnchanged(ctx context.Context, client filer_pb.SeaweedFilerClient, entry *filer_pb.Entry) (bool, error) {
|
|
fullPath := f.basePath + "/" + entry.Name
|
|
resp, err := client.ObjectTransaction(ctx, &filer_pb.ObjectTransactionRequest{
|
|
LockKey: fullPath,
|
|
RouteKey: s3_constants.ObjectWriteRouteKeyPrefix + fullPath,
|
|
Condition: &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{{
|
|
Kind: filer_pb.WriteCondition_IF_ENTRY_EQUAL, ExpectedEntry: entry,
|
|
}}},
|
|
Mutations: []*filer_pb.ObjectMutation{{
|
|
Type: filer_pb.ObjectMutation_DELETE, Directory: f.basePath, Name: entry.Name, IsDeleteData: true,
|
|
}},
|
|
})
|
|
if err != nil {
|
|
if status.Code(err) == codes.FailedPrecondition {
|
|
return false, nil
|
|
}
|
|
return false, err
|
|
}
|
|
if resp.ErrorCode == filer_pb.FilerError_PRECONDITION_FAILED {
|
|
return false, nil
|
|
}
|
|
if resp.Error != "" {
|
|
return false, errors.New(resp.Error)
|
|
}
|
|
return true, nil
|
|
}
|
|
|
|
// isEntryWriteConflict reports a write the filer refused because its condition
|
|
// no longer held: in the response, or as FailedPrecondition when the write
|
|
// was forwarded to the entry's owner filer.
|
|
func isEntryWriteConflict(resp *filer_pb.CreateEntryResponse, err error) bool {
|
|
if err != nil {
|
|
return status.Code(err) == codes.FailedPrecondition
|
|
}
|
|
return resp != nil && resp.ErrorCode == filer_pb.FilerError_PRECONDITION_FAILED
|
|
}
|
|
|
|
// GetRole retrieves a role definition from filer
|
|
func (f *FilerRoleStore) GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error) {
|
|
// Use provider function if filerAddress is not provided
|
|
if filerAddress == "" && f.filerAddressProvider != nil {
|
|
filerAddress = f.filerAddressProvider()
|
|
}
|
|
if filerAddress == "" {
|
|
return nil, fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
if roleName == "" {
|
|
return nil, fmt.Errorf("role name cannot be empty")
|
|
}
|
|
|
|
var roleData []byte
|
|
err := f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
|
|
request := &filer_pb.LookupDirectoryEntryRequest{
|
|
Directory: f.basePath,
|
|
Name: f.getRoleFileName(roleName),
|
|
}
|
|
|
|
glog.V(3).Infof("Looking up role %s", roleName)
|
|
response, err := filer_pb.LookupEntry(ctx, client, request)
|
|
if errors.Is(err, filer_pb.ErrNotFound) {
|
|
return fmt.Errorf("%w: %s", ErrRoleNotFound, roleName)
|
|
}
|
|
if err != nil {
|
|
return fmt.Errorf("lookup role %s: %w", roleName, err)
|
|
}
|
|
|
|
if response.Entry == nil {
|
|
return fmt.Errorf("%w: %s", ErrRoleNotFound, roleName)
|
|
}
|
|
|
|
roleData = response.Entry.Content
|
|
return nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// Deserialize role from JSON
|
|
var role RoleDefinition
|
|
if err := json.Unmarshal(roleData, &role); err != nil {
|
|
return nil, fmt.Errorf("failed to deserialize role: %v", err)
|
|
}
|
|
|
|
return &role, nil
|
|
}
|
|
|
|
// ListRoles lists all role names in filer
|
|
func (f *FilerRoleStore) ListRoles(ctx context.Context, filerAddress string) ([]string, error) {
|
|
// Use provider function if filerAddress is not provided
|
|
if filerAddress == "" && f.filerAddressProvider != nil {
|
|
filerAddress = f.filerAddressProvider()
|
|
}
|
|
if filerAddress == "" {
|
|
return nil, fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
|
|
// Page through the directory, and fail on a broken stream rather than
|
|
// return what arrived: a truncated list would let DeletePolicy miss a role
|
|
// that still attaches the policy (RolesAttachingPolicy).
|
|
var roleNames []string
|
|
err := f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
|
|
glog.V(3).Infof("Listing roles in %s", f.basePath)
|
|
startFrom := ""
|
|
for {
|
|
stream, err := client.ListEntries(ctx, &filer_pb.ListEntriesRequest{
|
|
Directory: f.basePath,
|
|
StartFromFileName: startFrom,
|
|
Limit: roleListPageSize,
|
|
})
|
|
if err != nil {
|
|
return err
|
|
}
|
|
received := 0
|
|
for {
|
|
resp, err := stream.Recv()
|
|
if errors.Is(err, io.EOF) {
|
|
break
|
|
}
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if resp.Entry == nil {
|
|
continue
|
|
}
|
|
received++
|
|
startFrom = resp.Entry.Name
|
|
if resp.Entry.IsDirectory {
|
|
continue
|
|
}
|
|
if name, ok := strings.CutSuffix(resp.Entry.Name, ".json"); ok {
|
|
roleNames = append(roleNames, name)
|
|
}
|
|
}
|
|
if received < roleListPageSize {
|
|
return nil
|
|
}
|
|
}
|
|
})
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to list roles: %w", err)
|
|
}
|
|
|
|
return roleNames, nil
|
|
}
|
|
|
|
// DeleteRole deletes a role definition from filer
|
|
func (f *FilerRoleStore) DeleteRole(ctx context.Context, filerAddress string, roleName string) error {
|
|
// Use provider function if filerAddress is not provided
|
|
if filerAddress == "" && f.filerAddressProvider != nil {
|
|
filerAddress = f.filerAddressProvider()
|
|
}
|
|
if filerAddress == "" {
|
|
return fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
if roleName == "" {
|
|
return fmt.Errorf("role name cannot be empty")
|
|
}
|
|
|
|
return f.withFilerClient(filerAddress, func(client filer_pb.SeaweedFilerClient) error {
|
|
request := &filer_pb.DeleteEntryRequest{
|
|
Directory: f.basePath,
|
|
Name: f.getRoleFileName(roleName),
|
|
IsDeleteData: true,
|
|
}
|
|
|
|
glog.V(3).Infof("Deleting role %s", roleName)
|
|
resp, err := client.DeleteEntry(ctx, request)
|
|
if err != nil {
|
|
if strings.Contains(err.Error(), "not found") {
|
|
return nil // Idempotent: deletion of non-existent role is successful
|
|
}
|
|
return fmt.Errorf("failed to delete role %s: %v", roleName, err)
|
|
}
|
|
|
|
if resp.Error != "" {
|
|
if strings.Contains(resp.Error, "not found") {
|
|
return nil // Idempotent: deletion of non-existent role is successful
|
|
}
|
|
return fmt.Errorf("failed to delete role %s: %s", roleName, resp.Error)
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|
|
|
|
// Helper methods for FilerRoleStore
|
|
|
|
func (f *FilerRoleStore) getRoleFileName(roleName string) string {
|
|
return roleName + ".json"
|
|
}
|
|
|
|
func (f *FilerRoleStore) getRolePath(roleName string) string {
|
|
return f.basePath + "/" + f.getRoleFileName(roleName)
|
|
}
|
|
|
|
func (f *FilerRoleStore) withFilerClient(filerAddress string, fn func(filer_pb.SeaweedFilerClient) error) error {
|
|
if filerAddress == "" {
|
|
return fmt.Errorf("filer address is required for FilerRoleStore")
|
|
}
|
|
return pb.WithGrpcFilerClient(false, 0, pb.ServerAddress(filerAddress), f.grpcDialOption, fn)
|
|
}
|
|
|
|
// CachedFilerRoleStore implements RoleStore with TTL caching on top of FilerRoleStore
|
|
type CachedFilerRoleStore struct {
|
|
filerStore *FilerRoleStore
|
|
cache *ccache.Cache
|
|
listCache *ccache.Cache
|
|
ttl time.Duration
|
|
listTTL time.Duration
|
|
}
|
|
|
|
// CachedFilerRoleStoreConfig holds configuration for the cached role store
|
|
type CachedFilerRoleStoreConfig struct {
|
|
BasePath string `json:"basePath,omitempty"`
|
|
TTL string `json:"ttl,omitempty"` // e.g., "5m", "1h"
|
|
ListTTL string `json:"listTtl,omitempty"` // e.g., "1m", "30s"
|
|
MaxCacheSize int `json:"maxCacheSize,omitempty"` // Maximum number of cached roles
|
|
}
|
|
|
|
// RolesAttachingPolicy returns the names of the roles that attach the policy.
|
|
// A policy is attached to a role by name, so deleting it while attached would
|
|
// let a policy created later under the same name take effect on the role;
|
|
// callers refuse the delete instead, as AWS does (DeleteConflict).
|
|
//
|
|
// It sees the stored roles, which every server sharing the store sees, and
|
|
// this server's config-file roles. A role defined only in another server's
|
|
// config file is invisible here: servers sharing a role store may have
|
|
// different config files, but a config-file role that attaches a managed
|
|
// (API-created) policy is protected only on the servers whose file defines it.
|
|
// Keep such roles in every server's file, or attach only config-file policies
|
|
// to config-file roles.
|
|
func RolesAttachingPolicy(ctx context.Context, store RoleStore, policyName string) ([]string, error) {
|
|
names, err := store.ListRoles(ctx, "")
|
|
if err != nil {
|
|
return nil, fmt.Errorf("list roles: %w", err)
|
|
}
|
|
var attaching []string
|
|
for _, name := range names {
|
|
role, err := store.GetRole(ctx, "", name)
|
|
if errors.Is(err, ErrRoleNotFound) {
|
|
continue // deleted between list and read
|
|
}
|
|
if err != nil {
|
|
return nil, fmt.Errorf("get role %s: %w", name, err)
|
|
}
|
|
for _, attached := range role.AttachedPolicies {
|
|
if attached == policyName {
|
|
attaching = append(attaching, name)
|
|
break
|
|
}
|
|
}
|
|
}
|
|
return attaching, nil
|
|
}
|
|
|
|
// MaxManagedPoliciesPerRole caps the managed policies attached to one role,
|
|
// as AWS's default quota does (and as MaxManagedPoliciesPerUser does for users).
|
|
const MaxManagedPoliciesPerRole = 10
|
|
|
|
var roleNamePattern = regexp.MustCompile(`^[\w+=,.@-]{1,64}$`)
|
|
|
|
// ValidateRoleName checks a role name against AWS's rules. A role is stored as
|
|
// <name>.json in the filer, so the rules also keep a name from leaving the
|
|
// role store's directory.
|
|
func ValidateRoleName(name string) error {
|
|
if !roleNamePattern.MatchString(name) {
|
|
return fmt.Errorf("invalid role name %q: must be 1-64 characters of letters, digits and +=,.@_-", name)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// staticRoleOverlay serves the IAM config file's roles from memory beside a
|
|
// persistent store, which never holds them (see IAMManager.LoadStaticRoles).
|
|
// A role stored under the same name takes precedence, as a stored OIDC
|
|
// provider does over a config-file one; deleting it brings the config-file
|
|
// role back. A config-file role itself cannot be stored over or deleted.
|
|
type staticRoleOverlay struct {
|
|
static map[string]*RoleDefinition
|
|
inner RoleStore
|
|
}
|
|
|
|
// storedRoleExists reports whether the store itself holds the role.
|
|
func (o *staticRoleOverlay) storedRoleExists(ctx context.Context, filerAddress, roleName string) (bool, error) {
|
|
_, err := o.inner.GetRole(ctx, filerAddress, roleName)
|
|
if errors.Is(err, ErrRoleNotFound) {
|
|
return false, nil
|
|
}
|
|
return err == nil, err
|
|
}
|
|
|
|
func (o *staticRoleOverlay) StoreRole(ctx context.Context, filerAddress string, roleName string, role *RoleDefinition) error {
|
|
if _, ok := o.static[roleName]; ok {
|
|
stored, err := o.storedRoleExists(ctx, filerAddress, roleName)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if !stored {
|
|
return fmt.Errorf("%w: %s", ErrRoleStatic, roleName)
|
|
}
|
|
}
|
|
return o.inner.StoreRole(ctx, filerAddress, roleName, role)
|
|
}
|
|
|
|
// UpdateRole refuses to create a stored role over a config-file role, as
|
|
// StoreRole does; a stored role of that name may be changed.
|
|
func (o *staticRoleOverlay) UpdateRole(ctx context.Context, filerAddress string, roleName string, update RoleUpdate) error {
|
|
_, static := o.static[roleName]
|
|
return o.inner.UpdateRole(ctx, filerAddress, roleName, func(current *RoleDefinition) (*RoleDefinition, error) {
|
|
if current == nil && static {
|
|
return nil, fmt.Errorf("%w: %s", ErrRoleStatic, roleName)
|
|
}
|
|
return update(current)
|
|
})
|
|
}
|
|
|
|
func (o *staticRoleOverlay) GetRole(ctx context.Context, filerAddress string, roleName string) (*RoleDefinition, error) {
|
|
role, err := o.inner.GetRole(ctx, filerAddress, roleName)
|
|
if errors.Is(err, ErrRoleNotFound) {
|
|
if static, ok := o.static[roleName]; ok {
|
|
return copyRoleDefinition(static), nil
|
|
}
|
|
}
|
|
return role, err
|
|
}
|
|
|
|
func (o *staticRoleOverlay) ListRoles(ctx context.Context, filerAddress string) ([]string, error) {
|
|
names, err := o.inner.ListRoles(ctx, filerAddress)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
seen := make(map[string]bool, len(names))
|
|
for _, name := range names {
|
|
seen[name] = true
|
|
}
|
|
for name := range o.static {
|
|
if !seen[name] {
|
|
names = append(names, name)
|
|
}
|
|
}
|
|
return names, nil
|
|
}
|
|
|
|
func (o *staticRoleOverlay) DeleteRole(ctx context.Context, filerAddress string, roleName string) error {
|
|
if _, ok := o.static[roleName]; ok {
|
|
stored, err := o.storedRoleExists(ctx, filerAddress, roleName)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if !stored {
|
|
return fmt.Errorf("%w: %s", ErrRoleStatic, roleName)
|
|
}
|
|
}
|
|
return o.inner.DeleteRole(ctx, filerAddress, roleName)
|
|
}
|
|
|
|
// ClearCache forwards cache invalidation to the store underneath.
|
|
func (o *staticRoleOverlay) ClearCache() {
|
|
if cached, ok := o.inner.(interface{ ClearCache() }); ok {
|
|
cached.ClearCache()
|
|
}
|
|
}
|