1 Commits

Include renames

Author	SHA1	Message	Date
3 people	30069f3e45	iam: manage OIDC providers and roles over the filer IAM gRPC service (#11523) ... * s3/iam: manage roles through the IAM API, with an opt-in persistent role store Roles could only come from the IAM config file: the S3 server pinned the role store to memory and the embedded IAM API had no role actions, so a role could not be created, retrusted or revoked without editing the file and restarting every gateway. Role store - Read the `roleStore` key (the IAMConfig field already existed). With an IAM config file the default stays memory; with none it is the filer, as for OIDC providers, so zero-config clusters keep runtime-created roles. - Roles from the IAM config file never go into a persistent role store, which outlives the file and may be shared by S3 servers with different files. They are served from memory beneath the store, as OIDC providers are: a stored role of the same name takes precedence, and deleting it restores the file's. A config-file role cannot be changed or deleted through the API (UnmodifiableEntity), and removing one from the file removes it at the next start. An in-memory store holds them as records, as before. They have no creation time, so CreateDate is omitted rather than reporting when this server started. SetRoleStore installs a store the same way, so a store set after startup keeps the config-file roles, as SetOIDCProviderStore does for providers. - Watch /etc/iam/roles and drop the cached role definitions on change. The cached filer store otherwise serves a peer's stale role for up to its 5m TTL, which keeps a revoked trust policy in force on the other gateways. - Role stores wrap ErrRoleNotFound for a missing role; the filer store used to report any failed lookup as "role not found". CreateRole proceeds only on a confirmed absence, so an unreadable store cannot let it write over an existing role. IAM actions - CreateRole, GetRole, ListRoles, DeleteRole, UpdateAssumeRolePolicy, AttachRolePolicy, DetachRolePolicy, ListAttachedRolePolicies. The reads are allowed in read-only mode. - A role defined in the config file is reloaded from it at every start, so changing or deleting it through the API is refused (UnmodifiableEntity) rather than silently reverted. - DeleteRole with policies attached is refused (DeleteConflict), as on AWS. - Role names follow AWS's rules ([\w+=,.@-]{1,64}); a role is stored as <name>.json in the filer, so this also keeps a name from leaving the role store's directory. At most 10 managed policies per role (AWS's default quota; MaxManagedPoliciesPerUser is 10 too), LimitExceeded beyond. - DeletePolicy is refused (DeleteConflict) while a role attaches the policy, as it already is for users and groups: roles attach policies by name, so a policy created later under the deleted one's name would otherwise take effect on the role. - Role paths other than "/" and role tags are not stored, so they are refused rather than dropped. Role IDs and sessions - Roles get a unique RoleId when first stored (random, AWS AROA form), kept across updates; a config-file role gets a stable ID derived from its name, since it is created again at every start. - Sessions issued through AssumeRoleWithWebIdentity, AssumeRoleWithCredentials and AssumeRole carry the role's ID (claim "rid"), and a request under a role whose current ID differs is denied. Resolving a session's policies by role name let a session outlive its role: once a role was deleted, a role later created under the same name — with a different trust policy and different policies — revived every unexpired session of the old one with the new role's permissions. Sessions issued before this change carry no ID and are unaffected until they expire. Integration test (test/s3/iam, run with `make start-services`): TestWebIdentityWithProviderAndRoleManagedThroughIAMAPI configures an OIDC provider, a managed policy and a role entirely through the IAM API against a JWKS served by the test, then checks the trusted subject gets credentials scoped to the attached policy; another subject, a token signed by another key, an unsigned token and a token for another audience are refused; and UpdateAssumeRolePolicy moves the trust at once. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * iam: manage OIDC providers and roles over the filer IAM gRPC service The filer's SeaweedIdentityAccessManagement service covers users, access keys, policies and service accounts, but not the OIDC providers and roles that STS web-identity federation needs. A controller that already manages IAM over this service (seaweedfs-operator's S3OIDCProvider) has no transport for them; its swadmin client returns ErrOIDCNotWired and names this as the recommended fix. - PutOIDCProvider / GetOIDCProvider / DeleteOIDCProvider / ListOIDCProviders and PutRole / GetRole / DeleteRole / ListRoles. - They write the filer-backed stores at their default paths, which S3 servers read when configured with a filer-typed "oidcProviderStore" and "roleStore"; the S3 servers' /etc/iam subscription applies changes without a restart. - Put is an upsert, so a controller can reconcile to it. Deleting a provider or role that does not exist returns NotFound, as DeleteUser does for a user; clients treat that as already deleted. The provider's account ID travels in the request, since the filer does not know the STS accountId. - PutRole applies the IAM API's rules: AWS role names, at most 10 managed policies. - An S3 server serves the roles and providers of its own IAM config file ahead of the store, so a stored entry with the same name has no effect on that server. - PutRole keeps a replaced role's RoleId and gives a role created anew a fresh one, so sessions of a deleted role do not carry over to a later role of the same name. - DeletePolicy returns FailedPrecondition while a role attaches the policy (see the IAM API's DeleteConflict in the previous change). DeletePolicy on this service still does not check user attachments, which predates this. - PutOIDCProvider requires an https issuer (http only for a loopback host): STS fetches the issuer's signing keys from it, so over plain HTTP anyone on the network path could substitute their own. - The OIDC provider and role RPCs refuse to run on an unauthenticated service (FailedPrecondition until jwt.filer_signing.key is set). Users and policies keep the service's opt-in auth, but these grant STS access outright: otherwise anyone who can reach the port could register an issuer they control, create a role trusting it, and exchange a token for S3 credentials. The filer's unauthenticated notice becomes a warning that says so. - A store that cannot be read is Unavailable, never "not found", so a Put never writes over an entry it could not see. - Validation is shared with the IAM API through PrepareRoleDefinition and PrepareOIDCProviderRecord. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * s3/iam: bind every role session to its role, and change roles atomically Review follow-ups. Session binding - The role-ID check ran only when a session carried no policy names, and AssumeRole embeds the role's attached policies, so those sessions kept their permissions after the role was deleted or recreated. The check now runs for every session carrying a role ID, before policy selection. - A named role that cannot be resolved at issuance gets no session, instead of one with no role ID (which nothing binds). - A config-file role's ID is derived from its name and trust policy, not the name alone: a different role put in the file under the same name gets a new ID, while an unchanged role keeps its sessions across restarts. Role writes - RoleStore gains UpdateRole, a read-modify-write that lands only if the role is unchanged since the read, and otherwise re-reads and retries. The filer store uses the filer's write conditions (IF_NOT_EXISTS for a new role, IF_ENTRY_EQUAL otherwise). CreateRole, UpdateAssumeRolePolicy and Attach/DetachRolePolicy all go through it, so two gateways no longer overwrite each other's changes, a change racing a delete no longer writes the role back, and of two concurrent creates one gets EntityAlreadyExists. - The filer store's ListRoles pages past 1,000 entries and fails on a broken stream instead of returning what arrived, so DeletePolicy's attachment check sees every role. ListRoles skips a role deleted between listing and reading it. - CreateRole validates first; a failed write is ServiceFailure, not InvalidInput. Any Tags.* parameter is refused, not only the first key. - ExecuteAction's skipPersist covers the S3ApiConfiguration only; the comment now says so. Role and OIDC provider actions write their own stores. Each fix has a test that fails without it. Against a real filer with two gateways, concurrent AttachRolePolicy calls lost 1-4 of 8 attachments per run before this change and none after. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * iam: PutRole changes roles atomically and checks its ARN; https issuers' keys stay on https Review follow-ups on top of the role-store changes. - PutRole goes through RoleStore.UpdateRole, so the decision to keep an existing role's ID or mint a new one is made against the role as it is when written. A PutRole racing a DeleteRole can no longer write the deleted role back with its old ID, which would revive its sessions. A failed store read or write is Unavailable. - PutRole refuses a role_arn that does not name the role: STS resolves a role by the name in the ARN it is given. - PutOIDCProvider requires an https issuer, but discovery could still name a plain-http jwks_uri, and a key fetch could be redirected to http. For an https issuer, a non-https jwks_uri from discovery is refused (the issuer's own /.well-known/jwks.json is used instead), and the client that fetches discovery and keys refuses any https-to-http redirect. An operator-set jwksUri is left as configured. Each has a test that fails without its guard. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * s3/iam: one role snapshot per decision; DeleteRole is atomic; watch a custom role store path Review follow-ups. - Authorization evaluates the policies of the role definition the session's binding was checked against, instead of reading the role again: a role replaced in between cannot lend a session its policies. - AssumeRole and AssumeRoleWithLDAPIdentity issue the session from the definition whose trust admits the caller (IAMManager.ResolveRoleForPrincipal), and take its ID, duration cap and embedded policies from that same definition. A role replaced after the caller's trust check by one that does not trust the caller now yields AccessDenied, not a session bound to the replacement. - A RoleUpdate that returns nil deletes the role, on the same condition as a write: the filer store deletes with ObjectTransaction on IF_ENTRY_EQUAL, routed and locked like the conditional CreateEntry. DeleteRole decides against the role it deletes, so a policy attached meanwhile on another server is a DeleteConflict, and a delete never removes a role written after its check. - S3 servers watch the role store's configured basePath, not only /etc/iam/roles, so a custom path also drops peers' cached roles on change. Each has a test that fails without it. Live against a real filer: DeleteRole refuses while a policy is attached and removes the entry once detached; all test/s3/iam CI stages pass. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * s3/iam: state which roles DeletePolicy's attachment check can see RolesAttachingPolicy sees the stored roles and this server's config-file roles. A role defined only in another server's IAM config file is invisible to it, so a config-file role that attaches a managed policy is protected only on the servers whose file defines it. The doc comment now says so and how to avoid it: keep such roles in every server's file, or attach only config-file policies to config-file roles. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * iam: note that a role store set after startup is not watched for peer changes S3 servers build their metadata watch list once, at startup, from the role store installed then. SetRoleStore's doc now says that a filer-backed store installed later with a different basePath is not watched, so peers' changes to it reach this server's cached roles only when the cache expires. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * iam: DeleteRole deletes only the role it saw; issuer URLs are bare Review follow-ups. - The filer IAM service's DeleteRole looked the role up, then deleted by name, so a PutRole landing in between had its new definition deleted. It now deletes through RoleStore.UpdateRole, conditional on the entry it read. If the role was replaced meanwhile, it returns Aborted rather than deleting the replacement, and the caller decides again. - PutOIDCProvider refuses an issuer URL with userinfo, a query or a fragment. The provider's ARN comes from host and path alone, while STS matches a token's iss claim against the stored URL exactly, so such a provider shared the bare issuer's ARN and matched no token. A loopback "localhost" is now matched without regard to case. Both have tests that fail without them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * iam: write OIDC providers atomically over the filer IAM gRPC service PutOIDCProvider read the record, then stored unconditionally; a racing DeleteOIDCProvider left the put's stale read merged into the rewritten record. DeleteOIDCProvider read, then deleted unconditionally; a racing PutOIDCProvider's newer record could be removed instead. These are the races the role RPCs closed with UpdateRole. OIDCProviderStore gains UpdateProvider with the same contract: memory under its lock, filer as a conditional write (IF_ENTRY_EQUAL / IF_NOT_EXISTS) or conditional delete retrying a changed entry. PutOIDCProvider merges the fields the request cannot carry against the record as it is written; DeleteOIDCProvider aborts rather than delete a record replaced meanwhile. isRoleWriteConflict is renamed isEntryWriteConflict — the conditional- write check is shared by both stores now. * iam: guard PutRole against a nil credential manager, fix its doc comment PutRole read attached policies through s.credentialManager without the nil check its sibling handlers make, so a server built without one panicked on a PutRole naming a policy. It now fails the call as FailedPrecondition like the others. The doc comment also had the store/static precedence backwards: a stored role shadows a same-named config-file role (as the overlay serves it), not the other way around. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	2026-10-03 08:36:19 +08:00