s3/iam: persist IAM-managed OIDC providers in the filer, and trust them after a restart

The S3 server's IAM config loader never read the documented
`oidcProviderStore` key, so the OIDC provider store was always in memory:
a provider created with CreateOpenIDConnectProvider lived in one gateway's
process, was lost on restart, and was never seen by peers. The
/etc/iam/oidc-providers metadata subscription refreshed from that empty
in-memory store.

- Read `oidcProviderStore` and pass it to the IAM manager. With an IAM
  config file the default stays memory. With no config file (zero-config
  IAM, as `weed filer -s3` and operator-managed clusters run) it defaults
  to the filer: there is nothing static to shadow, and providers created at
  runtime otherwise vanish on restart.
- With a store that outlives the process, load the STS runtime view from it
  at startup, so providers created on an earlier boot or on a peer are
  trusted without waiting for the next mutation.
- If the store cannot be read at startup (a filer not up yet), the load is
  retried in the background with backoff until it succeeds: the metadata
  subscription reports only later changes, so providers already stored would
  otherwise stay unknown to STS until one of them changed.
- Mark records mirrored from STS.Providers as `source: static-config`, and
  at startup delete such records whose provider has left the config, so
  removing a provider from the config file still revokes it. Records created
  through the IAM API are never pruned.
- The filer store reported every failed lookup, an unreachable filer
  included, as ErrOIDCProviderNotFound, which CreateOIDCProvider reads as
  "free to create". Only a confirmed absence is now not-found.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Khris RichardsonandClaude Opus 5.5 committed 2026-09-28 12:43:50 -07:00
1 parent 150a69fe11
commit 1207c37542
5 files changed
+336 -4

No files matched your search

+14 -1
View File
@@ -1168,7 +1168,10 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str
Policy *policy.PolicyEngineConfig `json:"policy"`
Providers []map[string]interface{} `json:"providers"`
Roles []*integration.RoleDefinition `json:"roles"`
Policies []struct {
// OIDCProviderStore selects where IAM-managed OIDC providers persist.
// Absent, they live in memory and are lost on restart.
OIDCProviderStore *integration.OIDCProviderStoreConfig `json:"oidcProviderStore"`
Policies []struct {
Name string `json:"name"`
Document *policy.PolicyDocument `json:"document"`
} `json:"policies"`
@@ -1213,6 +1216,15 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str
glog.V(1).Infof("Using policy defaults: DefaultEffect=%s, StoreType=%s", configRoot.Policy.DefaultEffect, configRoot.Policy.StoreType)
}
// With no IAM config file there is nothing static for a persisted
// provider to shadow or outlive, so providers created at runtime default
// to the filer, where restarts and peer S3 servers see them. A config
// file keeps the in-memory default unless it sets oidcProviderStore.
oidcProviderStore := configRoot.OIDCProviderStore
if oidcProviderStore == nil && configPath == "" && filerAddressProvider != nil {
oidcProviderStore = &integration.OIDCProviderStoreConfig{StoreType: "filer"}
}
// Create IAM configuration
iamConfig := &integration.IAMConfig{
STS: configRoot.STS,
@@ -1220,6 +1232,7 @@ func loadIAMManagerFromConfig(configPath string, filerAddressProvider func() str
Roles: &integration.RoleStoreConfig{
StoreType: sts.StoreTypeMemory, // Use memory store for JSON config-based setup
},
OIDCProviders: oidcProviderStore,
}
// Apply default signing key if not present in config