Commit Graph
2 Commits
Author SHA1 Message Date
Chris LuandClaude Opus 5.5 c248ab7865 credential: keep IAM changes made during the legacy-file migration (#12614)
* credential: migrate the legacy IAM file without overwriting newer entries

The migration wrote every identity, service account and group from the
snapshot the load read, over whatever the per-entry files held by then, so
a revoke or delete landing in between was undone for good. It now migrates
only the legacy file's own entries, creates each per-entry file only when
none exists, and waits for the next load when the legacy file changed
after the snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: test changes landing between the IAM load and its migration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: keep a per-entry file deleted after the IAM load deleted

An entry the load read from its own file is already migrated. When that
file is removed before the migration runs, the legacy copy must not
recreate it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: test a deleted per-entry file the legacy file also holds

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:17:45 -08:00
Chris Lu 5ff49909a0 fix(s3api/iam): avoid transient AccessDenied from full reloads on single IAM file changes (#11271)
* fix(s3api/iam): fail config snapshot on empty or malformed IAM files

A full IAM reload reads every identity/policy/service-account/group file
from the filer. When an external secrets tool rewrites a file, a reload
that reads it mid-rewrite sees empty or partially-written content. The
identity, policy and service-account loaders silently skipped such files
(``continue``), so the snapshot was missing entries that still existed
on disk. The atomic swap then installed an incomplete identity set while
``isAuthEnabled`` stayed on, denying unrelated clients mid-reload
(#11259).

The group loader and the read-error paths already fail the snapshot in
this situation (a skipped entry reads as deleted). Apply the same
behavior to empty content and unmarshal failures across the identity,
policy, service-account and group loaders, so a transient mid-rewrite
fails the reload (preserving the last known-good state) instead of
silently dropping entries.

* fix(s3api/iam): coalesce burst IAM config reloads through the reload queue

onIamConfigChange did a full synchronous reload for every identity/policy
file change event. When several independently-refreshing credentials
rewrite their files within the same second, that produced a burst of
dozens of back-to-back full reloads, each reading the whole store and
widening the window where a mid-rewrite file is observed (#11259).

Route every IAM config change through the existing coalescing reload
queue (scheduleReload/reloadRetryLoop) instead. A burst of N events now
collapses into a single reload (plus one tail reload for events that
arrived while one was in flight). scheduleReload gains a reason argument
for the existing log line; the reloadRetryLoop already retries failed
reloads, so the per-event failure handoff is no longer needed.

Tests that asserted on the synchronous reload now wire up the queue
(centralized in newTestS3ApiServerWithMemoryIAM) and poll via
waitForIdentity/waitForIdentityGone. Adds TestOnIamConfigChangeCoalescesBurstReloads
showing 50 events coalesce into <=3 reloads.

* fix(s3api/iam): skip non-JSON auxiliary files before failing IAM snapshot

Per review: the multi-file loaders unmarshal every entry in an IAM
directory, so a non-JSON auxiliary file (README, .DS_Store, a migration
backup such as identity.json.old) would hit the new empty/malformed
errors and reject the whole snapshot, blocking all later IAM reloads.

Only *.json files are IAM objects (SeaweedFS writes identities,
policies, service accounts and groups as <name>.json, and other call
sites already gate on the .json suffix). Skip non-.json entries at the
top of each loader loop, before reading content, so auxiliary files are
ignored while empty/malformed .json files still fail the snapshot.

Adds TestLoadConfigurationIgnoresNonJsonAuxiliaryFiles.

* fix(s3api/iam): reject IAM files with empty identifiers and skip aux in listing

Per review:

- ListPolicyNames listed every regular entry in the policies directory as a
  policy name, including non-JSON auxiliary files, but GetPolicy cannot
  retrieve them. Apply the same .json suffix filter used by the loader so
  the list only exposes retrievable policies.

- json.Unmarshal accepts `{}` and unknown fields. The identity and group
  loaders merge by the decoded Name (not the file name), so a `{}` file
  could install an empty-key record and displace a real one; the
  service-account loader accepted an empty Id. Validate Identity.Name,
  Group.Name and ServiceAccount.Id (via validateServiceAccountId) after
  unmarshal and fail the snapshot on empty identifiers.

Adds TestFilerEtcStoreListPolicyNamesSkipsNonJsonAuxiliary and
empty-identifier regression tests for identity, group and service-account
files.
2026-09-11 10:42:19 -07:00