2 Commits

Include renames

Author	SHA1	Message	Date
Chris Lu	c934b5dab6	fix(credential/postgres,s3api/iam): rename safety + pgxutil follow-ups to #9226 (#9230) ... * refactor(util): extract pgx OpenDB + DSN builder into shared pgxutil The postgres filer store had OpenPGXDB plus duplicated key=value DSN assembly across postgres/ and postgres2/. Move the connection helper to weed/util/pgxutil and add BuildDSN so the credential postgres store can land on the same code path. filer/postgres/pgx_conn.go keeps OpenPGXDB as a thin alias so postgres2 keeps building unchanged. * refactor(credential/postgres): use shared pgxutil for connection setup Replace the bespoke fmt.Sprintf DSN + sql.Open("pgx", ...) path with pgxutil.BuildDSN + pgxutil.OpenDB so the credential store mirrors the postgres filer store. This also drops the leaky RegisterConnConfig-style init in favor of stdlib.OpenDB(*config), which doesn't accumulate entries in the global pgx config map. Adds parity knobs the filer store already exposes: sslcrl, and configurable connection_max_idle / connection_max_open / connection_max_lifetime_seconds (with the previous hardcoded 25/5/5min as defaults). Also moves the jsonbParam helper here so other store files can reuse it. (Helper is also referenced by postgres_identity.go, which is migrated to it in the next commit.) * refactor(credential/postgres): use jsonbParam helper across all writers Consolidate JSONB write handling on the new pgxutil-adjacent helper jsonbParam(b []byte) interface{}, which returns nil (driver writes SQL NULL) when the marshaled JSON is empty and string(b) otherwise. postgres_identity.go: replace the inline 'var fooParam any' / 'fooParam = string(b)' pattern with the helper. Same in CreateUser and UpdateUser. postgres_inline_policy.go, postgres_policy.go, postgres_service_account.go, postgres_group.go: every JSONB writer was still passing []byte. Under pgx simple_protocol (pgbouncer_compatible=true), []byte is encoded as bytea and Postgres rejects that against a JSONB column with "invalid input syntax for type json". Route them through jsonbParam too. * fix(credential/postgres): rework SaveConfiguration to handle rename + UNIQUE access keys The IAM rename path (s3api UpdateUser) renames an identity in place and keeps its access keys. With the previous flow — upsert each user, then per-user delete-and-insert credentials, then prune absent users — the renamed user's access keys were still owned by the old row when the INSERT for the new name ran, tripping credentials.access_key's global UNIQUE constraint and failing every rename of a user with credentials. Reorder the SaveConfiguration body so the prune step runs BEFORE the credential replace. CASCADE on the old user releases its access keys in the same transaction, and the new name can then claim them. While here: - Replace the per-user loop DELETE FROM users WHERE username = $1 with a single DELETE ... WHERE username = ANY($1), one round trip instead of N inside the transaction. - Surface inline-policy CASCADE losses: count user_inline_policies for the prune set and emit a Warningf when the count is non-zero so rename-driven drops are visible in operator logs (the structural fix for renames lives at the IAM layer in a follow-up commit). - Two-pass credential replace: clear credentials for every user we are about to rewrite first, then insert, so an access key can be moved between two users in the same SaveConfiguration call. - credErr := credRows.Err() before credRows.Close() in LoadConfiguration — Err() is documented as safe after Close, but the leading-capture pattern matches the rest of the file. * fix(s3api/iam): preserve inline policies when renaming a user EmbeddedIamApi.UpdateUser renames an identity in place and the caller persists via SaveConfiguration, which prunes the old username and CASCADE-drops its rows from user_inline_policies. GetUserPolicy and ListUserPolicies then return nothing under the new name even though the API reported success — silent data loss. Before flipping sourceIdent.Name, list the user's stored inline policies and re-attach each one under the new name. The subsequent SaveConfiguration prune still CASCADE-removes the old-name rows; only the duplicates we just wrote under the new name survive. Adds a regression test that puts a policy on the old name, renames, and asserts the policy is readable under the new name. * perf(credential/postgres): batch the credential clear in SaveConfiguration The two-pass credential replace was clearing each incoming user's credentials with its own DELETE statement — N round-trips inside the transaction. Match the pattern already used for the user prune and issue a single DELETE FROM credentials WHERE username = ANY($1) instead. * refactor(s3api/iam): plumb context through UpdateUser UpdateUser was synthesizing a fresh context.Background() inside the inline-policy migration block, which discards the request deadline, cancellation, and tracing carried by the caller. Add ctx as the first parameter and pass r.Context() in via the ExecuteAction dispatcher, mirroring the signature already used by CreatePolicy / AttachUserPolicy / DetachUserPolicy. * fix(util/pgxutil): quote DSN values per libpq rules BuildDSN was concatenating values directly, so any password / cert path / database name with a space, single quote, or backslash produced a malformed connection string and pgx.ParseConfig either errored or mis-parsed the remainder. Critical now that the helper is shared with the credential store: mTLS deployments routinely sourcing passwords or secret-mounted cert paths from a vault are exactly the case where spaces and quotes show up. Add quoteDSNValue: empty values and values containing whitespace, `'`, or `\` are wrapped in single quotes with `'` and `\` escaped per PostgreSQL libpq rules; plain alphanumeric values pass through unchanged. Apply it to every variable field in BuildDSN. Adds a test that round-trips a password containing spaces, quotes and backslashes through pgx.ParseConfig and confirms the parsed Config matches the input. * fix(credential,s3api/iam): atomic UserRenamer to avoid FK violation on rename The previous IAM rename path called PutUserInlinePolicy(newName, ...) before SaveConfiguration created the new users row. user_inline_policies has a non-deferrable FOREIGN KEY (username) REFERENCES users(username), which Postgres validates at statement time, so every rename of a user that owned at least one inline policy failed with an FK violation. The existing memory-store regression test missed it because the memory backend has no FK enforcement. Add an optional credential.UserRenamer interface plus a CredentialManager.RenameUser thin shim that returns (supported, err). Implement it on PostgresStore as an atomic in-transaction migration: INSERT the new users row by SELECT-copying from the old, UPDATE credentials.username and user_inline_policies.username to the new name (FK satisfied because both rows now exist), then DELETE the old row. ErrUserNotFound / ErrUserAlreadyExists are surfaced cleanly. Implement it on MemoryStore by re-binding store.users / store.accessKeys / store.inlinePolicies under the new name. Also fixes a small leak in DeleteUser, which was forgetting to drop the user's inline-policy bucket. EmbeddedIamApi.UpdateUser now calls RenameUser first; if the store implements the interface, that's the whole migration. If it doesn't (stores without FK enforcement), fall back to the previous list / get / put copy. Adds a focused test for MemoryStore.RenameUser that asserts the identity, the access-key index, and the inline policies all land under the new name.	2026-04-26 16:31:53 -07:00
Chris Lu	5a7c74feac	migrate IAM policies to multi-file storage (#8114) ... * Add IAM gRPC service definition - Add GetConfiguration/PutConfiguration for config management - Add CreateUser/GetUser/UpdateUser/DeleteUser/ListUsers for user management - Add CreateAccessKey/DeleteAccessKey/GetUserByAccessKey for access key management - Methods mirror existing IAM HTTP API functionality * Add IAM gRPC handlers on filer server - Implement IamGrpcServer with CredentialManager integration - Handle configuration get/put operations - Handle user CRUD operations - Handle access key create/delete operations - All methods delegate to CredentialManager for actual storage * Wire IAM gRPC service to filer server - Add CredentialManager field to FilerOption and FilerServer - Import credential store implementations in filer command - Initialize CredentialManager from credential.toml if available - Register IAM gRPC service on filer gRPC server - Enable credential management via gRPC alongside existing filer services * Regenerate IAM protobuf with gRPC service methods * fix: compilation error in DeleteUser * fix: address code review comments for IAM migration * feat: migrate policies to multi-file layout and fix identity duplicated content * refactor: remove configuration.json and migrate Service Accounts to multi-file layout * refactor: standardize Service Accounts as distinct store entities and fix Admin Server persistence * config: set ServiceAccountsDirectory to /etc/iam/service_accounts * Fix Chrome dialog auto-dismiss with Bootstrap modals - Add modal-alerts.js library with Bootstrap modal replacements - Replace all 15 confirm() calls with showConfirm/showDeleteConfirm - Auto-override window.alert() for all alert() calls - Fixes Chrome 132+ aggressively blocking native dialogs * Upgrade Bootstrap from 5.3.2 to 5.3.8 * Fix syntax error in object_store_users.templ - remove duplicate closing braces * create policy * display errors * migrate to multi-file policies * address PR feedback: use showDeleteConfirm and showErrorMessage in policies.templ, refine migration check * Update policies_templ.go * add service account to iam grpc * iam: fix potential path traversal in policy names by validating name pattern * iam: add GetServiceAccountByAccessKey to CredentialStore interface * iam: implement service account support for PostgresStore Includes full CRUD operations and efficient lookup by access key. * iam: implement GetServiceAccountByAccessKey for filer_etc, grpc, and memory stores Provides efficient lookup of service accounts by access key where possible, with linear scan fallbacks for file-based stores. * iam: remove filer_multiple support Deleted its implementation and references in imports, scaffold config, and core interface constants. Redundant with filer_etc. * clear comment * dash: robustify service account construction - Guard against nil sa.Credential when constructing responses - Fix Expiration logic to only set if > 0, avoiding Unix epoch 1970 - Ensure consistency across Get, Create, and Update handlers * credential/filer_etc: improve error propagation in configuration handlers - Return error from loadServiceAccountsFromMultiFile to callers - Ensure listEntries errors in SaveConfiguration (cleanup logic) are propagated unless they are "not found" failures. - Fixes potential silent failures during IAM configuration sync. * credential/filer_etc: add existence check to CreateServiceAccount Ensures consistency with other stores by preventing accidental overwrite of existing service accounts during creation. * credential/memory: improve store robustness and Reset logic - Enforce ID immutability in UpdateServiceAccount to prevent orphans - Update Reset() to also clear the policies map, ensuring full state cleanup for tests. * dash: improve service account robustness and policy docs - Wrap parent user lookup errors to preserve context - Strictly validate Status field in UpdateServiceAccount - Add deprecation comments to legacy policy management methods * credential/filer_etc: protect against path traversal in service accounts Implemented ID validation (alphanumeric, underscores, hyphens) and applied it to Get, Save, and Delete operations to ensure no directory traversal via saId.json filenames. * credential/postgres: improve robustness and cleanup comments - Removed brainstorming comments in GetServiceAccountByAccessKey - Added missing rows.Err() check during iteration - Properly propagate Scan and Unmarshal errors instead of swallowing them * admin: unify UI alerts and confirmations using Bootstrap modals - Updated modal-alerts.js with improved automated alert type detection - Replaced native alert() and confirm() with showAlert(), showConfirm(), and showDeleteConfirm() across various Templ components - Improved UX for delete operations by providing better context and styling - Ensured consistent error reporting across IAM and Maintenance views * admin: additional UI consistency fixes for alerts and confirmations - Replaced native alert() and confirm() with Bootstrap modals in: - EC volumes (repair flow) - Collection details (repair flow) - File browser (properties and delete) - Maintenance config schema (save and reset) - Improved delete confirmation in file browser with item context - Ensured consistent success/error/info styling for all feedbacks * make * iam: add GetServiceAccountByAccessKey RPC and update GetConfiguration * iam: implement GetServiceAccountByAccessKey on server and client * iam: centralize policy and service account validation * iam: optimize MemoryStore service account lookups with indexing * iam: fix postgres service_accounts table and optimize lookups * admin: refactor modal alerts and clean up dashboard logic * admin: fix EC shards table layout mismatch * admin: URL-encode IAM path parameters for safety * admin: implement pauseWorker logic in maintenance view * iam: add rows.Err() check to postgres ListServiceAccounts * iam: standardize ErrServiceAccountNotFound across credential stores * iam: map ErrServiceAccountNotFound to codes.NotFound in DeleteServiceAccount * iam: refine service account store logic, errors and schema * iam: add validation to GetServiceAccountByAccessKey * admin: refine modal titles and ensure URL safety * admin: address bot review comments for alerts and async usage * iam: fix syntax error by restoring missing function declaration * [FilerEtcStore] improve error handling in CreateServiceAccount Refine error handling to provide clearer messages when checking for existing service accounts. * [PostgresStore] add nil guards and validation to service account methods Ensure input parameters are not nil and required IDs are present to prevent runtime panics and ensure data integrity. * [JS] add shared IAM utility script Consolidate common IAM operations like deleteUser and deleteAccessKey into a shared utility script for better maintainability. * [View] include shared IAM utilities in layout Include iam-utils.js in the main layout to make IAM functions available across all administrative pages. * [View] refactor IAM logic and restore async in EC Shards view Remove redundant local IAM functions and ensure that delete confirmation callbacks are properly marked as async. * [View] consolidate IAM logic in Object Store Users view Remove redundant local definitions of deleteUser and deleteAccessKey, relying on the shared utilities instead. * [View] update generated templ files for UI consistency * credential/postgres: remove redundant name column from service_accounts table The id is already used as the unique identifier and was being copied to the name column. This removes the name column from the schema and updates the INSERT/UPDATE queries. * credential/filer_etc: improve logging for policy migration failures Added Errorf log if AtomicRenameEntry fails during migration to ensure visibility of common failure points. * credential: allow uppercase characters in service account ID username Updated ServiceAccountIdPattern to allow [A-Za-z0-9_-]+ for the username component, matching the actual service account creation logic which uses the parent user name directly. * Update object_store_users_templ.go * admin: fix ec_shards pagination to handle numeric page arguments Updated goToPage in cluster_ec_shards.templ to accept either an Event or a numeric page argument. This prevents errors when goToPage(1) is called directly. Corrected both the .templ source and generated Go code. * credential/filer_etc: improve service account storage robustness Added nil guard to saveServiceAccount, updated GetServiceAccount to return ErrServiceAccountNotFound for empty data, and improved deleteServiceAccount to handle response-level Filer errors.	2026-01-26 11:28:23 -08:00