mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 13:30:46 +02:00
c2cdefd06d89f4510dfe06508eee7ea5aa49d607
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris Lu
|
4e5839ce82 |
fix(iam): return a valid user ARN from CreateUser and GetUser (#9794)
* fix(iam): return a valid user ARN from CreateUser and GetUser The terraform aws provider 6.41 reads a user back after creating it and blocks until GetUser returns a value that passes arn.IsARN. We only set UserName, so the ARN was empty and apply hung until the 2m timeout. Populate Arn (and Path) via a shared iam.NewUser helper in both the embedded and standalone IAM handlers. * fix(iam): use the userName parameter directly in NewUser Drop the redundant local copy; the value parameter is already function-local. * fix(iam): return full user objects with ARNs from GetGroup GetGroup listed members with only UserName set. Build them via the shared NewUser helper so group members carry a valid Arn and Path like the other user responses, in both the embedded and standalone IAM handlers. |
||
|
Jon E NesvoldandChris Lu
|
c6302fcb54 |
feat(iam): allow caller-supplied AccessKeyId and SecretAccessKey in CreateAccessKey (#9172)
* feat(iam): support caller-supplied AccessKeyId and SecretAccessKey in CreateAccessKey Both IAM implementations (standalone and embedded) now check for caller-supplied AccessKeyId and SecretAccessKey form parameters before generating random credentials. If provided, the caller-supplied values are used. If empty, random keys are generated as before. This enables programmatic identity provisioning where the caller needs to control the S3 credentials. Backward-compatible: no behavior change for callers that omit these parameters. * refactor(iam): extract shared caller-supplied credential validation Move the AccessKeyId/SecretAccessKey format checks and the in-memory collision scan into weed/iam so the standalone IAM API, the embedded IAM in s3api, and the admin dashboard all enforce the same rules. - ValidateCallerSuppliedAccessKeyId: 4-128 alphanumeric (rejects SigV4-breaking characters like '/' and '='). - ValidateCallerSuppliedSecretAccessKey: 8-128 chars. - FindAccessKeyOwner: scans identities and service accounts and returns the owning entity type + name for debug logging, without exposing the owner in caller-facing error messages. The admin dashboard previously only length-checked caller-supplied keys; it now enforces the same alphanumeric rule, which matches what SigV4 actually accepts anyway. * fix(iam): reject partial caller-supplied AccessKeyId/SecretAccessKey Previously, if a caller supplied only one of AccessKeyId or SecretAccessKey, CreateAccessKey logged a warning and auto-generated the missing half. That silently returns a credential the caller did not fully choose, which is surprising and easy to miss in a response they expected to echo back their input. Return ErrCodeInvalidInputException instead: either both are supplied or neither is. Updates the mixed-supply tests in weed/iamapi and weed/s3api to assert the rejection. * chore(iam): centralize and broaden sensitive form redaction DoActions and ExecuteAction both had an inline loop that redacted SecretAccessKey from their debug-level request log. Replace the two copies with iam.RedactSensitiveFormValues, backed by an explicit sensitive-keys set. The set now also covers Password, OldPassword, NewPassword, PrivateKey, and SessionToken. None of those parameters are used by today's IAM actions, but naming them here makes the log-safety guarantee survive future additions such as LoginProfile / STS. * test(iam): cover the upper length bound for CreateAccessKey TestCreateAccessKeyBoundary / TestEmbeddedIamCreateAccessKeyBoundary only exercised the 3/4-char lower edge. Add cases for 128 (accepted) and 129 (rejected) for AccessKeyId, plus 7 / 128 / 129-char cases for SecretAccessKey, so both ends of the validator are locked in at the handler level (the pure validators in weed/iam already cover this). * fix(s3api/iam): verify user existence before RNG and collision scan In the embedded IAM CreateAccessKey, the user lookup ran last: a request for a non-existent user still walked the whole identity / service-account list for collisions and, if no caller-supplied keys were present, generated fresh random credentials with crypto/rand before the NoSuchEntity error finally surfaced. Reorder: validate inputs, then find the target identity, then do the collision scan, then generate keys. A missing user now fails fast and consumes no entropy, and the handler returns NoSuchEntity instead of a misleading EntityAlreadyExists when both the user is missing and the supplied AccessKeyId happens to collide with another identity's key. Add TestEmbeddedIamCreateAccessKeyRejectsMissingUser to lock in the "no mutation on unknown user" guarantee. The standalone iamapi CreateAccessKey intentionally keeps its pre-existing "create-or-attach" semantics where a missing user is implicitly provisioned — that is a behavior change beyond the scope of this PR. * test(iam): tighten collision leak assertion and cover 8-char secret - Rename the collision-owner identity in TestCreateAccessKeyRejectsCollision (both iamapi and the embedded s3api test) from "existing" / "ExistingUser" to "ownerAlpha". The old assert.NotContains check was effectively a no-op because the error message never contained those substrings; a distinctive name shared with no part of the expected error body makes the leak guard actually meaningful if the wording ever drifts. The embedded test also adds a NotContains assertion that was previously missing entirely. - Add an explicit 8-char SecretAccessKey pass case to both boundary tests so the lower edge of the validator is locked in at the handler level alongside the 7 / 128 / 129-char cases. * fix(iamapi): enforce both-or-none before the collision lookup In the standalone IAM CreateAccessKey, FindAccessKeyOwner ran before the partial-credential check. If a caller supplied only AccessKeyId and it happened to collide with an existing key, the response was EntityAlreadyExists instead of the more fundamental InvalidInput for omitting SecretAccessKey — wrong error class, and leaked the fact that the probed key is already in use. Swap the order: validate both-or-none first, then do the collision scan. Matches the embedded IAM path and AWS behavior. Add a case to TestCreateAccessKeyRejectsPartialSupply that combines partial supply with a collision to lock in the ordering. * fix(admin): reject partial caller-supplied AccessKey/SecretKey The admin dashboard path silently generated the missing half when a caller supplied only one of AccessKey or SecretKey, while the IAM API and embedded IAM paths now reject this. Align the three: if exactly one is provided, return ErrInvalidInput. Also simplifies the generator block — either both are provided or neither is, so there is no mixed path to handle. * test(s3api/iam): guard dereferences in caller-supplied-keys test TestEmbeddedIamCreateAccessKeyWithCallerSuppliedKeys dereferenced *AccessKeyId/*SecretAccessKey/*UserName and indexed Identities[0].Credentials[0] without first verifying shape, so any future regression that returns a partial response or skips the config mutation would panic mid-assertion instead of failing with a clear message. Add require.NotNil on the response pointers and require.Len on the identities/credentials slices before the asserts. * test(iamapi): exercise the service-account branch of the collision check FindAccessKeyOwner scans both Identities[*].Credentials and ServiceAccounts[*].Credential, but TestCreateAccessKeyRejectsCollision only covered the identity branch. Split the test into two subtests — one per branch — so a future refactor that drops the service-account scan (or mutates the existing credential) trips a failure. Also asserts the existing service-account credential is not mutated and no credential is attached to the target identity on rejection. * test(iam): isolate 129-char secret subcase from prior credential In both TestCreateAccessKeyBoundary (iamapi) and TestEmbeddedIamCreateAccessKeyBoundary (s3api), the 129-char SecretAccessKey subcase reused the "validkey" AccessKeyId that the preceding 8-char subcase had just persisted into the config. The test still asserted the right outcome because the handler validates secret length before running the collision scan — but if the two checks ever swap, the subcase would pass (or fail) for the wrong reason. Reset the in-memory credentials before the 129-char subcase, matching the pattern already used by the 3/128/129-char AccessKeyId and 7-char secret subcases. No behavior change; purely test isolation. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Chris Lu
|
e4bcfb96d8 |
fix(iam): preserve actions/resources in GetUserPolicy fallback (#9009)
* fix(iam): preserve actions/resources in GetUserPolicy fallback (#9008) When GetUserPolicy cannot find a stored inline policy document and falls back to reconstructing one from the aggregated ident.Actions, it produced mangled output: bare-bucket paths like "b-le*/*" got another "/*" appended (becoming "b-le*/*/*"), and distinct s3 actions that map to the same coarse verb (e.g. s3:GetObject and s3:GetBucketLocation -> s3:Get*) were emitted multiple times in the same statement. - Use SplitN so paths containing ':' are not shredded. - Only append "/*" to bare bucket patterns; paths already containing '/' are used as-is. - Dedupe reconstructed actions per resource. Adds a regression test using the exact reproducer from the issue. * fix(iam): preserve bucket-level ARNs in fallback reconstruction Addresses CodeRabbit review feedback on #9009: - Use stored path verbatim in the GetUserPolicy fallback so bucket-level resources (e.g. arn:aws:s3:::b-le*) are not rewritten to object-level ARNs (arn:aws:s3:::b-le*/*). Previously bare bucket patterns had "/*" appended, conflating bucket and object resources. - Extend TestPutGetUserPolicyIssue9008 to also exercise the fallback reconstruction path by clearing the persisted inline policy between the two GetUserPolicy calls, validating that bucket and object resources stay distinct. * chore: revert accidental scheduled_tasks.lock change |
||
|
Chris Lu
|
9984ce7dcb |
fix(s3): omit NotResource:null from bucket policy JSON response (#8658)
* fix(s3): omit NotResource:null from bucket policy JSON response (#8657) Change NotResource from value type to pointer (*StringOrStringSlice) so that omitempty properly omits it when unset, matching the existing Principal field pattern. This prevents IaC tools (Terraform, Ansible) from detecting false configuration drift. Add bucket policy round-trip idempotency integration tests. * simplify JSON comparison in bucket policy idempotency test Use require.JSONEq directly on the raw JSON strings instead of round-tripping through unmarshal/marshal, since JSONEq already handles normalization internally. * fix bucket policy test cases that locked out the admin user The Deny+NotResource test cases used Action:"s3:*" which denied the admin's own GetBucketPolicy call. Scope deny to s3:GetObject only, and add an Allow+NotResource variant instead. * fix(s3): also make Resource a pointer to fix empty string in JSON Apply the same omitempty pointer fix to the Resource field, which was emitting "Resource":"" when only NotResource was set. Add NewStringOrStringSlicePtr helper, make Strings() nil-safe, and handle *StringOrStringSlice in normalizeToStringSliceWithError. * improve bucket policy integration tests per review feedback - Replace time.Sleep with waitForClusterReady using ListBuckets - Use structural hasKey check instead of brittle substring NotContains - Assert specific NoSuchBucketPolicy error code after delete - Handle single-statement policies in hasKey helper |
||
|
Chris LuandClaude Opus 4.6
|
df5e8210df |
Implement IAM managed policy operations (#8507)
* feat: Implement IAM managed policy operations (GetPolicy, ListPolicies, DeletePolicy, AttachUserPolicy, DetachUserPolicy) - Add response type aliases in iamapi_response.go for managed policy operations - Implement 6 handler methods in iamapi_management_handlers.go: - GetPolicy: Lookup managed policy by ARN - DeletePolicy: Remove managed policy - ListPolicies: List all managed policies - AttachUserPolicy: Attach managed policy to user, aggregating inline + managed actions - DetachUserPolicy: Detach managed policy from user - ListAttachedUserPolicies: List user's attached managed policies - Add computeAllActionsForUser() to aggregate actions from both inline and managed policies - Wire 6 new DoActions switch cases for policy operations - Add comprehensive tests for all new handlers - Fixes #8506 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback for IAM managed policy operations - Add parsePolicyArn() helper with proper ARN prefix validation, replacing fragile strings.Split parsing in GetPolicy, DeletePolicy, AttachUserPolicy, and DetachUserPolicy - DeletePolicy now detaches the policy from all users and recomputes their aggregated actions, preventing stale permissions after deletion - Set changed=true for DeletePolicy DoActions case so identity updates persist - Make PolicyId consistent: CreatePolicy now uses Hash(&policyName) matching GetPolicy and ListPolicies - Remove redundant nil map checks (Go handles nil map lookups safely) - DRY up action deduplication in computeAllActionsForUser with addUniqueActions closure - Add tests for invalid/empty ARN rejection and DeletePolicy identity cleanup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add integration tests for managed policy lifecycle (#8506) Add two integration tests covering the user-reported use case where managed policy operations returned 500 errors: - TestS3IAMManagedPolicyLifecycle: end-to-end workflow matching the issue report — CreatePolicy, ListPolicies, GetPolicy, AttachUserPolicy, ListAttachedUserPolicies, idempotent re-attach, DeletePolicy while attached (expects DeleteConflict), DetachUserPolicy, DeletePolicy, and verification that deleted policy is gone - TestS3IAMManagedPolicyErrorCases: covers error paths — nonexistent policy/user for GetPolicy, DeletePolicy, AttachUserPolicy, DetachUserPolicy, and ListAttachedUserPolicies Also fixes DeletePolicy to reject deletion when policy is still attached to a user (AWS-compatible DeleteConflictException), and adds the 409 status code mapping for DeleteConflictException in the error response handler. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: nil map panic in CreatePolicy, add PolicyId test assertions - Initialize policies.Policies map in CreatePolicy if nil (prevents panic when no policies exist yet); also handle filer_pb.ErrNotFound like other callers - Add PolicyId assertions in TestGetPolicy and TestListPolicies to lock in the consistent Hash(&policyName) behavior - Remove redundant time.Sleep calls from new integration tests (startMiniCluster already blocks on waitForS3Ready) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: PutUserPolicy and DeleteUserPolicy now preserve managed policy actions PutUserPolicy and DeleteUserPolicy were calling computeAggregatedActionsForUser (inline-only), overwriting ident.Actions and dropping managed policy actions. Both now call computeAllActionsForUser which unions inline + managed actions. Add TestManagedPolicyActionsPreservedAcrossInlineMutations regression test: attaches a managed policy, adds an inline policy (verifies both actions present), deletes the inline policy, then asserts managed policy actions still persist. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: PutUserPolicy verifies user exists before persisting inline policy Previously the inline policy was written to storage before checking if the target user exists in s3cfg.Identities, leaving orphaned policy data when the user was absent. Now validates the user first, returning NoSuchEntityException immediately if not found. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent stale/lost actions on computeAllActionsForUser failure - PutUserPolicy: on recomputation failure, preserve existing ident.Actions instead of falling back to only the current inline policy's actions - DeleteUserPolicy: on recomputation failure, preserve existing ident.Actions instead of assigning nil (which wiped all permissions) - AttachUserPolicy: roll back ident.PolicyNames and return error if action recomputation fails, keeping identity consistent - DetachUserPolicy: roll back ident.PolicyNames and return error if GetPolicies or action recomputation fails - Add doc comment on newTestIamApiServer noting it only sets s3ApiConfig Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Chris Lu
|
a3cb7fa8cc | go fmt | ||
|
Chris Lu
|
27e763222a |
Fix inline user policy retrieval (#8437)
* Fix IAM inline user policy retrieval * fmt * Persist inline user policies to avoid loss on server restart - Use s3ApiConfig.PutPolicies/GetPolicies for persistent storage instead of non-persistent global map - Remove unused global policyDocuments map - Update PutUserPolicy to store policies in persistent storage - Update GetUserPolicy to read from persistent storage - Update DeleteUserPolicy to clean up persistent storage - Add mock IamS3ApiConfig for testing - Improve test to verify policy statements are not merged or lost * Fix inline policy key collision and action aggregation * Improve error handling and optimize inline policy management - GetUserPolicy: Propagate GetPolicies errors instead of silently falling through - DeleteUserPolicy: Return error immediately on GetPolicies failure - computeAggregatedActionsForUser: Add optional Policies parameter for I/O optimization - PutUserPolicy: Reuse fetched policies to avoid redundant GetPolicies call - Improve logging with clearer messages about best-effort aggregation - Update test to use exact action string matching instead of substring checks All 15 tests pass with no regressions. * Add per-user policy index for O(1) lookup performance - Extend Policies struct with InlinePolicies map[userName]map[policyName] - Add getOrCreateUserPolicies() helper for safe user map management - Update computeAggregatedActionsForUser to use direct user map access - Update PutUserPolicy, GetUserPolicy, DeleteUserPolicy for new structure - Performance: O(1) user lookups instead of O(all_policies) iteration - Eliminates string prefix matching loop - All tests pass; backward compatible with managed policies * Fix DeleteUserPolicy to validate user existence before storage modification Refactor DeleteUserPolicy handler to check user existence early: - First iterate s3cfg.Identities to verify user exists - Return NoSuchEntity error immediately if user not found - Only then proceed with GetPolicies and policy deletion - Capture reference to found identity for direct update This ensures consistency: if user doesn't exist, storage is not modified. Previously the code would delete from storage first and check identity afterwards, potentially leaving orphaned policies. Benefits: - Fail-fast validation before storage operations - No orphaned policies in storage if validation fails - Atomic from logical perspective - Direct identity reference eliminates redundant loop - All error paths preserved and tested All 15 tests pass; no functional changes to behavior. * Fix GetUserPolicy to return NoSuchEntity when inline policy not found When InlinePolicies[userName] exists but does not contain policyName, the handler now immediately returns NoSuchEntity error instead of falling through to the reconstruction logic. Changes: - Add else clause after userPolicies[policyName] lookup - Return IamError(NoSuchEntityException, "policy not found") immediately - Prevents incorrect fallback to reconstructing ident.Actions - Ensures explicit error when policy explicitly doesn't exist This improves error semantics: - Policy exists in stored inline policies → return error (not reconstruct) - Policy doesn't exist in stored inline policies → try reconstruction (backward compat) - Storage error → return service failure error All 15 tests pass; no behavioral changes to existing error or success paths. |
||
|
Chris Lu
|
7cb1ca1308 | Add policy engine (#6970) | ||
|
Tom Crasset
|
7c3a0ed874 | return error on invalid action in PutUserPolicy (#6482) | ||
|
Tom CrassetandChris Lu
|
c5f21b2b01 |
fix S3 per-user-directory Policy (#6443)
* fix S3 per-user-directory Policy * Delete docker/config.json * add tests * remove logs * undo modifications of weed/shell/command_volume_balance.go * remove modifications of docker-compose * fix failing test --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |