From ddde794d3343a54f56b0e3569c275f29d6898aa5 Mon Sep 17 00:00:00 2001 From: chrislusf Date: Wed, 10 Dec 2025 22:23:26 -0800 Subject: [PATCH] re-org --- AWS-CLI-with-SeaweedFS.md | 2 +- Amazon-S3-API.md | 14 +- Docker-Compose-for-S3.md | 2 +- Keycloak-Integration.md | 187 --------------------- OIDC-Integration.md | 336 ++++++++++++++++++++++++++++++++++++++ S3-Configuration.md | 204 +++++++++++++++++++++++ S3-Credentials.md | 8 + S3-Rate-Limiting.md | 1 + TUS-Resumable-Uploads.md | 1 + _Sidebar.md | 23 +-- 10 files changed, 575 insertions(+), 203 deletions(-) delete mode 100644 Keycloak-Integration.md create mode 100644 OIDC-Integration.md create mode 100644 S3-Configuration.md diff --git a/AWS-CLI-with-SeaweedFS.md b/AWS-CLI-with-SeaweedFS.md index 77ff515..c5ce118 100644 --- a/AWS-CLI-with-SeaweedFS.md +++ b/AWS-CLI-with-SeaweedFS.md @@ -108,4 +108,4 @@ aws --endpoint-url http://localhost:8333 s3 cp file.txt s3://bucket/sse-s3.txt \ ``` ## OIDC/JWT to S3 -For Keycloak and other OIDC providers, you can obtain a JWT and access S3 directly (or use STS to assume a role). See [[Keycloak Integration]]. +For Keycloak and other OIDC providers, you can obtain a JWT and access S3 directly (or use STS to assume a role). See [[OIDC Integration]]. diff --git a/Amazon-S3-API.md b/Amazon-S3-API.md index 0e52fe3..03c1e74 100644 --- a/Amazon-S3-API.md +++ b/Amazon-S3-API.md @@ -261,10 +261,16 @@ curl -X PUT -H "If-None-Match: *" -d "new content" "http://localhost:8333/mybuck # S3 Authentication -By default, the access key and secret key to access `weed s3` is not authenticated. To enable credential based access, you can choose static or dynamic configuration: -* **Dynamic Configuration**: setup auth with `s3.configure` in `weed shell` -* **Static Configuration**: create a config.json file similar to the example below, and specify it via `weed s3 -config=config.json` -* **OIDC/JWT (Web Identity)**: for Keycloak and other OpenID providers, see [[Keycloak Integration]] for STS and JWT to S3 usage +> **For a complete overview of S3 configuration options, see [[S3 Configuration]].** + +By default, the access key and secret key to access `weed s3` is not authenticated. To enable credential based access, you can choose from: + +| Method | Option | Documentation | +|--------|--------|---------------| +| **Static Configuration** | `-s3.config=config.json` | [[S3 Credentials]] | +| **Dynamic Configuration** | `s3.configure` in weed shell | See below | +| **Admin UI** | Web interface | [[Admin UI]] | +| **OIDC/JWT (Web Identity)** | `-s3.iam.config=iam.json` | [[OIDC Integration]] | ## Dynamic Configuration diff --git a/Docker-Compose-for-S3.md b/Docker-Compose-for-S3.md index dadbf80..7359129 100644 --- a/Docker-Compose-for-S3.md +++ b/Docker-Compose-for-S3.md @@ -40,6 +40,6 @@ services: ``` ## Notes -- To enable advanced IAM (STS, OIDC providers), mount a JSON and add `-iam.config=/etc/seaweedfs/iam.json` to the S3 command. See [[Keycloak Integration]]. +- To enable advanced IAM (STS, OIDC providers), mount a JSON and add `-iam.config=/etc/seaweedfs/iam.json` to the S3 command. See [[OIDC Integration]]. - To use SSE-KMS (with OpenBao/Vault, AWS KMS, GCP KMS), mount KMS config (e.g. `s3_kms.json`) and start with `-config=/etc/seaweedfs/s3_kms.json`. See [[Server-Side-Encryption-SSE-KMS]]. - SSE-S3 and bucket default encryption work without external KMS; see [[Server-Side-Encryption]]. \ No newline at end of file diff --git a/Keycloak-Integration.md b/Keycloak-Integration.md deleted file mode 100644 index 38a551d..0000000 --- a/Keycloak-Integration.md +++ /dev/null @@ -1,187 +0,0 @@ -# Keycloak Integration (OIDC) with SeaweedFS S3 Gateway - -This guide shows how to integrate Keycloak (OpenID Connect) with SeaweedFS S3 Gateway using the advanced IAM and STS configuration. It supports both: - -- Direct OIDC authentication to S3 with Bearer tokens -- OIDC to STS role assumption using trust policies and role mapping - -## Prerequisites - -- A running Keycloak server and a realm (e.g. `seaweedfs`) -- A Keycloak client (e.g. `seaweedfs-s3`) created in that realm -- SeaweedFS with advanced IAM enabled via `-iam.config` - -## Step 1: Configure Keycloak - -1) Create a client -- Client ID: `seaweedfs-s3` -- Access Type: public or confidential (confidential requires a client secret) -- Standard Flow: enabled (for browser login flows) - -2) Add role/group claims to tokens -- Add a mapper of type "Group Membership" or "User Realm Role" that puts roles/groups into a top-level claim: - - Claim name: `groups` (recommended) or `roles` - - Add to ID token: true - - Add to Access token: true - -3) Confirm OIDC discovery -- Open your realm discovery document and note the issuer and certs endpoints (Keycloak Quarkus defaults): - - Issuer: `https://KEYCLOAK/realms/` - - JWKS (certs): `https://KEYCLOAK/realms//protocol/openid-connect/certs` - - UserInfo: `https://KEYCLOAK/realms//protocol/openid-connect/userinfo` - -Note: For older Keycloak distributions, issuer may include `/auth` in the path. - -## Step 2: Prepare SeaweedFS IAM config - -Create an IAM configuration JSON file (e.g. `/etc/seaweed/iam_keycloak.json`) and reference it with `weed s3 -iam.config=...`. - -Minimal example with Keycloak OIDC provider, role mapping, roles, and policies: - -```json -{ - "sts": { - "tokenDuration": "1h", - "maxSessionLength": "12h", - "issuer": "seaweedfs-sts", - "signingKey": "c2Vhd2VlZGZzLXNpZ25pbmcta2V5LTMyLWNoYXJzLWxvbmc=" - }, - "providers": [ - { - "name": "keycloak", - "type": "oidc", - "enabled": true, - "config": { - "issuer": "https://KEYCLOAK/realms/seaweedfs", - "clientId": "seaweedfs-s3", - "clientSecret": "", - "jwksUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/certs", - "userInfoUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/userinfo", - "scopes": ["openid", "profile", "email", "roles", "groups"], - "roleMapping": { - "rules": [ - { "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" }, - { "claim": "groups", "value": "developers", "role": "arn:aws:iam::role/S3WriteRole" } - ], - "defaultRole": "arn:aws:iam::role/S3ReadOnlyRole" - } - } - } - ], - "policies": [ - { - "name": "S3ReadOnlyPolicy", - "document": { - "Version": "2012-10-17", - "Statement": [ - { "Effect": "Allow", "Action": ["s3:List*", "s3:Get*"], "Resource": ["*"] } - ] - } - }, - { - "name": "S3WritePolicy", - "document": { - "Version": "2012-10-17", - "Statement": [ - { "Effect": "Allow", "Action": ["s3:List*", "s3:Get*", "s3:Put*", "s3:DeleteObject"], "Resource": ["*"] } - ] - } - }, - { - "name": "S3AdminPolicy", - "document": { - "Version": "2012-10-17", - "Statement": [ - { "Effect": "Allow", "Action": ["s3:*"] , "Resource": ["*"] } - ] - } - } - ], - "roles": [ - { - "roleName": "S3ReadOnlyRole", - "roleArn": "arn:aws:iam::role/S3ReadOnlyRole", - "attachedPolicies": ["S3ReadOnlyPolicy"], - "trustPolicy": { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { "Federated": "*" }, - "Action": ["sts:AssumeRoleWithWebIdentity"], - "Condition": { - "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } - } - } - ] - } - }, - { - "roleName": "S3WriteRole", - "roleArn": "arn:aws:iam::role/S3WriteRole", - "attachedPolicies": ["S3WritePolicy"], - "trustPolicy": { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { "Federated": "*" }, - "Action": ["sts:AssumeRoleWithWebIdentity"], - "Condition": { - "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } - } - } - ] - } - }, - { - "roleName": "S3AdminRole", - "roleArn": "arn:aws:iam::role/S3AdminRole", - "attachedPolicies": ["S3AdminPolicy"], - "trustPolicy": { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { "Federated": "*" }, - "Action": ["sts:AssumeRoleWithWebIdentity"], - "Condition": { - "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } - } - } - ] - } - } - ] -} -``` - -Notes: -- Set `signingKey` to a strong random secret (base64-encoded 32+ bytes). All S3 gateway instances must share the same STS `issuer` and `signingKey`. -- Explicit `jwksUri` and `userInfoUri` are recommended for Keycloak. -- Ensure your Keycloak mappers populate a top-level `groups` (or `roles`) claim. - -## Step 3: Start the S3 Gateway - -```bash -weed s3 -filer=filer:8888 -port=8333 -iam.config=/etc/seaweed/iam_keycloak.json -``` - -For multi-instance deployments, use the same IAM config on each instance. - -## Using It - -- Direct OIDC to S3 (Bearer): obtain a Keycloak access or ID token for client `seaweedfs-s3` and call S3 with: - -```bash -curl -H "Authorization: Bearer $KEYCLOAK_TOKEN" http://s3-gateway:8333/ -``` - -- Role selection: SeaweedFS maps OIDC claims via `roleMapping`. With the example above, users in `admins` get `S3AdminRole`, `developers` get `S3WriteRole`, others default to `S3ReadOnlyRole`. - -## Troubleshooting - -- Invalid token: verify token `iss` equals the configured provider `issuer` and `aud` or `azp` equals the client ID. -- JWKS errors: ensure `jwksUri` is reachable from the S3 gateway. For Keycloak, use the `.../protocol/openid-connect/certs` endpoint. -- No roles applied: confirm Keycloak mapper emits `groups` (or `roles`) at top-level in the token, and adjust `roleMapping` rules accordingly. -- Trust policy denied: ensure `seaweed:Issuer` in the trust policy matches your Keycloak realm issuer exactly. diff --git a/OIDC-Integration.md b/OIDC-Integration.md new file mode 100644 index 0000000..ce745f2 --- /dev/null +++ b/OIDC-Integration.md @@ -0,0 +1,336 @@ +# OIDC Integration with SeaweedFS S3 Gateway + +> **This page covers advanced IAM features** using the `-s3.iam.config` / `-iam.config` option. +> +> For basic S3 credentials with access keys, see [[S3 Credentials]] which uses the `-s3.config` / `-config` option. +> +> For a comparison of both options, see [[S3 Configuration]]. + +--- + +This guide shows how to integrate OpenID Connect (OIDC) identity providers with SeaweedFS S3 Gateway using the advanced IAM and STS configuration. It supports: + +- Direct OIDC authentication to S3 with Bearer tokens +- OIDC to STS role assumption using trust policies and role mapping + +## Supported Identity Providers + +SeaweedFS works with any OIDC-compliant identity provider, including: + +| Provider | Configuration Notes | +|----------|-------------------| +| **Keycloak** | See [Keycloak Example](#keycloak-example) below | +| **Okta** | Use your Okta domain as issuer | +| **Auth0** | Use `https://YOUR_DOMAIN.auth0.com/` as issuer | +| **Azure AD** | Use `https://login.microsoftonline.com/TENANT_ID/v2.0` as issuer | +| **Google** | Use `https://accounts.google.com` as issuer | +| **AWS Cognito** | Use `https://cognito-idp.REGION.amazonaws.com/POOL_ID` as issuer | + +## When to Use This + +Use `-s3.iam.config` when you need: +- **OIDC/OAuth2 integration** with identity providers +- **STS (Security Token Service)** for temporary credentials +- **AWS IAM-style policies** with fine-grained permissions +- **Role-based access control** with trust policies + +> **Important**: The `-s3.iam.config` does NOT support the `identities` field with access keys. If you need simple access key authentication, use `-s3.config` instead. You can use both options together. + +## Configuration Structure + +The IAM configuration file has these main sections: + +```json +{ + "sts": { ... }, // Security Token Service settings + "providers": [ ... ], // OIDC identity providers + "policies": [ ... ], // IAM policy documents + "roles": [ ... ] // IAM roles with trust policies +} +``` + +### STS Configuration + +```json +{ + "sts": { + "tokenDuration": "1h", + "maxSessionLength": "12h", + "issuer": "seaweedfs-sts", + "signingKey": "base64-encoded-32-byte-key" + } +} +``` + +| Field | Description | +|-------|-------------| +| `tokenDuration` | Duration of issued STS tokens | +| `maxSessionLength` | Maximum session length | +| `issuer` | Issuer name for STS tokens | +| `signingKey` | Base64-encoded signing key (32+ bytes) | + +### OIDC Provider Configuration + +```json +{ + "providers": [ + { + "name": "my-idp", + "type": "oidc", + "enabled": true, + "config": { + "issuer": "https://idp.example.com", + "clientId": "seaweedfs-s3", + "clientSecret": "optional-for-confidential-clients", + "jwksUri": "https://idp.example.com/.well-known/jwks.json", + "userInfoUri": "https://idp.example.com/userinfo", + "scopes": ["openid", "profile", "email", "groups"], + "roleMapping": { + "rules": [ + { "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" } + ], + "defaultRole": "arn:aws:iam::role/S3ReadOnlyRole" + } + } + } + ] +} +``` + +### Policies + +```json +{ + "policies": [ + { + "name": "S3ReadOnlyPolicy", + "document": { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", "Action": ["s3:Get*", "s3:List*"], "Resource": ["*"] } + ] + } + } + ] +} +``` + +### Roles + +```json +{ + "roles": [ + { + "roleName": "S3ReadOnlyRole", + "roleArn": "arn:aws:iam::role/S3ReadOnlyRole", + "attachedPolicies": ["S3ReadOnlyPolicy"], + "trustPolicy": { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { "Federated": "*" }, + "Action": ["sts:AssumeRoleWithWebIdentity"], + "Condition": { + "StringEquals": { "seaweed:Issuer": "https://idp.example.com" } + } + } + ] + } + } + ] +} +``` + +--- + +## Keycloak Example + +### Prerequisites + +- A running Keycloak server and a realm (e.g. `seaweedfs`) +- A Keycloak client (e.g. `seaweedfs-s3`) created in that realm +- SeaweedFS with advanced IAM enabled via `-iam.config` + +### Step 1: Configure Keycloak + +1) **Create a client** + - Client ID: `seaweedfs-s3` + - Access Type: public or confidential (confidential requires a client secret) + - Standard Flow: enabled (for browser login flows) + +2) **Add role/group claims to tokens** + - Add a mapper of type "Group Membership" or "User Realm Role" that puts roles/groups into a top-level claim: + - Claim name: `groups` (recommended) or `roles` + - Add to ID token: true + - Add to Access token: true + +3) **Confirm OIDC discovery** + - Open your realm discovery document and note the issuer and certs endpoints: + - Issuer: `https://KEYCLOAK/realms/` + - JWKS (certs): `https://KEYCLOAK/realms//protocol/openid-connect/certs` + - UserInfo: `https://KEYCLOAK/realms//protocol/openid-connect/userinfo` + + Note: For older Keycloak distributions, issuer may include `/auth` in the path. + +### Step 2: Create IAM Config + +Create `/etc/seaweed/iam.json`: + +```json +{ + "sts": { + "tokenDuration": "1h", + "maxSessionLength": "12h", + "issuer": "seaweedfs-sts", + "signingKey": "c2Vhd2VlZGZzLXNpZ25pbmcta2V5LTMyLWNoYXJzLWxvbmc=" + }, + "providers": [ + { + "name": "keycloak", + "type": "oidc", + "enabled": true, + "config": { + "issuer": "https://KEYCLOAK/realms/seaweedfs", + "clientId": "seaweedfs-s3", + "clientSecret": "", + "jwksUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/certs", + "userInfoUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/userinfo", + "scopes": ["openid", "profile", "email", "roles", "groups"], + "roleMapping": { + "rules": [ + { "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" }, + { "claim": "groups", "value": "developers", "role": "arn:aws:iam::role/S3WriteRole" } + ], + "defaultRole": "arn:aws:iam::role/S3ReadOnlyRole" + } + } + } + ], + "policies": [ + { + "name": "S3ReadOnlyPolicy", + "document": { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", "Action": ["s3:List*", "s3:Get*"], "Resource": ["*"] } + ] + } + }, + { + "name": "S3WritePolicy", + "document": { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", "Action": ["s3:List*", "s3:Get*", "s3:Put*", "s3:DeleteObject"], "Resource": ["*"] } + ] + } + }, + { + "name": "S3AdminPolicy", + "document": { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", "Action": ["s3:*"], "Resource": ["*"] } + ] + } + } + ], + "roles": [ + { + "roleName": "S3ReadOnlyRole", + "roleArn": "arn:aws:iam::role/S3ReadOnlyRole", + "attachedPolicies": ["S3ReadOnlyPolicy"], + "trustPolicy": { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { "Federated": "*" }, + "Action": ["sts:AssumeRoleWithWebIdentity"], + "Condition": { + "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } + } + } + ] + } + }, + { + "roleName": "S3WriteRole", + "roleArn": "arn:aws:iam::role/S3WriteRole", + "attachedPolicies": ["S3WritePolicy"], + "trustPolicy": { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { "Federated": "*" }, + "Action": ["sts:AssumeRoleWithWebIdentity"], + "Condition": { + "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } + } + } + ] + } + }, + { + "roleName": "S3AdminRole", + "roleArn": "arn:aws:iam::role/S3AdminRole", + "attachedPolicies": ["S3AdminPolicy"], + "trustPolicy": { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { "Federated": "*" }, + "Action": ["sts:AssumeRoleWithWebIdentity"], + "Condition": { + "StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" } + } + } + ] + } + } + ] +} +``` + +Notes: +- Set `signingKey` to a strong random secret (base64-encoded 32+ bytes). All S3 gateway instances must share the same STS `issuer` and `signingKey`. +- Explicit `jwksUri` and `userInfoUri` are recommended for Keycloak. +- Ensure your Keycloak mappers populate a top-level `groups` (or `roles`) claim. + +### Step 3: Start the S3 Gateway + +```bash +weed s3 -filer=filer:8888 -port=8333 -iam.config=/etc/seaweed/iam.json +``` + +For multi-instance deployments, use the same IAM config on each instance. + +--- + +## Using OIDC Authentication + +Once configured, obtain an access token from your identity provider and call S3 with: + +```bash +curl -H "Authorization: Bearer $OIDC_TOKEN" http://s3-gateway:8333/ +``` + +Role selection is automatic based on your `roleMapping` configuration. With the Keycloak example: +- Users in `admins` group → `S3AdminRole` +- Users in `developers` group → `S3WriteRole` +- All other users → `S3ReadOnlyRole` + +--- + +## Troubleshooting + +| Problem | Solution | +|---------|----------| +| **Invalid token** | Verify token `iss` equals the configured provider `issuer` and `aud` or `azp` equals the client ID | +| **JWKS errors** | Ensure `jwksUri` is reachable from the S3 gateway | +| **No roles applied** | Confirm your IdP emits `groups` (or `roles`) at top-level in the token | +| **Trust policy denied** | Ensure `seaweed:Issuer` in the trust policy matches your IdP issuer exactly | + diff --git a/S3-Configuration.md b/S3-Configuration.md new file mode 100644 index 0000000..48cead8 --- /dev/null +++ b/S3-Configuration.md @@ -0,0 +1,204 @@ +# S3 Configuration Overview + +SeaweedFS S3 gateway has two separate configuration systems for different purposes. Understanding the difference is crucial for proper setup. + +## Quick Reference + +| Option | Purpose | Use When | +|--------|---------|----------| +| `-s3.config` / `-config` | Basic S3 credentials (identities, access keys, actions) | You need simple user authentication with access keys | +| `-s3.iam.config` / `-iam.config` | Advanced IAM (STS, OIDC, policies, roles) | You need OIDC integration, role-based access, or AWS IAM-style policies | + +## Basic Credentials (`-s3.config`) + +**Use this for**: Simple username/password style authentication with access keys. + +**Documentation**: [[S3 Credentials]] + +```bash +weed s3 -config=/path/to/s3.json -filer=localhost:8888 +# OR +weed server -s3 -s3.config=/path/to/s3.json +``` + +### Configuration Format + +```json +{ + "identities": [ + { + "name": "admin_user", + "credentials": [ + { + "accessKey": "your_access_key", + "secretKey": "your_secret_key" + } + ], + "actions": ["Admin", "Read", "Write", "List", "Tagging"] + }, + { + "name": "readonly_user", + "credentials": [ + { + "accessKey": "readonly_key", + "secretKey": "readonly_secret" + } + ], + "actions": ["Read", "List"] + } + ] +} +``` + +### Available Actions + +| Action | Description | +|--------|-------------| +| `Admin` | Full access, create/delete buckets | +| `Read` | Read objects | +| `Write` | Write/upload objects | +| `List` | List buckets and objects | +| `Tagging` | Manage object tags | +| `Read:bucket1` | Read access to specific bucket | +| `Write:bucket1` | Write access to specific bucket | + +--- + +## Advanced IAM (`-s3.iam.config`) + +**Use this for**: Enterprise features like OIDC/Keycloak integration, STS (Security Token Service), IAM policies, and role-based access control. + +**Documentation**: [[OIDC Integration]] + +```bash +weed s3 -filer=localhost:8888 -iam.config=/path/to/iam.json +# OR +weed server -s3 -s3.iam.config=/path/to/iam.json +``` + +### Configuration Format + +```json +{ + "sts": { + "tokenDuration": "1h", + "maxSessionLength": "12h", + "issuer": "seaweedfs-sts", + "signingKey": "base64-encoded-32-byte-key" + }, + "providers": [ + { + "name": "keycloak", + "type": "oidc", + "enabled": true, + "config": { + "issuer": "https://keycloak.example.com/realms/myrealm", + "clientId": "seaweedfs-s3", + "jwksUri": "https://keycloak.example.com/realms/myrealm/protocol/openid-connect/certs" + } + } + ], + "policies": [ + { + "name": "ReadOnlyPolicy", + "document": { + "Version": "2012-10-17", + "Statement": [ + { "Effect": "Allow", "Action": ["s3:Get*", "s3:List*"], "Resource": ["*"] } + ] + } + } + ], + "roles": [ + { + "roleName": "ReadOnlyRole", + "roleArn": "arn:aws:iam::role/ReadOnlyRole", + "attachedPolicies": ["ReadOnlyPolicy"], + "trustPolicy": { ... } + } + ] +} +``` + +### Key Components + +| Component | Description | +|-----------|-------------| +| `sts` | Security Token Service configuration for temporary credentials | +| `providers` | OIDC identity providers (Keycloak, Okta, Auth0, etc.) | +| `policies` | AWS IAM-style policy documents | +| `roles` | IAM roles with trust policies for role assumption | + +> **Important**: The `-s3.iam.config` does NOT support the `identities` field. For basic user credentials, use `-s3.config` instead. + +--- + +## Using Both Together + +You can use both configuration options together: + +```bash +weed s3 \ + -config=/path/to/s3-credentials.json \ + -iam.config=/path/to/iam-advanced.json \ + -filer=localhost:8888 +``` + +This allows: +- Basic users to authenticate with access keys (from `-s3.config`) +- OIDC users to authenticate with JWT tokens (from `-s3.iam.config`) + +--- + +## Common Mistakes + +### Wrong: Using identities in `-s3.iam.config` + +```bash +# This will NOT load identities! +weed s3 -iam.config=/path/to/config.json +``` + +With config file: +```json +{ + "identities": [...] // This is IGNORED by -iam.config +} +``` + +### Correct: Using identities in `-s3.config` + +```bash +weed s3 -config=/path/to/config.json +``` + +With config file: +```json +{ + "identities": [...] // This works with -config +} +``` + +--- + +## Configuration Methods Summary + +| Method | Priority | Auto-Reload | Best For | +|--------|----------|-------------|----------| +| `-config` file | Highest | SIGHUP | Production static config | +| Filer storage | Medium | Yes | Dynamic management | +| [[Admin UI]] | Medium | Yes | Web-based management | +| Environment variables | Fallback | No | Development/testing | + +See [[S3 Credentials]] for detailed information on each method. + +--- + +## Related Documentation + +- [[S3 Credentials]] - Detailed documentation for basic credentials +- [[OIDC Integration]] - OIDC/STS integration guide +- [[Amazon S3 API]] - S3 API compatibility reference +- [[Amazon IAM API]] - IAM API support +- [[Admin UI]] - Web-based credential management + diff --git a/S3-Credentials.md b/S3-Credentials.md index 007b34c..4b2582f 100644 --- a/S3-Credentials.md +++ b/S3-Credentials.md @@ -1,5 +1,13 @@ # S3 Credentials +> **This page covers basic S3 credentials** using the `-s3.config` / `-config` option. +> +> For advanced IAM features (OIDC, STS, IAM policies), see [[OIDC Integration]] which uses the `-s3.iam.config` / `-iam.config` option. +> +> For a comparison of both options, see [[S3 Configuration]]. + +--- + SeaweedFS S3 API supports multiple authentication methods with a clear priority system. This page explains how to configure S3 credentials for your SeaweedFS setup. ## Authentication Methods diff --git a/S3-Rate-Limiting.md b/S3-Rate-Limiting.md index 8843cef..5219d42 100644 --- a/S3-Rate-Limiting.md +++ b/S3-Rate-Limiting.md @@ -160,3 +160,4 @@ Relevant metrics: - `seaweedfs_s3_request_total` - Total requests by action and status (look for 429 status) + diff --git a/TUS-Resumable-Uploads.md b/TUS-Resumable-Uploads.md index 6535d51..3c5c51a 100644 --- a/TUS-Resumable-Uploads.md +++ b/TUS-Resumable-Uploads.md @@ -266,3 +266,4 @@ func main() { - [[Filer Setup]] + diff --git a/_Sidebar.md b/_Sidebar.md index b84aeca..3aa5843 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -68,17 +68,11 @@ * [[Gateway to Remote Object Storage]] ### AWS S3 API -* [[S3 Credentials]] * [[Amazon S3 API]] * [[S3 Conditional Operations]] * [[S3 CORS]] * [[S3 Object Lock and Retention]] * [[S3 Object Versioning]] -* [[AWS CLI with SeaweedFS]] -* [[s3cmd with SeaweedFS]] -* [[rclone with SeaweedFS]] -* [[restic with SeaweedFS]] -* [[nodejs with Seaweed S3]] * [[S3 API Benchmark]] * [[S3 API FAQ]] * [[S3 Bucket Quota]] @@ -87,15 +81,24 @@ * [[S3 Nginx Proxy]] * [[Docker Compose for S3]] +### S3 Authentication & IAM +* [[S3 Configuration]] - Start Here +* [[S3 Credentials]] (`-s3.config`) +* [[OIDC Integration]] (`-s3.iam.config`) +* [[Amazon IAM API]] +* [[AWS IAM CLI]] + ### Server-Side Encryption * [[Server-Side Encryption]] * [[Server-Side Encryption SSE-KMS]] * [[Server-Side Encryption SSE-C]] -### AWS IAM -* [[Amazon IAM API]] -* [[AWS IAM CLI]] -* [[Keycloak Integration]] +### S3 Client Tools +* [[AWS CLI with SeaweedFS]] +* [[s3cmd with SeaweedFS]] +* [[rclone with SeaweedFS]] +* [[restic with SeaweedFS]] +* [[nodejs with Seaweed S3]] ### Machine Learning * [[TensorFlow with SeaweedFS]]