mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-23 14:30:46 +02:00
re-org
1 parent
5c3f4054cf
commit
ddde794d33
10 files changed
+575
-203
No files matched your search
@@ -108,4 +108,4 @@ aws --endpoint-url http://localhost:8333 s3 cp file.txt s3://bucket/sse-s3.txt \
|
||||
```
|
||||
|
||||
## OIDC/JWT to S3
|
||||
For Keycloak and other OIDC providers, you can obtain a JWT and access S3 directly (or use STS to assume a role). See [[Keycloak Integration]].
|
||||
For Keycloak and other OIDC providers, you can obtain a JWT and access S3 directly (or use STS to assume a role). See [[OIDC Integration]].
|
||||
+10
-4
@@ -261,10 +261,16 @@ curl -X PUT -H "If-None-Match: *" -d "new content" "http://localhost:8333/mybuck
|
||||
|
||||
# S3 Authentication
|
||||
|
||||
By default, the access key and secret key to access `weed s3` is not authenticated. To enable credential based access, you can choose static or dynamic configuration:
|
||||
* **Dynamic Configuration**: setup auth with `s3.configure` in `weed shell`
|
||||
* **Static Configuration**: create a config.json file similar to the example below, and specify it via `weed s3 -config=config.json`
|
||||
* **OIDC/JWT (Web Identity)**: for Keycloak and other OpenID providers, see [[Keycloak Integration]] for STS and JWT to S3 usage
|
||||
> **For a complete overview of S3 configuration options, see [[S3 Configuration]].**
|
||||
|
||||
By default, the access key and secret key to access `weed s3` is not authenticated. To enable credential based access, you can choose from:
|
||||
|
||||
| Method | Option | Documentation |
|
||||
|--------|--------|---------------|
|
||||
| **Static Configuration** | `-s3.config=config.json` | [[S3 Credentials]] |
|
||||
| **Dynamic Configuration** | `s3.configure` in weed shell | See below |
|
||||
| **Admin UI** | Web interface | [[Admin UI]] |
|
||||
| **OIDC/JWT (Web Identity)** | `-s3.iam.config=iam.json` | [[OIDC Integration]] |
|
||||
|
||||
## Dynamic Configuration
|
||||
|
||||
|
||||
+1
-1
@@ -40,6 +40,6 @@ services:
|
||||
```
|
||||
|
||||
## Notes
|
||||
- To enable advanced IAM (STS, OIDC providers), mount a JSON and add `-iam.config=/etc/seaweedfs/iam.json` to the S3 command. See [[Keycloak Integration]].
|
||||
- To enable advanced IAM (STS, OIDC providers), mount a JSON and add `-iam.config=/etc/seaweedfs/iam.json` to the S3 command. See [[OIDC Integration]].
|
||||
- To use SSE-KMS (with OpenBao/Vault, AWS KMS, GCP KMS), mount KMS config (e.g. `s3_kms.json`) and start with `-config=/etc/seaweedfs/s3_kms.json`. See [[Server-Side-Encryption-SSE-KMS]].
|
||||
- SSE-S3 and bucket default encryption work without external KMS; see [[Server-Side-Encryption]].
|
||||
@@ -1,187 +0,0 @@
|
||||
# Keycloak Integration (OIDC) with SeaweedFS S3 Gateway
|
||||
|
||||
This guide shows how to integrate Keycloak (OpenID Connect) with SeaweedFS S3 Gateway using the advanced IAM and STS configuration. It supports both:
|
||||
|
||||
- Direct OIDC authentication to S3 with Bearer tokens
|
||||
- OIDC to STS role assumption using trust policies and role mapping
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A running Keycloak server and a realm (e.g. `seaweedfs`)
|
||||
- A Keycloak client (e.g. `seaweedfs-s3`) created in that realm
|
||||
- SeaweedFS with advanced IAM enabled via `-iam.config`
|
||||
|
||||
## Step 1: Configure Keycloak
|
||||
|
||||
1) Create a client
|
||||
- Client ID: `seaweedfs-s3`
|
||||
- Access Type: public or confidential (confidential requires a client secret)
|
||||
- Standard Flow: enabled (for browser login flows)
|
||||
|
||||
2) Add role/group claims to tokens
|
||||
- Add a mapper of type "Group Membership" or "User Realm Role" that puts roles/groups into a top-level claim:
|
||||
- Claim name: `groups` (recommended) or `roles`
|
||||
- Add to ID token: true
|
||||
- Add to Access token: true
|
||||
|
||||
3) Confirm OIDC discovery
|
||||
- Open your realm discovery document and note the issuer and certs endpoints (Keycloak Quarkus defaults):
|
||||
- Issuer: `https://KEYCLOAK/realms/<realm>`
|
||||
- JWKS (certs): `https://KEYCLOAK/realms/<realm>/protocol/openid-connect/certs`
|
||||
- UserInfo: `https://KEYCLOAK/realms/<realm>/protocol/openid-connect/userinfo`
|
||||
|
||||
Note: For older Keycloak distributions, issuer may include `/auth` in the path.
|
||||
|
||||
## Step 2: Prepare SeaweedFS IAM config
|
||||
|
||||
Create an IAM configuration JSON file (e.g. `/etc/seaweed/iam_keycloak.json`) and reference it with `weed s3 -iam.config=...`.
|
||||
|
||||
Minimal example with Keycloak OIDC provider, role mapping, roles, and policies:
|
||||
|
||||
```json
|
||||
{
|
||||
"sts": {
|
||||
"tokenDuration": "1h",
|
||||
"maxSessionLength": "12h",
|
||||
"issuer": "seaweedfs-sts",
|
||||
"signingKey": "c2Vhd2VlZGZzLXNpZ25pbmcta2V5LTMyLWNoYXJzLWxvbmc="
|
||||
},
|
||||
"providers": [
|
||||
{
|
||||
"name": "keycloak",
|
||||
"type": "oidc",
|
||||
"enabled": true,
|
||||
"config": {
|
||||
"issuer": "https://KEYCLOAK/realms/seaweedfs",
|
||||
"clientId": "seaweedfs-s3",
|
||||
"clientSecret": "<optional-if-confidential>",
|
||||
"jwksUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/certs",
|
||||
"userInfoUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/userinfo",
|
||||
"scopes": ["openid", "profile", "email", "roles", "groups"],
|
||||
"roleMapping": {
|
||||
"rules": [
|
||||
{ "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" },
|
||||
{ "claim": "groups", "value": "developers", "role": "arn:aws:iam::role/S3WriteRole" }
|
||||
],
|
||||
"defaultRole": "arn:aws:iam::role/S3ReadOnlyRole"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"policies": [
|
||||
{
|
||||
"name": "S3ReadOnlyPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:List*", "s3:Get*"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "S3WritePolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:List*", "s3:Get*", "s3:Put*", "s3:DeleteObject"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "S3AdminPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:*"] , "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"roles": [
|
||||
{
|
||||
"roleName": "S3ReadOnlyRole",
|
||||
"roleArn": "arn:aws:iam::role/S3ReadOnlyRole",
|
||||
"attachedPolicies": ["S3ReadOnlyPolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"roleName": "S3WriteRole",
|
||||
"roleArn": "arn:aws:iam::role/S3WriteRole",
|
||||
"attachedPolicies": ["S3WritePolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"roleName": "S3AdminRole",
|
||||
"roleArn": "arn:aws:iam::role/S3AdminRole",
|
||||
"attachedPolicies": ["S3AdminPolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Set `signingKey` to a strong random secret (base64-encoded 32+ bytes). All S3 gateway instances must share the same STS `issuer` and `signingKey`.
|
||||
- Explicit `jwksUri` and `userInfoUri` are recommended for Keycloak.
|
||||
- Ensure your Keycloak mappers populate a top-level `groups` (or `roles`) claim.
|
||||
|
||||
## Step 3: Start the S3 Gateway
|
||||
|
||||
```bash
|
||||
weed s3 -filer=filer:8888 -port=8333 -iam.config=/etc/seaweed/iam_keycloak.json
|
||||
```
|
||||
|
||||
For multi-instance deployments, use the same IAM config on each instance.
|
||||
|
||||
## Using It
|
||||
|
||||
- Direct OIDC to S3 (Bearer): obtain a Keycloak access or ID token for client `seaweedfs-s3` and call S3 with:
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer $KEYCLOAK_TOKEN" http://s3-gateway:8333/
|
||||
```
|
||||
|
||||
- Role selection: SeaweedFS maps OIDC claims via `roleMapping`. With the example above, users in `admins` get `S3AdminRole`, `developers` get `S3WriteRole`, others default to `S3ReadOnlyRole`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- Invalid token: verify token `iss` equals the configured provider `issuer` and `aud` or `azp` equals the client ID.
|
||||
- JWKS errors: ensure `jwksUri` is reachable from the S3 gateway. For Keycloak, use the `.../protocol/openid-connect/certs` endpoint.
|
||||
- No roles applied: confirm Keycloak mapper emits `groups` (or `roles`) at top-level in the token, and adjust `roleMapping` rules accordingly.
|
||||
- Trust policy denied: ensure `seaweed:Issuer` in the trust policy matches your Keycloak realm issuer exactly.
|
||||
+336
@@ -0,0 +1,336 @@
|
||||
# OIDC Integration with SeaweedFS S3 Gateway
|
||||
|
||||
> **This page covers advanced IAM features** using the `-s3.iam.config` / `-iam.config` option.
|
||||
>
|
||||
> For basic S3 credentials with access keys, see [[S3 Credentials]] which uses the `-s3.config` / `-config` option.
|
||||
>
|
||||
> For a comparison of both options, see [[S3 Configuration]].
|
||||
|
||||
---
|
||||
|
||||
This guide shows how to integrate OpenID Connect (OIDC) identity providers with SeaweedFS S3 Gateway using the advanced IAM and STS configuration. It supports:
|
||||
|
||||
- Direct OIDC authentication to S3 with Bearer tokens
|
||||
- OIDC to STS role assumption using trust policies and role mapping
|
||||
|
||||
## Supported Identity Providers
|
||||
|
||||
SeaweedFS works with any OIDC-compliant identity provider, including:
|
||||
|
||||
| Provider | Configuration Notes |
|
||||
|----------|-------------------|
|
||||
| **Keycloak** | See [Keycloak Example](#keycloak-example) below |
|
||||
| **Okta** | Use your Okta domain as issuer |
|
||||
| **Auth0** | Use `https://YOUR_DOMAIN.auth0.com/` as issuer |
|
||||
| **Azure AD** | Use `https://login.microsoftonline.com/TENANT_ID/v2.0` as issuer |
|
||||
| **Google** | Use `https://accounts.google.com` as issuer |
|
||||
| **AWS Cognito** | Use `https://cognito-idp.REGION.amazonaws.com/POOL_ID` as issuer |
|
||||
|
||||
## When to Use This
|
||||
|
||||
Use `-s3.iam.config` when you need:
|
||||
- **OIDC/OAuth2 integration** with identity providers
|
||||
- **STS (Security Token Service)** for temporary credentials
|
||||
- **AWS IAM-style policies** with fine-grained permissions
|
||||
- **Role-based access control** with trust policies
|
||||
|
||||
> **Important**: The `-s3.iam.config` does NOT support the `identities` field with access keys. If you need simple access key authentication, use `-s3.config` instead. You can use both options together.
|
||||
|
||||
## Configuration Structure
|
||||
|
||||
The IAM configuration file has these main sections:
|
||||
|
||||
```json
|
||||
{
|
||||
"sts": { ... }, // Security Token Service settings
|
||||
"providers": [ ... ], // OIDC identity providers
|
||||
"policies": [ ... ], // IAM policy documents
|
||||
"roles": [ ... ] // IAM roles with trust policies
|
||||
}
|
||||
```
|
||||
|
||||
### STS Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"sts": {
|
||||
"tokenDuration": "1h",
|
||||
"maxSessionLength": "12h",
|
||||
"issuer": "seaweedfs-sts",
|
||||
"signingKey": "base64-encoded-32-byte-key"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `tokenDuration` | Duration of issued STS tokens |
|
||||
| `maxSessionLength` | Maximum session length |
|
||||
| `issuer` | Issuer name for STS tokens |
|
||||
| `signingKey` | Base64-encoded signing key (32+ bytes) |
|
||||
|
||||
### OIDC Provider Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": [
|
||||
{
|
||||
"name": "my-idp",
|
||||
"type": "oidc",
|
||||
"enabled": true,
|
||||
"config": {
|
||||
"issuer": "https://idp.example.com",
|
||||
"clientId": "seaweedfs-s3",
|
||||
"clientSecret": "optional-for-confidential-clients",
|
||||
"jwksUri": "https://idp.example.com/.well-known/jwks.json",
|
||||
"userInfoUri": "https://idp.example.com/userinfo",
|
||||
"scopes": ["openid", "profile", "email", "groups"],
|
||||
"roleMapping": {
|
||||
"rules": [
|
||||
{ "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" }
|
||||
],
|
||||
"defaultRole": "arn:aws:iam::role/S3ReadOnlyRole"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Policies
|
||||
|
||||
```json
|
||||
{
|
||||
"policies": [
|
||||
{
|
||||
"name": "S3ReadOnlyPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:Get*", "s3:List*"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Roles
|
||||
|
||||
```json
|
||||
{
|
||||
"roles": [
|
||||
{
|
||||
"roleName": "S3ReadOnlyRole",
|
||||
"roleArn": "arn:aws:iam::role/S3ReadOnlyRole",
|
||||
"attachedPolicies": ["S3ReadOnlyPolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://idp.example.com" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Keycloak Example
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- A running Keycloak server and a realm (e.g. `seaweedfs`)
|
||||
- A Keycloak client (e.g. `seaweedfs-s3`) created in that realm
|
||||
- SeaweedFS with advanced IAM enabled via `-iam.config`
|
||||
|
||||
### Step 1: Configure Keycloak
|
||||
|
||||
1) **Create a client**
|
||||
- Client ID: `seaweedfs-s3`
|
||||
- Access Type: public or confidential (confidential requires a client secret)
|
||||
- Standard Flow: enabled (for browser login flows)
|
||||
|
||||
2) **Add role/group claims to tokens**
|
||||
- Add a mapper of type "Group Membership" or "User Realm Role" that puts roles/groups into a top-level claim:
|
||||
- Claim name: `groups` (recommended) or `roles`
|
||||
- Add to ID token: true
|
||||
- Add to Access token: true
|
||||
|
||||
3) **Confirm OIDC discovery**
|
||||
- Open your realm discovery document and note the issuer and certs endpoints:
|
||||
- Issuer: `https://KEYCLOAK/realms/<realm>`
|
||||
- JWKS (certs): `https://KEYCLOAK/realms/<realm>/protocol/openid-connect/certs`
|
||||
- UserInfo: `https://KEYCLOAK/realms/<realm>/protocol/openid-connect/userinfo`
|
||||
|
||||
Note: For older Keycloak distributions, issuer may include `/auth` in the path.
|
||||
|
||||
### Step 2: Create IAM Config
|
||||
|
||||
Create `/etc/seaweed/iam.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"sts": {
|
||||
"tokenDuration": "1h",
|
||||
"maxSessionLength": "12h",
|
||||
"issuer": "seaweedfs-sts",
|
||||
"signingKey": "c2Vhd2VlZGZzLXNpZ25pbmcta2V5LTMyLWNoYXJzLWxvbmc="
|
||||
},
|
||||
"providers": [
|
||||
{
|
||||
"name": "keycloak",
|
||||
"type": "oidc",
|
||||
"enabled": true,
|
||||
"config": {
|
||||
"issuer": "https://KEYCLOAK/realms/seaweedfs",
|
||||
"clientId": "seaweedfs-s3",
|
||||
"clientSecret": "<optional-if-confidential>",
|
||||
"jwksUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/certs",
|
||||
"userInfoUri": "https://KEYCLOAK/realms/seaweedfs/protocol/openid-connect/userinfo",
|
||||
"scopes": ["openid", "profile", "email", "roles", "groups"],
|
||||
"roleMapping": {
|
||||
"rules": [
|
||||
{ "claim": "groups", "value": "admins", "role": "arn:aws:iam::role/S3AdminRole" },
|
||||
{ "claim": "groups", "value": "developers", "role": "arn:aws:iam::role/S3WriteRole" }
|
||||
],
|
||||
"defaultRole": "arn:aws:iam::role/S3ReadOnlyRole"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"policies": [
|
||||
{
|
||||
"name": "S3ReadOnlyPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:List*", "s3:Get*"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "S3WritePolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:List*", "s3:Get*", "s3:Put*", "s3:DeleteObject"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "S3AdminPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:*"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"roles": [
|
||||
{
|
||||
"roleName": "S3ReadOnlyRole",
|
||||
"roleArn": "arn:aws:iam::role/S3ReadOnlyRole",
|
||||
"attachedPolicies": ["S3ReadOnlyPolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"roleName": "S3WriteRole",
|
||||
"roleArn": "arn:aws:iam::role/S3WriteRole",
|
||||
"attachedPolicies": ["S3WritePolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"roleName": "S3AdminRole",
|
||||
"roleArn": "arn:aws:iam::role/S3AdminRole",
|
||||
"attachedPolicies": ["S3AdminPolicy"],
|
||||
"trustPolicy": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": { "Federated": "*" },
|
||||
"Action": ["sts:AssumeRoleWithWebIdentity"],
|
||||
"Condition": {
|
||||
"StringEquals": { "seaweed:Issuer": "https://KEYCLOAK/realms/seaweedfs" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- Set `signingKey` to a strong random secret (base64-encoded 32+ bytes). All S3 gateway instances must share the same STS `issuer` and `signingKey`.
|
||||
- Explicit `jwksUri` and `userInfoUri` are recommended for Keycloak.
|
||||
- Ensure your Keycloak mappers populate a top-level `groups` (or `roles`) claim.
|
||||
|
||||
### Step 3: Start the S3 Gateway
|
||||
|
||||
```bash
|
||||
weed s3 -filer=filer:8888 -port=8333 -iam.config=/etc/seaweed/iam.json
|
||||
```
|
||||
|
||||
For multi-instance deployments, use the same IAM config on each instance.
|
||||
|
||||
---
|
||||
|
||||
## Using OIDC Authentication
|
||||
|
||||
Once configured, obtain an access token from your identity provider and call S3 with:
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer $OIDC_TOKEN" http://s3-gateway:8333/
|
||||
```
|
||||
|
||||
Role selection is automatic based on your `roleMapping` configuration. With the Keycloak example:
|
||||
- Users in `admins` group → `S3AdminRole`
|
||||
- Users in `developers` group → `S3WriteRole`
|
||||
- All other users → `S3ReadOnlyRole`
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Solution |
|
||||
|---------|----------|
|
||||
| **Invalid token** | Verify token `iss` equals the configured provider `issuer` and `aud` or `azp` equals the client ID |
|
||||
| **JWKS errors** | Ensure `jwksUri` is reachable from the S3 gateway |
|
||||
| **No roles applied** | Confirm your IdP emits `groups` (or `roles`) at top-level in the token |
|
||||
| **Trust policy denied** | Ensure `seaweed:Issuer` in the trust policy matches your IdP issuer exactly |
|
||||
|
||||
+204
@@ -0,0 +1,204 @@
|
||||
# S3 Configuration Overview
|
||||
|
||||
SeaweedFS S3 gateway has two separate configuration systems for different purposes. Understanding the difference is crucial for proper setup.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Option | Purpose | Use When |
|
||||
|--------|---------|----------|
|
||||
| `-s3.config` / `-config` | Basic S3 credentials (identities, access keys, actions) | You need simple user authentication with access keys |
|
||||
| `-s3.iam.config` / `-iam.config` | Advanced IAM (STS, OIDC, policies, roles) | You need OIDC integration, role-based access, or AWS IAM-style policies |
|
||||
|
||||
## Basic Credentials (`-s3.config`)
|
||||
|
||||
**Use this for**: Simple username/password style authentication with access keys.
|
||||
|
||||
**Documentation**: [[S3 Credentials]]
|
||||
|
||||
```bash
|
||||
weed s3 -config=/path/to/s3.json -filer=localhost:8888
|
||||
# OR
|
||||
weed server -s3 -s3.config=/path/to/s3.json
|
||||
```
|
||||
|
||||
### Configuration Format
|
||||
|
||||
```json
|
||||
{
|
||||
"identities": [
|
||||
{
|
||||
"name": "admin_user",
|
||||
"credentials": [
|
||||
{
|
||||
"accessKey": "your_access_key",
|
||||
"secretKey": "your_secret_key"
|
||||
}
|
||||
],
|
||||
"actions": ["Admin", "Read", "Write", "List", "Tagging"]
|
||||
},
|
||||
{
|
||||
"name": "readonly_user",
|
||||
"credentials": [
|
||||
{
|
||||
"accessKey": "readonly_key",
|
||||
"secretKey": "readonly_secret"
|
||||
}
|
||||
],
|
||||
"actions": ["Read", "List"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Available Actions
|
||||
|
||||
| Action | Description |
|
||||
|--------|-------------|
|
||||
| `Admin` | Full access, create/delete buckets |
|
||||
| `Read` | Read objects |
|
||||
| `Write` | Write/upload objects |
|
||||
| `List` | List buckets and objects |
|
||||
| `Tagging` | Manage object tags |
|
||||
| `Read:bucket1` | Read access to specific bucket |
|
||||
| `Write:bucket1` | Write access to specific bucket |
|
||||
|
||||
---
|
||||
|
||||
## Advanced IAM (`-s3.iam.config`)
|
||||
|
||||
**Use this for**: Enterprise features like OIDC/Keycloak integration, STS (Security Token Service), IAM policies, and role-based access control.
|
||||
|
||||
**Documentation**: [[OIDC Integration]]
|
||||
|
||||
```bash
|
||||
weed s3 -filer=localhost:8888 -iam.config=/path/to/iam.json
|
||||
# OR
|
||||
weed server -s3 -s3.iam.config=/path/to/iam.json
|
||||
```
|
||||
|
||||
### Configuration Format
|
||||
|
||||
```json
|
||||
{
|
||||
"sts": {
|
||||
"tokenDuration": "1h",
|
||||
"maxSessionLength": "12h",
|
||||
"issuer": "seaweedfs-sts",
|
||||
"signingKey": "base64-encoded-32-byte-key"
|
||||
},
|
||||
"providers": [
|
||||
{
|
||||
"name": "keycloak",
|
||||
"type": "oidc",
|
||||
"enabled": true,
|
||||
"config": {
|
||||
"issuer": "https://keycloak.example.com/realms/myrealm",
|
||||
"clientId": "seaweedfs-s3",
|
||||
"jwksUri": "https://keycloak.example.com/realms/myrealm/protocol/openid-connect/certs"
|
||||
}
|
||||
}
|
||||
],
|
||||
"policies": [
|
||||
{
|
||||
"name": "ReadOnlyPolicy",
|
||||
"document": {
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{ "Effect": "Allow", "Action": ["s3:Get*", "s3:List*"], "Resource": ["*"] }
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"roles": [
|
||||
{
|
||||
"roleName": "ReadOnlyRole",
|
||||
"roleArn": "arn:aws:iam::role/ReadOnlyRole",
|
||||
"attachedPolicies": ["ReadOnlyPolicy"],
|
||||
"trustPolicy": { ... }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
| Component | Description |
|
||||
|-----------|-------------|
|
||||
| `sts` | Security Token Service configuration for temporary credentials |
|
||||
| `providers` | OIDC identity providers (Keycloak, Okta, Auth0, etc.) |
|
||||
| `policies` | AWS IAM-style policy documents |
|
||||
| `roles` | IAM roles with trust policies for role assumption |
|
||||
|
||||
> **Important**: The `-s3.iam.config` does NOT support the `identities` field. For basic user credentials, use `-s3.config` instead.
|
||||
|
||||
---
|
||||
|
||||
## Using Both Together
|
||||
|
||||
You can use both configuration options together:
|
||||
|
||||
```bash
|
||||
weed s3 \
|
||||
-config=/path/to/s3-credentials.json \
|
||||
-iam.config=/path/to/iam-advanced.json \
|
||||
-filer=localhost:8888
|
||||
```
|
||||
|
||||
This allows:
|
||||
- Basic users to authenticate with access keys (from `-s3.config`)
|
||||
- OIDC users to authenticate with JWT tokens (from `-s3.iam.config`)
|
||||
|
||||
---
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
### Wrong: Using identities in `-s3.iam.config`
|
||||
|
||||
```bash
|
||||
# This will NOT load identities!
|
||||
weed s3 -iam.config=/path/to/config.json
|
||||
```
|
||||
|
||||
With config file:
|
||||
```json
|
||||
{
|
||||
"identities": [...] // This is IGNORED by -iam.config
|
||||
}
|
||||
```
|
||||
|
||||
### Correct: Using identities in `-s3.config`
|
||||
|
||||
```bash
|
||||
weed s3 -config=/path/to/config.json
|
||||
```
|
||||
|
||||
With config file:
|
||||
```json
|
||||
{
|
||||
"identities": [...] // This works with -config
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Methods Summary
|
||||
|
||||
| Method | Priority | Auto-Reload | Best For |
|
||||
|--------|----------|-------------|----------|
|
||||
| `-config` file | Highest | SIGHUP | Production static config |
|
||||
| Filer storage | Medium | Yes | Dynamic management |
|
||||
| [[Admin UI]] | Medium | Yes | Web-based management |
|
||||
| Environment variables | Fallback | No | Development/testing |
|
||||
|
||||
See [[S3 Credentials]] for detailed information on each method.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [[S3 Credentials]] - Detailed documentation for basic credentials
|
||||
- [[OIDC Integration]] - OIDC/STS integration guide
|
||||
- [[Amazon S3 API]] - S3 API compatibility reference
|
||||
- [[Amazon IAM API]] - IAM API support
|
||||
- [[Admin UI]] - Web-based credential management
|
||||
|
||||
@@ -1,5 +1,13 @@
|
||||
# S3 Credentials
|
||||
|
||||
> **This page covers basic S3 credentials** using the `-s3.config` / `-config` option.
|
||||
>
|
||||
> For advanced IAM features (OIDC, STS, IAM policies), see [[OIDC Integration]] which uses the `-s3.iam.config` / `-iam.config` option.
|
||||
>
|
||||
> For a comparison of both options, see [[S3 Configuration]].
|
||||
|
||||
---
|
||||
|
||||
SeaweedFS S3 API supports multiple authentication methods with a clear priority system. This page explains how to configure S3 credentials for your SeaweedFS setup.
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
@@ -160,3 +160,4 @@ Relevant metrics:
|
||||
- `seaweedfs_s3_request_total` - Total requests by action and status (look for 429 status)
|
||||
|
||||
|
||||
|
||||
@@ -266,3 +266,4 @@ func main() {
|
||||
- [[Filer Setup]]
|
||||
|
||||
|
||||
|
||||
+13
-10
@@ -68,17 +68,11 @@
|
||||
* [[Gateway to Remote Object Storage]]
|
||||
|
||||
### AWS S3 API
|
||||
* [[S3 Credentials]]
|
||||
* [[Amazon S3 API]]
|
||||
* [[S3 Conditional Operations]]
|
||||
* [[S3 CORS]]
|
||||
* [[S3 Object Lock and Retention]]
|
||||
* [[S3 Object Versioning]]
|
||||
* [[AWS CLI with SeaweedFS]]
|
||||
* [[s3cmd with SeaweedFS]]
|
||||
* [[rclone with SeaweedFS]]
|
||||
* [[restic with SeaweedFS]]
|
||||
* [[nodejs with Seaweed S3]]
|
||||
* [[S3 API Benchmark]]
|
||||
* [[S3 API FAQ]]
|
||||
* [[S3 Bucket Quota]]
|
||||
@@ -87,15 +81,24 @@
|
||||
* [[S3 Nginx Proxy]]
|
||||
* [[Docker Compose for S3]]
|
||||
|
||||
### S3 Authentication & IAM
|
||||
* [[S3 Configuration]] - Start Here
|
||||
* [[S3 Credentials]] (`-s3.config`)
|
||||
* [[OIDC Integration]] (`-s3.iam.config`)
|
||||
* [[Amazon IAM API]]
|
||||
* [[AWS IAM CLI]]
|
||||
|
||||
### Server-Side Encryption
|
||||
* [[Server-Side Encryption]]
|
||||
* [[Server-Side Encryption SSE-KMS]]
|
||||
* [[Server-Side Encryption SSE-C]]
|
||||
|
||||
### AWS IAM
|
||||
* [[Amazon IAM API]]
|
||||
* [[AWS IAM CLI]]
|
||||
* [[Keycloak Integration]]
|
||||
### S3 Client Tools
|
||||
* [[AWS CLI with SeaweedFS]]
|
||||
* [[s3cmd with SeaweedFS]]
|
||||
* [[rclone with SeaweedFS]]
|
||||
* [[restic with SeaweedFS]]
|
||||
* [[nodejs with Seaweed S3]]
|
||||
|
||||
### Machine Learning
|
||||
* [[TensorFlow with SeaweedFS]]
|
||||
|
||||
Reference in new issue
Block a user