add s3 credentials

2026-10-11 08:47:46 +02:00 · 2025-07-23 11:45:04 -07:00
1 parent 36ddae236f
commit 6503963c27
@@ -14,50 +14,28 @@ weed master
 For `v`, `logtostderr`, `stderrthreshold`, `vmoudle`, `options`, `logdir`, `alsologtostderr`, `log_backtrace_at` , and `config_dir` you have to use `WEED_` as prefix for environment variable like this `WEED_CONFIG_DIR=/tmp`

 # S3 Admin Credentials
-For S3 API server, you can use standard AWS environment variables to set default admin credentials:
-
-```shell
-export AWS_ACCESS_KEY_ID=your_access_key
-export AWS_SECRET_ACCESS_KEY=your_secret_key
-weed s3 -filer=localhost:8888
-```
-
-These environment variables will supplement existing configuration by adding admin credentials when:
- Both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are set
- No identity with the same access key already exists
-
-**Configuration Priority:**
-1. **File or Filer configuration** is loaded first (if provided)
-2. **Environment variables** are added second, supplementing the existing configuration
-3. Environment variables are **skipped** if an access key conflict exists
-
-This approach ensures that:
- Existing S3 configuration files (with `admin_access` etc.) are preserved
- Environment variables provide additional admin access without conflicts
- AWS standard environment variables work seamlessly alongside existing setups
-
-The admin identity from environment variables will have:
- Name: `admin-` followed by the access key (or first 8 characters if longer than 8)
- Access Key: value of `AWS_ACCESS_KEY_ID`
- Secret Key: value of `AWS_SECRET_ACCESS_KEY`
- Permissions: Full admin access to all S3 operations
-
-This follows the standard AWS credential convention, making it compatible with existing AWS tooling and workflows.
+For S3 API server authentication, see the dedicated **[S3 Credentials](S3-Credentials)** page which covers:
+- Configuration file setup (highest priority)
+- Filer configuration (medium priority) 
+- Environment variables as fallback (lowest priority)
+- AWS standard environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
+- Complete authentication examples and troubleshooting

 # Docker
-This is useful for using docker and docker compose  
-You have to override entrypoint to `weed` because defautl [entrypoint](https://github.com/seaweedfs/seaweedfs/blob/master/docker/entrypoint.sh) use default values for `volumeSizeLimitMB`, `volumePreallocate`, `mdir`, `dir`, and `max` and setting environment variables won't change these values.
-## Docker
+You can set environment variables easily in Docker:
 ```shell
-docker run --entrypoint weed -it -e IP_BIND=0.0.0.0 -e MDIR=/tmp -e PORT=5000 -e VOLUMEPREALLOCATE=true chrislusf/seaweedfs:3.45 master
+docker run --name master -d -p 9333:9333 -p 19333:19333 \
+  -e MDIR="/data" -e PORT="9333" \
+  chrislusf/seaweedfs:latest \
+  master 
 ```

-## Docker Compose
+## Docker Compose with Environment Variables
 ```yaml
 version: '3.9'
 services:
  master:
-    image: chrislusf/seaweedfs:3.45
+    image: chrislusf/seaweedfs:latest
    ports:
      - 9333:9333
      - 19333:19333
@@ -69,12 +47,7 @@ services:
      # or `VOLUMEPREALLOCATE:`
    entrypoint: weed
    command: master 
-```

-## Docker Compose with S3 Admin Credentials
-```yaml
-version: '3.9'
-services:
  filer:
    image: chrislusf/seaweedfs:latest
    ports:
@@ -0,0 +1,257 @@
+# S3 Credentials
+
+SeaweedFS S3 API supports multiple authentication methods with a clear priority system. This page explains how to configure S3 credentials for your SeaweedFS setup.
+
+## Authentication Methods
+
+### 1. Configuration File (Highest Priority)
+
+Create a JSON configuration file and use the `-config` option:
+
+```json
+{
+  "identities": [
+    {
+      "name": "admin_user",
+      "credentials": [
+        {
+          "accessKey": "admin_access_key",
+          "secretKey": "admin_secret_key"
+        }
+      ],
+      "actions": ["Admin", "Read", "Write"]
+    },
+    {
+      "name": "read_only_user",
+      "credentials": [
+        {
+          "accessKey": "readonly_access_key",
+          "secretKey": "readonly_secret_key"
+        }
+      ],
+      "actions": ["Read"]
+    }
+  ]
+}
+```
+
+Start S3 server with config file:
+```bash
+weed s3 -config=/path/to/s3.json -filer=localhost:8888
+```
+
+### 2. Filer Configuration (Medium Priority)
+
+Store configuration in the filer using the credential manager. This allows dynamic configuration updates without restarting the S3 server.
+
+### 3. Environment Variables (Fallback)
+
+Use AWS standard environment variables as a fallback when no other configuration is available:
+
+```bash
+export AWS_ACCESS_KEY_ID=your_access_key
+export AWS_SECRET_ACCESS_KEY=your_secret_key
+weed s3 -filer=localhost:8888
+```
+
+**Important**: Environment variables are only used when:
+- No `-config` option is provided
+- No configuration is available from the filer
+- Both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are set
+
+## Priority System
+
+SeaweedFS uses the following priority order for S3 credentials:
+
+1. **Configuration File** (if `-config` option is provided)
+2. **Filer Configuration** (if available and no config file)
+3. **Environment Variables** (fallback only)
+
+Higher priority methods completely override lower priority methods - there is no merging or supplementing.
+
+## Configuration Examples
+
+### Production Setup
+```bash
+# Use dedicated configuration file
+weed s3 -config=/etc/seaweedfs/s3.json -filer=filer1:8888,filer2:8888
+```
+
+### Development Setup
+```bash
+# Use environment variables for quick setup
+export AWS_ACCESS_KEY_ID=dev_access_key
+export AWS_SECRET_ACCESS_KEY=dev_secret_key
+weed s3 -filer=localhost:8888
+```
+
+### Docker Compose
+```yaml
+version: '3.9'
+services:
+  s3:
+    image: chrislusf/seaweedfs:latest
+    ports:
+      - 8333:8333
+    environment:
+      AWS_ACCESS_KEY_ID: s3admin
+      AWS_SECRET_ACCESS_KEY: s3secret
+    entrypoint: weed
+    command: s3 -filer=filer:8888
+    depends_on:
+      - filer
+```
+
+## Credential Features
+
+### Actions
+Identities can have different permission levels:
+- `Admin`: Full access to all S3 operations
+- `Read`: Read-only access
+- `Write`: Read and write access
+- `Read_ACP`: Read access control permissions
+- `Write_ACP`: Write access control permissions
+
+### Multiple Credentials
+Each identity can have multiple access key/secret key pairs:
+
+```json
+{
+  "name": "multi_key_user",
+  "credentials": [
+    {
+      "accessKey": "key1",
+      "secretKey": "secret1"
+    },
+    {
+      "accessKey": "key2", 
+      "secretKey": "secret2"
+    }
+  ],
+  "actions": ["Read", "Write"]
+}
+```
+
+### Account Management
+Identities can be associated with accounts for better organization and cross-account access control.
+
+## Anonymous Access
+
+By default, if no credentials are configured, SeaweedFS allows anonymous access to all S3 operations. To enable authentication:
+
+1. Configure at least one identity using any of the methods above
+2. Authentication will be automatically enabled
+3. All requests will require valid credentials
+
+## Configuration Reloading
+
+SeaweedFS supports different reloading mechanisms depending on which authentication method you use:
+
+| Configuration Method | Auto Reload | Manual Reload | Live Reload |
+|---------------------|-------------|---------------|-------------|
+| **Configuration File** (`-config` option) | ❌ No | ✅ SIGHUP | ❌ No |
+| **Filer Configuration** (credential manager) | ✅ Yes | ✅ Yes | ✅ Yes |
+| **Environment Variables** | ❌ No | ❌ No | ❌ No |
+
+### Static Configuration Files
+
+When using the `-config` option, you can reload the configuration by sending a SIGHUP signal:
+
+```bash
+# Find the S3 server process ID
+ps aux | grep "weed s3"
+
+# Send SIGHUP signal to reload configuration
+kill -HUP <seaweedfs_s3_pid>
+
+# Or if using systemd
+systemctl reload seaweedfs-s3
+```
+
+The server will log the reload:
+```
+I0723 12:34:56.789 s3api_server.go:98] Loaded 3 identities from config file /etc/seaweedfs/s3.json
+```
+
+### Filer-based Configuration
+
+Filer-based configurations automatically reload when changes are detected:
+
+```bash
+# Changes are automatically applied
+weed shell
+> s3.configure -user=newuser -access_key=key123 -secret_key=secret123 -actions=Admin -apply
+```
+
+The server will automatically detect and apply changes:
+```
+I0723 12:35:12.456 auth_credentials_subscribe.go:55] updated /etc/seaweedfs/iam/identity.json
+```
+
+### Environment Variables
+
+Environment variable changes require a complete restart of the S3 server:
+
+```bash
+# Update environment variables
+export AWS_ACCESS_KEY_ID=new_access_key
+export AWS_SECRET_ACCESS_KEY=new_secret_key
+
+# Restart the S3 server
+systemctl restart seaweedfs-s3
+```
+
+### Verifying Configuration Reloads
+
+Monitor the logs to verify configuration updates:
+
+```bash
+# Watch for reload messages
+tail -f /var/log/seaweedfs/s3.log | grep -E "updated|Loaded.*identities"
+
+# Check current configuration via shell
+weed shell
+> s3.configure
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**Environment variables not working:**
+- Check that no `-config` option is provided
+- Verify no configuration exists in the filer
+- Ensure both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are set
+
+**Configuration file not loading:**
+- Verify the file path is correct
+- Check JSON syntax is valid
+- Ensure the file is readable by the SeaweedFS process
+
+**Invalid credentials error:**
+- Verify access key and secret key are correct
+- Check that the identity has the required actions/permissions
+- Ensure the credential store is properly configured
+
+### Debug Commands
+
+Check current configuration:
+```bash
+# View current identities (if using filer store)
+weed shell
+> s3.configure -list
+```
+
+Test credentials:
+```bash
+# Test with AWS CLI
+aws --endpoint-url=http://localhost:8333 s3 ls
+```
+
+## Security Best Practices
+
+1. **Use Configuration Files in Production**: Environment variables are visible in process lists
+2. **Rotate Credentials Regularly**: Update access keys and secret keys periodically  
+3. **Principle of Least Privilege**: Grant only the minimum required permissions
+4. **Secure Storage**: Store configuration files with appropriate file permissions
+5. **Monitor Access**: Enable audit logging to track S3 API usage 
@@ -66,6 +66,7 @@
 * [[Gateway to Remote Object Storage]]

 ### AWS S3 API
+* [[S3 Credentials]]
 * [[Amazon S3 API]]
 * [[S3 Object Lock and Retention]]
 * [[AWS CLI with SeaweedFS]]