custom attributes

chrislusf
2025-12-02 22:41:37 -08:00
parent 757e493b6a
commit 487e96f640
3 changed files with 148 additions and 18 deletions
+53
@@ -55,6 +55,11 @@ To be sure, you can look at the function defined in the files `weed/s3api/s3api_
* PutObjectTagging
* DeleteObjectTagging
// User Metadata
* PutObject (with x-amz-meta-* headers)
* GetObject / HeadObject (returns x-amz-meta-* headers)
* CopyObject (with metadata directive)
// Server-Side Encryption (NEW)
* PutObject (with SSE-KMS, SSE-C, SSE-S3)
* GetObject (with automatic decryption)
@@ -114,6 +119,54 @@ To be sure, you can look at the function defined in the files `weed/s3api/s3api_
Not included:
* Policy
# User Metadata (Custom Attributes)
SeaweedFS supports S3 user-defined metadata via `x-amz-meta-*` headers. This allows you to attach custom key-value pairs to objects.
## Setting User Metadata
```bash
# Using AWS CLI
aws s3 cp myfile.txt s3://mybucket/myfile.txt \
--metadata "expire=2025-12-01,author=john,project=demo"
# Using curl
curl -X PUT \
-H "x-amz-meta-expire: 2025-12-01" \
-H "x-amz-meta-author: john" \
--data-binary @myfile.txt \
"http://localhost:8333/mybucket/myfile.txt"
```
## Reading User Metadata
User metadata is returned in response headers when you GET or HEAD an object:
```bash
# Using AWS CLI
aws s3api head-object --bucket mybucket --key myfile.txt
# Using curl
curl -I "http://localhost:8333/mybucket/myfile.txt"
# Response includes:
# x-amz-meta-expire: 2025-12-01
# x-amz-meta-author: john
```
## Updating User Metadata
To update metadata, use CopyObject with `x-amz-metadata-directive: REPLACE`:
```bash
aws s3 cp s3://mybucket/myfile.txt s3://mybucket/myfile.txt \
--metadata "expire=2026-01-01" \
--metadata-directive REPLACE
```
## Limits
Metadata keys are case-insensitive and stored in canonical format (e.g., `x-amz-meta-My-Key` becomes `X-Amz-Meta-My-Key`).
# Feature difference
| Feature | SeaweedFS | Amazon S3 |
+42
@@ -13,6 +13,48 @@ With "weed mount", the files can be operated as a local file. The following oper
* display free disk space
* copy file range
* lseek
* extended attributes (xattr)
### Extended Attributes (xattr)
SeaweedFS supports POSIX extended attributes on files and directories via FUSE mount. You can use standard tools like `setfattr`, `getfattr`, and `xattr` to manage custom metadata.
#### Examples
```bash
# Set a custom attribute
setfattr -n user.expire -v "2025-12-01" /mnt/seaweedfs/path/to/file
# Get a custom attribute
getfattr -n user.expire /mnt/seaweedfs/path/to/file
# List all extended attributes
getfattr -d /mnt/seaweedfs/path/to/file
# Remove an extended attribute
setfattr -x user.expire /mnt/seaweedfs/path/to/file
# On macOS, use xattr instead
xattr -w user.expire "2025-12-01" /mnt/seaweedfs/path/to/file
xattr -p user.expire /mnt/seaweedfs/path/to/file
xattr -l /mnt/seaweedfs/path/to/file
xattr -d user.expire /mnt/seaweedfs/path/to/file
```
#### Limits
| Limit | Value |
| ----- | ----- |
| Max attribute name size | 255 bytes |
| Max attribute value size | 64 KB (65536 bytes) |
#### Disabling xattr
If you don't need extended attributes, you can disable them with the `-disableXAttr` flag:
```bash
weed mount -filer=localhost:8888 -dir=/mnt/seaweedfs -disableXAttr
```
### Mount as FUSE
+53 -18
@@ -203,29 +203,64 @@ Examples:
| response-content-disposition | used as response content-disposition | empty |
### PUT/DELETE file tagging
```
# put 2 pairs of meta data
curl -X PUT -H "Seaweed-Name1: value1" -H "Seaweed-some: some string value" http://localhost:8888/path/to/a/file?tagging
# read the meta data from HEAD request
curl -I "http://localhost:8888/path/to/a/file"
...
Seaweed-Name1: value1
Seaweed-Some: some string value
...
# delete all "Seaweed-" prefixed meta data
curl -X DELETE http://localhost:8888/path/to/a/file?tagging
# delete specific "Seaweed-" prefixed meta data
curl -X DELETE http://localhost:8888/path/to/a/file?tagging=Name1,Some
### PUT/DELETE file tagging (Custom Attributes)
The Filer provides a tagging API to attach custom metadata to files and directories using `Seaweed-` prefixed headers. This is useful for storing application-specific attributes like expiration dates, ownership, or classification tags.
#### Setting Custom Attributes
```bash
# Add custom attributes to an existing file
curl -X PUT \
-H "Seaweed-Expire: 2025-12-01" \
-H "Seaweed-Author: john" \
-H "Seaweed-Project: demo" \
"http://localhost:8888/path/to/file?tagging"
# You can also set attributes during file upload
curl -F "file=@myfile.txt" \
-H "Seaweed-Expire: 2025-12-01" \
"http://localhost:8888/path/to/myfile.txt"
```
#### Reading Custom Attributes
Custom attributes are returned as response headers when you HEAD or GET a file:
```bash
curl -I "http://localhost:8888/path/to/file"
# Response includes:
# Seaweed-Expire: 2025-12-01
# Seaweed-Author: john
# Seaweed-Project: demo
```
#### Deleting Custom Attributes
```bash
# Delete all Seaweed-prefixed attributes
curl -X DELETE "http://localhost:8888/path/to/file?tagging"
# Delete specific attributes
curl -X DELETE "http://localhost:8888/path/to/file?tagging=Expire,Author"
```
#### Summary
| Method | Request | Header | Operation |
| ---- | ---- | -- | -- |
| PUT | <file_url>?tagging | Prefixed with "Seaweed-" | set the meta data |
| DELETE | <file_url>?tagging | | remove all the "Seaweed-" prefixed header |
| DELETE | <file_url>?tagging=Some,Name | | remove the headers "Seaweed-Some", "Seaweed-Name" |
| PUT | `<file_url>?tagging` | Prefixed with "Seaweed-" | Set custom attributes |
| DELETE | `<file_url>?tagging` | | Remove all "Seaweed-" prefixed attributes |
| DELETE | `<file_url>?tagging=Key1,Key2` | | Remove specific attributes |
Notice that the tag names follow http header key convention, with the first character capitalized.
Notice that the attribute names follow HTTP header key convention, with the first character capitalized (e.g., `Seaweed-my-key` becomes `Seaweed-My-Key`).
#### Performance
Custom attributes are stored in the filer's metadata database alongside other file metadata. They are:
- Efficiently stored with no separate lookups required
- Replicated with other metadata for consistency
- Not indexed by default (queries by attribute require custom implementation)
### Move files and directories
```bash