From 487e96f640c868301e59e2cd485b4d2217c0b722 Mon Sep 17 00:00:00 2001 From: chrislusf Date: Tue, 2 Dec 2025 22:41:37 -0800 Subject: [PATCH] custom attributes --- Amazon-S3-API.md | 53 +++++++++++++++++++++++++++++++++ FUSE-Mount.md | 42 +++++++++++++++++++++++++++ Filer-Server-API.md | 71 +++++++++++++++++++++++++++++++++------------ 3 files changed, 148 insertions(+), 18 deletions(-) diff --git a/Amazon-S3-API.md b/Amazon-S3-API.md index 8de6317..9166164 100644 --- a/Amazon-S3-API.md +++ b/Amazon-S3-API.md @@ -55,6 +55,11 @@ To be sure, you can look at the function defined in the files `weed/s3api/s3api_ * PutObjectTagging * DeleteObjectTagging +// User Metadata +* PutObject (with x-amz-meta-* headers) +* GetObject / HeadObject (returns x-amz-meta-* headers) +* CopyObject (with metadata directive) + // Server-Side Encryption (NEW) * PutObject (with SSE-KMS, SSE-C, SSE-S3) * GetObject (with automatic decryption) @@ -114,6 +119,54 @@ To be sure, you can look at the function defined in the files `weed/s3api/s3api_ Not included: * Policy +# User Metadata (Custom Attributes) + +SeaweedFS supports S3 user-defined metadata via `x-amz-meta-*` headers. This allows you to attach custom key-value pairs to objects. + +## Setting User Metadata + +```bash +# Using AWS CLI +aws s3 cp myfile.txt s3://mybucket/myfile.txt \ + --metadata "expire=2025-12-01,author=john,project=demo" + +# Using curl +curl -X PUT \ + -H "x-amz-meta-expire: 2025-12-01" \ + -H "x-amz-meta-author: john" \ + --data-binary @myfile.txt \ + "http://localhost:8333/mybucket/myfile.txt" +``` + +## Reading User Metadata + +User metadata is returned in response headers when you GET or HEAD an object: + +```bash +# Using AWS CLI +aws s3api head-object --bucket mybucket --key myfile.txt + +# Using curl +curl -I "http://localhost:8333/mybucket/myfile.txt" +# Response includes: +# x-amz-meta-expire: 2025-12-01 +# x-amz-meta-author: john +``` + +## Updating User Metadata + +To update metadata, use CopyObject with `x-amz-metadata-directive: REPLACE`: + +```bash +aws s3 cp s3://mybucket/myfile.txt s3://mybucket/myfile.txt \ + --metadata "expire=2026-01-01" \ + --metadata-directive REPLACE +``` + +## Limits + +Metadata keys are case-insensitive and stored in canonical format (e.g., `x-amz-meta-My-Key` becomes `X-Amz-Meta-My-Key`). + # Feature difference | Feature | SeaweedFS | Amazon S3 | diff --git a/FUSE-Mount.md b/FUSE-Mount.md index 2c168a7..3e590e1 100644 --- a/FUSE-Mount.md +++ b/FUSE-Mount.md @@ -13,6 +13,48 @@ With "weed mount", the files can be operated as a local file. The following oper * display free disk space * copy file range * lseek +* extended attributes (xattr) + +### Extended Attributes (xattr) + +SeaweedFS supports POSIX extended attributes on files and directories via FUSE mount. You can use standard tools like `setfattr`, `getfattr`, and `xattr` to manage custom metadata. + +#### Examples + +```bash +# Set a custom attribute +setfattr -n user.expire -v "2025-12-01" /mnt/seaweedfs/path/to/file + +# Get a custom attribute +getfattr -n user.expire /mnt/seaweedfs/path/to/file + +# List all extended attributes +getfattr -d /mnt/seaweedfs/path/to/file + +# Remove an extended attribute +setfattr -x user.expire /mnt/seaweedfs/path/to/file + +# On macOS, use xattr instead +xattr -w user.expire "2025-12-01" /mnt/seaweedfs/path/to/file +xattr -p user.expire /mnt/seaweedfs/path/to/file +xattr -l /mnt/seaweedfs/path/to/file +xattr -d user.expire /mnt/seaweedfs/path/to/file +``` + +#### Limits + +| Limit | Value | +| ----- | ----- | +| Max attribute name size | 255 bytes | +| Max attribute value size | 64 KB (65536 bytes) | + +#### Disabling xattr + +If you don't need extended attributes, you can disable them with the `-disableXAttr` flag: + +```bash +weed mount -filer=localhost:8888 -dir=/mnt/seaweedfs -disableXAttr +``` ### Mount as FUSE diff --git a/Filer-Server-API.md b/Filer-Server-API.md index 7e63d34..97b33f8 100644 --- a/Filer-Server-API.md +++ b/Filer-Server-API.md @@ -203,29 +203,64 @@ Examples: | response-content-disposition | used as response content-disposition | empty | -### PUT/DELETE file tagging -``` -# put 2 pairs of meta data -curl -X PUT -H "Seaweed-Name1: value1" -H "Seaweed-some: some string value" http://localhost:8888/path/to/a/file?tagging -# read the meta data from HEAD request -curl -I "http://localhost:8888/path/to/a/file" -... -Seaweed-Name1: value1 -Seaweed-Some: some string value -... -# delete all "Seaweed-" prefixed meta data -curl -X DELETE http://localhost:8888/path/to/a/file?tagging -# delete specific "Seaweed-" prefixed meta data -curl -X DELETE http://localhost:8888/path/to/a/file?tagging=Name1,Some +### PUT/DELETE file tagging (Custom Attributes) +The Filer provides a tagging API to attach custom metadata to files and directories using `Seaweed-` prefixed headers. This is useful for storing application-specific attributes like expiration dates, ownership, or classification tags. + +#### Setting Custom Attributes + +```bash +# Add custom attributes to an existing file +curl -X PUT \ + -H "Seaweed-Expire: 2025-12-01" \ + -H "Seaweed-Author: john" \ + -H "Seaweed-Project: demo" \ + "http://localhost:8888/path/to/file?tagging" + +# You can also set attributes during file upload +curl -F "file=@myfile.txt" \ + -H "Seaweed-Expire: 2025-12-01" \ + "http://localhost:8888/path/to/myfile.txt" ``` + +#### Reading Custom Attributes + +Custom attributes are returned as response headers when you HEAD or GET a file: + +```bash +curl -I "http://localhost:8888/path/to/file" +# Response includes: +# Seaweed-Expire: 2025-12-01 +# Seaweed-Author: john +# Seaweed-Project: demo +``` + +#### Deleting Custom Attributes + +```bash +# Delete all Seaweed-prefixed attributes +curl -X DELETE "http://localhost:8888/path/to/file?tagging" + +# Delete specific attributes +curl -X DELETE "http://localhost:8888/path/to/file?tagging=Expire,Author" +``` + +#### Summary + | Method | Request | Header | Operation | | ---- | ---- | -- | -- | -| PUT | ?tagging | Prefixed with "Seaweed-" | set the meta data | -| DELETE | ?tagging | | remove all the "Seaweed-" prefixed header | -| DELETE | ?tagging=Some,Name | | remove the headers "Seaweed-Some", "Seaweed-Name" | +| PUT | `?tagging` | Prefixed with "Seaweed-" | Set custom attributes | +| DELETE | `?tagging` | | Remove all "Seaweed-" prefixed attributes | +| DELETE | `?tagging=Key1,Key2` | | Remove specific attributes | -Notice that the tag names follow http header key convention, with the first character capitalized. +Notice that the attribute names follow HTTP header key convention, with the first character capitalized (e.g., `Seaweed-my-key` becomes `Seaweed-My-Key`). + +#### Performance + +Custom attributes are stored in the filer's metadata database alongside other file metadata. They are: +- Efficiently stored with no separate lookups required +- Replicated with other metadata for consistency +- Not indexed by default (queries by attribute require custom implementation) ### Move files and directories ```bash