Files
seaweedfs/weed
Chris LuandClaude Opus 5.5 fdbaba52f9 s3 sink: forward user metadata to the destination object (#12286)
* s3 sink: forward user metadata to the destination object

filer.backup and filer.sync to an S3 sink copied the object body and tags
but dropped the X-Amz-Meta- attributes the S3 gateway stores with each
object, so the destination object came back without its user metadata.
Send them as the upload's metadata, keyed by the lowercased suffix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: send mtime as metadata without writing it into the entry

The sink stored X-Amz-Meta-Mtime into the replicated entry's Extended map,
modifying the caller's entry, and since tags were limited to X-Amz-Tagging-
keys it was never sent anywhere. Add it to the upload metadata instead,
keeping an mtime the source object already carries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: RFC 2047 encode metadata values that are not printable ASCII

The S3 gateway keeps user metadata values as raw bytes. Forwarded as is, a
UTF-8 value fails SigV4 on AWS and a control character makes the HTTP client
reject the request, and filer.backup retries such an object forever. Encode
those values as RFC 2047 encoded words, which S3 decodes on write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: upload without user metadata past the S3 2 KB limit

The S3 gateway does not cap user metadata, but S3 rejects more than 2 KB of
it, and filer.backup would retry that object forever. Upload such an object
without its metadata and log a warning, as the sink did before metadata was
forwarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: skip metadata keys that are not valid header names

Filer gRPC clients can store any Extended key. An X-Amz-Meta- key with a
space or control character makes the HTTP client reject the whole upload,
which filer.backup retries forever. Leave such keys out with a warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: pick the same value for metadata keys that differ only in case

An entry holding both X-Amz-Meta-Project and the legacy x-amz-meta-project
maps both to project, and map order decided which value was uploaded. Keep
the lowest key, so the canonical form wins over the legacy lowercase one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 sink: use a test value codespell accepts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 14:15:13 -07:00
..
2026-04-10 17:31:14 -07:00
2026-04-14 20:48:24 -07:00
2026-04-23 10:05:51 -07:00