Fixes#8712: S3 API directory markers created via PutObject with an
explicit Content-Type header (e.g. application/octet-stream) were not
appearing in ListObjects results.
Root cause: The filer strips "application/octet-stream" from the Mime
field during InsertEntry/UpdateEntry (filerstore_wrapper.go). This
causes IsDirectoryKeyObject() to return false, so the directory markers
are excluded from listings.
Additionally, when AWS CLI includes a Content-Type header, it may use
AWS chunked encoding. r.ContentLength then includes chunked overhead,
which can exceed 1024 bytes even for zero-byte payloads, preventing the
directory marker fast-path from triggering.
Changes:
- Use x-amz-decoded-content-length header for accurate payload size
- Require actualContentLength >= 0 to reject unknown/negative lengths
- Wrap body reader with io.LimitReader(1024+1) to prevent OOM
- Reject payloads > 1024 bytes with HTTP 413
- Store MIME in ExtMimeType extended attribute for zero-byte markers
(persists even when the filer strips the Mime field)
- Update IsDirectoryKeyObject() to check ExtMimeType as fallback
see https://blog.aqwari.net/xml-schema-go/
1. go get aqwari.net/xml/cmd/xsdgen
2. Add EncodingType element for ListBucketResult in AmazonS3.xsd
3. xsdgen -o s3api_xsd_generated.go -pkg s3api AmazonS3.xsd
4. Remove empty Grantee struct in s3api_xsd_generated.go
5. Remove xmlns: sed s'/http:\/\/s3.amazonaws.com\/doc\/2006-03-01\/\ //' s3api_xsd_generated.go
Chris Lu
7e1c2c812f