s3api: listing argument validation and encoding-type=url for versions, prefix and delimiter (#12753)

* s3api: reject an empty ListObjectsV2 continuation-token

AWS answers continuation-token= with 400 InvalidArgument ("The continuation
token provided is incorrect"); SeaweedFS listed from the start of the bucket.
No token this server issues is empty, and raw-key and base64: tokens keep
resuming as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* s3api: reject an encoding-type other than url on listings

AWS answers any encoding-type other than url with 400 InvalidArgument
("Invalid Encoding Method specified in Request") on ListObjects,
ListObjectsV2, ListObjectVersions, ListMultipartUploads and ListParts;
SeaweedFS ignored the value and listed unencoded keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* s3api: apply encoding-type=url to ListObjectVersions

The handler accepted encoding-type=url but returned raw keys. It now
URL-encodes Key in Version and DeleteMarker entries, CommonPrefixes, Prefix,
Delimiter, KeyMarker and NextKeyMarker with the urlPathEscape form ListObjects
uses, and echoes EncodingType.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* s3api: URL-encode Prefix and Delimiter in ListObjects responses

Under encoding-type=url, ListObjects and ListObjectsV2 encoded keys,
CommonPrefixes, markers and StartAfter but echoed Prefix and Delimiter raw,
so a client decoding them once corrupted a prefix holding '%' or '+'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chris LuandClaude Opus 5.5 authored and GitHub committed 2026-11-18 15:55:41 -08:00
1 parent 646549eee1
commit 1fd5008d63
5 files changed
+181 -2

No files matched your search

+29
View File
@@ -18,6 +18,7 @@ import (
"sync"
"time"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/seaweedfs/seaweedfs/weed/filer"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
@@ -494,6 +495,26 @@ func (s3a *S3ApiServer) truncateAndSetMarkers(combinedList []versionListItem, ma
return combinedList, nextKeyMarker, nextVersionIdMarker, isTruncated
}
// encodeURL applies encoding-type=url to every key-bearing field, the form
// ListObjects uses for its keys; clients decode them once.
func (result *S3ListObjectVersionsResult) encodeURL() {
result.EncodingType = s3.EncodingTypeUrl
result.Prefix = urlPathEscape(result.Prefix)
result.Delimiter = urlPathEscape(result.Delimiter)
result.KeyMarker = urlPathEscape(result.KeyMarker)
result.NextKeyMarker = urlPathEscape(result.NextKeyMarker)
for _, e := range result.Entries {
switch {
case e.Version != nil:
e.Version.Key = urlPathEscape(e.Version.Key)
case e.DeleteMarker != nil:
e.DeleteMarker.Key = urlPathEscape(e.DeleteMarker.Key)
case e.Prefix != nil:
e.Prefix.Prefix = urlPathEscape(e.Prefix.Prefix)
}
}
}
// splitIntoResult builds the final S3ListObjectVersionsResult from the combined list.
// It populates a single Entries slice that preserves the interleaved sort order.
func (s3a *S3ApiServer) splitIntoResult(combinedList []versionListItem, bucket, prefix, keyMarker, versionIdMarker, delimiter string, maxKeys int, isTruncated bool, nextKeyMarker, nextVersionIdMarker string) *S3ListObjectVersionsResult {
@@ -1991,6 +2012,11 @@ func (s3a *S3ApiServer) ListObjectVersionsHandler(w http.ResponseWriter, r *http
versionIdMarker := query.Get("version-id-marker")
delimiter := query.Get("delimiter")
encodingTypeUrl, errCode := parseEncodingType(query)
if errCode != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, errCode)
return
}
maxKeys, errCode := parseMaxKeys(query.Get("max-keys"))
if errCode != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, errCode)
@@ -2007,6 +2033,9 @@ func (s3a *S3ApiServer) ListObjectVersionsHandler(w http.ResponseWriter, r *http
// Set the original prefix in the response (not the normalized internal prefix)
result.Prefix = originalPrefix
if encodingTypeUrl {
result.encodeURL()
}
glog.V(3).Infof("ListObjectVersionsHandler response: %+v", result)
writeSuccessResponseXML(w, r, result)