mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-19 10:54:08 +01:00
test/s3: run the s3-listing-study listing tools and strategies in CI (#12750)
* test(s3): s3-listing-study integration suite (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): run the s3-listing-study tools against SeaweedFS (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): s3-listing-study deleted-prefix probe, fresh data dir per run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): listing study ports keep clear of every gRPC companion free_ports.py compared only base ports, so a base port could land on another server's base+10000 gRPC listener. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study ports stay below the ephemeral range Base ports of 20000-55000 put both them and their gRPC companions inside 32768-60999, so an outgoing connection made while weed server starts could take a port before the server that was given it binds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study checks the order inside each listing page parse_page sorted Contents and CommonPrefixes together before the probes compared them, so keys returned out of order within one page passed. Check each list's byte order first; the versions pager does the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study applies --only before running a probe The filter was applied to the probe results, so a run limited to one tool still sent every reference listing request and the deleted-prefix check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study requires CommonPrefixes from shallow modes The CommonPrefixes comparison ran only when a tool returned at least one prefix, so a delimiter listing of the canary root that came back empty passed. Compare them for every shallow mode except the two whose normalizer keeps keys only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
04c7b12238
commit
23eef59417
9 files changed
+1377
No files matched your search
@@ -0,0 +1,29 @@
|
||||
# Cases that fail for reasons outside SeaweedFS. A listed case that passes is
|
||||
# reported as XPASS and fails the run, so this list has to shrink with fixes.
|
||||
# Format: <case name> # reason
|
||||
|
||||
# aws-cli prints "None" for a --query over a page with no Contents (the canary
|
||||
# has no root-level keys); the capsule normalizer reads it as a malformed row.
|
||||
aws-cli:s3api-v2-remainder:canary:/ # capsule normalizer cannot parse aws-cli's "None"
|
||||
|
||||
# The edge bucket holds zero-byte directory markers (markers/dir/, markers/empty/).
|
||||
# These modes list them as directories, not objects: rclone's --files-only and
|
||||
# lsf --files-only drop them, s5cmd types any key ending in "/" as DIR and the
|
||||
# capsule normalizer drops DIR rows from its recursive modes. The reference
|
||||
# probes check that SeaweedFS returns both markers as Contents.
|
||||
rclone:recursive-fastlist:edge:/ # tool hides directory marker objects
|
||||
rclone:recursive-hierarchical:edge:/ # tool hides directory marker objects
|
||||
rclone:recursive-walk:edge:/ # tool hides directory marker objects
|
||||
rclone:listv1:edge:/ # tool hides directory marker objects
|
||||
rclone:lsf:edge:/ # tool hides directory marker objects
|
||||
s5cmd:recursive:edge:/ # tool hides directory marker objects
|
||||
s5cmd:listv1:edge:/ # tool hides directory marker objects
|
||||
s5cmd:allversions:edge:/ # tool hides directory marker objects
|
||||
|
||||
# Deleting the last key under a prefix leaves its filer directory behind, and an
|
||||
# empty directory is still listed as a CommonPrefix until the filer's empty-folder
|
||||
# cleanup removes it (2 minutes by default). This is deliberate: directories made
|
||||
# through mount or the filer stay visible to S3 clients (dirHoldsOnlyHiddenEntries
|
||||
# in weed/s3api/s3api_object_handlers_list.go). AWS derives CommonPrefixes from
|
||||
# keys only, so prefix-discovery tools briefly see the deleted prefix.
|
||||
reference:study-churn:deleted-prefixes # empty directories are listed until cleanup
|
||||
Reference in new issue
Block a user