Commit Graph
7 Commits
Author SHA1 Message Date
Chris Lu 358fd314ea test(s3/versioning): read the whole version body instead of one Read (#10815)
A single Read on the response body can return the last bytes together
with io.EOF, so asserting NoError on it fails even though the body is
complete. Use io.ReadAll, like every other test in this package.
2026-08-18 18:34:01 -07:00
Lars Lehtonen ed75a61fb0 fix(test/s3/versioning): dropped test error (#10813) 2026-08-18 17:26:51 -07:00
Chris Lu a4b896a224 fix(s3): skip directories before marker in ListObjectVersions pagination (#8890)
* fix(s3): skip directories before marker in ListObjectVersions pagination

ListObjectVersions was re-traversing the entire directory tree from the
beginning on every paginated request, only skipping entries at the leaf
level. For buckets with millions of objects in deep hierarchies, this
caused exponentially slower responses as pagination progressed.

Two optimizations:
1. Use keyMarker to compute a startFrom position at each directory level,
   skipping directly to the relevant entry instead of scanning from the
   beginning (mirroring how ListObjects uses marker descent).
2. Skip recursing into subdirectories whose keys are entirely before the
   keyMarker.

Changes per-page cost from O(entries_before_marker) to O(tree_depth).

* test(s3): add integration test for deep-hierarchy version listing pagination

Adds TestVersioningPaginationDeepDirectoryHierarchy which creates objects
across 20 subdirectories at depth 6 (mimicking Veeam 365 backup layout)
and paginates through them with small maxKeys. Verifies correctness
(no duplicates, sorted order, all objects found) and checks that later
pages don't take dramatically longer than earlier ones — the symptom
of the pre-fix re-traversal bug. Also tests delimiter+pagination
interaction across subdirectories.

* test(s3): strengthen deep-hierarchy pagination assertions

- Replace timing warning (t.Logf) with a failing assertion (t.Errorf)
  so pagination regressions actually fail the test.
- Replace generic count/uniqueness/sort checks on CommonPrefixes with
  exact equality against the expected prefix slice, catching wrong-but-
  sorted results.

* test(s3): use allKeys for exact assertion in deep-hierarchy pagination test

Wire the allKeys slice (previously unused dead code) into the version
listing assertion, replacing generic count/uniqueness/sort checks with
an exact equality comparison against the keys that were created.
2026-04-02 15:59:52 -07:00
promalertandChris Lu 9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
chrislu 26121c55c9 test: improve pagination stress test with QUICK_TEST option and better assertions 2025-12-15 23:23:50 -08:00
chrislu f517bc39fc test: fix nil pointer dereference and add debugging to pagination stress tests 2025-12-15 23:16:01 -08:00
chrislu 0972a0acf3 test: add pagination stress tests for S3 versioning with >1000 versions 2025-12-15 23:06:57 -08:00