Chris Lu
470339aa21
Tests share one weed mini server. Two pollution modes were producing order-dependent failures: - A later test's shard sweep would still load the prior test's lifecycle config (the worker reads every bucket's XML from filer state, and DeleteBucket alone doesn't drop lifecycle config cleanly on this codebase). - Versioned-bucket tests left versions + delete markers behind that ListObjectsV2 can't see, so the existing best-effort empty-then- delete didn't actually empty those buckets. - The AbortMPU test intentionally leaves an in-flight upload; without an explicit AbortMultipartUpload the bucket DELETE hits NotEmpty. Cleanup now runs DeleteBucketLifecycle, ListObjectVersions → DeleteObject(versionId), ListObjectsV2 → DeleteObject (catches what ListObjectVersions missed), ListMultipartUploads → AbortMultipartUpload, then DeleteBucket. Best-effort throughout so a half-torn-down bucket doesn't fail the cleanup chain.
S3 Lifecycle Integration Tests
End-to-end test of the event-driven S3 lifecycle worker, exercised through
the s3.lifecycle.run-shard shell command.
Why backdate mtimes?
The S3 API rejects Expiration.Days < 1, so a literal "wait one day"
integration test isn't workable. Each test sets up a 1-day expiration rule,
puts the target object, then rewrites its filer entry's Mtime to ~30 days
ago via filer_pb.UpdateEntry. From the engine's perspective the object is
past its expiration window the moment the shell command starts.
Running
# build the binary, start a local mini cluster, run tests, stop it
make test-with-server
# or, if a cluster is already running on the default ports
make test
The test runs the shell command once with -shards 0-15 (one filer
subscription covering all 16 shards) rather than computing the target
object's shard up front. This keeps the test independent of the
ShardID(bucket, key) hash function — only that some shard reaches the
deletion within the polling window.
Environment
| variable | default | description |
|---|---|---|
WEED_BINARY |
required | path to weed_binary |
S3_ENDPOINT |
http://localhost:8333 |
S3 API URL |
S3_GRPC_ENDPOINT |
localhost:18333 |
S3 gRPC for lifecycle dispatch |
MASTER_ENDPOINT |
http://localhost:9333 |
master HTTP |
FILER_GRPC_ADDRESS |
localhost:18888 |
filer gRPC for UpdateEntry |