Chris Lu
01bb3b3053
* s3api: add Snowflake s3compat API integration tests Run the upstream snowflakedb/snowflake-s3compat-api-test-suite against a local SeaweedFS server in CI. test/s3/snowflake/run.sh starts weed server with S3 (-s3.autoCreateBucket=false so missing-bucket PUTs return NoSuchBucket), prepares the fixtures the suite needs (versioned bucket, deny-all-policy bucket, >1000-object prefix), clones the suite, patches it to path-style addressing, and runs mvn -Dtest=S3CompatApiTest. The suite also exposed that GetBucketLocation returned 404 NoSuchBucket for a malformed bucket name; validate the name first and return 400 InvalidBucketName like AWS. * test: harden snowflake s3compat runner per review - Pin the upstream suite to a tested commit (SUITE_REV) instead of the moving default branch - Bind the test server to loopback only - Require the AccessDenied error code when verifying the denied bucket - Fix README so go install runs in a subshell - checkout with persist-credentials: false - Make the concurrency group unique per PR, and widen path filters to the storage/operation/wdclient/cluster/pb packages the S3 stack uses * test: advertise loopback ip for snowflake test server -ip.bind 127.0.0.1 alone left the volume server advertising the host's primary address, so chunk uploads were refused. Also set -ip 127.0.0.1 and disable the Iceberg/Lance listeners so the harness is loopback-only and does not collide with other local services.
Snowflake S3Compat API test suite
Integration tests that run the upstream
Snowflake s3compat API test suite
against SeaweedFS. The suite covers getBucketLocation, getObject (including
range reads), getObjectMetadata, putObject (including a 5 GB upload),
listObjectsV2 (including paged listing of >1000 objects), deleteObject,
deleteObjects, copyObject, and generatePresignedUrl.
Running locally
Requires weed (or WEED_BIN), the aws CLI, mvn, and JDK 11+ on PATH.
(cd weed && go install -buildvcs=false) # build weed first
bash test/s3/snowflake/run.sh
run.sh starts a weed server with S3 enabled, calls prepare.sh to create
the fixtures, clones the suite into a scratch dir, and runs
mvn -Dtest=S3CompatApiTest. Set WORK_DIR to keep the server log and suite
clone around, SKIP_SERVER_START=1 with ENDPOINT_URL to run against an
already-running server, and see the top of run.sh for the other overrides.
Notes
- The server runs with
-s3.autoCreateBucket=falseso PUTs to a missing bucket returnNoSuchBucketlike AWS; the suite asserts this. - The suite forces virtual-hosted-style bucket addressing, which requires
wildcard DNS that does not exist for a local endpoint.
run.shswitches it to path-style access with asedpatch. NOT_ACCESSIBLE_BUCKETis a real bucket carrying a deny-all bucket policy, which is how the suite'sAccessDeniednegative tests are satisfied.