2 Commits

Include renames

Author	SHA1	Message	Date
Chris Lu	aeb5db1b13	tests: pin weed mini admin ports and disable unused aux services (#11997) ... * tests: pin weed mini admin ports and disable unused aux services Every weed mini that does not configure them still binds the fixed default ports for the admin server (12535 + derived 22535 gRPC), its legacy compat listener (23646 + 33646), WebDAV (7333), Lance (9101) and the Iceberg REST catalog (8181). Sequential or concurrent minis on one host then collide on them: the port check warns and shifts, and the admin's late bind can land on a stolen ephemeral port, adding noise and flakiness to test logs (e.g. run 40017181958, job 120659267479). Add testutil.MiniAdminArgs to pin the admin server to an allocated mini-style port (HTTP + port+10000 gRPC, both below the Linux ephemeral port floor since admin binds late) and disable the legacy compat listener. Every test that spawns weed mini now also disables the aux services it does not use: WebDAV, Lance and the Iceberg REST endpoint get port=0 unless the test actually exercises them, in which case their ports stay pinned as before. * tests: take mini admin ports from the below-ephemeral range * tests: reuse MiniAdminArgs for the remote-s3 fetch mini * tests: fully disable webdav and co-allocate mini admin ports - -webdav.port=0 only makes the service bind a random port; add -webdav=false so minis that do not test WebDAV skip it entirely. - Spark, RisingWave and s3 Spark launchers allocated the admin port in a second MustFreeMiniPort call, which could pick a port already handed out by the first batch. Fold 'Admin' into the same MustFreeMiniPorts call. - test/fuse_integration is a separate Go module and cannot import testutil; allocate the admin ports with its local freePort helper instead. * test/fuse: derive the mini admin gRPC port from its HTTP port Two independent freePort calls could return the same port, making admin.port equal admin.port.grpc (or collide with the filer port). Allocate once and derive the gRPC port with the +10000 convention. * test/fuse: pin the mini admin port in the fuse_test framework copy The fuse_test package framework spawns weed mini too; without a pinned admin port it still binds the fixed defaults 12535/23646 and collides with other minis. * tests: drop the redundant -webdav.port=0 flag -webdav=false alone disables the service; the port flag is dead config.	2026-10-23 15:38:09 -07:00
Chris Lu	f72983c1fd	fix(s3): stop S3 Tables routes from swallowing buckets named "buckets" or "get-table" (#9566) ... * fix(s3): stop S3 Tables routes from swallowing buckets named "buckets" or "get-table" The S3 Tables REST endpoints share top-level paths with the regular S3 API (/buckets for ListTableBuckets/CreateTableBucket, /get-table for GetTable). They are registered first on the same router as the bucket subrouter, so a path-style request such as GET /buckets?list-type=2 on a bucket actually named "buckets" matched ListTableBuckets and returned JSON. AWS SDK V2 (and Hadoop s3a / Spark) then failed XML parsing with "Unexpected character '{' (code 123) in prolog". Disambiguate by requiring the AWS V4 credential scope to name the s3tables service on the colliding routes. Regular S3 SDKs sign with service=s3, S3 Tables SDKs sign with service=s3tables, and the scope is present in both the Authorization header and the X-Amz-Credential query parameter for presigned URLs, so the matcher works for both flavors. ARN-bearing S3 Tables routes (/buckets/<arn>, /namespaces/<arn>, etc.) already cannot collide because colons are not valid in bucket names, so they are left untouched. * fix(s3): accept AWS JSON RPC content type as S3 Tables intent signal The Iceberg catalog integration tests send unsigned PUT /buckets with Content-Type: application/x-amz-json-1.1 to create table buckets. With only the credential-scope check, those requests fell through to the regular S3 CreateBucket handler and the suite went red on this branch. Extend the matcher so a request is recognized as S3 Tables when either: - its AWS V4 credential scope names SERVICE=s3tables; or - it carries the canonical AWS JSON RPC 1.1 content type and is unsigned (a request explicitly signed for SERVICE=s3 still wins). The regular S3 SDKs do not send application/x-amz-json-1.1, so the signal is safe for the colliding paths (/buckets, /get-table). Also add an AWS SDK V2 for Go integration test under test/s3/sdk_v2_routing/ that drives the SDK's own XML deserializer against a bucket literally named "buckets" and "get-table" — the SDK errors before the test asserts if the server returns the wrong body shape. Wired up via .github/workflows/s3-sdk-v2-routing-tests.yml, mirroring the etag/acl workflow. * s3api: extend service matcher to all S3 Tables routes; simplify scope check - Apply serviceMatcher to every S3 Tables route, not just the bare-path ones. ARN-bearing paths could otherwise be hit by an S3 object key that starts with arn:aws:s3tables:..., inside a bucket named "buckets", "namespaces", "tables", or "tag". One matcher everywhere closes both collision classes. - Replace strings.Split + index lookup with strings.Contains for the credential-scope check. The scope shape is fixed at AK/DATE/REGION/SERVICE/aws4_request, slashes only delimit components, and access keys are alphanumeric — so /s3tables/ matches iff SERVICE is exactly s3tables. Existing unit cases (including the access-key-substring case) still pass. - Read the GetObject body in the SDK v2 routing test with io.ReadAll; the single Read could return short and make the equality check flaky. * s3api: drop content-type fallback; sign s3 tables harness traffic instead The content-type fallback in isS3TablesSignedRequest let an anonymous regular-S3 request whose body type is application/x-amz-json-1.1 hit an S3 Tables route when the path-style object key happened to be shaped like an S3 Tables ARN (e.g. PutObject on bucket "buckets" with key arn:aws:s3tables:.../bucket/foo/policy). Narrow the matcher back to the AWS V4 credential scope so only requests signed for SERVICE=s3tables match the S3 Tables routes. Update the Iceberg catalog test harness — the only caller still sending unsigned PUT /buckets — to sign with SERVICE=s3tables. The mini instance runs in default-allow mode, so the signature itself is not verified; only the credential scope matters for the route match. Drop the stale unit cases for the JSON-RPC content-type signal and the routing test that exercised unsigned harness traffic.	2026-05-19 14:24:25 -07:00