3 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandDevin	d15396fc4d	s3tables: scope the anonymous identity's grant on the table catalogs (#12056) ... * s3tables: scope the resolved anonymous identity instead of defaulting to allow Catalog authorization fell back to the open default whenever the request resolved to the configured anonymous identity, ignoring its grant. The fallback now applies only when no principal was resolved (zero-config), so anonymous requests are checked against the identity's configured actions like any other principal. The S3 Tables API and the Iceberg and Lance catalogs share this handler, so all three are covered. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: authenticate the catalog clients in the s3tables suites Anonymous calls into the Lance and Iceberg catalogs are now scoped by the anonymous identity's grant, so the suites that used to rely on the anonymous pass authenticate like their real deployments do: x-api-key for the Lance probes and REST drivers, client-credentials OAuth for pyiceberg. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * worker(lance): authenticate namespace calls with the storage credential Anonymous catalog calls are now scoped by the anonymous grant, so a worker that holds an S3 credential sends it as x-api-key on the namespace requests instead of relying on the anonymous pass. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: authenticate the pushdown catalog connection in catalog_lancedb Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: follow catalog redirects only within the same host reqwest forwards custom headers on a follow, so a redirect to another host would carry the x-api-key credential to a server it was never meant for. Keep same-host hops for proxies that normalize the path or upgrade the scheme. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: warn when the catalog api key travels in cleartext The x-api-key pair is a real storage credential: over plain http to a non-loopback host the network can read it. Warn at client build so the operator sees it; https and loopback stay quiet, and the key still sends since an http catalog is a valid deployment. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: require the same origin on catalog redirects A same-host hop can still downgrade https to http and put the api key on the wire; matching scheme, host, and port keeps the credential on the listener it was issued for. * lance: bound catalog redirect hops so loops fail fast The custom redirect policy drops reqwest's default 10-hop limit, so a catalog that redirects back to itself would hold each call open until the request timeout. Keep the bound alongside the origin check. * lance: gofmt the redirect helpers --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-25 06:13:22 -07:00
Chris Lu	aeb5db1b13	tests: pin weed mini admin ports and disable unused aux services (#11997) ... * tests: pin weed mini admin ports and disable unused aux services Every weed mini that does not configure them still binds the fixed default ports for the admin server (12535 + derived 22535 gRPC), its legacy compat listener (23646 + 33646), WebDAV (7333), Lance (9101) and the Iceberg REST catalog (8181). Sequential or concurrent minis on one host then collide on them: the port check warns and shifts, and the admin's late bind can land on a stolen ephemeral port, adding noise and flakiness to test logs (e.g. run 40017181958, job 120659267479). Add testutil.MiniAdminArgs to pin the admin server to an allocated mini-style port (HTTP + port+10000 gRPC, both below the Linux ephemeral port floor since admin binds late) and disable the legacy compat listener. Every test that spawns weed mini now also disables the aux services it does not use: WebDAV, Lance and the Iceberg REST endpoint get port=0 unless the test actually exercises them, in which case their ports stay pinned as before. * tests: take mini admin ports from the below-ephemeral range * tests: reuse MiniAdminArgs for the remote-s3 fetch mini * tests: fully disable webdav and co-allocate mini admin ports - -webdav.port=0 only makes the service bind a random port; add -webdav=false so minis that do not test WebDAV skip it entirely. - Spark, RisingWave and s3 Spark launchers allocated the admin port in a second MustFreeMiniPort call, which could pick a port already handed out by the first batch. Fold 'Admin' into the same MustFreeMiniPorts call. - test/fuse_integration is a separate Go module and cannot import testutil; allocate the admin ports with its local freePort helper instead. * test/fuse: derive the mini admin gRPC port from its HTTP port Two independent freePort calls could return the same port, making admin.port equal admin.port.grpc (or collide with the filer port). Allocate once and derive the gRPC port with the +10000 convention. * test/fuse: pin the mini admin port in the fuse_test framework copy The fuse_test package framework spawns weed mini too; without a pinned admin port it still binds the fixed defaults 12535/23646 and collides with other minis. * tests: drop the redundant -webdav.port=0 flag -webdav=false alone disables the service; the port flag is dead config.	2026-10-23 15:38:09 -07:00
Chris Lu	5e7ab43ddd	test: read Lance tables from DuckDB (#10866) ... * test: read Lance tables from DuckDB The LanceDB and Spark suites go through the catalog. DuckDB does not: its lance extension reaches the data over S3 with no namespace involved, which exercises the other half of the design - a table bucket's layout is a valid Lance dataset directory, so a table stays readable when the catalog is not in the path. scan_rows=128 scan_columns=id,title,vector filtered_rows=5 nearest=1,0,2 It also pins the one place the layout costs us. DuckDB's replacement scan recognises a dataset by a .lance path suffix, and tables created through this catalog deliberately have none: the catalog entry is the dataset directory, a table name may not contain a dot, and a suffix would leak into ARNs and policies. So __lance_scan is the way in, and the bare SELECT ... FROM 's3://...' form does not see these tables. The test asserts both halves - a suffixed path is read, a suffix-less one is not - so if the extension ever recognises a bare directory, it fails and says to update the documentation rather than leaving it wrong. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: require the catalog error from the suffix-less read Any failure satisfied the old check - a missing extension, bad credentials, an unreachable endpoint - so the assertion could pass without the replacement scan ever classifying the path. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: verify the Lance table bucket was actually created weed shell prints a command's own failure and still exits 0, so the harness would go on to blame DuckDB for a bucket that was never made. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: bound the Docker probe An unhealthy daemon makes docker version hang, and the probe runs before the test has a timeout of its own. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: order the aggregates the assertions read string_agg over an unordered relation may return the names, and the vector search's ids, in any order, so the expectations could fail on a run where nothing changed. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: do not persist credentials in the DuckDB Lance checkout The job only uploads a log on failure; nothing in it pushes. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm	2026-08-21 15:22:38 -07:00