2 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandDevin	d15396fc4d	s3tables: scope the anonymous identity's grant on the table catalogs (#12056) ... * s3tables: scope the resolved anonymous identity instead of defaulting to allow Catalog authorization fell back to the open default whenever the request resolved to the configured anonymous identity, ignoring its grant. The fallback now applies only when no principal was resolved (zero-config), so anonymous requests are checked against the identity's configured actions like any other principal. The S3 Tables API and the Iceberg and Lance catalogs share this handler, so all three are covered. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: authenticate the catalog clients in the s3tables suites Anonymous calls into the Lance and Iceberg catalogs are now scoped by the anonymous identity's grant, so the suites that used to rely on the anonymous pass authenticate like their real deployments do: x-api-key for the Lance probes and REST drivers, client-credentials OAuth for pyiceberg. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * worker(lance): authenticate namespace calls with the storage credential Anonymous catalog calls are now scoped by the anonymous grant, so a worker that holds an S3 credential sends it as x-api-key on the namespace requests instead of relying on the anonymous pass. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: authenticate the pushdown catalog connection in catalog_lancedb Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: follow catalog redirects only within the same host reqwest forwards custom headers on a follow, so a redirect to another host would carry the x-api-key credential to a server it was never meant for. Keep same-host hops for proxies that normalize the path or upgrade the scheme. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: warn when the catalog api key travels in cleartext The x-api-key pair is a real storage credential: over plain http to a non-loopback host the network can read it. Warn at client build so the operator sees it; https and loopback stay quiet, and the key still sends since an http catalog is a valid deployment. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * lance: require the same origin on catalog redirects A same-host hop can still downgrade https to http and put the api key on the wire; matching scheme, host, and port keeps the credential on the listener it was issued for. * lance: bound catalog redirect hops so loops fail fast The custom redirect policy drops reqwest's default 10-hop limit, so a catalog that redirects back to itself would hold each call open until the request timeout. Keep the bound alongside the origin check. * lance: gofmt the redirect helpers --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-25 06:13:22 -07:00
Chris Lu	34bb444f33	test: drive the Lance namespace with Spark (#10864) ... * test: drive the Lance namespace with Spark The counterpart of catalog_spark, which does this for the Iceberg REST catalog. Spark is the engine most likely to be pointed at a lakehouse, and it reaches the Lance catalog through the connector's DSV2 catalog - org.lance.spark.LanceNamespaceSparkCatalog with impl=rest - over the same routes every other client uses. SHOW NAMESPACES -> ['`sparklance-lcephd80`.ml'] SHOW TABLES -> ['sparklance-lcephd80$ml$embeddings'] count -> 3 filtered -> [(2, 'two'), (3, 'three')] count after a second commit -> 4 The second insert is there on purpose: a store that cannot order commits fails on the second one, not the first. Two things the run settled that were guesses beforehand. CREATE TABLE works, because the connector declares through the namespace and writes the data itself rather than pushing Arrow at the server. And SHOW TABLES returns the namespace's own identifiers - bucket, namespace and name joined by the delimiter - not bare Spark table names. Credentials go under the catalog's storage.* prefix, which is handed to lance as object_store options; a gateway without STS vends none, the same trap the LanceDB suite documents. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: verify the Lance table bucket was actually created weed shell prints a command's own failure and still exits 0, so the harness would go on to blame Spark for a bucket that was never made. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: bound the Docker probe An unhealthy daemon makes docker version hang, and the probe runs before the test has a timeout of its own. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: keep the ivy cache under the user's cache directory It is mounted into a container running as root, so a shared temp path lets another local user pre-create it and choose what Spark loads. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: assert the vector column's type, not only its name A column that came back as array<double> or array<string> would still be called vector and still pass. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: read the dataset off its location for real The catalog being optional is the property that lets duckdb and pandas read these tables; it was asserted in a comment and printed, never exercised. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: do not persist credentials in the Spark Lance checkout The job only uploads a log on failure; nothing in it pushes. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm * test: say the hosts in the README are placeholders The suite passes dynamically allocated host.docker.internal ports. Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm	2026-08-21 23:49:09 -07:00