Spark Lance Integration
Chris Lu edited this page 2026-08-21 13:17:24 -07:00
Clone

Wiki Page Revisions

2 Commits

Author SHA1 Message Date
Chris Lu b9475cdf7f Document both ways to create a Lance table bucket
The shell command against a running cluster, and weed mini -tableBucket at
startup, which takes name[:FORMAT] entries and also reads S3_TABLE_BUCKET.
Notes that mini skips a bucket in a format it does not serve rather than
creating one nothing can reach.
2026-08-21 13:17:24 -07:00
Chris Lu 489ebb7dd4 Add Spark Lance Integration
Spark reaches the Lance catalog through the connector's DSV2 catalog, the
same way it reaches the Iceberg one. Covers the storage.* prefix, which is
object_store's key names rather than Spark's s3a ones, and the two things
the integration suite settled: CREATE TABLE works because the connector
declares and writes rather than pushing Arrow at the server, and SHOW
TABLES returns namespace identifiers rather than bare table names.
2026-08-21 12:33:49 -07:00