test: drive the Lance namespace with LanceDB

The Iceberg catalog is checked against Spark, Trino, ClickHouse, Doris,
Dremio and RisingWave. The Lance one had only its own reference client,
which is the same thing as checking it against ourselves.

LanceDB connects with connect_namespace("rest", ...), which speaks the
routes this catalog implements, so the suite exercises the protocol rather
than our idea of it: list the catalog, open a table through it, read the
schema, run a vector search and a filtered scan, create a table, and read
the same dataset straight off its URI with no catalog at all.

    table_names -> ['lancedb-p0guidmm$ml$embeddings']
    open_table  -> 64 rows
    search      -> [1, 0, 2]
    create_table -> 4 rows, listed by the catalog
    direct read without the catalog -> 64 rows

Seeding is pylance, because the namespace records where a table lives and
does not carry its data. That split is the design rather than a limit of
the test.

One interop note the test encodes: a gateway without STS vends
storage_options carrying an endpoint and a region but no credentials, and
LanceDB uses what the namespace vends on some paths. The container gets
credentials in its environment as well, which is what a deployment without
STS would do.

Claude-Session: https://claude.ai/code/session_01Rkp1Mw5E89Jp6dzJFYiMrm
This commit is contained in:
Chris Lu
2026-08-20 12:38:06 -07:00
parent f56a7a1557
commit 2481641abf
5 changed files with 639 additions and 0 deletions
+70
View File
@@ -874,6 +874,76 @@ jobs:
path: test/s3tables/unity_catalog/test-output.log
retention-days: 3
lancedb-namespace-tests:
name: LanceDB Namespace Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
id: go
- name: Configure Docker Hub mirror
run: |
echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
- name: Pre-pull images
run: |
pull() { for i in 1 2 3; do docker pull "$1" && return 0; sleep 15; done; return 1; }
pull python:3.11-slim
- name: Run go mod tidy
run: go mod tidy
- name: Build SeaweedFS
run: |
cd weed && go build -buildvcs=false .
- name: Run LanceDB Namespace Integration Tests
timeout-minutes: 25
working-directory: test/s3tables/catalog_lancedb
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
docker info
echo "=== Starting LanceDB Namespace Tests ==="
go test -v -timeout 20m . 2>&1 | tee test-output.log || {
echo "LanceDB namespace integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog_lancedb
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker)" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v7
with:
name: lancedb-namespace-test-logs
path: test/s3tables/catalog_lancedb/test-output.log
retention-days: 3
s3-tables-build-verification:
name: S3 Tables Build Verification
runs-on: ubuntu-22.04