Commit Graph
5 Commits
Author SHA1 Message Date
Chris Lu 462e4445e2 test: skip catalog client images that cannot execute on the host CPU (#12014)
New upstream releases build for x86-64-v3 (AVX2) - ClickHouse since
26.5, plus the DuckDB iceberg extension and pylance wheels - so on a
runner whose CPU lacks AVX2 the container dies on SIGILL before the
test reaches any SeaweedFS code. Probe each third-party image first
and skip on the illegal-instruction exit instead of failing; other
errors still surface normally.
2026-10-23 18:46:26 -07:00
Chris Lu aeb5db1b13 tests: pin weed mini admin ports and disable unused aux services (#11997)
* tests: pin weed mini admin ports and disable unused aux services

Every weed mini that does not configure them still binds the fixed
default ports for the admin server (12535 + derived 22535 gRPC), its
legacy compat listener (23646 + 33646), WebDAV (7333), Lance (9101) and
the Iceberg REST catalog (8181). Sequential or concurrent minis on one
host then collide on them: the port check warns and shifts, and the
admin's late bind can land on a stolen ephemeral port, adding noise and
flakiness to test logs (e.g. run 40017181958, job 120659267479).

Add testutil.MiniAdminArgs to pin the admin server to an allocated
mini-style port (HTTP + port+10000 gRPC, both below the Linux ephemeral
port floor since admin binds late) and disable the legacy compat
listener. Every test that spawns weed mini now also disables the aux
services it does not use: WebDAV, Lance and the Iceberg REST endpoint
get port=0 unless the test actually exercises them, in which case their
ports stay pinned as before.

* tests: take mini admin ports from the below-ephemeral range

* tests: reuse MiniAdminArgs for the remote-s3 fetch mini

* tests: fully disable webdav and co-allocate mini admin ports

- -webdav.port=0 only makes the service bind a random port; add
  -webdav=false so minis that do not test WebDAV skip it entirely.
- Spark, RisingWave and s3 Spark launchers allocated the admin port in
  a second MustFreeMiniPort call, which could pick a port already
  handed out by the first batch. Fold 'Admin' into the same
  MustFreeMiniPorts call.
- test/fuse_integration is a separate Go module and cannot import
  testutil; allocate the admin ports with its local freePort helper
  instead.

* test/fuse: derive the mini admin gRPC port from its HTTP port

Two independent freePort calls could return the same port, making admin.port equal admin.port.grpc (or collide with the filer port). Allocate once and derive the gRPC port with the +10000 convention.

* test/fuse: pin the mini admin port in the fuse_test framework copy

The fuse_test package framework spawns weed mini too; without a pinned admin port it still binds the fixed defaults 12535/23646 and collides with other minis.

* tests: drop the redundant -webdav.port=0 flag

-webdav=false alone disables the service; the port flag is dead config.
2026-10-23 15:38:09 -07:00
Chris Lu 790e8d3fd6 clickhouse catalog test: cover latest ClickHouse and catalog-side CREATE TABLE (#10707)
* clickhouse catalog test: cover latest ClickHouse and catalog-side CREATE TABLE

* verify catalog registration structurally and fix README image wording
2026-08-10 20:04:33 -07:00
Chris Lu 923d0bd20c iceberg: repair non-compliant manifests at commit (#10641)
* iceberg: stamp a default name mapping on new tables

* iceberg: repair non-compliant manifests at commit

* s3tables: verify ClickHouse writes read back through PyIceberg

* iceberg: carry the manifest-list content into repaired manifests

* iceberg: refresh the default name mapping on schema evolution

* iceberg: merge historical names into the refreshed name mapping

* iceberg: never fail a commit on repair fallout

* iceberg: harden manifest repair against writer dialects

* s3tables: keep PyIceberg reader stderr out of row data

* iceberg: keep name mappings unambiguous across field id reassignment

* iceberg: align existing manifest content metadata with the list entry
2026-08-08 21:24:37 -07:00
Chris Lu 213eb4c23a s3tables: add ClickHouse iceberg catalog integration test (#10637)
* s3tables: add ClickHouse iceberg catalog integration test

* ci: run the ClickHouse iceberg catalog test

* s3tables: bound setup HTTP calls in the ClickHouse test

* s3tables: pin the ClickHouse writer image dependencies
2026-08-07 22:39:50 -07:00