* tests: pin weed mini admin ports and disable unused aux services
Every weed mini that does not configure them still binds the fixed
default ports for the admin server (12535 + derived 22535 gRPC), its
legacy compat listener (23646 + 33646), WebDAV (7333), Lance (9101) and
the Iceberg REST catalog (8181). Sequential or concurrent minis on one
host then collide on them: the port check warns and shifts, and the
admin's late bind can land on a stolen ephemeral port, adding noise and
flakiness to test logs (e.g. run 40017181958, job 120659267479).
Add testutil.MiniAdminArgs to pin the admin server to an allocated
mini-style port (HTTP + port+10000 gRPC, both below the Linux ephemeral
port floor since admin binds late) and disable the legacy compat
listener. Every test that spawns weed mini now also disables the aux
services it does not use: WebDAV, Lance and the Iceberg REST endpoint
get port=0 unless the test actually exercises them, in which case their
ports stay pinned as before.
* tests: take mini admin ports from the below-ephemeral range
* tests: reuse MiniAdminArgs for the remote-s3 fetch mini
* tests: fully disable webdav and co-allocate mini admin ports
- -webdav.port=0 only makes the service bind a random port; add
-webdav=false so minis that do not test WebDAV skip it entirely.
- Spark, RisingWave and s3 Spark launchers allocated the admin port in
a second MustFreeMiniPort call, which could pick a port already
handed out by the first batch. Fold 'Admin' into the same
MustFreeMiniPorts call.
- test/fuse_integration is a separate Go module and cannot import
testutil; allocate the admin ports with its local freePort helper
instead.
* test/fuse: derive the mini admin gRPC port from its HTTP port
Two independent freePort calls could return the same port, making admin.port equal admin.port.grpc (or collide with the filer port). Allocate once and derive the gRPC port with the +10000 convention.
* test/fuse: pin the mini admin port in the fuse_test framework copy
The fuse_test package framework spawns weed mini too; without a pinned admin port it still binds the fixed defaults 12535/23646 and collides with other minis.
* tests: drop the redundant -webdav.port=0 flag
-webdav=false alone disables the service; the port flag is dead config.
OLake (github.com/datazip-inc/olake) is a CDC and ingestion engine that writes
Apache Iceberg. It covers two paths none of the existing catalog suites reach.
It is a strict Java Iceberg client. OLake does not write Iceberg from Go — its
Go process spawns a Java sidecar over gRPC and writes through the official
Apache Iceberg library, because the Go library has no equality deletes and CDC
needs them. That makes it the client class weed/s3api/iceberg's metadata
compliance backfill exists to serve: the one that fails with "Cannot parse
missing long current-snapshot-id" when spec-required keys are omitted.
And it produces equality deletes. Its upsert path commits operation=overwrite
with an equality-delete file and a delete manifest. ClickHouse, Doris, Trino,
Spark and DuckDB all only append, so nothing else in this directory exercises
a delete manifest at all.
Six subtests: the destination check reaches SUCCEEDED and actually loads the
REST catalog; discover enumerates the source; a full sync commits a snapshot;
PyIceberg reads back what the Java writer committed; an update plus a re-sync
records an overwrite carrying equality deletes with a delete manifest in the
current snapshot; and the catalog does not rewrite manifests the official Java
writer produced, which gives the manifest-repair path a negative test to go
with the ClickHouse positive one.
What it deliberately does not assert is a delete-applied read. PyIceberg
refuses to scan a table carrying equality deletes (apache/iceberg#6568) while
reading its metadata fine, and an engine that can apply them costs a
multi-gigabyte image. Recording the commit correctly is the catalog's
contract; applying deletes on read is the engine's. The README says so, says
the read half was verified by hand once with StarRocks, and warns against
later "upgrading" this to a PyIceberg rows read — which would either fail or,
if PyIceberg ever starts skipping deletes instead of raising, pass by not
looking.
The workflow job asserts the suite actually ran rather than trusting a green
exit, for the same reason: at least one top-level PASS and zero SKIP.
No product change — the destination config is the generic catalog_type=rest
with the standard OAuth2 client-credentials flow, and s3_path_style is not even
set, since OLake turns it on itself whenever s3_endpoint is non-empty.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chris Lu