Phase 5: federation

Implements PLAN.md Phase 5: the Server-Server API. Warp joins rooms on
other servers and lets their users join its rooms, invites across servers
with the invitee's server countersigning, rejects remote invites (locally
if the inviter is unreachable), and exchanges events, typing, receipts and
to-device messages with other homeservers — all behind the runtime switch,
which now starts and stops real outbound traffic.

The protocol lives in internal/federation with no knowledge of rooms:
X-Matrix signing (fuzzed parser), the spec's discovery algorithm keeping
the dial address and the TLS name apart, a signed client that refuses
every request while the switch is off or the policy denies the
destination, and a key ring that caches other servers' keys for at most a
week, with optional notaries.

The roomserver takes events from elsewhere through the spec's intake:
signatures and hashes (a hash mismatch keeps the redacted form), the rules
against the event's own auth events, against the state before it — with
/get_missing_events for small gaps and /state_ids for wide ones — and
against the current state, where a failure is a soft failure. Events held
without their state are outliers: a column on roomserver_events, because
the current state of a room joined over federation is almost all
outliers, and excluded from every timeline query. Forks are resolved at
the room's head. Membership across servers is make_join/send_join,
make_leave/send_leave and countersigned invites, in both directions;
m.room.server_acl is enforced on everything another server sends.
History before a remote join is backfilled when /messages pages past it,
stored at negative stream positions.

internal/federationapi is the running component: the inbound endpoints,
and an outbound queue that persists what it owes each destination, sends
it in transactions of 50 PDUs and 100 EDUs, backs off from ten seconds to
six hours, survives restarts, and drops events persisted while federation
is off rather than delivering a backlog in one burst. Its Module is what
the switch starts and stops.

Also: a TLS listener for port 8448, trusted key servers, key validity and
an outbound CA bundle in the config; per-destination health and backoff
reset in the admin API; remote profiles, aliases and directories in the
Client-Server API; a Complement federation profile. See
docs/federation.md, including its known gaps (no partial-state joins, no
key rotation).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

server/test/complement/Dockerfile

@@ -16,10 +16,10 @@
 #
 #   docker build -t complement-warp -f server/test/complement/Dockerfile .
 #
-# Federation (port 8448, served over TLS with a certificate signed by the CA
-# Complement mounts at /complement/ca) arrives in Phase 5. Until then this
-# image is for the federation=off profile: the C-S tests. The port is still
-# exposed, because Complement expects it to be.
+# Federation is served on 8448 over TLS, with a certificate the entrypoint
+# has the CA Complement mounts at /complement/ca sign. The federation
+# profile of run.sh switches it on; the default profile leaves it off and
+# runs the C-S tests only.
 
 FROM golang:1.26-alpine AS build
 
@@ -38,9 +38,9 @@ RUN go build -trimpath -o /out/warp ./cmd/warp
 
 FROM alpine:3.21
 
-# curl is what the health check uses; ca-certificates is for the outbound
-# federation requests of Phase 5.
-RUN apk add --no-cache ca-certificates curl
+# curl is what the health check uses, ca-certificates the roots outbound
+# federation trusts, and openssl signs the federation certificate.
+RUN apk add --no-cache ca-certificates curl openssl
 
 COPY --from=build /out/warp /usr/local/bin/warp
 COPY server/test/complement/entrypoint.sh /usr/local/bin/warp-complement

server/test/complement/README.md

@@ -6,7 +6,8 @@ Matrix integration suite and this project's primary progress metric
 drives it over the real HTTP API.
 
 ```sh
-make complement                          # the current target set
+make complement                          # the C-S set, federation off
+make complement-federation               # the federation set
 bash run.sh TestLogin                    # one test
 COMPLEMENT_DEBUG=1 make complement       # with the container logs
 ```
@@ -25,12 +26,26 @@ the phases that are done.
 
 ## Profiles
 
-**`federation=off`** — what runs today. Only the Client-Server tests;
-`WARP_FEDERATION_ENABLED=false`, port 8448 exposed but unserved.
+`COMPLEMENT_PROFILE` picks one; each has its own test set in `run.sh`.
 
-**`federation=on`** — Phase 5. It needs a TLS listener on 8448 with a
-certificate signed by the CA that Complement mounts at `/complement/ca`,
-which is the first thing to add here when federation lands.
+**`csapi`** (the default, `make complement`) — the Client-Server tests in
+`tests/csapi`, with `WARP_FEDERATION_ENABLED=false`. Port 8448 is still
+served over TLS, because the CA is mounted either way, but the gate answers
+every federation request with `M_UNRECOGNIZED`.
+
+**`federation`** (`make complement-federation`) — the federation tests in
+`tests/` and the multi-server tests in `tests/csapi`, with federation on.
+The entrypoint has Complement's CA at `/complement/ca` sign a certificate
+for the server's name, serves the API on 8448 with it, and points
+`federation.tls_ca_file` at the same CA, so that Warp trusts the other
+homeservers of the run and the servers Complement itself pretends to be.
+The switch reaches the container through `COMPLEMENT_SHARE_ENV_PREFIX`:
+`run.sh` sets `PASS_WARP_FEDERATION_ENABLED`, which Complement passes on
+as `WARP_FEDERATION_ENABLED`.
+
+The federation set leaves out partial-state joins (MSC3706,
+`TestPartialStateJoin*`), which Warp does not implement, and anything that
+needs application services (Phase 9) or device lists (Phase 7).
 
 ## The test set
 

server/test/complement/entrypoint.sh

@@ -29,11 +29,35 @@ export WARP_RATE_LIMIT_ENABLED=false
 # Open registration: the tests create their own users.
 export WARP_REGISTRATION_ENABLED=true
 
-# Federation stays off unless the profile asks for it. Phase 5 adds the
-# TLS listener on 8448 that the federation profile needs; until then this
-# image only serves the Client-Server API.
+# Federation stays off unless the profile asks for it: run.sh passes
+# WARP_FEDERATION_ENABLED=true into the container for the federation
+# profile.
 export WARP_FEDERATION_ENABLED="${WARP_FEDERATION_ENABLED:-false}"
 
+# Federation over TLS on 8448. Complement mounts a CA at /complement/ca and
+# expects every homeserver of a run to present a certificate it signed;
+# the same CA is what Warp trusts when it connects to the others, and to
+# the servers Complement itself pretends to be.
+CA_DIR=/complement/ca
+if [ -f "$CA_DIR/ca.crt" ] && [ -f "$CA_DIR/ca.key" ]; then
+	if [ ! -f "$DATA_DIR/tls.crt" ]; then
+		openssl genrsa -out "$DATA_DIR/tls.key" 2048 2>/dev/null
+		openssl req -new -sha256 -key "$DATA_DIR/tls.key" \
+			-subj "/CN=$WARP_SERVER_NAME" -out "$DATA_DIR/tls.csr"
+		printf 'subjectAltName=DNS:%s\n' "$WARP_SERVER_NAME" > "$DATA_DIR/san.ext"
+		# The serial file goes in the data directory: the CA mount may be
+		# read-only.
+		openssl x509 -req -in "$DATA_DIR/tls.csr" \
+			-CA "$CA_DIR/ca.crt" -CAkey "$CA_DIR/ca.key" \
+			-CAserial "$DATA_DIR/ca.srl" -CAcreateserial \
+			-out "$DATA_DIR/tls.crt" -days 1 -sha256 -extfile "$DATA_DIR/san.ext" 2>/dev/null
+	fi
+	export WARP_LISTEN_FEDERATION_ADDRESS=":8448"
+	export WARP_TLS_CERT_FILE="$DATA_DIR/tls.crt"
+	export WARP_TLS_KEY_FILE="$DATA_DIR/tls.key"
+	export WARP_FEDERATION_TLS_CA_FILE="$CA_DIR/ca.crt"
+fi
+
 # Complement captures the container's output, so text at debug level is the
 # most useful thing to have when a test fails.
 export WARP_LOG_LEVEL="${WARP_LOG_LEVEL:-debug}"

server/test/complement/run.sh

@@ -1,8 +1,9 @@
 #!/usr/bin/env bash
 # Run Complement against a freshly built Warp image.
 #
-#   server/test/complement/run.sh                 # the test set for the phases that are done
-#   server/test/complement/run.sh TestLogin       # one test
+#   server/test/complement/run.sh                          # the C-S test set, federation off
+#   COMPLEMENT_PROFILE=federation server/test/complement/run.sh   # the federation test set
+#   server/test/complement/run.sh TestLogin                # one test
 #   COMPLEMENT_TESTS='Test.*' server/test/complement/run.sh
 #
 # Complement is a separate Go module, so it has to be checked out and run on
@@ -16,17 +17,42 @@ REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
 : "${COMPLEMENT_REF:=main}"
 : "${COMPLEMENT_DIR:=${REPO_ROOT}/.complement}"
 : "${COMPLEMENT_IMAGE:=complement-warp}"
+: "${COMPLEMENT_PROFILE:=csapi}"
 
-# The target set for the phases that are done, from PLAN.md section 7. Add
-# to it as phases land rather than switching to the whole suite: a pass
+# The target sets for the phases that are done, from PLAN.md section 7. Add
+# to them as phases land rather than switching to the whole suite: a pass
 # count that goes down because a later phase is missing is noise.
 #
-# Phase 4 adds the stream tests: /sync in all its forms, /messages,
-# /context, filters, typing, receipts and to-device messaging. What is still
-# out is anything needing a later phase — the notification counts of the
-# push rules (Phase 8), the device lists of the keyserver (Phase 7), and
-# the whole of federation (Phase 5).
-: "${COMPLEMENT_TESTS:=TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData}"
+# csapi: the Client-Server tests, with federation off. Phase 4 added the
+# stream tests: /sync in all its forms, /messages, /context, filters,
+# typing, receipts and to-device messaging. What is still out is anything
+# needing a later phase — the notification counts of the push rules
+# (Phase 8) and the device lists of the keyserver (Phase 7).
+CSAPI_TESTS='TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData'
+
+# federation: Phase 5. Joins, leaves and invites across servers, keys,
+# transactions, missing events, server ACLs and the queries. Partial-state
+# joins (MSC3706, TestPartialStateJoin*) are not implemented and are left
+# out; so is anything needing an application service (Phase 9) or device
+# lists (Phase 7).
+FEDERATION_TESTS='TestFederation.*|TestJoinViaRoomIDAndServerName|TestJoinFederatedRoom.*|TestBannedUserCannotSendJoin|TestCannotSendNon.*Via.*|TestCannotSendKnockViaSend.*|TestOutboundFederation.*|TestInboundFederation.*|TestInboundCanReturnMissingEvents|TestRemoteAliasRequestsUnderstandUnicode|TestNetworkPartitionOrdering|TestACLs|TestGetMissingEventsGapFilling|TestEventAuth|TestUnbanViaInvite|TestJoinRoomWithFederation.*|TestRemote.*'
+
+case "$COMPLEMENT_PROFILE" in
+	csapi)
+		: "${COMPLEMENT_TESTS:=$CSAPI_TESTS}"
+		PACKAGES=(./tests/csapi/...)
+		FEDERATION_ENABLED=false
+		;;
+	federation)
+		: "${COMPLEMENT_TESTS:=$FEDERATION_TESTS}"
+		PACKAGES=(./tests/... )
+		FEDERATION_ENABLED=true
+		;;
+	*)
+		echo "complement: unknown COMPLEMENT_PROFILE '$COMPLEMENT_PROFILE' (want csapi or federation)" >&2
+		exit 2
+		;;
+esac
 
 if [[ $# -gt 0 ]]; then
 	COMPLEMENT_TESTS="$(IFS='|'; echo "$*")"
@@ -49,12 +75,18 @@ else
 	git -C "$COMPLEMENT_DIR" checkout -q FETCH_HEAD
 fi
 
-echo "complement: running '${COMPLEMENT_TESTS}' with federation off"
+echo "complement: running '${COMPLEMENT_TESTS}' (profile ${COMPLEMENT_PROFILE}, federation ${FEDERATION_ENABLED})"
 cd "$COMPLEMENT_DIR"
 
 # COMPLEMENT_SPAWN_HS_TIMEOUT_SECS is generous because the first container
 # of a run has to apply the migrations on a cold SQLite file.
+#
+# COMPLEMENT_SHARE_ENV_PREFIX hands every host variable starting with
+# PASS_ to the homeserver containers, prefix removed: it is how the profile
+# reaches the entrypoint's federation switch.
 COMPLEMENT_BASE_IMAGE="$COMPLEMENT_IMAGE" \
 	COMPLEMENT_SPAWN_HS_TIMEOUT_SECS="${COMPLEMENT_SPAWN_HS_TIMEOUT_SECS:-60}" \
 	COMPLEMENT_DEBUG="${COMPLEMENT_DEBUG:-0}" \
-	go test -count=1 -v -timeout 20m -run "^(${COMPLEMENT_TESTS})$" ./tests/csapi/...
+	COMPLEMENT_SHARE_ENV_PREFIX=PASS_ \
+	PASS_WARP_FEDERATION_ENABLED="$FEDERATION_ENABLED" \
+	go test -count=1 -v -timeout 30m -run "^(${COMPLEMENT_TESTS})$" "${PACKAGES[@]}"