Phase 5: federation

Implements PLAN.md Phase 5: the Server-Server API. Warp joins rooms on
other servers and lets their users join its rooms, invites across servers
with the invitee's server countersigning, rejects remote invites (locally
if the inviter is unreachable), and exchanges events, typing, receipts and
to-device messages with other homeservers — all behind the runtime switch,
which now starts and stops real outbound traffic.

The protocol lives in internal/federation with no knowledge of rooms:
X-Matrix signing (fuzzed parser), the spec's discovery algorithm keeping
the dial address and the TLS name apart, a signed client that refuses
every request while the switch is off or the policy denies the
destination, and a key ring that caches other servers' keys for at most a
week, with optional notaries.

The roomserver takes events from elsewhere through the spec's intake:
signatures and hashes (a hash mismatch keeps the redacted form), the rules
against the event's own auth events, against the state before it — with
/get_missing_events for small gaps and /state_ids for wide ones — and
against the current state, where a failure is a soft failure. Events held
without their state are outliers: a column on roomserver_events, because
the current state of a room joined over federation is almost all
outliers, and excluded from every timeline query. Forks are resolved at
the room's head. Membership across servers is make_join/send_join,
make_leave/send_leave and countersigned invites, in both directions;
m.room.server_acl is enforced on everything another server sends.
History before a remote join is backfilled when /messages pages past it,
stored at negative stream positions.

internal/federationapi is the running component: the inbound endpoints,
and an outbound queue that persists what it owes each destination, sends
it in transactions of 50 PDUs and 100 EDUs, backs off from ten seconds to
six hours, survives restarts, and drops events persisted while federation
is off rather than delivering a backlog in one burst. Its Module is what
the switch starts and stops.

Also: a TLS listener for port 8448, trusted key servers, key validity and
an outbound CA bundle in the config; per-destination health and backoff
reset in the admin API; remote profiles, aliases and directories in the
Client-Server API; a Complement federation profile. See
docs/federation.md, including its known gaps (no partial-state joins, no
key rotation).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

server/internal/storage/postgres/migrations/0005_federation.sql

@@ -0,0 +1,113 @@
+-- Federation: other servers' keys, the outbound queue and its per-server
+-- backoff, the inbound transaction log, and the two additions the room
+-- tables need once a room's events can come from somewhere else.
+--
+-- Timestamps are milliseconds since the Unix epoch, as everywhere else in
+-- Warp.
+
+-- An outlier is an event this server holds without holding the room's
+-- state at it: the state and auth chain a join is handed by the resident
+-- server, an auth event fetched to judge something else, an invite to a
+-- room nobody here is in, an event that soft-failed. It is a real event —
+-- it is signed, it was authorised against its own auth events, and other
+-- events can cite it — but it has no place in the timeline: no state
+-- snapshot (state_before_id stays 0), never a forward extremity, never
+-- returned by /sync or /messages.
+--
+-- It goes on the events table rather than into a table of its own because
+-- every read of a room's current state joins the events table, and the
+-- current state of a room joined over federation is made almost entirely
+-- of outliers. An outlier that later arrives as an ordinary event — fetched
+-- by /get_missing_events, say — is promoted in place.
+ALTER TABLE roomserver_events ADD COLUMN outlier BOOLEAN NOT NULL DEFAULT FALSE;
+
+-- The stripped state an invite over federation arrives with.
+--
+-- An invite to a room this server is not in is the invite event alone: the
+-- room's state is on the inviting server. What a client shows in its
+-- invite list — the room's name, its avatar, who invited you — comes from
+-- the invite_room_state the inviting server sent along, which is stored
+-- here until the invite is answered and then deleted with it.
+CREATE TABLE IF NOT EXISTS roomserver_invite_state (
+    room_id    TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    user_id    TEXT NOT NULL,
+    -- A JSON array of stripped state events, exactly as received.
+    state_json TEXT NOT NULL,
+    PRIMARY KEY (room_id, user_id)
+);
+
+-- Other servers' signing keys, as their key servers or a notary reported
+-- them. A key is never deleted: an event signed years ago is still
+-- verified with the key that signed it, and a server that has rotated its
+-- key lists the old one only as an old_verify_key with the time it expired.
+CREATE TABLE IF NOT EXISTS federation_server_keys (
+    server_name    TEXT   NOT NULL,
+    key_id         TEXT   NOT NULL,
+    -- Unpadded base64, as it is on the wire.
+    public_key     TEXT   NOT NULL,
+    -- How long the server vouched for the key when it was fetched. A key
+    -- is re-fetched once this passes, and an event from room version 5 on
+    -- may not be verified with a key whose validity ended before the event
+    -- was sent.
+    valid_until_ts BIGINT NOT NULL,
+    -- When the key stopped being used, for an old_verify_key; 0 for a key
+    -- the server still signs with.
+    expired_ts     BIGINT NOT NULL DEFAULT 0,
+    fetched_at_ms  BIGINT NOT NULL,
+    PRIMARY KEY (server_name, key_id)
+);
+
+-- One counter row per stream; see roomserver_sequences for why these are
+-- rows and not database sequences.
+CREATE TABLE IF NOT EXISTS federation_sequences (
+    name  TEXT   NOT NULL PRIMARY KEY,
+    value BIGINT NOT NULL
+);
+
+INSERT INTO federation_sequences (name, value) VALUES ('queue', 0)
+    ON CONFLICT (name) DO NOTHING;
+
+-- The outbound queue: every PDU and EDU waiting for a destination.
+--
+-- A row is deleted once the destination has acknowledged the transaction
+-- that carried it, so what is here after a restart is exactly what still
+-- has to be sent. The payload is the JSON that goes on the wire, copied at
+-- enqueue time: re-reading a PDU from the room tables at send time would
+-- send its redacted form if it was redacted in the meantime, and that is
+-- not the event the destination has to verify.
+CREATE TABLE IF NOT EXISTS federation_queue (
+    queue_id     BIGINT NOT NULL PRIMARY KEY,
+    destination  TEXT   NOT NULL,
+    -- 'pdu' or 'edu'.
+    kind         TEXT   NOT NULL,
+    room_id      TEXT   NOT NULL DEFAULT '',
+    event_id     TEXT   NOT NULL DEFAULT '',
+    payload      TEXT   NOT NULL,
+    queued_at_ms BIGINT NOT NULL
+);
+
+CREATE INDEX IF NOT EXISTS federation_queue_destination_idx
+    ON federation_queue (destination, kind, queue_id);
+
+-- Per-destination delivery health: how often sending has failed in a row
+-- and when it may next be tried. A destination with no row has never
+-- failed.
+CREATE TABLE IF NOT EXISTS federation_destinations (
+    server_name     TEXT    NOT NULL PRIMARY KEY,
+    failures        INTEGER NOT NULL DEFAULT 0,
+    retry_at_ms     BIGINT  NOT NULL DEFAULT 0,
+    last_success_ms BIGINT  NOT NULL DEFAULT 0,
+    last_failure_ms BIGINT  NOT NULL DEFAULT 0,
+    last_error      TEXT    NOT NULL DEFAULT ''
+);
+
+-- Transactions received from other servers, so that a retransmission is
+-- answered with the response the first delivery got instead of being
+-- processed a second time.
+CREATE TABLE IF NOT EXISTS federation_inbound_txns (
+    origin         TEXT   NOT NULL,
+    txn_id         TEXT   NOT NULL,
+    response_json  TEXT   NOT NULL,
+    received_at_ms BIGINT NOT NULL,
+    PRIMARY KEY (origin, txn_id)
+);

server/internal/storage/postgres/migrations/0006_backfill.sql

@@ -0,0 +1,15 @@
+-- Outbound backfill: history fetched from other servers for a room this
+-- server joined part-way through.
+--
+-- roomserver_events.stream_ordering is the order clients are shown a room
+-- in, and the stream_ordering counter only ever goes up: every event it
+-- numbers arrived after the ones before it. Backfilled history arrives
+-- after everything too, but belongs before all of it. So it is numbered by
+-- a second counter that runs the other way — from -1 downwards — and every
+-- backfilled event sits below every event this server held when it came.
+--
+-- Zero is neither counter's: the stream_ordering counter's first value is
+-- 1 and this one's is -1, and a window bound of zero keeps meaning
+-- "unbounded at that end".
+INSERT INTO roomserver_sequences (name, value) VALUES ('backfill_ordering', 0)
+    ON CONFLICT (name) DO NOTHING;