Phase 3: rooms, local only

Implements PLAN.md Phase 3: the roomserver, the room half of storage, the
Client-Server room endpoints and the admin room endpoints. A client can now
create a room, invite, join, leave, kick, ban, send messages and state,
redact, name a room with an alias and find one in the public directory.
Nothing talks to another server; every operation that would need to is
refused with a message that says so rather than a bare M_FORBIDDEN.

roomserver: one input pipeline every event goes through, whatever sent it —
validate, authorise, resolve, persist, emit. The rules are not
reimplemented here; internal/matrix is pure library code and this is what
connects it to a database and to a room's history. The Client-Server
handlers do no authorisation of their own, deliberately: whether Alice may
kick Bob is a question about the room's power levels, and the one place
that is answered correctly is the authorisation rules inside the pipeline.

A mutex per room serialises writers. Two events racing into one room would
each read the same forward extremities, both become one, and fork the room
for no reason — which would then need state resolution to put back
together. State resolution is wired in anyway for the multi-extremity
case. It is unreachable from outside in this phase, and the alternative is
a pipeline that authorises against one arbitrary branch of a fork without
saying so.

Room versions 3 to 12 all work and are each tested end to end. Version 12
earns that coverage: its room ID is the create event's own ID, its create
event carries no room_id, and the create event is excluded from every other
event's auth_events — so the state auth.Check is given is the event's
auth_events plus the create event, which that version identifies by room ID
rather than citing.

createRoom sends the spec's events in the spec's order, each authorised
against what the ones before it produced. A failure part-way leaves an
incomplete room, which is the lesser evil: the alternative is holding every
event of a new room in memory and writing them in one transaction, making
the pipeline that authorises them a special case. The one failure a second
client can cause at any time, an alias clash, takes the room row back out.

Redactions stay two decisions, as the spec keeps them. The rules let almost
any redaction into the room; whether it applies to its target is a separate
test, run inside the same transaction as the redaction itself so that a
client holding an event ID can always read the stripped event.

storage: events keyed by event ID rather than by a surrogate, since an
event ID appears in prev_events, auth_events, state snapshots and client
requests alike. Stream ordering comes from a counter row bumped inside the
write transaction, because a PostgreSQL sequence is explicitly not
transactional and a rolled-back write would leave a hole in the stream that
/sync would have to be taught to skip. State snapshots are stored per event
and deduplicated by content hash, so the table grows with state changes
rather than with traffic — and that is what makes "the state after a
forward extremity" answerable at all. The room's listing columns are
denormalised onto its row because the public directory orders every
published room by member count and pages the result, which cannot be done
by reading each room's state in turn.

Federation-disabled behaviour from PLAN.md section 3 is enforced here.
Restricted joins are the exception that works: the rule needs a signature
from the authorising user's server verified, and while federation is off
that server is always this one.

adminapi: room list and detail, purge, force-leave, and room and event
counters on the dashboard. clientapi gains its own rate-limit buckets for
createRoom and for joining, both of which write events rather than read
them.

Tested against the real event core and a real database rather than fakes:
what is under test is whether an event this server signs is one the
authorisation rules accept, and a fake of either would only prove that the
pipeline calls what the fake expects. The storage tests run against both
SQL drivers. docs/rooms.md describes the pipeline, the storage decisions
and the known gaps — chief among them that history visibility is
room-level rather than per-event, and that /sync still reports no rooms,
both of which belong with the event stream in Phase 4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

server/internal/storage/postgres/migrations/0003_roomserver.sql

@@ -0,0 +1,165 @@
+-- Rooms, events and room state: the roomserver tables.
+--
+-- Events are keyed by their event ID rather than by a numeric surrogate.
+-- An event ID is 44 bytes of base64 and appears in prev_events,
+-- auth_events, state snapshots and client requests alike, so a surrogate
+-- would have to be translated at every one of those boundaries; the
+-- indexes below are what make the text key cheap enough.
+--
+-- Timestamps are milliseconds since the Unix epoch, as everywhere else in
+-- Warp.
+
+CREATE TABLE IF NOT EXISTS roomserver_rooms (
+    room_id       TEXT    NOT NULL PRIMARY KEY,
+    -- Fixed at creation. Nothing can read an event without it: the room
+    -- version decides the event format, the redaction algorithm and the
+    -- authorisation rules.
+    room_version  TEXT    NOT NULL,
+    creator       TEXT    NOT NULL,
+    is_published  BOOLEAN NOT NULL DEFAULT FALSE,
+    created_at_ms BIGINT  NOT NULL,
+
+    -- The denormalised room listing columns. Everything below is derivable
+    -- from the room's current state and is kept here anyway, because the
+    -- public room directory orders every published room by member count
+    -- and pages the result: that cannot be done by reading each room's
+    -- state in turn. The roomserver recomputes them when the state behind
+    -- them changes; joined_members the store counts for itself.
+    name               TEXT   NOT NULL DEFAULT '',
+    topic              TEXT   NOT NULL DEFAULT '',
+    avatar_url         TEXT   NOT NULL DEFAULT '',
+    canonical_alias    TEXT   NOT NULL DEFAULT '',
+    join_rule          TEXT   NOT NULL DEFAULT '',
+    history_visibility TEXT   NOT NULL DEFAULT '',
+    guest_access       TEXT   NOT NULL DEFAULT '',
+    room_type          TEXT   NOT NULL DEFAULT '',
+    joined_members     BIGINT NOT NULL DEFAULT 0
+);
+
+-- The public room directory's own ordering: published rooms, biggest first.
+CREATE INDEX IF NOT EXISTS roomserver_rooms_published_idx
+    ON roomserver_rooms (is_published, joined_members);
+
+-- One counter row per stream, incremented inside the write transaction.
+--
+-- Neither AUTOINCREMENT nor a sequence would do: SQLite and PostgreSQL
+-- spell those differently, and a PostgreSQL sequence is explicitly not
+-- transactional, so a rolled-back write would leave a hole in the stream
+-- that /sync would have to be taught to skip. An UPDATE on this row takes
+-- a row lock in both engines and gives out positions with no gaps.
+CREATE TABLE IF NOT EXISTS roomserver_sequences (
+    name  TEXT   NOT NULL PRIMARY KEY,
+    value BIGINT NOT NULL
+);
+
+INSERT INTO roomserver_sequences (name, value) VALUES ('stream_ordering', 0)
+    ON CONFLICT (name) DO NOTHING;
+INSERT INTO roomserver_sequences (name, value) VALUES ('state_snapshot', 0)
+    ON CONFLICT (name) DO NOTHING;
+
+-- A state snapshot is the complete state of a room at one point, stored
+-- once and shared by every event it applies to. Consecutive messages in a
+-- room whose state is not changing all point at the same snapshot, so the
+-- cost is one snapshot per state change rather than one per event.
+CREATE TABLE IF NOT EXISTS roomserver_state_snapshots (
+    snapshot_id BIGINT NOT NULL PRIMARY KEY,
+    room_id     TEXT   NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    -- SHA-256 over the sorted (type, state_key, event_id) triples: the
+    -- identity of the snapshot's contents, which is what lets two
+    -- identical states share a row.
+    state_hash  TEXT   NOT NULL,
+    UNIQUE (room_id, state_hash)
+);
+
+CREATE TABLE IF NOT EXISTS roomserver_state_entries (
+    snapshot_id BIGINT NOT NULL REFERENCES roomserver_state_snapshots (snapshot_id) ON DELETE CASCADE,
+    event_type  TEXT   NOT NULL,
+    state_key   TEXT   NOT NULL,
+    event_id    TEXT   NOT NULL,
+    PRIMARY KEY (snapshot_id, event_type, state_key)
+);
+
+CREATE TABLE IF NOT EXISTS roomserver_events (
+    event_id         TEXT    NOT NULL PRIMARY KEY,
+    room_id          TEXT    NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    -- This server's own monotonic position for the event: the order it was
+    -- persisted in, which is the order clients are shown it in.
+    stream_ordering  BIGINT  NOT NULL,
+    event_type       TEXT    NOT NULL,
+    -- NULL for a message event. The empty string is a valid state key, so
+    -- "no state key" cannot be spelled as ''.
+    state_key        TEXT,
+    sender           TEXT    NOT NULL,
+    depth            BIGINT  NOT NULL,
+    origin_server_ts BIGINT  NOT NULL,
+    -- The event as canonical JSON, byte for byte as it was signed. Every
+    -- other column here is a projection of it, written out to be indexed.
+    event_json       TEXT    NOT NULL,
+    -- The state of the room as of this event's prev_events, 0 for the
+    -- create event. Keeping it per event is what lets the state after a
+    -- forward extremity be worked out, which state resolution needs when a
+    -- room has more than one.
+    state_before_id  BIGINT  NOT NULL DEFAULT 0,
+    -- The redaction that applies to this event, or ''. The JSON above is
+    -- rewritten to its redacted form at the same time; this records which
+    -- event did it so that unsigned.redacted_because can be filled in.
+    redacted_by      TEXT    NOT NULL DEFAULT ''
+);
+
+CREATE UNIQUE INDEX IF NOT EXISTS roomserver_events_stream_idx
+    ON roomserver_events (stream_ordering);
+
+-- Paging a room's timeline, which is what /messages and /sync do.
+CREATE INDEX IF NOT EXISTS roomserver_events_room_stream_idx
+    ON roomserver_events (room_id, stream_ordering);
+
+-- The forward extremities: the events with no children, and so the
+-- prev_events of whatever is sent next.
+CREATE TABLE IF NOT EXISTS roomserver_extremities (
+    room_id  TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    event_id TEXT NOT NULL,
+    PRIMARY KEY (room_id, event_id)
+);
+
+-- The room's resolved current state: the state after its extremities.
+--
+-- It is derivable from the events, and is stored anyway because every
+-- authorisation decision reads it. The membership column is a denormalised
+-- copy of content.membership on m.room.member rows, so that "the rooms
+-- @alice is joined to" is an index lookup rather than a scan that parses
+-- JSON.
+CREATE TABLE IF NOT EXISTS roomserver_current_state (
+    room_id    TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    event_type TEXT NOT NULL,
+    state_key  TEXT NOT NULL,
+    event_id   TEXT NOT NULL,
+    membership TEXT NOT NULL DEFAULT '',
+    PRIMARY KEY (room_id, event_type, state_key)
+);
+
+CREATE INDEX IF NOT EXISTS roomserver_current_state_membership_idx
+    ON roomserver_current_state (event_type, state_key, membership);
+
+-- Client transaction IDs, so that a retried request returns the event the
+-- first attempt created instead of sending a second one.
+CREATE TABLE IF NOT EXISTS roomserver_transactions (
+    localpart TEXT   NOT NULL,
+    device_id TEXT   NOT NULL,
+    txn_id    TEXT   NOT NULL,
+    event_id  TEXT   NOT NULL,
+    room_id   TEXT   NOT NULL,
+    sent_at_ms BIGINT NOT NULL,
+    PRIMARY KEY (localpart, device_id, txn_id)
+);
+
+-- Local room aliases. Remote aliases are never stored: they are resolved
+-- over federation every time, because the other server may have moved them.
+CREATE TABLE IF NOT EXISTS roomserver_aliases (
+    alias         TEXT   NOT NULL PRIMARY KEY,
+    room_id       TEXT   NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
+    creator       TEXT   NOT NULL,
+    created_at_ms BIGINT NOT NULL
+);
+
+CREATE INDEX IF NOT EXISTS roomserver_aliases_room_idx
+    ON roomserver_aliases (room_id);