Phase 10: hardening and ops

Implements PLAN.md Phase 10.

Simplified Sliding Sync (MSC4186) is served at the unstable
org.matrix.simplified_msc3575 endpoint and advertised in /versions. Lists
are sorted by activity and filtered by DM, invite, encryption, room type,
space and tag; room subscriptions, required_state with wildcards, $ME and
$LAZY, and the to_device, e2ee, account_data, receipts and typing
extensions are supported. Per-connection state is kept in memory and an
unknown pos answers M_UNKNOWN_POS.

Threads and relations: m.relates_to is indexed on roomserver_events
(migration 0012), /relations (with recursion) and /threads are served, and
every event a timeline read returns carries its m.thread, m.replace and
m.reference summaries in unsigned.m.relations. Notifications carry a
thread ID (migration 0013): threaded receipts clear only their thread, and
sync reports unread_thread_notifications when the filter asks.

Spaces: /hierarchy walks a space breadth first, asking other servers about
rooms this one is not in through the new federation hierarchy endpoint;
room_summary describes one room. /search runs on FTS5 (sqlite, pocketbase)
or tsvector (postgres), bounded to what each user may see, with context,
profiles, state, grouping and paging.

The PocketBase driver is real: the sqlite schema and store on an embedded
PocketBase app's database, with media in its filesystem, local or S3
(database.pocketbase.s3). It runs the full storage test suite.

Ops: OpenTelemetry tracing over OTLP/HTTP (tracing.*), with server spans
per route, component spans, and trace propagation on federation, push and
application service calls. New metrics for persisted events, waiting
syncs, sliding sync connections and the database pool, with a Grafana
dashboard. deploy/ adds a Compose setup (PostgreSQL, Prometheus, Grafana,
Jaeger) and a Helm chart; cmd/warp-loadtest loads a server and reports
request and delivery latencies.

The load tester found a long-standing bug, fixed here: on a server where
no account data had been written, every incremental /sync re-sent the push
rules and returned at once, and an all-zero token was taken for an initial
sync, so long polls became busy loops.

The PostgreSQL migrations and search dialect compile but were not run
against a live PostgreSQL in this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

server/internal/storage/postgres/migrations/0012_relations_search.sql

@@ -0,0 +1,77 @@
+-- Phase 10: relations (threads, edits, reactions, references) and
+-- full-text search.
+
+-- content.m.relates_to of each event, written out so that "the children of
+-- this event" is an index lookup. Both are '' for an event that relates to
+-- nothing, and are cleared when the event is redacted: the redaction
+-- algorithm strips the content they came from.
+ALTER TABLE roomserver_events ADD COLUMN relates_to_id TEXT NOT NULL DEFAULT '';
+ALTER TABLE roomserver_events ADD COLUMN rel_type TEXT NOT NULL DEFAULT '';
+
+-- The events already stored. Only a relation with both fields a string
+-- counts, which is the rule the roomserver applies to new events. The LIKE
+-- keeps the cast to jsonb off the events that cannot have one.
+UPDATE roomserver_events e SET
+    relates_to_id = r.rel ->> 'event_id',
+    rel_type      = r.rel ->> 'rel_type'
+FROM (
+    SELECT event_id, event_json::jsonb -> 'content' -> 'm.relates_to' AS rel
+    FROM roomserver_events
+    WHERE redacted_by = '' AND event_json LIKE '%m.relates_to%'
+) r
+WHERE e.event_id = r.event_id
+  AND jsonb_typeof(r.rel -> 'event_id') = 'string'
+  AND jsonb_typeof(r.rel -> 'rel_type') = 'string';
+
+-- /relations, the bundled summaries and /threads all start from a parent
+-- and read its children in stream order.
+CREATE INDEX IF NOT EXISTS roomserver_events_relations_idx
+    ON roomserver_events (room_id, relates_to_id, rel_type, stream_ordering);
+
+-- The full-text index /search reads: one row per searchable event, keyed by
+-- the event's stream ordering, which is unique across the server and is
+-- what a redaction finds the row by. Outliers are not here: an event joins
+-- the index when it joins a timeline.
+--
+-- The 'simple' configuration lower-cases and splits on word boundaries
+-- without stemming for any one language, which is the right default for a
+-- server whose rooms are in every language at once.
+CREATE TABLE IF NOT EXISTS roomserver_event_search (
+    stream_ordering BIGINT NOT NULL PRIMARY KEY,
+    room_id         TEXT   NOT NULL,
+    event_id        TEXT   NOT NULL,
+    search_key      TEXT   NOT NULL,
+    sender          TEXT   NOT NULL,
+    event_type      TEXT   NOT NULL,
+    body            TEXT   NOT NULL,
+    vector          TSVECTOR GENERATED ALWAYS AS (to_tsvector('simple', body)) STORED
+);
+
+CREATE INDEX IF NOT EXISTS roomserver_event_search_vector_idx
+    ON roomserver_event_search USING GIN (vector);
+CREATE INDEX IF NOT EXISTS roomserver_event_search_room_idx
+    ON roomserver_event_search (room_id, stream_ordering);
+
+INSERT INTO roomserver_event_search (stream_ordering, room_id, event_id, search_key, sender, event_type, body)
+SELECT stream_ordering, room_id, event_id, k.search_key, sender, event_type, k.body
+FROM (
+    SELECT stream_ordering, room_id, event_id, sender, event_type, event_json::jsonb -> 'content' AS content
+    FROM roomserver_events
+    WHERE event_type IN ('m.room.message', 'm.room.name', 'm.room.topic')
+      AND NOT outlier AND redacted_by = ''
+) e
+CROSS JOIN LATERAL (
+    SELECT CASE e.event_type
+               WHEN 'm.room.message' THEN 'content.body'
+               WHEN 'm.room.name'    THEN 'content.name'
+               ELSE                       'content.topic'
+           END AS search_key,
+           CASE e.event_type
+               WHEN 'm.room.message' THEN e.content ->> 'body'
+               WHEN 'm.room.name'    THEN e.content ->> 'name'
+               ELSE                       e.content ->> 'topic'
+           END AS body
+) k
+WHERE k.body IS NOT NULL
+  AND jsonb_typeof(e.content -> substr(k.search_key, 9)) = 'string'
+ON CONFLICT (stream_ordering) DO NOTHING;

server/internal/storage/postgres/migrations/0013_thread_notifications.sql

@@ -0,0 +1,21 @@
+-- Phase 10: notifications counted per thread.
+--
+-- thread_id is the thread an event that notified a user belongs to: the
+-- root its m.relates_to names when rel_type is m.thread, and '' for an
+-- event on the main timeline. A read receipt for a thread clears that
+-- thread's rows and a receipt for the main timeline clears the '' rows,
+-- which is what lets reading one thread leave the rest of the room unread.
+ALTER TABLE push_notifications ADD COLUMN thread_id TEXT NOT NULL DEFAULT '';
+
+-- The notifications already stored, from the relation 0012 projected onto
+-- their events. A notification whose event has gone stays on the main
+-- timeline, where an unthreaded or main receipt still clears it.
+UPDATE push_notifications SET thread_id = COALESCE((
+    SELECT e.relates_to_id FROM roomserver_events e
+    WHERE e.event_id = push_notifications.event_id AND e.rel_type = 'm.thread'
+), '');
+
+-- A threaded receipt marks one thread's rows read; the per-thread counts a
+-- sync asks for group a room's unread rows by thread.
+CREATE INDEX IF NOT EXISTS push_notifications_thread_idx
+    ON push_notifications (user_id, room_id, thread_id, is_read, event_pos);

server/internal/storage/postgres/postgres.go

@@ -9,6 +9,7 @@ import (
 	"embed"
 	"fmt"
 	"log/slog"
+	"strings"
 
 	_ "github.com/jackc/pgx/v5/stdlib" // registers the "pgx" database/sql driver
 
@@ -44,6 +45,33 @@ var settingsQueries = sqlstore.SettingsQueries{
 	Delete: `DELETE FROM server_settings WHERE setting_key = $1`,
 }
 
+// searchDialect is the full-text index on tsvector, with the 'simple'
+// configuration the migration builds the column with.
+var searchDialect = &sqlstore.SearchDialect{
+	Insert: `INSERT INTO roomserver_event_search (stream_ordering, room_id, event_id, search_key, sender, event_type, body)
+		VALUES (?, ?, ?, ?, ?, ?, ?) ON CONFLICT (stream_ordering) DO NOTHING`,
+	DeleteEvent: `DELETE FROM roomserver_event_search WHERE room_id = ? AND event_id = ?`,
+	DeleteRoom:  `DELETE FROM roomserver_event_search WHERE room_id = ?`,
+	From:        `roomserver_event_search s`,
+	Name:        `s`,
+	Position:    `s.stream_ordering`,
+	Match:       `s.vector @@ to_tsquery('simple', ?)`,
+	Rank:        `ts_rank(s.vector, to_tsquery('simple', ?))`,
+	RankArgs:    1,
+	Query:       tsQuery,
+}
+
+// tsQuery builds a tsquery from a search term: every word as a prefix
+// match, ANDed. The words are letters and digits only, so none of them can
+// carry tsquery syntax of its own.
+func tsQuery(term string) string {
+	words := storage.SearchWords(term)
+	for i, w := range words {
+		words[i] = w + ":*"
+	}
+	return strings.Join(words, " & ")
+}
+
 // Database is the PostgreSQL implementation of storage.Database.
 type Database struct {
 	*sqlstore.Store
@@ -77,6 +105,7 @@ func Open(ctx context.Context, cfg storage.Config, log *slog.Logger) (storage.Da
 		Settings:          settingsQueries,
 		Rebind:            rebind,
 		IsUniqueViolation: isUniqueViolation,
+		Search:            searchDialect,
 		Logger:            log,
 	})
 	if err != nil {