Phase 10: hardening and ops

Implements PLAN.md Phase 10.

Simplified Sliding Sync (MSC4186) is served at the unstable
org.matrix.simplified_msc3575 endpoint and advertised in /versions. Lists
are sorted by activity and filtered by DM, invite, encryption, room type,
space and tag; room subscriptions, required_state with wildcards, $ME and
$LAZY, and the to_device, e2ee, account_data, receipts and typing
extensions are supported. Per-connection state is kept in memory and an
unknown pos answers M_UNKNOWN_POS.

Threads and relations: m.relates_to is indexed on roomserver_events
(migration 0012), /relations (with recursion) and /threads are served, and
every event a timeline read returns carries its m.thread, m.replace and
m.reference summaries in unsigned.m.relations. Notifications carry a
thread ID (migration 0013): threaded receipts clear only their thread, and
sync reports unread_thread_notifications when the filter asks.

Spaces: /hierarchy walks a space breadth first, asking other servers about
rooms this one is not in through the new federation hierarchy endpoint;
room_summary describes one room. /search runs on FTS5 (sqlite, pocketbase)
or tsvector (postgres), bounded to what each user may see, with context,
profiles, state, grouping and paging.

The PocketBase driver is real: the sqlite schema and store on an embedded
PocketBase app's database, with media in its filesystem, local or S3
(database.pocketbase.s3). It runs the full storage test suite.

Ops: OpenTelemetry tracing over OTLP/HTTP (tracing.*), with server spans
per route, component spans, and trace propagation on federation, push and
application service calls. New metrics for persisted events, waiting
syncs, sliding sync connections and the database pool, with a Grafana
dashboard. deploy/ adds a Compose setup (PostgreSQL, Prometheus, Grafana,
Jaeger) and a Helm chart; cmd/warp-loadtest loads a server and reports
request and delivery latencies.

The load tester found a long-standing bug, fixed here: on a server where
no account data had been written, every incremental /sync re-sent the push
rules and returned at once, and an all-zero token was taken for an initial
sync, so long polls became busy loops.

The PostgreSQL migrations and search dialect compile but were not run
against a live PostgreSQL in this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/README.md

@@ -6,12 +6,16 @@
 | [development.md](development.md) | building, testing, Complement, adding a migration |
 | [event-core.md](event-core.md) | `internal/matrix`: canonical JSON, events, auth rules, state resolution |
 | [rooms.md](rooms.md) | `internal/roomserver`: the input pipeline, room storage, known gaps |
-| [sync.md](sync.md) | `internal/syncapi`: stream tokens, long polling, filters, the ephemeral streams |
+| [sync.md](sync.md) | `internal/syncapi`: stream tokens, long polling, filters, the ephemeral streams, Simplified Sliding Sync |
+| [relations.md](relations.md) | threads, edits, reactions and references: `/relations`, `/threads` and `unsigned.m.relations` |
+| [spaces.md](spaces.md) | `/hierarchy` across servers, and `room_summary` |
+| [search.md](search.md) | `/search` on FTS5 and `tsvector` |
 | [federation.md](federation.md) | the Server-Server API: discovery, keys, taking in events, joins across servers, the outbound queue, the switch |
 | [media.md](media.md) | `internal/mediaapi`: uploads, downloads, thumbnails, other servers' media, where the files go |
 | [e2ee.md](e2ee.md) | `internal/keyserver`: device and one-time keys, cross-signing, device list tracking, key backup |
 | [push.md](push.md) | `internal/pushrules` and `internal/pushserver`: push rules, notification counts, pushers, the push gateway client |
 | [appservices.md](appservices.md) | `internal/appservice`: registration files, namespaces, acting as users, delivering events to bridges |
 | [admin-api.md](admin-api.md) | `/_warp/admin/v1`: the operator API behind the admin panel |
+| [deployment.md](deployment.md) | storage drivers, the image, Compose, Helm, metrics and the dashboard, tracing, load testing |
 
 The design and roadmap are in [PLAN.md](../PLAN.md) at the repository root.

docs/configuration.md

@@ -36,12 +36,20 @@ List-valued variables are comma-separated. A single `-` clears a list:
 | `listen.federation_address` | `WARP_LISTEN_FEDERATION_ADDRESS` | empty | A second listener serving the same API over TLS, usually `:8448`. Empty = none; see *Federation over TLS*. |
 | `listen.tls_cert_file` | `WARP_TLS_CERT_FILE` | empty | PEM certificate chain for the federation listener. Required with it. |
 | `listen.tls_key_file` | `WARP_TLS_KEY_FILE` | empty | PEM key for the federation listener. Required with it. |
-| `database.driver` | `WARP_DB_DRIVER` | `sqlite` | `sqlite`, `postgres`, `pocketbase` (not implemented). |
-| `database.connection_string` | `WARP_DB_CONNECTION_STRING` | SQLite file in the working directory | |
+| `database.driver` | `WARP_DB_DRIVER` | `sqlite` | `sqlite`, `postgres`, `pocketbase`. |
+| `database.connection_string` | `WARP_DB_CONNECTION_STRING` | SQLite file in the working directory | A DSN for postgres, a `file:` URI for sqlite, the data directory (PocketBase's `pb_data`) for pocketbase. |
 | `database.max_open_conns` | `WARP_DB_MAX_OPEN_CONNS` | `10` | Ignored by SQLite: writes are serialised onto one connection. |
 | `database.max_idle_conns` | `WARP_DB_MAX_IDLE_CONNS` | `5` | |
 | `database.conn_max_lifetime` | `WARP_DB_CONN_MAX_LIFETIME` | `1h` | |
 | `database.auto_migrate` | `WARP_DB_AUTO_MIGRATE` | `true` | `false` makes startup fail while migrations are pending. |
+| `database.pocketbase.s3.enabled` | `WARP_PB_S3_ENABLED` | `false` | pocketbase only: media content goes to an S3-compatible bucket instead of `pb_data/storage`. |
+| `database.pocketbase.s3.bucket` / `.region` / `.endpoint` | `WARP_PB_S3_BUCKET` / `WARP_PB_S3_REGION` / `WARP_PB_S3_ENDPOINT` | empty | All required when S3 is on. |
+| `database.pocketbase.s3.access_key` / `.secret` | `WARP_PB_S3_ACCESS_KEY` / `WARP_PB_S3_SECRET` | empty | All required when S3 is on. |
+| `database.pocketbase.s3.force_path_style` | `WARP_PB_S3_FORCE_PATH_STYLE` | `false` | For MinIO and other servers without virtual-host buckets. |
+| `database.pocketbase.s3.enabled` | `WARP_PB_S3_ENABLED` | `false` | pocketbase only: media content in an S3-compatible bucket instead of `pb_data/storage`. |
+| `database.pocketbase.s3.bucket` / `.region` / `.endpoint` | `WARP_PB_S3_BUCKET` / `WARP_PB_S3_REGION` / `WARP_PB_S3_ENDPOINT` | empty | All required with S3 on. |
+| `database.pocketbase.s3.access_key` / `.secret` | `WARP_PB_S3_ACCESS_KEY` / `WARP_PB_S3_SECRET` | empty | All required with S3 on. |
+| `database.pocketbase.s3.force_path_style` | `WARP_PB_S3_FORCE_PATH_STYLE` | `false` | For MinIO and other servers without virtual-host buckets. |
 | `client_api.cors_allowed_origins` | `WARP_CORS_ALLOWED_ORIGINS` | `["*"]` | The spec expects any origin. |
 | `client_api.trust_x_forwarded_for` | `WARP_TRUST_X_FORWARDED_FOR` | `false` | Only behind a proxy that overwrites the header; the rate limiter keys on the result. |
 | `client_api.access_token_lifetime` | `WARP_ACCESS_TOKEN_LIFETIME` | `0s` | `0` = never expire. With a lifetime, clients may ask for a refresh token at login. |
@@ -91,6 +99,10 @@ List-valued variables are comma-separated. A single `-` clears a list:
 | `logging.add_source` | `WARP_LOG_ADD_SOURCE` | `false` | File and line in every record. |
 | `metrics.enabled` | `WARP_METRICS_ENABLED` | `true` | |
 | `metrics.path` | `WARP_METRICS_PATH` | `/metrics` | Served on the same listener. |
+| `tracing.enabled` | `WARP_TRACING_ENABLED` | `false` | OpenTelemetry tracing; see [deployment.md](deployment.md#tracing). |
+| `tracing.endpoint` | `WARP_TRACING_ENDPOINT` | empty | The OTLP/HTTP collector, e.g. `http://localhost:4318`. Empty falls back to `OTEL_EXPORTER_OTLP_ENDPOINT`, then to `localhost:4318`. |
+| `tracing.sample_ratio` | `WARP_TRACING_SAMPLE_RATIO` | `1` | The fraction of new traces kept, 0 to 1. A request that arrives inside a sampled trace is always kept. |
+| `tracing.service_name` | `WARP_TRACING_SERVICE_NAME` | `warp` | |
 
 The rate limit buckets share one Go type, so per-bucket environment
 variables would all collide on the same name; they are YAML only.

docs/deployment.md

@@ -0,0 +1,171 @@
+# Deployment and operations
+
+What there is for running Warp somewhere other than a laptop: the container
+image, a Compose setup, a Helm chart, a Grafana dashboard, tracing, and a
+load tester. All of it is under [`deploy/`](../deploy) except the image,
+which is the [`Dockerfile`](../Dockerfile) at the root, and the load
+tester, which is `server/cmd/warp-loadtest`.
+
+## Choosing a storage driver
+
+| Driver | For | Notes |
+|---|---|---|
+| `postgres` | production | Several connections, so reads and writes run side by side. |
+| `sqlite` | development, a single user, a small server | One writer. Everything is one file plus `media.base_path`. |
+| `pocketbase` | an "appliance": one binary, one data directory | The sqlite driver's tables inside an embedded [PocketBase](https://pocketbase.io) app's `data.db`, written through PocketBase's own single-writer connection. Media goes into PocketBase's filesystem, which can be an S3 bucket (`database.pocketbase.s3`). |
+
+The pocketbase driver is the SQLite schema on the SQLite engine, so it is
+at parity with sqlite by construction, and the storage tests run against
+both. What it adds is PocketBase around it: its system tables in the same
+database, its `auxiliary.db` for its own logs, and its filesystem
+abstraction for media. Warp does not serve PocketBase's admin UI or API;
+the app is reachable from Go through `pocketbase.Database.App()` for
+features that want it later. Back up the whole data directory.
+
+## The container image
+
+```sh
+docker build -t warp .
+```
+
+One static binary on Alpine, running as UID 10001, with `/data` as its
+working directory and volume. The defaults put the SQLite database, the
+signing key and media under `/data`. The image does not create a signing
+key on its own; the Compose file and the Helm chart do it on first start,
+and a plain `docker run` needs one first:
+
+```sh
+docker run --rm -v warp-data:/data warp generate-keys --output /data/signing.key
+```
+
+## Compose
+
+[`deploy/compose`](../deploy/compose) runs Warp with PostgreSQL, and with
+the `observability` profile, Prometheus, Grafana (with the dashboard
+provisioned) and Jaeger:
+
+```sh
+cd deploy/compose
+WARP_SERVER_NAME=matrix.example.org docker compose up -d
+docker compose --profile observability up -d
+docker compose exec warp warp create-account --username admin --admin
+```
+
+Settings come from the environment, so anything in
+[configuration.md](configuration.md) can be added to the `warp` service.
+`WARP_TRACING_ENABLED=true` sends traces to the Jaeger container.
+
+## Helm
+
+[`deploy/helm/warp`](../deploy/helm/warp) is a chart for one Warp pod.
+Warp is one process, so the deployment always runs exactly one replica and
+replaces it with the `Recreate` strategy: two pods would be two
+homeservers with the same name.
+
+```sh
+helm install warp deploy/helm/warp \
+  --set serverName=example.org \
+  --set publicBaseURL=https://matrix.example.org \
+  --set image.repository=registry.example.org/warp --set image.tag=0.10.0 \
+  --set ingress.enabled=true
+```
+
+There is no published image; build one and push it to a registry the
+cluster can pull from. The chart creates a volume for `/data` that is kept
+when the release is deleted, because the signing key is on it unless
+`signingKey.existingSecret` names a secret holding one. For PostgreSQL, set
+`database.driver=postgres` and either `database.connectionString` or
+`database.existingSecret`. `metrics.serviceMonitor.enabled` adds a
+prometheus-operator `ServiceMonitor`. The ingress needs a read timeout
+above the 30-second long poll; `values.yaml` has the ingress-nginx
+annotations.
+
+## Reverse proxy
+
+Behind a proxy that terminates TLS:
+
+- Route `/_matrix`, `/.well-known/matrix` and `/_warp` to Warp.
+- Keep the proxy's read timeout above `sync.long_poll_timeout` (30 s), or
+  every idle `/sync` is cut off.
+- Allow request bodies of `media.max_upload_bytes` (50 MiB by default).
+- Set `client_api.trust_x_forwarded_for: true` only if the proxy
+  overwrites `X-Forwarded-For`; the rate limiter keys on it.
+
+## Metrics and the dashboard
+
+`/metrics` is Prometheus text, on by default. Besides the Go runtime and
+process collectors:
+
+| Metric | |
+|---|---|
+| `warp_http_requests_total{route,method,status}` | Requests by route pattern (never by path, so room and user IDs cannot explode the cardinality). |
+| `warp_http_request_duration_seconds{route,method}` | Latency histogram. Sync endpoints block by design, so leave them out of latency quantiles. |
+| `warp_http_requests_in_flight` | |
+| `warp_events_persisted_total{origin,kind}` | Room events persisted: `origin` is `local` or `federation`, `kind` is `state` or `message`. |
+| `warp_sync_waiting` | Syncs blocked in a long poll, classic and sliding: roughly the connected clients. |
+| `warp_sync_sliding_connections` | Sliding sync connections whose state the server holds. |
+| `warp_federation_enabled` | 1 while federation is on. |
+| `go_sql_*{db_name}` | The connection pool: open, in use, waits. |
+| `warp_build_info{version,commit}` | |
+
+[`deploy/grafana/warp.json`](../deploy/grafana/warp.json) is a Grafana
+dashboard over these: overview stats, requests and latency by route,
+events, sync connections, the database pool and the Go runtime. Import it
+and pick the Prometheus data source; the `job` and `instance` variables
+select the server.
+
+## Tracing
+
+With `tracing.enabled`, Warp exports OpenTelemetry traces over OTLP/HTTP to
+`tracing.endpoint` (or to `OTEL_EXPORTER_OTLP_ENDPOINT`, or to
+`localhost:4318`): an OpenTelemetry Collector, Jaeger, Tempo, Honeycomb.
+
+- Every request is a server span named after its route pattern, with the
+  method, path, user agent and status.
+- Persisting an event, building a sync or a sliding sync response, and
+  running a search are spans of their own inside it.
+- Every request Warp sends — to other servers, to push gateways, to
+  application services — is a client span, and carries the W3C
+  `traceparent` on, so a federation request shows up in the same trace as
+  the client request that caused it.
+- A request that arrives with a `traceparent` continues that trace.
+
+`tracing.sample_ratio` keeps that fraction of new traces; a request inside
+a sampled trace is always kept. Off, tracing costs a function call per
+span, and the trace context of incoming requests is still passed on.
+
+## Load testing
+
+`warp-loadtest` drives a server the way a crowd of clients would:
+
+```sh
+make loadtest ARGS="-users 50 -rooms 10 -rate 0.5 -duration 2m"
+# or
+go run ./server/cmd/warp-loadtest -url http://localhost:8008 -users 50 -rooms 10 -rate 0.5 -duration 2m -sliding
+```
+
+It registers the users (answering the dummy and registration-token
+stages), creates the rooms and joins everybody to all of them, then for the
+length of the run each user sends at `-rate` messages per second while
+syncing in a long-poll loop — `/sync`, or sliding sync with `-sliding`.
+Each message carries the time it was sent, so the report has, beside the
+latency of every kind of request, the **delivery** time: from one user
+sending to another user's sync handing it over, which is the number a user
+of a chat server feels.
+
+Point it at a test server with registration open and rate limiting off
+(`WARP_RATE_LIMIT_ENABLED=false`), or it measures the rate limiter. It
+creates accounts and rooms and never cleans them up.
+
+A run on a laptop, PocketBase driver, 20 users in 4 rooms at one message a
+second each (`-users 20 -rooms 4 -rate 1 -duration 15s`):
+
+| operation | count | p50 | p99 |
+|---|---|---|---|
+| send | 280 | 24 ms | 52 ms |
+| delivery (`/sync`) | 5320 | 147 ms | 280 ms |
+| delivery (sliding sync) | 5320 | 205 ms | 368 ms |
+
+Every message reached every other user. Those numbers are one writer
+serving twenty clients that each re-sync on every message; PostgreSQL's
+parallel reads are what moves them.

docs/development.md

@@ -47,7 +47,8 @@ WARP_SERVER_NAME=dev.local WARP_PRIVATE_KEY=./keys/dev.key ./bin/warp serve
 1. Write the same change twice, once per SQL dialect:
    `server/internal/storage/sqlite/migrations/NNNN_name.sql` and
    `server/internal/storage/postgres/migrations/NNNN_name.sql`.
-   Use the next free number; `NNNN` is zero-padded to four digits.
+   Use the next free number; `NNNN` is zero-padded to four digits. The
+   pocketbase driver uses the SQLite files as they are.
 2. The files are embedded with `embed.FS`, so a rebuild picks them up.
 3. `./bin/warp migrate --status` lists what would run, `./bin/warp migrate`
    applies it. A server started with `database.auto_migrate: true` (the
@@ -124,7 +125,9 @@ says why.
   smaller set that goes through the whole wrapped handler, to catch a route
   that was written but never registered.
 - Storage tests run against every driver that is available:
-  SQLite always, PostgreSQL when `WARP_TEST_POSTGRES_DSN` is set. The
+  SQLite and PocketBase always, PostgreSQL when `WARP_TEST_POSTGRES_DSN` is
+  set. PocketBase runs the SQLite migrations on PocketBase's own database,
+  so a SQLite migration is a PocketBase one too. The
   PostgreSQL tests share one database, so each group of them truncates the
   tables it owns first; add a table to `resetRoomTables` or
   `resetSyncTables` when you add one, or the second test to insert the same

docs/media.md

@@ -42,9 +42,13 @@ refers to content until it is all there.
 
 The blob store is chosen by the storage driver. SQLite and PostgreSQL use
 the local filesystem under `media.base_path`. A driver that brings its own
-file storage implements `storage.BlobProvider`, and PocketBase will — its
-filesystem abstraction is local disk or S3 (PLAN.md section 4) — when that
-driver is built in Phase 10.
+file storage implements `storage.BlobProvider`, and PocketBase does: media
+goes into PocketBase's filesystem, under `warp_media/` in
+`<data directory>/storage`, or in an S3-compatible bucket when
+`database.pocketbase.s3` is on ([configuration.md](configuration.md)).
+PocketBase's filesystem is written a whole file at a time, so an upload is
+spooled to `<data directory>/.warp_tmp` and hashed on the way, then handed
+over under its hash; `media.base_path` is not used by that driver.
 
 ## Endpoints
 
@@ -235,5 +239,5 @@ with a message saying federation is disabled.
   `413 M_TOO_LARGE`. See [admin-api.md](admin-api.md).
 - **No quarantine**, which belongs with the same admin work.
 - **No animated thumbnails**; `animated=true` gets a still image.
-- **No S3** until the PocketBase driver brings its filesystem abstraction;
-  the `BlobProvider` seam is where it plugs in.
+- **S3 only on PocketBase.** The sqlite and postgres drivers keep media on
+  local disk; S3 comes with the pocketbase driver's filesystem.

docs/push.md

@@ -135,7 +135,7 @@ write to the rules, which is what makes the cache safe in one process.
 
 | Table (migration `0010_push.sql`) | What |
 |---|---|
-| `push_notifications` | one row per event that notified a user: the event, its position in the room event stream, the actions, highlight, read |
+| `push_notifications` | one row per event that notified a user: the event, its position in the room event stream, its thread (`0013_thread_notifications.sql`), the actions, highlight, read |
 | `push_room_counts` | each user's unread and highlight counts in each room, kept current by the writes, stamped with a notification stream position |
 | `push_pushers` | pushers |
 | `push_sequences` | the notification stream's counter |
@@ -165,10 +165,46 @@ then recounted from the rows that are still unread rather than decremented,
 so if anything ever put them out of step, the next receipt puts them right.
 `/read_markers` goes through the same path.
 
-Notifications are not tracked per thread (threads are Phase 10). So a
-receipt counts when it is for the whole room or for its `main` timeline. A
-receipt inside a thread changes nothing; thread notifications are cleared
-when the main receipt moves past them.
+Each notification is filed under the thread its event is in: the root
+that its `m.relates_to` names when `rel_type` is `m.thread`, or the main
+timeline for anything else (a thread root itself included). A receipt
+reads the part of the room its `thread_id` names:
+
+| `thread_id` | Reads, up to the receipt's event |
+|---|---|
+| none | the whole room, every thread included |
+| `main` | the main timeline only |
+| a thread root's event ID | that thread only |
+
+So reading one thread leaves the main timeline and the other threads
+unread, and a client that knows nothing of threads sends unthreaded
+receipts and clears the room as it always did. `/read_markers` sends
+unthreaded receipts. A local user leaving or being banned reads the whole
+room.
+
+### Per-thread counts
+
+`push_room_counts` stays the room's total across every thread, which is
+what `unread_notifications` reports by default, what sliding sync reports,
+and what a push's badge counts. A sync whose filter sets
+`room.timeline.unread_thread_notifications` gets the counts split instead:
+`unread_notifications` counts the main timeline only, and
+`unread_thread_notifications` maps each thread root with something unread
+to its `notification_count` and `highlight_count`. A thread with nothing
+unread is left out of the map, and a room with no unread threads has no
+map, both of which the spec allows.
+
+The per-thread counts are counted from the unread rows of
+`push_notifications` when a sync asks, rather than kept in a table like
+the room totals: a room has few unread notifications at a time, they are a
+range of an index, and a second table of counts would be one more thing
+every write and receipt had to keep in step. They need no stream of their
+own either: any change to a thread's counts is a change to its room's,
+which stamps the room and brings it down the next sync.
+
+Notifications stored before migration 0013 are filed by the relation the
+roomserver projected onto their events in 0012; one whose event is gone
+stays on the main timeline.
 
 Read notifications are kept for `push.notification_retention` (30 days by
 default) for `GET /notifications`, then pruned hourly. Unread ones are kept
@@ -227,7 +263,7 @@ to again.
 
 | Thing | Why |
 |---|---|
-| `unread_thread_notifications` | needs threads (Phase 10) |
+| Per-thread counts in sliding sync and in pushes | sliding sync reports each room's total, and a push's badge is the user's total across rooms; neither has anywhere the spec puts a per-thread count |
 | Skipping senders in `m.ignored_user_list` | ignored users' events still notify; a client hides them, but they count |
 | Email pushers | needs a mail server; see above |
 | Durable evaluation | the emit stage is in memory, so events persisted just before a crash are never evaluated. Their messages are still there; only the count and the push are missed. |

docs/relations.md

@@ -0,0 +1,97 @@
+# Relations and threads
+
+An event can point at another through `content.m.relates_to`: a thread
+reply (`m.thread`), an edit (`m.replace`), a reaction (`m.annotation`), a
+reference (`m.reference`). None of that is checked by the authorisation
+rules. A relation is a client convention, not part of the room's DAG, and a
+server can accept an event without understanding what it relates to. What
+the server adds is three things:
+
+1. **The index.** Every event's `m.relates_to` is written out with it, so
+   that "the children of this event" is an index lookup.
+2. **The listings.** `/relations` lists an event's children and `/threads`
+   lists a room's threads.
+3. **The summaries.** Wherever an event is served, its children are
+   summarised into `unsigned.m.relations`, so a client drawing a timeline
+   knows a message has a thread under it, or has been edited, without
+   asking about every event.
+
+The code is in `internal/roomserver/relations.go` (the reads and the
+summaries) and `internal/syncapi/relations.go` (the tokens), over the
+relation reads of the room store in `internal/storage/relations.go`.
+
+## The index
+
+`roomserver_events` has two more columns, `relates_to_id` and `rel_type`,
+taken from `content.m.relates_to.event_id` and `.rel_type` when both are
+strings. Migration 0012 fills them in for the events already stored. An
+`m.in_reply_to` on its own is not a relation in this sense: a reply points
+at an event without being one of its children.
+
+A redaction clears both columns. The redaction algorithm strips the
+content they came from, so a redacted thread reply leaves its thread, and
+a redacted edit stops being one.
+
+## `GET /_matrix/client/v1/rooms/{roomID}/relations/{eventID}[/{relType}[/{eventType}]]`
+
+The children of one event, newest first by default.
+
+| Parameter | |
+|---|---|
+| `relType`, `eventType` | Narrow by relation type, then by event type. |
+| `dir` | `b` (default) or `f`. |
+| `from`, `to` | Pagination tokens. They are `/messages` tokens: a sync's `prev_batch` works as a `from`. |
+| `limit` | 50 by default, 1000 at most. |
+| `recurse` | Include the children's children, down to three levels. The response then carries `recursion_depth`: how deep it found anything. The filters apply at every level. |
+
+The parent has to be visible to the user, and the children are bounded to
+what they may see, by the same visible range as a timeline read
+([rooms.md](rooms.md)). A parent the user may not see is `404`, not `403`:
+whether a particular event exists in a room is itself not theirs to know.
+
+## `GET /_matrix/client/v1/rooms/{roomID}/threads`
+
+The room's threads, the most recently active first. A thread is a root
+that has at least one `m.thread` child; its activity is its newest child.
+`include=participated` keeps the threads the user started or replied in.
+Each root carries its `m.thread` summary. `from` takes the `next_batch` of
+the previous page, a token of its own (`t<position>`), because threads are
+ordered by activity rather than by any one stream.
+
+A thread whose root this server does not hold — sent before this server
+joined the room and never backfilled — is not listed: there is no event to
+show it under.
+
+## Unread counts and receipts
+
+Each notification is filed under its thread, a threaded read receipt reads
+only that thread (`thread_id: "main"` only the main timeline), and a sync
+whose filter sets `unread_thread_notifications` gets the counts per thread.
+How, and what stays a room total, is in [push.md](push.md).
+
+## Bundled aggregations
+
+Every event a timeline read returns — `/sync`, sliding sync, `/messages`,
+`/context`, `GET /event`, `/relations` and `/threads` — gets
+`unsigned.m.relations` when it has children:
+
+| Key | What |
+|---|---|
+| `m.thread` | `latest_event` (the newest reply, in full), `count`, and `current_user_participated` — true for a user who sent the root or any reply. |
+| `m.replace` | The newest edit, in full. Only an edit from the original's sender counts, and only one of the same event type; a state event cannot be edited, and a redacted event's edits are not shown. |
+| `m.reference` | `chunk`: the IDs of the events referencing it, oldest first, at most 50. |
+
+Annotations (reactions) are not summarised. The spec dropped their
+aggregation, and every client counts reactions from the events themselves.
+
+The summaries are computed when the event is read, by one grouped query
+over the batch of events being returned, bounded to what the user may
+see: a user who has left the room sees a thread's count and latest reply
+as of when they left.
+
+## What is deliberately missing
+
+| Thing | Why |
+|---|---|
+| Rejecting a thread whose root is itself a relation | The spec says a server *should*; Warp accepts it, and clients render it as an ordinary reply. |
+| The relation filters of a room event filter (`related_by_rel_types`, `related_by_senders`) | Ignored, like every filter field Warp does not act on; the client gets a superset. |

docs/search.md

@@ -0,0 +1,79 @@
+# Search
+
+`POST /_matrix/client/v3/search` searches the rooms a user is in, or was
+in, for words in messages, room names and topics.
+
+The roomserver decides where a user may look (`internal/roomserver/search.go`),
+the store runs the query against a full-text index
+(`internal/storage/sqlstore/relations.go`), and the sync API shapes the
+response (`internal/syncapi/search.go`).
+
+## The index
+
+Each storage engine uses its own full-text search, and the drivers hand the
+shared store the statements for it:
+
+| Driver | Index | Ranking |
+|---|---|---|
+| sqlite, pocketbase | an FTS5 table, tokenised with `unicode61 remove_diacritics 2`, so `cafe` finds `café` | `bm25` |
+| postgres | a `tsvector` column with the `simple` configuration and a GIN index | `ts_rank` |
+
+`simple` lower-cases and splits words without stemming for any one
+language, which suits a server whose rooms are in every language at once.
+It keeps diacritics, so on PostgreSQL `cafe` does not find `café`.
+
+An event joins the index when it joins a timeline: `content.body` of an
+`m.room.message`, `content.name` of `m.room.name`, `content.topic` of
+`m.room.topic`. Outliers are not indexed; an outlier promoted into the
+timeline is, unless it had already been redacted. A redaction removes the
+event from the index, and purging a room removes the room's rows.
+Migration 0012 indexes the events already stored.
+
+An encrypted message has no body the server can read and is never found.
+That is what end-to-end encryption means; clients search encrypted rooms
+themselves.
+
+## What a user may search
+
+Every room the user is joined to, or has left or been banned from, each
+bounded by the same visible range as a timeline read
+([rooms.md](rooms.md)). A user who left a room finds what was said while
+they were in it and nothing since; a user who joined a room with `joined`
+history visibility finds nothing from before they came. The bounds go into
+the query itself, one clause per room, so a page is never short because
+results it could not show were filtered out afterwards.
+
+## The query
+
+What the user typed is reduced to words — runs of letters and digits —
+and each is matched as a prefix, all of them required. Both engines'
+query languages have operators (FTS5's quotes, `NEAR` and column filters;
+tsquery's `&`, `|`, `!`), and a search box is not the place to expose
+them: a user searching for `C++ & Go` means the words. A term with no words
+in it finds nothing.
+
+## Request and response
+
+| Request field | |
+|---|---|
+| `search_term` | Required. |
+| `keys` | Any of `content.body`, `content.name`, `content.topic`; all three by default. |
+| `filter` | A room event filter: `rooms`, `not_rooms`, `types`, `not_types`, `senders`, `not_senders`, and `limit` (10 by default, 100 at most). |
+| `order_by` | `rank` (default) or `recent`. |
+| `event_context` | `before_limit` and `after_limit` (5 each by default, 20 at most), and `include_profile` for the senders' names and avatars. |
+| `include_state` | The current state of every room with a result. |
+| `groupings.group_by` | `room_id` and/or `sender`. |
+
+The response carries `count` (every match, not just this page),
+`highlights` (the words searched for), the `results` with their `rank`, and
+`next_batch` when there is more. A ranked search pages by offset; a search
+by recency pages by stream position, which keeps a page from repeating a
+result when new messages arrive between two requests.
+
+## What is deliberately missing
+
+| Thing | Why |
+|---|---|
+| Ranking across engines | `rank` is only comparable within one search on one server. |
+| Highlighting stemmed forms | Highlights are the words as typed. |
+| Searching rooms by their contents over federation | Search covers what this server holds. |

docs/spaces.md

@@ -0,0 +1,85 @@
+# Spaces
+
+A space is a room whose create event says `"type": "m.space"`, and whose
+`m.space.child` state events name the rooms in it. As with relations, the
+authorisation rules know nothing of it: a child event is a pointer, and
+the room it points at need not know. What the server provides is the walk.
+
+The code is in `internal/roomserver/hierarchy.go`.
+
+## `GET /_matrix/client/v1/rooms/{roomID}/hierarchy`
+
+The space's tree, breadth first, a page at a time. Each room is described
+the way the public room directory describes it — name, topic, avatar,
+alias, member count, join rule, room type, room version and encryption —
+plus its `children_state`: the `m.space.child` events, each with its
+`origin_server_ts`.
+
+| Parameter | |
+|---|---|
+| `max_depth` | How far below the root to go; `0` is the root alone. Capped at 50. |
+| `suggested_only` | Follow only children marked `"suggested": true`. |
+| `limit` | Rooms per page: 50 by default, 100 at most. |
+| `from` | The previous page's `next_batch`. |
+
+Children are visited in the spec's order: those with a valid `order` (printable ASCII, at most 50 characters) by that string, then by when the child event was sent, then by room ID.
+A child event whose `via` is missing or empty is how a space removes a
+child — a state event cannot be deleted, only emptied — and is skipped.
+Only a space's children are walked into: an ordinary room that happens to
+hold an `m.space.child` event is not a space.
+
+Paging walks from the root again and skips what earlier pages returned.
+That keeps the server free of per-walk state, and a walk is capped at 2000
+rooms whatever the page size, which bounds the cost of the last page of a
+very large space.
+
+### Who sees what
+
+A room is shown to a user who:
+
+- is joined to it or invited to it; or
+- may know about it anyway: its join rule is `public`, `knock` or
+  `knock_restricted`, or its history is world-readable; or
+- is joined to one of the rooms a `restricted` room lets in.
+
+Everything else is left out, and so are its children. The root itself is
+the exception: a space the user may not see is `403`.
+
+### Rooms this server is not in
+
+A child this server is not in is asked about over federation, of the
+servers its `via` names, with `GET /_matrix/federation/v1/hierarchy`. The
+answer is remembered for the rest of the walk. The other server has
+already judged the room for this server as a whole; Warp then checks a
+restricted room's allow list against its own user. A child that no server
+will describe is left out. With federation off, rooms this server is not
+in are simply not shown.
+
+## `GET /_matrix/federation/v1/hierarchy/{roomID}`
+
+Another server walking one of Warp's spaces: the room and the children
+Warp is in, each judged for the asking server. A room is shown to a server
+when anybody could be shown it (see above), when it is restricted — with
+`allowed_room_ids`, so the asking server can check its own users — or
+when a user of that server is joined or invited. Children it may not see
+are listed in `inaccessible_children`. A root it may not see is `404`,
+which is also the answer for a room Warp is not in.
+
+## `GET /_matrix/client/v1/room_summary/{roomIDOrAlias}`
+
+One room's face, for a link preview or an invite: the same fields as a
+hierarchy room without its children, plus the user's `membership`
+(`leave` for a user who was never in it). An alias is resolved first, and
+the servers it names are added to `?via=`. The same visibility rules apply;
+a room the user may not see is `403`.
+
+The endpoint is in the spec from v1.15. Warp still advertises v1.14,
+because v1.15 also brought requirements Warp does not meet yet.
+
+## What is deliberately missing
+
+| Thing | Why |
+|---|---|
+| Unauthenticated `room_summary` | The spec allows it for guests; Warp asks for a token. |
+| Caching other servers' hierarchy answers between walks | Each walk asks again; a space with many remote children costs a federation request per child per page. |
+| `m.space.parent` checks | Parent events are state like any other, and are not used to validate children. |

docs/sync.md

@@ -256,6 +256,14 @@ So the test is whether the response has anything in it, with an initial
 sync as the exception: a client that has just logged in needs an answer
 whatever is in it.
 
+"Initial" means the client sent no `?since=`, not that its token is all
+zeroes. On a server whose streams have not moved yet, `s0` is every
+client's token, and taking it for a first sync made every long poll
+return at once. The push rules a first sync carries are sent on the
+client's first sync for the same reason, not on any sync whose account
+data position is zero: before anybody had written account data, that was
+every sync, and each one counted as news.
+
 The same trap caught presence. A client says it is still online by syncing,
 so `?set_presence=` makes every sync a presence write; a stream that moved
 each time would wake every peer's long poll, which would sync, which would
@@ -280,11 +288,12 @@ What Warp acts on:
   tuples, not a window of a stream, so there is no index to push it to.
 - `room.rooms` / `not_rooms`, at both levels.
 - `room.timeline.lazy_load_members`.
+- `room.timeline.unread_thread_notifications`: the unread counts split by
+  thread; see [push.md](push.md).
 - `room.include_leave`.
 - `presence` and `account_data` types and senders.
 
-What it parses and does not act on: `event_fields`, `contains_url`,
-`unread_thread_notifications`. A field Warp ignores is **not** an error — a
+What it parses and does not act on: `event_fields`, `contains_url`. A field Warp ignores is **not** an error — a
 server is allowed to return a superset of what a filter asked for, and
 refusing the filter is the one outcome that would stop a client syncing at
 all. `event_format: federation` *is* refused: that is a server's view of an
@@ -400,16 +409,120 @@ its own response type for that reason. Reporting it as a room event with
 empty fields would be worse: a client would see an event with no sender and
 no timestamp and have to decide what that meant.
 
+## Simplified Sliding Sync
+
+`POST /_matrix/client/unstable/org.matrix.simplified_msc3575/sync` is
+MSC4186, which Element X and every client on the matrix-rust-sdk use when
+`/versions` advertises `org.matrix.simplified_msc3575` — as Warp's does.
+
+A classic initial sync sends every room before a client can draw
+anything, which for a large account takes minutes. Sliding sync turns that
+round: the client asks for *lists* — "my rooms, most recently active
+first, positions 0 to 19" — and for the few pieces of state it needs to
+draw each one, and is sent those rooms and a count of the rest. As the
+user scrolls the client widens the window; as rooms become active they
+move into it and are sent.
+
+The code is `internal/syncapi/sliding.go` and `slidingconn.go`, over a few
+roomserver reads in `internal/roomserver/sliding.go`.
+
+### Connections
+
+What a client has been sent depends on what it asked for, so a token
+cannot just be a position in each stream. The server remembers, per
+connection — one user, one device, one `conn_id` — which rooms it has sent
+and up to which position, and `pos` names one of those memories. A room is
+sent once in full (`"initial": true`) and then only when it changes; a room
+that leaves the window and comes back is not sent again unless something
+happened to it.
+
+The memory is in the process and is lost on restart. An unknown `pos` —
+expired after 30 minutes idle, from before a restart, or one of the
+connection's older positions beyond the last three — is answered with
+`400 M_UNKNOWN_POS`, and the client starts a new connection. That is how
+the protocol is meant to recover. A request without `pos` starts its
+connection over. A device holds at most ten connections; the oldest goes
+first.
+
+Request parameters are not sticky: every request says what it wants.
+
+### Lists
+
+A list holds the rooms the user is joined to, invited to or knocking on,
+plus rooms they left or were banned from while the connection was open,
+which is how a client hears that a room it is showing is gone. Rooms are
+sorted by `bump_stamp`, the position of the last message, encrypted
+message, sticker, call invite, poll or live location in the room (not
+membership changes, reactions or receipts, which would keep a busy room on
+top for reasons nobody could see). An invite, a knock or a room the user
+has left is bumped by their own membership event, because they cannot see
+what happens in it.
+
+| Filter | |
+|---|---|
+| `is_dm` | rooms in the user's `m.direct` account data |
+| `is_invite` | |
+| `is_encrypted` | an `m.room.encryption` in the room's state |
+| `room_types`, `not_room_types` | the create event's `type`; `null` is an ordinary room |
+| `spaces` | children of the named spaces this server is in |
+| `tags`, `not_tags` | the room's `m.tag` account data |
+
+`ranges` are inclusive windows into the sorted list. The response gives
+each list's `count`, and the rooms in any window, merged with the room
+subscriptions.
+
+### Rooms
+
+`room_subscriptions` names rooms directly, whatever list they are or are
+not in: the room a client has open. A room in several lists and
+subscriptions is sent with the largest `timeline_limit` and the union of
+the `required_state`.
+
+`required_state` is a list of `[type, state_key]`. `*` is a wildcard in
+either place, `$ME` is the user, and `["m.room.member", "$LAZY"]` is the
+members who sent something in the timeline, plus the user. Only those
+tuples are read: the room's state is read as a set of keys first and the
+event bodies after, so a room with ten thousand members costs one
+member event, not ten thousand. On an incremental response it is the
+tuples that changed since the room was last sent.
+
+Each room carries `name` and `avatar`, `heroes` (with names and avatars)
+when it has no name, `is_dm`, `joined_count`, `invited_count`, the unread
+counts, `bump_stamp`, and the `timeline` with `prev_batch` (a `/messages`
+token), `limited` and `num_live` (how many of the timeline's events are
+new since the previous response). An invite is its `invite_state`, a knock
+its `knock_state`, and neither has a timeline.
+
+### Extensions
+
+| Extension | What |
+|---|---|
+| `to_device` | The device's queued messages, 100 at a time by default. It has a `since` token of its own; coming back with it acknowledges the batch. |
+| `e2ee` | `device_lists`, `device_one_time_keys_count`, `device_unused_fallback_key_types`, worked out as for `/sync`. |
+| `account_data` | Global, and per room for the rooms in the response's lists and subscriptions — all of it for a room sent in full. |
+| `receipts` | Per room, the receipts since the last response; all of them for a room sent in full. |
+| `typing` | Per room, the typing list when it changed — including to empty, so a client stops showing somebody typing. |
+
+`lists` and `rooms` narrow the rooms a per-room extension covers. Naming
+any list the request has is every room in the windows: the windows are
+merged before the extensions run.
+
+### Long polling
+
+A response is returned at once on a new connection; otherwise only when a
+room is sent, a list's count moved, or an extension has something to say.
+Until then it waits on the same notifier as `/sync`.
+
 ## What is deliberately missing
 
-| Thing | Needs | Phase |
-|---|---|---|
-| `unread_thread_notifications` | threads | 10 |
-| Remote presence | presence over federation | — |
-| Threads, and threaded receipts beyond storing `thread_id` | threads | 10 |
-| `contains_url`, `event_fields` | nothing; they are parsed and ignored | — |
-| Per-event history visibility | state storage (open question 1) | — |
-| Simplified Sliding Sync (MSC4186) | — | 10 |
+| Thing | Needs |
+|---|---|
+| Remote presence | presence over federation |
+| `contains_url`, `event_fields` | nothing; they are parsed and ignored |
+| Per-event history visibility | state storage (open question 1) |
+
+Threads, relations and the summaries every timeline event carries in
+`unsigned.m.relations` are in [relations.md](relations.md).
 
 `device_lists` is sent as two empty lists rather than left out: a client
 reads a missing one as a malformed response, where two empty lists say
@@ -417,7 +530,9 @@ reads a missing one as a malformed response, where two empty lists say
 that ride along with them, are in [e2ee.md](e2ee.md).
 
 Each joined room carries `unread_notifications` from the push server,
-whose counts are the seventh stream, and the global account data carries
+whose counts are the seventh stream — the main timeline's alone, beside an
+`unread_thread_notifications` map of each thread's, when the filter sets
+`unread_thread_notifications` — and the global account data carries
 `m.push_rules`, rendered over the server defaults for a user who never
 changed a rule. Both are described in [push.md](push.md). A build with no
 push server leaves the counts out entirely rather than sending zeroes,
@@ -429,6 +544,9 @@ is not.
 | Endpoint | Notes |
 |---|---|
 | `GET /sync` | Long-polls. `?since=`, `?filter=`, `?timeout=`, `?full_state=`, `?set_presence=`. |
+| `POST /_matrix/client/unstable/org.matrix.simplified_msc3575/sync` | Simplified Sliding Sync; see below. `?pos=`, `?timeout=`, `?set_presence=`. |
+| `GET /_matrix/client/v1/rooms/{id}/relations/{eventId}` · `GET /_matrix/client/v1/rooms/{id}/threads` | See [relations.md](relations.md). |
+| `POST /search` | See [search.md](search.md). |
 | `GET /rooms/{id}/messages` | `?dir=` (`b` default), `?from=`, `?to=`, `?limit=`, `?filter=`. No `end` at the end of the room. |
 | `GET /rooms/{id}/context/{eventId}` | `?limit=`, `?filter=`. Half the budget each side, the odd one backwards. |
 | `POST /user/{userId}/filter` | Returns an opaque ID. |