Group voice and video calls with MatrixRTC and LiveKit

Every call in the web client now goes through MatrixRTC with a LiveKit
SFU, the model Element Call and Element X use; the legacy 1:1 m.call.*
calls are gone. In a room of two the other person is rung and can answer
or decline; in a bigger room a banner offers to join the call. Calls show
a grid of participant tiles with mute, camera, screen sharing and leave,
and media is end-to-end encrypted in encrypted rooms.

Server:
- MSC4140 delayed events (send/state with a delay, plus the
  delayed_events list/restart/cancel/send API), stored and scheduled, so
  a member whose client drops off is taken out of the call.
- POST /user/{userId}/openid/request_token, which lk-jwt-service uses to
  hand out LiveKit tokens.
- org.matrix.msc4143.rtc_foci in /.well-known/matrix/client, configured
  under client_api.rtc.
- Room presets let members send call membership state.
- Migration 0014 for both SQL dialects.

Deploy: LiveKit and lk-jwt-service in compose (calls.yml) and the helm
chart. Docs: docs/calls.md, configuration and deployment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/README.md

@@ -13,6 +13,7 @@
 | [federation.md](federation.md) | the Server-Server API: discovery, keys, taking in events, joins across servers, the outbound queue, the switch |
 | [media.md](media.md) | `internal/mediaapi`: uploads, downloads, thumbnails, other servers' media, where the files go |
 | [e2ee.md](e2ee.md) | `internal/keyserver`: device and one-time keys, cross-signing, device list tracking, key backup |
+| [calls.md](calls.md) | voice and video calls with MatrixRTC and LiveKit: ringing, delayed events, OpenID, the JWT service, deploying it |
 | [push.md](push.md) | `internal/pushrules` and `internal/pushserver`: push rules, notification counts, pushers, the push gateway client |
 | [appservices.md](appservices.md) | `internal/appservice`: registration files, namespaces, acting as users, delivering events to bridges |
 | [admin-api.md](admin-api.md) | `/_warp/admin/v1`: the operator API behind the admin panel |

docs/calls.md

@@ -0,0 +1,171 @@
+# Voice and video calls
+
+Every call in Warp's web client, between two people or twenty, is a
+MatrixRTC call (MSC4143) with its media through a LiveKit SFU: the Element
+Call model, which Element X uses too. A call of two is a call with two
+members, plus a ring. The server needs a LiveKit SFU and the LiveKit JWT
+service (`client_api.rtc`); without them there are no calls.
+
+The old Matrix 1:1 VoIP (`m.call.*` events, WebRTC between the two
+browsers) is not used: the web client starts the SDK with `disableVoip`, so
+it neither places nor answers such calls, and shows them only as history in
+the timeline. The server still serves `GET /_matrix/client/v3/voip/turnServer`
+(`client_api.turn`) for other clients that do.
+
+## How a call works
+
+Who is in a room's call is room state: every device in it has an
+`org.matrix.msc3401.call.member` state event (state key
+`_@user:server_DEVICE_m.call`), with the LiveKit service it publishes to.
+The media goes through a LiveKit SFU, which Warp does not run itself.
+
+```
+browser ── GET /.well-known/matrix/client ──────────▶ warp   org.matrix.msc4143.rtc_foci
+        ── GET …/org.matrix.msc4143/rtc/transports ─▶ warp   (the same list, authenticated)
+        ── PUT …/state/org.matrix.msc3401.call.member/<key>
+             ?org.matrix.msc4140.delay=…  (the delayed "left")
+             and then without a delay  (the "joined") ──▶ warp
+        ── POST /user/<me>/openid/request_token ────▶ warp   an OpenID token
+        ── POST <livekit_service_url>/sfu/get ──────▶ lk-jwt-service
+             lk-jwt-service ── GET /_matrix/federation/v1/openid/userinfo ──▶ warp
+        ◀── {url, jwt}
+        ── WebSocket + UDP ─────────────────────────▶ LiveKit
+```
+
+### What the server does
+
+- **Discovery.** With `client_api.rtc.livekit_service_url` set, Warp adds
+  `org.matrix.msc4143.rtc_foci: [{"type": "livekit", "livekit_service_url": …}]`
+  to `/.well-known/matrix/client` and serves the same list at
+  `GET /_matrix/client/unstable/org.matrix.msc4143/rtc/transports`.
+- **Delayed events (MSC4140)**, advertised as `org.matrix.msc4140` in
+  `/versions`. `PUT /rooms/{roomId}/state/{type}/{key}` and
+  `PUT /rooms/{roomId}/send/{type}/{txnId}` take
+  `?org.matrix.msc4140.delay=<ms>`: the event is stored instead of sent,
+  and the answer is `{"delay_id": …}`. Under
+  `/_matrix/client/unstable/org.matrix.msc4140`:
+  - `GET /delayed_events` lists your events as `scheduled` and `finalised`
+    (`?status=` and `?delay_id=` narrow it); `GET /delayed_events/{id}`
+    returns one.
+  - `POST /delayed_events/{id}/{restart|cancel|send}`, or the older
+    `POST /delayed_events/{id}` with `{"action": …}`. These need no access
+    token: the delay ID is the capability.
+
+  A scheduler goroutine sleeps until the next event is due and sends it
+  as the user who scheduled it, through the same pipeline as any event, so
+  power levels and membership still apply; a send that fails is finalised
+  with the error the client would have got. Someone *else* setting the
+  same piece of state cancels a delayed state event, so a stale "left"
+  can't overwrite a newer membership; the owner's own updates don't. Events
+  live in the database (`delayed_events`, migration 14) and survive a
+  restart. Finalised events are kept for a day. `client_api.delayed_events`
+  caps the delay (24h) and how many a user may have (100).
+
+  This is what makes a call robust: a joining member schedules its own
+  empty `call.member` a few seconds out and keeps restarting it while it
+  is in the call. A browser that crashes or loses its network stops
+  restarting it, and Warp removes the member.
+- **OpenID.** `POST /_matrix/client/v3/user/{userId}/openid/request_token`
+  gives a user a one-hour token, which `GET /_matrix/federation/v1/openid/userinfo?access_token=…`
+  turns back into `{"sub": "@user:server"}` for whoever it was handed to.
+  Tokens are stored hashed (`openid_tokens`). The userinfo endpoint is
+  served with federation switched off too: a server that does not
+  federate can still have calls.
+- **Power levels.** New rooms let anyone who may talk join the call:
+  `org.matrix.msc3401.call.member` is at level 0 in the `events` of the
+  default power levels. Rooms created before this keep `state_default`
+  (50) for it, so an ordinary member gets `M_FORBIDDEN` until a moderator
+  sets it to 0.
+
+### Ringing
+
+Starting a call in a room with one other person rings them. The caller's
+MatrixRTC session sends an `org.matrix.msc4075.rtc.notification`
+(`notification_type: "ring"`, a `lifetime` of a minute, `m.call.intent`
+audio or video) when its membership is the first; in a bigger room it
+sends `notification`, which the web client doesn't ring for, since the
+"Join call" banner says enough. The other side shows an incoming-call card
+for a ring that is someone else's, mentions them or the room, and is
+within its lifetime, until they answer (join the call), decline (an
+`org.matrix.msc4310.rtc.decline` referencing the notification, which also
+stops the ring on their other devices), the caller leaves, or it expires.
+The caller hears a ringback until someone joins, and gives up on a decline
+or after the minute. In an encrypted room both events are encrypted like
+any other.
+
+The SDK's own validator for received notifications expects MSC4354 sticky
+events and `m.rtc.slot` state, which Warp doesn't serve yet, so the web
+client applies the receiving rules itself.
+
+### What the web client does
+
+`src/stores/call.ts` uses the SDK's MatrixRTC session
+(`client.matrixRTC.getRoomSession(room)`), publishing the transport
+`{type: "livekit", livekit_service_url, livekit_alias: <room ID>}` (the one
+the call already uses, else this server's), gets a LiveKit token from
+`/sfu/get`, and connects with `livekit-client`. In an encrypted room it
+joins with `manageMediaKeys`: each member makes its own media key and
+sends it to the others over encrypted to-device messages, and a
+`BaseKeyProvider` hands each key to LiveKit's frame encryption for that
+participant (`@user:server:DEVICE`). The settings (no shared key, no
+ratcheting) match Element Call's, so the two can share a call.
+
+### Running it
+
+Locally, [`deploy/compose/calls.yml`](../deploy/compose/calls.yml) adds
+LiveKit and lk-jwt-service to the Compose setup:
+
+```sh
+cd deploy/compose
+WARP_REGISTRATION_ENABLED=true docker compose -f docker-compose.yml -f calls.yml --profile web up -d --build
+```
+
+Then sign in as different users at <http://localhost:8080>,
+<http://127.0.0.1:8080> and <http://[::1]:8080> (different origins keep
+separate sessions in one browser). A voice or video call in a room of two
+rings the other person; in a room of three or more the others see a "Join
+call" banner. `WARP_PORT` and `WARP_WEB_PORT` move the homeserver and web client
+off 8008 and 8080 (then set `WARP_PUBLIC_BASEURL` to match).
+
+The JWT service checks an OpenID token by asking the user's homeserver
+over its federation port with TLS. For the server name `localhost` that
+means `https://localhost:8448`, so the file gives Warp a federation
+listener with a self-signed certificate (`LIVEKIT_INSECURE_SKIP_VERIFY_TLS`
+accepts it) and runs LiveKit and the JWT service in Warp's network
+namespace, where `localhost` is Warp. One consequence: when the `warp`
+container is recreated on its own, the other two lose their network. Run
+`up` for the whole file, not for one service, or recreate them with
+`docker compose -f docker-compose.yml -f calls.yml up -d --force-recreate livekit livekit-jwt`.
+LiveKit advertises `127.0.0.1` for media; set `LIVEKIT_NODE_IP` to the
+machine's LAN address to call from other devices.
+
+In production, give each piece its own hostname and a real certificate:
+
+- **LiveKit**, reachable by browsers on TCP 7880 behind TLS (`wss://`),
+  and UDP (7882, or a port range) and TCP 7881 for media. See LiveKit's
+  own deployment guide or its Helm chart.
+- **lk-jwt-service** behind TLS, with `LIVEKIT_URL` (the `wss://` URL),
+  `LIVEKIT_KEY` / `LIVEKIT_SECRET` (LiveKit's API key) and
+  `LIVEKIT_FULL_ACCESS_HOMESERVERS` (your server name). The Helm chart
+  runs it with `calls.jwtService.enabled`.
+- **Warp** with `client_api.rtc.livekit_service_url` set to the JWT
+  service's URL. The JWT service must reach
+  `https://<server_name>/.well-known/matrix/server` (a delegation to the
+  host that serves Warp) or `<server_name>:8448`. Note that Warp serves
+  `/.well-known/matrix/server` only with federation on; with it off,
+  serve the delegation file from the server name's web server, as you
+  would for `/.well-known/matrix/client`.
+- **warp-web**: `WARP_WEB_CALL_ORIGINS="https://livekit-jwt.example.org wss://livekit.example.org"`
+  lets the browser reach both through its Content-Security-Policy.
+
+### Known gaps
+
+- Calls only use the legacy `org.matrix.msc3401.call.member` state events,
+  not the sticky `m.rtc.member` events of newer MatrixRTC drafts.
+- `org.matrix.msc4140.parent_delay_id` (chained delayed events) is
+  refused, and a delayed send's transaction ID is not deduplicated.
+- No ring for calls in rooms of more than two (only the banner), no
+  MSC4354 sticky events, and no MSC4195 homeserver-side LiveKit token
+  endpoint (`rtc/livekit/get_token`).
+- Calls from clients that only do legacy 1:1 VoIP (`m.call.*`) don't ring
+  in the web client.

docs/configuration.md

@@ -62,10 +62,13 @@ List-valued variables are comma-separated. A single `-` clears a list:
 | `client_api.rate_limit.join.burst` / `.interval` | — | `10` / `1s` | The two join endpoints. |
 | `client_api.rate_limit.sync.burst` / `.interval` | — | `100` / `20ms` | `/sync` and the writes behind it: typing, receipts, to-device, presence. |
 | `client_api.rate_limit.media.burst` / `.interval` | — | `100` / `20ms` | Media downloads and thumbnails: a client opening a room asks for every avatar at once. Uploads are in the default bucket. |
-| `client_api.turn.uris` | `WARP_TURN_URIS` | `[]` | `turn:`, `turns:` and `stun:` URIs for calls. Empty = no relay; `/voip/turnServer` answers `{}`. |
+| `client_api.turn.uris` | `WARP_TURN_URIS` | `[]` | `turn:`, `turns:` and `stun:` URIs for other clients' legacy 1:1 VoIP calls (the web client calls through LiveKit and doesn't use them). Empty = no relay; `/voip/turnServer` answers `{}`. |
 | `client_api.turn.shared_secret` | `WARP_TURN_SHARED_SECRET` | — | coturn's `static-auth-secret`: each user gets expiring TURN REST API credentials. |
 | `client_api.turn.user_lifetime` | `WARP_TURN_USER_LIFETIME` | `1h` | How long shared-secret credentials last. |
 | `client_api.turn.username` / `.password` | `WARP_TURN_USERNAME` / `WARP_TURN_PASSWORD` | — | Static credentials, used only without a shared secret. |
+| `client_api.rtc.livekit_service_url` | `WARP_RTC_LIVEKIT_SERVICE_URL` | — | Base URL of the LiveKit JWT service (lk-jwt-service) for calls, advertised in `/.well-known/matrix/client` as `org.matrix.msc4143.rtc_foci`. Empty = no calls. See [calls.md](calls.md). |
+| `client_api.delayed_events.max_delay` | `WARP_DELAYED_EVENTS_MAX_DELAY` | `24h` | Longest delay a client may give a delayed event (MSC4140). `0` turns delayed events off and stops advertising `org.matrix.msc4140`; calls then can't remove a member whose client vanished. |
+| `client_api.delayed_events.max_per_user` | `WARP_DELAYED_EVENTS_MAX_PER_USER` | `100` | How many delayed events one user may have scheduled at once. |
 | `sync.presence` | `WARP_SYNC_PRESENCE` | `false` | Off by default; see below. |
 | `sync.typing_timeout` | `WARP_SYNC_TYPING_TIMEOUT` | `30s` | How long a typing notification lasts, and the cap on what a client may ask for. |
 | `sync.long_poll_timeout` | `WARP_SYNC_LONG_POLL_TIMEOUT` | `30s` | The cap on a `/sync` `?timeout=`. |

docs/deployment.md

@@ -135,6 +135,21 @@ already public behind its own proxy, set `WARP_WEB_PUBLIC_HOMESERVER` to
 its URL instead: the browser then calls it directly and warp-web serves
 only the client. See [apps/web](../apps/web/README.md).
 
+## Calls
+
+Calls need a LiveKit SFU and the LiveKit JWT service next to Warp.
+For a local try, `deploy/compose/calls.yml` adds both:
+
+```sh
+cd deploy/compose
+WARP_REGISTRATION_ENABLED=true docker compose -f docker-compose.yml -f calls.yml --profile web up -d --build
+```
+
+The Helm chart can run the JWT service (`calls.jwtService.enabled`) and
+sets `client_api.rtc.livekit_service_url` from it; LiveKit itself runs
+from its own chart. See [calls.md](calls.md) for how the pieces find each
+other and what production needs.
+
 ## Reverse proxy
 
 Behind a proxy that terminates TLS: