Warp/docs/calls.md

# Voice and video calls

Every call in Warp's web client, between two people or twenty, is a
MatrixRTC call (MSC4143) with its media through a LiveKit SFU: the Element
Call model, which Element X uses too. A call of two is a call with two
members, plus a ring. The server needs a LiveKit SFU and the LiveKit JWT
service (`client_api.rtc`); without them there are no calls.

The old Matrix 1:1 VoIP (`m.call.*` events, WebRTC between the two
browsers) is not used: the web client starts the SDK with `disableVoip`, so
it neither places nor answers such calls, and shows them only as history in
the timeline. The server still serves `GET /_matrix/client/v3/voip/turnServer`
(`client_api.turn`) for other clients that do.

## How a call works

Who is in a room's call is room state: every device in it has an
`org.matrix.msc3401.call.member` state event (state key
`_@user:server_DEVICE_m.call`), with the LiveKit service it publishes to.
The media goes through a LiveKit SFU, which Warp does not run itself.

```
browser ── GET /.well-known/matrix/client ──────────▶ warp   org.matrix.msc4143.rtc_foci
        ── GET …/org.matrix.msc4143/rtc/transports ─▶ warp   (the same list, authenticated)
        ── PUT …/state/org.matrix.msc3401.call.member/<key>
             ?org.matrix.msc4140.delay=…  (the delayed "left")
             and then without a delay  (the "joined") ──▶ warp
        ── POST /user/<me>/openid/request_token ────▶ warp   an OpenID token
        ── POST <livekit_service_url>/sfu/get ──────▶ lk-jwt-service
             lk-jwt-service ── GET /_matrix/federation/v1/openid/userinfo ──▶ warp
        ◀── {url, jwt}
        ── WebSocket + UDP ─────────────────────────▶ LiveKit
```

### What the server does

- **Discovery.** With `client_api.rtc.livekit_service_url` set, Warp adds
  `org.matrix.msc4143.rtc_foci: [{"type": "livekit", "livekit_service_url": …}]`
  to `/.well-known/matrix/client` and serves the same list at
  `GET /_matrix/client/unstable/org.matrix.msc4143/rtc/transports`.
- **Delayed events (MSC4140)**, advertised as `org.matrix.msc4140` in
  `/versions`. `PUT /rooms/{roomId}/state/{type}/{key}` and
  `PUT /rooms/{roomId}/send/{type}/{txnId}` take
  `?org.matrix.msc4140.delay=<ms>`: the event is stored instead of sent,
  and the answer is `{"delay_id": …}`. Under
  `/_matrix/client/unstable/org.matrix.msc4140`:
  - `GET /delayed_events` lists your events as `scheduled` and `finalised`
    (`?status=` and `?delay_id=` narrow it); `GET /delayed_events/{id}`
    returns one.
  - `POST /delayed_events/{id}/{restart|cancel|send}`, or the older
    `POST /delayed_events/{id}` with `{"action": …}`. These need no access
    token: the delay ID is the capability.

  A scheduler goroutine sleeps until the next event is due and sends it
  as the user who scheduled it, through the same pipeline as any event, so
  power levels and membership still apply; a send that fails is finalised
  with the error the client would have got. Someone *else* setting the
  same piece of state cancels a delayed state event, so a stale "left"
  can't overwrite a newer membership; the owner's own updates don't. Events
  live in the database (`delayed_events`, migration 14) and survive a
  restart. Finalised events are kept for a day. `client_api.delayed_events`
  caps the delay (24h) and how many a user may have (100).

  This is what makes a call robust: a joining member schedules its own
  empty `call.member` a few seconds out and keeps restarting it while it
  is in the call. A browser that crashes or loses its network stops
  restarting it, and Warp removes the member.
- **OpenID.** `POST /_matrix/client/v3/user/{userId}/openid/request_token`
  gives a user a one-hour token, which `GET /_matrix/federation/v1/openid/userinfo?access_token=…`
  turns back into `{"sub": "@user:server"}` for whoever it was handed to.
  Tokens are stored hashed (`openid_tokens`). The userinfo endpoint is
  served with federation switched off too: a server that does not
  federate can still have calls.
- **Power levels.** New rooms let anyone who may talk join the call:
  `org.matrix.msc3401.call.member` is at level 0 in the `events` of the
  default power levels. Rooms created before this keep `state_default`
  (50) for it, so an ordinary member gets `M_FORBIDDEN` until a moderator
  sets it to 0.

### Ringing

Starting a call in a room with one other person rings them. The caller's
MatrixRTC session sends an `org.matrix.msc4075.rtc.notification`
(`notification_type: "ring"`, a `lifetime` of a minute, `m.call.intent`
audio or video) when its membership is the first; in a bigger room it
sends `notification`, which the web client doesn't ring for, since the
"Join call" banner says enough. The other side shows an incoming-call card
for a ring that is someone else's, mentions them or the room, and is
within its lifetime, until they answer (join the call), decline (an
`org.matrix.msc4310.rtc.decline` referencing the notification, which also
stops the ring on their other devices), the caller leaves, or it expires.
The caller hears a ringback until someone joins, and gives up on a decline
or after the minute. In an encrypted room both events are encrypted like
any other.

The SDK's own validator for received notifications expects MSC4354 sticky
events and `m.rtc.slot` state, which Warp doesn't serve yet, so the web
client applies the receiving rules itself.

### What the web client does

`src/stores/call.ts` uses the SDK's MatrixRTC session
(`client.matrixRTC.getRoomSession(room)`), publishing the transport
`{type: "livekit", livekit_service_url, livekit_alias: <room ID>}` (the one
the call already uses, else this server's), gets a LiveKit token from
`/sfu/get`, and connects with `livekit-client`. In an encrypted room it
joins with `manageMediaKeys`: each member makes its own media key and
sends it to the others over encrypted to-device messages, and a
`BaseKeyProvider` hands each key to LiveKit's frame encryption for that
participant (`@user:server:DEVICE`). The settings (no shared key, no
ratcheting) match Element Call's, so the two can share a call.

### Running it

Locally, [`deploy/compose/calls.yml`](../deploy/compose/calls.yml) adds
LiveKit and lk-jwt-service to the Compose setup:

```sh
cd deploy/compose
WARP_REGISTRATION_ENABLED=true docker compose -f docker-compose.yml -f calls.yml --profile web up -d --build
```

Then sign in as different users at <http://localhost:8080>,
<http://127.0.0.1:8080> and <http://[::1]:8080> (different origins keep
separate sessions in one browser). A voice or video call in a room of two
rings the other person; in a room of three or more the others see a "Join
call" banner. `WARP_PORT` and `WARP_WEB_PORT` move the homeserver and web client
off 8008 and 8080 (then set `WARP_PUBLIC_BASEURL` to match).

The JWT service checks an OpenID token by asking the user's homeserver
over its federation port with TLS. For the server name `localhost` that
means `https://localhost:8448`, so the file gives Warp a federation
listener with a self-signed certificate (`LIVEKIT_INSECURE_SKIP_VERIFY_TLS`
accepts it) and runs LiveKit and the JWT service in Warp's network
namespace, where `localhost` is Warp. One consequence: when the `warp`
container is recreated on its own, the other two lose their network. Run
`up` for the whole file, not for one service, or recreate them with
`docker compose -f docker-compose.yml -f calls.yml up -d --force-recreate livekit livekit-jwt`.
LiveKit advertises `127.0.0.1` for media; set `LIVEKIT_NODE_IP` to the
machine's LAN address to call from other devices.

In production, give each piece its own hostname and a real certificate:

- **LiveKit**, reachable by browsers on TCP 7880 behind TLS (`wss://`),
  and UDP (7882, or a port range) and TCP 7881 for media. See LiveKit's
  own deployment guide or its Helm chart.
- **lk-jwt-service** behind TLS, with `LIVEKIT_URL` (the `wss://` URL),
  `LIVEKIT_KEY` / `LIVEKIT_SECRET` (LiveKit's API key) and
  `LIVEKIT_FULL_ACCESS_HOMESERVERS` (your server name). The Helm chart
  runs it with `calls.jwtService.enabled`.
- **Warp** with `client_api.rtc.livekit_service_url` set to the JWT
  service's URL. The JWT service must reach
  `https://<server_name>/.well-known/matrix/server` (a delegation to the
  host that serves Warp) or `<server_name>:8448`. Note that Warp serves
  `/.well-known/matrix/server` only with federation on; with it off,
  serve the delegation file from the server name's web server, as you
  would for `/.well-known/matrix/client`.
- **warp-web**: `WARP_WEB_CALL_ORIGINS="https://livekit-jwt.example.org wss://livekit.example.org"`
  lets the browser reach both through its Content-Security-Policy.

### Known gaps

- Calls only use the legacy `org.matrix.msc3401.call.member` state events,
  not the sticky `m.rtc.member` events of newer MatrixRTC drafts.
- `org.matrix.msc4140.parent_delay_id` (chained delayed events) is
  refused, and a delayed send's transaction ID is not deduplicated.
- No ring for calls in rooms of more than two (only the banner), no
  MSC4354 sticky events, and no MSC4195 homeserver-side LiveKit token
  endpoint (`rtc/livekit/get_token`).
- Calls from clients that only do legacy 1:1 VoIP (`m.call.*`) don't ring
  in the web client.