keyserver: cache remote users' device and cross-signing keys

POST /keys/query for a remote user who shares a room with a local user is
now answered from a cache, fetched whole with GET /user/devices/{userId}
on a miss. Incoming m.device_list_update EDUs are applied when their
prev_id matches the cached stream_id; anything else marks the copy stale
and it is fetched again in the background, and a stale copy is never
served. m.signing_key_update replaces the cached cross-signing keys. A
user's copy is dropped once no local user shares a room with them, by a
keyserver listener on the roomserver's emit stage.

Migration 0009 adds keyserver_remote_users, keyserver_remote_device_keys
and keyserver_remote_cross_signing_keys for sqlite and postgres.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/e2ee.md

@@ -27,6 +27,20 @@ servers' users too.
 | `keyserver_backup_versions`, `keyserver_backup_keys` | server-side key backup |
 | `keyserver_sequences` | the device list stream's counter |
 
+Migration `0009_remote_keys.sql` adds the cache of other servers' users'
+keys (see [The remote key cache](#the-remote-key-cache)):
+
+| Table | What |
+|---|---|
+| `keyserver_remote_users` | one row per cached user: the `stream_id` the copy is at, and whether it is `stale` |
+| `keyserver_remote_device_keys` | their devices' keys and display names, as their server sent them |
+| `keyserver_remote_cross_signing_keys` | their master and self-signing keys |
+
+They are tables of their own rather than rows in the local ones: every
+write to a local user's keys moves the device list stream and goes out over
+federation, and keeping the copies apart means no path that announces a
+local change can announce a copy.
+
 Keys are stored in canonical JSON. That is what lets "the same key
 uploaded again" be a byte comparison: a client that re-sends a one-time key
 it already uploaded, encoded differently, is not told it conflicts.
@@ -40,7 +54,7 @@ exactly what its owner uploaded.
 | Endpoint | Notes |
 |---|---|
 | `POST /keys/upload` | device keys must be for the caller's own device; one-time and fallback keys (`fallback_keys`, or MSC2732's unstable name); answers the one-time key counts, `signed_curve25519` always included |
-| `POST /keys/query` | local users from the store, remote users asked of their servers in parallel; `failures` for a server that did not answer |
+| `POST /keys/query` | local users from the store; remote users who share a room with somebody here from the cache, fetched on a miss; anybody else asked of their server; all servers in parallel, `failures` for one that did not answer |
 | `POST /keys/claim` | a one-time key if there is one, the fallback key otherwise; remote users claimed from their servers |
 | `GET /keys/changes` | whose device lists changed between two sync tokens |
 | `POST /keys/device_signing/upload` | cross-signing keys; asks for the password, except where the spec says not to |
@@ -147,23 +161,71 @@ uploads one.
 
 | EDU | Out | In |
 |---|---|---|
-| `m.device_list_update` | one per device change, to every server sharing a room with the user, with the keys or `deleted: true`, a `stream_id` and the `prev_id` before it | moves the remote user's position in the device list stream |
-| `m.signing_key_update` | when a user's master or self-signing key changes, or the master key gains a device's signature | likewise |
+| `m.device_list_update` | one per device change, to every server sharing a room with the user, with the keys or `deleted: true`, a `stream_id` and the `prev_id` before it | applied to the cached copy when its `prev_id` includes the copy's `stream_id`, otherwise the copy is marked stale and fetched again; moves the remote user's position in the device list stream |
+| `m.signing_key_update` | when a user's master or self-signing key changes, or the master key gains a device's signature | replaces the cached master and self-signing keys; moves the position likewise |
 
 Each device of a multi-device deletion gets a stream ID of its own:
 remote servers treat a stream ID they have seen as a duplicate, so three
 deletions sharing one would reach them as one deletion and two repeats.
 
-Warp does **not cache other servers' keys**. A query about a remote user is
-answered by asking their server, so an incoming update has nothing to
-invalidate; what it does is tell local clients, who do cache, by moving the
-user in the device list stream. That only happens when a local user shares
-a room with them: an update about a stranger concerns nobody here, and
-recording it would let any server grow the table with users of its own
-invention.
+An incoming update about a remote user always tells local clients, who
+keep their own copies, by moving the user in the device list stream — and
+only does anything when a local user shares a room with them: an update
+about a stranger concerns nobody here, and recording it would let any
+server grow the tables with users of its own invention. A repeat of an
+update already applied changes nothing, not even the stream.
 
 Device display names are not sent over federation, in answers or in EDUs.
-Local users see them in `unsigned.device_display_name`.
+Local users see them in `unsigned.device_display_name`: their own
+devices', and those of remote devices whose server did send one.
+
+### The remote key cache
+
+A client asks for the keys of everybody in a room before it sends an
+encrypted message, and again whenever a sync says one of them moved.
+Without a cache each of those is a federation round trip in front of the
+message. So the keys of remote users who **share a room with a local
+user** are kept (`internal/keyserver/remotekeys.go`):
+
+- **On a miss**, `/keys/query` fetches the user's whole device list with
+  `GET /user/devices/{userId}` (which also gives the `stream_id` the list
+  is at), caches it and serves it. A device whose keys name another user
+  or device is left out, and so is a cross-signing key that is not that
+  user's key for its purpose. The next query does not leave the server.
+- **An `m.device_list_update`** whose `prev_id` includes the cached
+  `stream_id` is applied: the device's keys replaced, or deleted. One
+  whose `stream_id` is at or before the copy's is a repeat and ignored.
+  Anything else (a gap, an update with no keys for a device it says
+  changed, keys that are not that device's) marks the copy **stale**, and
+  it is fetched again in the background, not inside the transaction that
+  brought the update, which would have the sending server wait on a
+  request back to itself.
+- **A stale copy is never served.** A query that finds one does what it
+  does on a miss and asks synchronously: a device list known to be missing
+  a change would have the sender skip the device just added, and the
+  message would be undecryptable on it. If the user's server is down, the
+  query reports it in `failures`.
+- **An `m.signing_key_update`** replaces the cached master and
+  self-signing keys with the ones it carries; one it leaves out is
+  dropped.
+- **A copy goes** once no local user shares a room with the user. The
+  keyserver listens on the roomserver's emit stage: a remote user's leave
+  or ban has their copy checked, a local user's has every copy checked, in
+  the background. A query or update that finds a cached user has become a
+  stranger drops the copy on the spot. Strangers are never cached; a query
+  about one is passed to their server as before.
+
+The cache and the updates race in one place: a snapshot fetched just
+before an update arrives. An update about a user with nothing cached
+leaves a stale marker at its `stream_id`, an update to a stale copy raises
+its `stream_id`, and a snapshot older than the `stream_id` held is not
+stored. So a snapshot that predates an update is never cached as current.
+
+Local moves that Warp does not send as an `m.device_list_update` (a device
+rename, a cross-signing change, a signature) still move the user's
+`stream_id`, so a Warp server caching another Warp server's user sees the
+next update as a gap and fetches again. That costs one request and is
+always correct.
 
 With federation off, remote users in a query or claim are reported in
 `failures` straight away; nothing is sent.
@@ -187,8 +249,6 @@ not exist, `404`. A version's algorithm cannot be changed.
 
 | Thing | Why |
 |---|---|
-| A cache of remote users' keys | Every query asks their server. Correct, and slower for a client in rooms with many remote users; a cache invalidated by the EDUs above is the obvious next step. |
-| Resync on a gap in a remote server's `prev_id` | Follows from the above: with nothing cached, a missed update costs nothing. |
 | Dehydrated devices (MSC3814) | — |
 | Claiming several keys per device (MSC3983), appservice key claims and queries (MSC3983/MSC3984) | application services are Phase 9 |
 | Purging a deleted device's to-device queue | the messages stay queued; a new device later logged in under the same ID is handed them, encrypted to the old device's keys and so unreadable to it |
@@ -199,6 +259,8 @@ not exist, `404`. A version's algorithm cannot be changed.
 | Where | What |
 |---|---|
 | `internal/storage/keys_test.go` | the key store against every driver: the device list stream, claims, fallback keys, signatures, the backup replacement rule |
+| `internal/storage/remotekeys_test.go` | the remote key cache against every driver: snapshots, updates that follow on, repeats, gaps, stale markers, older snapshots refused, dropping |
 | `internal/keyserver/keyserver_test.go` | the checks, with real ed25519 signatures: uploads, conflicts, cross-signing, the three signature kinds and their visibility, deletions, remote updates, backups |
-| `internal/setup/phase7_test.go` | through the whole HTTP stack: upload, sync, query, claim, `keys/changes`, cross-signing with and without UIA, backup, device deletion, leaving a room; and two federated servers exchanging keys and device list updates |
+| `internal/keyserver/remotekeys_test.go` | the cache: a second query not leaving the server, updates applied, a gap fetched again in the background, a stale copy not served, signing key updates, strangers dropped |
+| `internal/setup/phase7_test.go` | through the whole HTTP stack: upload, sync, query, claim, `keys/changes`, cross-signing with and without UIA, backup, device deletion, leaving a room; and two federated servers exchanging keys and device list updates, with a second query and an update served from B's cache without a request to A |
 | Complement | `TestUploadKey*`, `TestKeyClaimOrdering`, `TestKeyChanges*`, `TestDeviceListUpdates*`, `TestE2EKeyBackup*`, `TestFallbackKeys*` and `TestCrossSigning*` are in the target sets |