Phase 7: E2EE support

Implements PLAN.md Phase 7: the server half of end-to-end encryption.
Devices upload identity, one-time and fallback keys; other devices query
and claim them, across servers too, and every one-time key is handed out
exactly once. Cross-signing, signature uploads, device list tracking and
server-side key backup complete the set.

internal/keyserver is the component. Keys are stored in canonical JSON
so that re-uploading the same key is a byte comparison, and signatures
uploaded later are kept apart from the key they sign and merged in on
read, so a key's own JSON stays what its owner uploaded. Device keys must
be for the caller's own device; cross-signing keys must be the
uploader's, carry the right usage and, for self- and user-signing keys,
be signed by the master key. Signatures are verified against the key as
stored. Replacing a master key asks for the password; a first upload or
an upload of the keys already stored does not, as the spec allows. A
signature of another user's master key is shown only to the user who
made it and never federated.

Every write that changes what /keys/query answers moves the device list
stream in the same transaction, one position per device, and wakes the
syncs of everyone sharing a room. The stream is the sixth field of a sync
token; older tokens still parse. The syncapi works out device_lists
changed and left from the stream and from membership moves in the
window, skipping invites and knocks, and reports
device_one_time_keys_count and device_unused_fallback_key_types, the
latter as [] when empty. GET /keys/changes uses the same logic.

Over federation: /user/keys/query, /user/keys/claim and
/user/devices/{userId}, and m.device_list_update and m.signing_key_update
both ways. Remote keys are not cached; an update about a remote user
sharing a room moves them in the stream so local clients re-query.
Device display names are not sent to other servers. With federation off,
remote users are reported in failures at once.

Deleting a device deletes its keys, through a new userapi
DeviceObserver. Key backup keeps the better copy of each session by the
spec's rule, never reuses a deleted version's number, and answers writes
to a stale version with M_WRONG_ROOM_KEYS_VERSION.

Storage is migration 0008 for sqlite and postgres, with a PocketBase
stub. The Complement target sets gain the key, device list, fallback,
cross-signing and backup tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5.5 committed 2026-10-17 10:34:08 +02:00
1 parent 92836f8dcd
commit fc49f56339
45 files changed
+6050 -63

No files matched your search

@@ -0,0 +1,122 @@
-- The keyserver: what end-to-end encryption needs the server to hold for
-- its users' devices, none of which it can read or use itself.
--
-- Users are full user IDs throughout, not localparts. Most of what is here
-- is about local devices, but signatures can be of other servers' users'
-- keys, and the device list stream records their changes too: a local
-- client sharing a room with a remote user has to be told when that user's
-- keys move, and the stream is how a sync finds out.
-- The counter of the device list stream. It is the keyserver's own table
-- for the same reason every component has one, and the sync store reads it
-- next to its own counters so that a sync token names one moment in all of
-- them.
CREATE TABLE IF NOT EXISTS keyserver_sequences (
name TEXT NOT NULL PRIMARY KEY,
value BIGINT NOT NULL
);
INSERT INTO keyserver_sequences (name, value) VALUES ('device_list', 0) ON CONFLICT (name) DO NOTHING;
-- A device's identity keys: its curve25519 and ed25519 keys, signed by the
-- ed25519 one, exactly as the device uploaded them.
CREATE TABLE IF NOT EXISTS keyserver_device_keys (
user_id TEXT NOT NULL,
device_id TEXT NOT NULL,
key_json TEXT NOT NULL,
PRIMARY KEY (user_id, device_id)
);
-- One-time keys. A row is deleted the moment it is claimed: each key is
-- used for exactly one Olm session, and handing it out twice would let two
-- senders set up sessions the device can only tell apart by luck.
CREATE TABLE IF NOT EXISTS keyserver_one_time_keys (
user_id TEXT NOT NULL,
device_id TEXT NOT NULL,
-- "algorithm:id", as the key is named in the upload.
key_id TEXT NOT NULL,
algorithm TEXT NOT NULL,
key_json TEXT NOT NULL,
PRIMARY KEY (user_id, device_id, key_id)
);
-- Counting a device's keys by algorithm, and claiming the oldest.
CREATE INDEX IF NOT EXISTS keyserver_one_time_keys_algorithm_idx
ON keyserver_one_time_keys (user_id, device_id, algorithm);
-- Fallback keys: one per algorithm per device, handed out when the
-- one-time keys have run out. Unlike a one-time key it survives being
-- claimed, marked used, until the device uploads a new one.
CREATE TABLE IF NOT EXISTS keyserver_fallback_keys (
user_id TEXT NOT NULL,
device_id TEXT NOT NULL,
algorithm TEXT NOT NULL,
key_id TEXT NOT NULL,
key_json TEXT NOT NULL,
used BOOLEAN NOT NULL DEFAULT 0,
PRIMARY KEY (user_id, device_id, algorithm)
);
-- Cross-signing keys: a user's master, self-signing and user-signing keys.
CREATE TABLE IF NOT EXISTS keyserver_cross_signing_keys (
user_id TEXT NOT NULL,
-- master, self_signing or user_signing.
purpose TEXT NOT NULL,
key_json TEXT NOT NULL,
PRIMARY KEY (user_id, purpose)
);
-- Signatures uploaded after the key they sign, through
-- /keys/signatures/upload. They are merged into the key when it is read
-- rather than written into it, so the key's own JSON stays what its owner
-- uploaded.
CREATE TABLE IF NOT EXISTS keyserver_signatures (
target_user_id TEXT NOT NULL,
-- A device ID, or a cross-signing key's public key.
target_key_id TEXT NOT NULL,
signer_user_id TEXT NOT NULL,
-- "ed25519:" and the signing key's ID.
signer_key_id TEXT NOT NULL,
signature TEXT NOT NULL,
PRIMARY KEY (target_user_id, target_key_id, signer_user_id, signer_key_id)
);
-- The device list stream: each user's latest position, local and remote.
-- Only the latest is kept, as for presence: a sync asks whose list moved
-- after its token, and the history of the moves does not answer that any
-- better.
CREATE TABLE IF NOT EXISTS keyserver_device_lists (
user_id TEXT NOT NULL PRIMARY KEY,
stream_pos BIGINT NOT NULL
);
CREATE INDEX IF NOT EXISTS keyserver_device_lists_stream_idx
ON keyserver_device_lists (stream_pos);
-- Server-side key backup versions. A deleted version keeps its row, so
-- that its number is never handed out again: a client still holding the
-- old version number must not find itself writing into a new backup that
-- is encrypted to a different key.
CREATE TABLE IF NOT EXISTS keyserver_backup_versions (
user_id TEXT NOT NULL,
version BIGINT NOT NULL,
algorithm TEXT NOT NULL,
auth_data TEXT NOT NULL,
-- Bumped whenever a key in the version changes.
etag BIGINT NOT NULL DEFAULT 0,
deleted BOOLEAN NOT NULL DEFAULT 0,
PRIMARY KEY (user_id, version)
);
-- The backed-up room keys, one per Megolm session.
CREATE TABLE IF NOT EXISTS keyserver_backup_keys (
user_id TEXT NOT NULL,
version BIGINT NOT NULL,
room_id TEXT NOT NULL,
session_id TEXT NOT NULL,
first_message_index BIGINT NOT NULL,
forwarded_count BIGINT NOT NULL,
is_verified BOOLEAN NOT NULL,
session_data TEXT NOT NULL,
PRIMARY KEY (user_id, version, room_id, session_id)
);