Phase 7: E2EE support

Implements PLAN.md Phase 7: the server half of end-to-end encryption.
Devices upload identity, one-time and fallback keys; other devices query
and claim them, across servers too, and every one-time key is handed out
exactly once. Cross-signing, signature uploads, device list tracking and
server-side key backup complete the set.

internal/keyserver is the component. Keys are stored in canonical JSON
so that re-uploading the same key is a byte comparison, and signatures
uploaded later are kept apart from the key they sign and merged in on
read, so a key's own JSON stays what its owner uploaded. Device keys must
be for the caller's own device; cross-signing keys must be the
uploader's, carry the right usage and, for self- and user-signing keys,
be signed by the master key. Signatures are verified against the key as
stored. Replacing a master key asks for the password; a first upload or
an upload of the keys already stored does not, as the spec allows. A
signature of another user's master key is shown only to the user who
made it and never federated.

Every write that changes what /keys/query answers moves the device list
stream in the same transaction, one position per device, and wakes the
syncs of everyone sharing a room. The stream is the sixth field of a sync
token; older tokens still parse. The syncapi works out device_lists
changed and left from the stream and from membership moves in the
window, skipping invites and knocks, and reports
device_one_time_keys_count and device_unused_fallback_key_types, the
latter as [] when empty. GET /keys/changes uses the same logic.

Over federation: /user/keys/query, /user/keys/claim and
/user/devices/{userId}, and m.device_list_update and m.signing_key_update
both ways. Remote keys are not cached; an update about a remote user
sharing a room moves them in the stream so local clients re-query.
Device display names are not sent to other servers. With federation off,
remote users are reported in failures at once.

Deleting a device deletes its keys, through a new userapi
DeviceObserver. Key backup keeps the better copy of each session by the
spec's rule, never reuses a deleted version's number, and answers writes
to a stale version with M_WRONG_ROOM_KEYS_VERSION.

Storage is migration 0008 for sqlite and postgres, with a PocketBase
stub. The Complement target sets gain the key, device list, fallback,
cross-signing and backup tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

server/internal/clientapi/clientapi.go

@@ -20,6 +20,7 @@ import (
 	"github.com/dariuszjaniak/warp/server/internal/config"
 	"github.com/dariuszjaniak/warp/server/internal/federation"
 	"github.com/dariuszjaniak/warp/server/internal/httputil"
+	"github.com/dariuszjaniak/warp/server/internal/keyserver"
 	"github.com/dariuszjaniak/warp/server/internal/mediaapi"
 	"github.com/dariuszjaniak/warp/server/internal/ratelimit"
 	"github.com/dariuszjaniak/warp/server/internal/roomserver"
@@ -35,6 +36,7 @@ type ClientAPI struct {
 	users         *userapi.API
 	rooms         *roomserver.API
 	sync          *syncapi.API
+	keys          *keyserver.API
 	media         *mediaapi.API
 	legacyMedia   bool
 	settings      *settings.Service
@@ -92,6 +94,11 @@ type Options struct {
 	// pretending there is a stream to read.
 	Sync *syncapi.API
 
+	// Keys is the keyserver. Without it the end-to-end encryption
+	// endpoints answer M_UNRECOGNIZED, which is how a client learns it
+	// cannot encrypt here.
+	Keys *keyserver.API
+
 	// Media is the media repository. Without it the media endpoints
 	// answer M_UNRECOGNIZED.
 	Media *mediaapi.API
@@ -153,6 +160,7 @@ func New(opts Options) (*ClientAPI, error) {
 		users:         opts.Users,
 		rooms:         opts.Rooms,
 		sync:          opts.Sync,
+		keys:          opts.Keys,
 		media:         opts.Media,
 		legacyMedia:   opts.LegacyMediaDownloads,
 		settings:      opts.Settings,
@@ -247,6 +255,7 @@ func (c *ClientAPI) RegisterRoutes(r *httputil.Router) {
 
 	c.registerRoomRoutes(r)
 	c.registerSyncRoutes(r)
+	c.registerKeyRoutes(r)
 	c.registerMediaRoutes(r)
 }
 

server/internal/clientapi/keys.go

@@ -0,0 +1,374 @@
+package clientapi
+
+import (
+	"encoding/json"
+	"errors"
+	"net/http"
+	"time"
+
+	"github.com/dariuszjaniak/warp/server/internal/httputil"
+	"github.com/dariuszjaniak/warp/server/internal/keyserver"
+	"github.com/dariuszjaniak/warp/server/internal/userapi"
+)
+
+// The end-to-end encryption endpoints: device keys, one-time keys,
+// cross-signing, signatures and the server-side key backup. Every one of
+// them is the keyserver's; the handlers decode and shape.
+
+// registerKeyRoutes adds the keyserver endpoints, if there is one.
+func (c *ClientAPI) registerKeyRoutes(r *httputil.Router) {
+	if c.keys == nil {
+		return
+	}
+
+	c.authed(r, "POST "+v3+"/keys/upload", c.limitDefault, c.handleKeysUpload)
+	c.authed(r, "POST "+v3+"/keys/query", c.limitDefault, c.handleKeysQuery)
+	c.authed(r, "POST "+v3+"/keys/claim", c.limitDefault, c.handleKeysClaim)
+	c.authed(r, "POST "+v3+"/keys/device_signing/upload", c.limitDefault, c.handleCrossSigningUpload)
+	c.authed(r, "POST "+v3+"/keys/signatures/upload", c.limitDefault, c.handleSignaturesUpload)
+	if c.sync != nil {
+		c.authed(r, "GET "+v3+"/keys/changes", c.limitDefault, c.handleKeysChanges)
+	}
+
+	c.authed(r, "POST "+v3+"/room_keys/version", c.limitDefault, c.handleCreateBackup)
+	c.authed(r, "GET "+v3+"/room_keys/version", c.limitDefault, c.handleGetBackup)
+	c.authed(r, "GET "+v3+"/room_keys/version/{version}", c.limitDefault, c.handleGetBackup)
+	c.authed(r, "PUT "+v3+"/room_keys/version/{version}", c.limitDefault, c.handlePutBackup)
+	c.authed(r, "DELETE "+v3+"/room_keys/version/{version}", c.limitDefault, c.handleDeleteBackup)
+
+	// The keys themselves, at the three granularities the spec gives:
+	// the whole backup, one room, one session.
+	for _, path := range []string{
+		v3 + "/room_keys/keys",
+		v3 + "/room_keys/keys/{roomID}",
+		v3 + "/room_keys/keys/{roomID}/{sessionID}",
+	} {
+		c.authed(r, "PUT "+path, c.limitDefault, c.handlePutRoomKeys)
+		c.authed(r, "GET "+path, c.limitDefault, c.handleGetRoomKeys)
+		c.authed(r, "DELETE "+path, c.limitDefault, c.handleDeleteRoomKeys)
+	}
+}
+
+// keyError maps a keyserver error onto the Matrix error a client is sent.
+func keyError(err error) error {
+	var (
+		matrixErr   *httputil.MatrixError
+		wrongBackup *keyserver.WrongBackupVersionError
+	)
+	switch {
+	case err == nil:
+		return nil
+	case errors.As(err, &matrixErr):
+		return err
+	case errors.As(err, &wrongBackup):
+		e := httputil.NewError(http.StatusForbidden, httputil.ErrCodeWrongRoomKeysVersion, "%s", err.Error())
+		e.CurrentVersion = wrongBackup.Current
+		return e
+	case errors.Is(err, keyserver.ErrNoSuchBackup):
+		return httputil.NotFound("Unknown backup version")
+	case errors.Is(err, keyserver.ErrNoSuchBackupKey):
+		return httputil.NotFound("No room keys found")
+	case errors.Is(err, keyserver.ErrInvalidSignature):
+		return httputil.NewError(http.StatusBadRequest, httputil.ErrCodeInvalidSignature, "%s", err.Error())
+	case errors.Is(err, keyserver.ErrMissingParam):
+		return httputil.MissingParam("%s", err.Error())
+	case errors.Is(err, keyserver.ErrInvalidParam), errors.Is(err, keyserver.ErrKeyConflict):
+		return httputil.InvalidParam("%s", err.Error())
+	}
+	return err
+}
+
+// --- device and one-time keys ---------------------------------------------------
+
+type keysUploadResponse struct {
+	OneTimeKeyCounts map[string]int `json:"one_time_key_counts"`
+}
+
+// handleKeysUpload implements POST /_matrix/client/v3/keys/upload.
+func (c *ClientAPI) handleKeysUpload(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req keyserver.UploadRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	counts, err := c.keys.UploadKeys(r.Context(), auth.UserID, auth.DeviceID, req)
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, keysUploadResponse{OneTimeKeyCounts: counts})
+	return nil
+}
+
+// keysQueryRequest is the body of POST /keys/query.
+type keysQueryRequest struct {
+	DeviceKeys map[string][]string `json:"device_keys"`
+
+	// Timeout is in milliseconds.
+	Timeout int64 `json:"timeout"`
+}
+
+// handleKeysQuery implements POST /_matrix/client/v3/keys/query.
+func (c *ClientAPI) handleKeysQuery(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req keysQueryRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	resp, err := c.keys.QueryKeys(r.Context(), auth.UserID, keyserver.QueryRequest{
+		DeviceKeys: req.DeviceKeys,
+		Timeout:    time.Duration(req.Timeout) * time.Millisecond,
+	})
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, resp)
+	return nil
+}
+
+// keysClaimRequest is the body of POST /keys/claim.
+type keysClaimRequest struct {
+	OneTimeKeys map[string]map[string]string `json:"one_time_keys"`
+	Timeout     int64                        `json:"timeout"`
+}
+
+// handleKeysClaim implements POST /_matrix/client/v3/keys/claim.
+func (c *ClientAPI) handleKeysClaim(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error {
+	var req keysClaimRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	resp, err := c.keys.ClaimKeys(r.Context(), keyserver.ClaimRequest{
+		OneTimeKeys: req.OneTimeKeys,
+		Timeout:     time.Duration(req.Timeout) * time.Millisecond,
+	})
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, resp)
+	return nil
+}
+
+// handleKeysChanges implements GET /_matrix/client/v3/keys/changes.
+func (c *ClientAPI) handleKeysChanges(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	query := r.URL.Query()
+	from := query.Get("from")
+	if from == "" {
+		return httputil.MissingParam("from is required")
+	}
+	lists, err := c.sync.KeyChanges(r.Context(), auth.UserID, from, query.Get("to"))
+	if err != nil {
+		return syncError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, lists)
+	return nil
+}
+
+// --- cross-signing ------------------------------------------------------------
+
+// crossSigningUploadRequest is the body of POST /keys/device_signing/upload.
+type crossSigningUploadRequest struct {
+	keyserver.CrossSigningRequest
+	Auth *uiaRequest `json:"auth"`
+}
+
+// handleCrossSigningUpload implements
+// POST /_matrix/client/v3/keys/device_signing/upload.
+//
+// Replacing cross-signing keys asks for the password first, for the same
+// reason deleting a device does: a stolen access token must not be enough
+// to become the user to everyone who has verified them. The keyserver says
+// when the upload is one of the two the spec exempts.
+func (c *ClientAPI) handleCrossSigningUpload(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req crossSigningUploadRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+
+	needsAuth, err := c.keys.CrossSigningNeedsAuth(r.Context(), auth.UserID, req.CrossSigningRequest)
+	if err != nil {
+		return keyError(err)
+	}
+	sessionID := ""
+	if needsAuth {
+		flows := []uiaFlow{{Stages: []string{stagePassword}}}
+		var done bool
+		sessionID, done, err = c.beginOrAdvanceUIA(w, flows, req.Auth,
+			func(_ string, stage *uiaRequest) (*httputil.MatrixError, error) {
+				return c.passwordStage(r, auth, stage)
+			})
+		if err != nil || !done {
+			return err
+		}
+	}
+
+	if err := c.keys.UploadCrossSigningKeys(r.Context(), auth.UserID, req.CrossSigningRequest); err != nil {
+		return keyError(err)
+	}
+	if sessionID != "" {
+		c.uia.remove(sessionID)
+	}
+	httputil.WriteJSON(w, http.StatusOK, struct{}{})
+	return nil
+}
+
+type signaturesUploadResponse struct {
+	Failures map[string]map[string]keyserver.SignatureFailure `json:"failures"`
+}
+
+// handleSignaturesUpload implements
+// POST /_matrix/client/v3/keys/signatures/upload.
+func (c *ClientAPI) handleSignaturesUpload(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req map[string]map[string]json.RawMessage
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	failures, err := c.keys.UploadSignatures(r.Context(), auth.UserID, req)
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, signaturesUploadResponse{Failures: failures})
+	return nil
+}
+
+// --- key backup ---------------------------------------------------------------
+
+// backupVersionRequest is the body of POST and PUT /room_keys/version.
+type backupVersionRequest struct {
+	Algorithm string          `json:"algorithm"`
+	AuthData  json.RawMessage `json:"auth_data"`
+	Version   string          `json:"version"`
+}
+
+// handleCreateBackup implements POST /_matrix/client/v3/room_keys/version.
+func (c *ClientAPI) handleCreateBackup(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req backupVersionRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	version, err := c.keys.CreateBackup(r.Context(), auth.UserID, req.Algorithm, req.AuthData)
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, map[string]string{"version": version})
+	return nil
+}
+
+// handleGetBackup implements GET /_matrix/client/v3/room_keys/version and
+// GET /_matrix/client/v3/room_keys/version/{version}.
+func (c *ClientAPI) handleGetBackup(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	version := r.PathValue("version")
+	info, err := c.keys.Backup(r.Context(), auth.UserID, version)
+	if errors.Is(err, keyserver.ErrNoSuchBackup) && version == "" {
+		return httputil.NotFound("No current backup version")
+	}
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, info)
+	return nil
+}
+
+// handlePutBackup implements PUT /_matrix/client/v3/room_keys/version/{version}.
+func (c *ClientAPI) handlePutBackup(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	var req backupVersionRequest
+	if err := httputil.DecodeJSON(w, r, &req); err != nil {
+		return err
+	}
+	if err := c.keys.UpdateBackup(r.Context(), auth.UserID, r.PathValue("version"),
+		req.Algorithm, req.Version, req.AuthData); err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, struct{}{})
+	return nil
+}
+
+// handleDeleteBackup implements DELETE /_matrix/client/v3/room_keys/version/{version}.
+func (c *ClientAPI) handleDeleteBackup(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	if err := c.keys.DeleteBackup(r.Context(), auth.UserID, r.PathValue("version")); err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, struct{}{})
+	return nil
+}
+
+// roomKeysRequest is the body of PUT /room_keys/keys at whatever
+// granularity the path names; only the matching field is read.
+type roomKeysRequest struct {
+	Rooms    map[string]keyserver.RoomKeyBackup `json:"rooms"`
+	Sessions map[string]keyserver.KeyBackupData `json:"sessions"`
+}
+
+// handlePutRoomKeys implements PUT /_matrix/client/v3/room_keys/keys,
+// .../{roomId} and .../{roomId}/{sessionId}.
+func (c *ClientAPI) handlePutRoomKeys(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	roomID, sessionID := r.PathValue("roomID"), r.PathValue("sessionID")
+
+	var rooms map[string]keyserver.RoomKeyBackup
+	switch {
+	case sessionID != "":
+		var data keyserver.KeyBackupData
+		if err := httputil.DecodeJSON(w, r, &data); err != nil {
+			return err
+		}
+		rooms = map[string]keyserver.RoomKeyBackup{
+			roomID: {Sessions: map[string]keyserver.KeyBackupData{sessionID: data}},
+		}
+	case roomID != "":
+		var req roomKeysRequest
+		if err := httputil.DecodeJSON(w, r, &req); err != nil {
+			return err
+		}
+		rooms = map[string]keyserver.RoomKeyBackup{roomID: {Sessions: req.Sessions}}
+	default:
+		var req roomKeysRequest
+		if err := httputil.DecodeJSON(w, r, &req); err != nil {
+			return err
+		}
+		rooms = req.Rooms
+	}
+
+	result, err := c.keys.PutBackupKeys(r.Context(), auth.UserID, r.URL.Query().Get("version"), rooms)
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, result)
+	return nil
+}
+
+// handleGetRoomKeys implements GET /_matrix/client/v3/room_keys/keys,
+// .../{roomId} and .../{roomId}/{sessionId}.
+func (c *ClientAPI) handleGetRoomKeys(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	roomID, sessionID := r.PathValue("roomID"), r.PathValue("sessionID")
+	rooms, err := c.keys.BackupKeys(r.Context(), auth.UserID, r.URL.Query().Get("version"), roomID, sessionID)
+	if err != nil {
+		return keyError(err)
+	}
+
+	switch {
+	case sessionID != "":
+		data, ok := rooms[roomID].Sessions[sessionID]
+		if !ok {
+			return httputil.NotFound("No room key found for that session")
+		}
+		httputil.WriteJSON(w, http.StatusOK, data)
+	case roomID != "":
+		room, ok := rooms[roomID]
+		if !ok {
+			room = keyserver.RoomKeyBackup{Sessions: map[string]keyserver.KeyBackupData{}}
+		}
+		httputil.WriteJSON(w, http.StatusOK, room)
+	default:
+		httputil.WriteJSON(w, http.StatusOK, map[string]any{"rooms": rooms})
+	}
+	return nil
+}
+
+// handleDeleteRoomKeys implements DELETE /_matrix/client/v3/room_keys/keys,
+// .../{roomId} and .../{roomId}/{sessionId}.
+func (c *ClientAPI) handleDeleteRoomKeys(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error {
+	result, err := c.keys.DeleteBackupKeys(r.Context(), auth.UserID, r.URL.Query().Get("version"),
+		r.PathValue("roomID"), r.PathValue("sessionID"))
+	if err != nil {
+		return keyError(err)
+	}
+	httputil.WriteJSON(w, http.StatusOK, result)
+	return nil
+}

server/internal/clientapi/sync_test.go

@@ -260,7 +260,7 @@ func TestSyncRejectsAnUnreadableToken(t *testing.T) {
 	// "s-1" is not here: a negative event position is a /messages token
 	// from inside backfilled history, and a valid ?since=. Only the event
 	// stream goes below zero.
-	for _, since := range []string{"nonsense", "s1_-1", "sx", "s1_2_3_4_5_6"} {
+	for _, since := range []string{"nonsense", "s1_-1", "sx", "s1_2_3_4_5_6_7"} {
 		rec := ts.get("/_matrix/client/v3/sync?since="+url.QueryEscape(since), alice.AccessToken)
 		expectError(t, rec, http.StatusBadRequest, "M_INVALID_PARAM")
 	}

server/internal/federation/types.go

@@ -31,6 +31,7 @@ const (
 	EDUDirectToDev  = "m.direct_to_device"
 	EDUPresence     = "m.presence"
 	EDUDeviceListUp = "m.device_list_update"
+	EDUSigningKeys  = "m.signing_key_update"
 )
 
 // SendTransactionResponse is the answer to PUT /send: a result per PDU.

server/internal/federation/userkeys.go

@@ -0,0 +1,109 @@
+package federation
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+)
+
+// The end-to-end encryption half of the Server-Server API: asking another
+// server for its users' device keys, claiming their one-time keys, and the
+// EDUs that say a user's keys have changed.
+//
+// Keys travel as json.RawMessage. They are signed objects, and the only
+// safe thing to do with a signed object on its way through is not to
+// re-encode it.
+
+// UserKeysQueryRequest is the body of POST /user/keys/query: user ID to the
+// device IDs wanted, where an empty list asks for every device.
+type UserKeysQueryRequest struct {
+	DeviceKeys map[string][]string `json:"device_keys"`
+}
+
+// UserKeysQueryResponse is the answer to POST /user/keys/query.
+type UserKeysQueryResponse struct {
+	DeviceKeys      map[string]map[string]json.RawMessage `json:"device_keys"`
+	MasterKeys      map[string]json.RawMessage            `json:"master_keys,omitempty"`
+	SelfSigningKeys map[string]json.RawMessage            `json:"self_signing_keys,omitempty"`
+}
+
+// UserKeysClaimRequest is the body of POST /user/keys/claim: user ID, then
+// device ID, then the algorithm of the key wanted.
+type UserKeysClaimRequest struct {
+	OneTimeKeys map[string]map[string]string `json:"one_time_keys"`
+}
+
+// UserKeysClaimResponse is the answer to POST /user/keys/claim: user ID,
+// then device ID, then key ID, then the key.
+type UserKeysClaimResponse struct {
+	OneTimeKeys map[string]map[string]map[string]json.RawMessage `json:"one_time_keys"`
+}
+
+// UserDevicesResponse is the answer to GET /user/devices/{userId}: the
+// whole of a user's device list, which a server asks for when the updates
+// it was sent do not add up.
+type UserDevicesResponse struct {
+	UserID         string          `json:"user_id"`
+	StreamID       int64           `json:"stream_id"`
+	Devices        []UserDevice    `json:"devices"`
+	MasterKey      json.RawMessage `json:"master_key,omitempty"`
+	SelfSigningKey json.RawMessage `json:"self_signing_key,omitempty"`
+}
+
+// UserDevice is one device in a UserDevicesResponse.
+type UserDevice struct {
+	DeviceID          string          `json:"device_id"`
+	Keys              json.RawMessage `json:"keys"`
+	DeviceDisplayName string          `json:"device_display_name,omitempty"`
+}
+
+// DeviceListUpdateContent is the content of an m.device_list_update EDU:
+// one device of one user changed, or went away.
+type DeviceListUpdateContent struct {
+	UserID            string          `json:"user_id"`
+	DeviceID          string          `json:"device_id"`
+	DeviceDisplayName string          `json:"device_display_name,omitempty"`
+	StreamID          int64           `json:"stream_id"`
+	PrevID            []int64         `json:"prev_id,omitempty"`
+	Deleted           bool            `json:"deleted,omitempty"`
+	Keys              json.RawMessage `json:"keys,omitempty"`
+}
+
+// SigningKeyUpdateContent is the content of an m.signing_key_update EDU: a
+// user's cross-signing keys changed.
+type SigningKeyUpdateContent struct {
+	UserID         string          `json:"user_id"`
+	MasterKey      json.RawMessage `json:"master_key,omitempty"`
+	SelfSigningKey json.RawMessage `json:"self_signing_key,omitempty"`
+}
+
+// QueryUserKeys asks a server for its users' device and cross-signing
+// keys.
+func (c *Client) QueryUserKeys(ctx context.Context, destination string, req UserKeysQueryRequest) (UserKeysQueryResponse, error) {
+	var out UserKeysQueryResponse
+	err := c.do(ctx, request{
+		method: http.MethodPost, destination: destination, signed: true,
+		path: "/_matrix/federation/v1/user/keys/query", body: req,
+	}, &out)
+	return out, err
+}
+
+// ClaimUserKeys claims one-time keys of a server's users' devices.
+func (c *Client) ClaimUserKeys(ctx context.Context, destination string, req UserKeysClaimRequest) (UserKeysClaimResponse, error) {
+	var out UserKeysClaimResponse
+	err := c.do(ctx, request{
+		method: http.MethodPost, destination: destination, signed: true,
+		path: "/_matrix/federation/v1/user/keys/claim", body: req,
+	}, &out)
+	return out, err
+}
+
+// UserDevices asks a server for one of its users' whole device list.
+func (c *Client) UserDevices(ctx context.Context, destination, userID string) (UserDevicesResponse, error) {
+	var out UserDevicesResponse
+	err := c.do(ctx, request{
+		method: http.MethodGet, destination: destination, signed: true,
+		path: "/_matrix/federation/v1/user/devices/" + esc(userID),
+	}, &out)
+	return out, err
+}

server/internal/federationapi/keys.go

@@ -0,0 +1,96 @@
+package federationapi
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+
+	"github.com/dariuszjaniak/warp/server/internal/federation"
+	"github.com/dariuszjaniak/warp/server/internal/httputil"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/id"
+)
+
+// The end-to-end encryption endpoints other servers call: the keys of this
+// server's users, their one-time keys, and their whole device lists.
+
+// KeyService answers other servers' questions about this server's users'
+// keys, and takes in their news about their own users'. *keyserver.API
+// satisfies it.
+type KeyService interface {
+	QueryLocalKeys(ctx context.Context, req federation.UserKeysQueryRequest) (federation.UserKeysQueryResponse, error)
+	ClaimLocalKeys(ctx context.Context, want map[string]map[string]string) (map[string]map[string]map[string]json.RawMessage, error)
+	UserDevices(ctx context.Context, userID string) (federation.UserDevicesResponse, error)
+	RemoteKeysChanged(ctx context.Context, userID string) error
+}
+
+// registerKeyRoutes adds the key endpoints, if there is a keyserver.
+func (s *Server) registerKeyRoutes(authed func(string, authedHandler)) {
+	if s.keyService == nil {
+		return
+	}
+	const v1 = "/_matrix/federation/v1"
+	authed("POST "+v1+"/user/keys/query", s.handleUserKeysQuery)
+	authed("POST "+v1+"/user/keys/claim", s.handleUserKeysClaim)
+	authed("GET "+v1+"/user/devices/{userID}", s.handleUserDevices)
+}
+
+func (s *Server) handleUserKeysQuery(w http.ResponseWriter, r *http.Request, _ string, body []byte) error {
+	var req federation.UserKeysQueryRequest
+	if err := decode(body, &req); err != nil {
+		return err
+	}
+	resp, err := s.keyService.QueryLocalKeys(r.Context(), req)
+	if err != nil {
+		return err
+	}
+	httputil.WriteJSON(w, http.StatusOK, resp)
+	return nil
+}
+
+func (s *Server) handleUserKeysClaim(w http.ResponseWriter, r *http.Request, _ string, body []byte) error {
+	var req federation.UserKeysClaimRequest
+	if err := decode(body, &req); err != nil {
+		return err
+	}
+	claimed, err := s.keyService.ClaimLocalKeys(r.Context(), req.OneTimeKeys)
+	if err != nil {
+		return err
+	}
+	httputil.WriteJSON(w, http.StatusOK, federation.UserKeysClaimResponse{OneTimeKeys: claimed})
+	return nil
+}
+
+func (s *Server) handleUserDevices(w http.ResponseWriter, r *http.Request, _ string, _ []byte) error {
+	userID := r.PathValue("userID")
+	server, err := id.ServerNameOf(userID)
+	if err != nil {
+		return httputil.InvalidParam("%s is not a user ID", userID)
+	}
+	if server != s.serverName {
+		return httputil.NotFound("%s is not a user of this server", userID)
+	}
+	resp, err := s.keyService.UserDevices(r.Context(), userID)
+	if err != nil {
+		return err
+	}
+	httputil.WriteJSON(w, http.StatusOK, resp)
+	return nil
+}
+
+// receiveKeyEDU takes in an m.device_list_update or m.signing_key_update.
+// Either says only that a user of the sending server changed their keys;
+// the keyserver passes that on to the local clients that care.
+func (s *Server) receiveKeyEDU(ctx context.Context, origin string, edu federation.EDU) {
+	if s.keyService == nil {
+		return
+	}
+	var c struct {
+		UserID string `json:"user_id"`
+	}
+	if json.Unmarshal(edu.Content, &c) != nil || !sameServer(c.UserID, origin) {
+		return
+	}
+	if err := s.keyService.RemoteKeysChanged(ctx, c.UserID); err != nil {
+		s.log.Debug("ignored a key update EDU", "origin", origin, "type", edu.Type, "error", err)
+	}
+}

server/internal/federationapi/server.go

@@ -61,6 +61,11 @@ type ServerOptions struct {
 	// endpoints unregistered.
 	Media *mediaapi.API
 
+	// UserKeys answers queries and claims of this server's users'
+	// end-to-end keys, and takes in key update EDUs. Nil leaves the key
+	// endpoints unregistered.
+	UserKeys KeyService
+
 	Logger *slog.Logger
 	Now    func() time.Time
 }
@@ -75,6 +80,7 @@ type Server struct {
 	profiles   Profiles
 	store      storage.FederationStore
 	media      *mediaapi.API
+	keyService KeyService
 	log        *slog.Logger
 	now        func() time.Time
 }
@@ -102,6 +108,7 @@ func NewServer(opts ServerOptions) (*Server, error) {
 		profiles:   opts.Profiles,
 		store:      opts.Store,
 		media:      opts.Media,
+		keyService: opts.UserKeys,
 		log:        opts.Logger,
 		now:        opts.Now,
 	}, nil
@@ -152,6 +159,7 @@ func (s *Server) RegisterRoutes(r *httputil.Router) {
 	authed("POST "+v1+"/publicRooms", s.handlePublicRoomsPost)
 
 	s.registerMediaRoutes(authed)
+	s.registerKeyRoutes(authed)
 }
 
 // authenticated checks X-Matrix and the allow/deny lists before the
@@ -382,9 +390,11 @@ func (s *Server) receiveEDU(ctx context.Context, origin string, edu federation.E
 			s.log.Debug("ignored a to-device EDU", "origin", origin, "error", err)
 		}
 
+	case federation.EDUDeviceListUp, federation.EDUSigningKeys:
+		s.receiveKeyEDU(ctx, origin, edu)
+
 	default:
-		// Presence and device list updates belong to phases that have not
-		// landed (presence is off by default; device lists are Phase 7).
+		// Presence is off by default and not federated yet.
 		s.log.Debug("ignored an EDU", "origin", origin, "type", edu.Type)
 	}
 }

server/internal/httputil/errors.go

@@ -39,6 +39,10 @@ const (
 	// Media codes, added with the mediaapi in Phase 6.
 	ErrCodeNotYetUploaded  = "M_NOT_YET_UPLOADED"
 	ErrCodeCannotOverwrite = "M_CANNOT_OVERWRITE_MEDIA"
+
+	// End-to-end encryption codes, added with the keyserver in Phase 7.
+	ErrCodeInvalidSignature     = "M_INVALID_SIGNATURE"
+	ErrCodeWrongRoomKeysVersion = "M_WRONG_ROOM_KEYS_VERSION"
 )
 
 // MatrixError is the standard Matrix error envelope. Every non-2xx response
@@ -58,6 +62,10 @@ type MatrixError struct {
 	// encryption state and re-login.
 	SoftLogout bool `json:"soft_logout,omitempty"`
 
+	// CurrentVersion accompanies M_WRONG_ROOM_KEYS_VERSION: the backup
+	// version the client should be writing to instead.
+	CurrentVersion string `json:"current_version,omitempty"`
+
 	// status is the HTTP status to send; not part of the JSON body.
 	status int
 }

server/internal/keyserver/backup.go

@@ -0,0 +1,204 @@
+package keyserver
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+
+	"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Server-side key backup: a user's room keys, encrypted to a backup key
+// only the user holds, so that a new device can read history the user's
+// other devices received.
+//
+// The server stores what it is given and decides one thing: when two
+// copies of the same session's key arrive, which to keep. Everything else
+// about a backup — what the algorithm is, what the auth data says, whether
+// the backup key is trusted — is the clients' business.
+
+// BackupInfo is a backup version as GET /room_keys/version reports it.
+type BackupInfo struct {
+	Algorithm string          `json:"algorithm"`
+	AuthData  json.RawMessage `json:"auth_data"`
+	Count     int             `json:"count"`
+	ETag      string          `json:"etag"`
+	Version   string          `json:"version"`
+}
+
+// KeyBackupData is one backed-up session key on the wire.
+type KeyBackupData struct {
+	FirstMessageIndex int64           `json:"first_message_index"`
+	ForwardedCount    int64           `json:"forwarded_count"`
+	IsVerified        bool            `json:"is_verified"`
+	SessionData       json.RawMessage `json:"session_data"`
+}
+
+// RoomKeyBackup is one room's keys: session ID to key.
+type RoomKeyBackup struct {
+	Sessions map[string]KeyBackupData `json:"sessions"`
+}
+
+// BackupKeysResult is what a write or delete of keys answers with.
+type BackupKeysResult struct {
+	Count int    `json:"count"`
+	ETag  string `json:"etag"`
+}
+
+func infoOf(v storage.BackupVersion) BackupInfo {
+	return BackupInfo{
+		Algorithm: v.Algorithm,
+		AuthData:  v.AuthData,
+		Count:     v.Count,
+		ETag:      v.ETag,
+		Version:   v.Version,
+	}
+}
+
+// authDataOf checks auth_data is an object and canonicalises it.
+func authDataOf(raw json.RawMessage) ([]byte, error) {
+	if len(raw) == 0 {
+		return nil, missingParamf("auth_data is required")
+	}
+	if _, err := object(raw); err != nil {
+		return nil, invalidParamf("auth_data must be an object")
+	}
+	out, err := canonicaljson.Encode(raw)
+	if err != nil {
+		return nil, invalidParamf("auth_data is not valid JSON: %v", err)
+	}
+	return out, nil
+}
+
+// CreateBackup creates a new backup version, which becomes the current
+// one.
+func (k *API) CreateBackup(ctx context.Context, userID, algorithm string, authData json.RawMessage) (string, error) {
+	if algorithm == "" {
+		return "", missingParamf("algorithm is required")
+	}
+	encoded, err := authDataOf(authData)
+	if err != nil {
+		return "", err
+	}
+	return k.store.CreateBackup(ctx, userID, algorithm, encoded)
+}
+
+// Backup reads a backup version, or the current one when version is empty.
+func (k *API) Backup(ctx context.Context, userID, version string) (BackupInfo, error) {
+	v, err := k.store.Backup(ctx, userID, version)
+	if err != nil {
+		return BackupInfo{}, err
+	}
+	return infoOf(v), nil
+}
+
+// UpdateBackup replaces a version's auth data. The algorithm cannot
+// change — every key in the version is encrypted with it — and a version
+// in the body has to be the one in the path.
+func (k *API) UpdateBackup(ctx context.Context, userID, version, algorithm, bodyVersion string, authData json.RawMessage) error {
+	if bodyVersion != "" && bodyVersion != version {
+		return invalidParamf("the version in the body, %s, is not the one in the path, %s", bodyVersion, version)
+	}
+	existing, err := k.store.Backup(ctx, userID, version)
+	if err != nil {
+		return err
+	}
+	if algorithm != existing.Algorithm {
+		return invalidParamf("a backup version's algorithm cannot change from %s", existing.Algorithm)
+	}
+	encoded, err := authDataOf(authData)
+	if err != nil {
+		return err
+	}
+	return k.store.UpdateBackup(ctx, userID, version, encoded)
+}
+
+// DeleteBackup deletes a backup version and its keys.
+func (k *API) DeleteBackup(ctx context.Context, userID, version string) error {
+	return k.store.DeleteBackup(ctx, userID, version)
+}
+
+// PutBackupKeys stores keys in a version, which has to be the current one:
+// a client writing into an old version after another device replaced it
+// would be backing up keys nobody can restore from.
+func (k *API) PutBackupKeys(ctx context.Context, userID, version string, rooms map[string]RoomKeyBackup) (BackupKeysResult, error) {
+	if version == "" {
+		return BackupKeysResult{}, missingParamf("the version parameter is required")
+	}
+	current, err := k.store.Backup(ctx, userID, "")
+	switch {
+	case errors.Is(err, storage.ErrNoSuchBackup):
+		return BackupKeysResult{}, ErrNoSuchBackup
+	case err != nil:
+		return BackupKeysResult{}, err
+	case current.Version != version:
+		if _, err := k.store.Backup(ctx, userID, version); errors.Is(err, storage.ErrNoSuchBackup) {
+			return BackupKeysResult{}, ErrNoSuchBackup
+		}
+		return BackupKeysResult{}, &WrongBackupVersionError{Current: current.Version}
+	}
+
+	var keys []storage.BackupKey
+	for roomID, room := range rooms {
+		for sessionID, data := range room.Sessions {
+			if len(data.SessionData) == 0 {
+				return BackupKeysResult{}, missingParamf("session_data is required for %s in %s", sessionID, roomID)
+			}
+			sessionData, err := canonicaljson.Encode(data.SessionData)
+			if err != nil {
+				return BackupKeysResult{}, invalidParamf("session_data of %s is not valid JSON: %v", sessionID, err)
+			}
+			keys = append(keys, storage.BackupKey{
+				RoomID: roomID, SessionID: sessionID,
+				FirstMessageIndex: data.FirstMessageIndex,
+				ForwardedCount:    data.ForwardedCount,
+				IsVerified:        data.IsVerified,
+				SessionData:       sessionData,
+			})
+		}
+	}
+	v, err := k.store.PutBackupKeys(ctx, userID, version, keys)
+	if err != nil {
+		return BackupKeysResult{}, err
+	}
+	return BackupKeysResult{Count: v.Count, ETag: v.ETag}, nil
+}
+
+// BackupKeys reads keys of a version, grouped by room.
+func (k *API) BackupKeys(ctx context.Context, userID, version, roomID, sessionID string) (map[string]RoomKeyBackup, error) {
+	if version == "" {
+		return nil, missingParamf("the version parameter is required")
+	}
+	keys, err := k.store.BackupKeys(ctx, userID, version, roomID, sessionID)
+	if err != nil {
+		return nil, err
+	}
+	out := map[string]RoomKeyBackup{}
+	for _, key := range keys {
+		room, ok := out[key.RoomID]
+		if !ok {
+			room = RoomKeyBackup{Sessions: map[string]KeyBackupData{}}
+			out[key.RoomID] = room
+		}
+		room.Sessions[key.SessionID] = KeyBackupData{
+			FirstMessageIndex: key.FirstMessageIndex,
+			ForwardedCount:    key.ForwardedCount,
+			IsVerified:        key.IsVerified,
+			SessionData:       key.SessionData,
+		}
+	}
+	return out, nil
+}
+
+// DeleteBackupKeys deletes keys from a version.
+func (k *API) DeleteBackupKeys(ctx context.Context, userID, version, roomID, sessionID string) (BackupKeysResult, error) {
+	if version == "" {
+		return BackupKeysResult{}, missingParamf("the version parameter is required")
+	}
+	v, err := k.store.DeleteBackupKeys(ctx, userID, version, roomID, sessionID)
+	if err != nil {
+		return BackupKeysResult{}, err
+	}
+	return BackupKeysResult{Count: v.Count, ETag: v.ETag}, nil
+}

server/internal/keyserver/crosssigning.go

@@ -0,0 +1,429 @@
+package keyserver
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"slices"
+	"strings"
+
+	"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Cross-signing: a user's master key, the self-signing key it signs and
+// that signs the user's devices, and the user-signing key it signs and
+// that signs other users' master keys. Verifying one device of a user then
+// verifies the user, which is what makes verification bearable for a user
+// with five devices.
+//
+// The server checks what it can: that each key is the uploader's own, has
+// the usage it is uploaded as, and — for the self-signing and user-signing
+// keys — is signed by the master key. It cannot check that the master key
+// is the user's, which is what verification between users is for.
+
+// CrossSigningRequest is the body of POST /keys/device_signing/upload.
+type CrossSigningRequest struct {
+	MasterKey      json.RawMessage `json:"master_key,omitempty"`
+	SelfSigningKey json.RawMessage `json:"self_signing_key,omitempty"`
+	UserSigningKey json.RawMessage `json:"user_signing_key,omitempty"`
+}
+
+// keys pairs each uploaded key with its purpose, leaving out the ones
+// absent.
+func (r CrossSigningRequest) keys() map[string]json.RawMessage {
+	out := map[string]json.RawMessage{}
+	for purpose, raw := range map[string]json.RawMessage{
+		storage.CrossSigningMaster:      r.MasterKey,
+		storage.CrossSigningSelfSigning: r.SelfSigningKey,
+		storage.CrossSigningUserSigning: r.UserSigningKey,
+	} {
+		if len(raw) > 0 && string(raw) != "null" {
+			out[purpose] = raw
+		}
+	}
+	return out
+}
+
+// CrossSigningNeedsAuth reports whether an upload has to go through
+// user-interactive authentication.
+//
+// The spec exempts two cases: a user who has no master key yet — a new
+// account setting up encryption, with nothing to take over — and an upload
+// that contains nothing but the keys already stored, which is what a
+// client does when it is unsure whether its first attempt landed.
+// Replacing a master key is the one thing that would let somebody holding
+// a stolen access token pass as the user to everyone who has verified
+// them, so everything else needs the password.
+func (k *API) CrossSigningNeedsAuth(ctx context.Context, userID string, req CrossSigningRequest) (bool, error) {
+	existing, err := k.store.CrossSigningKeys(ctx, userID)
+	if err != nil {
+		return false, err
+	}
+	if _, ok := existing[storage.CrossSigningMaster]; !ok {
+		return false, nil
+	}
+	for purpose, raw := range req.keys() {
+		encoded, err := canonicaljson.Encode(raw)
+		if err != nil {
+			return false, invalidParamf("the %s key is not valid JSON: %v", purpose, err)
+		}
+		if string(encoded) != string(existing[purpose]) {
+			return true, nil
+		}
+	}
+	return false, nil
+}
+
+// checkCrossSigningKey validates one uploaded key and returns it in
+// canonical form with its public key.
+func checkCrossSigningKey(userID, purpose string, raw json.RawMessage) ([]byte, string, error) {
+	var key crossSigningKeyObject
+	if err := json.Unmarshal(raw, &key); err != nil {
+		return nil, "", invalidParamf("the %s key is malformed: %v", purpose, err)
+	}
+	if key.UserID != userID {
+		return nil, "", invalidParamf("the %s key's user_id must be %s", purpose, userID)
+	}
+	if !slices.Contains(key.Usage, purpose) {
+		return nil, "", invalidParamf("the %s key's usage must include %q", purpose, purpose)
+	}
+	pub, ok := key.ed25519Key()
+	if !ok {
+		return nil, "", invalidParamf("the %s key must hold exactly one ed25519 key, named by its own public key", purpose)
+	}
+	if _, err := publicKey(pub); err != nil {
+		return nil, "", invalidParamf("the %s key: %v", purpose, err)
+	}
+	encoded, err := canonical(raw)
+	if err != nil {
+		return nil, "", err
+	}
+	if encoded, err = withoutField(encoded, "unsigned"); err != nil {
+		return nil, "", err
+	}
+	return encoded, pub, nil
+}
+
+// UploadCrossSigningKeys stores a user's cross-signing keys. The caller has
+// already done the user-interactive authentication CrossSigningNeedsAuth
+// asked for.
+func (k *API) UploadCrossSigningKeys(ctx context.Context, userID string, req CrossSigningRequest) error {
+	uploaded := req.keys()
+	if len(uploaded) == 0 {
+		return missingParamf("at least one of master_key, self_signing_key and user_signing_key is required")
+	}
+
+	existing, err := k.store.CrossSigningKeys(ctx, userID)
+	if err != nil {
+		return err
+	}
+	oldMasterPub := ""
+	if raw, ok := existing[storage.CrossSigningMaster]; ok {
+		var key crossSigningKeyObject
+		if json.Unmarshal(raw, &key) == nil {
+			oldMasterPub, _ = key.ed25519Key()
+		}
+	}
+
+	writes := map[string][]byte{}
+	masterPub := oldMasterPub
+	if raw, ok := uploaded[storage.CrossSigningMaster]; ok {
+		encoded, pub, err := checkCrossSigningKey(userID, storage.CrossSigningMaster, raw)
+		if err != nil {
+			return err
+		}
+		writes[storage.CrossSigningMaster] = encoded
+		masterPub = pub
+	}
+	if masterPub == "" {
+		return missingParamf("a master key is required before any key it signs")
+	}
+
+	for _, purpose := range []string{storage.CrossSigningSelfSigning, storage.CrossSigningUserSigning} {
+		raw, ok := uploaded[purpose]
+		if !ok {
+			// A new master key orphans the keys the old one signed: they
+			// would go on being served, signed by a key that is no
+			// longer the user's.
+			if masterPub != oldMasterPub {
+				if _, had := existing[purpose]; had {
+					writes[purpose] = nil
+				}
+			}
+			continue
+		}
+		encoded, _, err := checkCrossSigningKey(userID, purpose, raw)
+		if err != nil {
+			return err
+		}
+		if err := verify(encoded, userID, masterPub, masterPub); err != nil {
+			return invalidSignaturef("the %s key is not signed by the master key", purpose)
+		}
+		writes[purpose] = encoded
+	}
+
+	if _, err := k.store.SetCrossSigningKeys(ctx, userID, writes); err != nil {
+		return err
+	}
+	k.announceSigningKeys(ctx, userID)
+	return nil
+}
+
+// SignatureFailure is why one signature of an upload was refused.
+type SignatureFailure struct {
+	ErrCode string `json:"errcode"`
+	Error   string `json:"error"`
+}
+
+// The errcodes a signature upload reports per key.
+const (
+	errCodeInvalidParam     = "M_INVALID_PARAM"
+	errCodeInvalidSignature = "M_INVALID_SIGNATURE"
+	errCodeNotFound         = "M_NOT_FOUND"
+)
+
+// signatureTarget is a key a signature upload is about, and which of the
+// uploader's keys may sign it.
+type signatureTarget struct {
+	// JSON is the key as stored, which is what a signature has to verify
+	// against: a client sending a modified copy with a valid signature
+	// over its own modification signs nothing that is here.
+	JSON []byte
+
+	// Signers maps the key IDs allowed to sign it to their public keys.
+	Signers map[string]string
+}
+
+// UploadSignatures stores signatures from POST /keys/signatures/upload:
+// user ID, then key ID (a device ID, or a master key's public key), then
+// the key with the uploader's signatures on it.
+//
+// Three kinds are accepted, which are the three the spec describes: the
+// uploader's self-signing key signing one of their devices, one of their
+// devices signing their master key, and their user-signing key signing
+// another user's master key. A signature that is none of those — the
+// device's own signature still on its keys, typically — is ignored rather
+// than refused, because clients send the whole signed object back.
+func (k *API) UploadSignatures(ctx context.Context, userID string, body map[string]map[string]json.RawMessage) (map[string]map[string]SignatureFailure, error) {
+	failures := map[string]map[string]SignatureFailure{}
+	fail := func(targetUser, keyID, code, msg string) {
+		if failures[targetUser] == nil {
+			failures[targetUser] = map[string]SignatureFailure{}
+		}
+		failures[targetUser][keyID] = SignatureFailure{ErrCode: code, Error: msg}
+	}
+
+	own, err := k.ownKeys(ctx, userID)
+	if err != nil {
+		return nil, err
+	}
+
+	var accepted []storage.KeySignature
+	for _, targetUser := range sortedKeys(body) {
+		for _, keyID := range sortedKeys(body[targetUser]) {
+			raw := body[targetUser][keyID]
+			target, err := k.signatureTarget(ctx, userID, own, targetUser, keyID, raw)
+			if err != nil {
+				code := errCodeInvalidParam
+				if errors.Is(err, errUnknownKey) {
+					code = errCodeNotFound
+				}
+				fail(targetUser, keyID, code, err.Error())
+				continue
+			}
+
+			sigs, err := signaturesOf(raw)
+			if err != nil {
+				fail(targetUser, keyID, errCodeInvalidParam, err.Error())
+				continue
+			}
+			for _, signerKeyID := range sortedKeys(sigs[userID]) {
+				pub, allowed := target.Signers[signerKeyID]
+				if !allowed {
+					continue
+				}
+				signature := sigs[userID][signerKeyID]
+				if err := checkSignature(target.JSON, userID, signerKeyID, pub, signature); err != nil {
+					fail(targetUser, keyID, errCodeInvalidSignature, err.Error())
+					continue
+				}
+				accepted = append(accepted, storage.KeySignature{
+					TargetUserID: targetUser, TargetKeyID: keyID,
+					SignerUserID: userID, SignerKeyID: signerKeyID, Signature: signature,
+				})
+			}
+		}
+	}
+
+	updates, err := k.store.AddSignatures(ctx, accepted)
+	if err != nil {
+		return nil, err
+	}
+	k.announceSignatures(ctx, userID, own, updates)
+	return failures, nil
+}
+
+// ownKeys is what UploadSignatures needs to know about the uploader.
+type ownKeys struct {
+	masterPub      string
+	selfSigningPub string
+	userSigningPub string
+	master         []byte
+}
+
+func (k *API) ownKeys(ctx context.Context, userID string) (ownKeys, error) {
+	stored, err := k.store.CrossSigningKeys(ctx, userID)
+	if err != nil {
+		return ownKeys{}, err
+	}
+	var out ownKeys
+	pubOf := func(purpose string) string {
+		var key crossSigningKeyObject
+		if json.Unmarshal(stored[purpose], &key) != nil {
+			return ""
+		}
+		pub, _ := key.ed25519Key()
+		return pub
+	}
+	out.master = stored[storage.CrossSigningMaster]
+	out.masterPub = pubOf(storage.CrossSigningMaster)
+	out.selfSigningPub = pubOf(storage.CrossSigningSelfSigning)
+	out.userSigningPub = pubOf(storage.CrossSigningUserSigning)
+	return out, nil
+}
+
+// errUnknownKey is a signature of a key this server does not have.
+var errUnknownKey = errors.New("no such key")
+
+// signatureTarget works out what a signature upload names and who may
+// sign it.
+func (k *API) signatureTarget(ctx context.Context, userID string, own ownKeys, targetUser, keyID string, raw json.RawMessage) (signatureTarget, error) {
+	if targetUser == userID {
+		if own.masterPub != "" && keyID == own.masterPub {
+			// The user's master key, signed by one of their devices.
+			devices, err := k.store.DeviceKeys(ctx, userID, nil)
+			if err != nil {
+				return signatureTarget{}, err
+			}
+			signers := map[string]string{}
+			for _, d := range devices {
+				var keys deviceKeysObject
+				if json.Unmarshal(d.JSON, &keys) != nil {
+					continue
+				}
+				if pub, ok := keys.Keys["ed25519:"+d.DeviceID]; ok {
+					signers["ed25519:"+d.DeviceID] = pub
+				}
+			}
+			return signatureTarget{JSON: own.master, Signers: signers}, nil
+		}
+
+		// One of the user's devices, signed by their self-signing key.
+		devices, err := k.store.DeviceKeys(ctx, userID, []string{keyID})
+		if err != nil {
+			return signatureTarget{}, err
+		}
+		if len(devices) == 0 {
+			return signatureTarget{}, errUnknownKey
+		}
+		if own.selfSigningPub == "" {
+			return signatureTarget{}, missingParamf("a self-signing key has to be uploaded before it can sign a device")
+		}
+		return signatureTarget{
+			JSON:    devices[0].JSON,
+			Signers: map[string]string{"ed25519:" + own.selfSigningPub: own.selfSigningPub},
+		}, nil
+	}
+
+	// Another user's master key, signed by the user-signing key.
+	if own.userSigningPub == "" {
+		return signatureTarget{}, missingParamf("a user-signing key has to be uploaded before it can sign another user")
+	}
+	signers := map[string]string{"ed25519:" + own.userSigningPub: own.userSigningPub}
+
+	if k.isLocal(targetUser) {
+		stored, err := k.store.CrossSigningKeys(ctx, targetUser)
+		if err != nil {
+			return signatureTarget{}, err
+		}
+		var key crossSigningKeyObject
+		raw, ok := stored[storage.CrossSigningMaster]
+		if !ok || json.Unmarshal(raw, &key) != nil {
+			return signatureTarget{}, errUnknownKey
+		}
+		if pub, _ := key.ed25519Key(); pub != keyID {
+			return signatureTarget{}, errUnknownKey
+		}
+		return signatureTarget{JSON: raw, Signers: signers}, nil
+	}
+
+	// A remote user's master key is not stored here, so the signature is
+	// checked against the copy the client sent, which has to be a master
+	// key of that user named by keyID. Nothing is lost by trusting the
+	// copy: the signature only ever goes back to the user who made it.
+	encoded, pub, err := checkCrossSigningKey(targetUser, storage.CrossSigningMaster, raw)
+	if err != nil {
+		return signatureTarget{}, err
+	}
+	if pub != keyID {
+		return signatureTarget{}, invalidParamf("the key is named %s but holds %s", keyID, pub)
+	}
+	return signatureTarget{JSON: encoded, Signers: signers}, nil
+}
+
+// checkSignature verifies one signature against a key, with every other
+// signature on the key taken off: the signature covers neither.
+func checkSignature(target []byte, userID, signerKeyID, pub, signature string) error {
+	obj, err := object(target)
+	if err != nil {
+		return err
+	}
+	delete(obj, "unsigned")
+	obj["signatures"], err = json.Marshal(map[string]map[string]string{userID: {signerKeyID: signature}})
+	if err != nil {
+		return err
+	}
+	msg, err := canonicaljson.Marshal(obj)
+	if err != nil {
+		return err
+	}
+	return verify(msg, userID, strings.TrimPrefix(signerKeyID, "ed25519:"), pub)
+}
+
+// announceSignatures tells whoever has to know about the signatures just
+// stored. A signature of one of the uploader's devices goes out as an
+// update of that device and one of their master key as a signing key
+// update. A signature of somebody else's master key goes nowhere but the
+// uploader's own devices: who a user has verified is nobody else's
+// business.
+func (k *API) announceSignatures(ctx context.Context, userID string, own ownKeys, updates []storage.DeviceListUpdate) {
+	var (
+		devices    []storage.DeviceListUpdate
+		deviceIDs  []string
+		masterSeen bool
+	)
+	for _, u := range updates {
+		switch {
+		case u.UserID != userID:
+			if k.notifier != nil {
+				k.notifier.NotifyUsers([]string{userID})
+			}
+		case u.DeviceID == own.masterPub:
+			masterSeen = true
+		default:
+			devices = append(devices, u)
+			deviceIDs = append(deviceIDs, u.DeviceID)
+		}
+	}
+	if len(devices) > 0 {
+		published, err := k.publishedDeviceKeys(ctx, userID, deviceIDs)
+		if err != nil {
+			k.log.Warn("could not read back signed device keys", "user_id", userID, "error", err)
+		}
+		k.announceDevices(ctx, userID, devices, published)
+	}
+	if masterSeen {
+		k.announceSigningKeys(ctx, userID)
+	}
+}

server/internal/keyserver/devicelists.go

@@ -0,0 +1,162 @@
+package keyserver
+
+import (
+	"context"
+	"encoding/json"
+	"slices"
+
+	"github.com/dariuszjaniak/warp/server/internal/federation"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/id"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// The writing side of device list tracking: telling everyone who shares a
+// room with a user that the user's keys changed.
+//
+// Locally that is a wake-up. The change is already in the device list
+// stream, moved in the same transaction as the keys themselves, and every
+// sync works out from the stream whose changes concern it; all a sync
+// blocked in a long poll needs is to be told to look. Remotely it is an
+// EDU to every server with a user in one of those rooms, which is how a
+// server that has the user's keys cached knows to drop them.
+
+// sharers is who shares a room with a user, the user included.
+func (k *API) sharers(ctx context.Context, userID string) []string {
+	out := []string{userID}
+	if k.rooms == nil {
+		return out
+	}
+	users, err := k.rooms.UsersSharingRooms(ctx, userID)
+	if err != nil {
+		// A missed wake-up costs a client one sync timeout, and a missed
+		// EDU costs a remote server one stale key until the next change;
+		// neither is worth failing the upload that caused it.
+		k.log.Warn("could not read who shares a room with a user", "user_id", userID, "error", err)
+		return out
+	}
+	for _, u := range users {
+		if !slices.Contains(out, u) {
+			out = append(out, u)
+		}
+	}
+	return out
+}
+
+// changed reacts to a move of a user's device list: it wakes the local
+// syncs it concerns and returns the other servers that have to hear of
+// it.
+func (k *API) changed(ctx context.Context, userID string) []string {
+	users := k.sharers(ctx, userID)
+
+	var (
+		local   []string
+		servers []string
+	)
+	for _, u := range users {
+		server, err := id.ServerNameOf(u)
+		switch {
+		case err != nil:
+			continue
+		case server == k.serverName:
+			local = append(local, u)
+		case !slices.Contains(servers, server):
+			servers = append(servers, server)
+		}
+	}
+	if k.notifier != nil {
+		k.notifier.NotifyUsers(local)
+	}
+	slices.Sort(servers)
+	return servers
+}
+
+// federating reports whether there is a running queue to send EDUs to.
+func (k *API) federating() bool {
+	return k.outbound != nil && k.outbound.Running()
+}
+
+// announceDevices tells everyone about moves of a local user's devices:
+// keys uploaded, changed or deleted. keys holds the current keys of the
+// devices that still have them; a device missing from it is announced as
+// deleted.
+func (k *API) announceDevices(ctx context.Context, userID string, updates []storage.DeviceListUpdate, keys map[string]json.RawMessage) {
+	if len(updates) == 0 {
+		return
+	}
+	servers := k.changed(ctx, userID)
+	if len(servers) == 0 || !k.federating() {
+		return
+	}
+	for _, u := range updates {
+		content := federation.DeviceListUpdateContent{
+			UserID:   userID,
+			DeviceID: u.DeviceID,
+			StreamID: u.StreamPos,
+		}
+		if u.PrevPos > 0 {
+			content.PrevID = []int64{u.PrevPos}
+		}
+		if raw, ok := keys[u.DeviceID]; ok {
+			content.Keys = raw
+		} else {
+			content.Deleted = true
+		}
+		encoded, err := json.Marshal(content)
+		if err != nil {
+			continue
+		}
+		k.outbound.QueueEDU(ctx, servers, federation.EDU{Type: federation.EDUDeviceListUp, Content: encoded})
+	}
+}
+
+// announceSigningKeys tells everyone a local user's cross-signing keys
+// changed: the master key, or a signature on it, or the self-signing key.
+// The user-signing key is not announced — nobody but its owner ever sees
+// it.
+func (k *API) announceSigningKeys(ctx context.Context, userID string) {
+	servers := k.changed(ctx, userID)
+	if len(servers) == 0 || !k.federating() {
+		return
+	}
+	master, selfSigning, _, err := k.crossSigningKeys(ctx, userID, "")
+	if err != nil {
+		k.log.Warn("could not read cross-signing keys to announce them", "user_id", userID, "error", err)
+		return
+	}
+	encoded, err := json.Marshal(federation.SigningKeyUpdateContent{
+		UserID: userID, MasterKey: master, SelfSigningKey: selfSigning,
+	})
+	if err != nil {
+		return
+	}
+	k.outbound.QueueEDU(ctx, servers, federation.EDU{Type: federation.EDUSigningKeys, Content: encoded})
+}
+
+// RemoteKeysChanged takes in an m.device_list_update or
+// m.signing_key_update from another server: one of its users' keys
+// changed.
+//
+// Warp does not cache other servers' keys — a query for them is answered
+// by asking their server — so there is nothing to invalidate. What there
+// is to do is tell local clients, which do cache them, by moving the
+// user's position in the device list stream. That only happens when a
+// local user shares a room with them: an update about a stranger concerns
+// nobody here, and recording it would let any server grow the table with
+// users of its own invention.
+func (k *API) RemoteKeysChanged(ctx context.Context, userID string) error {
+	if k.isLocal(userID) || k.rooms == nil {
+		return nil
+	}
+	users, err := k.rooms.UsersSharingRooms(ctx, userID)
+	if err != nil {
+		return err
+	}
+	if !slices.ContainsFunc(users, k.isLocal) {
+		return nil
+	}
+	if _, err := k.store.MarkDeviceListsChanged(ctx, []string{userID}); err != nil {
+		return err
+	}
+	k.changed(ctx, userID)
+	return nil
+}

server/internal/keyserver/json.go

@@ -0,0 +1,160 @@
+package keyserver
+
+import (
+	"crypto/ed25519"
+	"encoding/json"
+	"fmt"
+	"strings"
+
+	"github.com/dariuszjaniak/warp/server/internal/matrix/canonicaljson"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// The shapes of the keys a client uploads, as far as the server reads
+// them. Everything else in a key is kept as it was sent: a key is a signed
+// object, and the server re-encoding a field it did not understand would
+// break the signature.
+
+// deviceKeysObject is a device's identity keys.
+type deviceKeysObject struct {
+	UserID     string            `json:"user_id"`
+	DeviceID   string            `json:"device_id"`
+	Algorithms []string          `json:"algorithms"`
+	Keys       map[string]string `json:"keys"`
+}
+
+// crossSigningKeyObject is a cross-signing key.
+type crossSigningKeyObject struct {
+	UserID string            `json:"user_id"`
+	Usage  []string          `json:"usage"`
+	Keys   map[string]string `json:"keys"`
+}
+
+// canonical re-encodes a key in canonical JSON, which is what the store
+// compares when it decides whether an upload is the same key again.
+func canonical(raw json.RawMessage) ([]byte, error) {
+	out, err := canonicaljson.Encode(raw)
+	if err != nil {
+		return nil, invalidParamf("a key is not valid JSON: %v", err)
+	}
+	return out, nil
+}
+
+// object decodes a JSON object into its fields.
+func object(raw []byte) (map[string]json.RawMessage, error) {
+	var obj map[string]json.RawMessage
+	if err := json.Unmarshal(raw, &obj); err != nil || obj == nil {
+		return nil, invalidParamf("a key must be a JSON object")
+	}
+	return obj, nil
+}
+
+// withoutField drops one field from a JSON object, in canonical form.
+func withoutField(raw []byte, field string) ([]byte, error) {
+	obj, err := object(raw)
+	if err != nil {
+		return nil, err
+	}
+	delete(obj, field)
+	return canonicaljson.Marshal(obj)
+}
+
+// signaturesOf reads a key's signatures block.
+func signaturesOf(raw []byte) (signing.Signatures, error) {
+	sigs, err := signing.ExtractSignatures(raw)
+	if err != nil {
+		return nil, invalidParamf("a key's signatures are malformed: %v", err)
+	}
+	return sigs, nil
+}
+
+// withSignatures merges stored signatures into a key, and optionally sets
+// its unsigned.device_display_name. Signatures the key already carries are
+// kept; a stored one under the same signer and key ID wins, since it is
+// the more recent.
+func withSignatures(raw []byte, sigs []storage.KeySignature, displayName string) (json.RawMessage, error) {
+	if len(sigs) == 0 && displayName == "" {
+		return raw, nil
+	}
+	obj, err := object(raw)
+	if err != nil {
+		return nil, err
+	}
+	if len(sigs) > 0 {
+		merged, err := signaturesOf(raw)
+		if err != nil {
+			return nil, err
+		}
+		if merged == nil {
+			merged = signing.Signatures{}
+		}
+		for _, sig := range sigs {
+			if merged[sig.SignerUserID] == nil {
+				merged[sig.SignerUserID] = map[string]string{}
+			}
+			merged[sig.SignerUserID][sig.SignerKeyID] = sig.Signature
+		}
+		if obj["signatures"], err = json.Marshal(merged); err != nil {
+			return nil, err
+		}
+	}
+	if displayName != "" {
+		unsigned := map[string]json.RawMessage{}
+		if existing, ok := obj["unsigned"]; ok {
+			_ = json.Unmarshal(existing, &unsigned)
+		}
+		if unsigned["device_display_name"], err = json.Marshal(displayName); err != nil {
+			return nil, err
+		}
+		if obj["unsigned"], err = json.Marshal(unsigned); err != nil {
+			return nil, err
+		}
+	}
+	return canonicaljson.Marshal(obj)
+}
+
+// ed25519Key returns the one ed25519 key of a cross-signing key, by its
+// unpadded base64 public key, which is also its key ID.
+func (o crossSigningKeyObject) ed25519Key() (string, bool) {
+	if len(o.Keys) != 1 {
+		return "", false
+	}
+	for keyID, value := range o.Keys {
+		algorithm, name, ok := signing.SplitKeyID(keyID)
+		if !ok || algorithm != "ed25519" || name != value {
+			return "", false
+		}
+		return value, true
+	}
+	return "", false
+}
+
+// publicKey decodes an unpadded base64 ed25519 public key.
+func publicKey(encoded string) (ed25519.PublicKey, error) {
+	raw, err := signing.DecodeBase64(encoded)
+	if err != nil || len(raw) != ed25519.PublicKeySize {
+		return nil, fmt.Errorf("%q is not an ed25519 public key", encoded)
+	}
+	return ed25519.PublicKey(raw), nil
+}
+
+// verify checks that a key carries a valid signature by userID's key
+// "ed25519:"+keyName, whose public key is pub.
+func verify(raw []byte, userID, keyName, pub string) error {
+	key, err := publicKey(pub)
+	if err != nil {
+		return invalidSignaturef("%v", err)
+	}
+	if err := signing.VerifyJSON(userID, "ed25519:"+keyName, key, raw); err != nil {
+		return invalidSignaturef("%v", err)
+	}
+	return nil
+}
+
+// validKeyID reports whether a one-time key ID has the "algorithm:id"
+// shape.
+func validKeyID(keyID string) bool {
+	algorithm, name, ok := strings.Cut(keyID, ":")
+	return ok && algorithm != "" && name != ""
+}

server/internal/keyserver/keyserver.go

@@ -0,0 +1,223 @@
+// Package keyserver holds what end-to-end encryption needs a homeserver to
+// keep for its users' devices: their identity keys, their one-time and
+// fallback keys, the user's cross-signing keys and the signatures between
+// all of those, the server-side backup of room keys, and the device list
+// stream that tells other devices when any of it changed.
+//
+// None of it is something the server can use. Every key here is public, or
+// encrypted to a key the server never sees; what the server provides is a
+// place to publish them, the guarantee that a one-time key is handed out
+// once, and the bookkeeping of who has to be told about a change. That is
+// the whole of the design, and the reason the package is mostly checks:
+// the server cannot tell a good key from a bad one, but it can refuse a
+// key uploaded for somebody else's device, a cross-signing key that its
+// master key did not sign, and a signature that does not verify.
+//
+// The transport of encrypted messages between devices — to-device
+// messaging — is the syncapi's, because a device only ever receives one in
+// a /sync. So is the reporting of device list changes, for the same
+// reason: whose keys changed is one of the streams of a sync response.
+// What is here is the writing of that stream.
+//
+// Phase 7 of PLAN.md.
+package keyserver
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"log/slog"
+	"time"
+
+	"github.com/dariuszjaniak/warp/server/internal/federation"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/id"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Errors callers have to tell apart. Each is wrapped with the detail of
+// what was wrong, so a handler can answer with the right errcode and still
+// tell the client which key it was.
+var (
+	// ErrInvalidParam is a request that is well-formed JSON but not a
+	// valid one: keys for another user's device, a key ID without an
+	// algorithm, a cross-signing key with the wrong usage.
+	ErrInvalidParam = errors.New("invalid parameter")
+
+	// ErrMissingParam is a request that leaves out something it needs,
+	// such as a self-signing key with no master key to have signed it.
+	ErrMissingParam = errors.New("missing parameter")
+
+	// ErrInvalidSignature is a signature that does not verify.
+	ErrInvalidSignature = errors.New("invalid signature")
+
+	// ErrKeyConflict is a one-time key uploaded under an ID that already
+	// holds a different one.
+	ErrKeyConflict = storage.ErrKeyConflict
+
+	// ErrNoSuchBackup is a key backup version that does not exist.
+	ErrNoSuchBackup = storage.ErrNoSuchBackup
+
+	// ErrNoSuchBackupKey is a room key that is not in the backup.
+	ErrNoSuchBackupKey = storage.ErrNoSuchBackupKey
+)
+
+// WrongBackupVersionError is a write to a key backup version that is not
+// the user's current one. The spec answers it with the current version, so
+// that a client whose backup was replaced from another device finds out
+// which one to use.
+type WrongBackupVersionError struct {
+	Current string
+}
+
+func (e *WrongBackupVersionError) Error() string {
+	return fmt.Sprintf("backup version is not the current one (%s)", e.Current)
+}
+
+// Rooms says who shares a room with whom, which is who has to be told when
+// a user's keys change. *roomserver.API satisfies it.
+type Rooms interface {
+	UsersSharingRooms(ctx context.Context, userID string) ([]string, error)
+}
+
+// Notifier wakes the long polls of users. *syncapi.Notifier satisfies it.
+type Notifier interface {
+	NotifyUsers(userIDs []string)
+}
+
+// Devices is the part of the userapi the keyserver reads: a user's devices,
+// for their display names and to tell a device that still exists from one
+// that does not. *userapi.API satisfies it.
+type Devices interface {
+	Devices(ctx context.Context, localpart string) ([]storage.Device, error)
+}
+
+// Remote asks other servers about their users' keys. *federation.Client
+// satisfies it; it refuses every request while federation is off, which is
+// what keeps a key query from reaching out with the switch down.
+type Remote interface {
+	QueryUserKeys(ctx context.Context, destination string, req federation.UserKeysQueryRequest) (federation.UserKeysQueryResponse, error)
+	ClaimUserKeys(ctx context.Context, destination string, req federation.UserKeysClaimRequest) (federation.UserKeysClaimResponse, error)
+}
+
+// Outbound queues EDUs to other servers. *federationapi.Queue satisfies it.
+type Outbound interface {
+	Running() bool
+	QueueEDU(ctx context.Context, destinations []string, edu federation.EDU)
+}
+
+// API is the keyserver component.
+type API struct {
+	serverName string
+	store      storage.KeyStore
+	rooms      Rooms
+	notifier   Notifier
+	devices    Devices
+	remote     Remote
+	outbound   Outbound
+	log        *slog.Logger
+}
+
+// Options configure the component.
+type Options struct {
+	// ServerName is the Matrix server name.
+	ServerName string
+
+	// Store is the key half of storage.
+	Store storage.KeyStore
+
+	// Rooms says who shares a room with a user. Nil means nobody does,
+	// which is only useful in a test.
+	Rooms Rooms
+
+	// Notifier wakes the syncs a key change concerns. Nil wakes nobody.
+	Notifier Notifier
+
+	// Devices is the userapi.
+	Devices Devices
+
+	// Remote asks other servers for their users' keys. Nil fails every
+	// remote user in a query or claim as unreachable.
+	Remote Remote
+
+	// Outbound sends device list updates to other servers. Nil, or a
+	// queue that is not running, sends nothing.
+	Outbound Outbound
+
+	Logger *slog.Logger
+}
+
+// New builds the component.
+func New(opts Options) (*API, error) {
+	switch {
+	case opts.ServerName == "":
+		return nil, errors.New("keyserver: ServerName must not be empty")
+	case opts.Store == nil:
+		return nil, errors.New("keyserver: Store must not be nil")
+	case opts.Devices == nil:
+		return nil, errors.New("keyserver: Devices must not be nil")
+	}
+	if opts.Logger == nil {
+		opts.Logger = slog.Default()
+	}
+	return &API{
+		serverName: opts.ServerName,
+		store:      opts.Store,
+		rooms:      opts.Rooms,
+		notifier:   opts.Notifier,
+		devices:    opts.Devices,
+		remote:     opts.Remote,
+		outbound:   opts.Outbound,
+		log:        opts.Logger,
+	}, nil
+}
+
+// SetOutbound connects the federation queue, which is built after the
+// keyserver's own dependencies. Nil disconnects it.
+func (k *API) SetOutbound(o Outbound) { k.outbound = o }
+
+// isLocal reports whether a user belongs to this server.
+func (k *API) isLocal(userID string) bool {
+	server, err := id.ServerNameOf(userID)
+	return err == nil && server == k.serverName
+}
+
+// localpartOf returns the localpart of a local user.
+func (k *API) localpartOf(userID string) (string, bool) {
+	localpart, server, err := id.SplitUserID(userID)
+	if err != nil || server != k.serverName {
+		return "", false
+	}
+	return localpart, true
+}
+
+// defaultRemoteTimeout is how long a key query or claim waits for other
+// servers when the client did not say. A client asking for keys is about
+// to send a message, and a server that has not answered in ten seconds is
+// better reported as a failure than waited on.
+const defaultRemoteTimeout = 10 * time.Second
+
+// maxRemoteTimeout caps what a client may ask for.
+const maxRemoteTimeout = time.Minute
+
+// remoteTimeout clamps a client's timeout.
+func remoteTimeout(d time.Duration) time.Duration {
+	switch {
+	case d <= 0:
+		return defaultRemoteTimeout
+	case d > maxRemoteTimeout:
+		return maxRemoteTimeout
+	}
+	return d
+}
+
+func invalidParamf(format string, args ...any) error {
+	return fmt.Errorf("%w: %s", ErrInvalidParam, fmt.Sprintf(format, args...))
+}
+
+func missingParamf(format string, args ...any) error {
+	return fmt.Errorf("%w: %s", ErrMissingParam, fmt.Sprintf(format, args...))
+}
+
+func invalidSignaturef(format string, args ...any) error {
+	return fmt.Errorf("%w: %s", ErrInvalidSignature, fmt.Sprintf(format, args...))
+}

server/internal/keyserver/keyserver_test.go

@@ -0,0 +1,623 @@
+package keyserver_test
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"path/filepath"
+	"slices"
+	"sync"
+	"testing"
+
+	"github.com/dariuszjaniak/warp/server/internal/federation"
+	"github.com/dariuszjaniak/warp/server/internal/keyserver"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+	_ "github.com/dariuszjaniak/warp/server/internal/storage/drivers"
+)
+
+const (
+	alice = "@alice:warp.example"
+	bob   = "@bob:warp.example"
+	carol = "@carol:remote.example"
+)
+
+// --- fakes --------------------------------------------------------------------
+
+type fakeRooms struct{ sharing map[string][]string }
+
+func (f fakeRooms) UsersSharingRooms(_ context.Context, userID string) ([]string, error) {
+	return f.sharing[userID], nil
+}
+
+type fakeNotifier struct {
+	mu    sync.Mutex
+	woken []string
+}
+
+func (n *fakeNotifier) NotifyUsers(userIDs []string) {
+	n.mu.Lock()
+	defer n.mu.Unlock()
+	n.woken = append(n.woken, userIDs...)
+}
+
+func (n *fakeNotifier) wasWoken(userID string) bool {
+	n.mu.Lock()
+	defer n.mu.Unlock()
+	return slices.Contains(n.woken, userID)
+}
+
+type fakeDevices struct{ byLocalpart map[string][]storage.Device }
+
+func (f fakeDevices) Devices(_ context.Context, localpart string) ([]storage.Device, error) {
+	return f.byLocalpart[localpart], nil
+}
+
+type sentEDU struct {
+	destinations []string
+	edu          federation.EDU
+}
+
+type fakeOutbound struct {
+	mu   sync.Mutex
+	sent []sentEDU
+}
+
+func (o *fakeOutbound) Running() bool { return true }
+
+func (o *fakeOutbound) QueueEDU(_ context.Context, destinations []string, edu federation.EDU) {
+	o.mu.Lock()
+	defer o.mu.Unlock()
+	o.sent = append(o.sent, sentEDU{destinations: destinations, edu: edu})
+}
+
+func (o *fakeOutbound) ofType(t string) []sentEDU {
+	o.mu.Lock()
+	defer o.mu.Unlock()
+	var out []sentEDU
+	for _, s := range o.sent {
+		if s.edu.Type == t {
+			out = append(out, s)
+		}
+	}
+	return out
+}
+
+type harness struct {
+	keys     *keyserver.API
+	store    storage.KeyStore
+	notifier *fakeNotifier
+	outbound *fakeOutbound
+}
+
+func newHarness(t *testing.T, remote keyserver.Remote) harness {
+	t.Helper()
+	path := filepath.ToSlash(filepath.Join(t.TempDir(), "warp.db"))
+	db, err := storage.Open(t.Context(), storage.Config{
+		Driver:           storage.DriverSQLite,
+		ConnectionString: "file:" + path + "?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)",
+	}, nil)
+	if err != nil {
+		t.Fatalf("storage.Open: %v", err)
+	}
+	t.Cleanup(func() { _ = db.Close() })
+	if err := db.Migrate(t.Context()); err != nil {
+		t.Fatalf("Migrate: %v", err)
+	}
+
+	h := harness{store: db.Keys(), notifier: &fakeNotifier{}, outbound: &fakeOutbound{}}
+	h.keys, err = keyserver.New(keyserver.Options{
+		ServerName: "warp.example",
+		Store:      db.Keys(),
+		Rooms: fakeRooms{sharing: map[string][]string{
+			alice: {alice, bob, carol},
+			bob:   {alice, bob},
+			carol: {alice, carol},
+		}},
+		Notifier: h.notifier,
+		Devices: fakeDevices{byLocalpart: map[string][]storage.Device{
+			"alice": {{Localpart: "alice", ID: "D1", DisplayName: "Alice's phone"}, {Localpart: "alice", ID: "D2"}},
+			"bob":   {{Localpart: "bob", ID: "B1"}},
+		}},
+		Remote:   remote,
+		Outbound: h.outbound,
+	})
+	if err != nil {
+		t.Fatalf("keyserver.New: %v", err)
+	}
+	return h
+}
+
+// --- keys ---------------------------------------------------------------------
+
+// newKey makes an ed25519 key named by its own public key, which is how a
+// cross-signing key is named; a device key is renamed to its device ID.
+func newKey(t *testing.T) *signing.KeyPair {
+	t.Helper()
+	kp, err := signing.Generate("")
+	if err != nil {
+		t.Fatal(err)
+	}
+	kp.ID = kp.PublicBase64()
+	return kp
+}
+
+func signed(t *testing.T, userID string, kp *signing.KeyPair, v any) json.RawMessage {
+	t.Helper()
+	raw, err := json.Marshal(v)
+	if err != nil {
+		t.Fatal(err)
+	}
+	out, err := signing.SignJSON(userID, kp, raw)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return out
+}
+
+// device is a device's identity key pair and its signed device_keys.
+func device(t *testing.T, userID, deviceID string) (*signing.KeyPair, json.RawMessage) {
+	t.Helper()
+	kp := newKey(t)
+	pub := kp.PublicBase64()
+	kp.ID = deviceID
+	return kp, signed(t, userID, kp, map[string]any{
+		"user_id":    userID,
+		"device_id":  deviceID,
+		"algorithms": []string{"m.olm.v1.curve25519-aes-sha2", "m.megolm.v1.aes-sha2"},
+		"keys": map[string]string{
+			"ed25519:" + deviceID:    pub,
+			"curve25519:" + deviceID: "curve-" + deviceID,
+		},
+	})
+}
+
+func crossSigningKey(userID, usage string, kp *signing.KeyPair) map[string]any {
+	return map[string]any{
+		"user_id": userID,
+		"usage":   []string{usage},
+		"keys":    map[string]string{"ed25519:" + kp.PublicBase64(): kp.PublicBase64()},
+	}
+}
+
+// crossSigning is a user's three cross-signing keys, uploaded.
+type crossSigning struct {
+	master, selfSigning, userSigning *signing.KeyPair
+	req                              keyserver.CrossSigningRequest
+}
+
+func newCrossSigning(t *testing.T, userID string) crossSigning {
+	t.Helper()
+	cs := crossSigning{master: newKey(t), selfSigning: newKey(t), userSigning: newKey(t)}
+	cs.req = keyserver.CrossSigningRequest{
+		MasterKey:      signed(t, userID, newKey(t), crossSigningKey(userID, "master", cs.master)),
+		SelfSigningKey: signed(t, userID, cs.master, crossSigningKey(userID, "self_signing", cs.selfSigning)),
+		UserSigningKey: signed(t, userID, cs.master, crossSigningKey(userID, "user_signing", cs.userSigning)),
+	}
+	// The master key's own signature above is by a throwaway key, which
+	// the server does not check; what it checks is that the master key
+	// signed the other two.
+	raw, _ := json.Marshal(crossSigningKey(userID, "master", cs.master))
+	cs.req.MasterKey = raw
+	return cs
+}
+
+func decodeObject(t *testing.T, raw json.RawMessage) map[string]any {
+	t.Helper()
+	var out map[string]any
+	if err := json.Unmarshal(raw, &out); err != nil {
+		t.Fatalf("decode %s: %v", raw, err)
+	}
+	return out
+}
+
+func signaturesBy(t *testing.T, raw json.RawMessage, userID string) map[string]any {
+	t.Helper()
+	sigs, _ := decodeObject(t, raw)["signatures"].(map[string]any)
+	out, _ := sigs[userID].(map[string]any)
+	return out
+}
+
+// --- tests --------------------------------------------------------------------
+
+func TestUploadAndQueryDeviceKeys(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+	_, d1 := device(t, alice, "D1")
+
+	counts, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		DeviceKeys: d1,
+		OneTimeKeys: map[string]json.RawMessage{
+			"signed_curve25519:AAA": json.RawMessage(`{"key":"a"}`),
+			"signed_curve25519:BBB": json.RawMessage(`{"key":"b"}`),
+		},
+		FallbackKeys: map[string]json.RawMessage{
+			"signed_curve25519:FFF": json.RawMessage(`{"key":"f","fallback":true}`),
+		},
+	})
+	if err != nil {
+		t.Fatalf("UploadKeys: %v", err)
+	}
+	if counts["signed_curve25519"] != 2 {
+		t.Errorf("one_time_key_counts = %v, want 2 signed_curve25519", counts)
+	}
+
+	// Everybody sharing a room hears of it: the local ones are woken, the
+	// remote server gets an m.device_list_update with the keys in it.
+	if !h.notifier.wasWoken(bob) {
+		t.Error("a user sharing a room was not woken")
+	}
+	updates := h.outbound.ofType(federation.EDUDeviceListUp)
+	if len(updates) != 1 || !slices.Equal(updates[0].destinations, []string{"remote.example"}) {
+		t.Fatalf("device list EDUs = %+v, want one to remote.example", updates)
+	}
+	var content federation.DeviceListUpdateContent
+	_ = json.Unmarshal(updates[0].edu.Content, &content)
+	if content.UserID != alice || content.DeviceID != "D1" || content.Deleted || len(content.Keys) == 0 || content.StreamID == 0 {
+		t.Errorf("EDU content = %+v", content)
+	}
+
+	resp, err := h.keys.QueryKeys(ctx, bob, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil}})
+	if err != nil {
+		t.Fatalf("QueryKeys: %v", err)
+	}
+	got, ok := resp.DeviceKeys[alice]["D1"]
+	if !ok {
+		t.Fatalf("query did not return D1: %+v", resp.DeviceKeys)
+	}
+	unsigned, _ := decodeObject(t, got)["unsigned"].(map[string]any)
+	if unsigned["device_display_name"] != "Alice's phone" {
+		t.Errorf("unsigned = %v, want the device's display name", unsigned)
+	}
+	if _, err := signing.SigningBytes(got); err != nil {
+		t.Errorf("queried keys are not a signed object: %v", err)
+	}
+
+	// Somebody else's device, or keys naming another device, are refused.
+	_, other := device(t, alice, "D2")
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{DeviceKeys: other}); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("keys for another device = %v, want ErrInvalidParam", err)
+	}
+	_, bobs := device(t, bob, "D1")
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{DeviceKeys: bobs}); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("keys for another user = %v, want ErrInvalidParam", err)
+	}
+
+	// The same one-time key again is fine, encoded differently or not; a
+	// different key under its ID is not.
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		OneTimeKeys: map[string]json.RawMessage{"signed_curve25519:AAA": json.RawMessage(`{ "key" : "a" }`)},
+	}); err != nil {
+		t.Errorf("re-uploading the same key: %v", err)
+	}
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		OneTimeKeys: map[string]json.RawMessage{"signed_curve25519:AAA": json.RawMessage(`{"key":"z"}`)},
+	}); !errors.Is(err, keyserver.ErrKeyConflict) {
+		t.Errorf("a conflicting key = %v, want ErrKeyConflict", err)
+	}
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		OneTimeKeys: map[string]json.RawMessage{"no-algorithm": json.RawMessage(`"x"`)},
+	}); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("a key ID with no algorithm = %v, want ErrInvalidParam", err)
+	}
+}
+
+func TestClaimKeys(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+	_, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		OneTimeKeys:  map[string]json.RawMessage{"signed_curve25519:AAA": json.RawMessage(`{"key":"a"}`)},
+		FallbackKeys: map[string]json.RawMessage{"signed_curve25519:FFF": json.RawMessage(`{"key":"f"}`)},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	claim := func() keyserver.ClaimResponse {
+		t.Helper()
+		resp, err := h.keys.ClaimKeys(ctx, keyserver.ClaimRequest{OneTimeKeys: map[string]map[string]string{
+			alice: {"D1": "signed_curve25519", "D2": "signed_curve25519"},
+			carol: {"C1": "signed_curve25519"},
+		}})
+		if err != nil {
+			t.Fatalf("ClaimKeys: %v", err)
+		}
+		return resp
+	}
+
+	first := claim()
+	if _, ok := first.OneTimeKeys[alice]["D1"]["signed_curve25519:AAA"]; !ok {
+		t.Errorf("first claim = %v, want the one-time key", first.OneTimeKeys)
+	}
+	if _, ok := first.OneTimeKeys[alice]["D2"]; ok {
+		t.Error("a device with no keys was given one")
+	}
+	if _, ok := first.Failures["remote.example"]; !ok {
+		t.Errorf("a server that cannot be asked is not a failure: %v", first.Failures)
+	}
+
+	second := claim()
+	if _, ok := second.OneTimeKeys[alice]["D1"]["signed_curve25519:FFF"]; !ok {
+		t.Errorf("second claim = %v, want the fallback key", second.OneTimeKeys)
+	}
+	if types, _ := h.keys.UnusedFallbackKeyTypes(ctx, alice, "D1"); len(types) != 0 {
+		t.Errorf("a claimed fallback key is still unused: %v", types)
+	}
+}
+
+// fakeRemote answers for remote.example.
+type fakeRemote struct{}
+
+func (fakeRemote) QueryUserKeys(_ context.Context, _ string, req federation.UserKeysQueryRequest) (federation.UserKeysQueryResponse, error) {
+	return federation.UserKeysQueryResponse{
+		DeviceKeys: map[string]map[string]json.RawMessage{
+			carol:                 {"C1": json.RawMessage(`{"device_id":"C1"}`)},
+			"@mallory:other.test": {"M1": json.RawMessage(`{}`)},
+		},
+	}, nil
+}
+
+func (fakeRemote) ClaimUserKeys(context.Context, string, federation.UserKeysClaimRequest) (federation.UserKeysClaimResponse, error) {
+	return federation.UserKeysClaimResponse{}, errors.New("unreachable")
+}
+
+func TestQueryAsksOtherServers(t *testing.T) {
+	h := newHarness(t, fakeRemote{})
+	resp, err := h.keys.QueryKeys(t.Context(), alice, keyserver.QueryRequest{DeviceKeys: map[string][]string{carol: nil}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, ok := resp.DeviceKeys[carol]["C1"]; !ok {
+		t.Errorf("remote keys = %v", resp.DeviceKeys)
+	}
+	if _, ok := resp.DeviceKeys["@mallory:other.test"]; ok {
+		t.Error("a server's answer about a user it was not asked about was passed on")
+	}
+
+	claimed, _ := h.keys.ClaimKeys(t.Context(), keyserver.ClaimRequest{OneTimeKeys: map[string]map[string]string{carol: {"C1": "signed_curve25519"}}})
+	if failure, ok := claimed.Failures["remote.example"].(map[string]any); !ok || failure["status"] != 503 {
+		t.Errorf("failures = %v, want a 503 for remote.example", claimed.Failures)
+	}
+}
+
+func TestCrossSigning(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+	cs := newCrossSigning(t, alice)
+
+	if needs, err := h.keys.CrossSigningNeedsAuth(ctx, alice, cs.req); err != nil || needs {
+		t.Fatalf("a first upload needs auth = %v, %v; want false", needs, err)
+	}
+	if err := h.keys.UploadCrossSigningKeys(ctx, alice, cs.req); err != nil {
+		t.Fatalf("UploadCrossSigningKeys: %v", err)
+	}
+	if updates := h.outbound.ofType(federation.EDUSigningKeys); len(updates) != 1 {
+		t.Errorf("signing key EDUs = %d, want 1", len(updates))
+	}
+
+	// The same keys again need no password; a new master key does.
+	if needs, _ := h.keys.CrossSigningNeedsAuth(ctx, alice, cs.req); needs {
+		t.Error("re-uploading the same keys needs auth")
+	}
+	replacement := newCrossSigning(t, alice)
+	if needs, _ := h.keys.CrossSigningNeedsAuth(ctx, alice, replacement.req); !needs {
+		t.Error("replacing the master key needs no auth")
+	}
+
+	// The user-signing key is only shown to its owner.
+	mine, _ := h.keys.QueryKeys(ctx, alice, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil}})
+	theirs, _ := h.keys.QueryKeys(ctx, bob, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil}})
+	if _, ok := mine.UserSigningKeys[alice]; !ok {
+		t.Error("the owner does not see their user-signing key")
+	}
+	if _, ok := theirs.UserSigningKeys[alice]; ok {
+		t.Error("another user sees the user-signing key")
+	}
+	if _, ok := theirs.MasterKeys[alice]; !ok {
+		t.Error("another user does not see the master key")
+	}
+	if _, ok := theirs.SelfSigningKeys[alice]; !ok {
+		t.Error("another user does not see the self-signing key")
+	}
+
+	// A self-signing key the master key did not sign is refused.
+	forged := keyserver.CrossSigningRequest{
+		SelfSigningKey: signed(t, alice, newKey(t), crossSigningKey(alice, "self_signing", newKey(t))),
+	}
+	if err := h.keys.UploadCrossSigningKeys(ctx, alice, forged); !errors.Is(err, keyserver.ErrInvalidSignature) {
+		t.Errorf("a self-signing key not signed by the master = %v, want ErrInvalidSignature", err)
+	}
+	wrongUsage := keyserver.CrossSigningRequest{
+		SelfSigningKey: signed(t, alice, cs.master, crossSigningKey(alice, "user_signing", newKey(t))),
+	}
+	if err := h.keys.UploadCrossSigningKeys(ctx, alice, wrongUsage); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("a key with the wrong usage = %v, want ErrInvalidParam", err)
+	}
+
+	// A new master key on its own drops the keys the old one signed.
+	if err := h.keys.UploadCrossSigningKeys(ctx, alice, keyserver.CrossSigningRequest{MasterKey: replacement.req.MasterKey}); err != nil {
+		t.Fatalf("replacing the master key: %v", err)
+	}
+	after, _ := h.keys.QueryKeys(ctx, alice, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil}})
+	if _, ok := after.SelfSigningKeys[alice]; ok {
+		t.Error("the old self-signing key survived a new master key")
+	}
+
+	// And nothing but a master key needs one first.
+	if err := h.keys.UploadCrossSigningKeys(ctx, bob, keyserver.CrossSigningRequest{
+		SelfSigningKey: signed(t, bob, newKey(t), crossSigningKey(bob, "self_signing", newKey(t))),
+	}); !errors.Is(err, keyserver.ErrMissingParam) {
+		t.Errorf("a self-signing key with no master = %v, want ErrMissingParam", err)
+	}
+}
+
+func TestSignatureUploads(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+
+	deviceKey, d1 := device(t, alice, "D1")
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{DeviceKeys: d1}); err != nil {
+		t.Fatal(err)
+	}
+	cs := newCrossSigning(t, alice)
+	if err := h.keys.UploadCrossSigningKeys(ctx, alice, cs.req); err != nil {
+		t.Fatal(err)
+	}
+	bobs := newCrossSigning(t, bob)
+	if err := h.keys.UploadCrossSigningKeys(ctx, bob, bobs.req); err != nil {
+		t.Fatal(err)
+	}
+
+	var stripped map[string]any
+	_ = json.Unmarshal(d1, &stripped)
+	delete(stripped, "signatures")
+
+	masterPub := cs.master.PublicBase64()
+	failures, err := h.keys.UploadSignatures(ctx, alice, map[string]map[string]json.RawMessage{
+		alice: {
+			// The device, by the self-signing key.
+			"D1": signed(t, alice, cs.selfSigning, stripped),
+			// The master key, by the device.
+			masterPub: signed(t, alice, deviceKey, crossSigningKey(alice, "master", cs.master)),
+		},
+		// Bob's master key, by Alice's user-signing key.
+		bob: {bobs.master.PublicBase64(): signed(t, alice, cs.userSigning, crossSigningKey(bob, "master", bobs.master))},
+	})
+	if err != nil || len(failures) != 0 {
+		t.Fatalf("UploadSignatures = %v, %v", failures, err)
+	}
+
+	resp, _ := h.keys.QueryKeys(ctx, alice, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil, bob: nil}})
+	if _, ok := signaturesBy(t, resp.DeviceKeys[alice]["D1"], alice)["ed25519:"+cs.selfSigning.PublicBase64()]; !ok {
+		t.Error("the device is not shown signed by the self-signing key")
+	}
+	if _, ok := signaturesBy(t, resp.MasterKeys[alice], alice)["ed25519:D1"]; !ok {
+		t.Error("the master key is not shown signed by the device")
+	}
+	if _, ok := signaturesBy(t, resp.MasterKeys[bob], alice)["ed25519:"+cs.userSigning.PublicBase64()]; !ok {
+		t.Error("Alice does not see her signature of Bob's master key")
+	}
+
+	// Who Alice has verified is between her and her devices.
+	asCarol, _ := h.keys.QueryKeys(ctx, carol, keyserver.QueryRequest{DeviceKeys: map[string][]string{bob: nil}})
+	if sigs := signaturesBy(t, asCarol.MasterKeys[bob], alice); len(sigs) != 0 {
+		t.Errorf("another user sees Alice's signature of Bob's key: %v", sigs)
+	}
+
+	// A signature that does not verify against the stored key is refused,
+	// and so is one of a key that does not exist.
+	stripped["algorithms"] = []string{"tampered"}
+	failures, err = h.keys.UploadSignatures(ctx, alice, map[string]map[string]json.RawMessage{
+		alice: {
+			"D1":   signed(t, alice, cs.selfSigning, stripped),
+			"NOPE": signed(t, alice, cs.selfSigning, stripped),
+		},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if failures[alice]["D1"].ErrCode != "M_INVALID_SIGNATURE" {
+		t.Errorf("a signature over a different object = %+v", failures[alice]["D1"])
+	}
+	if failures[alice]["NOPE"].ErrCode != "M_NOT_FOUND" {
+		t.Errorf("a signature of an unknown device = %+v", failures[alice]["NOPE"])
+	}
+}
+
+func TestDeletedDevicesLoseTheirKeys(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+	_, d1 := device(t, alice, "D1")
+	if _, err := h.keys.UploadKeys(ctx, alice, "D1", keyserver.UploadRequest{
+		DeviceKeys:  d1,
+		OneTimeKeys: map[string]json.RawMessage{"signed_curve25519:AAA": json.RawMessage(`{}`)},
+	}); err != nil {
+		t.Fatal(err)
+	}
+
+	h.keys.DevicesDeleted(ctx, "alice", []string{"D1"})
+
+	resp, _ := h.keys.QueryKeys(ctx, bob, keyserver.QueryRequest{DeviceKeys: map[string][]string{alice: nil}})
+	if len(resp.DeviceKeys[alice]) != 0 {
+		t.Errorf("keys of a deleted device are still served: %v", resp.DeviceKeys[alice])
+	}
+	if counts, _ := h.keys.OneTimeKeyCounts(ctx, alice, "D1"); counts["signed_curve25519"] != 0 {
+		t.Errorf("one-time keys of a deleted device survived: %v", counts)
+	}
+	updates := h.outbound.ofType(federation.EDUDeviceListUp)
+	var last federation.DeviceListUpdateContent
+	_ = json.Unmarshal(updates[len(updates)-1].edu.Content, &last)
+	if !last.Deleted || last.DeviceID != "D1" || len(last.PrevID) != 1 {
+		t.Errorf("the last device list EDU = %+v, want D1 deleted after its upload", last)
+	}
+}
+
+func TestRemoteKeyChangesOnlyConcernSharedUsers(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+
+	if err := h.keys.RemoteKeysChanged(ctx, "@stranger:remote.example"); err != nil {
+		t.Fatal(err)
+	}
+	if pos, _ := h.store.DeviceListPosition(ctx, "@stranger:remote.example"); pos != 0 {
+		t.Error("an update about a user nobody here shares a room with was recorded")
+	}
+
+	if err := h.keys.RemoteKeysChanged(ctx, carol); err != nil {
+		t.Fatal(err)
+	}
+	if pos, _ := h.store.DeviceListPosition(ctx, carol); pos == 0 {
+		t.Error("an update about a user sharing a room was not recorded")
+	}
+	if !h.notifier.wasWoken(alice) {
+		t.Error("the local user sharing a room was not woken")
+	}
+	if len(h.outbound.sent) != 0 {
+		t.Errorf("a remote update was echoed over federation: %+v", h.outbound.sent)
+	}
+}
+
+func TestKeyBackupVersions(t *testing.T) {
+	ctx := t.Context()
+	h := newHarness(t, nil)
+
+	v1, err := h.keys.CreateBackup(ctx, alice, "m.megolm_backup.v1.curve25519-aes-sha2", json.RawMessage(`{"public_key":"a"}`))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err := h.keys.CreateBackup(ctx, alice, "", json.RawMessage(`{}`)); !errors.Is(err, keyserver.ErrMissingParam) {
+		t.Errorf("a backup with no algorithm = %v", err)
+	}
+	if _, err := h.keys.CreateBackup(ctx, alice, "alg", json.RawMessage(`[1]`)); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("auth_data that is not an object = %v", err)
+	}
+
+	rooms := map[string]keyserver.RoomKeyBackup{"!r:x": {Sessions: map[string]keyserver.KeyBackupData{
+		"s1": {FirstMessageIndex: 1, SessionData: json.RawMessage(`{"ciphertext":"c"}`)},
+	}}}
+	result, err := h.keys.PutBackupKeys(ctx, alice, v1, rooms)
+	if err != nil || result.Count != 1 {
+		t.Fatalf("PutBackupKeys = %+v, %v", result, err)
+	}
+
+	v2, _ := h.keys.CreateBackup(ctx, alice, "m.megolm_backup.v1.curve25519-aes-sha2", json.RawMessage(`{"public_key":"b"}`))
+	var wrong *keyserver.WrongBackupVersionError
+	if _, err := h.keys.PutBackupKeys(ctx, alice, v1, rooms); !errors.As(err, &wrong) || wrong.Current != v2 {
+		t.Errorf("writing to an old version = %v, want WrongBackupVersionError{%s}", err, v2)
+	}
+	if _, err := h.keys.PutBackupKeys(ctx, alice, "99", rooms); !errors.Is(err, keyserver.ErrNoSuchBackup) {
+		t.Errorf("writing to a version that does not exist = %v", err)
+	}
+
+	if err := h.keys.UpdateBackup(ctx, alice, v2, "another.algorithm", "", json.RawMessage(`{}`)); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("changing a version's algorithm = %v", err)
+	}
+	if err := h.keys.UpdateBackup(ctx, alice, v2, "m.megolm_backup.v1.curve25519-aes-sha2", v1, json.RawMessage(`{}`)); !errors.Is(err, keyserver.ErrInvalidParam) {
+		t.Errorf("a body version that is not the path's = %v", err)
+	}
+
+	old, err := h.keys.BackupKeys(ctx, alice, v1, "", "")
+	if err != nil || len(old["!r:x"].Sessions) != 1 {
+		t.Errorf("an old version's keys = %v, %v; still readable", old, err)
+	}
+}

server/internal/keyserver/query.go

@@ -0,0 +1,465 @@
+package keyserver
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"slices"
+	"sync"
+	"time"
+
+	"github.com/dariuszjaniak/warp/server/internal/federation"
+	"github.com/dariuszjaniak/warp/server/internal/matrix/id"
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Reading keys: POST /keys/query and /keys/claim, and their federation
+// counterparts.
+
+// QueryRequest is the body of POST /keys/query.
+type QueryRequest struct {
+	// DeviceKeys is user ID to the device IDs wanted; an empty list asks
+	// for every device.
+	DeviceKeys map[string][]string
+
+	// Timeout is how long to wait for other servers.
+	Timeout time.Duration
+}
+
+// QueryResponse is the answer to POST /keys/query.
+type QueryResponse struct {
+	DeviceKeys      map[string]map[string]json.RawMessage `json:"device_keys"`
+	Failures        map[string]any                        `json:"failures"`
+	MasterKeys      map[string]json.RawMessage            `json:"master_keys"`
+	SelfSigningKeys map[string]json.RawMessage            `json:"self_signing_keys"`
+	UserSigningKeys map[string]json.RawMessage            `json:"user_signing_keys"`
+}
+
+// ClaimRequest is the body of POST /keys/claim.
+type ClaimRequest struct {
+	// OneTimeKeys is user ID, then device ID, then the algorithm wanted.
+	OneTimeKeys map[string]map[string]string
+
+	Timeout time.Duration
+}
+
+// ClaimResponse is the answer to POST /keys/claim.
+type ClaimResponse struct {
+	OneTimeKeys map[string]map[string]map[string]json.RawMessage `json:"one_time_keys"`
+	Failures    map[string]any                                   `json:"failures"`
+}
+
+// remoteFailure is what a server that could not be asked is reported as,
+// in the shape other homeservers use.
+func remoteFailure(err error) map[string]any {
+	status := 503
+	var httpErr *federation.HTTPError
+	if errors.As(err, &httpErr) {
+		status = httpErr.Status
+	}
+	return map[string]any{"status": status, "message": err.Error()}
+}
+
+// --- reading keys ---------------------------------------------------------------
+
+// publishedDeviceKeys reads the keys of a user's devices with every stored
+// signature merged in. For a local user only devices that still exist are
+// returned, and names maps them to their display names when the caller
+// wants those.
+func (k *API) publishedDeviceKeys(ctx context.Context, userID string, deviceIDs []string) (map[string]json.RawMessage, error) {
+	return k.deviceKeys(ctx, userID, deviceIDs, false)
+}
+
+func (k *API) deviceKeys(ctx context.Context, userID string, deviceIDs []string, withNames bool) (map[string]json.RawMessage, error) {
+	stored, err := k.store.DeviceKeys(ctx, userID, deviceIDs)
+	if err != nil {
+		return nil, err
+	}
+	out := map[string]json.RawMessage{}
+	if len(stored) == 0 {
+		return out, nil
+	}
+
+	names := map[string]string{}
+	if localpart, ok := k.localpartOf(userID); ok {
+		devices, err := k.devices.Devices(ctx, localpart)
+		if err != nil {
+			return nil, err
+		}
+		for _, d := range devices {
+			names[d.ID] = d.DisplayName
+		}
+	}
+
+	sigs, err := k.store.Signatures(ctx, userID)
+	if err != nil {
+		return nil, err
+	}
+	for _, d := range stored {
+		name, exists := names[d.DeviceID]
+		if k.isLocal(userID) && !exists {
+			continue
+		}
+		var mine []storage.KeySignature
+		for _, sig := range sigs {
+			if sig.TargetKeyID == d.DeviceID {
+				mine = append(mine, sig)
+			}
+		}
+		if !withNames {
+			name = ""
+		}
+		merged, err := withSignatures(d.JSON, mine, name)
+		if err != nil {
+			return nil, err
+		}
+		out[d.DeviceID] = merged
+	}
+	return out, nil
+}
+
+// crossSigningKeys reads a user's cross-signing keys as requester is
+// allowed to see them.
+//
+// The master key carries the signatures of the user's own devices, which
+// everybody may see, and the signature of requester's user-signing key if
+// they made one: who a user has verified is between them and their own
+// devices. The user-signing key is only ever shown to its owner.
+func (k *API) crossSigningKeys(ctx context.Context, userID, requester string) (master, selfSigning, userSigning json.RawMessage, err error) {
+	keys, err := k.store.CrossSigningKeys(ctx, userID)
+	if err != nil || len(keys) == 0 {
+		return nil, nil, nil, err
+	}
+	sigs, err := k.store.Signatures(ctx, userID)
+	if err != nil {
+		return nil, nil, nil, err
+	}
+
+	read := func(purpose string) (json.RawMessage, error) {
+		raw, ok := keys[purpose]
+		if !ok {
+			return nil, nil
+		}
+		var key crossSigningKeyObject
+		if err := json.Unmarshal(raw, &key); err != nil {
+			return nil, err
+		}
+		pub, _ := key.ed25519Key()
+		var visible []storage.KeySignature
+		for _, sig := range sigs {
+			if sig.TargetKeyID != pub {
+				continue
+			}
+			if sig.SignerUserID == userID || (requester != "" && sig.SignerUserID == requester) {
+				visible = append(visible, sig)
+			}
+		}
+		return withSignatures(raw, visible, "")
+	}
+
+	if master, err = read(storage.CrossSigningMaster); err != nil {
+		return nil, nil, nil, err
+	}
+	if selfSigning, err = read(storage.CrossSigningSelfSigning); err != nil {
+		return nil, nil, nil, err
+	}
+	if requester == userID {
+		if userSigning, err = read(storage.CrossSigningUserSigning); err != nil {
+			return nil, nil, nil, err
+		}
+	}
+	return master, selfSigning, userSigning, nil
+}
+
+// --- /keys/query ----------------------------------------------------------------
+
+// QueryKeys answers POST /keys/query for requester: local users from the
+// store, remote ones by asking their servers, all at once.
+func (k *API) QueryKeys(ctx context.Context, requester string, req QueryRequest) (QueryResponse, error) {
+	resp := QueryResponse{
+		DeviceKeys:      map[string]map[string]json.RawMessage{},
+		Failures:        map[string]any{},
+		MasterKeys:      map[string]json.RawMessage{},
+		SelfSigningKeys: map[string]json.RawMessage{},
+		UserSigningKeys: map[string]json.RawMessage{},
+	}
+
+	remote := map[string]map[string][]string{}
+	for userID, deviceIDs := range req.DeviceKeys {
+		server, err := id.ServerNameOf(userID)
+		if err != nil {
+			continue
+		}
+		if server != k.serverName {
+			if remote[server] == nil {
+				remote[server] = map[string][]string{}
+			}
+			remote[server][userID] = deviceIDs
+			continue
+		}
+
+		devices, err := k.deviceKeys(ctx, userID, deviceIDs, true)
+		if err != nil {
+			return QueryResponse{}, err
+		}
+		resp.DeviceKeys[userID] = devices
+
+		master, selfSigning, userSigning, err := k.crossSigningKeys(ctx, userID, requester)
+		if err != nil {
+			return QueryResponse{}, err
+		}
+		setIfPresent(resp.MasterKeys, userID, master)
+		setIfPresent(resp.SelfSigningKeys, userID, selfSigning)
+		setIfPresent(resp.UserSigningKeys, userID, userSigning)
+	}
+
+	k.queryRemote(ctx, requester, remote, remoteTimeout(req.Timeout), &resp)
+	return resp, nil
+}
+
+// queryRemote asks each server for its users' keys in parallel and merges
+// the answers in. A server that does not answer in time is a failure, not
+// an error: the keys of everybody else are still worth returning.
+func (k *API) queryRemote(ctx context.Context, requester string, byServer map[string]map[string][]string, timeout time.Duration, resp *QueryResponse) {
+	if len(byServer) == 0 {
+		return
+	}
+	ctx, cancel := context.WithTimeout(ctx, timeout)
+	defer cancel()
+
+	var (
+		mu sync.Mutex
+		wg sync.WaitGroup
+	)
+	for server, users := range byServer {
+		if k.remote == nil {
+			resp.Failures[server] = remoteFailure(errors.New("federation is not available on this server"))
+			continue
+		}
+		wg.Go(func() {
+			answer, err := k.remote.QueryUserKeys(ctx, server, federation.UserKeysQueryRequest{DeviceKeys: users})
+
+			mu.Lock()
+			defer mu.Unlock()
+			if err != nil {
+				resp.Failures[server] = remoteFailure(err)
+				return
+			}
+			for userID, devices := range answer.DeviceKeys {
+				// A server answers for its own users and nobody else's.
+				if _, asked := users[userID]; asked && devices != nil {
+					resp.DeviceKeys[userID] = devices
+				}
+			}
+			for userID, master := range answer.MasterKeys {
+				if _, asked := users[userID]; asked {
+					resp.MasterKeys[userID] = k.withRequesterSignature(ctx, userID, requester, master)
+				}
+			}
+			for userID, selfSigning := range answer.SelfSigningKeys {
+				if _, asked := users[userID]; asked {
+					resp.SelfSigningKeys[userID] = selfSigning
+				}
+			}
+		})
+	}
+	wg.Wait()
+}
+
+// withRequesterSignature adds requester's own signature of a remote user's
+// master key, which is stored here and nowhere else.
+func (k *API) withRequesterSignature(ctx context.Context, userID, requester string, master json.RawMessage) json.RawMessage {
+	if requester == "" {
+		return master
+	}
+	sigs, err := k.store.Signatures(ctx, userID)
+	if err != nil || len(sigs) == 0 {
+		return master
+	}
+	var key crossSigningKeyObject
+	if json.Unmarshal(master, &key) != nil {
+		return master
+	}
+	pub, _ := key.ed25519Key()
+	var mine []storage.KeySignature
+	for _, sig := range sigs {
+		if sig.TargetKeyID == pub && sig.SignerUserID == requester {
+			mine = append(mine, sig)
+		}
+	}
+	merged, err := withSignatures(master, mine, "")
+	if err != nil {
+		return master
+	}
+	return merged
+}
+
+func setIfPresent(m map[string]json.RawMessage, key string, value json.RawMessage) {
+	if len(value) > 0 {
+		m[key] = value
+	}
+}
+
+// QueryLocalKeys answers another server's POST /user/keys/query: the keys
+// of this server's users, without device display names, which Warp keeps
+// to itself.
+func (k *API) QueryLocalKeys(ctx context.Context, req federation.UserKeysQueryRequest) (federation.UserKeysQueryResponse, error) {
+	resp := federation.UserKeysQueryResponse{
+		DeviceKeys:      map[string]map[string]json.RawMessage{},
+		MasterKeys:      map[string]json.RawMessage{},
+		SelfSigningKeys: map[string]json.RawMessage{},
+	}
+	for userID, deviceIDs := range req.DeviceKeys {
+		if !k.isLocal(userID) {
+			continue
+		}
+		devices, err := k.publishedDeviceKeys(ctx, userID, deviceIDs)
+		if err != nil {
+			return federation.UserKeysQueryResponse{}, err
+		}
+		resp.DeviceKeys[userID] = devices
+
+		master, selfSigning, _, err := k.crossSigningKeys(ctx, userID, "")
+		if err != nil {
+			return federation.UserKeysQueryResponse{}, err
+		}
+		setIfPresent(resp.MasterKeys, userID, master)
+		setIfPresent(resp.SelfSigningKeys, userID, selfSigning)
+	}
+	return resp, nil
+}
+
+// UserDevices answers another server's GET /user/devices/{userId}: a local
+// user's whole device list, and its position in the device list stream,
+// which is the stream ID the server's later updates count from.
+func (k *API) UserDevices(ctx context.Context, userID string) (federation.UserDevicesResponse, error) {
+	resp := federation.UserDevicesResponse{UserID: userID, Devices: []federation.UserDevice{}}
+	if !k.isLocal(userID) {
+		return resp, invalidParamf("%s is not a user of this server", userID)
+	}
+	pos, err := k.store.DeviceListPosition(ctx, userID)
+	if err != nil {
+		return resp, err
+	}
+	resp.StreamID = pos
+
+	devices, err := k.publishedDeviceKeys(ctx, userID, nil)
+	if err != nil {
+		return resp, err
+	}
+	for _, deviceID := range sortedKeys(devices) {
+		resp.Devices = append(resp.Devices, federation.UserDevice{DeviceID: deviceID, Keys: devices[deviceID]})
+	}
+	resp.MasterKey, resp.SelfSigningKey, _, err = k.crossSigningKeys(ctx, userID, "")
+	return resp, err
+}
+
+// --- /keys/claim ----------------------------------------------------------------
+
+// ClaimKeys answers POST /keys/claim.
+func (k *API) ClaimKeys(ctx context.Context, req ClaimRequest) (ClaimResponse, error) {
+	resp := ClaimResponse{
+		OneTimeKeys: map[string]map[string]map[string]json.RawMessage{},
+		Failures:    map[string]any{},
+	}
+
+	local := map[string]map[string]string{}
+	remote := map[string]map[string]map[string]string{}
+	for userID, devices := range req.OneTimeKeys {
+		server, err := id.ServerNameOf(userID)
+		switch {
+		case err != nil:
+			continue
+		case server == k.serverName:
+			local[userID] = devices
+		default:
+			if remote[server] == nil {
+				remote[server] = map[string]map[string]string{}
+			}
+			remote[server][userID] = devices
+		}
+	}
+
+	claimed, err := k.ClaimLocalKeys(ctx, local)
+	if err != nil {
+		return ClaimResponse{}, err
+	}
+	for userID, devices := range claimed {
+		resp.OneTimeKeys[userID] = devices
+	}
+
+	k.claimRemote(ctx, remote, remoteTimeout(req.Timeout), &resp)
+	return resp, nil
+}
+
+// ClaimLocalKeys claims keys of this server's users' devices, for a local
+// client or another server. A device with no key of the algorithm asked
+// for is left out, which is how the spec says "none left".
+func (k *API) ClaimLocalKeys(ctx context.Context, want map[string]map[string]string) (map[string]map[string]map[string]json.RawMessage, error) {
+	out := map[string]map[string]map[string]json.RawMessage{}
+	for userID, devices := range want {
+		if !k.isLocal(userID) {
+			continue
+		}
+		for deviceID, algorithm := range devices {
+			key, ok, err := k.store.ClaimKey(ctx, userID, deviceID, algorithm)
+			if err != nil {
+				return nil, err
+			}
+			if !ok {
+				continue
+			}
+			if out[userID] == nil {
+				out[userID] = map[string]map[string]json.RawMessage{}
+			}
+			out[userID][deviceID] = map[string]json.RawMessage{key.KeyID: key.JSON}
+		}
+	}
+	return out, nil
+}
+
+// claimRemote claims keys from other servers in parallel.
+func (k *API) claimRemote(ctx context.Context, byServer map[string]map[string]map[string]string, timeout time.Duration, resp *ClaimResponse) {
+	if len(byServer) == 0 {
+		return
+	}
+	ctx, cancel := context.WithTimeout(ctx, timeout)
+	defer cancel()
+
+	var (
+		mu sync.Mutex
+		wg sync.WaitGroup
+	)
+	for server, users := range byServer {
+		if k.remote == nil {
+			resp.Failures[server] = remoteFailure(errors.New("federation is not available on this server"))
+			continue
+		}
+		wg.Go(func() {
+			answer, err := k.remote.ClaimUserKeys(ctx, server, federation.UserKeysClaimRequest{OneTimeKeys: users})
+
+			mu.Lock()
+			defer mu.Unlock()
+			if err != nil {
+				resp.Failures[server] = remoteFailure(err)
+				return
+			}
+			for userID, devices := range answer.OneTimeKeys {
+				if _, asked := users[userID]; asked {
+					resp.OneTimeKeys[userID] = devices
+				}
+			}
+		})
+	}
+	wg.Wait()
+}
+
+// sortedKeys is a map's keys in order.
+func sortedKeys[V any](m map[string]V) []string {
+	out := make([]string, 0, len(m))
+	for key := range m {
+		out = append(out, key)
+	}
+	slices.Sort(out)
+	return out
+}

server/internal/keyserver/upload.go

@@ -0,0 +1,202 @@
+package keyserver
+
+import (
+	"cmp"
+	"context"
+	"encoding/json"
+	"slices"
+
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Uploading keys: POST /keys/upload, and what deleting a device does to
+// them.
+
+// Algorithm names the server has to know about.
+const (
+	// AlgorithmSignedCurve25519 is the one-time key algorithm of Olm. A
+	// client counts on its count being reported even at zero: it is how
+	// the client learns it has to upload more.
+	AlgorithmSignedCurve25519 = "signed_curve25519"
+)
+
+// maxKeysPerUpload caps one upload's one-time keys. A client uploads up to
+// half of what its Olm account holds, which is fifty for libolm and
+// vodozemac alike; a thousand is a client that has gone wrong.
+const maxKeysPerUpload = 1000
+
+// UploadRequest is the body of POST /keys/upload.
+type UploadRequest struct {
+	DeviceKeys   json.RawMessage            `json:"device_keys,omitempty"`
+	OneTimeKeys  map[string]json.RawMessage `json:"one_time_keys,omitempty"`
+	FallbackKeys map[string]json.RawMessage `json:"fallback_keys,omitempty"`
+
+	// UnstableFallbackKeys is MSC2732's name for fallback_keys, which
+	// clients older than Matrix 1.2 still send.
+	UnstableFallbackKeys map[string]json.RawMessage `json:"org.matrix.msc2732.fallback_keys,omitempty"`
+}
+
+// UploadKeys stores what a device uploaded and returns how many one-time
+// keys it now has.
+//
+// The device keys are checked to be the caller's own device, and nothing
+// more. Whether they are signed by the key they say they are is checked by
+// every client that reads them, which is the check that matters: a server
+// that verified them would only be adding a point at which a key could be
+// refused and none at which it could be trusted.
+func (k *API) UploadKeys(ctx context.Context, userID, deviceID string, req UploadRequest) (map[string]int, error) {
+	if len(req.OneTimeKeys) > maxKeysPerUpload {
+		return nil, invalidParamf("at most %d one-time keys can be uploaded at once", maxKeysPerUpload)
+	}
+
+	if len(req.DeviceKeys) > 0 && string(req.DeviceKeys) != "null" {
+		if err := k.uploadDeviceKeys(ctx, userID, deviceID, req.DeviceKeys); err != nil {
+			return nil, err
+		}
+	}
+
+	otks, err := oneTimeKeys(req.OneTimeKeys)
+	if err != nil {
+		return nil, err
+	}
+	if err := k.store.AddOneTimeKeys(ctx, userID, deviceID, otks); err != nil {
+		return nil, err
+	}
+
+	fallback := req.FallbackKeys
+	if len(fallback) == 0 {
+		fallback = req.UnstableFallbackKeys
+	}
+	fallbackKeys, err := oneTimeKeys(fallback)
+	if err != nil {
+		return nil, err
+	}
+	if err := k.store.SetFallbackKeys(ctx, userID, deviceID, fallbackKeys); err != nil {
+		return nil, err
+	}
+
+	return k.OneTimeKeyCounts(ctx, userID, deviceID)
+}
+
+// uploadDeviceKeys checks and stores a device's identity keys.
+func (k *API) uploadDeviceKeys(ctx context.Context, userID, deviceID string, raw json.RawMessage) error {
+	var keys deviceKeysObject
+	if err := json.Unmarshal(raw, &keys); err != nil {
+		return invalidParamf("device_keys is malformed: %v", err)
+	}
+	switch {
+	case keys.UserID != userID:
+		return invalidParamf("device_keys.user_id must be %s", userID)
+	case keys.DeviceID != deviceID:
+		return invalidParamf("device_keys.device_id must be %s, the device the access token belongs to", deviceID)
+	case len(keys.Keys) == 0:
+		return missingParamf("device_keys.keys must not be empty")
+	}
+
+	encoded, err := canonical(raw)
+	if err != nil {
+		return err
+	}
+	// A client has no business writing unsigned: the server fills it in
+	// on the way out, and nothing in it is covered by the signature.
+	if encoded, err = withoutField(encoded, "unsigned"); err != nil {
+		return err
+	}
+
+	update, err := k.store.SetDeviceKeys(ctx, storage.DeviceKeys{UserID: userID, DeviceID: deviceID, JSON: encoded})
+	if err != nil {
+		return err
+	}
+	if update.StreamPos == 0 {
+		return nil
+	}
+	published, err := k.publishedDeviceKeys(ctx, userID, []string{deviceID})
+	if err != nil {
+		k.log.Warn("could not read back uploaded device keys", "user_id", userID, "device_id", deviceID, "error", err)
+	}
+	k.announceDevices(ctx, userID, []storage.DeviceListUpdate{update}, published)
+	return nil
+}
+
+// oneTimeKeys checks and canonicalises a map of one-time or fallback keys.
+func oneTimeKeys(in map[string]json.RawMessage) ([]storage.OneTimeKey, error) {
+	out := make([]storage.OneTimeKey, 0, len(in))
+	for keyID, raw := range in {
+		if !validKeyID(keyID) {
+			return nil, invalidParamf("key ID %q must have the form algorithm:id", keyID)
+		}
+		encoded, err := canonical(raw)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, storage.OneTimeKey{KeyID: keyID, JSON: encoded})
+	}
+	// In order, so that which of two conflicting keys is reported does
+	// not depend on map iteration.
+	slices.SortFunc(out, func(a, b storage.OneTimeKey) int { return cmp.Compare(a.KeyID, b.KeyID) })
+	return out, nil
+}
+
+// OneTimeKeyCounts reports a device's unclaimed one-time keys by
+// algorithm, with signed_curve25519 always present.
+func (k *API) OneTimeKeyCounts(ctx context.Context, userID, deviceID string) (map[string]int, error) {
+	counts, err := k.store.OneTimeKeyCounts(ctx, userID, deviceID)
+	if err != nil {
+		return nil, err
+	}
+	if _, ok := counts[AlgorithmSignedCurve25519]; !ok {
+		counts[AlgorithmSignedCurve25519] = 0
+	}
+	return counts, nil
+}
+
+// UnusedFallbackKeyTypes lists the algorithms a device has an unclaimed
+// fallback key for.
+func (k *API) UnusedFallbackKeyTypes(ctx context.Context, userID, deviceID string) ([]string, error) {
+	types, err := k.store.UnusedFallbackKeyTypes(ctx, userID, deviceID)
+	if types == nil && err == nil {
+		types = []string{}
+	}
+	return types, err
+}
+
+// DevicesDeleted drops the keys of deleted devices and tells everyone. It
+// is the userapi's hook, called after the devices are gone; an empty
+// deviceIDs means every device of the user.
+//
+// It runs after the deletion rather than inside it, so a crash between
+// the two leaves keys behind for a device that no longer exists. Those
+// keys are never served: a query reads the device list first and skips
+// keys for a device that is not on it.
+func (k *API) DevicesDeleted(ctx context.Context, localpart string, deviceIDs []string) {
+	userID := "@" + localpart + ":" + k.serverName
+	updates, err := k.store.DeleteDeviceKeys(ctx, userID, deviceIDs)
+	if err != nil {
+		k.log.Warn("could not delete the keys of deleted devices", "user_id", userID, "error", err)
+		return
+	}
+	k.announceDevices(ctx, userID, updates, nil)
+}
+
+// DeviceRenamed moves the device list of a user whose device has a new
+// display name, which /keys/query reports to local users. Nothing goes
+// over federation: Warp does not tell other servers what a user's devices
+// are called.
+func (k *API) DeviceRenamed(ctx context.Context, localpart, deviceID string) {
+	userID := "@" + localpart + ":" + k.serverName
+	keys, err := k.store.DeviceKeys(ctx, userID, []string{deviceID})
+	if err != nil || len(keys) == 0 {
+		return
+	}
+	if _, err := k.store.MarkDeviceListsChanged(ctx, []string{userID}); err != nil {
+		k.log.Warn("could not record a device rename", "user_id", userID, "error", err)
+		return
+	}
+	k.changed(ctx, userID)
+}
+
+// DeviceListChanges returns which of the given users' device lists moved
+// in a window of the device list stream. It is what a sync reads.
+func (k *API) DeviceListChanges(ctx context.Context, userIDs []string, from, to int64) ([]storage.DeviceListChange, error) {
+	return k.store.DeviceListChanges(ctx, userIDs, from, to)
+}

server/internal/roomserver/stream.go

@@ -565,3 +565,30 @@ func stateEntryIDs(entries []storage.StateEntry) []string {
 	}
 	return out
 }
+
+// UsersSharingRooms lists the users joined to a room the given user is
+// joined to, that user included. It is who hears about the user's device
+// list changes.
+func (r *API) UsersSharingRooms(ctx context.Context, userID string) ([]string, error) {
+	return r.store.UsersSharingRooms(ctx, userID)
+}
+
+// MemberChanges lists the membership events of the given rooms in a window
+// of the event stream, each with the user's membership now.
+func (r *API) MemberChanges(ctx context.Context, roomIDs []string, from, to int64) ([]storage.MemberChange, error) {
+	return r.store.MemberChanges(ctx, roomIDs, from, to)
+}
+
+// JoinedUserIDs lists the users joined to a room, as this server last
+// knew it.
+func (r *API) JoinedUserIDs(ctx context.Context, roomID string) ([]string, error) {
+	entries, err := r.store.MembershipEntries(ctx, roomID, []string{event.MembershipJoin})
+	if err != nil {
+		return nil, err
+	}
+	out := make([]string, 0, len(entries))
+	for _, e := range entries {
+		out = append(out, e.UserID)
+	}
+	return out, nil
+}

server/internal/setup/phase7_test.go

@@ -0,0 +1,329 @@
+package setup_test
+
+import (
+	"crypto/ed25519"
+	"crypto/rand"
+	"encoding/base64"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"testing"
+
+	"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
+	"github.com/dariuszjaniak/warp/server/internal/setup"
+)
+
+// The Phase 7 tests: end-to-end encryption's server half, through the
+// whole stack.
+
+// sendJSON sends a JSON body with any method.
+func sendJSON(t *testing.T, srv *setup.Server, method, path string, body any, token string) *httptest.ResponseRecorder {
+	t.Helper()
+	raw, err := json.Marshal(body)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return send(t, srv, method, path, "application/json", raw, token, nil)
+}
+
+func mustOK(t *testing.T, rec *httptest.ResponseRecorder, what string) map[string]any {
+	t.Helper()
+	if rec.Code != http.StatusOK {
+		t.Fatalf("%s: status %d: %s", what, rec.Code, rec.Body.String())
+	}
+	return decode[map[string]any](t, rec)
+}
+
+func deviceIDOf(t *testing.T, srv *setup.Server, token string) string {
+	t.Helper()
+	whoami := mustOK(t, getWithToken(t, srv, "/_matrix/client/v3/account/whoami", token), "whoami")
+	deviceID, _ := whoami["device_id"].(string)
+	return deviceID
+}
+
+// signedObject signs a JSON object as userID with an ed25519 key named
+// keyName.
+func signedObject(t *testing.T, userID, keyName string, priv ed25519.PrivateKey, obj map[string]any) map[string]any {
+	t.Helper()
+	raw, _ := json.Marshal(obj)
+	out, err := signing.SignJSON(userID, &signing.KeyPair{ID: keyName, Private: priv}, raw)
+	if err != nil {
+		t.Fatal(err)
+	}
+	var signed map[string]any
+	_ = json.Unmarshal(out, &signed)
+	return signed
+}
+
+func newEd25519(t *testing.T) (string, ed25519.PrivateKey) {
+	t.Helper()
+	pub, priv, err := ed25519.GenerateKey(rand.Reader)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return base64.RawStdEncoding.EncodeToString(pub), priv
+}
+
+// uploadDeviceKeys uploads a device's identity keys and some one-time
+// keys, and returns the response.
+func uploadDeviceKeys(t *testing.T, srv *setup.Server, token, userID, deviceID string, otks int) map[string]any {
+	t.Helper()
+	pub, priv := newEd25519(t)
+	deviceKeys := signedObject(t, userID, deviceID, priv, map[string]any{
+		"user_id":    userID,
+		"device_id":  deviceID,
+		"algorithms": []string{"m.olm.v1.curve25519-aes-sha2", "m.megolm.v1.aes-sha2"},
+		"keys": map[string]string{
+			"ed25519:" + deviceID:    pub,
+			"curve25519:" + deviceID: "curve" + deviceID,
+		},
+	})
+	oneTime := map[string]any{}
+	for i := range otks {
+		oneTime["signed_curve25519:"+deviceID+string(rune('a'+i))] = map[string]any{"key": "otk" + string(rune('a'+i))}
+	}
+	return mustOK(t, post(t, srv, "/_matrix/client/v3/keys/upload", map[string]any{
+		"device_keys":   deviceKeys,
+		"one_time_keys": oneTime,
+		"fallback_keys": map[string]any{"signed_curve25519:fb" + deviceID: map[string]any{"key": "fb", "fallback": true}},
+	}, token), "keys/upload")
+}
+
+// syncResponse is the part of a sync these tests read.
+type syncResponse struct {
+	NextBatch   string `json:"next_batch"`
+	DeviceLists struct {
+		Changed []string `json:"changed"`
+		Left    []string `json:"left"`
+	} `json:"device_lists"`
+	OTKCounts      map[string]int `json:"device_one_time_keys_count"`
+	UnusedFallback *[]string      `json:"device_unused_fallback_key_types"`
+}
+
+func syncOnce(t *testing.T, srv *setup.Server, token, since string) syncResponse {
+	t.Helper()
+	path := "/_matrix/client/v3/sync?timeout=0"
+	if since != "" {
+		path += "&since=" + url.QueryEscape(since)
+	}
+	rec := getWithToken(t, srv, path, token)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("sync: %d %s", rec.Code, rec.Body.String())
+	}
+	return decode[syncResponse](t, rec)
+}
+
+func createPublicRoom(t *testing.T, srv *setup.Server, token string) string {
+	t.Helper()
+	resp := mustOK(t, post(t, srv, "/_matrix/client/v3/createRoom", map[string]any{"preset": "public_chat"}, token), "createRoom")
+	roomID, _ := resp["room_id"].(string)
+	return roomID
+}
+
+func TestEndToEndEncryptionThroughTheAPI(t *testing.T) {
+	srv := newSigningServer(t, nil)
+	aliceToken := registerThrough(t, srv, "alice")
+	bobToken := registerThrough(t, srv, "bob")
+	const alice, bob = "@alice:warp.example", "@bob:warp.example"
+	aliceDevice := deviceIDOf(t, srv, aliceToken)
+
+	roomID := createPublicRoom(t, srv, aliceToken)
+	mustOK(t, post(t, srv, "/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/join", map[string]any{}, bobToken), "join")
+	bobSince := syncOnce(t, srv, bobToken, "").NextBatch
+
+	// Alice uploads her keys, and is told how many one-time keys she has.
+	counts, _ := uploadDeviceKeys(t, srv, aliceToken, alice, aliceDevice, 3)["one_time_key_counts"].(map[string]any)
+	if counts["signed_curve25519"] != float64(3) {
+		t.Errorf("one_time_key_counts = %v", counts)
+	}
+
+	// Bob's next sync says Alice's keys changed; Alice's carries her key
+	// counts, and both carry the fallback key list even when it is empty.
+	bobSync := syncOnce(t, srv, bobToken, bobSince)
+	if !contains(bobSync.DeviceLists.Changed, alice) {
+		t.Errorf("bob's device_lists.changed = %v, want alice", bobSync.DeviceLists.Changed)
+	}
+	if bobSync.UnusedFallback == nil || len(*bobSync.UnusedFallback) != 0 {
+		t.Errorf("bob's device_unused_fallback_key_types = %v, want []", bobSync.UnusedFallback)
+	}
+	if bobSync.OTKCounts["signed_curve25519"] != 0 {
+		t.Errorf("bob's one-time key count = %v", bobSync.OTKCounts)
+	}
+	aliceSync := syncOnce(t, srv, aliceToken, "")
+	if aliceSync.OTKCounts["signed_curve25519"] != 3 || aliceSync.UnusedFallback == nil || len(*aliceSync.UnusedFallback) != 1 {
+		t.Errorf("alice's key counts = %v, fallback %v", aliceSync.OTKCounts, aliceSync.UnusedFallback)
+	}
+
+	changes := mustOK(t, getWithToken(t, srv, "/_matrix/client/v3/keys/changes?from="+url.QueryEscape(bobSince), bobToken), "keys/changes")
+	if changed, _ := changes["changed"].([]any); len(changed) != 1 || changed[0] != alice {
+		t.Errorf("keys/changes = %v", changes)
+	}
+
+	// Bob fetches Alice's keys and claims one of her one-time keys.
+	query := mustOK(t, post(t, srv, "/_matrix/client/v3/keys/query", map[string]any{
+		"device_keys": map[string]any{alice: []string{}},
+	}, bobToken), "keys/query")
+	devices, _ := query["device_keys"].(map[string]any)[alice].(map[string]any)
+	if _, ok := devices[aliceDevice]; !ok {
+		t.Fatalf("keys/query = %v", query)
+	}
+	claim := mustOK(t, post(t, srv, "/_matrix/client/v3/keys/claim", map[string]any{
+		"one_time_keys": map[string]any{alice: map[string]string{aliceDevice: "signed_curve25519"}},
+	}, bobToken), "keys/claim")
+	if keys, _ := claim["one_time_keys"].(map[string]any)[alice].(map[string]any)[aliceDevice].(map[string]any); len(keys) != 1 {
+		t.Errorf("keys/claim = %v", claim)
+	}
+
+	// Cross-signing: the first upload needs no password, replacing the
+	// master key does.
+	masterPub, masterPriv := newEd25519(t)
+	sskPub, _ := newEd25519(t)
+	crossSigning := func(master string, priv ed25519.PrivateKey, ssk string) map[string]any {
+		return map[string]any{
+			"master_key": map[string]any{"user_id": alice, "usage": []string{"master"},
+				"keys": map[string]string{"ed25519:" + master: master}},
+			"self_signing_key": signedObject(t, alice, master, priv, map[string]any{
+				"user_id": alice, "usage": []string{"self_signing"},
+				"keys": map[string]string{"ed25519:" + ssk: ssk}}),
+		}
+	}
+	mustOK(t, post(t, srv, "/_matrix/client/v3/keys/device_signing/upload", crossSigning(masterPub, masterPriv, sskPub), aliceToken),
+		"first device_signing/upload")
+
+	newMaster, newMasterPriv := newEd25519(t)
+	body := crossSigning(newMaster, newMasterPriv, sskPub)
+	rec := post(t, srv, "/_matrix/client/v3/keys/device_signing/upload", body, aliceToken)
+	if rec.Code != http.StatusUnauthorized {
+		t.Fatalf("replacing the master key without auth = %d, want 401: %s", rec.Code, rec.Body.String())
+	}
+	body["auth"] = map[string]any{
+		"type": "m.login.password", "session": decode[map[string]any](t, rec)["session"],
+		"identifier": map[string]string{"type": "m.id.user", "user": "alice"}, "password": "sUp3rs3kr1t",
+	}
+	mustOK(t, post(t, srv, "/_matrix/client/v3/keys/device_signing/upload", body, aliceToken), "device_signing/upload with auth")
+	query = mustOK(t, post(t, srv, "/_matrix/client/v3/keys/query", map[string]any{
+		"device_keys": map[string]any{alice: []string{}},
+	}, bobToken), "keys/query after cross-signing")
+	if master, _ := query["master_keys"].(map[string]any)[alice].(map[string]any); master == nil ||
+		master["keys"].(map[string]any)["ed25519:"+newMaster] != newMaster {
+		t.Errorf("master_keys = %v, want the replacement", query["master_keys"])
+	}
+
+	// Key backup, version and keys.
+	version := mustOK(t, post(t, srv, "/_matrix/client/v3/room_keys/version", map[string]any{
+		"algorithm": "m.megolm_backup.v1.curve25519-aes-sha2", "auth_data": map[string]any{"public_key": "abc"},
+	}, aliceToken), "create backup")["version"].(string)
+	put := mustOK(t, sendJSON(t, srv, http.MethodPut, "/_matrix/client/v3/room_keys/keys/"+url.PathEscape(roomID)+"/sess1?version="+version,
+		map[string]any{"first_message_index": 0, "forwarded_count": 0, "is_verified": false,
+			"session_data": map[string]any{"ciphertext": "c"}}, aliceToken), "put room key")
+	if put["count"] != float64(1) {
+		t.Errorf("put room key = %v", put)
+	}
+	backup := mustOK(t, getWithToken(t, srv, "/_matrix/client/v3/room_keys/version", aliceToken), "get backup")
+	if backup["version"] != version || backup["count"] != float64(1) {
+		t.Errorf("backup = %v", backup)
+	}
+	keys := mustOK(t, getWithToken(t, srv, "/_matrix/client/v3/room_keys/keys?version="+version, aliceToken), "get room keys")
+	if rooms, _ := keys["rooms"].(map[string]any); rooms[roomID] == nil {
+		t.Errorf("room keys = %v", keys)
+	}
+	rec = sendJSON(t, srv, http.MethodPut, "/_matrix/client/v3/room_keys/keys?version=999", map[string]any{"rooms": map[string]any{}}, aliceToken)
+	if rec.Code != http.StatusNotFound {
+		t.Errorf("writing to an unknown backup version = %d: %s", rec.Code, rec.Body.String())
+	}
+	if rec := getWithToken(t, srv, "/_matrix/client/v3/room_keys/version", bobToken); rec.Code != http.StatusNotFound {
+		t.Errorf("a user with no backup = %d, want 404", rec.Code)
+	}
+
+	// Logging in a second device, uploading its keys and deleting it: the
+	// keys go with the device, and Bob is told.
+	login := mustOK(t, post(t, srv, "/_matrix/client/v3/login", map[string]any{
+		"type": "m.login.password", "identifier": map[string]string{"type": "m.id.user", "user": "alice"},
+		"password": "sUp3rs3kr1t", "device_id": "PHONE",
+	}, ""), "login")
+	uploadDeviceKeys(t, srv, login["access_token"].(string), alice, "PHONE", 1)
+	bobSince = syncOnce(t, srv, bobToken, "").NextBatch
+
+	rec = sendJSON(t, srv, http.MethodDelete, "/_matrix/client/v3/devices/PHONE", map[string]any{}, aliceToken)
+	if rec.Code != http.StatusUnauthorized {
+		t.Fatalf("delete device = %d, want 401", rec.Code)
+	}
+	mustOK(t, sendJSON(t, srv, http.MethodDelete, "/_matrix/client/v3/devices/PHONE", map[string]any{"auth": map[string]any{
+		"type": "m.login.password", "session": decode[map[string]any](t, rec)["session"],
+		"identifier": map[string]string{"type": "m.id.user", "user": "alice"}, "password": "sUp3rs3kr1t",
+	}}, aliceToken), "delete device with auth")
+
+	if got := syncOnce(t, srv, bobToken, bobSince); !contains(got.DeviceLists.Changed, alice) {
+		t.Errorf("after the device was deleted, bob's changed = %v", got.DeviceLists.Changed)
+	}
+	query = mustOK(t, post(t, srv, "/_matrix/client/v3/keys/query", map[string]any{
+		"device_keys": map[string]any{alice: []string{}},
+	}, bobToken), "keys/query after deletion")
+	if _, ok := query["device_keys"].(map[string]any)[alice].(map[string]any)["PHONE"]; ok {
+		t.Error("a deleted device's keys are still served")
+	}
+
+	// Bob leaving the only room they share puts him in Alice's left list.
+	aliceSince := syncOnce(t, srv, aliceToken, "").NextBatch
+	mustOK(t, post(t, srv, "/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/leave", map[string]any{}, bobToken), "leave")
+	if got := syncOnce(t, srv, aliceToken, aliceSince); !contains(got.DeviceLists.Left, bob) {
+		t.Errorf("after bob left, alice's device_lists = %+v", got.DeviceLists)
+	}
+}
+
+func TestKeysOverFederation(t *testing.T) {
+	if testing.Short() {
+		t.Skip("starts two servers")
+	}
+	nodes := newFederation(t, "hs-a.example", "hs-b.example")
+	a, b := nodes["hs-a.example"].srv, nodes["hs-b.example"].srv
+	const alice, bob = "@alice:hs-a.example", "@bob:hs-b.example"
+
+	aliceToken := registerThrough(t, a, "alice")
+	bobToken := registerThrough(t, b, "bob")
+	aliceDevice := deviceIDOf(t, a, aliceToken)
+
+	roomID := createPublicRoom(t, a, aliceToken)
+	mustOK(t, post(t, b, "/_matrix/client/v3/join/"+url.PathEscape(roomID)+"?server_name=hs-a.example",
+		map[string]any{}, bobToken), "remote join")
+	bobSince := syncOnce(t, b, bobToken, "").NextBatch
+
+	// Alice's upload reaches Bob's server as an m.device_list_update, and
+	// Bob's next sync says so.
+	uploadDeviceKeys(t, a, aliceToken, alice, aliceDevice, 2)
+	eventually(t, "alice in bob's device_lists.changed", func() bool {
+		got := syncOnce(t, b, bobToken, bobSince)
+		return contains(got.DeviceLists.Changed, alice)
+	})
+
+	// Bob's query and claim are answered by Alice's server.
+	query := mustOK(t, post(t, b, "/_matrix/client/v3/keys/query", map[string]any{
+		"device_keys": map[string]any{alice: []string{}},
+	}, bobToken), "remote keys/query")
+	if devices, _ := query["device_keys"].(map[string]any)[alice].(map[string]any); devices[aliceDevice] == nil {
+		t.Fatalf("remote keys/query = %v", query)
+	}
+	claim := mustOK(t, post(t, b, "/_matrix/client/v3/keys/claim", map[string]any{
+		"one_time_keys": map[string]any{alice: map[string]string{aliceDevice: "signed_curve25519"}},
+	}, bobToken), "remote keys/claim")
+	if keys, _ := claim["one_time_keys"].(map[string]any)[alice].(map[string]any)[aliceDevice].(map[string]any); len(keys) != 1 {
+		t.Errorf("remote keys/claim = %v", claim)
+	}
+	counts := syncOnce(t, a, aliceToken, "").OTKCounts
+	if counts["signed_curve25519"] != 1 {
+		t.Errorf("after a remote claim alice has %v one-time keys, want 1", counts)
+	}
+
+	// With A's federation off, B reports A as a failure instead of
+	// hanging.
+	if err := a.Settings().SetFederationEnabled(t.Context(), false); err != nil {
+		t.Fatal(err)
+	}
+	query = mustOK(t, post(t, b, "/_matrix/client/v3/keys/query", map[string]any{
+		"device_keys": map[string]any{alice: []string{}}, "timeout": 5000,
+	}, bobToken), "keys/query with A's federation off")
+	if failures, _ := query["failures"].(map[string]any); failures["hs-a.example"] == nil {
+		t.Errorf("failures = %v, want hs-a.example", query["failures"])
+	}
+}

server/internal/setup/routes.go

@@ -12,14 +12,13 @@ import (
 //
 // Keep it honest: a version belongs here once the endpoints a client
 // actually uses from it work. Accounts, sessions, devices, profiles,
-// account data, rooms, the event stream and — with Phase 6 — media all
-// do, which is enough for a client to be used. Advertising v1.11 or later
-// is also what makes a client use the authenticated media endpoints, the
-// only download endpoints Warp serves by default. What is still missing
-// is end-to-end encryption (Phase 7) and push (Phase 8); a client
-// discovers each of those from the endpoint it needs rather than from
-// this list, which is why their absence does not make a version listed
-// here a lie.
+// account data, rooms, the event stream, media (Phase 6) and end-to-end
+// encryption (Phase 7) all do, which is enough for a client to be used.
+// Advertising v1.11 or later is also what makes a client use the
+// authenticated media endpoints, the only download endpoints Warp serves
+// by default. What is still missing is push (Phase 8); a client discovers
+// that from the endpoint it needs rather than from this list, which is why
+// its absence does not make a version listed here a lie.
 var SpecVersions = []string{"v1.13", "v1.14"}
 
 // UnstableFeatures advertises MSC support. Simplified Sliding Sync

server/internal/setup/server.go

@@ -21,6 +21,7 @@ import (
 	"github.com/dariuszjaniak/warp/server/internal/federation"
 	"github.com/dariuszjaniak/warp/server/internal/federationapi"
 	"github.com/dariuszjaniak/warp/server/internal/httputil"
+	"github.com/dariuszjaniak/warp/server/internal/keyserver"
 	"github.com/dariuszjaniak/warp/server/internal/matrix/signing"
 	"github.com/dariuszjaniak/warp/server/internal/mediaapi"
 	"github.com/dariuszjaniak/warp/server/internal/metrics"
@@ -46,6 +47,7 @@ type Server struct {
 	users     *userapi.API
 	rooms     *roomserver.API
 	sync      *syncapi.API
+	keys      *keyserver.API
 	media     *mediaapi.API
 	clientAPI *clientapi.ClientAPI
 	adminAPI  *adminapi.AdminAPI
@@ -273,10 +275,32 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error)
 		emit.add(queue)
 		s.fedModule = federationapi.NewModule(queue, log)
 
+		// The keyserver comes with the rooms too: device list tracking is
+		// "who shares a room with whom", and the sync it reports through
+		// is the roomserver's. It tells other servers about key changes
+		// through the same queue as everything else, and hears about
+		// deleted devices from the userapi.
+		s.keys, err = keyserver.New(keyserver.Options{
+			ServerName: cfg.Global.ServerName,
+			Store:      s.db.Keys(),
+			Rooms:      s.rooms,
+			Notifier:   notifier,
+			Devices:    s.users,
+			Remote:     s.fedClient,
+			Outbound:   queue,
+			Logger:     log,
+		})
+		if err != nil {
+			_ = s.closeDB()
+			return nil, err
+		}
+		s.users.SetDeviceObserver(s.keys)
+
 		s.sync, err = syncapi.New(syncapi.Options{
 			ServerName:      cfg.Global.ServerName,
 			Rooms:           s.rooms,
 			Accounts:        s.users,
+			Keys:            s.keys,
 			Store:           s.db.Sync(),
 			Notifier:        notifier,
 			PresenceEnabled: cfg.Sync.Presence,
@@ -299,6 +323,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error)
 			Profiles:   s.users,
 			Store:      s.db.Federation(),
 			Media:      s.media,
+			UserKeys:   s.keys,
 			Logger:     log,
 		})
 		if err != nil {
@@ -306,7 +331,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error)
 			return nil, err
 		}
 	} else {
-		log.Warn("started without a signing key: rooms and /sync are unavailable")
+		log.Warn("started without a signing key: rooms, /sync and end-to-end encryption are unavailable")
 	}
 
 	// The interfaces below are left nil rather than set to a nil pointer
@@ -325,6 +350,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error)
 		Users:                s.users,
 		Rooms:                s.rooms,
 		Sync:                 s.sync,
+		Keys:                 s.keys,
 		Media:                s.media,
 		LegacyMediaDownloads: cfg.Media.LegacyUnauthenticatedDownloads,
 		Settings:             settingsService,
@@ -414,6 +440,10 @@ func (s *Server) Rooms() *roomserver.API { return s.rooms }
 // without a signing key.
 func (s *Server) Sync() *syncapi.API { return s.sync }
 
+// Keys exposes the keyserver, or nil when the server started without a
+// signing key.
+func (s *Server) Keys() *keyserver.API { return s.keys }
+
 // Media exposes the media repository.
 func (s *Server) Media() *mediaapi.API { return s.media }
 

server/internal/storage/keys.go

@@ -0,0 +1,278 @@
+package storage
+
+import (
+	"context"
+	"errors"
+)
+
+// Errors the key store returns for cases a caller has to tell apart.
+var (
+	// ErrKeyConflict is a one-time key uploaded under an ID that already
+	// holds a different key. The spec has a key ID name one key for its
+	// whole life: a client that reuses one has lost track of what it
+	// uploaded, and silently replacing the key would leave a peer that
+	// already claimed the old one unable to decrypt.
+	ErrKeyConflict = errors.New("a different key is already stored under this key ID")
+
+	// ErrNoSuchBackup is a key backup version that does not exist, or has
+	// been deleted.
+	ErrNoSuchBackup = errors.New("no such key backup version")
+
+	// ErrNoSuchBackupKey is a room key that is not in a backup.
+	ErrNoSuchBackupKey = errors.New("no such key in the backup")
+)
+
+// The purposes a cross-signing key can have, as the spec spells its usage.
+const (
+	CrossSigningMaster      = "master"
+	CrossSigningSelfSigning = "self_signing"
+	CrossSigningUserSigning = "user_signing"
+)
+
+// DeviceKeys is the identity keys of one device, as the device uploaded and
+// signed them.
+type DeviceKeys struct {
+	UserID   string
+	DeviceID string
+
+	// JSON is the device_keys object as uploaded, without `unsigned`: the
+	// server writes that itself on the way out, and anything a client put
+	// there is not something it signed.
+	JSON []byte
+}
+
+// OneTimeKey is one one-time or fallback key of a device.
+type OneTimeKey struct {
+	// KeyID is "algorithm:id", the key's name in the map it was uploaded
+	// in.
+	KeyID string
+
+	// JSON is the key as uploaded: a bare string for an unsigned
+	// curve25519 key, an object with signatures for a signed one.
+	JSON []byte
+}
+
+// Algorithm is the part of the key ID before the colon.
+func (k OneTimeKey) Algorithm() string {
+	for i := 0; i < len(k.KeyID); i++ {
+		if k.KeyID[i] == ':' {
+			return k.KeyID[:i]
+		}
+	}
+	return k.KeyID
+}
+
+// ClaimedKey is a key handed out by a claim, and whether it was a fallback
+// key rather than a one-time one.
+type ClaimedKey struct {
+	OneTimeKey
+	Fallback bool
+}
+
+// KeySignature is one signature of a key by another key, uploaded apart
+// from the key it signs.
+//
+// The keys a client uploads come signed by the device or key that made
+// them. Cross-signing adds signatures afterwards — a device by the user's
+// self-signing key, the user's master key by a device, another user's
+// master key by the user-signing key — and those arrive through
+// /keys/signatures/upload, long after the key they sign. They are kept
+// apart and merged in when the key is read, rather than written into it,
+// so that a key's own JSON stays exactly what its owner uploaded.
+type KeySignature struct {
+	// TargetUserID owns the signed key, and TargetKeyID names it: a device
+	// ID for a device key, the unpadded base64 public key for a
+	// cross-signing key.
+	TargetUserID string
+	TargetKeyID  string
+
+	// SignerUserID made the signature with SignerKeyID, which is
+	// "ed25519:" and the signing key's ID.
+	SignerUserID string
+	SignerKeyID  string
+
+	Signature string
+}
+
+// DeviceListUpdate is one move of a user's device list: the position this
+// change was given, and the position of the change before it. A remote
+// server is told both, and asks for the whole list when the previous one
+// is not the last it heard of.
+type DeviceListUpdate struct {
+	UserID string
+
+	// DeviceID is the device the change was about, or empty for a change
+	// to the user's cross-signing keys.
+	DeviceID string
+
+	StreamPos int64
+
+	// PrevPos is the user's position before this change, or 0 for their
+	// first.
+	PrevPos int64
+}
+
+// DeviceListChange is a user whose device list moved, at the position of
+// the latest move.
+type DeviceListChange struct {
+	UserID    string
+	StreamPos int64
+}
+
+// BackupVersion is one version of a user's server-side key backup.
+type BackupVersion struct {
+	UserID  string
+	Version string
+
+	Algorithm string
+
+	// AuthData is the algorithm's parameters as the client wrote them: the
+	// backup's public key and the signatures over it.
+	AuthData []byte
+
+	// ETag changes every time a key in the backup does, so a client can
+	// tell its copy of the backup is stale without downloading it.
+	ETag string
+
+	// Count is how many keys the backup holds.
+	Count int
+}
+
+// BackupKey is one room key in a backup.
+type BackupKey struct {
+	RoomID    string
+	SessionID string
+
+	FirstMessageIndex int64
+	ForwardedCount    int64
+	IsVerified        bool
+
+	// SessionData is the encrypted key, which the server cannot read.
+	SessionData []byte
+}
+
+// Replaces reports whether a key should replace one already backed up for
+// the same session, by the rule the spec gives: a verified key beats an
+// unverified one, then the key that decrypts more of the session — the
+// lower first message index — and then the one that has been forwarded
+// fewer times.
+func (k BackupKey) Replaces(existing BackupKey) bool {
+	if k.IsVerified != existing.IsVerified {
+		return k.IsVerified
+	}
+	if k.FirstMessageIndex != existing.FirstMessageIndex {
+		return k.FirstMessageIndex < existing.FirstMessageIndex
+	}
+	return k.ForwardedCount < existing.ForwardedCount
+}
+
+// KeyStore persists end-to-end encryption keys: the storage half of the
+// keyserver component.
+//
+// Users are full user IDs here rather than localparts, because most of what
+// is stored is about users of other servers too: signatures of their keys,
+// and the device list stream, which records their changes as well as ours
+// so that a local client knows to fetch their keys again.
+//
+// Every write that changes what /keys/query would answer about a user moves
+// the device list stream, inside the same transaction, and reports the
+// position it moved to. That is what makes "whose keys changed since this
+// sync token" a question with one exact answer.
+type KeyStore interface {
+	// SetDeviceKeys stores a device's identity keys. Storing what is
+	// already there changes nothing and returns a zero update; anything
+	// else moves the device list stream.
+	SetDeviceKeys(ctx context.Context, keys DeviceKeys) (DeviceListUpdate, error)
+
+	// DeviceKeys reads the identity keys of a user's devices. An empty
+	// deviceIDs reads every device that has uploaded keys.
+	DeviceKeys(ctx context.Context, userID string, deviceIDs []string) ([]DeviceKeys, error)
+
+	// DeleteDeviceKeys drops everything stored for the given devices —
+	// identity keys, one-time keys and fallback keys — and moves the
+	// device list stream once for each device that had identity keys. An
+	// empty deviceIDs drops every device of the user.
+	DeleteDeviceKeys(ctx context.Context, userID string, deviceIDs []string) ([]DeviceListUpdate, error)
+
+	// AddOneTimeKeys stores one-time keys. A key under an ID that already
+	// holds the same key is ignored; one under an ID holding a different
+	// key fails the whole batch with ErrKeyConflict.
+	AddOneTimeKeys(ctx context.Context, userID, deviceID string, keys []OneTimeKey) error
+
+	// OneTimeKeyCounts counts a device's unclaimed one-time keys by
+	// algorithm.
+	OneTimeKeyCounts(ctx context.Context, userID, deviceID string) (map[string]int, error)
+
+	// SetFallbackKeys stores fallback keys, replacing any earlier one of
+	// the same algorithm. A key the same as the one stored keeps its used
+	// flag: a client re-uploading its fallback key on every start must not
+	// make a used one look fresh.
+	SetFallbackKeys(ctx context.Context, userID, deviceID string, keys []OneTimeKey) error
+
+	// UnusedFallbackKeyTypes lists the algorithms a device has a fallback
+	// key for that has not been claimed.
+	UnusedFallbackKeyTypes(ctx context.Context, userID, deviceID string) ([]string, error)
+
+	// ClaimKey hands out one key of an algorithm for a device, deleting it:
+	// a one-time key if there is one, the fallback key otherwise, which is
+	// marked used and kept. The boolean is false when the device has
+	// neither.
+	ClaimKey(ctx context.Context, userID, deviceID, algorithm string) (ClaimedKey, bool, error)
+
+	// SetCrossSigningKeys stores a user's cross-signing keys by purpose and
+	// moves the device list stream. A purpose mapped to nil is deleted,
+	// which is what replacing the master key does to the keys it signed.
+	SetCrossSigningKeys(ctx context.Context, userID string, keys map[string][]byte) (DeviceListUpdate, error)
+
+	// CrossSigningKeys reads a user's cross-signing keys by purpose.
+	CrossSigningKeys(ctx context.Context, userID string) (map[string][]byte, error)
+
+	// AddSignatures stores signatures, replacing an earlier one by the same
+	// key of the same key, and moves the device list stream of every user
+	// whose key gained one.
+	AddSignatures(ctx context.Context, sigs []KeySignature) ([]DeviceListUpdate, error)
+
+	// Signatures reads the stored signatures of a user's keys.
+	Signatures(ctx context.Context, targetUserID string) ([]KeySignature, error)
+
+	// MarkDeviceListsChanged moves the device list stream for users whose
+	// keys changed somewhere this store does not hold them: on another
+	// server, which said so in an m.device_list_update.
+	MarkDeviceListsChanged(ctx context.Context, userIDs []string) (int64, error)
+
+	// DeviceListChanges returns which of the given users' device lists
+	// moved after from and at or before to.
+	DeviceListChanges(ctx context.Context, userIDs []string, from, to int64) ([]DeviceListChange, error)
+
+	// DeviceListPosition is a user's latest position in the device list
+	// stream, or 0 when their list has never moved.
+	DeviceListPosition(ctx context.Context, userID string) (int64, error)
+
+	// CreateBackup creates a new key backup version, which becomes the
+	// user's current one, and returns its version.
+	CreateBackup(ctx context.Context, userID, algorithm string, authData []byte) (string, error)
+
+	// Backup reads one backup version, or the current one when version is
+	// empty. A deleted or unknown version is ErrNoSuchBackup.
+	Backup(ctx context.Context, userID, version string) (BackupVersion, error)
+
+	// UpdateBackup replaces a backup version's auth data.
+	UpdateBackup(ctx context.Context, userID, version string, authData []byte) error
+
+	// DeleteBackup deletes a backup version and every key in it. Its
+	// number is not handed out again.
+	DeleteBackup(ctx context.Context, userID, version string) error
+
+	// PutBackupKeys stores room keys in a backup version, each replacing
+	// the key already there for its session only when BackupKey.Replaces
+	// says it should, and returns the version as it then stands.
+	PutBackupKeys(ctx context.Context, userID, version string, keys []BackupKey) (BackupVersion, error)
+
+	// BackupKeys reads the keys of a backup version: all of them, one
+	// room's (roomID set), or one session's (both set).
+	BackupKeys(ctx context.Context, userID, version, roomID, sessionID string) ([]BackupKey, error)
+
+	// DeleteBackupKeys deletes keys from a backup version with the same
+	// scoping as BackupKeys, and returns the version as it then stands.
+	DeleteBackupKeys(ctx context.Context, userID, version, roomID, sessionID string) (BackupVersion, error)
+}

server/internal/storage/keys_test.go

@@ -0,0 +1,342 @@
+package storage_test
+
+import (
+	"database/sql"
+	"errors"
+	"os"
+	"slices"
+	"testing"
+
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// migratedKeysDB opens a driver with the keyserver tables empty. The
+// device list counter is left where it is: every assertion below is about
+// positions relative to one read before the write.
+func migratedKeysDB(t *testing.T, driver string) storage.Database {
+	t.Helper()
+	db := openTestDB(t, driver)
+	if err := db.Migrate(t.Context()); err != nil {
+		t.Fatalf("Migrate: %v", err)
+	}
+	if driver == storage.DriverPostgres {
+		pool, err := sql.Open("pgx", os.Getenv("WARP_TEST_POSTGRES_DSN"))
+		if err != nil {
+			t.Fatalf("open postgres to reset the keyserver tables: %v", err)
+		}
+		defer func() { _ = pool.Close() }()
+		if _, err := pool.ExecContext(t.Context(), `TRUNCATE keyserver_device_keys, keyserver_one_time_keys,
+			keyserver_fallback_keys, keyserver_cross_signing_keys, keyserver_signatures,
+			keyserver_device_lists, keyserver_backup_versions, keyserver_backup_keys`); err != nil {
+			t.Fatalf("reset the keyserver tables: %v", err)
+		}
+	}
+	return db
+}
+
+const alice = "@alice:warp.example"
+
+func TestDeviceKeysAndTheDeviceListStream(t *testing.T) {
+	for _, driver := range testDrivers {
+		t.Run(driver, func(t *testing.T) {
+			ctx := t.Context()
+			db := migratedKeysDB(t, driver)
+			keys := db.Keys()
+
+			before, err := db.Sync().StreamPositions(ctx)
+			if err != nil {
+				t.Fatalf("StreamPositions: %v", err)
+			}
+
+			first, err := keys.SetDeviceKeys(ctx, storage.DeviceKeys{UserID: alice, DeviceID: "D1", JSON: []byte(`{"k":1}`)})
+			if err != nil {
+				t.Fatalf("SetDeviceKeys: %v", err)
+			}
+			if first.StreamPos != before.DeviceList+1 || first.PrevPos != 0 || first.DeviceID != "D1" {
+				t.Errorf("first update = %+v, want position %d with no previous", first, before.DeviceList+1)
+			}
+
+			// The same keys again are not a change.
+			again, err := keys.SetDeviceKeys(ctx, storage.DeviceKeys{UserID: alice, DeviceID: "D1", JSON: []byte(`{"k":1}`)})
+			if err != nil {
+				t.Fatalf("SetDeviceKeys again: %v", err)
+			}
+			if again != (storage.DeviceListUpdate{}) {
+				t.Errorf("re-uploading the same keys moved the stream: %+v", again)
+			}
+
+			second, err := keys.SetDeviceKeys(ctx, storage.DeviceKeys{UserID: alice, DeviceID: "D2", JSON: []byte(`{"k":2}`)})
+			if err != nil {
+				t.Fatalf("SetDeviceKeys D2: %v", err)
+			}
+			if second.PrevPos != first.StreamPos {
+				t.Errorf("second update's previous = %d, want %d", second.PrevPos, first.StreamPos)
+			}
+
+			after, _ := db.Sync().StreamPositions(ctx)
+			if after.DeviceList != second.StreamPos {
+				t.Errorf("StreamPositions.DeviceList = %d, want %d", after.DeviceList, second.StreamPos)
+			}
+
+			all, err := keys.DeviceKeys(ctx, alice, nil)
+			if err != nil || len(all) != 2 || all[0].DeviceID != "D1" || string(all[1].JSON) != `{"k":2}` {
+				t.Fatalf("DeviceKeys = %+v, %v", all, err)
+			}
+			one, _ := keys.DeviceKeys(ctx, alice, []string{"D2", "nope"})
+			if len(one) != 1 || one[0].DeviceID != "D2" {
+				t.Errorf("DeviceKeys(D2) = %+v", one)
+			}
+
+			changes, err := keys.DeviceListChanges(ctx, []string{alice, "@bob:warp.example"}, first.StreamPos, 0)
+			if err != nil || len(changes) != 1 || changes[0].StreamPos != second.StreamPos {
+				t.Errorf("DeviceListChanges after the first = %+v, %v", changes, err)
+			}
+			if changes, _ := keys.DeviceListChanges(ctx, []string{alice}, second.StreamPos, 0); len(changes) != 0 {
+				t.Errorf("DeviceListChanges after the last = %+v, want none", changes)
+			}
+
+			// Deleting two devices moves the stream once for each, so a
+			// remote server sees two updates rather than one and a repeat.
+			if err := keys.AddOneTimeKeys(ctx, alice, "D1", []storage.OneTimeKey{{KeyID: "curve25519:a", JSON: []byte(`"x"`)}}); err != nil {
+				t.Fatal(err)
+			}
+			deleted, err := keys.DeleteDeviceKeys(ctx, alice, nil)
+			if err != nil || len(deleted) != 2 {
+				t.Fatalf("DeleteDeviceKeys = %+v, %v", deleted, err)
+			}
+			if deleted[0].StreamPos == deleted[1].StreamPos || deleted[1].PrevPos != deleted[0].StreamPos {
+				t.Errorf("deletions share or skip positions: %+v", deleted)
+			}
+			if left, _ := keys.DeviceKeys(ctx, alice, nil); len(left) != 0 {
+				t.Errorf("keys survived DeleteDeviceKeys: %+v", left)
+			}
+			if counts, _ := keys.OneTimeKeyCounts(ctx, alice, "D1"); len(counts) != 0 {
+				t.Errorf("one-time keys survived DeleteDeviceKeys: %v", counts)
+			}
+			if pos, _ := keys.DeviceListPosition(ctx, alice); pos != deleted[1].StreamPos {
+				t.Errorf("DeviceListPosition = %d, want %d", pos, deleted[1].StreamPos)
+			}
+
+			// Deleting devices that had no keys moves nothing.
+			if none, err := keys.DeleteDeviceKeys(ctx, alice, []string{"D9"}); err != nil || len(none) != 0 {
+				t.Errorf("DeleteDeviceKeys of a keyless device = %+v, %v", none, err)
+			}
+
+			top, err := keys.MarkDeviceListsChanged(ctx, []string{"@carol:remote.example", "@carol:remote.example"})
+			if err != nil || top != deleted[1].StreamPos+1 {
+				t.Errorf("MarkDeviceListsChanged = %d, %v; want one move to %d", top, err, deleted[1].StreamPos+1)
+			}
+		})
+	}
+}
+
+func TestOneTimeAndFallbackKeys(t *testing.T) {
+	for _, driver := range testDrivers {
+		t.Run(driver, func(t *testing.T) {
+			ctx := t.Context()
+			keys := migratedKeysDB(t, driver).Keys()
+
+			otks := []storage.OneTimeKey{
+				{KeyID: "signed_curve25519:AAAA", JSON: []byte(`{"key":"a"}`)},
+				{KeyID: "signed_curve25519:BBBB", JSON: []byte(`{"key":"b"}`)},
+				{KeyID: "curve25519:CCCC", JSON: []byte(`"c"`)},
+			}
+			if err := keys.AddOneTimeKeys(ctx, alice, "D1", otks); err != nil {
+				t.Fatalf("AddOneTimeKeys: %v", err)
+			}
+			// The same keys again are fine; a different key under a used
+			// ID is not, and nothing of that batch is kept.
+			if err := keys.AddOneTimeKeys(ctx, alice, "D1", otks[:1]); err != nil {
+				t.Errorf("re-uploading the same key: %v", err)
+			}
+			err := keys.AddOneTimeKeys(ctx, alice, "D1", []storage.OneTimeKey{
+				{KeyID: "signed_curve25519:DDDD", JSON: []byte(`{"key":"d"}`)},
+				{KeyID: "signed_curve25519:AAAA", JSON: []byte(`{"key":"other"}`)},
+			})
+			if !errors.Is(err, storage.ErrKeyConflict) {
+				t.Errorf("conflicting key = %v, want ErrKeyConflict", err)
+			}
+
+			counts, err := keys.OneTimeKeyCounts(ctx, alice, "D1")
+			if err != nil || counts["signed_curve25519"] != 2 || counts["curve25519"] != 1 {
+				t.Fatalf("OneTimeKeyCounts = %v, %v", counts, err)
+			}
+
+			fallback := storage.OneTimeKey{KeyID: "signed_curve25519:FFFF", JSON: []byte(`{"key":"f","fallback":true}`)}
+			if err := keys.SetFallbackKeys(ctx, alice, "D1", []storage.OneTimeKey{fallback}); err != nil {
+				t.Fatalf("SetFallbackKeys: %v", err)
+			}
+			if types, _ := keys.UnusedFallbackKeyTypes(ctx, alice, "D1"); !slices.Equal(types, []string{"signed_curve25519"}) {
+				t.Errorf("UnusedFallbackKeyTypes = %v", types)
+			}
+
+			// Two one-time keys, oldest ID first, then the fallback key,
+			// as often as it is asked for.
+			var claimed []string
+			for range 4 {
+				k, ok, err := keys.ClaimKey(ctx, alice, "D1", "signed_curve25519")
+				if err != nil || !ok {
+					t.Fatalf("ClaimKey = %+v, %v, %v", k, ok, err)
+				}
+				claimed = append(claimed, k.KeyID)
+				if k.Fallback != (k.KeyID == fallback.KeyID) {
+					t.Errorf("claimed %s with Fallback=%v", k.KeyID, k.Fallback)
+				}
+			}
+			want := []string{"signed_curve25519:AAAA", "signed_curve25519:BBBB", fallback.KeyID, fallback.KeyID}
+			if !slices.Equal(claimed, want) {
+				t.Errorf("claimed %v, want %v", claimed, want)
+			}
+			if types, _ := keys.UnusedFallbackKeyTypes(ctx, alice, "D1"); len(types) != 0 {
+				t.Errorf("a claimed fallback key is still reported unused: %v", types)
+			}
+
+			// Re-uploading the same fallback key keeps it used; a new one
+			// is fresh.
+			_ = keys.SetFallbackKeys(ctx, alice, "D1", []storage.OneTimeKey{fallback})
+			if types, _ := keys.UnusedFallbackKeyTypes(ctx, alice, "D1"); len(types) != 0 {
+				t.Errorf("re-uploading a used fallback key made it unused: %v", types)
+			}
+			_ = keys.SetFallbackKeys(ctx, alice, "D1", []storage.OneTimeKey{{KeyID: "signed_curve25519:GGGG", JSON: []byte(`{}`)}})
+			if types, _ := keys.UnusedFallbackKeyTypes(ctx, alice, "D1"); len(types) != 1 {
+				t.Errorf("a new fallback key is not reported unused: %v", types)
+			}
+
+			if _, ok, err := keys.ClaimKey(ctx, alice, "D2", "signed_curve25519"); ok || err != nil {
+				t.Errorf("claiming from a device with no keys = %v, %v", ok, err)
+			}
+		})
+	}
+}
+
+func TestCrossSigningKeysAndSignatures(t *testing.T) {
+	for _, driver := range testDrivers {
+		t.Run(driver, func(t *testing.T) {
+			ctx := t.Context()
+			keys := migratedKeysDB(t, driver).Keys()
+
+			u1, err := keys.SetCrossSigningKeys(ctx, alice, map[string][]byte{
+				storage.CrossSigningMaster:      []byte(`{"m":1}`),
+				storage.CrossSigningSelfSigning: []byte(`{"s":1}`),
+			})
+			if err != nil || u1.StreamPos == 0 {
+				t.Fatalf("SetCrossSigningKeys = %+v, %v", u1, err)
+			}
+			_, err = keys.SetCrossSigningKeys(ctx, alice, map[string][]byte{
+				storage.CrossSigningMaster:      []byte(`{"m":2}`),
+				storage.CrossSigningSelfSigning: nil,
+			})
+			if err != nil {
+				t.Fatal(err)
+			}
+			got, err := keys.CrossSigningKeys(ctx, alice)
+			if err != nil || len(got) != 1 || string(got[storage.CrossSigningMaster]) != `{"m":2}` {
+				t.Errorf("CrossSigningKeys = %q, %v", got, err)
+			}
+
+			updates, err := keys.AddSignatures(ctx, []storage.KeySignature{
+				{TargetUserID: alice, TargetKeyID: "D1", SignerUserID: alice, SignerKeyID: "ed25519:ssk", Signature: "s1"},
+				{TargetUserID: alice, TargetKeyID: "D1", SignerUserID: alice, SignerKeyID: "ed25519:ssk", Signature: "s2"},
+				{TargetUserID: "@bob:warp.example", TargetKeyID: "mk", SignerUserID: alice, SignerKeyID: "ed25519:usk", Signature: "u1"},
+			})
+			if err != nil || len(updates) != 2 {
+				t.Fatalf("AddSignatures = %+v, %v; want one update per signed key", updates, err)
+			}
+			sigs, err := keys.Signatures(ctx, alice)
+			if err != nil || len(sigs) != 1 || sigs[0].Signature != "s2" {
+				t.Errorf("Signatures = %+v, %v; want the later signature only", sigs, err)
+			}
+		})
+	}
+}
+
+func TestKeyBackup(t *testing.T) {
+	for _, driver := range testDrivers {
+		t.Run(driver, func(t *testing.T) {
+			ctx := t.Context()
+			keys := migratedKeysDB(t, driver).Keys()
+
+			if _, err := keys.Backup(ctx, alice, ""); !errors.Is(err, storage.ErrNoSuchBackup) {
+				t.Fatalf("Backup with none = %v, want ErrNoSuchBackup", err)
+			}
+			v1, err := keys.CreateBackup(ctx, alice, "m.megolm_backup.v1.curve25519-aes-sha2", []byte(`{"public_key":"a"}`))
+			if err != nil || v1 != "1" {
+				t.Fatalf("CreateBackup = %q, %v", v1, err)
+			}
+
+			k := storage.BackupKey{RoomID: "!r:x", SessionID: "s1", FirstMessageIndex: 5, ForwardedCount: 1, SessionData: []byte(`{"d":1}`)}
+			v, err := keys.PutBackupKeys(ctx, alice, v1, []storage.BackupKey{k})
+			if err != nil || v.Count != 1 || v.ETag != "1" {
+				t.Fatalf("PutBackupKeys = %+v, %v", v, err)
+			}
+
+			// A worse key does not replace the one there and does not
+			// move the etag; a better one does.
+			worse := k
+			worse.FirstMessageIndex, worse.SessionData = 9, []byte(`{"d":"worse"}`)
+			if v, _ = keys.PutBackupKeys(ctx, alice, v1, []storage.BackupKey{worse}); v.ETag != "1" {
+				t.Errorf("a worse key moved the etag to %s", v.ETag)
+			}
+			verified := worse
+			verified.IsVerified, verified.SessionData = true, []byte(`{"d":"verified"}`)
+			if v, _ = keys.PutBackupKeys(ctx, alice, v1, []storage.BackupKey{verified}); v.ETag != "2" {
+				t.Errorf("a verified key did not replace an unverified one: etag %s", v.ETag)
+			}
+			got, err := keys.BackupKeys(ctx, alice, v1, "!r:x", "s1")
+			if err != nil || len(got) != 1 || string(got[0].SessionData) != `{"d":"verified"}` {
+				t.Fatalf("BackupKeys = %+v, %v", got, err)
+			}
+
+			_, _ = keys.PutBackupKeys(ctx, alice, v1, []storage.BackupKey{{RoomID: "!r:x", SessionID: "s2", SessionData: []byte(`{}`)}})
+			if all, _ := keys.BackupKeys(ctx, alice, v1, "", ""); len(all) != 2 {
+				t.Errorf("BackupKeys(all) = %d keys, want 2", len(all))
+			}
+			v, err = keys.DeleteBackupKeys(ctx, alice, v1, "!r:x", "s2")
+			if err != nil || v.Count != 1 {
+				t.Errorf("DeleteBackupKeys = %+v, %v", v, err)
+			}
+
+			if err := keys.UpdateBackup(ctx, alice, v1, []byte(`{"public_key":"b"}`)); err != nil {
+				t.Fatalf("UpdateBackup: %v", err)
+			}
+			current, _ := keys.Backup(ctx, alice, "")
+			if current.Version != v1 || string(current.AuthData) != `{"public_key":"b"}` {
+				t.Errorf("Backup = %+v", current)
+			}
+
+			// Deleting a version deletes its keys, and the next version
+			// does not reuse its number.
+			if err := keys.DeleteBackup(ctx, alice, v1); err != nil {
+				t.Fatalf("DeleteBackup: %v", err)
+			}
+			if _, err := keys.Backup(ctx, alice, v1); !errors.Is(err, storage.ErrNoSuchBackup) {
+				t.Errorf("Backup of a deleted version = %v", err)
+			}
+			if _, err := keys.BackupKeys(ctx, alice, v1, "", ""); !errors.Is(err, storage.ErrNoSuchBackup) {
+				t.Errorf("BackupKeys of a deleted version = %v", err)
+			}
+			if v2, _ := keys.CreateBackup(ctx, alice, "alg", []byte(`{}`)); v2 != "2" {
+				t.Errorf("the next version is %q, want 2", v2)
+			}
+			if _, err := keys.Backup(ctx, alice, "not-a-number"); !errors.Is(err, storage.ErrNoSuchBackup) {
+				t.Errorf("Backup of a malformed version = %v", err)
+			}
+		})
+	}
+}
+
+func TestUsersSharingRoomsWithNobody(t *testing.T) {
+	for _, driver := range testDrivers {
+		t.Run(driver, func(t *testing.T) {
+			db := migratedKeysDB(t, driver)
+			users, err := db.Rooms().UsersSharingRooms(t.Context(), "@nobody:warp.example")
+			if err != nil || len(users) != 0 {
+				t.Errorf("UsersSharingRooms = %v, %v", users, err)
+			}
+			changes, err := db.Rooms().MemberChanges(t.Context(), []string{"!none:warp.example"}, 0, 0)
+			if err != nil || len(changes) != 0 {
+				t.Errorf("MemberChanges = %v, %v", changes, err)
+			}
+		})
+	}
+}

server/internal/storage/pocketbase/pocketbase.go

@@ -66,6 +66,9 @@ func (*Database) Federation() storage.FederationStore { return notImplementedFed
 // Media implements storage.Database.
 func (*Database) Media() storage.MediaStore { return notImplementedMedia{} }
 
+// Keys implements storage.Database.
+func (*Database) Keys() storage.KeyStore { return notImplementedKeys{} }
+
 // Close implements storage.Database.
 func (*Database) Close() error { return nil }
 
@@ -324,6 +327,14 @@ func (notImplementedRooms) EventByID(context.Context, string) (storage.StoredEve
 	return storage.StoredEvent{}, storage.ErrNotImplemented
 }
 
+func (notImplementedRooms) UsersSharingRooms(context.Context, string) ([]string, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedRooms) MemberChanges(context.Context, []string, int64, int64) ([]storage.MemberChange, error) {
+	return nil, storage.ErrNotImplemented
+}
+
 // notImplementedFederation satisfies storage.FederationStore the same way.
 // A queue that accepted events and never stored them would lose them the
 // moment federation was switched on.
@@ -475,4 +486,98 @@ var (
 	_ storage.SyncStore       = notImplementedSync{}
 	_ storage.FederationStore = notImplementedFederation{}
 	_ storage.MediaStore      = notImplementedMedia{}
+	_ storage.KeyStore        = notImplementedKeys{}
 )
+
+// notImplementedKeys satisfies storage.KeyStore the same way. A key upload
+// that reported success and kept nothing would leave every message sent to
+// the device undecryptable.
+type notImplementedKeys struct{}
+
+func (notImplementedKeys) SetDeviceKeys(context.Context, storage.DeviceKeys) (storage.DeviceListUpdate, error) {
+	return storage.DeviceListUpdate{}, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeviceKeys(context.Context, string, []string) ([]storage.DeviceKeys, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeleteDeviceKeys(context.Context, string, []string) ([]storage.DeviceListUpdate, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) AddOneTimeKeys(context.Context, string, string, []storage.OneTimeKey) error {
+	return storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) OneTimeKeyCounts(context.Context, string, string) (map[string]int, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) SetFallbackKeys(context.Context, string, string, []storage.OneTimeKey) error {
+	return storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) UnusedFallbackKeyTypes(context.Context, string, string) ([]string, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) ClaimKey(context.Context, string, string, string) (storage.ClaimedKey, bool, error) {
+	return storage.ClaimedKey{}, false, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) SetCrossSigningKeys(context.Context, string, map[string][]byte) (storage.DeviceListUpdate, error) {
+	return storage.DeviceListUpdate{}, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) CrossSigningKeys(context.Context, string) (map[string][]byte, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) AddSignatures(context.Context, []storage.KeySignature) ([]storage.DeviceListUpdate, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) Signatures(context.Context, string) ([]storage.KeySignature, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) MarkDeviceListsChanged(context.Context, []string) (int64, error) {
+	return 0, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeviceListChanges(context.Context, []string, int64, int64) ([]storage.DeviceListChange, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeviceListPosition(context.Context, string) (int64, error) {
+	return 0, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) CreateBackup(context.Context, string, string, []byte) (string, error) {
+	return "", storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) Backup(context.Context, string, string) (storage.BackupVersion, error) {
+	return storage.BackupVersion{}, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) UpdateBackup(context.Context, string, string, []byte) error {
+	return storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeleteBackup(context.Context, string, string) error {
+	return storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) PutBackupKeys(context.Context, string, string, []storage.BackupKey) (storage.BackupVersion, error) {
+	return storage.BackupVersion{}, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) BackupKeys(context.Context, string, string, string, string) ([]storage.BackupKey, error) {
+	return nil, storage.ErrNotImplemented
+}
+
+func (notImplementedKeys) DeleteBackupKeys(context.Context, string, string, string, string) (storage.BackupVersion, error) {
+	return storage.BackupVersion{}, storage.ErrNotImplemented
+}

server/internal/storage/postgres/migrations/0008_keyserver.sql

@@ -0,0 +1,122 @@
+-- The keyserver: what end-to-end encryption needs the server to hold for
+-- its users' devices, none of which it can read or use itself.
+--
+-- Users are full user IDs throughout, not localparts. Most of what is here
+-- is about local devices, but signatures can be of other servers' users'
+-- keys, and the device list stream records their changes too: a local
+-- client sharing a room with a remote user has to be told when that user's
+-- keys move, and the stream is how a sync finds out.
+
+-- The counter of the device list stream. It is the keyserver's own table
+-- for the same reason every component has one, and the sync store reads it
+-- next to its own counters so that a sync token names one moment in all of
+-- them.
+CREATE TABLE IF NOT EXISTS keyserver_sequences (
+    name  TEXT   NOT NULL PRIMARY KEY,
+    value BIGINT NOT NULL
+);
+
+INSERT INTO keyserver_sequences (name, value) VALUES ('device_list', 0) ON CONFLICT (name) DO NOTHING;
+
+-- A device's identity keys: its curve25519 and ed25519 keys, signed by the
+-- ed25519 one, exactly as the device uploaded them.
+CREATE TABLE IF NOT EXISTS keyserver_device_keys (
+    user_id   TEXT NOT NULL,
+    device_id TEXT NOT NULL,
+    key_json  TEXT NOT NULL,
+    PRIMARY KEY (user_id, device_id)
+);
+
+-- One-time keys. A row is deleted the moment it is claimed: each key is
+-- used for exactly one Olm session, and handing it out twice would let two
+-- senders set up sessions the device can only tell apart by luck.
+CREATE TABLE IF NOT EXISTS keyserver_one_time_keys (
+    user_id   TEXT NOT NULL,
+    device_id TEXT NOT NULL,
+    -- "algorithm:id", as the key is named in the upload.
+    key_id    TEXT NOT NULL,
+    algorithm TEXT NOT NULL,
+    key_json  TEXT NOT NULL,
+    PRIMARY KEY (user_id, device_id, key_id)
+);
+
+-- Counting a device's keys by algorithm, and claiming the oldest.
+CREATE INDEX IF NOT EXISTS keyserver_one_time_keys_algorithm_idx
+    ON keyserver_one_time_keys (user_id, device_id, algorithm);
+
+-- Fallback keys: one per algorithm per device, handed out when the
+-- one-time keys have run out. Unlike a one-time key it survives being
+-- claimed, marked used, until the device uploads a new one.
+CREATE TABLE IF NOT EXISTS keyserver_fallback_keys (
+    user_id   TEXT    NOT NULL,
+    device_id TEXT    NOT NULL,
+    algorithm TEXT    NOT NULL,
+    key_id    TEXT    NOT NULL,
+    key_json  TEXT    NOT NULL,
+    used      BOOLEAN NOT NULL DEFAULT FALSE,
+    PRIMARY KEY (user_id, device_id, algorithm)
+);
+
+-- Cross-signing keys: a user's master, self-signing and user-signing keys.
+CREATE TABLE IF NOT EXISTS keyserver_cross_signing_keys (
+    user_id  TEXT NOT NULL,
+    -- master, self_signing or user_signing.
+    purpose  TEXT NOT NULL,
+    key_json TEXT NOT NULL,
+    PRIMARY KEY (user_id, purpose)
+);
+
+-- Signatures uploaded after the key they sign, through
+-- /keys/signatures/upload. They are merged into the key when it is read
+-- rather than written into it, so the key's own JSON stays what its owner
+-- uploaded.
+CREATE TABLE IF NOT EXISTS keyserver_signatures (
+    target_user_id TEXT NOT NULL,
+    -- A device ID, or a cross-signing key's public key.
+    target_key_id  TEXT NOT NULL,
+    signer_user_id TEXT NOT NULL,
+    -- "ed25519:" and the signing key's ID.
+    signer_key_id  TEXT NOT NULL,
+    signature      TEXT NOT NULL,
+    PRIMARY KEY (target_user_id, target_key_id, signer_user_id, signer_key_id)
+);
+
+-- The device list stream: each user's latest position, local and remote.
+-- Only the latest is kept, as for presence: a sync asks whose list moved
+-- after its token, and the history of the moves does not answer that any
+-- better.
+CREATE TABLE IF NOT EXISTS keyserver_device_lists (
+    user_id    TEXT   NOT NULL PRIMARY KEY,
+    stream_pos BIGINT NOT NULL
+);
+
+CREATE INDEX IF NOT EXISTS keyserver_device_lists_stream_idx
+    ON keyserver_device_lists (stream_pos);
+
+-- Server-side key backup versions. A deleted version keeps its row, so
+-- that its number is never handed out again: a client still holding the
+-- old version number must not find itself writing into a new backup that
+-- is encrypted to a different key.
+CREATE TABLE IF NOT EXISTS keyserver_backup_versions (
+    user_id   TEXT    NOT NULL,
+    version   BIGINT  NOT NULL,
+    algorithm TEXT    NOT NULL,
+    auth_data TEXT    NOT NULL,
+    -- Bumped whenever a key in the version changes.
+    etag      BIGINT  NOT NULL DEFAULT 0,
+    deleted   BOOLEAN NOT NULL DEFAULT FALSE,
+    PRIMARY KEY (user_id, version)
+);
+
+-- The backed-up room keys, one per Megolm session.
+CREATE TABLE IF NOT EXISTS keyserver_backup_keys (
+    user_id             TEXT    NOT NULL,
+    version             BIGINT  NOT NULL,
+    room_id             TEXT    NOT NULL,
+    session_id          TEXT    NOT NULL,
+    first_message_index BIGINT  NOT NULL,
+    forwarded_count     BIGINT  NOT NULL,
+    is_verified         BOOLEAN NOT NULL,
+    session_data        TEXT    NOT NULL,
+    PRIMARY KEY (user_id, version, room_id, session_id)
+);

server/internal/storage/rooms.go

@@ -249,6 +249,20 @@ type MembershipEntry struct {
 	EventID    string
 }
 
+// MemberChange is one membership event in a window of the event stream.
+type MemberChange struct {
+	RoomID string
+
+	// UserID is the state key: the user whose membership it set.
+	UserID string
+
+	StreamOrdering int64
+
+	// Membership is the user's membership of the room now, which may
+	// have moved again since this event.
+	Membership string
+}
+
 // RoomFilter narrows and pages a room listing.
 type RoomFilter struct {
 	// Search matches a substring of the room ID, name, topic or canonical
@@ -462,6 +476,18 @@ type RoomStore interface {
 	// thousands.
 	HasJoinedMemberOn(ctx context.Context, roomID, serverName string) (bool, error)
 
+	// UsersSharingRooms lists the users joined to at least one room the
+	// given user is joined to, that user included when they are joined
+	// anywhere. It is whose device list changes a client is told about,
+	// and who is woken when this user's keys change.
+	UsersSharingRooms(ctx context.Context, userID string) ([]string, error)
+
+	// MemberChanges lists the m.room.member events of the given rooms
+	// after from and at or before to: whose membership moved in a window,
+	// which is what tells a sync that it now shares a room with somebody
+	// it did not before, or no longer does.
+	MemberChanges(ctx context.Context, roomIDs []string, from, to int64) ([]MemberChange, error)
+
 	// EventByID reads an event without knowing its room, returning
 	// ErrNoSuchEvent if it is absent. The federation API's /event names
 	// only the event; everything else asks within a room.

server/internal/storage/sqlite/migrations/0008_keyserver.sql

@@ -0,0 +1,122 @@
+-- The keyserver: what end-to-end encryption needs the server to hold for
+-- its users' devices, none of which it can read or use itself.
+--
+-- Users are full user IDs throughout, not localparts. Most of what is here
+-- is about local devices, but signatures can be of other servers' users'
+-- keys, and the device list stream records their changes too: a local
+-- client sharing a room with a remote user has to be told when that user's
+-- keys move, and the stream is how a sync finds out.
+
+-- The counter of the device list stream. It is the keyserver's own table
+-- for the same reason every component has one, and the sync store reads it
+-- next to its own counters so that a sync token names one moment in all of
+-- them.
+CREATE TABLE IF NOT EXISTS keyserver_sequences (
+    name  TEXT   NOT NULL PRIMARY KEY,
+    value BIGINT NOT NULL
+);
+
+INSERT INTO keyserver_sequences (name, value) VALUES ('device_list', 0) ON CONFLICT (name) DO NOTHING;
+
+-- A device's identity keys: its curve25519 and ed25519 keys, signed by the
+-- ed25519 one, exactly as the device uploaded them.
+CREATE TABLE IF NOT EXISTS keyserver_device_keys (
+    user_id   TEXT NOT NULL,
+    device_id TEXT NOT NULL,
+    key_json  TEXT NOT NULL,
+    PRIMARY KEY (user_id, device_id)
+);
+
+-- One-time keys. A row is deleted the moment it is claimed: each key is
+-- used for exactly one Olm session, and handing it out twice would let two
+-- senders set up sessions the device can only tell apart by luck.
+CREATE TABLE IF NOT EXISTS keyserver_one_time_keys (
+    user_id   TEXT NOT NULL,
+    device_id TEXT NOT NULL,
+    -- "algorithm:id", as the key is named in the upload.
+    key_id    TEXT NOT NULL,
+    algorithm TEXT NOT NULL,
+    key_json  TEXT NOT NULL,
+    PRIMARY KEY (user_id, device_id, key_id)
+);
+
+-- Counting a device's keys by algorithm, and claiming the oldest.
+CREATE INDEX IF NOT EXISTS keyserver_one_time_keys_algorithm_idx
+    ON keyserver_one_time_keys (user_id, device_id, algorithm);
+
+-- Fallback keys: one per algorithm per device, handed out when the
+-- one-time keys have run out. Unlike a one-time key it survives being
+-- claimed, marked used, until the device uploads a new one.
+CREATE TABLE IF NOT EXISTS keyserver_fallback_keys (
+    user_id   TEXT    NOT NULL,
+    device_id TEXT    NOT NULL,
+    algorithm TEXT    NOT NULL,
+    key_id    TEXT    NOT NULL,
+    key_json  TEXT    NOT NULL,
+    used      BOOLEAN NOT NULL DEFAULT 0,
+    PRIMARY KEY (user_id, device_id, algorithm)
+);
+
+-- Cross-signing keys: a user's master, self-signing and user-signing keys.
+CREATE TABLE IF NOT EXISTS keyserver_cross_signing_keys (
+    user_id  TEXT NOT NULL,
+    -- master, self_signing or user_signing.
+    purpose  TEXT NOT NULL,
+    key_json TEXT NOT NULL,
+    PRIMARY KEY (user_id, purpose)
+);
+
+-- Signatures uploaded after the key they sign, through
+-- /keys/signatures/upload. They are merged into the key when it is read
+-- rather than written into it, so the key's own JSON stays what its owner
+-- uploaded.
+CREATE TABLE IF NOT EXISTS keyserver_signatures (
+    target_user_id TEXT NOT NULL,
+    -- A device ID, or a cross-signing key's public key.
+    target_key_id  TEXT NOT NULL,
+    signer_user_id TEXT NOT NULL,
+    -- "ed25519:" and the signing key's ID.
+    signer_key_id  TEXT NOT NULL,
+    signature      TEXT NOT NULL,
+    PRIMARY KEY (target_user_id, target_key_id, signer_user_id, signer_key_id)
+);
+
+-- The device list stream: each user's latest position, local and remote.
+-- Only the latest is kept, as for presence: a sync asks whose list moved
+-- after its token, and the history of the moves does not answer that any
+-- better.
+CREATE TABLE IF NOT EXISTS keyserver_device_lists (
+    user_id    TEXT   NOT NULL PRIMARY KEY,
+    stream_pos BIGINT NOT NULL
+);
+
+CREATE INDEX IF NOT EXISTS keyserver_device_lists_stream_idx
+    ON keyserver_device_lists (stream_pos);
+
+-- Server-side key backup versions. A deleted version keeps its row, so
+-- that its number is never handed out again: a client still holding the
+-- old version number must not find itself writing into a new backup that
+-- is encrypted to a different key.
+CREATE TABLE IF NOT EXISTS keyserver_backup_versions (
+    user_id   TEXT    NOT NULL,
+    version   BIGINT  NOT NULL,
+    algorithm TEXT    NOT NULL,
+    auth_data TEXT    NOT NULL,
+    -- Bumped whenever a key in the version changes.
+    etag      BIGINT  NOT NULL DEFAULT 0,
+    deleted   BOOLEAN NOT NULL DEFAULT 0,
+    PRIMARY KEY (user_id, version)
+);
+
+-- The backed-up room keys, one per Megolm session.
+CREATE TABLE IF NOT EXISTS keyserver_backup_keys (
+    user_id             TEXT    NOT NULL,
+    version             BIGINT  NOT NULL,
+    room_id             TEXT    NOT NULL,
+    session_id          TEXT    NOT NULL,
+    first_message_index BIGINT  NOT NULL,
+    forwarded_count     BIGINT  NOT NULL,
+    is_verified         BOOLEAN NOT NULL,
+    session_data        TEXT    NOT NULL,
+    PRIMARY KEY (user_id, version, room_id, session_id)
+);

server/internal/storage/sqlstore/keys.go

@@ -0,0 +1,930 @@
+package sqlstore
+
+import (
+	"cmp"
+	"context"
+	"database/sql"
+	"errors"
+	"fmt"
+	"slices"
+	"strconv"
+
+	"github.com/dariuszjaniak/warp/server/internal/storage"
+)
+
+// Keys is the shared SQL implementation of storage.KeyStore.
+//
+// Callers hand it key JSON already in canonical form. That is what lets
+// "the same key uploaded again" be a byte comparison here: a client that
+// re-sends a key it already uploaded may well encode it differently, and
+// must not be told it is a conflicting one.
+type Keys struct {
+	db     *sql.DB
+	rebind func(string) string
+}
+
+// NewKeys builds a key store.
+func NewKeys(db *sql.DB, rebind func(string) string) *Keys {
+	if rebind == nil {
+		rebind = func(query string) string { return query }
+	}
+	return &Keys{db: db, rebind: rebind}
+}
+
+// The keyserver's sequence table and its one counter.
+const (
+	keySequences  = "keyserver_sequences"
+	seqDeviceList = "device_list"
+)
+
+// maxInList caps the size of an IN list. SQLite's limit on bound
+// parameters is in the tens of thousands and PostgreSQL's 65535, and a
+// sync of a user in a large room asks about every member of it.
+const maxInList = 500
+
+// claimAttempts bounds the retries of a claim that lost a race for a key.
+const claimAttempts = 5
+
+func (s *Keys) q(query string) string { return s.rebind(query) }
+
+// tx runs fn in a transaction, rolling back on any error.
+func (s *Keys) tx(ctx context.Context, fn func(*sql.Tx) error) error {
+	tx, err := s.db.BeginTx(ctx, nil)
+	if err != nil {
+		return fmt.Errorf("begin transaction: %w", err)
+	}
+	defer func() { _ = tx.Rollback() }()
+
+	if err := fn(tx); err != nil {
+		return err
+	}
+	if err := tx.Commit(); err != nil {
+		return fmt.Errorf("commit transaction: %w", err)
+	}
+	return nil
+}
+
+// --- the device list stream ---------------------------------------------------
+
+const (
+	selectDeviceListPosSQL = `
+		SELECT stream_pos FROM keyserver_device_lists WHERE user_id = ?`
+
+	upsertDeviceListPosSQL = `
+		INSERT INTO keyserver_device_lists (user_id, stream_pos) VALUES (?, ?)
+		ON CONFLICT (user_id) DO UPDATE SET stream_pos = excluded.stream_pos`
+)
+
+// bumpDeviceList moves a user's device list once per device named, or once
+// for the user when none is, and returns an update per move.
+//
+// Each device gets a position of its own because a remote server treats a
+// stream ID it has already seen as a duplicate: one position shared by the
+// deletion of three devices would reach it as one deletion and two
+// repeats.
+func bumpDeviceList(ctx context.Context, tx *sql.Tx, q func(string) string, userID string, deviceIDs []string) ([]storage.DeviceListUpdate, error) {
+	if len(deviceIDs) == 0 {
+		deviceIDs = []string{""}
+	}
+	n := int64(len(deviceIDs))
+	last, err := advanceSequence(ctx, tx, q, keySequences, seqDeviceList, n)
+	if err != nil {
+		return nil, err
+	}
+
+	var prev int64
+	err = tx.QueryRowContext(ctx, q(selectDeviceListPosSQL), userID).Scan(&prev)
+	if err != nil && !errors.Is(err, sql.ErrNoRows) {
+		return nil, fmt.Errorf("read the device list position of %q: %w", userID, err)
+	}
+
+	out := make([]storage.DeviceListUpdate, 0, n)
+	for i, deviceID := range deviceIDs {
+		pos := last - n + 1 + int64(i)
+		out = append(out, storage.DeviceListUpdate{
+			UserID: userID, DeviceID: deviceID, StreamPos: pos, PrevPos: prev,
+		})
+		prev = pos
+	}
+	if _, err := tx.ExecContext(ctx, q(upsertDeviceListPosSQL), userID, last); err != nil {
+		return nil, fmt.Errorf("move the device list of %q: %w", userID, err)
+	}
+	return out, nil
+}
+
+// MarkDeviceListsChanged implements storage.KeyStore.
+func (s *Keys) MarkDeviceListsChanged(ctx context.Context, userIDs []string) (int64, error) {
+	var top int64
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		for _, userID := range dedupe(userIDs) {
+			updates, err := bumpDeviceList(ctx, tx, s.q, userID, nil)
+			if err != nil {
+				return err
+			}
+			top = updates[len(updates)-1].StreamPos
+		}
+		return nil
+	})
+	return top, err
+}
+
+// DeviceListChanges implements storage.KeyStore.
+func (s *Keys) DeviceListChanges(ctx context.Context, userIDs []string, from, to int64) ([]storage.DeviceListChange, error) {
+	var out []storage.DeviceListChange
+	for chunk := range slices.Chunk(dedupe(userIDs), maxInList) {
+		query := `
+			SELECT user_id, stream_pos FROM keyserver_device_lists
+			WHERE user_id IN (` + placeholders(len(chunk)) + `) AND stream_pos > ?`
+		args := make([]any, 0, len(chunk)+2)
+		for _, userID := range chunk {
+			args = append(args, userID)
+		}
+		args = append(args, from)
+		if to > 0 {
+			query += ` AND stream_pos <= ?`
+			args = append(args, to)
+		}
+
+		changes, err := s.deviceListChanges(ctx, query, args)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, changes...)
+	}
+	slices.SortFunc(out, func(a, b storage.DeviceListChange) int { return cmp.Compare(a.StreamPos, b.StreamPos) })
+	return out, nil
+}
+
+// deviceListChanges runs one chunk of DeviceListChanges.
+func (s *Keys) deviceListChanges(ctx context.Context, query string, args []any) ([]storage.DeviceListChange, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(query), args...)
+	if err != nil {
+		return nil, fmt.Errorf("read device list changes: %w", err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []storage.DeviceListChange
+	for rows.Next() {
+		var c storage.DeviceListChange
+		if err := rows.Scan(&c.UserID, &c.StreamPos); err != nil {
+			return nil, fmt.Errorf("read device list changes: %w", err)
+		}
+		out = append(out, c)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, fmt.Errorf("read device list changes: %w", err)
+	}
+	return out, nil
+}
+
+// DeviceListPosition implements storage.KeyStore.
+func (s *Keys) DeviceListPosition(ctx context.Context, userID string) (int64, error) {
+	var pos int64
+	err := s.db.QueryRowContext(ctx, s.q(selectDeviceListPosSQL), userID).Scan(&pos)
+	switch {
+	case errors.Is(err, sql.ErrNoRows):
+		return 0, nil
+	case err != nil:
+		return 0, fmt.Errorf("read the device list position of %q: %w", userID, err)
+	}
+	return pos, nil
+}
+
+// --- device keys --------------------------------------------------------------
+
+const (
+	selectDeviceKeyJSONSQL = `
+		SELECT key_json FROM keyserver_device_keys WHERE user_id = ? AND device_id = ?`
+
+	upsertDeviceKeysSQL = `
+		INSERT INTO keyserver_device_keys (user_id, device_id, key_json) VALUES (?, ?, ?)
+		ON CONFLICT (user_id, device_id) DO UPDATE SET key_json = excluded.key_json`
+)
+
+// SetDeviceKeys implements storage.KeyStore.
+func (s *Keys) SetDeviceKeys(ctx context.Context, keys storage.DeviceKeys) (storage.DeviceListUpdate, error) {
+	var update storage.DeviceListUpdate
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		var existing string
+		err := tx.QueryRowContext(ctx, s.q(selectDeviceKeyJSONSQL), keys.UserID, keys.DeviceID).Scan(&existing)
+		switch {
+		case err == nil && existing == string(keys.JSON):
+			return nil
+		case err != nil && !errors.Is(err, sql.ErrNoRows):
+			return fmt.Errorf("read the keys of %s/%s: %w", keys.UserID, keys.DeviceID, err)
+		}
+
+		if _, err := tx.ExecContext(ctx, s.q(upsertDeviceKeysSQL), keys.UserID, keys.DeviceID, string(keys.JSON)); err != nil {
+			return fmt.Errorf("store the keys of %s/%s: %w", keys.UserID, keys.DeviceID, err)
+		}
+		updates, err := bumpDeviceList(ctx, tx, s.q, keys.UserID, []string{keys.DeviceID})
+		if err != nil {
+			return err
+		}
+		update = updates[0]
+		return nil
+	})
+	return update, err
+}
+
+// DeviceKeys implements storage.KeyStore.
+func (s *Keys) DeviceKeys(ctx context.Context, userID string, deviceIDs []string) ([]storage.DeviceKeys, error) {
+	query := `SELECT device_id, key_json FROM keyserver_device_keys WHERE user_id = ?`
+	args := []any{userID}
+	if len(deviceIDs) > 0 {
+		deviceIDs = dedupe(deviceIDs)
+		if len(deviceIDs) > maxInList {
+			deviceIDs = deviceIDs[:maxInList]
+		}
+		query += ` AND device_id IN (` + placeholders(len(deviceIDs)) + `)`
+		for _, deviceID := range deviceIDs {
+			args = append(args, deviceID)
+		}
+	}
+	query += ` ORDER BY device_id`
+
+	rows, err := s.db.QueryContext(ctx, s.q(query), args...)
+	if err != nil {
+		return nil, fmt.Errorf("read the device keys of %q: %w", userID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []storage.DeviceKeys
+	for rows.Next() {
+		k := storage.DeviceKeys{UserID: userID}
+		var keyJSON string
+		if err := rows.Scan(&k.DeviceID, &keyJSON); err != nil {
+			return nil, fmt.Errorf("read the device keys of %q: %w", userID, err)
+		}
+		k.JSON = []byte(keyJSON)
+		out = append(out, k)
+	}
+	return out, rows.Err()
+}
+
+// DeleteDeviceKeys implements storage.KeyStore.
+func (s *Keys) DeleteDeviceKeys(ctx context.Context, userID string, deviceIDs []string) ([]storage.DeviceListUpdate, error) {
+	var updates []storage.DeviceListUpdate
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		where := ` WHERE user_id = ?`
+		args := []any{userID}
+		if len(deviceIDs) > 0 {
+			deviceIDs = dedupe(deviceIDs)
+			where += ` AND device_id IN (` + placeholders(len(deviceIDs)) + `)`
+			for _, deviceID := range deviceIDs {
+				args = append(args, deviceID)
+			}
+		}
+
+		had, err := s.devicesWithKeys(ctx, tx, userID, where, args)
+		if err != nil {
+			return err
+		}
+
+		for _, table := range []string{"keyserver_device_keys", "keyserver_one_time_keys", "keyserver_fallback_keys"} {
+			if _, err := tx.ExecContext(ctx, s.q(`DELETE FROM `+table+where), args...); err != nil {
+				return fmt.Errorf("delete from %s for %q: %w", table, userID, err)
+			}
+		}
+		if len(had) == 0 {
+			return nil
+		}
+		updates, err = bumpDeviceList(ctx, tx, s.q, userID, had)
+		return err
+	})
+	return updates, err
+}
+
+// devicesWithKeys lists the devices a DeleteDeviceKeys will take identity
+// keys from, which are the ones whose deletion moves the stream.
+func (s *Keys) devicesWithKeys(ctx context.Context, tx *sql.Tx, userID, where string, args []any) ([]string, error) {
+	rows, err := tx.QueryContext(ctx, s.q(`SELECT device_id FROM keyserver_device_keys`+where+` ORDER BY device_id`), args...)
+	if err != nil {
+		return nil, fmt.Errorf("read the devices of %q: %w", userID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []string
+	for rows.Next() {
+		var deviceID string
+		if err := rows.Scan(&deviceID); err != nil {
+			return nil, fmt.Errorf("read the devices of %q: %w", userID, err)
+		}
+		out = append(out, deviceID)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, fmt.Errorf("read the devices of %q: %w", userID, err)
+	}
+	return out, nil
+}
+
+// --- one-time and fallback keys -----------------------------------------------
+
+const (
+	selectOneTimeKeySQL = `
+		SELECT key_json FROM keyserver_one_time_keys
+		WHERE user_id = ? AND device_id = ? AND key_id = ?`
+
+	insertOneTimeKeySQL = `
+		INSERT INTO keyserver_one_time_keys (user_id, device_id, key_id, algorithm, key_json)
+		VALUES (?, ?, ?, ?, ?)`
+
+	countOneTimeKeysSQL = `
+		SELECT algorithm, COUNT(*) FROM keyserver_one_time_keys
+		WHERE user_id = ? AND device_id = ? GROUP BY algorithm`
+
+	selectFallbackKeySQL = `
+		SELECT key_id, key_json FROM keyserver_fallback_keys
+		WHERE user_id = ? AND device_id = ? AND algorithm = ?`
+
+	upsertFallbackKeySQL = `
+		INSERT INTO keyserver_fallback_keys (user_id, device_id, algorithm, key_id, key_json, used)
+		VALUES (?, ?, ?, ?, ?, ?)
+		ON CONFLICT (user_id, device_id, algorithm) DO UPDATE SET
+			key_id = excluded.key_id, key_json = excluded.key_json, used = excluded.used`
+
+	selectUnusedFallbackSQL = `
+		SELECT algorithm FROM keyserver_fallback_keys
+		WHERE user_id = ? AND device_id = ? AND used = ? ORDER BY algorithm`
+
+	selectClaimableKeySQL = `
+		SELECT key_id, key_json FROM keyserver_one_time_keys
+		WHERE user_id = ? AND device_id = ? AND algorithm = ?
+		ORDER BY key_id LIMIT 1`
+
+	deleteOneTimeKeySQL = `
+		DELETE FROM keyserver_one_time_keys WHERE user_id = ? AND device_id = ? AND key_id = ?`
+
+	markFallbackUsedSQL = `
+		UPDATE keyserver_fallback_keys SET used = ?
+		WHERE user_id = ? AND device_id = ? AND algorithm = ?`
+)
+
+// AddOneTimeKeys implements storage.KeyStore.
+func (s *Keys) AddOneTimeKeys(ctx context.Context, userID, deviceID string, keys []storage.OneTimeKey) error {
+	if len(keys) == 0 {
+		return nil
+	}
+	return s.tx(ctx, func(tx *sql.Tx) error {
+		for _, k := range keys {
+			var existing string
+			err := tx.QueryRowContext(ctx, s.q(selectOneTimeKeySQL), userID, deviceID, k.KeyID).Scan(&existing)
+			switch {
+			case err == nil && existing == string(k.JSON):
+				continue
+			case err == nil:
+				return fmt.Errorf("%w: %s", storage.ErrKeyConflict, k.KeyID)
+			case !errors.Is(err, sql.ErrNoRows):
+				return fmt.Errorf("read one-time key %s of %s/%s: %w", k.KeyID, userID, deviceID, err)
+			}
+			if _, err := tx.ExecContext(ctx, s.q(insertOneTimeKeySQL),
+				userID, deviceID, k.KeyID, k.Algorithm(), string(k.JSON)); err != nil {
+				return fmt.Errorf("store one-time key %s of %s/%s: %w", k.KeyID, userID, deviceID, err)
+			}
+		}
+		return nil
+	})
+}
+
+// OneTimeKeyCounts implements storage.KeyStore.
+func (s *Keys) OneTimeKeyCounts(ctx context.Context, userID, deviceID string) (map[string]int, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(countOneTimeKeysSQL), userID, deviceID)
+	if err != nil {
+		return nil, fmt.Errorf("count the one-time keys of %s/%s: %w", userID, deviceID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	out := map[string]int{}
+	for rows.Next() {
+		var (
+			algorithm string
+			n         int
+		)
+		if err := rows.Scan(&algorithm, &n); err != nil {
+			return nil, fmt.Errorf("count the one-time keys of %s/%s: %w", userID, deviceID, err)
+		}
+		out[algorithm] = n
+	}
+	return out, rows.Err()
+}
+
+// SetFallbackKeys implements storage.KeyStore.
+func (s *Keys) SetFallbackKeys(ctx context.Context, userID, deviceID string, keys []storage.OneTimeKey) error {
+	if len(keys) == 0 {
+		return nil
+	}
+	return s.tx(ctx, func(tx *sql.Tx) error {
+		for _, k := range keys {
+			var existingID, existingJSON string
+			err := tx.QueryRowContext(ctx, s.q(selectFallbackKeySQL), userID, deviceID, k.Algorithm()).
+				Scan(&existingID, &existingJSON)
+			switch {
+			case err == nil && existingID == k.KeyID && existingJSON == string(k.JSON):
+				// The same key again keeps its used flag.
+				continue
+			case err != nil && !errors.Is(err, sql.ErrNoRows):
+				return fmt.Errorf("read the fallback key of %s/%s: %w", userID, deviceID, err)
+			}
+			if _, err := tx.ExecContext(ctx, s.q(upsertFallbackKeySQL),
+				userID, deviceID, k.Algorithm(), k.KeyID, string(k.JSON), false); err != nil {
+				return fmt.Errorf("store the fallback key of %s/%s: %w", userID, deviceID, err)
+			}
+		}
+		return nil
+	})
+}
+
+// UnusedFallbackKeyTypes implements storage.KeyStore.
+func (s *Keys) UnusedFallbackKeyTypes(ctx context.Context, userID, deviceID string) ([]string, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(selectUnusedFallbackSQL), userID, deviceID, false)
+	if err != nil {
+		return nil, fmt.Errorf("read the fallback keys of %s/%s: %w", userID, deviceID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []string
+	for rows.Next() {
+		var algorithm string
+		if err := rows.Scan(&algorithm); err != nil {
+			return nil, fmt.Errorf("read the fallback keys of %s/%s: %w", userID, deviceID, err)
+		}
+		out = append(out, algorithm)
+	}
+	return out, rows.Err()
+}
+
+// ClaimKey implements storage.KeyStore.
+//
+// The one-time key is read and then deleted, and the claim is only won if
+// the delete removed the row: two servers claiming a key of the same
+// device at the same moment can both read it, and only one of them may
+// have it. The loser tries the next key.
+func (s *Keys) ClaimKey(ctx context.Context, userID, deviceID, algorithm string) (storage.ClaimedKey, bool, error) {
+	for range claimAttempts {
+		var (
+			claimed storage.ClaimedKey
+			found   bool
+			lost    bool
+		)
+		err := s.tx(ctx, func(tx *sql.Tx) error {
+			var keyID, keyJSON string
+			err := tx.QueryRowContext(ctx, s.q(selectClaimableKeySQL), userID, deviceID, algorithm).Scan(&keyID, &keyJSON)
+			switch {
+			case err == nil:
+				res, err := tx.ExecContext(ctx, s.q(deleteOneTimeKeySQL), userID, deviceID, keyID)
+				if err != nil {
+					return fmt.Errorf("claim one-time key %s of %s/%s: %w", keyID, userID, deviceID, err)
+				}
+				if n, _ := res.RowsAffected(); n != 1 {
+					lost = true
+					return nil
+				}
+				claimed = storage.ClaimedKey{OneTimeKey: storage.OneTimeKey{KeyID: keyID, JSON: []byte(keyJSON)}}
+				found = true
+				return nil
+			case !errors.Is(err, sql.ErrNoRows):
+				return fmt.Errorf("claim a one-time key of %s/%s: %w", userID, deviceID, err)
+			}
+
+			err = tx.QueryRowContext(ctx, s.q(selectFallbackKeySQL), userID, deviceID, algorithm).Scan(&keyID, &keyJSON)
+			switch {
+			case errors.Is(err, sql.ErrNoRows):
+				return nil
+			case err != nil:
+				return fmt.Errorf("claim the fallback key of %s/%s: %w", userID, deviceID, err)
+			}
+			if _, err := tx.ExecContext(ctx, s.q(markFallbackUsedSQL), true, userID, deviceID, algorithm); err != nil {
+				return fmt.Errorf("claim the fallback key of %s/%s: %w", userID, deviceID, err)
+			}
+			claimed = storage.ClaimedKey{OneTimeKey: storage.OneTimeKey{KeyID: keyID, JSON: []byte(keyJSON)}, Fallback: true}
+			found = true
+			return nil
+		})
+		if err != nil {
+			return storage.ClaimedKey{}, false, err
+		}
+		if !lost {
+			return claimed, found, nil
+		}
+	}
+	return storage.ClaimedKey{}, false, fmt.Errorf("claim a key of %s/%s: lost the race %d times", userID, deviceID, claimAttempts)
+}
+
+// --- cross-signing ------------------------------------------------------------
+
+const (
+	upsertCrossSigningKeySQL = `
+		INSERT INTO keyserver_cross_signing_keys (user_id, purpose, key_json) VALUES (?, ?, ?)
+		ON CONFLICT (user_id, purpose) DO UPDATE SET key_json = excluded.key_json`
+
+	deleteCrossSigningKeySQL = `
+		DELETE FROM keyserver_cross_signing_keys WHERE user_id = ? AND purpose = ?`
+
+	selectCrossSigningKeysSQL = `
+		SELECT purpose, key_json FROM keyserver_cross_signing_keys WHERE user_id = ?`
+)
+
+// SetCrossSigningKeys implements storage.KeyStore.
+func (s *Keys) SetCrossSigningKeys(ctx context.Context, userID string, keys map[string][]byte) (storage.DeviceListUpdate, error) {
+	var update storage.DeviceListUpdate
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		for _, purpose := range sortedKeys(keys) {
+			keyJSON := keys[purpose]
+			var err error
+			if keyJSON == nil {
+				_, err = tx.ExecContext(ctx, s.q(deleteCrossSigningKeySQL), userID, purpose)
+			} else {
+				_, err = tx.ExecContext(ctx, s.q(upsertCrossSigningKeySQL), userID, purpose, string(keyJSON))
+			}
+			if err != nil {
+				return fmt.Errorf("store the %s key of %q: %w", purpose, userID, err)
+			}
+		}
+		updates, err := bumpDeviceList(ctx, tx, s.q, userID, nil)
+		if err != nil {
+			return err
+		}
+		update = updates[0]
+		return nil
+	})
+	return update, err
+}
+
+// CrossSigningKeys implements storage.KeyStore.
+func (s *Keys) CrossSigningKeys(ctx context.Context, userID string) (map[string][]byte, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(selectCrossSigningKeysSQL), userID)
+	if err != nil {
+		return nil, fmt.Errorf("read the cross-signing keys of %q: %w", userID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	out := map[string][]byte{}
+	for rows.Next() {
+		var purpose, keyJSON string
+		if err := rows.Scan(&purpose, &keyJSON); err != nil {
+			return nil, fmt.Errorf("read the cross-signing keys of %q: %w", userID, err)
+		}
+		out[purpose] = []byte(keyJSON)
+	}
+	return out, rows.Err()
+}
+
+// --- signatures ---------------------------------------------------------------
+
+const (
+	upsertSignatureSQL = `
+		INSERT INTO keyserver_signatures
+			(target_user_id, target_key_id, signer_user_id, signer_key_id, signature)
+		VALUES (?, ?, ?, ?, ?)
+		ON CONFLICT (target_user_id, target_key_id, signer_user_id, signer_key_id)
+		DO UPDATE SET signature = excluded.signature`
+
+	selectSignaturesSQL = `
+		SELECT target_key_id, signer_user_id, signer_key_id, signature
+		FROM keyserver_signatures WHERE target_user_id = ?
+		ORDER BY target_key_id, signer_user_id, signer_key_id`
+)
+
+// AddSignatures implements storage.KeyStore.
+func (s *Keys) AddSignatures(ctx context.Context, sigs []storage.KeySignature) ([]storage.DeviceListUpdate, error) {
+	if len(sigs) == 0 {
+		return nil, nil
+	}
+	var updates []storage.DeviceListUpdate
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		targets := map[string][]string{}
+		for _, sig := range sigs {
+			if _, err := tx.ExecContext(ctx, s.q(upsertSignatureSQL),
+				sig.TargetUserID, sig.TargetKeyID, sig.SignerUserID, sig.SignerKeyID, sig.Signature); err != nil {
+				return fmt.Errorf("store a signature of %s/%s: %w", sig.TargetUserID, sig.TargetKeyID, err)
+			}
+			if !slices.Contains(targets[sig.TargetUserID], sig.TargetKeyID) {
+				targets[sig.TargetUserID] = append(targets[sig.TargetUserID], sig.TargetKeyID)
+			}
+		}
+		for _, userID := range sortedKeys(targets) {
+			moved, err := bumpDeviceList(ctx, tx, s.q, userID, targets[userID])
+			if err != nil {
+				return err
+			}
+			updates = append(updates, moved...)
+		}
+		return nil
+	})
+	return updates, err
+}
+
+// Signatures implements storage.KeyStore.
+func (s *Keys) Signatures(ctx context.Context, targetUserID string) ([]storage.KeySignature, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(selectSignaturesSQL), targetUserID)
+	if err != nil {
+		return nil, fmt.Errorf("read the signatures of %q's keys: %w", targetUserID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []storage.KeySignature
+	for rows.Next() {
+		sig := storage.KeySignature{TargetUserID: targetUserID}
+		if err := rows.Scan(&sig.TargetKeyID, &sig.SignerUserID, &sig.SignerKeyID, &sig.Signature); err != nil {
+			return nil, fmt.Errorf("read the signatures of %q's keys: %w", targetUserID, err)
+		}
+		out = append(out, sig)
+	}
+	return out, rows.Err()
+}
+
+// --- key backup ---------------------------------------------------------------
+
+const (
+	selectMaxBackupVersionSQL = `
+		SELECT COALESCE(MAX(version), 0) FROM keyserver_backup_versions WHERE user_id = ?`
+
+	insertBackupVersionSQL = `
+		INSERT INTO keyserver_backup_versions (user_id, version, algorithm, auth_data, etag, deleted)
+		VALUES (?, ?, ?, ?, 0, ?)`
+
+	selectCurrentBackupVersionSQL = `
+		SELECT COALESCE(MAX(version), 0) FROM keyserver_backup_versions
+		WHERE user_id = ? AND deleted = ?`
+
+	selectBackupVersionSQL = `
+		SELECT algorithm, auth_data, etag FROM keyserver_backup_versions
+		WHERE user_id = ? AND version = ? AND deleted = ?`
+
+	countBackupKeysSQL = `
+		SELECT COUNT(*) FROM keyserver_backup_keys WHERE user_id = ? AND version = ?`
+
+	updateBackupAuthDataSQL = `
+		UPDATE keyserver_backup_versions SET auth_data = ?
+		WHERE user_id = ? AND version = ? AND deleted = ?`
+
+	deleteBackupVersionSQL = `
+		UPDATE keyserver_backup_versions SET deleted = ?
+		WHERE user_id = ? AND version = ? AND deleted = ?`
+
+	bumpBackupETagSQL = `
+		UPDATE keyserver_backup_versions SET etag = etag + 1 WHERE user_id = ? AND version = ?`
+
+	selectBackupKeySQL = `
+		SELECT first_message_index, forwarded_count, is_verified, session_data
+		FROM keyserver_backup_keys
+		WHERE user_id = ? AND version = ? AND room_id = ? AND session_id = ?`
+
+	upsertBackupKeySQL = `
+		INSERT INTO keyserver_backup_keys
+			(user_id, version, room_id, session_id, first_message_index, forwarded_count,
+			 is_verified, session_data)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+		ON CONFLICT (user_id, version, room_id, session_id) DO UPDATE SET
+			first_message_index = excluded.first_message_index,
+			forwarded_count = excluded.forwarded_count,
+			is_verified = excluded.is_verified,
+			session_data = excluded.session_data`
+)
+
+// rowQuerier is the part of *sql.Tx and *sql.DB a single-row read needs.
+type rowQuerier interface {
+	QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
+}
+
+// parseBackupVersion reads a version number. A version that is not one is
+// a version that does not exist, not a malformed request: the spec makes
+// versions opaque strings, and a client is entitled to ask about any.
+func parseBackupVersion(version string) (int64, bool) {
+	n, err := strconv.ParseInt(version, 10, 64)
+	return n, err == nil && n > 0
+}
+
+// readBackup reads one live version, with its key count.
+func (s *Keys) readBackup(ctx context.Context, db rowQuerier, userID, version string) (storage.BackupVersion, int64, error) {
+	var n int64
+	if version == "" {
+		if err := db.QueryRowContext(ctx, s.q(selectCurrentBackupVersionSQL), userID, false).Scan(&n); err != nil {
+			return storage.BackupVersion{}, 0, fmt.Errorf("read the backup version of %q: %w", userID, err)
+		}
+		if n == 0 {
+			return storage.BackupVersion{}, 0, storage.ErrNoSuchBackup
+		}
+	} else {
+		var ok bool
+		if n, ok = parseBackupVersion(version); !ok {
+			return storage.BackupVersion{}, 0, storage.ErrNoSuchBackup
+		}
+	}
+
+	out := storage.BackupVersion{UserID: userID, Version: strconv.FormatInt(n, 10)}
+	var (
+		authData string
+		etag     int64
+	)
+	err := db.QueryRowContext(ctx, s.q(selectBackupVersionSQL), userID, n, false).Scan(&out.Algorithm, &authData, &etag)
+	switch {
+	case errors.Is(err, sql.ErrNoRows):
+		return storage.BackupVersion{}, 0, storage.ErrNoSuchBackup
+	case err != nil:
+		return storage.BackupVersion{}, 0, fmt.Errorf("read backup version %d of %q: %w", n, userID, err)
+	}
+	out.AuthData = []byte(authData)
+	out.ETag = strconv.FormatInt(etag, 10)
+	if err := db.QueryRowContext(ctx, s.q(countBackupKeysSQL), userID, n).Scan(&out.Count); err != nil {
+		return storage.BackupVersion{}, 0, fmt.Errorf("count the keys of backup version %d of %q: %w", n, userID, err)
+	}
+	return out, n, nil
+}
+
+// CreateBackup implements storage.KeyStore.
+func (s *Keys) CreateBackup(ctx context.Context, userID, algorithm string, authData []byte) (string, error) {
+	var version int64
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		if err := tx.QueryRowContext(ctx, s.q(selectMaxBackupVersionSQL), userID).Scan(&version); err != nil {
+			return fmt.Errorf("read the backup versions of %q: %w", userID, err)
+		}
+		version++
+		if _, err := tx.ExecContext(ctx, s.q(insertBackupVersionSQL), userID, version, algorithm, string(authData), false); err != nil {
+			return fmt.Errorf("create a backup version for %q: %w", userID, err)
+		}
+		return nil
+	})
+	if err != nil {
+		return "", err
+	}
+	return strconv.FormatInt(version, 10), nil
+}
+
+// Backup implements storage.KeyStore.
+func (s *Keys) Backup(ctx context.Context, userID, version string) (storage.BackupVersion, error) {
+	out, _, err := s.readBackup(ctx, s.db, userID, version)
+	return out, err
+}
+
+// UpdateBackup implements storage.KeyStore.
+func (s *Keys) UpdateBackup(ctx context.Context, userID, version string, authData []byte) error {
+	n, ok := parseBackupVersion(version)
+	if !ok {
+		return storage.ErrNoSuchBackup
+	}
+	res, err := s.db.ExecContext(ctx, s.q(updateBackupAuthDataSQL), string(authData), userID, n, false)
+	if err != nil {
+		return fmt.Errorf("update backup version %d of %q: %w", n, userID, err)
+	}
+	if affected, _ := res.RowsAffected(); affected == 0 {
+		return storage.ErrNoSuchBackup
+	}
+	return nil
+}
+
+// DeleteBackup implements storage.KeyStore.
+func (s *Keys) DeleteBackup(ctx context.Context, userID, version string) error {
+	n, ok := parseBackupVersion(version)
+	if !ok {
+		return storage.ErrNoSuchBackup
+	}
+	return s.tx(ctx, func(tx *sql.Tx) error {
+		res, err := tx.ExecContext(ctx, s.q(deleteBackupVersionSQL), true, userID, n, false)
+		if err != nil {
+			return fmt.Errorf("delete backup version %d of %q: %w", n, userID, err)
+		}
+		if affected, _ := res.RowsAffected(); affected == 0 {
+			return storage.ErrNoSuchBackup
+		}
+		if _, err := tx.ExecContext(ctx, s.q(`DELETE FROM keyserver_backup_keys WHERE user_id = ? AND version = ?`), userID, n); err != nil {
+			return fmt.Errorf("delete the keys of backup version %d of %q: %w", n, userID, err)
+		}
+		return nil
+	})
+}
+
+// PutBackupKeys implements storage.KeyStore.
+func (s *Keys) PutBackupKeys(ctx context.Context, userID, version string, keys []storage.BackupKey) (storage.BackupVersion, error) {
+	var out storage.BackupVersion
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		_, n, err := s.readBackup(ctx, tx, userID, version)
+		if err != nil {
+			return err
+		}
+
+		changed := false
+		for _, k := range keys {
+			var (
+				existing    storage.BackupKey
+				sessionData string
+			)
+			err := tx.QueryRowContext(ctx, s.q(selectBackupKeySQL), userID, n, k.RoomID, k.SessionID).
+				Scan(&existing.FirstMessageIndex, &existing.ForwardedCount, &existing.IsVerified, &sessionData)
+			switch {
+			case err == nil:
+				if !k.Replaces(existing) {
+					continue
+				}
+			case !errors.Is(err, sql.ErrNoRows):
+				return fmt.Errorf("read backed-up key %s/%s: %w", k.RoomID, k.SessionID, err)
+			}
+			if _, err := tx.ExecContext(ctx, s.q(upsertBackupKeySQL), userID, n, k.RoomID, k.SessionID,
+				k.FirstMessageIndex, k.ForwardedCount, k.IsVerified, string(k.SessionData)); err != nil {
+				return fmt.Errorf("back up key %s/%s: %w", k.RoomID, k.SessionID, err)
+			}
+			changed = true
+		}
+		if changed {
+			if _, err := tx.ExecContext(ctx, s.q(bumpBackupETagSQL), userID, n); err != nil {
+				return fmt.Errorf("move the etag of backup version %d of %q: %w", n, userID, err)
+			}
+		}
+		out, _, err = s.readBackup(ctx, tx, userID, version)
+		return err
+	})
+	return out, err
+}
+
+// backupKeyScope is the WHERE clause the key reads and deletes share.
+func backupKeyScope(userID string, version int64, roomID, sessionID string) (string, []any) {
+	where := ` WHERE user_id = ? AND version = ?`
+	args := []any{userID, version}
+	if roomID != "" {
+		where += ` AND room_id = ?`
+		args = append(args, roomID)
+		if sessionID != "" {
+			where += ` AND session_id = ?`
+			args = append(args, sessionID)
+		}
+	}
+	return where, args
+}
+
+// BackupKeys implements storage.KeyStore.
+func (s *Keys) BackupKeys(ctx context.Context, userID, version, roomID, sessionID string) ([]storage.BackupKey, error) {
+	_, n, err := s.readBackup(ctx, s.db, userID, version)
+	if err != nil {
+		return nil, err
+	}
+	where, args := backupKeyScope(userID, n, roomID, sessionID)
+	rows, err := s.db.QueryContext(ctx, s.q(`
+		SELECT room_id, session_id, first_message_index, forwarded_count, is_verified, session_data
+		FROM keyserver_backup_keys`+where+` ORDER BY room_id, session_id`), args...)
+	if err != nil {
+		return nil, fmt.Errorf("read the keys of backup version %d of %q: %w", n, userID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []storage.BackupKey
+	for rows.Next() {
+		var (
+			k           storage.BackupKey
+			sessionData string
+		)
+		if err := rows.Scan(&k.RoomID, &k.SessionID, &k.FirstMessageIndex, &k.ForwardedCount,
+			&k.IsVerified, &sessionData); err != nil {
+			return nil, fmt.Errorf("read the keys of backup version %d of %q: %w", n, userID, err)
+		}
+		k.SessionData = []byte(sessionData)
+		out = append(out, k)
+	}
+	return out, rows.Err()
+}
+
+// DeleteBackupKeys implements storage.KeyStore.
+func (s *Keys) DeleteBackupKeys(ctx context.Context, userID, version, roomID, sessionID string) (storage.BackupVersion, error) {
+	var out storage.BackupVersion
+	err := s.tx(ctx, func(tx *sql.Tx) error {
+		_, n, err := s.readBackup(ctx, tx, userID, version)
+		if err != nil {
+			return err
+		}
+		where, args := backupKeyScope(userID, n, roomID, sessionID)
+		res, err := tx.ExecContext(ctx, s.q(`DELETE FROM keyserver_backup_keys`+where), args...)
+		if err != nil {
+			return fmt.Errorf("delete keys from backup version %d of %q: %w", n, userID, err)
+		}
+		if affected, _ := res.RowsAffected(); affected > 0 {
+			if _, err := tx.ExecContext(ctx, s.q(bumpBackupETagSQL), userID, n); err != nil {
+				return fmt.Errorf("move the etag of backup version %d of %q: %w", n, userID, err)
+			}
+		}
+		out, _, err = s.readBackup(ctx, tx, userID, version)
+		return err
+	})
+	return out, err
+}
+
+// --- helpers ------------------------------------------------------------------
+
+// dedupe returns the distinct values of a list, sorted, so that a caller
+// naming a user twice does not move their stream twice.
+func dedupe(values []string) []string {
+	out := slices.Clone(values)
+	slices.Sort(out)
+	return slices.Compact(out)
+}
+
+// sortedKeys is the keys of a map in order, which makes the order of
+// writes — and so of the positions handed out — deterministic.
+func sortedKeys[V any](m map[string]V) []string {
+	out := make([]string, 0, len(m))
+	for k := range m {
+		out = append(out, k)
+	}
+	slices.Sort(out)
+	return out
+}
+
+var _ storage.KeyStore = (*Keys)(nil)

server/internal/storage/sqlstore/rooms.go

@@ -1195,3 +1195,78 @@ func (s *Rooms) HasJoinedMemberOn(ctx context.Context, roomID, serverName string
 	}
 	return true, nil
 }
+
+// selectUsersSharingRoomsSQL joins the current state to itself on the room:
+// the rows of every room the user is joined to, paired with every joined
+// member of the same room. Both sides are lookups on the membership index.
+const selectUsersSharingRoomsSQL = `
+	SELECT DISTINCT other.state_key
+	FROM roomserver_current_state mine
+	JOIN roomserver_current_state other
+		ON other.room_id = mine.room_id
+		AND other.event_type = 'm.room.member'
+		AND other.membership = 'join'
+	WHERE mine.event_type = 'm.room.member'
+		AND mine.state_key = ?
+		AND mine.membership = 'join'
+	ORDER BY other.state_key`
+
+// UsersSharingRooms implements storage.RoomStore.
+func (s *Rooms) UsersSharingRooms(ctx context.Context, userID string) ([]string, error) {
+	rows, err := s.db.QueryContext(ctx, s.q(selectUsersSharingRoomsSQL), userID)
+	if err != nil {
+		return nil, fmt.Errorf("read the users sharing a room with %q: %w", userID, err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []string
+	for rows.Next() {
+		var other string
+		if err := rows.Scan(&other); err != nil {
+			return nil, fmt.Errorf("read the users sharing a room with %q: %w", userID, err)
+		}
+		out = append(out, other)
+	}
+	return out, rows.Err()
+}
+
+// MemberChanges implements storage.RoomStore.
+func (s *Rooms) MemberChanges(ctx context.Context, roomIDs []string, from, to int64) ([]storage.MemberChange, error) {
+	if len(roomIDs) == 0 {
+		return nil, nil
+	}
+	query := `
+		SELECT e.room_id, e.state_key, e.stream_ordering, COALESCE(c.membership, '')
+		FROM roomserver_events e
+		LEFT JOIN roomserver_current_state c
+			ON c.room_id = e.room_id AND c.event_type = 'm.room.member' AND c.state_key = e.state_key
+		WHERE e.event_type = 'm.room.member' AND e.state_key IS NOT NULL AND NOT e.outlier
+			AND e.room_id IN (` + placeholders(len(roomIDs)) + `)
+			AND e.stream_ordering > ?`
+	args := make([]any, 0, len(roomIDs)+2)
+	for _, roomID := range roomIDs {
+		args = append(args, roomID)
+	}
+	args = append(args, from)
+	if to != 0 {
+		query += ` AND e.stream_ordering <= ?`
+		args = append(args, to)
+	}
+	query += ` ORDER BY e.stream_ordering`
+
+	rows, err := s.db.QueryContext(ctx, s.q(query), args...)
+	if err != nil {
+		return nil, fmt.Errorf("read membership changes: %w", err)
+	}
+	defer func() { _ = rows.Close() }()
+
+	var out []storage.MemberChange
+	for rows.Next() {
+		var c storage.MemberChange
+		if err := rows.Scan(&c.RoomID, &c.UserID, &c.StreamOrdering, &c.Membership); err != nil {
+			return nil, fmt.Errorf("read membership changes: %w", err)
+		}
+		out = append(out, c)
+	}
+	return out, rows.Err()
+}

server/internal/storage/sqlstore/sqlstore.go

@@ -61,6 +61,7 @@ type Store struct {
 	sync     *Sync
 	fed      *Federation
 	media    *Media
+	keys     *Keys
 }
 
 // New builds a Store. It does not touch the database.
@@ -81,6 +82,7 @@ func New(opts Options) (*Store, error) {
 		sync:     NewSync(opts.DB, opts.Rebind, opts.IsUniqueViolation),
 		fed:      NewFederation(opts.DB, opts.Rebind),
 		media:    NewMedia(opts.DB, opts.Rebind, opts.IsUniqueViolation),
+		keys:     NewKeys(opts.DB, opts.Rebind),
 	}, nil
 }
 
@@ -114,6 +116,9 @@ func (s *Store) Federation() storage.FederationStore { return s.fed }
 // Media implements storage.Database.
 func (s *Store) Media() storage.MediaStore { return s.media }
 
+// Keys implements storage.Database.
+func (s *Store) Keys() storage.KeyStore { return s.keys }
+
 // Migrate implements storage.Database.
 func (s *Store) Migrate(ctx context.Context) error {
 	ms, err := s.migrations()

server/internal/storage/sqlstore/sync.go

@@ -107,6 +107,14 @@ func (s *Sync) StreamPositions(ctx context.Context) (storage.StreamPositions, er
 	if err := rows.Err(); err != nil {
 		return storage.StreamPositions{}, fmt.Errorf("read the stream positions: %w", err)
 	}
+
+	// The device list stream is the keyserver's, and is read here all the
+	// same, for the same reason the event stream is: a token has to name
+	// one moment in every stream it carries.
+	out.DeviceList, err = currentSequence(ctx, s.db, s.q, keySequences, seqDeviceList)
+	if err != nil {
+		return storage.StreamPositions{}, err
+	}
 	return out, nil
 }
 

server/internal/storage/storage.go

@@ -79,6 +79,11 @@ type Database interface {
 	// remote media, and of their thumbnails.
 	Media() MediaStore
 
+	// Keys is the keyserver store: device keys, one-time and fallback
+	// keys, cross-signing keys and their signatures, the device list
+	// stream and the server-side key backup.
+	Keys() KeyStore
+
 	// Close releases the connection pool.
 	Close() error
 }

server/internal/storage/sync.go

@@ -28,6 +28,10 @@ type StreamPositions struct {
 	ToDevice    int64
 	AccountData int64
 	Presence    int64
+
+	// DeviceList is the keyserver's device list stream: the users whose
+	// end-to-end keys a client should fetch again.
+	DeviceList int64
 }
 
 // TimelineDirection is which way a timeline read walks the stream.

server/internal/syncapi/devicelists.go

@@ -0,0 +1,198 @@
+package syncapi
+
+import (
+	"context"
+	"slices"
+
+	"github.com/dariuszjaniak/warp/server/internal/matrix/event"
+	"github.com/dariuszjaniak/warp/server/internal/roomserver"
+)
+
+// Device list tracking, the reading side: whose end-to-end keys a client
+// should fetch again, and whose it can forget.
+//
+// A client encrypting for a room needs the keys of every device of every
+// member, and caches them. The cache goes stale in two ways, and the two
+// lists of a sync's device_lists are one each:
+//
+//   - changed: a user who shares a room with the client has a new device,
+//     lost one, or changed their cross-signing keys — or has only just come
+//     to share a room with the client, whose cache of their keys, if it has
+//     one at all, is from before.
+//   - left: a user who no longer shares any room with the client, whose
+//     keys it can stop tracking.
+//
+// The first kind comes from the keyserver's device list stream, which
+// moves for local users when they upload or delete keys and for remote
+// users when their server says so. The second kind comes from membership:
+// whose membership moved in a room this user is or was in, and whether
+// the two still share a room once it did. Both are read only on an
+// incremental sync; an initial one is the client's cue to fetch every key
+// it needs anyway.
+
+// deviceLists works out the device_lists of a sync between since and to.
+func (s *API) deviceLists(ctx context.Context, userID string, since, to Token, rooms []roomserver.MembershipAt) (DeviceLists, error) {
+	out := DeviceLists{Changed: []string{}, Left: []string{}}
+	if s.keys == nil || since.IsInitial() {
+		return out, nil
+	}
+	keysMoved := to.DeviceList > since.DeviceList
+	roomsMoved := to.PDU > since.PDU
+	if !keysMoved && !roomsMoved {
+		return out, nil
+	}
+
+	sharing, err := s.rooms.UsersSharingRooms(ctx, userID)
+	if err != nil {
+		return DeviceLists{}, err
+	}
+	shares := func(u string) bool { return slices.Contains(sharing, u) }
+
+	changed := map[string]bool{}
+	left := map[string]bool{}
+
+	if keysMoved {
+		// The user's own devices are reported too: another of their
+		// devices has to hear about the new one to start encrypting for
+		// it.
+		watched := append(slices.Clone(sharing), userID)
+		moves, err := s.keys.DeviceListChanges(ctx, watched, since.DeviceList, to.DeviceList)
+		if err != nil {
+			return DeviceLists{}, err
+		}
+		for _, m := range moves {
+			changed[m.UserID] = true
+		}
+	}
+
+	if roomsMoved {
+		if err := s.membershipDeviceLists(ctx, userID, since, to, rooms, shares, changed, left); err != nil {
+			return DeviceLists{}, err
+		}
+	}
+
+	for u := range changed {
+		out.Changed = append(out.Changed, u)
+		delete(left, u)
+	}
+	for u := range left {
+		out.Left = append(out.Left, u)
+	}
+	slices.Sort(out.Changed)
+	slices.Sort(out.Left)
+	return out, nil
+}
+
+// membershipDeviceLists adds the users whose sharing of a room with userID
+// began or ended inside the window.
+func (s *API) membershipDeviceLists(
+	ctx context.Context,
+	userID string,
+	since, to Token,
+	rooms []roomserver.MembershipAt,
+	shares func(string) bool,
+	changed, left map[string]bool,
+) error {
+	consider := func(other string) {
+		switch {
+		case other == userID:
+		case shares(other):
+			changed[other] = true
+		default:
+			left[other] = true
+		}
+	}
+
+	// A room whose membership of this user moved in the window: joined,
+	// everybody in it is new to share with; left, everybody in it may no
+	// longer be.
+	for _, room := range rooms {
+		if room.Position <= since.PDU || room.Position > to.PDU {
+			continue
+		}
+		if room.Membership != event.MembershipJoin && !isDeparted(room.Membership) {
+			continue
+		}
+		members, err := s.rooms.JoinedUserIDs(ctx, room.RoomID)
+		if err != nil {
+			return err
+		}
+		for _, member := range members {
+			consider(member)
+		}
+	}
+
+	// Everybody else whose membership moved in a room this user is or was
+	// in. An invite or a knock is not a change of who shares the room, and
+	// is skipped: telling a client to forget the keys of a user it was
+	// encrypting for as an invitee would have it stop.
+	changes, err := s.rooms.MemberChanges(ctx, roomIDsOf(rooms), since.PDU, to.PDU)
+	if err != nil {
+		return err
+	}
+	for _, c := range changes {
+		if c.Membership == event.MembershipJoin || isDeparted(c.Membership) {
+			consider(c.UserID)
+		}
+	}
+	return nil
+}
+
+// KeyChanges answers GET /keys/changes: whose device lists changed between
+// two tokens, for a client that missed some syncs, or is catching up with
+// the history of a room it was not syncing.
+func (s *API) KeyChanges(ctx context.Context, userID, from, to string) (DeviceLists, error) {
+	since, err := ParseToken(from)
+	if err != nil {
+		return DeviceLists{}, errInvalidTokenf("from: %v", err)
+	}
+	var until Token
+	if to == "" {
+		positions, err := s.store.StreamPositions(ctx)
+		if err != nil {
+			return DeviceLists{}, err
+		}
+		until = tokenAt(positions)
+	} else if until, err = ParseToken(to); err != nil {
+		return DeviceLists{}, errInvalidTokenf("to: %v", err)
+	}
+
+	if since.IsInitial() {
+		// From the beginning of time is every device list there is, which
+		// is the client fetching everybody's keys: what it would do after
+		// an initial sync anyway, and what since=0 means everywhere else.
+		// Reported as every user it shares a room with.
+		sharing, err := s.rooms.UsersSharingRooms(ctx, userID)
+		if err != nil {
+			return DeviceLists{}, err
+		}
+		if sharing == nil {
+			sharing = []string{}
+		}
+		return DeviceLists{Changed: sharing, Left: []string{}}, nil
+	}
+
+	rooms, err := s.rooms.UserMemberships(ctx, userID, nil)
+	if err != nil {
+		return DeviceLists{}, err
+	}
+	return s.deviceLists(ctx, userID, since, until, rooms)
+}
+
+// deviceKeyCounts reads the key counts of the syncing device.
+func (s *API) deviceKeyCounts(ctx context.Context, req SyncRequest, resp *Response) error {
+	if s.keys == nil || req.Device == "" {
+		return nil
+	}
+	counts, err := s.keys.OneTimeKeyCounts(ctx, req.User, req.Device)
+	if err != nil {
+		return err
+	}
+	unused, err := s.keys.UnusedFallbackKeyTypes(ctx, req.User, req.Device)
+	if err != nil {
+		return err
+	}
+	resp.DeviceOneTimeKeysCount = counts
+	resp.DeviceUnusedFallbackKeyTypes = &unused
+	return nil
+}

server/internal/syncapi/sync.go

@@ -61,11 +61,22 @@ type Response struct {
 	ToDevice    ToDeviceList    `json:"to_device"`
 
 	// DeviceLists reports whose device list changed, which a client needs
-	// in order to re-fetch their keys. The keyserver is PLAN.md phase 7,
-	// so both lists are always empty; the block is here all the same,
-	// because a client that finds no device_lists treats the response as
-	// malformed rather than as "nothing changed".
+	// in order to re-fetch their keys, and whose it can stop tracking.
+	// Both lists are present even when empty: a client that finds no
+	// device_lists treats the response as malformed rather than as
+	// "nothing changed".
 	DeviceLists DeviceLists `json:"device_lists"`
+
+	// DeviceOneTimeKeysCount is how many one-time keys the syncing device
+	// has left on the server, by algorithm, which is how it knows to
+	// upload more.
+	DeviceOneTimeKeysCount map[string]int `json:"device_one_time_keys_count,omitempty"`
+
+	// DeviceUnusedFallbackKeyTypes lists the algorithms the device has an
+	// unused fallback key for. It is a pointer so that an empty list goes
+	// out as [] rather than not at all: a client reads the field's
+	// absence as a server without fallback keys, and never uploads one.
+	DeviceUnusedFallbackKeyTypes *[]string `json:"device_unused_fallback_key_types,omitempty"`
 }
 
 // Rooms is the room half of a sync, split by the user's membership.
@@ -302,7 +313,6 @@ func (s *API) build(ctx context.Context, req SyncRequest, filter Filter, since T
 		Presence:    PresenceList{Events: []presenceEvent{}},
 		AccountData: AccountDataList{Events: []AccountDataEvent{}},
 		ToDevice:    ToDeviceList{Events: []toDeviceEvent{}},
-		// Both lists are empty until Phase 7 and both have to be present.
 		DeviceLists: DeviceLists{Changed: []string{}, Left: []string{}},
 	}
 
@@ -346,6 +356,13 @@ func (s *API) build(ctx context.Context, req SyncRequest, filter Filter, since T
 	}
 	resp.Presence.Events = presence
 
+	if resp.DeviceLists, err = s.deviceLists(ctx, req.User, since, to, rooms); err != nil {
+		return Response{}, false, err
+	}
+	if err := s.deviceKeyCounts(ctx, req, &resp); err != nil {
+		return Response{}, false, err
+	}
+
 	resp.NextBatch = next.String()
 	return resp, isInteresting(resp, since), nil
 }
@@ -369,7 +386,9 @@ func isInteresting(resp Response, since Token) bool {
 		len(resp.Rooms.Knock) > 0 ||
 		len(resp.AccountData.Events) > 0 ||
 		len(resp.ToDevice.Events) > 0 ||
-		len(resp.Presence.Events) > 0
+		len(resp.Presence.Events) > 0 ||
+		len(resp.DeviceLists.Changed) > 0 ||
+		len(resp.DeviceLists.Left) > 0
 }
 
 // buildRooms fills the four room sections.

server/internal/syncapi/syncapi.go

@@ -16,18 +16,20 @@
 // What this package owns instead is everything the roomserver has no
 // opinion about:
 //
-//   - Stream tokens: a position in each of the five streams a response
+//   - Stream tokens: a position in each of the six streams a response
 //     carries, in one opaque string (token.go).
 //   - The notifier: waking a client blocked in a long poll the moment
 //     something concerns it, rather than when its timeout expires
 //     (notifier.go).
 //   - Filters: what a client asked to be left out (filter.go).
-//   - The ephemeral and device streams: typing, receipts, presence and
-//     to-device messages, none of which are events in a room.
+//   - The ephemeral and device streams: typing, receipts, presence,
+//     to-device messages and device list changes, none of which are
+//     events in a room.
 //
-// Phase 4 of PLAN.md. What it does not do yet is notification counts, which
-// need push rules (Phase 8), and device list tracking, which needs the
-// keyserver (Phase 7); both are named in docs/sync.md as the gaps they are.
+// Phase 4 of PLAN.md, with device lists from Phase 7: the keyserver writes
+// the device list stream, and this package works out which of its changes
+// concern whom (devicelists.go). What it does not do yet is notification
+// counts, which need push rules (Phase 8); docs/sync.md names that gap.
 package syncapi
 
 import (
@@ -72,11 +74,20 @@ type Accounts interface {
 	Devices(ctx context.Context, localpart string) ([]storage.Device, error)
 }
 
+// Keys is the part of the keyserver a sync reads: a device's key counts,
+// and the device list stream. *keyserver.API satisfies it.
+type Keys interface {
+	OneTimeKeyCounts(ctx context.Context, userID, deviceID string) (map[string]int, error)
+	UnusedFallbackKeyTypes(ctx context.Context, userID, deviceID string) ([]string, error)
+	DeviceListChanges(ctx context.Context, userIDs []string, from, to int64) ([]storage.DeviceListChange, error)
+}
+
 // API is the syncapi component.
 type API struct {
 	serverName string
 	rooms      *roomserver.API
 	accounts   Accounts
+	keys       Keys
 	store      storage.SyncStore
 	notifier   *Notifier
 	typing     *typingTracker
@@ -111,6 +122,10 @@ type Options struct {
 	// Accounts is the userapi.
 	Accounts Accounts
 
+	// Keys is the keyserver. Nil leaves the key counts out of a sync and
+	// both device lists empty.
+	Keys Keys
+
 	// Store is the sync half of storage.
 	Store storage.SyncStore
 
@@ -190,6 +205,7 @@ func New(opts Options) (*API, error) {
 		serverName:      opts.ServerName,
 		rooms:           opts.Rooms,
 		accounts:        opts.Accounts,
+		keys:            opts.Keys,
 		store:           opts.Store,
 		notifier:        opts.Notifier,
 		typing:          newTypingTracker(opts.TypingTimeout, opts.Notifier, opts.Now),

server/internal/syncapi/token.go

@@ -18,10 +18,12 @@ import (
 //
 // The wire form is
 //
-//	s{pdu}_{receipt}_{to_device}_{account_data}_{presence}
+//	s{pdu}_{receipt}_{to_device}_{account_data}_{presence}_{device_list}
 //
 // with trailing zeroes left off, so a client that has only ever seen room
-// events carries "s12" rather than "s12_0_0_0_0". The leading "s" is there
+// events carries "s12" rather than "s12_0_0_0_0_0". The device list
+// position came with Phase 7 and was added at the end, which is what lets
+// every token handed out before it go on parsing. The leading "s" is there
 // to make a token recognisable in a log and to leave room for a second
 // token shape later — Simplified Sliding Sync (PLAN.md phase 10) has its
 // own — without any ambiguity about which is which.
@@ -35,8 +37,12 @@ type Token struct {
 	ToDevice    int64
 	AccountData int64
 	Presence    int64
+	DeviceList  int64
 }
 
+// tokenPositions is how many positions a token carries.
+const tokenPositions = 6
+
 // tokenPrefix marks a classic /sync token.
 const tokenPrefix = "s"
 
@@ -46,7 +52,7 @@ func (t Token) IsInitial() bool { return t == Token{} }
 
 // String renders the wire form.
 func (t Token) String() string {
-	parts := []int64{t.PDU, t.Receipt, t.ToDevice, t.AccountData, t.Presence}
+	parts := []int64{t.PDU, t.Receipt, t.ToDevice, t.AccountData, t.Presence, t.DeviceList}
 	// Trailing zeroes carry no information, and leaving them off keeps the
 	// common token short.
 	last := 0
@@ -88,12 +94,12 @@ func ParseToken(raw string) (Token, error) {
 	}
 
 	fields := strings.Split(body, "_")
-	if len(fields) > 5 {
-		return Token{}, fmt.Errorf("%q is not a sync token: it has %d positions, this server writes at most 5",
-			raw, len(fields))
+	if len(fields) > tokenPositions {
+		return Token{}, fmt.Errorf("%q is not a sync token: it has %d positions, this server writes at most %d",
+			raw, len(fields), tokenPositions)
 	}
 
-	var positions [5]int64
+	var positions [tokenPositions]int64
 	for i, field := range fields {
 		value, err := strconv.ParseInt(field, 10, 64)
 		// The PDU position alone may be negative: /messages hands out
@@ -110,6 +116,7 @@ func ParseToken(raw string) (Token, error) {
 		ToDevice:    positions[2],
 		AccountData: positions[3],
 		Presence:    positions[4],
+		DeviceList:  positions[5],
 	}, nil
 }
 
@@ -122,6 +129,7 @@ func tokenAt(positions storage.StreamPositions) Token {
 		ToDevice:    positions.ToDevice,
 		AccountData: positions.AccountData,
 		Presence:    positions.Presence,
+		DeviceList:  positions.DeviceList,
 	}
 }
 

server/internal/syncapi/token_test.go

@@ -17,8 +17,8 @@ func TestTokenRoundTrips(t *testing.T) {
 		{"events and receipts", syncapi.Token{PDU: 12, Receipt: 3}, "s12_3"},
 		{
 			"every stream",
-			syncapi.Token{PDU: 1, Receipt: 2, ToDevice: 3, AccountData: 4, Presence: 5},
-			"s1_2_3_4_5",
+			syncapi.Token{PDU: 1, Receipt: 2, ToDevice: 3, AccountData: 4, Presence: 5, DeviceList: 6},
+			"s1_2_3_4_5_6",
 		},
 		{
 			// /messages pages back into history backfilled from other
@@ -81,13 +81,13 @@ func TestParseTokenAcceptsAShortToken(t *testing.T) {
 
 func TestParseTokenRejectsNonsense(t *testing.T) {
 	for _, raw := range []string{
-		"nonsense",     // no prefix
-		"t1_2",         // a prefix this build does not write
-		"s",            // a prefix and nothing else
-		"s1_-1",        // a negative position outside the event stream
-		"sx",           // not a number
-		"s1_",          // a trailing separator
-		"s1_2_3_4_5_6", // more positions than there are streams
+		"nonsense",       // no prefix
+		"t1_2",           // a prefix this build does not write
+		"s",              // a prefix and nothing else
+		"s1_-1",          // a negative position outside the event stream
+		"sx",             // not a number
+		"s1_",            // a trailing separator
+		"s1_2_3_4_5_6_7", // more positions than there are streams
 		"s1.5",
 	} {
 		if _, err := syncapi.ParseToken(raw); err == nil {

server/internal/userapi/userapi.go

@@ -7,8 +7,9 @@
 // which is also what makes `warp create-account` able to create an account
 // without starting a server.
 //
-// Nothing in this package knows about rooms. Devices get their encryption
-// keys in Phase 7 (keyserver) and their push rules in Phase 8.
+// Nothing in this package knows about rooms. A device's encryption keys
+// are the keyserver's, which hears about a device going away through the
+// DeviceObserver; its push rules arrive in Phase 8.
 package userapi
 
 import (
@@ -67,8 +68,30 @@ type API struct {
 
 	// lastSeen throttles the last-seen bookkeeping; see RecordActivity.
 	lastSeen sync.Map
+
+	// observer hears about deleted and renamed devices; nil hears nothing.
+	observer DeviceObserver
 }
 
+// DeviceObserver is told about changes to devices that something else
+// keeps state for: the keyserver, whose keys for a deleted device have to
+// go with it. *keyserver.API satisfies it.
+//
+// It is called after the change has been made, on the request path, and
+// must not fail it: the device is already gone.
+type DeviceObserver interface {
+	// DevicesDeleted is called with the deleted device IDs, or with none
+	// when every device of the user was deleted.
+	DevicesDeleted(ctx context.Context, localpart string, deviceIDs []string)
+
+	// DeviceRenamed is called when a device's display name changes.
+	DeviceRenamed(ctx context.Context, localpart, deviceID string)
+}
+
+// SetDeviceObserver connects the observer. It is a setter because the
+// keyserver is built after the userapi it reads devices from.
+func (a *API) SetDeviceObserver(o DeviceObserver) { a.observer = o }
+
 // Options configure the component.
 type Options struct {
 	// ServerName is the Matrix server name; it becomes the domain part of
@@ -438,16 +461,28 @@ func (a *API) Devices(ctx context.Context, localpart string) ([]storage.Device,
 
 // SetDeviceDisplayName renames a device.
 func (a *API) SetDeviceDisplayName(ctx context.Context, localpart, deviceID, displayName string) error {
-	return a.store.SetDeviceDisplayName(ctx, localpart, deviceID, displayName)
+	if err := a.store.SetDeviceDisplayName(ctx, localpart, deviceID, displayName); err != nil {
+		return err
+	}
+	if a.observer != nil {
+		a.observer.DeviceRenamed(ctx, localpart, deviceID)
+	}
+	return nil
 }
 
 // DeleteDevices revokes devices and their tokens. It is what /logout,
 // /delete_devices and DELETE /devices/{id} all end up calling.
 func (a *API) DeleteDevices(ctx context.Context, localpart string, deviceIDs []string) error {
+	if len(deviceIDs) == 0 {
+		return nil
+	}
 	if _, err := a.store.DeleteDevices(ctx, localpart, deviceIDs); err != nil {
 		return err
 	}
 	a.forgetLastSeen(localpart, deviceIDs...)
+	if a.observer != nil {
+		a.observer.DevicesDeleted(ctx, localpart, deviceIDs)
+	}
 	return nil
 }
 
@@ -457,6 +492,9 @@ func (a *API) DeleteAllDevices(ctx context.Context, localpart string) error {
 		return err
 	}
 	a.forgetLastSeen(localpart)
+	if a.observer != nil {
+		a.observer.DevicesDeleted(ctx, localpart, nil)
+	}
 	return nil
 }
 

server/test/complement/README.md

@@ -45,7 +45,7 @@ as `WARP_FEDERATION_ENABLED`.
 
 The federation set leaves out partial-state joins (MSC3706,
 `TestPartialStateJoin*`), which Warp does not implement, and anything that
-needs application services (Phase 9) or device lists (Phase 7).
+needs application services (Phase 9).
 
 ## The test set
 

server/test/complement/run.sh

@@ -27,18 +27,19 @@ REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
 # stream tests: /sync in all its forms, /messages, /context, filters,
 # typing, receipts and to-device messaging. What is still out is anything
 # needing a later phase — the notification counts of the push rules
-# (Phase 8) and the device lists of the keyserver (Phase 7). Phase 6 added
-# the media tests: uploads, async uploads, downloads with their file names
-# and thumbnails. The remote-media tests are in the federation set, which
-# TestRemote.* already covers.
-CSAPI_TESTS='TestMedia.*|TestContent.*|TestLocalPngThumbnail|TestAsyncUpload.*|TestUploadLimit|TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData'
+# (Phase 8). Phase 6 added the media tests: uploads, async uploads,
+# downloads with their file names and thumbnails. The remote-media tests
+# are in the federation set, which TestRemote.* already covers. Phase 7
+# added the keyserver's: key uploads, claims, changes, device lists,
+# fallback keys, cross-signing and key backup.
+CSAPI_TESTS='TestUploadKey.*|TestKeyClaimOrdering|TestKeyChanges.*|TestDeviceListUpdates.*|TestE2EKeyBackup.*|TestFallbackKeys.*|TestCrossSigning.*|TestMedia.*|TestContent.*|TestLocalPngThumbnail|TestAsyncUpload.*|TestUploadLimit|TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData'
 
 # federation: Phase 5. Joins, leaves and invites across servers, keys,
 # transactions, missing events, server ACLs and the queries. Partial-state
 # joins (MSC3706, TestPartialStateJoin*) are not implemented and are left
-# out; so is anything needing an application service (Phase 9) or device
-# lists (Phase 7).
-FEDERATION_TESTS='TestFederation.*|TestJoinViaRoomIDAndServerName|TestJoinFederatedRoom.*|TestBannedUserCannotSendJoin|TestCannotSendNon.*Via.*|TestCannotSendKnockViaSend.*|TestOutboundFederation.*|TestInboundFederation.*|TestInboundCanReturnMissingEvents|TestRemoteAliasRequestsUnderstandUnicode|TestNetworkPartitionOrdering|TestACLs|TestGetMissingEventsGapFilling|TestEventAuth|TestUnbanViaInvite|TestJoinRoomWithFederation.*|TestRemote.*'
+# out; so is anything needing an application service (Phase 9). Phase 7
+# added device lists across servers.
+FEDERATION_TESTS='TestDeviceListUpdates.*|TestDeviceListsUpdateOverFederation.*|TestFederation.*|TestJoinViaRoomIDAndServerName|TestJoinFederatedRoom.*|TestBannedUserCannotSendJoin|TestCannotSendNon.*Via.*|TestCannotSendKnockViaSend.*|TestOutboundFederation.*|TestInboundFederation.*|TestInboundCanReturnMissingEvents|TestRemoteAliasRequestsUnderstandUnicode|TestNetworkPartitionOrdering|TestACLs|TestGetMissingEventsGapFilling|TestEventAuth|TestUnbanViaInvite|TestJoinRoomWithFederation.*|TestRemote.*'
 
 case "$COMPLEMENT_PROFILE" in
 	csapi)