Warp/README.md

Warp

A Matrix homeserver written from scratch in Go, plus a family of first-party clients. Federation is optional and can be switched on and off at runtime.

The full design and roadmap live in PLAN.md. This README covers what exists today.

Status

Phase 10 — hardening and ops. Complete. Simplified Sliding Sync (MSC4186) is served and advertised, so Element X and the matrix-rust-sdk skip /sync: room lists sorted by activity with filters for DMs, invites, encryption, room types, spaces and tags; room subscriptions; required state picked tuple by tuple, $LAZY members included; and the to-device, E2EE, account data, receipt and typing extensions, over per-connection state that answers M_UNKNOWN_POS when it has been forgotten. Threads work: /relations (with recursion) and /threads, and every event served carries its thread summary, latest edit and references in unsigned.m.relations. Spaces have /hierarchy, walked across servers through the federation hierarchy endpoint, and room_summary. /search runs on SQLite's FTS5 or PostgreSQL's tsvector, bounded to what each user may see, with context, profiles, state and grouping. The PocketBase driver is real: the SQLite schema inside an embedded PocketBase app, with media in its filesystem — local or S3 — and it passes the same storage tests as SQLite. OpenTelemetry tracing follows a request from the client through federation, push and application service calls; /metrics gains event, sync and database pool metrics, with a Grafana dashboard over them. There is a Compose setup, a Helm chart and a load tester (warp-loadtest), and the load tester found a long-standing bug fixed here: on a server where no account data had been written yet, every /sync long poll returned at once. See docs/deployment.md; the known gap — sliding sync connection state lost on restart — is in docs/sync.md. Unread counts are kept per thread for a client that asks (docs/push.md).

Phase 9 — application services and the admin API. Complete. Bridges and bots can be registered from their registration files. A service acts as its own sender or, with ?user_id=, as any user in its namespace; registers and logs in its users with m.login.application_service; and has its exclusive namespaces kept from everybody else with M_EXCLUSIVE. Events reach it as transactions read from the event stream at a cursor kept in the database. A service that was down is caught up from where it stopped, and a transaction cut off by a restart is resent under the same ID. Unknown aliases and users in a service's namespace are asked of the service first, so portals and puppets appear on demand, and the ping endpoint reports exactly why a service cannot be reached. Users can change their password and deactivate their account. The admin API is complete: deactivating and reactivating users, password resets, devices, the admin flag, media quotas, registration tokens with use limits and expiry, server notices, an editor for the federation allow and deny lists, a delivery view of every application service, and an audit log of every change made through it. Known gaps — no ephemeral events for services, no third-party lookups — are listed in docs/appservices.md.

Phase 8 — push and notifications. Complete. Every event is checked against the push rules of every local member of its room. Warp has the spec's full default ruleset, all six condition kinds, intentional mentions switching off the older body-matching rules, and user rules ordered the way Synapse orders them. Push rules live in m.push_rules account data, so clients find them where they already look, while a server upgrade still brings in new default rules. /sync reports unread_notifications for each joined room. The counts are the seventh stream of a sync token, so a receipt sent from another device clears them everywhere at once. Pushers register an app's push gateway; pushes go out from a bounded pool with retries, the event_id_only format is honoured, and push keys the gateway rejects are removed. Logging a device out removes its pushers. Known gaps — no thread counts, no email pushers — are listed in docs/push.md.

Phase 7 — end-to-end encryption. Complete. The server half of E2EE: devices upload their identity, one-time and fallback keys, other devices query and claim them — across servers too — and every one-time key is handed out exactly once. Cross-signing keys are checked to be signed by the master key, replacing a master key asks for the password, and signatures between devices and users are verified against the keys as stored; who a user has verified stays between them and their own devices. Device list tracking is real: /sync reports whose keys changed and whom a client no longer shares a room with, fed by a device list stream that moves in the same transaction as the keys and travels between servers as m.device_list_update and m.signing_key_update. Deleting a device takes its keys with it. Server-side key backup keeps the better copy of each room key by the spec's rule. Known gaps — no cache of other servers' keys, no dehydrated devices — are listed in docs/e2ee.md.

Phase 6 — media. Complete. Clients can upload files, reserve an mxc:// URI and upload to it later, download and ask for thumbnails, all through the authenticated media endpoints of Matrix 1.11. Content is stored once per distinct file, named by its SHA-256, on the local filesystem; thumbnails are made on first request from a fixed set of sizes, the right way up for a phone's photos, with the pixel count checked before anything is decoded. Media from other servers is fetched over federation on first use — with the fallback older servers need, and redirects that cannot be pointed into a private network — and cached, so it is still there with federation switched off. Every download carries the headers that keep an uploaded file from running as a page on the homeserver's origin. Known gaps — no URL previews, no deletion — are listed in docs/media.md.

Phase 5 — federation. Complete. Warp talks to other homeservers. It joins rooms on other servers and lets their users join its rooms, invites across servers with the invitee's server countersigning, rejects remote invites (locally, if the inviter is gone), and exchanges events, typing, read receipts and to-device messages with them. Events from elsewhere go through the spec's full intake — signatures and hashes, the rules against their own auth events, against the state before them and against the current state, with missing history fetched and soft failure for what fails only at the end — and forked rooms are resolved back together. The outbound queue persists what it owes each server, retries with backoff, and survives restarts. Server discovery, key fetching with an optional notary, X-Matrix request signing, server ACLs, and a TLS listener for port 8448. The switch is real now: turning federation off stops the queue and refuses every outbound request, turning it back on resumes from where it stopped, and the admin API shows each server's queue and backoff. Known gaps — no history backfill from other servers, no partial-state joins, no key rotation — are listed in docs/federation.md.

Phase 4 — sync. Complete. Milestone M1. Warp is now a homeserver a client can be used against. /sync is real: stream tokens across five independent streams, long polling that wakes the moment something concerns the client rather than when its timeout expires, incremental deltas, gappy timelines with a prev_batch to close them, filters, and lazy-loaded members. /messages pages a room's history and /context opens a permalink. The streams that are not room events are here too: typing, read receipts and read markers, to-device messaging for Phase 7's encryption to build on, and presence behind a config switch that is off by default. A user who has left a room is now served the room as it was when they left rather than refused, and ?at= on /members is honoured. See docs/sync.md.

Phase 3 — rooms, local only. Complete. A client can create a room at any supported version, invite, join, leave, kick, ban, send messages and state, redact, name a room with an alias and find one in the public directory. Every event goes through one pipeline — validate, authorise, resolve, persist, emit — so a membership change made by createRoom, by /invite or by a raw PUT .../state/m.room.member is judged by the same authorisation rules. Nothing talks to another server yet, and every operation that would need to is refused with a message that says so. See docs/rooms.md.

Phase 2 — the event core. Complete. The library every later phase is built on: canonical JSON, the event model with its hashes and identifiers, the redaction algorithm, event signing, the room version registry, the authorisation rules and state resolution v2. Pure library code under server/internal/matrix, verified against the spec's own test vectors. See docs/event-core.md.

Phase 1 — accounts and sessions. Complete. A client can register, log in, manage its devices and set its profile.

Client-Server API:

POST /register, GET /register/available	sign-up, with m.login.dummy and optional registration tokens; application services register their users
GET, POST /login · POST /logout, /logout/all · POST /refresh	sessions, with m.login.password, m.login.application_service and refresh tokens
GET /account/whoami · GET /capabilities	who am I, and what this build can do
POST /account/password · POST /account/deactivate	changing your password and closing your account, both behind the password
POST /_matrix/client/v1/appservice/{id}/ping	an application service checking this server can reach it; see docs/appservices.md
GET /devices, GET/PUT/DELETE /devices/{id} · POST /delete_devices	device management; deletion re-asks for the password
GET /profile/{user} and its displayname / avatar_url sub-resources	profiles — reads are public, remote ones asked of their server, writes are your own
GET/PUT /user/{user}/account_data/{type} and the per-room variant	account data, stored and returned byte for byte
POST /createRoom	room versions 3 to 12, presets, initial_state, invites, aliases
POST /join/{roomIdOrAlias} · POST /rooms/{id}/join, /leave	joining by ID or alias — on another server too, with federation on — and leaving or rejecting an invite
POST /rooms/{id}/invite, /kick, /ban, /unban	membership, decided by the room's power levels; invites reach users of other servers
PUT /rooms/{id}/send/{type}/{txn}	messages, idempotent per transaction ID
GET/PUT /rooms/{id}/state[/{type}[/{key}]]	room state, read whole or one piece at a time
PUT /rooms/{id}/redact/{event}/{txn}	redactions, applied only when they take effect
GET /rooms/{id}/event/{event} · /members (with ?at=) · /joined_members · /aliases	reading a room
GET /joined_rooms	the rooms you are in
GET/PUT/DELETE /directory/room/{alias}	the alias directory; remote aliases are asked of their server
GET/POST /publicRooms · GET/PUT /directory/list/room/{id}	the public room directory, ours or (?server=) another server's
GET /sync	the event stream: long-polled, incremental, filtered, with lazy-loaded members
GET /rooms/{id}/messages · /context/{event}	paging a room's history, and one event with what surrounds it
POST /user/{user}/filter · GET /user/{user}/filter/{id}	filters, stored and returned byte for byte
PUT /rooms/{id}/typing/{user}	typing notifications, kept in memory and expired
POST /rooms/{id}/receipt/{type}/{event} · /read_markers	read receipts, public and private, and the read marker
PUT /sendToDevice/{type}/{txn}	device-to-device messaging, which carries E2EE's session keys
POST /keys/upload, /keys/query, /keys/claim · GET /keys/changes	device, one-time and fallback keys, local and from other servers
POST /keys/device_signing/upload, /keys/signatures/upload	cross-signing, with the password re-asked to replace a master key
/room_keys/version[/{v}] · /room_keys/keys[/{room}[/{session}]]	server-side key backup; see docs/e2ee.md
GET /pushrules/ · GET/PUT/DELETE /pushrules/global/{kind}/{rule} and its enabled / actions	push rules, over the spec's defaults; see docs/push.md
GET /pushers · POST /pushers/set	where a device wants its notifications pushed
GET /notifications	what notified you, newest first
GET/PUT /presence/{user}/status	presence — only while sync.presence is on, and it is off by default
POST /_matrix/media/v3/upload · POST /_matrix/media/v1/create · PUT /_matrix/media/v3/upload/{server}/{id}	uploads, directly or to a reserved URI
GET /_matrix/client/v1/media/download/… · /thumbnail/… · /config	downloads and thumbnails, local and from other servers; see docs/media.md
GET /_matrix/client/versions · /.well-known/matrix/client	discovery
/.well-known/matrix/server	server discovery — only while federation is on
/_matrix/federation/* · /_matrix/key/*	the Server-Server API — only while federation is on; see docs/federation.md
GET /healthz · GET /metrics	liveness with a database ping, and Prometheus
anything else under /_matrix/	404 M_UNRECOGNIZED, as the spec requires

Admin API under /_warp/admin/v1: the federation and registration switches and the allow/deny list editor, per-server federation health, users (listing, creation, the admin flag, deactivation, password resets, devices, media quotas), registration tokens, server notices, the application services and their delivery, the room list and detail, purging a room, removing a user from one, the audit log and the dashboard counters. See docs/admin-api.md.

The admin panel is a web UI for all of it. It is optional and runs as its own process, warp-admin, next to the homeserver: docker compose --profile admin up -d, or see apps/admin.

Also in place: argon2id passwords, hashed access and refresh tokens, per-endpoint rate limiting, YAML + environment configuration, structured logging, graceful shutdown, the storage interfaces with SQLite, PostgreSQL and PocketBase drivers, the embedded migration runner and the runtime settings service.

Complement, the official integration suite, is wired up for both profiles. make complement runs the account, room and stream tests with federation off — TestRegistration, TestLogin, TestDeviceManagement, the Phase 3 room tests and the Phase 4 TestSync*, TestMessages*, TestContext*, TestFilter*, TestTyping*, TestReceipt* and TestToDevice*, the Phase 6 media tests, the Phase 7 key tests and the Phase 8 push rule tests. make complement-federation runs the Phase 5 Server-Server tests and the device list tests with federation on, over TLS with Complement's CA.

Quick start

Requires Go 1.26 or newer. No cgo, no C compiler, no database server: the default SQLite driver is pure Go.

cp warp.yaml.example warp.yaml
# edit global.server_name

make build
./bin/warp generate-keys
./bin/warp create-account --username you --admin
./bin/warp serve

Then:

curl localhost:8008/healthz
curl localhost:8008/_matrix/client/versions

Registration is off in a fresh install, which is why the first account is made from the command line. Turn it on from the admin API when you want others to sign up.

Without make, the same from the server directory:

go build -o ../bin/warp ./cmd/warp

Configuration

One YAML file, every field overridable from the environment. See warp.yaml.example — each setting lists its WARP_* variable. With no config file at all the server runs on defaults plus the environment, which is how the container image is meant to be used:

docker build -t warp .
docker run -p 8008:8008 -v warp-data:/data -e WARP_SERVER_NAME=warp.example warp

An unknown field in the config file is an error, not a warning: a typo in a security-relevant setting should never be silently ignored.

Commands

Command	What it does
warp serve	run the homeserver
warp migrate [--status]	apply (or list) pending database migrations
warp generate-keys [--output path] [--key-id id]	create the ed25519 signing key
warp create-account --username name [--admin]	create a local account, straight in the database
warp version	version and compiled-in storage drivers
warp-loadtest [-users n] [-rooms n] [-rate r] [-duration d] [-sliding]	a separate binary that loads a test server with users, rooms, messages and syncs, and reports latencies; see docs/deployment.md

create-account talks to the database rather than to a running server, so the first account on a new install can be made before anything is listening and without turning registration on. It prompts for the password twice; --password-file and --password are there for scripts.

The signing key is written as a single line, the same format Synapse uses: ed25519 <key-id> <unpadded-base64-seed>. generate-keys refuses to overwrite an existing key — losing it would invalidate every event this server has ever signed.

Federation switch

Federation is off in a fresh install. While it is off, the federation surface does not answer at all: /_matrix/federation/*, /_matrix/key/* and /.well-known/matrix/server return 404 M_UNRECOGNIZED, exactly as they would on a server that has no federation API, and nothing outbound runs.

The switch is stored in the server_settings table and cached in memory, so flipping it — PUT /_warp/admin/v1/federation {"enabled": true} — takes effect on the next request without a restart. Switching it on starts the outbound queue, which picks up whatever it still owed other servers; switching it off stops the queue mid-flight and makes the federation client refuse every request before it opens a connection. Set federation.allow_runtime_toggle: false to pin the switch to the config file.

To federate, other servers have to be able to reach this one: either set global.well_known.server_address to a reverse proxy that terminates TLS, or let Warp terminate it with listen.federation_address: ":8448" and a certificate. See docs/configuration.md and docs/federation.md.

Development

make test          # unit tests (SQLite)
make lint          # golangci-lint
make check         # vet + test, what CI runs
make cover         # coverage.html
make complement    # the Complement C-S subset, federation off (needs Docker)
make complement-federation  # the Complement federation subset

# the storage tests against PostgreSQL as well
WARP_TEST_POSTGRES_DSN='postgres://warp:warp@localhost/warp_test?sslmode=disable' make test-postgres

make test-race needs cgo and therefore a C toolchain; CI runs it on Linux.

Layout

server/                 the homeserver (Go module)
  cmd/warp/             the binary: serve, migrate, generate-keys, create-account, version
  cmd/warp-loadtest/    the load tester
  internal/
    config/             YAML + env configuration
    logging/  metrics/  slog and Prometheus setup
    tracing/            OpenTelemetry: server and client spans, trace propagation
    httputil/           Matrix error envelope, CORS, router, middleware
    ratelimit/          per-endpoint token buckets
    federation/         the Server-Server protocol: gate, X-Matrix, discovery, client, keys
    federationapi/      federation endpoints, the outbound queue, the switchable Module (docs/federation.md)
    settings/           runtime settings service (the switches)
    userapi/            accounts, passwords, devices, tokens, account data, deactivation, registration tokens
    roomserver/         rooms: the input pipeline (docs/rooms.md), relations and threads (docs/relations.md), spaces (docs/spaces.md), search (docs/search.md)
    syncapi/            /sync, sliding sync, /messages, the ephemeral streams (docs/sync.md)
    pushrules/  pushserver/  push rules, notification counts, pushers (docs/push.md)
    mediaapi/           uploads, downloads, thumbnails, remote media (docs/media.md)
    keyserver/          device keys, one-time keys, cross-signing, key backup (docs/e2ee.md)
    appservice/         application services: registrations, namespaces, delivery (docs/appservices.md)
    servernotices/      messages from the server to its users
    clientapi/          the Client-Server API handlers
    adminapi/           /_warp/admin/v1
    setup/              wiring: storage, routes, graceful shutdown
    matrix/             the event core (docs/event-core.md)
      canonicaljson/    the canonical JSON encoding
      signing/          ed25519 server keys, Signing JSON
      id/               identifier grammars as they appear in events
      roomversion/      the room version registry
      event/            PDUs: hashes, event IDs, redaction, signing
      auth/             the authorisation rules and power levels
      stateres/         state resolution v2
    storage/            interfaces + driver registry
      migrations/       embedded migration runner
      sqlstore/         shared database/sql implementation
      filestore/        media content on the local filesystem
      sqlite/ postgres/ pocketbase/
  test/complement/      the Complement homeserver image and runner
apps/                   clients: ui (design system), admin (warp-admin panel); web, desktop, mobile to come
deploy/                 compose/ (with Prometheus, Grafana, Jaeger), helm/warp, grafana/ dashboard
docs/