Warp/README.md

Warp

A Matrix homeserver written from scratch in Go, plus a family of first-party clients. Federation is optional and can be switched on and off at runtime.

The full design and roadmap live in PLAN.md. This README covers what exists today.

Status

Phase 6 — media. Complete. Clients can upload files, reserve an mxc:// URI and upload to it later, download and ask for thumbnails, all through the authenticated media endpoints of Matrix 1.11. Content is stored once per distinct file, named by its SHA-256, on the local filesystem; thumbnails are made on first request from a fixed set of sizes, the right way up for a phone's photos, with the pixel count checked before anything is decoded. Media from other servers is fetched over federation on first use — with the fallback older servers need, and redirects that cannot be pointed into a private network — and cached, so it is still there with federation switched off. Every download carries the headers that keep an uploaded file from running as a page on the homeserver's origin. Known gaps — no URL previews, no deletion or quotas until Phase 9 — are listed in docs/media.md.

Phase 5 — federation. Complete. Warp talks to other homeservers. It joins rooms on other servers and lets their users join its rooms, invites across servers with the invitee's server countersigning, rejects remote invites (locally, if the inviter is gone), and exchanges events, typing, read receipts and to-device messages with them. Events from elsewhere go through the spec's full intake — signatures and hashes, the rules against their own auth events, against the state before them and against the current state, with missing history fetched and soft failure for what fails only at the end — and forked rooms are resolved back together. The outbound queue persists what it owes each server, retries with backoff, and survives restarts. Server discovery, key fetching with an optional notary, X-Matrix request signing, server ACLs, and a TLS listener for port 8448. The switch is real now: turning federation off stops the queue and refuses every outbound request, turning it back on resumes from where it stopped, and the admin API shows each server's queue and backoff. Known gaps — no history backfill from other servers, no partial-state joins, no key rotation — are listed in docs/federation.md.

Phase 4 — sync. Complete. Milestone M1. Warp is now a homeserver a client can be used against. /sync is real: stream tokens across five independent streams, long polling that wakes the moment something concerns the client rather than when its timeout expires, incremental deltas, gappy timelines with a prev_batch to close them, filters, and lazy-loaded members. /messages pages a room's history and /context opens a permalink. The streams that are not room events are here too: typing, read receipts and read markers, to-device messaging for Phase 7's encryption to build on, and presence behind a config switch that is off by default. A user who has left a room is now served the room as it was when they left rather than refused, and ?at= on /members is honoured. See docs/sync.md.

Phase 3 — rooms, local only. Complete. A client can create a room at any supported version, invite, join, leave, kick, ban, send messages and state, redact, name a room with an alias and find one in the public directory. Every event goes through one pipeline — validate, authorise, resolve, persist, emit — so a membership change made by createRoom, by /invite or by a raw PUT .../state/m.room.member is judged by the same authorisation rules. Nothing talks to another server yet, and every operation that would need to is refused with a message that says so. See docs/rooms.md.

Phase 2 — the event core. Complete. The library every later phase is built on: canonical JSON, the event model with its hashes and identifiers, the redaction algorithm, event signing, the room version registry, the authorisation rules and state resolution v2. Pure library code under server/internal/matrix, verified against the spec's own test vectors. See docs/event-core.md.

Phase 1 — accounts and sessions. Complete. A client can register, log in, manage its devices and set its profile.

Client-Server API:

POST /register, GET /register/available	sign-up, with m.login.dummy and optional registration tokens
GET, POST /login · POST /logout, /logout/all · POST /refresh	sessions, with m.login.password and refresh tokens
GET /account/whoami · GET /capabilities	who am I, and what this build can do
GET /devices, GET/PUT/DELETE /devices/{id} · POST /delete_devices	device management; deletion re-asks for the password
GET /profile/{user} and its displayname / avatar_url sub-resources	profiles — reads are public, remote ones asked of their server, writes are your own
GET/PUT /user/{user}/account_data/{type} and the per-room variant	account data, stored and returned byte for byte
POST /createRoom	room versions 3 to 12, presets, initial_state, invites, aliases
POST /join/{roomIdOrAlias} · POST /rooms/{id}/join, /leave	joining by ID or alias — on another server too, with federation on — and leaving or rejecting an invite
POST /rooms/{id}/invite, /kick, /ban, /unban	membership, decided by the room's power levels; invites reach users of other servers
PUT /rooms/{id}/send/{type}/{txn}	messages, idempotent per transaction ID
GET/PUT /rooms/{id}/state[/{type}[/{key}]]	room state, read whole or one piece at a time
PUT /rooms/{id}/redact/{event}/{txn}	redactions, applied only when they take effect
GET /rooms/{id}/event/{event} · /members (with ?at=) · /joined_members · /aliases	reading a room
GET /joined_rooms	the rooms you are in
GET/PUT/DELETE /directory/room/{alias}	the alias directory; remote aliases are asked of their server
GET/POST /publicRooms · GET/PUT /directory/list/room/{id}	the public room directory, ours or (?server=) another server's
GET /sync	the event stream: long-polled, incremental, filtered, with lazy-loaded members
GET /rooms/{id}/messages · /context/{event}	paging a room's history, and one event with what surrounds it
POST /user/{user}/filter · GET /user/{user}/filter/{id}	filters, stored and returned byte for byte
PUT /rooms/{id}/typing/{user}	typing notifications, kept in memory and expired
POST /rooms/{id}/receipt/{type}/{event} · /read_markers	read receipts, public and private, and the read marker
PUT /sendToDevice/{type}/{txn}	device-to-device messaging, for E2EE in Phase 7
GET/PUT /presence/{user}/status	presence — only while sync.presence is on, and it is off by default
POST /_matrix/media/v3/upload · POST /_matrix/media/v1/create · PUT /_matrix/media/v3/upload/{server}/{id}	uploads, directly or to a reserved URI
GET /_matrix/client/v1/media/download/… · /thumbnail/… · /config	downloads and thumbnails, local and from other servers; see docs/media.md
GET /_matrix/client/versions · /.well-known/matrix/client	discovery
/.well-known/matrix/server	server discovery — only while federation is on
/_matrix/federation/* · /_matrix/key/*	the Server-Server API — only while federation is on; see docs/federation.md
GET /healthz · GET /metrics	liveness with a database ping, and Prometheus
anything else under /_matrix/	404 M_UNRECOGNIZED, as the spec requires

Admin API under /_warp/admin/v1: the federation and registration switches, per-server federation health, user listing and creation, the room list and detail, purging a room, removing a user from one, and the dashboard counters. See docs/admin-api.md.

Also in place: argon2id passwords, hashed access and refresh tokens, per-endpoint rate limiting, YAML + environment configuration, structured logging, graceful shutdown, the storage interfaces with a working SQLite and PostgreSQL driver (and a PocketBase stub), the embedded migration runner and the runtime settings service.

Complement, the official integration suite, is wired up for both profiles. make complement runs the account, room and stream tests with federation off — TestRegistration, TestLogin, TestDeviceManagement, the Phase 3 room tests and the Phase 4 TestSync*, TestMessages*, TestContext*, TestFilter*, TestTyping*, TestReceipt* and TestToDevice*, and the Phase 6 media tests. make complement-federation runs the Phase 5 Server-Server tests with federation on, over TLS with Complement's CA.

Quick start

Requires Go 1.26 or newer. No cgo, no C compiler, no database server: the default SQLite driver is pure Go.

cp warp.yaml.example warp.yaml
# edit global.server_name

make build
./bin/warp generate-keys
./bin/warp create-account --username you --admin
./bin/warp serve

Then:

curl localhost:8008/healthz
curl localhost:8008/_matrix/client/versions

Registration is off in a fresh install, which is why the first account is made from the command line. Turn it on from the admin API when you want others to sign up.

Without make, the same from the server directory:

go build -o ../bin/warp ./cmd/warp

Configuration

One YAML file, every field overridable from the environment. See warp.yaml.example — each setting lists its WARP_* variable. With no config file at all the server runs on defaults plus the environment, which is how the container image is meant to be used:

docker build -t warp .
docker run -p 8008:8008 -v warp-data:/data -e WARP_SERVER_NAME=warp.example warp

An unknown field in the config file is an error, not a warning: a typo in a security-relevant setting should never be silently ignored.

Commands

Command	What it does
warp serve	run the homeserver
warp migrate [--status]	apply (or list) pending database migrations
warp generate-keys [--output path] [--key-id id]	create the ed25519 signing key
warp create-account --username name [--admin]	create a local account, straight in the database
warp version	version and compiled-in storage drivers

create-account talks to the database rather than to a running server, so the first account on a new install can be made before anything is listening and without turning registration on. It prompts for the password twice; --password-file and --password are there for scripts.

The signing key is written as a single line, the same format Synapse uses: ed25519 <key-id> <unpadded-base64-seed>. generate-keys refuses to overwrite an existing key — losing it would invalidate every event this server has ever signed.

Federation switch

Federation is off in a fresh install. While it is off, the federation surface does not answer at all: /_matrix/federation/*, /_matrix/key/* and /.well-known/matrix/server return 404 M_UNRECOGNIZED, exactly as they would on a server that has no federation API, and nothing outbound runs.

The switch is stored in the server_settings table and cached in memory, so flipping it — PUT /_warp/admin/v1/federation {"enabled": true} — takes effect on the next request without a restart. Switching it on starts the outbound queue, which picks up whatever it still owed other servers; switching it off stops the queue mid-flight and makes the federation client refuse every request before it opens a connection. Set federation.allow_runtime_toggle: false to pin the switch to the config file.

To federate, other servers have to be able to reach this one: either set global.well_known.server_address to a reverse proxy that terminates TLS, or let Warp terminate it with listen.federation_address: ":8448" and a certificate. See docs/configuration.md and docs/federation.md.

Development

make test          # unit tests (SQLite)
make lint          # golangci-lint
make check         # vet + test, what CI runs
make cover         # coverage.html
make complement    # the Complement C-S subset, federation off (needs Docker)
make complement-federation  # the Complement federation subset

# the storage tests against PostgreSQL as well
WARP_TEST_POSTGRES_DSN='postgres://warp:warp@localhost/warp_test?sslmode=disable' make test-postgres

make test-race needs cgo and therefore a C toolchain; CI runs it on Linux.

Layout

server/                 the homeserver (Go module)
  cmd/warp/             the binary: serve, migrate, generate-keys, create-account, version
  internal/
    config/             YAML + env configuration
    logging/  metrics/  slog and Prometheus setup
    httputil/           Matrix error envelope, CORS, router, middleware
    ratelimit/          per-endpoint token buckets
    federation/         the Server-Server protocol: gate, X-Matrix, discovery, client, keys
    federationapi/      federation endpoints, the outbound queue, the switchable Module (docs/federation.md)
    settings/           runtime settings service (the switches)
    userapi/            accounts, passwords, devices, tokens, account data
    roomserver/         rooms: the input pipeline (docs/rooms.md)
    mediaapi/           uploads, downloads, thumbnails, remote media (docs/media.md)
    clientapi/          the Client-Server API handlers
    adminapi/           /_warp/admin/v1
    setup/              wiring: storage, routes, graceful shutdown
    matrix/             the event core (docs/event-core.md)
      canonicaljson/    the canonical JSON encoding
      signing/          ed25519 server keys, Signing JSON
      id/               identifier grammars as they appear in events
      roomversion/      the room version registry
      event/            PDUs: hashes, event IDs, redaction, signing
      auth/             the authorisation rules and power levels
      stateres/         state resolution v2
    storage/            interfaces + driver registry
      migrations/       embedded migration runner
      sqlstore/         shared database/sql implementation
      filestore/        media content on the local filesystem
      sqlite/ postgres/ pocketbase/
  test/complement/      the Complement homeserver image and runner
apps/                   clients: ui, admin, web, desktop, mobile (Phase 1+)
docs/