Warp/docs/development.md

Development

Prerequisites

• Go 1.26+. Nothing else for the default build: the SQLite driver is pure Go and CGO_ENABLED=0 works everywhere.
• A C toolchain only if you want go test -race.
• PostgreSQL only to run the storage tests against it.
• golangci-lint v2 for make lint: go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest

Everyday commands

make build         # bin/warp
make test          # go test ./... in server/
make check         # vet + test
make lint
make cover         # coverage.html
make clean

Each one is a two-line shell command if you would rather not use make; look at the Makefile.

Running a server locally

cp warp.yaml.example warp.yaml   # set global.server_name
make build
./bin/warp generate-keys
./bin/warp serve

The SQLite file, the signing key and (later) the media store are written relative to the working directory. They are in .gitignore.

For a throwaway server with no config file at all:

WARP_SERVER_NAME=dev.local WARP_PRIVATE_KEY=./keys/dev.key ./bin/warp generate-keys
WARP_SERVER_NAME=dev.local WARP_PRIVATE_KEY=./keys/dev.key ./bin/warp serve

Adding a database migration

1. Write the same change twice, once per SQL dialect: server/internal/storage/sqlite/migrations/NNNN_name.sql and server/internal/storage/postgres/migrations/NNNN_name.sql. Use the next free number; NNNN is zero-padded to four digits. The pocketbase driver uses the SQLite files as they are.
2. The files are embedded with embed.FS, so a rebuild picks them up.
3. ./bin/warp migrate --status lists what would run, ./bin/warp migrate applies it. A server started with database.auto_migrate: true (the default) runs them itself.

Rules the runner enforces:

• One transaction per migration. A failure rolls that file back entirely and leaves the following ones unapplied.
• A migration that has already been applied and whose file has since changed is an error. Write a new migration instead of editing an old one; the checksum of every applied file is recorded.
• A database that has migrations this binary does not know about refuses to start, so an old binary cannot run against a newer schema.

Long-polling endpoints

/sync holds a request open until something happens, which has two consequences worth knowing about when running a server locally.

listen.write_timeout must stay above sync.long_poll_timeout, or the listener cuts off a blocked sync before it can answer. The defaults (120s and 30s) leave room; a proxy in front of the server needs the same check.

A blocked sync is woken through syncapi.Notifier, not by polling, so a test that sends an event and then expects a blocked sync to return has to make sure the sync is actually blocked first — otherwise it finds the event without ever having waited, and passes without testing anything. Notifier.Listeners() exists for that, and internal/clientapi/sync_test.go shows the pattern.

Adding an endpoint

Routes are registered in server/internal/setup/routes.go on an httputil.Router, which is http.ServeMux plus per-route metrics:

r.HandleFunc("GET /_matrix/client/v3/profile/{userID}/displayname", s.handleGetDisplayName)

Handlers have the signature func(http.ResponseWriter, *http.Request) error and return Matrix errors directly:

func (s *Server) handleThing(w http.ResponseWriter, r *http.Request) error {
    if somethingWrong {
        return httputil.Forbidden("You are not allowed to do that")
    }
    httputil.WriteJSON(w, http.StatusOK, response{})
    return nil
}

Anything returned that is not a *httputil.MatrixError is logged in full and reported to the client as M_UNKNOWN, so internal detail never leaks.

Endpoints that belong to the federation surface are registered through the gate instead, so they disappear when federation is switched off:

r.Handle("GET /_matrix/federation/v1/version", s.federation.ProtectFunc(s.handleFederationVersion))

A Client-Server endpoint that would need federation to answer returns federation.RejectClientRequest("..."), which is an M_FORBIDDEN that says why.

Tests

• Unit tests live next to the code. Each API package holds its own HTTP-level tests, built on a throwaway SQLite database with the real components behind them rather than fakes; internal/setup holds the smaller set that goes through the whole wrapped handler, to catch a route that was written but never registered.
• Storage tests run against every driver that is available: SQLite and PocketBase always, PostgreSQL when WARP_TEST_POSTGRES_DSN is set. PocketBase runs the SQLite migrations on PocketBase's own database, so a SQLite migration is a PocketBase one too. The PostgreSQL tests share one database, so each group of them truncates the tables it owns first; add a table to resetRoomTables or resetSyncTables when you add one, or the second test to insert the same row will collide — and only in CI, where PostgreSQL exists.
• The event core under internal/matrix is tested against the spec's own cryptographic and canonical JSON test vectors, and has fuzz targets for the two functions that read bytes straight off the network. make test runs the unit tests; the fuzzers are opt-in. See event-core.md.
• CI runs gofmt -l, go mod tidy (as a diff check), go vet, the tests on Linux and Windows, the race detector on Linux, golangci-lint, and a smoke test that boots the real binary and curls it.

Complement

Complement is the official Matrix integration suite. It runs a container per homeserver per test and drives it over the real HTTP API, which makes its pass count the honest measure of how far along the server is — PLAN.md section 9 treats it as the primary progress metric.

make complement                                   # the C-S set, federation off
make complement-federation                        # the federation set
server/test/complement/run.sh TestLogin           # one test
COMPLEMENT_DEBUG=1 make complement                # with the container logs

It needs Docker. The script builds the image, clones Complement into .complement/ (ignored by git) and runs the subset for the phases that are done. COMPLEMENT_PROFILE picks the set: csapi (the default) runs the account, room and stream tests with federation off; federation runs the Server-Server tests with it on. Extend the sets in run.sh as each phase lands, rather than switching to the whole suite: a pass count that drops because a later phase is missing is noise.

The image is server/test/complement/Dockerfile and the deployment is entirely WARP_* environment variables — see entrypoint.sh. Two things there are worth knowing: rate limiting is off, because the whole suite comes from one address and would otherwise spend its time reading M_LIMIT_EXCEEDED; and port 8448 is served over TLS with a certificate the entrypoint has the CA Complement mounts at /complement/ca sign, the same CA Warp is told to trust for its outbound connections.

Creating the first account

A new install has registration off, so the first account is made from the command line against the database directly — no server needs to be running:

./bin/warp create-account --username you --admin

It prompts for the password twice. --password-file reads it from a file and --password takes it inline, for scripts that have already thought about the shell history. From there, the admin API can turn registration on:

curl -X PUT http://localhost:8008/_warp/admin/v1/registration   -H "Authorization: Bearer $TOKEN" -d '{"enabled": true}'