Warp/docs/deployment.md

Deployment and operations

What there is for running Warp somewhere other than a laptop: the container image, a Compose setup, a Helm chart, a Grafana dashboard, tracing, and a load tester. All of it is under deploy/ except the image, which is the Dockerfile at the root, and the load tester, which is server/cmd/warp-loadtest.

Choosing a storage driver

Driver	For	Notes
postgres	production	Several connections, so reads and writes run side by side.
sqlite	development, a single user, a small server	One writer. Everything is one file plus media.base_path.
pocketbase	an "appliance": one binary, one data directory	The sqlite driver's tables inside an embedded PocketBase app's data.db, written through PocketBase's own single-writer connection. Media goes into PocketBase's filesystem, which can be an S3 bucket (database.pocketbase.s3).

The pocketbase driver is the SQLite schema on the SQLite engine, so it is at parity with sqlite by construction, and the storage tests run against both. What it adds is PocketBase around it: its system tables in the same database, its auxiliary.db for its own logs, and its filesystem abstraction for media. Warp does not serve PocketBase's admin UI or API; the app is reachable from Go through pocketbase.Database.App() for features that want it later. Back up the whole data directory.

The container image

docker build -t warp .

One static binary on Alpine, running as UID 10001, with /data as its working directory and volume. The defaults put the SQLite database, the signing key and media under /data. The image does not create a signing key on its own; the Compose file and the Helm chart do it on first start, and a plain docker run needs one first:

docker run --rm -v warp-data:/data warp generate-keys --output /data/signing.key

Compose

deploy/compose runs Warp with PostgreSQL, and with the observability profile, Prometheus, Grafana (with the dashboard provisioned) and Jaeger:

cd deploy/compose
WARP_SERVER_NAME=matrix.example.org docker compose up -d
docker compose --profile observability up -d
docker compose exec warp warp create-account --username admin --admin

Settings come from the environment, so anything in configuration.md can be added to the warp service. WARP_TRACING_ENABLED=true sends traces to the Jaeger container.

Helm

deploy/helm/warp is a chart for one Warp pod. Warp is one process, so the deployment always runs exactly one replica and replaces it with the Recreate strategy: two pods would be two homeservers with the same name.

helm install warp deploy/helm/warp \
  --set serverName=example.org \
  --set publicBaseURL=https://matrix.example.org \
  --set image.repository=registry.example.org/warp --set image.tag=0.10.0 \
  --set ingress.enabled=true

There is no published image; build one and push it to a registry the cluster can pull from. The chart creates a volume for /data that is kept when the release is deleted, because the signing key is on it unless signingKey.existingSecret names a secret holding one. For PostgreSQL, set database.driver=postgres and either database.connectionString or database.existingSecret. metrics.serviceMonitor.enabled adds a prometheus-operator ServiceMonitor. The ingress needs a read timeout above the 30-second long poll; values.yaml has the ingress-nginx annotations.

Reverse proxy

Behind a proxy that terminates TLS:

• Route /_matrix, /.well-known/matrix and /_warp to Warp.
• Keep the proxy's read timeout above sync.long_poll_timeout (30 s), or every idle /sync is cut off.
• Allow request bodies of media.max_upload_bytes (50 MiB by default).
• Set client_api.trust_x_forwarded_for: true only if the proxy overwrites X-Forwarded-For; the rate limiter keys on it.

Metrics and the dashboard

/metrics is Prometheus text, on by default. Besides the Go runtime and process collectors:

Metric
warp_http_requests_total{route,method,status}	Requests by route pattern (never by path, so room and user IDs cannot explode the cardinality).
warp_http_request_duration_seconds{route,method}	Latency histogram. Sync endpoints block by design, so leave them out of latency quantiles.
warp_http_requests_in_flight
warp_events_persisted_total{origin,kind}	Room events persisted: origin is local or federation, kind is state or message.
warp_sync_waiting	Syncs blocked in a long poll, classic and sliding: roughly the connected clients.
warp_sync_sliding_connections	Sliding sync connections whose state the server holds.
warp_federation_enabled	1 while federation is on.
go_sql_*{db_name}	The connection pool: open, in use, waits.
warp_build_info{version,commit}

deploy/grafana/warp.json is a Grafana dashboard over these: overview stats, requests and latency by route, events, sync connections, the database pool and the Go runtime. Import it and pick the Prometheus data source; the job and instance variables select the server.

Tracing

With tracing.enabled, Warp exports OpenTelemetry traces over OTLP/HTTP to tracing.endpoint (or to OTEL_EXPORTER_OTLP_ENDPOINT, or to localhost:4318): an OpenTelemetry Collector, Jaeger, Tempo, Honeycomb.

• Every request is a server span named after its route pattern, with the method, path, user agent and status.
• Persisting an event, building a sync or a sliding sync response, and running a search are spans of their own inside it.
• Every request Warp sends — to other servers, to push gateways, to application services — is a client span, and carries the W3C traceparent on, so a federation request shows up in the same trace as the client request that caused it.
• A request that arrives with a traceparent continues that trace.

tracing.sample_ratio keeps that fraction of new traces; a request inside a sampled trace is always kept. Off, tracing costs a function call per span, and the trace context of incoming requests is still passed on.

Load testing

warp-loadtest drives a server the way a crowd of clients would:

make loadtest ARGS="-users 50 -rooms 10 -rate 0.5 -duration 2m"
# or
go run ./server/cmd/warp-loadtest -url http://localhost:8008 -users 50 -rooms 10 -rate 0.5 -duration 2m -sliding

It registers the users (answering the dummy and registration-token stages), creates the rooms and joins everybody to all of them, then for the length of the run each user sends at -rate messages per second while syncing in a long-poll loop — /sync, or sliding sync with -sliding. Each message carries the time it was sent, so the report has, beside the latency of every kind of request, the delivery time: from one user sending to another user's sync handing it over, which is the number a user of a chat server feels.

Point it at a test server with registration open and rate limiting off (WARP_RATE_LIMIT_ENABLED=false), or it measures the rate limiter. It creates accounts and rooms and never cleans them up.

A run on a laptop, PocketBase driver, 20 users in 4 rooms at one message a second each (-users 20 -rooms 4 -rate 1 -duration 15s):

operation	count	p50	p99
send	280	24 ms	52 ms
delivery (/sync)	5320	147 ms	280 ms
delivery (sliding sync)	5320	205 ms	368 ms

Every message reached every other user. Those numbers are one writer serving twenty clients that each re-sync on every message; PostgreSQL's parallel reads are what moves them.