Warp/docs/deployment.md

Deployment and operations

What there is for running Warp somewhere other than a laptop: the container image, a Compose setup, a Helm chart, a Grafana dashboard, tracing, and a load tester. All of it is under deploy/ except the image, which is the Dockerfile at the root, and the load tester, which is server/cmd/warp-loadtest.

Choosing a storage driver

Driver	For	Notes
postgres	production	Several connections, so reads and writes run side by side.
sqlite	development, a single user, a small server	One writer. Everything is one file plus media.base_path.
pocketbase	a PocketBase server you want to see and manage the data in	Warp's tables as PocketBase collections on a PocketBase server of its own (0.40 or later), reached only through its REST API. Media goes into PocketBase's storage, which can be an S3 bucket (database.pocketbase.s3).

The pocketbase driver connects to PocketBase the way the postgres driver connects to PostgreSQL: database.connection_string is the server's URL, and Warp signs in as the superuser in database.pocketbase. Every table the SQL drivers have is a superuser-only collection of the same name and columns, so the PocketBase admin UI shows Warp's accounts, rooms and events as records. Warp creates and updates the collections itself (warp migrate), reads them through PocketBase's SQL endpoint with the same queries the SQL drivers run, and writes them as record operations in batch requests, each one a transaction. It turns batch requests on in PocketBase's settings at startup.

Writes go through one lock in the Warp process, so one Warp server per PocketBase, and every database call is an HTTP request: expect it to be slower than PostgreSQL. Search has no full-text engine behind it; it matches the start of words, case-insensitively. The storage tests run against a real PocketBase. Back up PocketBase's whole pb_data.

The container image

docker build -t warp .

One static binary on Alpine, running as UID 10001, with /data as its working directory and volume. The defaults put the SQLite database, the signing key and media under /data. The image does not create a signing key on its own; the Compose file and the Helm chart do it on first start, and a plain docker run needs one first:

docker run --rm -v warp-data:/data warp generate-keys --output /data/signing.key

Compose

deploy/compose runs Warp with PostgreSQL, and with the observability profile, Prometheus, Grafana (with the dashboard provisioned) and Jaeger:

cd deploy/compose
WARP_SERVER_NAME=matrix.example.org docker compose up -d
docker compose --profile observability up -d
docker compose exec warp warp create-account --username admin --admin

Settings come from the environment, so anything in configuration.md can be added to the warp service. WARP_TRACING_ENABLED=true sends traces to the Jaeger container.

Helm

deploy/helm/warp is a chart for one Warp pod. Warp is one process, so the deployment always runs exactly one replica and replaces it with the Recreate strategy: two pods would be two homeservers with the same name.

helm install warp deploy/helm/warp \
  --set serverName=example.org \
  --set publicBaseURL=https://matrix.example.org \
  --set image.repository=registry.example.org/warp --set image.tag=0.10.0 \
  --set ingress.enabled=true

There is no published image; build one and push it to a registry the cluster can pull from. The chart creates a volume for /data that is kept when the release is deleted, because the signing key is on it unless signingKey.existingSecret names a secret holding one. For PostgreSQL, set database.driver=postgres and either database.connectionString or database.existingSecret. metrics.serviceMonitor.enabled adds a prometheus-operator ServiceMonitor. The ingress needs a read timeout above the 30-second long poll; values.yaml has the ingress-nginx annotations.

The admin panel is off by default. admin.enabled=true adds a Deployment and Service for warp-admin on port 8009, pointed at the homeserver's Service. Its image is built separately, also from the repository root:

docker build -t registry.example.org/warp-admin:0.10.0 -f apps/admin/Dockerfile .
helm upgrade warp deploy/helm/warp --reuse-values \
  --set admin.enabled=true \
  --set admin.image.repository=registry.example.org/warp-admin \
  --set admin.image.tag=0.10.0

Without an ingress, reach it with kubectl port-forward svc/warp-admin 8009. admin.ingress.enabled with admin.ingress.host gives it its own ingress on a separate host, ideally one only admins can reach. warp-admin reaches /_warp/admin inside the cluster, so you can then drop /_warp from ingress.hosts[].paths. admin.resources and admin.replicas are in values.yaml.

Admin panel

The admin panel is optional and is not part of the warp binary. It is a separate service, warp-admin, that serves the panel and proxies the login endpoints and /_warp/admin/v1 to the homeserver. Install it only if you want it:

cd deploy/compose
docker compose --profile admin up -d    # http://localhost:8009

Or build the image (make admin-docker) or the binary (make admin) and point it at the homeserver with WARP_ADMIN_HOMESERVER. Since only warp-admin needs /_warp/admin, your public proxy can leave that path out and keep the panel on an internal hostname. See apps/admin.

Web client

The web client is optional too, and separate from both: warp-web serves it and, by default, proxies the client's Matrix requests (/_matrix/client and /_matrix/media, nothing else) to the homeserver, so the browser only ever talks to warp-web.

cd deploy/compose
docker compose --profile web up -d      # http://localhost:8080

Or build the image (make web-docker) or the binary (make web) and point it at the homeserver with WARP_WEB_HOMESERVER. If your homeserver is already public behind its own proxy, set WARP_WEB_PUBLIC_HOMESERVER to its URL instead: the browser then calls it directly and warp-web serves only the client. See apps/web.

Calls

Calls need a LiveKit SFU and the LiveKit JWT service next to Warp. For a local try, deploy/compose/calls.yml adds both:

cd deploy/compose
WARP_REGISTRATION_ENABLED=true docker compose -f docker-compose.yml -f calls.yml --profile web up -d --build

The Helm chart can run the JWT service (calls.jwtService.enabled) and sets client_api.rtc.livekit_service_url from it; LiveKit itself runs from its own chart. See calls.md for how the pieces find each other and what production needs.

Reverse proxy

Behind a proxy that terminates TLS:

• Route /_matrix and /.well-known/matrix to Warp. Route /_warp too only if something outside reaches the admin API directly; warp-admin talks to Warp over its own connection.
• Keep the proxy's read timeout above sync.long_poll_timeout (30 s), or every idle /sync is cut off.
• Allow request bodies of media.max_upload_bytes (50 MiB by default).
• Set client_api.trust_x_forwarded_for: true only if the proxy overwrites X-Forwarded-For; the rate limiter keys on it.

Metrics and the dashboard

/metrics is Prometheus text, on by default. Besides the Go runtime and process collectors:

Metric
warp_http_requests_total{route,method,status}	Requests by route pattern (never by path, so room and user IDs cannot explode the cardinality).
warp_http_request_duration_seconds{route,method}	Latency histogram. Sync endpoints block by design, so leave them out of latency quantiles.
warp_http_requests_in_flight
warp_events_persisted_total{origin,kind}	Room events persisted: origin is local or federation, kind is state or message.
warp_sync_waiting	Syncs blocked in a long poll, classic and sliding: roughly the connected clients.
warp_sync_sliding_connections	Sliding sync connections whose state the server holds.
warp_federation_enabled	1 while federation is on.
go_sql_*{db_name}	The connection pool: open, in use, waits.
warp_build_info{version,commit}

deploy/grafana/warp.json is a Grafana dashboard over these: overview stats, requests and latency by route, events, sync connections, the database pool and the Go runtime. Import it and pick the Prometheus data source; the job and instance variables select the server.

Tracing

With tracing.enabled, Warp exports OpenTelemetry traces over OTLP/HTTP to tracing.endpoint (or to OTEL_EXPORTER_OTLP_ENDPOINT, or to localhost:4318): an OpenTelemetry Collector, Jaeger, Tempo, Honeycomb.

• Every request is a server span named after its route pattern, with the method, path, user agent and status.
• Persisting an event, building a sync or a sliding sync response, and running a search are spans of their own inside it.
• Every request Warp sends — to other servers, to push gateways, to application services — is a client span, and carries the W3C traceparent on, so a federation request shows up in the same trace as the client request that caused it.
• A request that arrives with a traceparent continues that trace.

tracing.sample_ratio keeps that fraction of new traces; a request inside a sampled trace is always kept. Off, tracing costs a function call per span, and the trace context of incoming requests is still passed on.

Load testing

warp-loadtest drives a server the way a crowd of clients would:

make loadtest ARGS="-users 50 -rooms 10 -rate 0.5 -duration 2m"
# or
go run ./server/cmd/warp-loadtest -url http://localhost:8008 -users 50 -rooms 10 -rate 0.5 -duration 2m -sliding

It registers the users (answering the dummy and registration-token stages), creates the rooms and joins everybody to all of them, then for the length of the run each user sends at -rate messages per second while syncing in a long-poll loop — /sync, or sliding sync with -sliding. Each message carries the time it was sent, so the report has, beside the latency of every kind of request, the delivery time: from one user sending to another user's sync handing it over, which is the number a user of a chat server feels.

Point it at a test server with registration open and rate limiting off (WARP_RATE_LIMIT_ENABLED=false), or it measures the rate limiter. It creates accounts and rooms and never cleans them up.

A run on a laptop, PocketBase driver, 20 users in 4 rooms at one message a second each (-users 20 -rooms 4 -rate 1 -duration 15s):

operation	count	p50	p99
send	280	24 ms	52 ms
delivery (/sync)	5320	147 ms	280 ms
delivery (sliding sync)	5320	205 ms	368 ms

Every message reached every other user. Those numbers are one writer serving twenty clients that each re-sync on every message; PostgreSQL's parallel reads are what moves them.