Phase 0: bootstrap the Warp homeserver
Scaffolds the server module and everything Phase 0 of PLAN.md calls for: a binary that boots, serves its discovery endpoints and nothing else yet. - config: one YAML file plus per-field WARP_* environment overrides, unknown fields rejected, validation that reports every problem at once - logging and metrics: slog, a Prometheus registry, per-route HTTP instrumentation labelled by pattern rather than path - httputil: the Matrix error envelope, error-returning handlers, CORS, request logging, panic recovery, and a router over http.ServeMux - endpoints: /_matrix/client/versions, /.well-known/matrix/client, /.well-known/matrix/server (gated), /healthz, /metrics, and M_UNRECOGNIZED for everything else - storage: driver registry, shared database/sql base, checksummed migration runner, sqlite and postgres drivers, pocketbase stub - settings: server_settings table behind a cached service, so the federation and registration switches change without a restart - federation: the gate that makes federation a runtime mode - cmd/warp: serve, migrate, generate-keys, version - Makefile, golangci-lint v2, GitHub Actions CI (Linux and Windows tests, race on Linux, storage tests against PostgreSQL, lint, boot smoke test), Dockerfile, example config and docs Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5 committed
commit
b4fb60a8ab
50 files changed
+5672
No files matched your search
@@ -0,0 +1,8 @@
|
||||
# Documentation
|
||||
|
||||
| Document | What |
|
||||
|---|---|
|
||||
| [configuration.md](configuration.md) | every setting, and its environment variable |
|
||||
| [development.md](development.md) | building, testing, adding a migration |
|
||||
|
||||
The design and roadmap are in [PLAN.md](../PLAN.md) at the repository root.
|
||||
@@ -0,0 +1,83 @@
|
||||
# Configuration
|
||||
|
||||
Warp reads one YAML file. Every setting can also come from the
|
||||
environment, and the environment wins. With no file at all the defaults
|
||||
plus `WARP_*` variables are used, which is how the container image runs.
|
||||
|
||||
Where the file comes from, in order:
|
||||
|
||||
1. `--config <path>`
|
||||
2. `$WARP_CONFIG`
|
||||
3. `./warp.yaml`, if it exists
|
||||
4. nothing — defaults and environment only
|
||||
|
||||
An unknown field is a fatal error. A typo in a security-relevant setting
|
||||
must not be ignored.
|
||||
|
||||
List-valued variables are comma-separated. A single `-` clears a list:
|
||||
`WARP_CORS_ALLOWED_ORIGINS=-`.
|
||||
|
||||
## Settings
|
||||
|
||||
| YAML | Environment | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `global.server_name` | `WARP_SERVER_NAME` | — | Required. Part of every ID; cannot be changed later. |
|
||||
| `global.private_key` | `WARP_PRIVATE_KEY` | `./keys/signing.key` | Written by `warp generate-keys`. |
|
||||
| `global.well_known.client_base_url` | `WARP_WELL_KNOWN_CLIENT_BASE_URL` | `https://<server_name>` | Advertised to clients. |
|
||||
| `global.well_known.identity_server_base_url` | `WARP_WELL_KNOWN_IDENTITY_SERVER_BASE_URL` | empty | Omitted from the document when empty. |
|
||||
| `global.well_known.server_address` | `WARP_WELL_KNOWN_SERVER_ADDRESS` | empty | `host:port` for other homeservers. Empty = no delegation, other servers fall back to SRV/A. |
|
||||
| `listen.address` | `WARP_LISTEN_ADDRESS` | `:8008` | |
|
||||
| `listen.read_header_timeout` | `WARP_READ_HEADER_TIMEOUT` | `10s` | |
|
||||
| `listen.read_timeout` | `WARP_READ_TIMEOUT` | `60s` | |
|
||||
| `listen.write_timeout` | `WARP_WRITE_TIMEOUT` | `120s` | Raise before long-polling `/sync` lands in Phase 4. |
|
||||
| `listen.idle_timeout` | `WARP_IDLE_TIMEOUT` | `120s` | |
|
||||
| `listen.max_header_bytes` | `WARP_MAX_HEADER_BYTES` | `1048576` | |
|
||||
| `listen.shutdown_timeout` | `WARP_SHUTDOWN_TIMEOUT` | `30s` | Grace period for in-flight requests. |
|
||||
| `database.driver` | `WARP_DB_DRIVER` | `sqlite` | `sqlite`, `postgres`, `pocketbase` (not implemented). |
|
||||
| `database.connection_string` | `WARP_DB_CONNECTION_STRING` | SQLite file in the working directory | |
|
||||
| `database.max_open_conns` | `WARP_DB_MAX_OPEN_CONNS` | `10` | Ignored by SQLite: writes are serialised onto one connection. |
|
||||
| `database.max_idle_conns` | `WARP_DB_MAX_IDLE_CONNS` | `5` | |
|
||||
| `database.conn_max_lifetime` | `WARP_DB_CONN_MAX_LIFETIME` | `1h` | |
|
||||
| `database.auto_migrate` | `WARP_DB_AUTO_MIGRATE` | `true` | `false` makes startup fail while migrations are pending. |
|
||||
| `client_api.cors_allowed_origins` | `WARP_CORS_ALLOWED_ORIGINS` | `["*"]` | The spec expects any origin. |
|
||||
| `federation.enabled` | `WARP_FEDERATION_ENABLED` | `false` | |
|
||||
| `federation.allowlist` | `WARP_FEDERATION_ALLOWLIST` | `[]` | Non-empty = federate only with these. |
|
||||
| `federation.denylist` | `WARP_FEDERATION_DENYLIST` | `[]` | Always wins over the allowlist. |
|
||||
| `federation.allow_runtime_toggle` | `WARP_FEDERATION_ALLOW_RUNTIME_TOGGLE` | `true` | See below. |
|
||||
| `registration.enabled` | `WARP_REGISTRATION_ENABLED` | `false` | |
|
||||
| `registration.allow_runtime_toggle` | `WARP_REGISTRATION_ALLOW_RUNTIME_TOGGLE` | `true` | |
|
||||
| `logging.level` | `WARP_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error`. |
|
||||
| `logging.format` | `WARP_LOG_FORMAT` | `text` | `text` or `json`. |
|
||||
| `logging.add_source` | `WARP_LOG_ADD_SOURCE` | `false` | File and line in every record. |
|
||||
| `metrics.enabled` | `WARP_METRICS_ENABLED` | `true` | |
|
||||
| `metrics.path` | `WARP_METRICS_PATH` | `/metrics` | Served on the same listener. |
|
||||
|
||||
## Settings that change at runtime
|
||||
|
||||
`federation.enabled` and `registration.enabled` are also stored in the
|
||||
`server_settings` database table, which is what the admin API writes to.
|
||||
|
||||
- With `allow_runtime_toggle: true` (the default) the stored value wins
|
||||
over the config file at boot, and the admin API may change it. The
|
||||
change takes effect on the next request; no restart.
|
||||
- With `allow_runtime_toggle: false` the config file is authoritative and
|
||||
the admin API refuses to change the setting. Any previously stored value
|
||||
is kept but ignored, so re-enabling the toggle restores the
|
||||
administrator's last choice.
|
||||
|
||||
Everything else is read once at startup.
|
||||
|
||||
## What federation off actually means
|
||||
|
||||
While `federation.enabled` is false:
|
||||
|
||||
- `/.well-known/matrix/server` answers `404 M_UNRECOGNIZED`, and so will
|
||||
`/_matrix/federation/*` and `/_matrix/key/*` once they exist.
|
||||
- Nothing outbound runs — in Phase 0 there is nothing outbound yet; from
|
||||
Phase 5 the transaction queue, key fetcher and remote media fetch stay
|
||||
stopped.
|
||||
- Client-Server endpoints that would need a remote server answer
|
||||
`M_FORBIDDEN` with a message saying federation is disabled.
|
||||
- Events are still signed with the server key, and room and event IDs are
|
||||
unchanged. A room created while federation is off federates normally
|
||||
once it is switched on.
|
||||
@@ -0,0 +1,113 @@
|
||||
# Development
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Go 1.26+. Nothing else for the default build: the SQLite driver is pure
|
||||
Go and `CGO_ENABLED=0` works everywhere.
|
||||
- A C toolchain only if you want `go test -race`.
|
||||
- PostgreSQL only to run the storage tests against it.
|
||||
- `golangci-lint` v2 for `make lint`:
|
||||
`go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest`
|
||||
|
||||
## Everyday commands
|
||||
|
||||
```sh
|
||||
make build # bin/warp
|
||||
make test # go test ./... in server/
|
||||
make check # vet + test
|
||||
make lint
|
||||
make cover # coverage.html
|
||||
make clean
|
||||
```
|
||||
|
||||
Each one is a two-line shell command if you would rather not use `make`;
|
||||
look at the [Makefile](../Makefile).
|
||||
|
||||
## Running a server locally
|
||||
|
||||
```sh
|
||||
cp warp.yaml.example warp.yaml # set global.server_name
|
||||
make build
|
||||
./bin/warp generate-keys
|
||||
./bin/warp serve
|
||||
```
|
||||
|
||||
The SQLite file, the signing key and (later) the media store are written
|
||||
relative to the working directory. They are in `.gitignore`.
|
||||
|
||||
For a throwaway server with no config file at all:
|
||||
|
||||
```sh
|
||||
WARP_SERVER_NAME=dev.local WARP_PRIVATE_KEY=./keys/dev.key ./bin/warp generate-keys
|
||||
WARP_SERVER_NAME=dev.local WARP_PRIVATE_KEY=./keys/dev.key ./bin/warp serve
|
||||
```
|
||||
|
||||
## Adding a database migration
|
||||
|
||||
1. Write the same change twice, once per SQL dialect:
|
||||
`server/internal/storage/sqlite/migrations/NNNN_name.sql` and
|
||||
`server/internal/storage/postgres/migrations/NNNN_name.sql`.
|
||||
Use the next free number; `NNNN` is zero-padded to four digits.
|
||||
2. The files are embedded with `embed.FS`, so a rebuild picks them up.
|
||||
3. `./bin/warp migrate --status` lists what would run, `./bin/warp migrate`
|
||||
applies it. A server started with `database.auto_migrate: true` (the
|
||||
default) runs them itself.
|
||||
|
||||
Rules the runner enforces:
|
||||
|
||||
- One transaction per migration. A failure rolls that file back entirely
|
||||
and leaves the following ones unapplied.
|
||||
- A migration that has already been applied and whose file has since
|
||||
changed is an error. Write a new migration instead of editing an old
|
||||
one; the checksum of every applied file is recorded.
|
||||
- A database that has migrations this binary does not know about refuses
|
||||
to start, so an old binary cannot run against a newer schema.
|
||||
|
||||
## Adding an endpoint
|
||||
|
||||
Routes are registered in `server/internal/setup/routes.go` on an
|
||||
`httputil.Router`, which is `http.ServeMux` plus per-route metrics:
|
||||
|
||||
```go
|
||||
r.HandleFunc("GET /_matrix/client/v3/profile/{userID}/displayname", s.handleGetDisplayName)
|
||||
```
|
||||
|
||||
Handlers have the signature `func(http.ResponseWriter, *http.Request) error`
|
||||
and return Matrix errors directly:
|
||||
|
||||
```go
|
||||
func (s *Server) handleThing(w http.ResponseWriter, r *http.Request) error {
|
||||
if somethingWrong {
|
||||
return httputil.Forbidden("You are not allowed to do that")
|
||||
}
|
||||
httputil.WriteJSON(w, http.StatusOK, response{})
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
Anything returned that is not a `*httputil.MatrixError` is logged in full
|
||||
and reported to the client as `M_UNKNOWN`, so internal detail never leaks.
|
||||
|
||||
Endpoints that belong to the federation surface are registered through the
|
||||
gate instead, so they disappear when federation is switched off:
|
||||
|
||||
```go
|
||||
r.Handle("GET /_matrix/federation/v1/version", s.federation.ProtectFunc(s.handleFederationVersion))
|
||||
```
|
||||
|
||||
A Client-Server endpoint that would need federation to answer returns
|
||||
`federation.RejectClientRequest("...")`, which is an `M_FORBIDDEN` that
|
||||
says why.
|
||||
|
||||
## Tests
|
||||
|
||||
- Unit tests live next to the code. `internal/setup` holds the HTTP-level
|
||||
tests; they build a whole server on an in-memory SQLite database.
|
||||
- Storage tests run against every driver that is available:
|
||||
SQLite always, PostgreSQL when `WARP_TEST_POSTGRES_DSN` is set.
|
||||
- CI runs `gofmt -l`, `go mod tidy` (as a diff check), `go vet`, the tests
|
||||
on Linux and Windows, the race detector on Linux, `golangci-lint`, and a
|
||||
smoke test that boots the real binary and curls it.
|
||||
|
||||
Complement, the official integration suite, is wired up in Phase 1 — there
|
||||
is nothing it can test before registration exists.
|
||||
Reference in new issue
Block a user