tajniak81andClaude Opus 5
c96f88e994
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Implements PLAN.md Phase 3: the roomserver, the room half of storage, the Client-Server room endpoints and the admin room endpoints. A client can now create a room, invite, join, leave, kick, ban, send messages and state, redact, name a room with an alias and find one in the public directory. Nothing talks to another server; every operation that would need to is refused with a message that says so rather than a bare M_FORBIDDEN. roomserver: one input pipeline every event goes through, whatever sent it — validate, authorise, resolve, persist, emit. The rules are not reimplemented here; internal/matrix is pure library code and this is what connects it to a database and to a room's history. The Client-Server handlers do no authorisation of their own, deliberately: whether Alice may kick Bob is a question about the room's power levels, and the one place that is answered correctly is the authorisation rules inside the pipeline. A mutex per room serialises writers. Two events racing into one room would each read the same forward extremities, both become one, and fork the room for no reason — which would then need state resolution to put back together. State resolution is wired in anyway for the multi-extremity case. It is unreachable from outside in this phase, and the alternative is a pipeline that authorises against one arbitrary branch of a fork without saying so. Room versions 3 to 12 all work and are each tested end to end. Version 12 earns that coverage: its room ID is the create event's own ID, its create event carries no room_id, and the create event is excluded from every other event's auth_events — so the state auth.Check is given is the event's auth_events plus the create event, which that version identifies by room ID rather than citing. createRoom sends the spec's events in the spec's order, each authorised against what the ones before it produced. A failure part-way leaves an incomplete room, which is the lesser evil: the alternative is holding every event of a new room in memory and writing them in one transaction, making the pipeline that authorises them a special case. The one failure a second client can cause at any time, an alias clash, takes the room row back out. Redactions stay two decisions, as the spec keeps them. The rules let almost any redaction into the room; whether it applies to its target is a separate test, run inside the same transaction as the redaction itself so that a client holding an event ID can always read the stripped event. storage: events keyed by event ID rather than by a surrogate, since an event ID appears in prev_events, auth_events, state snapshots and client requests alike. Stream ordering comes from a counter row bumped inside the write transaction, because a PostgreSQL sequence is explicitly not transactional and a rolled-back write would leave a hole in the stream that /sync would have to be taught to skip. State snapshots are stored per event and deduplicated by content hash, so the table grows with state changes rather than with traffic — and that is what makes "the state after a forward extremity" answerable at all. The room's listing columns are denormalised onto its row because the public directory orders every published room by member count and pages the result, which cannot be done by reading each room's state in turn. Federation-disabled behaviour from PLAN.md section 3 is enforced here. Restricted joins are the exception that works: the rule needs a signature from the authorising user's server verified, and while federation is off that server is always this one. adminapi: room list and detail, purge, force-leave, and room and event counters on the dashboard. clientapi gains its own rate-limit buckets for createRoom and for joining, both of which write events rather than read them. Tested against the real event core and a real database rather than fakes: what is under test is whether an event this server signs is one the authorisation rules accept, and a fake of either would only prove that the pipeline calls what the fake expects. The storage tests run against both SQL drivers. docs/rooms.md describes the pipeline, the storage decisions and the known gaps — chief among them that history visibility is room-level rather than per-event, and that /sync still reports no rooms, both of which belong with the event stream in Phase 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
204 lines
9.1 KiB
Markdown
204 lines
9.1 KiB
Markdown
# Warp
|
|
|
|
A [Matrix](https://spec.matrix.org) homeserver written from scratch in Go,
|
|
plus a family of first-party clients. Federation is optional and can be
|
|
switched on and off at runtime.
|
|
|
|
The full design and roadmap live in [PLAN.md](PLAN.md). This README covers
|
|
what exists today.
|
|
|
|
## Status
|
|
|
|
**Phase 3 — rooms, local only. Complete.** A client can create a room at any
|
|
supported version, invite, join, leave, kick, ban, send messages and state,
|
|
redact, name a room with an alias and find one in the public directory. Every
|
|
event goes through one pipeline — validate, authorise, resolve, persist,
|
|
emit — so a membership change made by `createRoom`, by `/invite` or by a raw
|
|
`PUT .../state/m.room.member` is judged by the same authorisation rules.
|
|
Nothing talks to another server yet, and every operation that would need to
|
|
is refused with a message that says so. There is no `/messages` or real
|
|
`/sync`, so a room's history cannot be paged: that is Phase 4. See
|
|
[docs/rooms.md](docs/rooms.md).
|
|
|
|
**Phase 2 — the event core. Complete.** The library every later phase is
|
|
built on: canonical JSON, the event model with its hashes and identifiers,
|
|
the redaction algorithm, event signing, the room version registry, the
|
|
authorisation rules and state resolution v2. Pure library code under
|
|
`server/internal/matrix`, verified against the spec's own test vectors.
|
|
See [docs/event-core.md](docs/event-core.md).
|
|
|
|
**Phase 1 — accounts and sessions. Complete.** A client can register, log
|
|
in, manage its devices and set its profile.
|
|
|
|
Client-Server API:
|
|
|
|
| | |
|
|
|---|---|
|
|
| `POST /register`, `GET /register/available` | sign-up, with `m.login.dummy` and optional registration tokens |
|
|
| `GET`, `POST /login` · `POST /logout`, `/logout/all` · `POST /refresh` | sessions, with `m.login.password` and refresh tokens |
|
|
| `GET /account/whoami` · `GET /capabilities` | who am I, and what this build can do |
|
|
| `GET /devices`, `GET`/`PUT`/`DELETE /devices/{id}` · `POST /delete_devices` | device management; deletion re-asks for the password |
|
|
| `GET /profile/{user}` and its `displayname` / `avatar_url` sub-resources | profiles — reads are public, writes are your own |
|
|
| `GET`/`PUT /user/{user}/account_data/{type}` and the per-room variant | account data, stored and returned byte for byte |
|
|
| `POST /createRoom` | room versions 3 to 12, presets, `initial_state`, invites, aliases |
|
|
| `POST /join/{roomIdOrAlias}` · `POST /rooms/{id}/join`, `/leave` | joining by ID or alias, and leaving or rejecting an invite |
|
|
| `POST /rooms/{id}/invite`, `/kick`, `/ban`, `/unban` | membership, decided by the room's power levels |
|
|
| `PUT /rooms/{id}/send/{type}/{txn}` | messages, idempotent per transaction ID |
|
|
| `GET`/`PUT /rooms/{id}/state[/{type}[/{key}]]` | room state, read whole or one piece at a time |
|
|
| `PUT /rooms/{id}/redact/{event}/{txn}` | redactions, applied only when they take effect |
|
|
| `GET /rooms/{id}/event/{event}` · `/members` · `/joined_members` · `/aliases` | reading a room |
|
|
| `GET /joined_rooms` | the rooms you are in |
|
|
| `GET`/`PUT`/`DELETE /directory/room/{alias}` | the alias directory |
|
|
| `GET`/`POST /publicRooms` · `GET`/`PUT /directory/list/room/{id}` | the public room directory |
|
|
| `GET /sync` | still a placeholder: a well-formed, empty response, because there is no event stream until Phase 4 |
|
|
| `GET /_matrix/client/versions` · `/.well-known/matrix/client` | discovery |
|
|
| `/.well-known/matrix/server` | server discovery — **only while federation is on** |
|
|
| `GET /healthz` · `GET /metrics` | liveness with a database ping, and Prometheus |
|
|
| anything else under `/_matrix/` | `404 M_UNRECOGNIZED`, as the spec requires |
|
|
|
|
Admin API under `/_warp/admin/v1`: the federation and registration
|
|
switches, user listing and creation, the room list and detail, purging a
|
|
room, removing a user from one, and the dashboard counters. See
|
|
[docs/admin-api.md](docs/admin-api.md).
|
|
|
|
Also in place: argon2id passwords, hashed access and refresh tokens,
|
|
per-endpoint rate limiting, YAML + environment configuration, structured
|
|
logging, graceful shutdown, the storage interfaces with a working SQLite and
|
|
PostgreSQL driver (and a PocketBase stub), the embedded migration runner and
|
|
the runtime settings service.
|
|
|
|
Complement, the official integration suite, is wired up for the
|
|
federation-off profile: `make complement` runs `TestRegistration`,
|
|
`TestLogin`, `TestDeviceManagement` and the Phase 3 room tests.
|
|
|
|
## Quick start
|
|
|
|
Requires Go 1.26 or newer. No cgo, no C compiler, no database server: the
|
|
default SQLite driver is pure Go.
|
|
|
|
```sh
|
|
cp warp.yaml.example warp.yaml
|
|
# edit global.server_name
|
|
|
|
make build
|
|
./bin/warp generate-keys
|
|
./bin/warp create-account --username you --admin
|
|
./bin/warp serve
|
|
```
|
|
|
|
Then:
|
|
|
|
```sh
|
|
curl localhost:8008/healthz
|
|
curl localhost:8008/_matrix/client/versions
|
|
```
|
|
|
|
Registration is off in a fresh install, which is why the first account is
|
|
made from the command line. Turn it on from the admin API when you want
|
|
others to sign up.
|
|
|
|
Without `make`, the same from the `server` directory:
|
|
|
|
```sh
|
|
go build -o ../bin/warp ./cmd/warp
|
|
```
|
|
|
|
## Configuration
|
|
|
|
One YAML file, every field overridable from the environment. See
|
|
[warp.yaml.example](warp.yaml.example) — each setting lists its `WARP_*`
|
|
variable. With no config file at all the server runs on defaults plus the
|
|
environment, which is how the container image is meant to be used:
|
|
|
|
```sh
|
|
docker build -t warp .
|
|
docker run -p 8008:8008 -v warp-data:/data -e WARP_SERVER_NAME=warp.example warp
|
|
```
|
|
|
|
An unknown field in the config file is an error, not a warning: a typo in a
|
|
security-relevant setting should never be silently ignored.
|
|
|
|
## Commands
|
|
|
|
| Command | What it does |
|
|
|---|---|
|
|
| `warp serve` | run the homeserver |
|
|
| `warp migrate [--status]` | apply (or list) pending database migrations |
|
|
| `warp generate-keys [--output path] [--key-id id]` | create the ed25519 signing key |
|
|
| `warp create-account --username name [--admin]` | create a local account, straight in the database |
|
|
| `warp version` | version and compiled-in storage drivers |
|
|
|
|
`create-account` talks to the database rather than to a running server, so
|
|
the first account on a new install can be made before anything is listening
|
|
and without turning registration on. It prompts for the password twice;
|
|
`--password-file` and `--password` are there for scripts.
|
|
|
|
The signing key is written as a single line, the same format Synapse uses:
|
|
`ed25519 <key-id> <unpadded-base64-seed>`. `generate-keys` refuses to
|
|
overwrite an existing key — losing it would invalidate every event this
|
|
server has ever signed.
|
|
|
|
## Federation switch
|
|
|
|
Federation is off in a fresh install. While it is off, the federation
|
|
surface does not answer at all; `/.well-known/matrix/server` returns
|
|
`404 M_UNRECOGNIZED`, exactly as it would on a server that has no
|
|
federation API.
|
|
|
|
The switch is stored in the `server_settings` table and cached in memory,
|
|
so flipping it takes effect on the next request without a restart. In
|
|
Phase 0 the only thing behind the gate is the delegation document; the
|
|
outbound queue, key fetcher and remote media fetch join it in Phase 5. Set
|
|
`federation.allow_runtime_toggle: false` to pin the switch to the config
|
|
file.
|
|
|
|
## Development
|
|
|
|
```sh
|
|
make test # unit tests (SQLite)
|
|
make lint # golangci-lint
|
|
make check # vet + test, what CI runs
|
|
make cover # coverage.html
|
|
make complement # the Complement subset for the phases that are done (needs Docker)
|
|
|
|
# the storage tests against PostgreSQL as well
|
|
WARP_TEST_POSTGRES_DSN='postgres://warp:warp@localhost/warp_test?sslmode=disable' make test-postgres
|
|
```
|
|
|
|
`make test-race` needs cgo and therefore a C toolchain; CI runs it on
|
|
Linux.
|
|
|
|
## Layout
|
|
|
|
```
|
|
server/ the homeserver (Go module)
|
|
cmd/warp/ the binary: serve, migrate, generate-keys, create-account, version
|
|
internal/
|
|
config/ YAML + env configuration
|
|
logging/ metrics/ slog and Prometheus setup
|
|
httputil/ Matrix error envelope, CORS, router, middleware
|
|
ratelimit/ per-endpoint token buckets
|
|
federation/ the federation gate and allow/deny policy
|
|
settings/ runtime settings service (the switches)
|
|
userapi/ accounts, passwords, devices, tokens, account data
|
|
roomserver/ rooms: the input pipeline (docs/rooms.md)
|
|
clientapi/ the Client-Server API handlers
|
|
adminapi/ /_warp/admin/v1
|
|
setup/ wiring: storage, routes, graceful shutdown
|
|
matrix/ the event core (docs/event-core.md)
|
|
canonicaljson/ the canonical JSON encoding
|
|
signing/ ed25519 server keys, Signing JSON
|
|
id/ identifier grammars as they appear in events
|
|
roomversion/ the room version registry
|
|
event/ PDUs: hashes, event IDs, redaction, signing
|
|
auth/ the authorisation rules and power levels
|
|
stateres/ state resolution v2
|
|
storage/ interfaces + driver registry
|
|
migrations/ embedded migration runner
|
|
sqlstore/ shared database/sql implementation
|
|
sqlite/ postgres/ pocketbase/
|
|
test/complement/ the Complement homeserver image and runner
|
|
apps/ clients: ui, admin, web, desktop, mobile (Phase 1+)
|
|
docs/
|
|
```
|