diff --git a/README.md b/README.md index 10ab788..7ea29c2 100644 --- a/README.md +++ b/README.md @@ -9,17 +9,26 @@ what exists today. ## Status -**Phase 2 — the event core. Complete.** The library that every later phase -is built on: canonical JSON, the event model with its hashes and -identifiers, the redaction algorithm, event signing, the room version -registry, the authorisation rules and state resolution v2. It is pure -library code under `server/internal/matrix`, verified against the spec's -own test vectors, and nothing serves it over HTTP yet — rooms are Phase 3. +**Phase 3 — rooms, local only. Complete.** A client can create a room at any +supported version, invite, join, leave, kick, ban, send messages and state, +redact, name a room with an alias and find one in the public directory. Every +event goes through one pipeline — validate, authorise, resolve, persist, +emit — so a membership change made by `createRoom`, by `/invite` or by a raw +`PUT .../state/m.room.member` is judged by the same authorisation rules. +Nothing talks to another server yet, and every operation that would need to +is refused with a message that says so. There is no `/messages` or real +`/sync`, so a room's history cannot be paged: that is Phase 4. See +[docs/rooms.md](docs/rooms.md). + +**Phase 2 — the event core. Complete.** The library every later phase is +built on: canonical JSON, the event model with its hashes and identifiers, +the redaction algorithm, event signing, the room version registry, the +authorisation rules and state resolution v2. Pure library code under +`server/internal/matrix`, verified against the spec's own test vectors. See [docs/event-core.md](docs/event-core.md). **Phase 1 — accounts and sessions. Complete.** A client can register, log -in, manage its devices and set its profile. There are no rooms and no real -`/sync` yet: those are Phases 3 and 4. +in, manage its devices and set its profile. Client-Server API: @@ -31,14 +40,25 @@ Client-Server API: | `GET /devices`, `GET`/`PUT`/`DELETE /devices/{id}` · `POST /delete_devices` | device management; deletion re-asks for the password | | `GET /profile/{user}` and its `displayname` / `avatar_url` sub-resources | profiles — reads are public, writes are your own | | `GET`/`PUT /user/{user}/account_data/{type}` and the per-room variant | account data, stored and returned byte for byte | -| `GET /sync` | a placeholder: an empty, well-formed response, because there is nothing to sync yet | +| `POST /createRoom` | room versions 3 to 12, presets, `initial_state`, invites, aliases | +| `POST /join/{roomIdOrAlias}` · `POST /rooms/{id}/join`, `/leave` | joining by ID or alias, and leaving or rejecting an invite | +| `POST /rooms/{id}/invite`, `/kick`, `/ban`, `/unban` | membership, decided by the room's power levels | +| `PUT /rooms/{id}/send/{type}/{txn}` | messages, idempotent per transaction ID | +| `GET`/`PUT /rooms/{id}/state[/{type}[/{key}]]` | room state, read whole or one piece at a time | +| `PUT /rooms/{id}/redact/{event}/{txn}` | redactions, applied only when they take effect | +| `GET /rooms/{id}/event/{event}` · `/members` · `/joined_members` · `/aliases` | reading a room | +| `GET /joined_rooms` | the rooms you are in | +| `GET`/`PUT`/`DELETE /directory/room/{alias}` | the alias directory | +| `GET`/`POST /publicRooms` · `GET`/`PUT /directory/list/room/{id}` | the public room directory | +| `GET /sync` | still a placeholder: a well-formed, empty response, because there is no event stream until Phase 4 | | `GET /_matrix/client/versions` · `/.well-known/matrix/client` | discovery | | `/.well-known/matrix/server` | server discovery — **only while federation is on** | | `GET /healthz` · `GET /metrics` | liveness with a database ping, and Prometheus | | anything else under `/_matrix/` | `404 M_UNRECOGNIZED`, as the spec requires | Admin API under `/_warp/admin/v1`: the federation and registration -switches, user listing and creation, and the dashboard counters. See +switches, user listing and creation, the room list and detail, purging a +room, removing a user from one, and the dashboard counters. See [docs/admin-api.md](docs/admin-api.md). Also in place: argon2id passwords, hashed access and refresh tokens, @@ -49,7 +69,7 @@ the runtime settings service. Complement, the official integration suite, is wired up for the federation-off profile: `make complement` runs `TestRegistration`, -`TestLogin` and `TestDeviceManagement`. +`TestLogin`, `TestDeviceManagement` and the Phase 3 room tests. ## Quick start @@ -161,6 +181,7 @@ server/ the homeserver (Go module) federation/ the federation gate and allow/deny policy settings/ runtime settings service (the switches) userapi/ accounts, passwords, devices, tokens, account data + roomserver/ rooms: the input pipeline (docs/rooms.md) clientapi/ the Client-Server API handlers adminapi/ /_warp/admin/v1 setup/ wiring: storage, routes, graceful shutdown diff --git a/docs/README.md b/docs/README.md index a996e49..22ef07c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,6 +5,7 @@ | [configuration.md](configuration.md) | every setting, and its environment variable | | [development.md](development.md) | building, testing, Complement, adding a migration | | [event-core.md](event-core.md) | `internal/matrix`: canonical JSON, events, auth rules, state resolution | +| [rooms.md](rooms.md) | `internal/roomserver`: the input pipeline, room storage, known gaps | | [admin-api.md](admin-api.md) | `/_warp/admin/v1`: the operator API behind the admin panel | The design and roadmap are in [PLAN.md](../PLAN.md) at the repository root. diff --git a/docs/admin-api.md b/docs/admin-api.md index 50e1f84..cfa51b0 100644 --- a/docs/admin-api.md +++ b/docs/admin-api.md @@ -101,6 +101,94 @@ the admin's own action, not a public sign-up. `username` is normalised (lower-cased) like any other localpart. A taken name is `400 M_USER_IN_USE`, an unusable one `400 M_INVALID_USERNAME`. +### `GET /rooms` + +``` +GET /rooms?search=ops&limit=50&offset=0 +``` + +Ordered by joined-member count, biggest first, then by room ID — the second +key is not decoration, because paging an order that is not total can show +the same room on two pages and skip another, and ties in member count are +the common case. `search` matches a substring of the room ID, name, topic or +canonical alias, case-insensitively. + +```json +{ + "rooms": [ + { + "room_id": "!abc:warp.example", + "room_version": "11", + "creator": "@alice:warp.example", + "name": "Operations", + "topic": "Ops chat", + "canonical_alias": "#ops:warp.example", + "join_rule": "public", + "history_visibility": "shared", + "guest_access": "forbidden", + "published": true, + "joined_members": 12, + "created_at_ms": 1760000000000 + } + ], + "total": 1, + "limit": 50, + "offset": 0 +} +``` + +Everything after `creator` is read from the room's current state and kept in +the room's own row. That is denormalisation on purpose: this listing and the +public room directory both order every room by member count and page the +result, which cannot be done by reading each room's state in turn. + +### `GET /rooms/{roomID}` + +The list row plus the things only a count can answer. + +```json +{ + "room_id": "!abc:warp.example", + "members": 15, + "invited_members": 2, + "joined_members": 12, + "aliases": ["#ops:warp.example"], + "forward_extremities": ["$sPfXR..."] +} +``` + +`members` counts every `m.room.member` event in the current state whatever +the membership, because a room with one member and forty bans is not the +same room as one with a single member. `forward_extremities` is where the +room's DAG ends; there is normally exactly one, and more than one means the +room has forked. + +### `DELETE /rooms/{roomID}` + +Purges the room: its events, its state, its aliases and its row. There is no +undo and the members are not told — the only way to tell them would have +been to add an event to a room that no longer exists. + +It is a `DELETE` on the room rather than a `POST` to an action because that +is what it is, and because the method makes a panel's confirm dialog +mandatory in a way a `POST` does not. Purging a room that is not there is +`404`, not a silent success: an operator who sees an `OK` has to be able to +believe the room was there. + +### `POST /rooms/{roomID}/force_leave` + +```json +{"user_id": "@alice:warp.example"} +``` + +Makes a user leave without asking the room's rules whether they may. A user +stuck in a room — one whose power levels are broken, or whose last moderator +has gone — is what this exists for, and in such a room the rules are exactly +what is in the way. It is still an ordinary membership event, signed and +added to the room, so the other members see what happened. + +A user who is not in the room gets `403`: there is nothing to leave. + ### `GET /stats` The dashboard payload. @@ -114,13 +202,16 @@ The dashboard payload. "admins": 1, "deactivated": 0, "devices": 19, + "rooms": 4, + "published_rooms": 1, + "events": 312, "federation_enabled": false, "registration_enabled": false } ``` -Room and event counters join it in Phase 3, when there is something to -count. +The three room counters are zero on a server that started without a signing +key, which has no roomserver and so no rooms. ## Errors @@ -134,4 +225,11 @@ configuration is what refuses it. Phase 9 completes the admin API: deactivating users, resetting passwords, revoking devices, granting the admin flag, registration token management, the federation allow/deny list editor, server notices, media quotas and an -audit log of admin actions. Rooms arrive with the roomserver in Phase 3. +audit log of admin actions. + +The room endpoints will grow with it: a room's state and its recent events +for an operator looking at a report, and a way to publish or withdraw a room +from the directory without borrowing a member's token. + +A server started without a signing key has no roomserver, and the four room +endpoints then answer `503` rather than failing on the first call. diff --git a/docs/configuration.md b/docs/configuration.md index b0ccbf8..c4333b0 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -47,6 +47,8 @@ List-valued variables are comma-separated. A single `-` clears a list: | `client_api.rate_limit.default.burst` / `.interval` | — | `30` / `100ms` | Every rate-limited endpoint without its own bucket. | | `client_api.rate_limit.login.burst` / `.interval` | — | `10` / `1s` | `/login` and `/refresh`. | | `client_api.rate_limit.register.burst` / `.interval` | — | `5` / `2s` | `/register` and `/register/available`. | +| `client_api.rate_limit.room_create.burst` / `.interval` | — | `10` / `1s` | `/createRoom`, which writes a dozen events per call. | +| `client_api.rate_limit.join.burst` / `.interval` | — | `10` / `1s` | The two join endpoints. | | `federation.enabled` | `WARP_FEDERATION_ENABLED` | `false` | | | `federation.allowlist` | `WARP_FEDERATION_ALLOWLIST` | `[]` | Non-empty = federate only with these. | | `federation.denylist` | `WARP_FEDERATION_DENYLIST` | `[]` | Always wins over the allowlist. | diff --git a/docs/rooms.md b/docs/rooms.md new file mode 100644 index 0000000..25b94b9 --- /dev/null +++ b/docs/rooms.md @@ -0,0 +1,215 @@ +# Rooms + +`internal/roomserver` and the room half of `internal/storage`: PLAN.md +Phase 3. This is what turns the event core of Phase 2 into rooms a client +can use — locally. Nothing here talks to another server; federation is +Phase 5, and every operation that would need it is refused with a message +that says so rather than a bare `M_FORBIDDEN`. + +## Where the decisions live + +| Question | Answered by | +|---|---| +| Is this event well-formed? | `internal/matrix/event` | +| May this event be in the room? | `internal/matrix/auth` | +| Which of two conflicting states wins? | `internal/matrix/stateres` | +| What are this room's prev_events, depth and auth_events? | `roomserver` | +| Who may call this endpoint? | nobody — see below | +| How is any of it stored? | `storage.RoomStore` | + +The third row is the important one. `internal/matrix` is pure library code +with no storage and no HTTP; the roomserver is what connects those rules to +a database and to a room's history. It is the only thing in Warp that +creates an event. + +The fifth row is not a joke. The Client-Server handlers do no authorisation +of their own: whether Alice may kick Bob, rename the room or send a message +is a question about the room's power levels and membership, and the one +place that is answered correctly is the authorisation rules inside the +pipeline. A handler-level check would be a second, worse copy of them. The +handlers do check the two things the rules cannot see — that the caller is +who their token says, and that a path parameter is a well-formed ID. + +## The input pipeline + +Every event goes through one function, `roomserver.send`, whatever sent it. +The stages are: + +1. **Validate** — the sender is a user of this server (we can only sign as + ourselves), and the content is a JSON object within the size limit. +2. **Authorise** — the event is built against the room's state with exactly + the `auth_events` the selection algorithm asks for, then run through + `auth.CheckAuthEvents` (rule 3) and `auth.Check` (everything else). +3. **Resolve** — the state the event is judged against is the state after + the room's forward extremities, resolved with state resolution v2 when + there is more than one. +4. **Persist** — the event, its state snapshot, the new current state, the + new forward extremity and any transaction ID go down in one transaction. +5. **Emit** — the `Notifier` is told. Phase 3 ships without one; this is the + seam `/sync` plugs into in Phase 4. + +### Why there is a lock per room + +Two events racing into one room would each pick up the same forward +extremities, both become one, and fork the room for no reason — which would +then need state resolution to put back together. A mutex per room instead +keeps every locally created event a child of the one before it, so a room +this server fills on its own always has exactly one forward extremity. The +admin room detail reports the extremities for that reason: more than one +means something unusual happened. + +### Why state resolution is here at all + +With that lock, a Phase 3 room cannot fork, so the resolution path is +unreachable from outside. It is implemented anyway because the alternative +is a pipeline that authorises against one arbitrary branch of a fork +without saying so — and because `stateAfter`, which it needs, is also what +federation will need in Phase 5. + +### Soft failure + +An event can be allowed by the rules against its own `auth_events` and +still fail against the room's *current* state; the spec calls that soft +failure and asks a server to keep the event but not let it into the forward +extremities. There is no soft-failure check here, because for an event +created on this server the two states are the same one. It arrives with +events that come from somewhere else. + +## Room versions + +Versions 3 to 12 all work, and the tests create a room at each of them and +send a message into it. Version 12 is the one that earns the coverage: its +room ID *is* the create event's ID, its create event carries no `room_id` +at all, and the create event is left out of every other event's +`auth_events`. Two consequences worth knowing: + +- `createRoom` has to build the create event before it can name the room, + which is why that one event does not go through `send`. +- The state `auth.Check` is given is the event's `auth_events` **plus the + create event**, because from version 12 the create event is identified by + the room ID rather than cited. The rules still need it: it is what says + who the room's creators are. + +`createRoom` defaults to version 11, which is what the registry says and +why (`internal/matrix/roomversion`): the spec asks servers to stay there for +a while after 1.16 so clients and bridges catch up. + +## Storage + +Events are keyed by event ID rather than by a numeric surrogate. An event ID +appears in `prev_events`, `auth_events`, state snapshots and client requests +alike, so a surrogate would have to be translated at every one of those +boundaries. + +**Stream ordering** is this server's own monotonic position for an event, +handed out by a counter row inside the write transaction. Neither +`AUTOINCREMENT` nor a sequence would do: the two engines spell those +differently, and a PostgreSQL sequence is explicitly not transactional, so a +rolled-back write would leave a hole that `/sync` would have to be taught to +skip. + +**State snapshots** are stored per event and deduplicated by a hash of their +contents. That is what makes a per-event snapshot affordable: every message +in a room whose state is not changing resolves to the snapshot the previous +one already stored, so the table grows with state changes rather than with +traffic. It is also what makes "the state after a forward extremity" +answerable, which state resolution needs. + +**Current state** is kept as its own table, which is derivable from the +events and stored anyway because every authorisation decision reads it. The +`membership` column is a denormalised copy of `content.membership` on member +rows, so "the rooms @alice is joined to" is an index lookup rather than a +scan that parses JSON. + +**The room's listing columns** — name, topic, canonical alias, join rule, +history visibility, guest access, room type and the joined-member count — +live on the room row. The public room directory orders every published room +by member count and pages the result, which cannot be done by reading each +room's state in turn. The roomserver recomputes them when one of the state +events behind them changes, and only then; a room full of chat does not +rewrite its own row on every line. + +## Federation-disabled behaviour + +PLAN.md section 3, enforced here: + +| Operation | While federation is off | +|---|---| +| `/join` a room this server does not have | `403`, naming federation | +| Inviting `@user:other.example` | `403`, naming federation | +| Resolving `#alias:other.example` | `403`, naming federation | +| `publicRooms?server=other.example` | `403`, naming federation | +| Room and event IDs | unchanged — a room created offline federates normally if federation is later turned on | +| Signing | unchanged — every event is signed with the server key | + +The messages are explicit on purpose. `M_FORBIDDEN` alone sends a user to +their operator and the operator hunting through power levels. + +Restricted joins are the interesting case. The rule needs a signature from +the authorising user's server checked, so it normally needs a key fetcher — +but while federation is off every room is local, this server's own key is in +hand, and the rule can be applied in full. A join authorised via a *remote* +user is refused, like everything else that would need another server. + +## Known gaps + +These are deliberate, and each one is somebody else's phase. + +**History visibility is room-level, not per-event.** A read of a room's +state or members is allowed for a joined member, and for anybody at all when +the room is `world_readable`. A user who has *left* a room should see the +state as it was when they left; they are refused instead. Doing it properly +means answering every state read from a historical snapshot and filtering +each event by the visibility in force when it was sent — the same machinery +`/messages` and `/context` need, so it belongs with them in Phase 4 rather +than being built twice. The current behaviour errs towards showing too +little. + +**`GET /rooms/{roomID}/members?at=` is accepted and ignored.** There is one +stream position in this build, because `/sync` has no stream to be at until +Phase 4, so the membership as of any token a client can hold *is* the +current membership. Phase 4 has to honour it. + +**No `/messages`, `/context` or `/sync` of rooms.** Phase 4. Until then a +client can create a room, send into it and read its state, but cannot page +its history. + +**No knocking endpoint.** The `knock` join rule is enforced by the +authorisation rules — a knock-rule room needs an invite to join — but +`POST /knock/{roomIdOrAlias}` is not served. + +**No third-party invites.** They go through an identity server, which Warp +does not talk to (Phase 9). `createRoom` with `invite_3pid`, and `/invite` +with `id_server`, are refused rather than silently dropped. + +**No room upgrades.** `POST /rooms/{roomID}/upgrade` and the tombstone +handling that goes with it are not served. + +**A half-built room can be left behind.** `createRoom` sends a dozen events +through the pipeline one at a time, each authorised against what the ones +before it produced, and a failure part-way through leaves a room that exists +and is incomplete. That is the lesser evil: the alternative is holding every +event of a new room in memory and writing them in one transaction, which +would make the pipeline that authorises them a special case — and the +pipeline is the one thing in here that must not have special cases. Such a +room is visible only to its creator and can be purged. The one failure that +is handled properly is an alias clash, which takes the room row back out, +because it is the one a second client can cause at any time. + +## Tests + +`internal/roomserver` tests run against the real event core and a real +SQLite database, not fakes: what is under test is whether an event this +server signs is one the authorisation rules accept, and a fake of either +would only prove that the pipeline calls what the fake expects. +`internal/storage` tests the room store against both SQL drivers. +`internal/clientapi` and `internal/adminapi` test the HTTP surface through +`httptest`. + +Two things the local-only shape makes untestable here, both covered in +`internal/matrix/auth`: + +- The power-level half of the redaction test. A redaction applies when it + comes from the same server as its target, and every user here is on one + server, so that branch covers every pair of them. +- Soft failure, and state resolution across a real fork. diff --git a/server/internal/adminapi/adminapi.go b/server/internal/adminapi/adminapi.go index 5a99340..8d23bec 100644 --- a/server/internal/adminapi/adminapi.go +++ b/server/internal/adminapi/adminapi.go @@ -22,6 +22,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/buildinfo" "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" "github.com/dariuszjaniak/warp/server/internal/storage" "github.com/dariuszjaniak/warp/server/internal/userapi" @@ -34,6 +35,7 @@ const Prefix = "/_warp/admin/v1" type AdminAPI struct { serverName string users *userapi.API + rooms *roomserver.API settings *settings.Service federation *federation.Gate database storage.Database @@ -44,6 +46,12 @@ type AdminAPI struct { type Options struct { ServerName string Users *userapi.API + + // Rooms is the roomserver component. A server that started without a + // signing key has none, and the room endpoints then report that rather + // than failing on the first call. + Rooms *roomserver.API + Settings *settings.Service Federation *federation.Gate @@ -73,6 +81,7 @@ func New(opts Options) (*AdminAPI, error) { return &AdminAPI{ serverName: opts.ServerName, users: opts.Users, + rooms: opts.Rooms, settings: opts.Settings, federation: opts.Federation, database: opts.Database, @@ -89,6 +98,11 @@ func (a *AdminAPI) RegisterRoutes(r *httputil.Router) { a.handle(r, "GET "+Prefix+"/users", a.handleGetUsers) a.handle(r, "POST "+Prefix+"/users", a.handlePostUsers) a.handle(r, "GET "+Prefix+"/stats", a.handleGetStats) + + a.handle(r, "GET "+Prefix+"/rooms", a.handleGetRooms) + a.handle(r, "GET "+Prefix+"/rooms/{roomID}", a.handleGetRoom) + a.handle(r, "DELETE "+Prefix+"/rooms/{roomID}", a.handleDeleteRoom) + a.handle(r, "POST "+Prefix+"/rooms/{roomID}/force_leave", a.handleForceLeave) } // adminHandler is a handler that has already been shown to be an admin. @@ -288,6 +302,10 @@ type statsResponse struct { Deactivated int `json:"deactivated"` Devices int `json:"devices"` + Rooms int `json:"rooms"` + PublishedRooms int `json:"published_rooms"` + Events int `json:"events"` + FederationEnabled bool `json:"federation_enabled"` RegistrationEnabled bool `json:"registration_enabled"` } @@ -311,6 +329,15 @@ func (a *AdminAPI) handleGetStats(w http.ResponseWriter, r *http.Request, _ user if a.database != nil { resp.Database = a.database.Driver() } + if a.rooms != nil { + roomStats, err := a.rooms.Stats(r.Context()) + if err != nil { + return err + } + resp.Rooms = roomStats.Rooms + resp.PublishedRooms = roomStats.Published + resp.Events = roomStats.Events + } httputil.WriteJSON(w, http.StatusOK, resp) return nil } diff --git a/server/internal/adminapi/adminapi_test.go b/server/internal/adminapi/adminapi_test.go index 2cf9929..44de070 100644 --- a/server/internal/adminapi/adminapi_test.go +++ b/server/internal/adminapi/adminapi_test.go @@ -13,6 +13,8 @@ import ( "github.com/dariuszjaniak/warp/server/internal/adminapi" "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" + "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" "github.com/dariuszjaniak/warp/server/internal/storage" _ "github.com/dariuszjaniak/warp/server/internal/storage/drivers" @@ -26,6 +28,7 @@ type testServer struct { t *testing.T handler http.Handler users *userapi.API + rooms *roomserver.API settings *settings.Service } @@ -61,9 +64,25 @@ func newTestServer(t *testing.T, defaults settings.Defaults) *testServer { t.Fatalf("settings.New: %v", err) } + key, err := signing.Generate("test") + if err != nil { + t.Fatalf("signing.Generate: %v", err) + } + rooms, err := roomserver.New(roomserver.Options{ + ServerName: testServerName, + Store: db.Rooms(), + Profiles: users, + SigningKey: key, + Logger: slog.New(slog.DiscardHandler), + }) + if err != nil { + t.Fatalf("roomserver.New: %v", err) + } + api, err := adminapi.New(adminapi.Options{ ServerName: testServerName, Users: users, + Rooms: rooms, Settings: settingsService, Federation: federation.NewGate(settingsService.FederationEnabled, federation.NewPolicy([]string{"friend.example"}, []string{"spam.example"})), @@ -76,7 +95,9 @@ func newTestServer(t *testing.T, defaults settings.Defaults) *testServer { router := httputil.NewRouter(nil) api.RegisterRoutes(router) - return &testServer{t: t, handler: router, users: users, settings: settingsService} + return &testServer{ + t: t, handler: router, users: users, rooms: rooms, settings: settingsService, + } } func (ts *testServer) do(method, path, token string, body any) *httptest.ResponseRecorder { diff --git a/server/internal/adminapi/rooms.go b/server/internal/adminapi/rooms.go new file mode 100644 index 0000000..ac38596 --- /dev/null +++ b/server/internal/adminapi/rooms.go @@ -0,0 +1,249 @@ +package adminapi + +import ( + "errors" + "net/http" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/storage" + "github.com/dariuszjaniak/warp/server/internal/userapi" +) + +// adminRoom is one row of the room list. Like the user list it is a flat +// shape, because the panel shows it as a table. +type adminRoom struct { + RoomID string `json:"room_id"` + RoomVersion string `json:"room_version"` + Creator string `json:"creator"` + Name string `json:"name,omitempty"` + Topic string `json:"topic,omitempty"` + CanonicalAlias string `json:"canonical_alias,omitempty"` + AvatarURL string `json:"avatar_url,omitempty"` + RoomType string `json:"room_type,omitempty"` + + JoinRule string `json:"join_rule,omitempty"` + HistoryVisibility string `json:"history_visibility,omitempty"` + GuestAccess string `json:"guest_access,omitempty"` + + Published bool `json:"published"` + JoinedMembers int `json:"joined_members"` + CreatedAtMS int64 `json:"created_at_ms"` +} + +type roomsResponse struct { + Rooms []adminRoom `json:"rooms"` + + // Total is the number of rooms matching the query, not the number + // returned, so the panel can page. + Total int `json:"total"` + + Limit int `json:"limit"` + Offset int `json:"offset"` +} + +// defaultRoomPageLimit is what a request without ?limit= gets. +const defaultRoomPageLimit = 50 + +// handleGetRooms implements GET /_warp/admin/v1/rooms. +func (a *AdminAPI) handleGetRooms(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error { + if err := a.requireRooms(); err != nil { + return err + } + query := r.URL.Query() + + limit, err := intParam(query.Get("limit"), defaultRoomPageLimit) + if err != nil { + return httputil.InvalidParam("limit must be a number") + } + offset, err := intParam(query.Get("offset"), 0) + if err != nil { + return httputil.InvalidParam("offset must be a number") + } + if limit < 0 || offset < 0 { + return httputil.InvalidParam("limit and offset must not be negative") + } + + summaries, total, err := a.rooms.ListRooms(r.Context(), storage.RoomFilter{ + Search: query.Get("search"), + Limit: limit, + Offset: offset, + }) + if err != nil { + return err + } + + resp := roomsResponse{ + Rooms: make([]adminRoom, 0, len(summaries)), + Total: total, + Limit: limit, + Offset: offset, + } + for _, summary := range summaries { + resp.Rooms = append(resp.Rooms, adminRoomOf(summary)) + } + httputil.WriteJSON(w, http.StatusOK, resp) + return nil +} + +// roomDetailResponse is GET /_warp/admin/v1/rooms/{roomID}: the list row +// plus what only a count can answer. +type roomDetailResponse struct { + adminRoom + + Members int `json:"members"` + InvitedMembers int `json:"invited_members"` + + Aliases []string `json:"aliases"` + + // ForwardExtremities is where the room's DAG currently ends. There is + // normally exactly one; more than one means the room has forked, which + // is worth an operator seeing. + ForwardExtremities []string `json:"forward_extremities"` +} + +// handleGetRoom implements GET /_warp/admin/v1/rooms/{roomID}. +func (a *AdminAPI) handleGetRoom(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error { + roomID, err := a.pathRoomID(r) + if err != nil { + return err + } + + details, err := a.rooms.RoomDetails(r.Context(), roomID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, roomDetailResponse{ + adminRoom: adminRoomOf(details.RoomSummary), + Members: details.Members, + InvitedMembers: details.InvitedMembers, + Aliases: details.Aliases, + ForwardExtremities: details.ForwardExtremities, + }) + return nil +} + +// handleDeleteRoom implements DELETE /_warp/admin/v1/rooms/{roomID}: the +// purge. +// +// It is a DELETE on the room rather than a POST to an action, because that +// is what it is, and because the method makes a panel's confirm dialog +// mandatory in a way a POST does not. There is no undo: the room's events +// are deleted, and the members are not told, because the only way to tell +// them would have been to add an event to the room that no longer exists. +func (a *AdminAPI) handleDeleteRoom(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := a.pathRoomID(r) + if err != nil { + return err + } + + if err := a.rooms.PurgeRoom(r.Context(), roomID); err != nil { + return roomError(err) + } + a.log.Warn("room purged through the admin API", + "room_id", roomID, "by", auth.UserID) + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// forceLeaveRequest is the POST /rooms/{roomID}/force_leave body. +type forceLeaveRequest struct { + UserID string `json:"user_id"` +} + +// handleForceLeave implements POST /_warp/admin/v1/rooms/{roomID}/force_leave. +// +// It makes a user leave a room without asking the room's rules whether they +// may. A user stuck in a room — one whose power levels are broken, or whose +// last moderator has gone — is what this exists for, and in such a room the +// rules are exactly what is in the way. It is still an ordinary membership +// event, signed and added to the room, so the other members see what +// happened. +func (a *AdminAPI) handleForceLeave(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := a.pathRoomID(r) + if err != nil { + return err + } + var req forceLeaveRequest + if err := httputil.DecodeJSON(w, r, &req); err != nil { + return err + } + if req.UserID == "" { + return httputil.MissingParam("A user ID is required") + } + + if err := a.rooms.ForceLeave(r.Context(), roomID, req.UserID); err != nil { + return roomError(err) + } + a.log.Warn("user removed from a room through the admin API", + "room_id", roomID, "user_id", req.UserID, "by", auth.UserID) + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// adminRoomOf projects a stored room onto the list row. +func adminRoomOf(summary storage.RoomSummary) adminRoom { + return adminRoom{ + RoomID: summary.ID, + RoomVersion: summary.Version, + Creator: summary.Creator, + Name: summary.Name, + Topic: summary.Topic, + CanonicalAlias: summary.CanonicalAlias, + AvatarURL: summary.AvatarURL, + RoomType: summary.RoomType, + JoinRule: summary.JoinRule, + HistoryVisibility: summary.HistoryVisibility, + GuestAccess: summary.GuestAccess, + Published: summary.Published, + JoinedMembers: summary.JoinedMembers, + CreatedAtMS: summary.CreatedAtMS, + } +} + +// pathRoomID reads the {roomID} path value and insists there is a +// roomserver to ask about it. +func (a *AdminAPI) pathRoomID(r *http.Request) (string, error) { + if err := a.requireRooms(); err != nil { + return "", err + } + roomID := r.PathValue("roomID") + if err := id.ValidateRoomID(roomID); err != nil { + return "", httputil.InvalidParam("%s is not a valid room ID", roomID) + } + return roomID, nil +} + +// requireRooms refuses a room endpoint on a server that started without a +// signing key, and so without a roomserver. +func (a *AdminAPI) requireRooms() error { + if a.rooms == nil { + return httputil.Unknown(http.StatusServiceUnavailable, + "This server has no signing key, so it has no rooms") + } + return nil +} + +// roomError maps a roomserver error onto a Matrix error. The admin API has +// a shorter list than the Client-Server one, because an admin endpoint +// never hits the authorisation rules. +func roomError(err error) error { + var matrixErr *httputil.MatrixError + switch { + case err == nil: + return nil + case errors.As(err, &matrixErr): + return err + case errors.Is(err, roomserver.ErrNoSuchRoom): + return httputil.NotFound("Unknown room") + case errors.Is(err, roomserver.ErrNoSuchEvent): + return httputil.NotFound("Unknown event") + case errors.Is(err, roomserver.ErrForbidden), errors.Is(err, roomserver.ErrRejected): + return httputil.Forbidden("%s", err.Error()) + case errors.Is(err, roomserver.ErrInvalidContent): + return httputil.BadJSON("%s", err.Error()) + default: + return err + } +} diff --git a/server/internal/adminapi/rooms_test.go b/server/internal/adminapi/rooms_test.go new file mode 100644 index 0000000..0a61bce --- /dev/null +++ b/server/internal/adminapi/rooms_test.go @@ -0,0 +1,208 @@ +package adminapi_test + +import ( + "net/http" + "net/url" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/adminapi" + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/settings" +) + +// createRoom makes a room through the roomserver directly: the admin API +// has no createRoom, and would be the wrong place for one. +func (ts *testServer) createRoom(creator string, req roomserver.CreateRoomRequest) string { + ts.t.Helper() + req.Creator = "@" + creator + ":" + testServerName + resp, err := ts.rooms.CreateRoom(ts.t.Context(), req) + if err != nil { + ts.t.Fatalf("CreateRoom: %v", err) + } + return resp.RoomID +} + +func TestAdminRoomList(t *testing.T) { + ts := newTestServer(t, settings.Defaults{}) + token := ts.admin("admin") + ts.user("alice") + + roomID := ts.createRoom("alice", roomserver.CreateRoomRequest{ + Name: "Operations", + Preset: roomserver.PresetPublicChat, + Visibility: roomserver.VisibilityPublic, + }) + ts.createRoom("alice", roomserver.CreateRoomRequest{Name: "Quiet"}) + + rec := ts.do(http.MethodGet, adminapi.Prefix+"/rooms", token, nil) + expectStatus(t, rec, http.StatusOK) + body := jsonBody(t, rec) + if total, _ := body["total"].(float64); total != 2 { + t.Errorf("total = %v, want 2", body["total"]) + } + rooms, _ := body["rooms"].([]any) + if len(rooms) != 2 { + t.Fatalf("the list has %d rooms, want 2: %s", len(rooms), rec.Body.String()) + } + + // The search covers the name, which is the column the panel sorts by. + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms?search=operations", token, nil) + expectStatus(t, rec, http.StatusOK) + body = jsonBody(t, rec) + rooms, _ = body["rooms"].([]any) + if len(rooms) != 1 { + t.Fatalf("the search returned %d rooms, want 1", len(rooms)) + } + found, _ := rooms[0].(map[string]any) + if found["room_id"] != roomID { + t.Errorf("the search found %v, want %q", found["room_id"], roomID) + } + if found["published"] != true { + t.Errorf("the published flag is %v, want true", found["published"]) + } + if found["joined_members"] != float64(1) { + t.Errorf("joined_members = %v, want 1", found["joined_members"]) + } + + // An ordinary user is refused, and told why rather than given a 404. + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms", ts.user("bob"), nil) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms?limit=lots", token, nil) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) +} + +func TestAdminRoomDetail(t *testing.T) { + ts := newTestServer(t, settings.Defaults{}) + token := ts.admin("admin") + ts.user("alice") + + roomID := ts.createRoom("alice", roomserver.CreateRoomRequest{ + Name: "Detailed", + Preset: roomserver.PresetPublicChat, + RoomAliasName: "detailed", + }) + + rec := ts.do(http.MethodGet, adminapi.Prefix+"/rooms/"+url.PathEscape(roomID), token, nil) + expectStatus(t, rec, http.StatusOK) + body := jsonBody(t, rec) + + if body["room_id"] != roomID || body["name"] != "Detailed" { + t.Errorf("the detail is %s", rec.Body.String()) + } + if body["creator"] != "@alice:"+testServerName { + t.Errorf("creator = %v", body["creator"]) + } + if body["members"] != float64(1) || body["joined_members"] != float64(1) { + t.Errorf("members = %v, joined = %v", body["members"], body["joined_members"]) + } + aliases, _ := body["aliases"].([]any) + if len(aliases) != 1 { + t.Errorf("aliases = %v, want one", body["aliases"]) + } + // A room filled by one server ends at a single event. + extremities, _ := body["forward_extremities"].([]any) + if len(extremities) != 1 { + t.Errorf("forward_extremities = %v, want one", body["forward_extremities"]) + } + + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms/"+ + url.PathEscape("!nope:"+testServerName), token, nil) + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) + + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms/not-a-room", token, nil) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) +} + +func TestAdminPurgeRoom(t *testing.T) { + ts := newTestServer(t, settings.Defaults{}) + token := ts.admin("admin") + ts.user("alice") + roomID := ts.createRoom("alice", roomserver.CreateRoomRequest{Name: "Doomed"}) + escaped := url.PathEscape(roomID) + + // An ordinary user cannot destroy a room. + rec := ts.do(http.MethodDelete, adminapi.Prefix+"/rooms/"+escaped, ts.user("bob"), nil) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + rec = ts.do(http.MethodDelete, adminapi.Prefix+"/rooms/"+escaped, token, nil) + expectStatus(t, rec, http.StatusOK) + + rec = ts.do(http.MethodGet, adminapi.Prefix+"/rooms/"+escaped, token, nil) + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) + + // Purging twice is a 404, not a silent success: an operator who sees + // an OK has to be able to believe the room was there. + rec = ts.do(http.MethodDelete, adminapi.Prefix+"/rooms/"+escaped, token, nil) + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) +} + +func TestAdminForceLeave(t *testing.T) { + ts := newTestServer(t, settings.Defaults{}) + token := ts.admin("admin") + ts.user("alice") + ts.user("bob") + + roomID := ts.createRoom("alice", roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + escaped := url.PathEscape(roomID) + bobID := "@bob:" + testServerName + + if _, err := ts.rooms.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: roomID, User: bobID, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + + rec := ts.do(http.MethodPost, adminapi.Prefix+"/rooms/"+escaped+"/force_leave", + token, map[string]any{"user_id": bobID}) + expectStatus(t, rec, http.StatusOK) + + membership, err := ts.rooms.Membership(t.Context(), roomID, bobID) + if err != nil { + t.Fatalf("Membership: %v", err) + } + if membership != "leave" { + t.Errorf("bob's membership is %q, want leave", membership) + } + + // A user who is not in the room has nothing to leave. + rec = ts.do(http.MethodPost, adminapi.Prefix+"/rooms/"+escaped+"/force_leave", + token, map[string]any{"user_id": "@carol:" + testServerName}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + rec = ts.do(http.MethodPost, adminapi.Prefix+"/rooms/"+escaped+"/force_leave", + token, map[string]any{}) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeMissingParam) +} + +// TestAdminStatsCountsRooms checks the dashboard counters follow the rooms. +func TestAdminStatsCountsRooms(t *testing.T) { + ts := newTestServer(t, settings.Defaults{}) + token := ts.admin("admin") + ts.user("alice") + + rec := ts.do(http.MethodGet, adminapi.Prefix+"/stats", token, nil) + expectStatus(t, rec, http.StatusOK) + if rooms, _ := jsonBody(t, rec)["rooms"].(float64); rooms != 0 { + t.Errorf("a fresh server reports %v rooms, want 0", rooms) + } + + ts.createRoom("alice", roomserver.CreateRoomRequest{ + Preset: roomserver.PresetPublicChat, + Visibility: roomserver.VisibilityPublic, + }) + + rec = ts.do(http.MethodGet, adminapi.Prefix+"/stats", token, nil) + expectStatus(t, rec, http.StatusOK) + body := jsonBody(t, rec) + if rooms, _ := body["rooms"].(float64); rooms != 1 { + t.Errorf("rooms = %v, want 1", body["rooms"]) + } + if published, _ := body["published_rooms"].(float64); published != 1 { + t.Errorf("published_rooms = %v, want 1", body["published_rooms"]) + } + if events, _ := body["events"].(float64); events == 0 { + t.Error("a server with a room reports no events") + } +} diff --git a/server/internal/clientapi/clientapi.go b/server/internal/clientapi/clientapi.go index 44df818..51b4e5e 100644 --- a/server/internal/clientapi/clientapi.go +++ b/server/internal/clientapi/clientapi.go @@ -21,6 +21,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" "github.com/dariuszjaniak/warp/server/internal/ratelimit" + "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" "github.com/dariuszjaniak/warp/server/internal/userapi" ) @@ -30,6 +31,7 @@ type ClientAPI struct { serverName string clientBaseURL string users *userapi.API + rooms *roomserver.API settings *settings.Service federation *federation.Gate registration config.Registration @@ -41,9 +43,11 @@ type ClientAPI struct { // One limiter per endpoint class, so a client hammering /login cannot // spend the budget of /profile. - limitDefault *ratelimit.Limiter - limitLogin *ratelimit.Limiter - limitRegister *ratelimit.Limiter + limitDefault *ratelimit.Limiter + limitLogin *ratelimit.Limiter + limitRegister *ratelimit.Limiter + limitRoomCreate *ratelimit.Limiter + limitJoin *ratelimit.Limiter } // Options configure the component. @@ -60,6 +64,12 @@ type Options struct { // Users is the userapi component. Users *userapi.API + // Rooms is the roomserver component. Without it the room endpoints are + // not registered at all, which is what the Phase 1 and Phase 2 tests + // relied on and what keeps a half-wired server from offering a + // createRoom it cannot honour. + Rooms *roomserver.API + // Settings is the runtime-settings service; the registration switch is // read from it on every registration attempt. Settings *settings.Service @@ -107,6 +117,7 @@ func New(opts Options) (*ClientAPI, error) { serverName: opts.ServerName, clientBaseURL: opts.ClientBaseURL, users: opts.Users, + rooms: opts.Rooms, settings: opts.Settings, federation: opts.Federation, registration: opts.Registration, @@ -120,9 +131,16 @@ func New(opts Options) (*ClientAPI, error) { c.limitDefault = ratelimit.New(bucket(opts.RateLimit.Default)) c.limitLogin = ratelimit.New(bucket(opts.RateLimit.Login)) c.limitRegister = ratelimit.New(bucket(opts.RateLimit.Register)) + // Creating a room writes a dozen events and joining one writes a + // membership event into a room everybody in it then sees, so both + // get their own, tighter bucket: the default one is sized for + // reads. + c.limitRoomCreate = ratelimit.New(bucket(opts.RateLimit.RoomCreate)) + c.limitJoin = ratelimit.New(bucket(opts.RateLimit.Join)) } else { c.limitDefault, c.limitLogin, c.limitRegister = ratelimit.Disabled(), ratelimit.Disabled(), ratelimit.Disabled() + c.limitRoomCreate, c.limitJoin = ratelimit.Disabled(), ratelimit.Disabled() } return c, nil } @@ -187,6 +205,74 @@ func (c *ClientAPI) RegisterRoutes(r *httputil.Router) { c.limitDefault, c.handlePutRoomAccountData) c.authed(r, "GET "+v3+"/sync", c.limitDefault, c.handleSync) + + c.registerRoomRoutes(r) +} + +// registerRoomRoutes adds the room endpoints, if there is a roomserver. +// +// A build without one answers them with M_UNRECOGNIZED, which is the right +// answer: a server that cannot store a room should say it has no createRoom +// rather than fail every call to it. +func (c *ClientAPI) registerRoomRoutes(r *httputil.Router) { + if c.rooms == nil { + return + } + + c.authed(r, "POST "+v3+"/createRoom", c.limitRoomCreate, c.handleCreateRoom) + + // Membership. The two join endpoints differ only in what they are + // addressed by: /join/{roomIdOrAlias} resolves an alias first. + c.authed(r, "POST "+v3+"/join/{roomIDOrAlias}", c.limitJoin, c.handleJoinRoom) + c.authed(r, "POST "+v3+"/rooms/{roomID}/join", c.limitJoin, c.handleJoinRoomByID) + c.authed(r, "POST "+v3+"/rooms/{roomID}/leave", c.limitDefault, c.handleLeaveRoom) + c.authed(r, "POST "+v3+"/rooms/{roomID}/invite", c.limitDefault, c.handleInvite) + c.authed(r, "POST "+v3+"/rooms/{roomID}/kick", c.limitDefault, c.handleKick) + c.authed(r, "POST "+v3+"/rooms/{roomID}/ban", c.limitDefault, c.handleBan) + c.authed(r, "POST "+v3+"/rooms/{roomID}/unban", c.limitDefault, c.handleUnban) + + // Sending. The state endpoint is registered twice, because the empty + // state key is both valid and the common case and arrives in two + // spellings: ".../state/m.room.name" and ".../state/m.room.name/". + // The trailing-slash form uses a multi-segment wildcard, which is the + // only kind Go's router matches against an empty segment — and which + // also means a state key a client did not escape arrives whole rather + // than as a 404. + c.authed(r, "PUT "+v3+"/rooms/{roomID}/send/{eventType}/{txnID}", + c.limitDefault, c.handleSendEvent) + c.authed(r, "PUT "+v3+"/rooms/{roomID}/state/{eventType}", + c.limitDefault, c.handlePutState) + c.authed(r, "PUT "+v3+"/rooms/{roomID}/state/{eventType}/{stateKey...}", + c.limitDefault, c.handlePutState) + c.authed(r, "PUT "+v3+"/rooms/{roomID}/redact/{eventID}/{txnID}", + c.limitDefault, c.handleRedact) + + // Reading. + c.authed(r, "GET "+v3+"/rooms/{roomID}/event/{eventID}", c.limitDefault, c.handleGetEvent) + c.authed(r, "GET "+v3+"/rooms/{roomID}/state", c.limitDefault, c.handleGetState) + c.authed(r, "GET "+v3+"/rooms/{roomID}/state/{eventType}", + c.limitDefault, c.handleGetStateEvent) + c.authed(r, "GET "+v3+"/rooms/{roomID}/state/{eventType}/{stateKey...}", + c.limitDefault, c.handleGetStateEvent) + c.authed(r, "GET "+v3+"/rooms/{roomID}/members", c.limitDefault, c.handleGetMembers) + c.authed(r, "GET "+v3+"/rooms/{roomID}/joined_members", + c.limitDefault, c.handleGetJoinedMembers) + c.authed(r, "GET "+v3+"/joined_rooms", c.limitDefault, c.handleGetJoinedRooms) + + // The directory. Resolving an alias and reading the public room list + // need no token: a client handed a link has to be able to follow it + // before it decides whether to log in. + c.public(r, "GET "+v3+"/directory/room/{roomAlias}", c.limitDefault, c.handleGetAlias) + c.authed(r, "PUT "+v3+"/directory/room/{roomAlias}", c.limitDefault, c.handlePutAlias) + c.authed(r, "DELETE "+v3+"/directory/room/{roomAlias}", c.limitDefault, c.handleDeleteAlias) + c.authed(r, "GET "+v3+"/rooms/{roomID}/aliases", c.limitDefault, c.handleGetRoomAliases) + + c.public(r, "GET "+v3+"/publicRooms", c.limitDefault, c.handleGetPublicRooms) + c.authed(r, "POST "+v3+"/publicRooms", c.limitDefault, c.handlePostPublicRooms) + c.public(r, "GET "+v3+"/directory/list/room/{roomID}", + c.limitDefault, c.handleGetRoomVisibility) + c.authed(r, "PUT "+v3+"/directory/list/room/{roomID}", + c.limitDefault, c.handlePutRoomVisibility) } // public registers an endpoint that needs no access token. diff --git a/server/internal/clientapi/clientapi_test.go b/server/internal/clientapi/clientapi_test.go index dabb397..e3d41c7 100644 --- a/server/internal/clientapi/clientapi_test.go +++ b/server/internal/clientapi/clientapi_test.go @@ -14,6 +14,8 @@ import ( "github.com/dariuszjaniak/warp/server/internal/config" "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" + "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" "github.com/dariuszjaniak/warp/server/internal/storage" _ "github.com/dariuszjaniak/warp/server/internal/storage/drivers" @@ -31,6 +33,7 @@ type testServer struct { t *testing.T handler http.Handler users *userapi.API + rooms *roomserver.API settings *settings.Service fedGate *federation.Gate } @@ -75,9 +78,28 @@ func newTestServer(t *testing.T, tweak func(*clientapi.Options)) *testServer { } gate := federation.NewGate(settingsService.FederationEnabled, nil) + // A throwaway signing key: the roomserver signs every event it builds, + // and the room endpoints are not registered without one. + key, err := signing.Generate("test") + if err != nil { + t.Fatalf("signing.Generate: %v", err) + } + rooms, err := roomserver.New(roomserver.Options{ + ServerName: testServerName, + Store: db.Rooms(), + Profiles: users, + SigningKey: key, + FederationEnabled: settingsService.FederationEnabled, + Logger: slog.New(slog.DiscardHandler), + }) + if err != nil { + t.Fatalf("roomserver.New: %v", err) + } + opts := clientapi.Options{ ServerName: testServerName, Users: users, + Rooms: rooms, Settings: settingsService, Federation: gate, Logger: slog.New(slog.DiscardHandler), @@ -97,6 +119,7 @@ func newTestServer(t *testing.T, tweak func(*clientapi.Options)) *testServer { t: t, handler: router, users: users, + rooms: rooms, settings: settingsService, fedGate: gate, } diff --git a/server/internal/clientapi/directory.go b/server/internal/clientapi/directory.go new file mode 100644 index 0000000..e3d7909 --- /dev/null +++ b/server/internal/clientapi/directory.go @@ -0,0 +1,211 @@ +package clientapi + +import ( + "net/http" + "strconv" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/userapi" +) + +// aliasResponse is the GET /directory/room/{roomAlias} body. +type aliasResponse struct { + RoomID string `json:"room_id"` + Servers []string `json:"servers"` +} + +// handleGetAlias implements GET /_matrix/client/v3/directory/room/{roomAlias}. +// +// The endpoint needs no token: a client that has been handed a link has to +// be able to turn it into a room ID before it can decide whether to log in, +// and the spec makes it unauthenticated for that reason. +func (c *ClientAPI) handleGetAlias(w http.ResponseWriter, r *http.Request) error { + alias, err := aliasFromPath(r) + if err != nil { + return err + } + resolved, err := c.rooms.ResolveAlias(r.Context(), alias) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, aliasResponse{ + RoomID: resolved.RoomID, + Servers: resolved.Servers, + }) + return nil +} + +// putAliasRequest is the PUT /directory/room/{roomAlias} body. +type putAliasRequest struct { + RoomID string `json:"room_id"` +} + +// handlePutAlias implements PUT /_matrix/client/v3/directory/room/{roomAlias}. +func (c *ClientAPI) handlePutAlias(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + alias, err := aliasFromPath(r) + if err != nil { + return err + } + var req putAliasRequest + if err := httputil.DecodeJSON(w, r, &req); err != nil { + return err + } + if req.RoomID == "" { + return httputil.MissingParam("A room ID is required") + } + + if err := c.rooms.CreateAlias(r.Context(), alias, req.RoomID, auth.UserID); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// handleDeleteAlias implements +// DELETE /_matrix/client/v3/directory/room/{roomAlias}. +func (c *ClientAPI) handleDeleteAlias(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + alias, err := aliasFromPath(r) + if err != nil { + return err + } + if err := c.rooms.DeleteAlias(r.Context(), alias, auth.UserID, auth.Admin); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// roomAliasesResponse is the GET /rooms/{roomID}/aliases body. +type roomAliasesResponse struct { + Aliases []string `json:"aliases"` +} + +// handleGetRoomAliases implements +// GET /_matrix/client/v3/rooms/{roomID}/aliases. +func (c *ClientAPI) handleGetRoomAliases(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + aliases, err := c.rooms.RoomAliases(r.Context(), roomID, auth.UserID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, roomAliasesResponse{Aliases: aliases}) + return nil +} + +// visibilityResponse is the GET /directory/list/room/{roomID} body. +type visibilityResponse struct { + Visibility string `json:"visibility"` +} + +// handleGetRoomVisibility implements +// GET /_matrix/client/v3/directory/list/room/{roomID}. +// +// Whether a room is in the public directory is itself public: the directory +// can be read without a token, so hiding the flag behind one would protect +// nothing. +func (c *ClientAPI) handleGetRoomVisibility(w http.ResponseWriter, r *http.Request) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + visibility, err := c.rooms.RoomVisibility(r.Context(), roomID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, visibilityResponse{Visibility: visibility}) + return nil +} + +// handlePutRoomVisibility implements +// PUT /_matrix/client/v3/directory/list/room/{roomID}. +func (c *ClientAPI) handlePutRoomVisibility(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + var req visibilityResponse + if err := httputil.DecodeJSON(w, r, &req); err != nil { + return err + } + if err := c.rooms.SetRoomVisibility(r.Context(), roomID, auth.UserID, req.Visibility, auth.Admin); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// publicRoomsFilter is the filter in a POST /publicRooms body. +type publicRoomsFilter struct { + GenericSearchTerm string `json:"generic_search_term"` + RoomTypes []string `json:"room_types"` +} + +// publicRoomsRequest is the POST /publicRooms body. +type publicRoomsRequest struct { + Limit int `json:"limit"` + Since string `json:"since"` + Filter publicRoomsFilter `json:"filter"` + + // IncludeAllNetworks and ThirdPartyInstanceID select an application + // service's own directory, which arrives with application service + // support in Phase 9. + IncludeAllNetworks bool `json:"include_all_networks"` + ThirdPartyInstanceID string `json:"third_party_instance_id"` +} + +// handleGetPublicRooms implements GET /_matrix/client/v3/publicRooms. +func (c *ClientAPI) handleGetPublicRooms(w http.ResponseWriter, r *http.Request) error { + query := r.URL.Query() + + limit := 0 + if raw := query.Get("limit"); raw != "" { + parsed, err := strconv.Atoi(raw) + if err != nil || parsed < 0 { + return httputil.InvalidParam("limit must be a non-negative integer") + } + limit = parsed + } + + return c.publicRooms(w, r, roomserver.PublicRoomsRequest{ + Limit: limit, + Since: query.Get("since"), + Server: query.Get("server"), + }) +} + +// handlePostPublicRooms implements POST /_matrix/client/v3/publicRooms, +// which is the same listing with a filter. +func (c *ClientAPI) handlePostPublicRooms(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error { + var req publicRoomsRequest + if _, err := httputil.DecodeOptionalJSON(w, r, &req); err != nil { + return err + } + if req.ThirdPartyInstanceID != "" { + return httputil.Unknown(http.StatusBadRequest, + "Application service room directories are not supported by this server") + } + if req.Limit < 0 { + return httputil.InvalidParam("limit must be a non-negative integer") + } + + return c.publicRooms(w, r, roomserver.PublicRoomsRequest{ + Limit: req.Limit, + Since: req.Since, + Search: req.Filter.GenericSearchTerm, + Server: r.URL.Query().Get("server"), + }) +} + +// publicRooms is the body of both directory endpoints. +func (c *ClientAPI) publicRooms(w http.ResponseWriter, r *http.Request, req roomserver.PublicRoomsRequest) error { + resp, err := c.rooms.PublicRooms(r.Context(), req) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, resp) + return nil +} diff --git a/server/internal/clientapi/membership.go b/server/internal/clientapi/membership.go new file mode 100644 index 0000000..0837e77 --- /dev/null +++ b/server/internal/clientapi/membership.go @@ -0,0 +1,205 @@ +package clientapi + +import ( + "net/http" + "strings" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/userapi" +) + +// joinRequest is the body of both join endpoints. +type joinRequest struct { + Reason string `json:"reason"` + + // ThirdPartySigned is the proof an identity server issued for a + // third-party invite. Warp does not talk to one, so it is refused + // rather than ignored. + ThirdPartySigned map[string]any `json:"third_party_signed"` +} + +type joinResponse struct { + RoomID string `json:"room_id"` +} + +// handleJoinRoomByID implements POST /_matrix/client/v3/rooms/{roomID}/join. +func (c *ClientAPI) handleJoinRoomByID(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + return c.join(w, r, auth, roomID) +} + +// handleJoinRoom implements POST /_matrix/client/v3/join/{roomIDOrAlias}, +// the endpoint a client uses when all it has is an alias. +func (c *ClientAPI) handleJoinRoom(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + target := r.PathValue("roomIDOrAlias") + switch { + case strings.HasPrefix(target, "!"): + if err := id.ValidateRoomID(target); err != nil { + return httputil.InvalidParam("%s is not a valid room ID", target) + } + return c.join(w, r, auth, target) + + case strings.HasPrefix(target, "#"): + resolved, err := c.rooms.ResolveAlias(r.Context(), target) + if err != nil { + return roomError(err) + } + return c.join(w, r, auth, resolved.RoomID) + + default: + return httputil.InvalidParam("%s is neither a room ID nor a room alias", target) + } +} + +// join is the body of both join endpoints. +func (c *ClientAPI) join(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated, roomID string) error { + var req joinRequest + if _, err := httputil.DecodeOptionalJSON(w, r, &req); err != nil { + return err + } + if len(req.ThirdPartySigned) > 0 { + return httputil.Unknown(http.StatusBadRequest, + "Third-party invites are not supported by this server") + } + + joined, err := c.rooms.JoinRoom(r.Context(), roomserver.JoinRequest{ + RoomID: roomID, + User: auth.UserID, + Reason: req.Reason, + ServerNames: r.URL.Query()["server_name"], + }) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, joinResponse{RoomID: joined}) + return nil +} + +// reasonRequest is the body of /leave: a note, at most. +type reasonRequest struct { + Reason string `json:"reason"` +} + +// handleLeaveRoom implements POST /_matrix/client/v3/rooms/{roomID}/leave. +// +// The same endpoint rejects an invite, which is why it works on a room the +// caller has never been joined to. +func (c *ClientAPI) handleLeaveRoom(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + var req reasonRequest + if _, err := httputil.DecodeOptionalJSON(w, r, &req); err != nil { + return err + } + if err := c.rooms.LeaveRoom(r.Context(), roomID, auth.UserID, req.Reason); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// targetRequest is the body of invite, kick, ban and unban. +type targetRequest struct { + UserID string `json:"user_id"` + Reason string `json:"reason"` + + // The identity-server fields of /invite. Their presence is what + // distinguishes a third-party invite from an ordinary one. + IDServer string `json:"id_server"` + Medium string `json:"medium"` + Address string `json:"address"` + IDAccessTok string `json:"id_access_token"` +} + +// handleInvite implements POST /_matrix/client/v3/rooms/{roomID}/invite. +func (c *ClientAPI) handleInvite(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, req, err := c.targetRequest(w, r) + if err != nil { + return err + } + if req.Address != "" || req.Medium != "" || req.IDServer != "" { + return httputil.Unknown(http.StatusBadRequest, + "Third-party invites are not supported by this server") + } + + if err := c.rooms.InviteUser(r.Context(), roomserver.InviteRequest{ + RoomID: roomID, + Sender: auth.UserID, + Target: req.UserID, + Reason: req.Reason, + }); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// handleKick implements POST /_matrix/client/v3/rooms/{roomID}/kick. +func (c *ClientAPI) handleKick(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, req, err := c.targetRequest(w, r) + if err != nil { + return err + } + if req.UserID == "" { + return httputil.MissingParam("A user ID is required") + } + if err := c.rooms.KickUser(r.Context(), roomID, auth.UserID, req.UserID, req.Reason); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// handleBan implements POST /_matrix/client/v3/rooms/{roomID}/ban. +func (c *ClientAPI) handleBan(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, req, err := c.targetRequest(w, r) + if err != nil { + return err + } + if req.UserID == "" { + return httputil.MissingParam("A user ID is required") + } + if err := c.rooms.BanUser(r.Context(), roomID, auth.UserID, req.UserID, req.Reason); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// handleUnban implements POST /_matrix/client/v3/rooms/{roomID}/unban. +func (c *ClientAPI) handleUnban(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, req, err := c.targetRequest(w, r) + if err != nil { + return err + } + if req.UserID == "" { + return httputil.MissingParam("A user ID is required") + } + if err := c.rooms.UnbanUser(r.Context(), roomID, auth.UserID, req.UserID, req.Reason); err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// targetRequest decodes the body the four moderation endpoints share. +func (c *ClientAPI) targetRequest(w http.ResponseWriter, r *http.Request) (string, targetRequest, error) { + roomID, err := pathRoomID(r) + if err != nil { + return "", targetRequest{}, err + } + var req targetRequest + if err := httputil.DecodeJSON(w, r, &req); err != nil { + return "", targetRequest{}, err + } + // A third-party invite has no user_id, which is why the check for one + // is left to the handler. + return roomID, req, nil +} diff --git a/server/internal/clientapi/rooms.go b/server/internal/clientapi/rooms.go new file mode 100644 index 0000000..bf5eb89 --- /dev/null +++ b/server/internal/clientapi/rooms.go @@ -0,0 +1,417 @@ +package clientapi + +import ( + "encoding/json" + "errors" + "net/http" + "strings" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/storage" + "github.com/dariuszjaniak/warp/server/internal/userapi" +) + +// roomError maps a roomserver error onto the Matrix error the spec answers +// it with. +// +// The roomserver's refusals carry their reason in the message — which rule +// said no, which membership was missing — and the message is passed through +// to the client. That is deliberate: "M_FORBIDDEN" on its own sends a user +// to their operator and the operator to the logs, and nothing in these +// messages is anything the caller could not work out by reading the room's +// state, which they are entitled to do. +func roomError(err error) error { + var matrixErr *httputil.MatrixError + switch { + case err == nil: + return nil + case errors.As(err, &matrixErr): + return err + + case errors.Is(err, roomserver.ErrNoSuchRoom): + return httputil.NotFound("Unknown room") + case errors.Is(err, roomserver.ErrNoSuchEvent): + return httputil.NotFound("Unknown event") + case errors.Is(err, roomserver.ErrNoSuchAlias): + return httputil.NotFound("Unknown room alias") + case errors.Is(err, roomserver.ErrAliasExists): + return httputil.NewError(http.StatusConflict, httputil.ErrCodeRoomInUse, + "That room alias is already taken") + + case errors.Is(err, roomserver.ErrUnknownRoomVersion), + errors.Is(err, roomserver.ErrUnsupportedRoomVersion): + return httputil.NewError(http.StatusBadRequest, httputil.ErrCodeUnsupportedVer, "%s", err.Error()) + + case errors.Is(err, roomserver.ErrFederationDisabled): + return httputil.Forbidden("%s", err.Error()) + case errors.Is(err, roomserver.ErrRejected), errors.Is(err, roomserver.ErrForbidden): + return httputil.Forbidden("%s", err.Error()) + case errors.Is(err, roomserver.ErrInvalidContent): + return httputil.BadJSON("%s", err.Error()) + + default: + return err + } +} + +// createRoomRequest is the POST /createRoom body. +type createRoomRequest struct { + RoomVersion string `json:"room_version"` + Preset string `json:"preset"` + Visibility string `json:"visibility"` + RoomAliasName string `json:"room_alias_name"` + Name string `json:"name"` + Topic string `json:"topic"` + Invite []string `json:"invite"` + CreationContent json.RawMessage `json:"creation_content"` + InitialState []roomserver.StateEvent `json:"initial_state"` + PowerLevelContentOverride json.RawMessage `json:"power_level_content_override"` + IsDirect bool `json:"is_direct"` + Invite3PID []map[string]json.RawMessage `json:"invite_3pid"` +} + +type createRoomResponse struct { + RoomID string `json:"room_id"` + RoomAlias string `json:"room_alias,omitempty"` +} + +// handleCreateRoom implements POST /_matrix/client/v3/createRoom. +func (c *ClientAPI) handleCreateRoom(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + var req createRoomRequest + if _, err := httputil.DecodeOptionalJSON(w, r, &req); err != nil { + return err + } + + // Third-party invites go through an identity server, which Warp does + // not talk to (PLAN.md phase 9). Saying so is better than creating the + // room and quietly dropping the invites. + if len(req.Invite3PID) > 0 { + return httputil.Unknown(http.StatusBadRequest, + "Third-party invites are not supported by this server") + } + + resp, err := c.rooms.CreateRoom(r.Context(), roomserver.CreateRoomRequest{ + Creator: auth.UserID, + Version: req.RoomVersion, + Preset: req.Preset, + Visibility: req.Visibility, + RoomAliasName: req.RoomAliasName, + Name: req.Name, + Topic: req.Topic, + Invite: req.Invite, + CreationContent: req.CreationContent, + InitialState: req.InitialState, + PowerLevelContentOverride: req.PowerLevelContentOverride, + IsDirect: req.IsDirect, + }) + if err != nil { + return roomError(err) + } + + httputil.WriteJSON(w, http.StatusOK, createRoomResponse{ + RoomID: resp.RoomID, + RoomAlias: resp.Alias, + }) + return nil +} + +// eventIDResponse is what every endpoint that sends an event returns. +type eventIDResponse struct { + EventID string `json:"event_id"` +} + +// sendEventRequest is the body of a send or state PUT: any JSON object. +type sendEventRequest = json.RawMessage + +// handleSendEvent implements +// PUT /_matrix/client/v3/rooms/{roomID}/send/{eventType}/{txnID}. +func (c *ClientAPI) handleSendEvent(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + eventType := r.PathValue("eventType") + if eventType == "" { + return httputil.MissingParam("An event type is required") + } + txnID := r.PathValue("txnID") + if txnID == "" { + return httputil.MissingParam("A transaction ID is required") + } + + var content sendEventRequest + if err := httputil.DecodeJSON(w, r, &content); err != nil { + return err + } + + eventID, err := c.rooms.SendEvent(r.Context(), roomserver.SendEventRequest{ + RoomID: roomID, + Sender: auth.UserID, + Type: eventType, + Content: content, + Transaction: transactionRef(auth, txnID), + }) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, eventIDResponse{EventID: eventID}) + return nil +} + +// handlePutState implements +// PUT /_matrix/client/v3/rooms/{roomID}/state/{eventType}[/{stateKey}]. +// +// State events carry no transaction ID: the spec gives none, because a +// state event is idempotent in a way a message is not — sending the same +// state twice leaves the room in the state you asked for either way. +func (c *ClientAPI) handlePutState(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + eventType := r.PathValue("eventType") + if eventType == "" { + return httputil.MissingParam("An event type is required") + } + stateKey := r.PathValue("stateKey") + + var content sendEventRequest + if err := httputil.DecodeJSON(w, r, &content); err != nil { + return err + } + + eventID, err := c.rooms.SendEvent(r.Context(), roomserver.SendEventRequest{ + RoomID: roomID, + Sender: auth.UserID, + Type: eventType, + StateKey: &stateKey, + Content: content, + }) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, eventIDResponse{EventID: eventID}) + return nil +} + +// redactRequest is the PUT /rooms/{roomID}/redact/{eventID}/{txnID} body. +type redactRequest struct { + Reason string `json:"reason"` +} + +// handleRedact implements +// PUT /_matrix/client/v3/rooms/{roomID}/redact/{eventID}/{txnID}. +func (c *ClientAPI) handleRedact(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + eventID := r.PathValue("eventID") + if err := id.ValidateEventID(eventID); err != nil { + return httputil.InvalidParam("%s is not a valid event ID", eventID) + } + txnID := r.PathValue("txnID") + if txnID == "" { + return httputil.MissingParam("A transaction ID is required") + } + + var req redactRequest + if _, err := httputil.DecodeOptionalJSON(w, r, &req); err != nil { + return err + } + + redactionID, err := c.rooms.Redact(r.Context(), roomserver.RedactRequest{ + RoomID: roomID, + Sender: auth.UserID, + EventID: eventID, + Reason: req.Reason, + Transaction: transactionRef(auth, txnID), + }) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, eventIDResponse{EventID: redactionID}) + return nil +} + +// handleGetEvent implements +// GET /_matrix/client/v3/rooms/{roomID}/event/{eventID}. +func (c *ClientAPI) handleGetEvent(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + eventID := r.PathValue("eventID") + if err := id.ValidateEventID(eventID); err != nil { + return httputil.InvalidParam("%s is not a valid event ID", eventID) + } + + clientEvent, err := c.rooms.Event(r.Context(), roomID, auth.UserID, eventID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, clientEvent) + return nil +} + +// handleGetState implements GET /_matrix/client/v3/rooms/{roomID}/state, +// which returns the room's whole current state as an array. +func (c *ClientAPI) handleGetState(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + state, err := c.rooms.RoomState(r.Context(), roomID, auth.UserID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, state) + return nil +} + +// handleGetStateEvent implements +// GET /_matrix/client/v3/rooms/{roomID}/state/{eventType}[/{stateKey}]. +// +// It returns the event's content rather than the event: an inconsistency in +// the Client-Server API that predates most of it, and one every client +// relies on. The ?format=event query parameter asks for the whole event +// instead, which is how a client that wants the sender gets it. +func (c *ClientAPI) handleGetStateEvent(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + eventType := r.PathValue("eventType") + if eventType == "" { + return httputil.MissingParam("An event type is required") + } + stateKey := r.PathValue("stateKey") + + if r.URL.Query().Get("format") == "event" { + state, err := c.rooms.StateEvent(r.Context(), roomID, auth.UserID, eventType, stateKey) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, state) + return nil + } + + content, err := c.rooms.StateEventContent(r.Context(), roomID, auth.UserID, eventType, stateKey) + if err != nil { + return roomError(err) + } + httputil.WriteRawJSON(w, http.StatusOK, content) + return nil +} + +// membersResponse is the GET /rooms/{roomID}/members body. +type membersResponse struct { + Chunk []roomserver.ClientEvent `json:"chunk"` +} + +// handleGetMembers implements GET /_matrix/client/v3/rooms/{roomID}/members. +// +// The ?at= parameter asks for the membership as of a point in the event +// stream, and is accepted and ignored. That is not a shortcut: there is one +// stream position in this build, because /sync has no stream to be at until +// Phase 4, so the membership as of any token a client can hold *is* the +// current membership. Phase 4 replaces this endpoint's filtering along with +// /sync and has to honour it then — see docs/rooms.md. +func (c *ClientAPI) handleGetMembers(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + query := r.URL.Query() + + membership, notMembership := query.Get("membership"), query.Get("not_membership") + for _, value := range []string{membership, notMembership} { + if err := roomserver.ValidateMembershipFilter(value); err != nil { + return roomError(err) + } + } + + members, err := c.rooms.Members(r.Context(), roomserver.MembersRequest{ + RoomID: roomID, + User: auth.UserID, + Membership: membership, + NotMembership: notMembership, + }) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, membersResponse{Chunk: members}) + return nil +} + +// joinedMembersResponse is the GET /rooms/{roomID}/joined_members body. +type joinedMembersResponse struct { + Joined map[string]roomserver.RoomMember `json:"joined"` +} + +// handleGetJoinedMembers implements +// GET /_matrix/client/v3/rooms/{roomID}/joined_members. +func (c *ClientAPI) handleGetJoinedMembers(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + roomID, err := pathRoomID(r) + if err != nil { + return err + } + joined, err := c.rooms.JoinedMembers(r.Context(), roomID, auth.UserID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, joinedMembersResponse{Joined: joined}) + return nil +} + +// joinedRoomsResponse is the GET /joined_rooms body. +type joinedRoomsResponse struct { + JoinedRooms []string `json:"joined_rooms"` +} + +// handleGetJoinedRooms implements GET /_matrix/client/v3/joined_rooms. +func (c *ClientAPI) handleGetJoinedRooms(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + rooms, err := c.rooms.JoinedRooms(r.Context(), auth.UserID) + if err != nil { + return roomError(err) + } + httputil.WriteJSON(w, http.StatusOK, joinedRoomsResponse{JoinedRooms: rooms}) + return nil +} + +// pathRoomID reads and validates the {roomID} path value. +func pathRoomID(r *http.Request) (string, error) { + roomID := r.PathValue("roomID") + if err := id.ValidateRoomID(roomID); err != nil { + return "", httputil.InvalidParam("%s is not a valid room ID", roomID) + } + return roomID, nil +} + +// transactionRef builds the idempotency key for a send. +// +// It is keyed on the device rather than the user: the spec scopes a +// transaction ID to the access token, and two of a user's clients each +// counting from "m1" must not collide. +func transactionRef(auth userapi.Authenticated, txnID string) *storage.TransactionRef { + return &storage.TransactionRef{ + Localpart: auth.Localpart, + DeviceID: auth.DeviceID, + TxnID: txnID, + } +} + +// aliasFromPath reads the {roomAlias} path value, which arrives percent- +// encoded because it contains a '#' and a ':'. +func aliasFromPath(r *http.Request) (string, error) { + alias := r.PathValue("roomAlias") + if alias == "" { + return "", httputil.MissingParam("A room alias is required") + } + if !strings.HasPrefix(alias, "#") { + return "", httputil.InvalidParam("%s is not a valid room alias", alias) + } + return alias, nil +} diff --git a/server/internal/clientapi/rooms_test.go b/server/internal/clientapi/rooms_test.go new file mode 100644 index 0000000..53b879a --- /dev/null +++ b/server/internal/clientapi/rooms_test.go @@ -0,0 +1,526 @@ +package clientapi_test + +import ( + "net/http" + "net/http/httptest" + "net/url" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/httputil" +) + +// createRoom calls POST /createRoom and returns the room ID. +func (ts *testServer) createRoom(acc account, body map[string]any) string { + ts.t.Helper() + rec := ts.do(request{ + method: http.MethodPost, + path: "/_matrix/client/v3/createRoom", + body: body, + token: acc.AccessToken, + }) + expectStatus(ts.t, rec, http.StatusOK) + roomID, _ := jsonBody(ts.t, rec)["room_id"].(string) + if roomID == "" { + ts.t.Fatalf("createRoom returned no room ID: %s", rec.Body.String()) + } + return roomID +} + +// put is a PUT with a body, which most of the room endpoints are. +func (ts *testServer) put(path, token string, body any) *httptest.ResponseRecorder { + ts.t.Helper() + return ts.do(request{method: http.MethodPut, path: path, body: body, token: token}) +} + +// post is a POST with a body. +func (ts *testServer) post(path, token string, body any) *httptest.ResponseRecorder { + ts.t.Helper() + return ts.do(request{method: http.MethodPost, path: path, body: body, token: token}) +} + +func TestCreateRoomEndpoint(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + + rec := ts.do(request{ + method: http.MethodPost, + path: "/_matrix/client/v3/createRoom", + body: map[string]any{ + "name": "Test Room", + "topic": "Testing", + "preset": "public_chat", + "visibility": "public", + "room_alias_name": "test", + }, + token: alice.AccessToken, + }) + expectStatus(t, rec, http.StatusOK) + + body := jsonBody(t, rec) + roomID, _ := body["room_id"].(string) + if roomID == "" || roomID[0] != '!' { + t.Fatalf("createRoom returned room_id %q", roomID) + } + if got, _ := body["room_alias"].(string); got != "#test:"+testServerName { + t.Errorf("createRoom returned room_alias %q", got) + } + + // A token is required: a room has to be created by somebody. + rec = ts.do(request{ + method: http.MethodPost, + path: "/_matrix/client/v3/createRoom", + body: map[string]any{}, + }) + expectError(t, rec, http.StatusUnauthorized, httputil.ErrCodeMissingToken) + + // An unknown room version is reported as such, so a client can offer + // the user the versions that do exist. + rec = ts.do(request{ + method: http.MethodPost, + path: "/_matrix/client/v3/createRoom", + body: map[string]any{"room_version": "99"}, + token: alice.AccessToken, + }) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeUnsupportedVer) + + // A taken alias is a conflict, not a bad request. + rec = ts.do(request{ + method: http.MethodPost, + path: "/_matrix/client/v3/createRoom", + body: map[string]any{"room_alias_name": "test"}, + token: alice.AccessToken, + }) + expectError(t, rec, http.StatusConflict, httputil.ErrCodeRoomInUse) +} + +func TestSendAndReadEvents(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat"}) + + rec := ts.put("/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/send/m.room.message/txn1", + alice.AccessToken, map[string]any{"msgtype": "m.text", "body": "hello"}) + expectStatus(t, rec, http.StatusOK) + eventID, _ := jsonBody(t, rec)["event_id"].(string) + if eventID == "" || eventID[0] != '$' { + t.Fatalf("send returned event_id %q", eventID) + } + + // A retry with the same transaction ID returns the first event. + rec = ts.put("/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/send/m.room.message/txn1", + alice.AccessToken, map[string]any{"msgtype": "m.text", "body": "hello"}) + expectStatus(t, rec, http.StatusOK) + if again, _ := jsonBody(t, rec)["event_id"].(string); again != eventID { + t.Errorf("a retried send returned %q, want %q", again, eventID) + } + + rec = ts.get("/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/event/"+ + url.PathEscape(eventID), alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + fetched := jsonBody(t, rec) + if fetched["event_id"] != eventID { + t.Errorf("the event came back as %v", fetched["event_id"]) + } + if fetched["room_id"] != roomID { + t.Errorf("the event's room_id is %v, want %q", fetched["room_id"], roomID) + } + if fetched["sender"] != alice.UserID { + t.Errorf("the event's sender is %v, want %q", fetched["sender"], alice.UserID) + } + // A client is given the ID explicitly and never the signatures: in + // every room version since 3 the ID is a hash, not a property, and the + // signatures are a server's business. + for _, forbidden := range []string{"hashes", "signatures", "auth_events", "prev_events", "depth"} { + if _, present := fetched[forbidden]; present { + t.Errorf("the client event carries %q", forbidden) + } + } + + // An unknown event in a known room is a 404. + rec = ts.get("/_matrix/client/v3/rooms/"+url.PathEscape(roomID)+"/event/%24nope", + alice.AccessToken) + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) + + // A malformed room ID is caught before anything is looked up. + rec = ts.get("/_matrix/client/v3/rooms/not-a-room/state", alice.AccessToken) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) +} + +func TestStateEndpoints(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat", "name": "Original"}) + escaped := url.PathEscape(roomID) + + // The whole state comes back as an array. + rec := ts.get("/_matrix/client/v3/rooms/"+escaped+"/state", alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + state := decode[[]map[string]any](t, rec) + if len(state) < 6 { + t.Fatalf("the room state has %d events, want the full setup: %s", len(state), rec.Body.String()) + } + + // One piece of state comes back as bare content, which is a quirk of + // the Client-Server API that every client relies on. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.name/", alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["name"].(string); got != "Original" { + t.Errorf("the state content is %s", rec.Body.String()) + } + // Without a trailing slash the state key is the empty string too. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.name", alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["name"].(string); got != "Original" { + t.Errorf("the state content without a trailing slash is %s", rec.Body.String()) + } + + // ?format=event asks for the whole event instead. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.name?format=event", + alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["type"].(string); got != "m.room.name" { + t.Errorf("?format=event returned %s", rec.Body.String()) + } + + rec = ts.put("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.topic", + alice.AccessToken, map[string]any{"topic": "Changed"}) + expectStatus(t, rec, http.StatusOK) + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.topic", alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["topic"].(string); got != "Changed" { + t.Errorf("the topic is %s", rec.Body.String()) + } + + // State this room has never set is a 404. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.avatar", alice.AccessToken) + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) + + // A stranger cannot read the state of a room they are not in. + bob := ts.register("bob", "correct-horse-battery-staple") + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state", bob.AccessToken) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + // Nor can they write it. + rec = ts.put("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.topic", + bob.AccessToken, map[string]any{"topic": "Hijacked"}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) +} + +func TestMembershipEndpoints(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + bob := ts.register("bob", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat"}) + escaped := url.PathEscape(roomID) + + rec := ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["room_id"].(string); got != roomID { + t.Errorf("join returned room_id %q, want %q", got, roomID) + } + + rec = ts.get("/_matrix/client/v3/joined_rooms", bob.AccessToken) + expectStatus(t, rec, http.StatusOK) + rooms := decode[struct { + JoinedRooms []string `json:"joined_rooms"` + }](t, rec) + if len(rooms.JoinedRooms) != 1 || rooms.JoinedRooms[0] != roomID { + t.Errorf("bob's joined rooms = %v", rooms.JoinedRooms) + } + + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/joined_members", bob.AccessToken) + expectStatus(t, rec, http.StatusOK) + joined := decode[struct { + Joined map[string]struct { + DisplayName string `json:"display_name"` + } `json:"joined"` + }](t, rec) + if len(joined.Joined) != 2 { + t.Errorf("joined_members returned %d members, want 2", len(joined.Joined)) + } + + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/members", bob.AccessToken) + expectStatus(t, rec, http.StatusOK) + members := decode[struct { + Chunk []map[string]any `json:"chunk"` + }](t, rec) + if len(members.Chunk) != 2 { + t.Errorf("members returned %d events, want 2", len(members.Chunk)) + } + + // An invented membership filter is refused rather than ignored. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/members?membership=lurking", + bob.AccessToken) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeBadJSON) + + // ?at= is accepted and ignored: there is one stream position in this + // build, so the membership as of any token a client can hold is the + // current membership. Phase 4 has to honour it properly. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/members?at=s0_0", bob.AccessToken) + expectStatus(t, rec, http.StatusOK) + + // Moderation. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/kick", alice.AccessToken, + map[string]any{"user_id": bob.UserID, "reason": "behave"}) + expectStatus(t, rec, http.StatusOK) + + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/ban", alice.AccessToken, + map[string]any{"user_id": bob.UserID}) + expectStatus(t, rec, http.StatusOK) + // A banned user cannot come back. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/unban", alice.AccessToken, + map[string]any{"user_id": bob.UserID}) + expectStatus(t, rec, http.StatusOK) + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/leave", bob.AccessToken, nil) + expectStatus(t, rec, http.StatusOK) + + // user_id is required where it is the whole point of the call. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/kick", alice.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeMissingParam) + + // Moderation without the power is M_FORBIDDEN, decided by the room's + // power levels rather than by this handler. + carol := ts.register("carol", "correct-horse-battery-staple") + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", carol.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/kick", carol.AccessToken, + map[string]any{"user_id": alice.UserID}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) +} + +func TestInviteEndpoint(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + bob := ts.register("bob", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "private_chat"}) + escaped := url.PathEscape(roomID) + + // An invite-only room cannot be walked into. + rec := ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/invite", alice.AccessToken, + map[string]any{"user_id": bob.UserID}) + expectStatus(t, rec, http.StatusOK) + + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + + // Inviting a user of another server needs federation, and says so. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/invite", alice.AccessToken, + map[string]any{"user_id": "@eve:elsewhere.example"}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + if msg, _ := jsonBody(t, rec)["error"].(string); msg == "" { + t.Error("the federation refusal carried no explanation") + } + + // A third-party invite goes through an identity server, which this + // server does not talk to. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/invite", alice.AccessToken, + map[string]any{"id_server": "id.example", "medium": "email", "address": "a@b.example"}) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeUnknown) +} + +func TestRedactEndpoint(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat"}) + escaped := url.PathEscape(roomID) + + rec := ts.put("/_matrix/client/v3/rooms/"+escaped+"/send/m.room.message/t1", + alice.AccessToken, map[string]any{"msgtype": "m.text", "body": "oops"}) + expectStatus(t, rec, http.StatusOK) + eventID, _ := jsonBody(t, rec)["event_id"].(string) + + rec = ts.put("/_matrix/client/v3/rooms/"+escaped+"/redact/"+ + url.PathEscape(eventID)+"/r1", alice.AccessToken, map[string]any{"reason": "a mistake"}) + expectStatus(t, rec, http.StatusOK) + redaction, _ := jsonBody(t, rec)["event_id"].(string) + if redaction == "" || redaction == eventID { + t.Fatalf("redact returned event_id %q", redaction) + } + + // The event is readable and empty, and says why. + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/event/"+url.PathEscape(eventID), + alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + fetched := jsonBody(t, rec) + content, _ := fetched["content"].(map[string]any) + if len(content) != 0 { + t.Errorf("the redacted event still has content: %v", content) + } + unsigned, _ := fetched["unsigned"].(map[string]any) + because, _ := unsigned["redacted_because"].(map[string]any) + if because["event_id"] != redaction { + t.Errorf("unsigned.redacted_because = %v, want the redaction %q", because, redaction) + } + + // A malformed event ID in the path is caught before anything else. + rec = ts.put("/_matrix/client/v3/rooms/"+escaped+"/redact/nope/r2", + alice.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) +} + +func TestDirectoryEndpoints(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + bob := ts.register("bob", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat"}) + escaped := url.PathEscape(roomID) + + const alias = "#chat:" + testServerName + escapedAlias := url.PathEscape(alias) + + rec := ts.put("/_matrix/client/v3/directory/room/"+escapedAlias, alice.AccessToken, + map[string]any{"room_id": roomID}) + expectStatus(t, rec, http.StatusOK) + + // Resolving an alias needs no token: a client handed a link has to be + // able to follow it before it decides whether to log in. + rec = ts.get("/_matrix/client/v3/directory/room/"+escapedAlias, "") + expectStatus(t, rec, http.StatusOK) + resolved := jsonBody(t, rec) + if resolved["room_id"] != roomID { + t.Errorf("the alias resolves to %v, want %q", resolved["room_id"], roomID) + } + + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/aliases", alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + aliases := decode[struct { + Aliases []string `json:"aliases"` + }](t, rec) + if len(aliases.Aliases) != 1 || aliases.Aliases[0] != alias { + t.Errorf("the room's aliases = %v", aliases.Aliases) + } + + // An unknown alias is a 404. + rec = ts.get("/_matrix/client/v3/directory/room/"+url.PathEscape("#nope:"+testServerName), "") + expectError(t, rec, http.StatusNotFound, httputil.ErrCodeNotFound) + + // The public room directory. + rec = ts.get("/_matrix/client/v3/publicRooms", "") + expectStatus(t, rec, http.StatusOK) + if total, _ := jsonBody(t, rec)["total_room_count_estimate"].(float64); total != 0 { + t.Errorf("an unpublished room is in the directory (total %v)", total) + } + + rec = ts.put("/_matrix/client/v3/directory/list/room/"+escaped, alice.AccessToken, + map[string]any{"visibility": "public"}) + expectStatus(t, rec, http.StatusOK) + + rec = ts.get("/_matrix/client/v3/directory/list/room/"+escaped, "") + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["visibility"].(string); got != "public" { + t.Errorf("visibility = %q, want public", got) + } + + rec = ts.get("/_matrix/client/v3/publicRooms?limit=10", "") + expectStatus(t, rec, http.StatusOK) + directory := decode[struct { + Chunk []map[string]any `json:"chunk"` + Total int `json:"total_room_count_estimate"` + }](t, rec) + if directory.Total != 1 || len(directory.Chunk) != 1 { + t.Fatalf("the directory has %d of %d rooms, want 1 of 1", + len(directory.Chunk), directory.Total) + } + if directory.Chunk[0]["room_id"] != roomID { + t.Errorf("the directory lists %v, want %q", directory.Chunk[0]["room_id"], roomID) + } + + // The POST form takes a filter. + rec = ts.post("/_matrix/client/v3/publicRooms", alice.AccessToken, map[string]any{ + "filter": map[string]any{"generic_search_term": "nothing matches this"}, + }) + expectStatus(t, rec, http.StatusOK) + filtered := decode[struct { + Chunk []map[string]any `json:"chunk"` + }](t, rec) + if len(filtered.Chunk) != 0 { + t.Errorf("a search that matches nothing returned %d rooms", len(filtered.Chunk)) + } + + // An ordinary member cannot publish or unpublish the room. + rec = ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + rec = ts.put("/_matrix/client/v3/directory/list/room/"+escaped, bob.AccessToken, + map[string]any{"visibility": "private"}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + // Another server's directory needs federation. + rec = ts.get("/_matrix/client/v3/publicRooms?server=other.example", "") + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) + + // A bad limit is a bad request rather than a default. + rec = ts.get("/_matrix/client/v3/publicRooms?limit=lots", "") + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) +} + +// TestJoinByAlias is the endpoint a client uses when all it was given is a +// link. +func TestJoinByAlias(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + bob := ts.register("bob", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{ + "preset": "public_chat", "room_alias_name": "open", + }) + + alias := "#open:" + testServerName + rec := ts.post("/_matrix/client/v3/join/"+url.PathEscape(alias), bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["room_id"].(string); got != roomID { + t.Errorf("joining by alias returned %q, want %q", got, roomID) + } + + // The same endpoint takes a room ID. + rec = ts.post("/_matrix/client/v3/join/"+url.PathEscape(roomID), bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + + // Anything that is neither is a bad request. + rec = ts.post("/_matrix/client/v3/join/nonsense", bob.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusBadRequest, httputil.ErrCodeInvalidParam) + + // A room on another server needs federation. + rec = ts.post("/_matrix/client/v3/join/"+url.PathEscape("!x:other.example"), + bob.AccessToken, map[string]any{}) + expectError(t, rec, http.StatusForbidden, httputil.ErrCodeForbidden) +} + +// TestStateKeyWithSpecialCharacters is about the router, not the rooms: a +// membership state key is a user ID, so it carries an '@' and a ':', and +// the empty state key arrives in two spellings. All of them have to route. +func TestStateKeyWithSpecialCharacters(t *testing.T) { + ts := newTestServer(t, nil) + alice := ts.register("alice", "correct-horse-battery-staple") + bob := ts.register("bob", "correct-horse-battery-staple") + roomID := ts.createRoom(alice, map[string]any{"preset": "public_chat"}) + escaped := url.PathEscape(roomID) + + rec := ts.post("/_matrix/client/v3/rooms/"+escaped+"/join", bob.AccessToken, map[string]any{}) + expectStatus(t, rec, http.StatusOK) + + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.member/"+ + url.PathEscape(bob.UserID), alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["membership"].(string); got != "join" { + t.Errorf("bob's membership content is %s", rec.Body.String()) + } + + // And writing one back through the state endpoint. + rec = ts.put("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.member/"+ + url.PathEscape(bob.UserID), alice.AccessToken, + map[string]any{"membership": "ban"}) + expectStatus(t, rec, http.StatusOK) + rec = ts.get("/_matrix/client/v3/rooms/"+escaped+"/state/m.room.member/"+ + url.PathEscape(bob.UserID), alice.AccessToken) + expectStatus(t, rec, http.StatusOK) + if got, _ := jsonBody(t, rec)["membership"].(string); got != "ban" { + t.Errorf("bob's membership after the ban is %s", rec.Body.String()) + } +} diff --git a/server/internal/clientapi/sync.go b/server/internal/clientapi/sync.go index 55ecd9f..1e88e5e 100644 --- a/server/internal/clientapi/sync.go +++ b/server/internal/clientapi/sync.go @@ -7,8 +7,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/userapi" ) -// syncResponse is the subset of the /sync body that is meaningful before -// there are any rooms. +// syncResponse is the subset of the /sync body this build can fill in. type syncResponse struct { NextBatch string `json:"next_batch"` Rooms syncRooms `json:"rooms"` @@ -34,27 +33,29 @@ type syncDeviceLists struct { Left []string `json:"left"` } -// syncPlaceholderToken is the only stream position Phase 1 can be at. +// syncPlaceholderToken is the only stream position this build has. // // It is a real token shape rather than an empty string so that a client -// storing it and sending it back as ?since= gets the same answer, which is -// the correct one: nothing has happened. +// storing it and sending it back as ?since= gets the same answer. const syncPlaceholderToken = "s0_0" -// handleSync implements GET /_matrix/client/v3/sync as far as Phase 1 can. +// handleSync implements GET /_matrix/client/v3/sync as far as this build +// can. // -// This is a placeholder, and deliberately a working one. There are no rooms -// until Phase 3 and no event streams until Phase 4, so an empty sync is not -// a lie — it is the complete state of this server. What it buys in the -// meantime is the thing every client and the test suite check first: that a -// valid token syncs and a revoked one does not. Without it, "logging out -// invalidates the session" cannot be observed at all. +// It is a placeholder, and since Phase 3 it is an incomplete answer rather +// than a complete one: there are rooms now, and this reports none of them. +// Filling it in means the event stream — stream tokens, incremental deltas, +// long polling, filters and lazy-loaded members — which is the syncapi of +// PLAN.md phase 4, and which replaces this wholesale. // -// Phase 4 replaces this wholesale with the syncapi: stream tokens, long -// polling, filters and lazy-loaded members. +// It stays because of what it does buy: a valid token syncs and a revoked +// one does not, which is the first thing every client and the test suite +// check, and which cannot be observed at all without an endpoint here. A +// client can meanwhile see its rooms through /joined_rooms and read them +// through /rooms/{roomID}/state. See docs/rooms.md. func (c *ClientAPI) handleSync(w http.ResponseWriter, _ *http.Request, _ userapi.Authenticated) error { // The `timeout` parameter is ignored rather than honoured: blocking for - // 30 seconds to return a response that cannot change would hold a + // 30 seconds to return a response this build cannot change would hold a // connection open for nothing. httputil.WriteJSON(w, http.StatusOK, syncResponse{ NextBatch: syncPlaceholderToken, diff --git a/server/internal/config/config.go b/server/internal/config/config.go index ae46413..e89209e 100644 --- a/server/internal/config/config.go +++ b/server/internal/config/config.go @@ -135,6 +135,14 @@ type RateLimit struct { // Register covers /register and /register/available. Register RateLimitBucket `yaml:"register"` + + // RoomCreate covers /createRoom, which writes a dozen events per call. + RoomCreate RateLimitBucket `yaml:"room_create"` + + // Join covers the two join endpoints. A join writes a membership event + // into a room that everybody in it then sees, so it is worth a tighter + // bucket than a read. + Join RateLimitBucket `yaml:"join"` } // RateLimitBucket is one token bucket: Burst requests may arrive back to @@ -240,6 +248,11 @@ func Default() *Config { Default: RateLimitBucket{Burst: 30, Interval: 100 * time.Millisecond}, Login: RateLimitBucket{Burst: 10, Interval: time.Second}, Register: RateLimitBucket{Burst: 5, Interval: 2 * time.Second}, + // Room creation and joining are deliberately slow. Both + // are rare for a real client and expensive for the server, + // and a client that hits either limit is a script. + RoomCreate: RateLimitBucket{Burst: 10, Interval: time.Second}, + Join: RateLimitBucket{Burst: 10, Interval: time.Second}, }, }, Federation: Federation{ @@ -344,6 +357,8 @@ func (c *Config) Validate() error { {"default", c.ClientAPI.RateLimit.Default}, {"login", c.ClientAPI.RateLimit.Login}, {"register", c.ClientAPI.RateLimit.Register}, + {"room_create", c.ClientAPI.RateLimit.RoomCreate}, + {"join", c.ClientAPI.RateLimit.Join}, } { if b.bucket.Burst < 1 { bad("client_api.rate_limit.%s.burst: must be at least 1", b.name) diff --git a/server/internal/httputil/errors.go b/server/internal/httputil/errors.go index dfdb8f5..95c8193 100644 --- a/server/internal/httputil/errors.go +++ b/server/internal/httputil/errors.go @@ -26,6 +26,9 @@ const ( ErrCodeTooLarge = "M_TOO_LARGE" ErrCodeUnsupportedVer = "M_UNSUPPORTED_ROOM_VERSION" + // Room codes, added with the roomserver in Phase 3. + ErrCodeRoomInUse = "M_ROOM_IN_USE" + // Account and session codes, added with the userapi in Phase 1. ErrCodeUserInUse = "M_USER_IN_USE" ErrCodeInvalidUsername = "M_INVALID_USERNAME" diff --git a/server/internal/httputil/json.go b/server/internal/httputil/json.go index d58a851..4735ef8 100644 --- a/server/internal/httputil/json.go +++ b/server/internal/httputil/json.go @@ -47,6 +47,25 @@ func WriteJSON(w http.ResponseWriter, status int, v any) { } } +// WriteRawJSON writes JSON that is already encoded. +// +// It exists for the one endpoint whose response body *is* a stored value: +// GET /rooms/{roomID}/state/{eventType}/{stateKey} answers with the state +// event's content and nothing else. Re-encoding it through a map would +// reorder its keys and rewrite its numbers, and a client comparing the +// content it set with the content it reads back would see a difference that +// is not there. +func WriteRawJSON(w http.ResponseWriter, status int, body []byte) { + if len(body) == 0 { + body = []byte("{}") + } + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + if _, err := w.Write(body); err != nil { + slog.Debug("writing response failed", "error", err) + } +} + // WriteError sends err as a Matrix error envelope. Anything that is not a // *MatrixError is logged in full and reported as M_UNKNOWN, so internal // details never leak to the client. diff --git a/server/internal/roomserver/admin.go b/server/internal/roomserver/admin.go new file mode 100644 index 0000000..230c064 --- /dev/null +++ b/server/internal/roomserver/admin.go @@ -0,0 +1,105 @@ +package roomserver + +import ( + "context" + + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// RoomDetails is a room as the admin panel shows it: the stored summary +// plus the things only a count can answer. +type RoomDetails struct { + storage.RoomSummary + + // Members counts every m.room.member event in the current state, + // whatever the membership — a room with one member and forty bans is + // not the same room as one with one member. + Members int + + // InvitedMembers counts the outstanding invites. + InvitedMembers int + + // Aliases are the local aliases pointing at the room. + Aliases []string + + // ForwardExtremities are the event IDs the room's DAG currently ends + // at. There is normally one; more than one means the room has forked, + // which is worth seeing on an operator's screen. + ForwardExtremities []string +} + +// ListRooms returns a page of every room on this server, for the admin API. +func (r *API) ListRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) { + return r.store.ListRooms(ctx, filter) +} + +// RoomDetails reads one room in full, for the admin API. +func (r *API) RoomDetails(ctx context.Context, roomID string) (RoomDetails, error) { + summary, err := r.store.RoomSummary(ctx, roomID) + if err != nil { + return RoomDetails{}, err + } + + memberships, err := r.store.MembershipEntries(ctx, roomID, nil) + if err != nil { + return RoomDetails{}, err + } + details := RoomDetails{RoomSummary: summary, Members: len(memberships)} + for _, m := range memberships { + if m.Membership == event.MembershipInvite { + details.InvitedMembers++ + } + } + + if details.Aliases, err = r.store.AliasesForRoom(ctx, roomID); err != nil { + return RoomDetails{}, err + } + if details.Aliases == nil { + details.Aliases = []string{} + } + + extremities, err := r.store.Extremities(ctx, roomID) + if err != nil { + return RoomDetails{}, err + } + details.ForwardExtremities = make([]string, 0, len(extremities)) + for _, e := range extremities { + details.ForwardExtremities = append(details.ForwardExtremities, e.EventID) + } + return details, nil +} + +// PurgeRoom deletes a room and everything belonging to it. +// +// This is the one operation in the roomserver that destroys history, and it +// is irreversible: the events are gone, not tombstoned. It exists because a +// room can become something an operator has to be able to remove — abuse, a +// room nobody is in any more, a room whose version this build no longer +// supports — and because the alternative an operator reaches for otherwise +// is deleting rows by hand. +// +// It does not tell the room's members. Nothing can: the room's events are +// what a membership change would have to be added to. +func (r *API) PurgeRoom(ctx context.Context, roomID string) error { + // The lock is taken so that a purge cannot race a send into the same + // room and leave the event pointing at a room row that is gone. + mu := r.lock(roomID) + mu.Lock() + defer mu.Unlock() + + if err := r.store.PurgeRoom(ctx, roomID); err != nil { + return err + } + // The lock itself is dropped: a purged room ID will not come back, and + // keeping its mutex would make the map grow with every purge. + r.locks.Delete(roomID) + + r.log.Warn("purged room", "room_id", roomID) + return nil +} + +// Stats returns the aggregate room counts for the admin dashboard. +func (r *API) Stats(ctx context.Context) (storage.RoomStats, error) { + return r.store.RoomStats(ctx) +} diff --git a/server/internal/roomserver/create.go b/server/internal/roomserver/create.go new file mode 100644 index 0000000..e85c65a --- /dev/null +++ b/server/internal/roomserver/create.go @@ -0,0 +1,540 @@ +package roomserver + +import ( + "context" + "crypto/rand" + "encoding/base64" + "encoding/json" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// The createRoom presets, which are shorthand for a join rule, a history +// visibility and a guest access policy together. +const ( + PresetPrivateChat = "private_chat" + PresetTrustedPrivateChat = "trusted_private_chat" + PresetPublicChat = "public_chat" +) + +// Directory visibility values for createRoom and +// /directory/list/room/{roomID}. +const ( + VisibilityPublic = "public" + VisibilityPrivate = "private" +) + +// CreateRoomRequest is the createRoom body, already parsed. +type CreateRoomRequest struct { + // Creator is the full user ID creating the room. + Creator string + + // Version is the room version to create at; empty uses this server's + // default. + Version string + + // Preset is one of the three presets; empty derives it from Visibility + // the way the spec does. + Preset string + + // Visibility is "public" to publish the room in this server's directory. + Visibility string + + // RoomAliasName is the localpart of an alias to create and set as the + // canonical alias. + RoomAliasName string + + Name string + Topic string + + // Invite lists user IDs to invite once the room exists. + Invite []string + + // CreationContent is merged into the m.room.create content. It is how a + // client sets m.federate and creates a space. + CreationContent json.RawMessage + + // InitialState are state events to send after the room's own setup + // events and before the invites. + InitialState []StateEvent + + // PowerLevelContentOverride is merged over the m.room.power_levels + // content this server would otherwise send. + PowerLevelContentOverride json.RawMessage + + // IsDirect marks the creator's and the invitees' membership events as a + // direct chat. + IsDirect bool +} + +// StateEvent is one state event in createRoom's initial_state. +type StateEvent struct { + Type string `json:"type"` + StateKey string `json:"state_key"` + Content json.RawMessage `json:"content"` +} + +// CreateRoomResponse is what the client is told. +type CreateRoomResponse struct { + RoomID string + + // Alias is the full alias created from RoomAliasName, or empty. + Alias string +} + +// CreateRoom builds a room from nothing. +// +// The order of the events is the spec's order, and it is not arbitrary: each +// one is authorised against the state the ones before it produced. The +// create event comes first because nothing can be authorised without it; the +// creator's join comes next because every later event needs a sender who is +// in the room; the power levels come before the join rules so that the +// creator still has the power to set them. +// +// A failure part-way through leaves a room that exists and is incomplete. +// That is the lesser evil: the alternative is holding every event of a new +// room in memory and writing them in one transaction, which would make the +// pipeline that authorises them a special case, and the pipeline is the one +// thing in here that must not have special cases. A half-built room is +// visible only to its creator and can be purged. +func (r *API) CreateRoom(ctx context.Context, req CreateRoomRequest) (CreateRoomResponse, error) { + if !r.isLocal(req.Creator) { + return CreateRoomResponse{}, fmt.Errorf("%w: %s is not a user of this server", ErrForbidden, req.Creator) + } + + version := r.defaultRoomVersion + if req.Version != "" { + if err := roomversion.Validate(req.Version); err != nil { + return CreateRoomResponse{}, fmt.Errorf("%w: %s", ErrUnknownRoomVersion, err) + } + version = roomversion.Version(req.Version) + } + rules, err := roomversion.Get(version) + if err != nil { + return CreateRoomResponse{}, err + } + + preset, err := presetFor(req) + if err != nil { + return CreateRoomResponse{}, err + } + + var alias string + if req.RoomAliasName != "" { + if alias, err = r.aliasFor(req.RoomAliasName); err != nil { + return CreateRoomResponse{}, err + } + } + + roomID, createEvent, err := r.buildCreateEvent(ctx, req, rules) + if err != nil { + return CreateRoomResponse{}, err + } + + if err := r.store.CreateRoom(ctx, storage.NewRoom{ + ID: roomID, + Version: string(version), + Creator: req.Creator, + Published: req.Visibility == VisibilityPublic, + CreatedAtMS: r.nowMS(), + }); err != nil { + return CreateRoomResponse{}, err + } + + // The alias is claimed before any event is signed, so that a name + // already taken does not leave a room behind. It cannot be claimed + // before the room row, because an alias points at a room; so a clash + // takes the room row back out, which is safe precisely because nothing + // has been written into the room yet. + if alias != "" { + if err := r.store.CreateAlias(ctx, storage.RoomAlias{ + Alias: alias, + RoomID: roomID, + Creator: req.Creator, + CreatedAtMS: r.nowMS(), + }); err != nil { + if purgeErr := r.store.PurgeRoom(ctx, roomID); purgeErr != nil { + r.log.Error("could not remove the room an alias clash abandoned", + "room_id", roomID, "error", purgeErr) + } + return CreateRoomResponse{}, err + } + } + + // The create event is the one event that cannot go through send: there + // is no room state to read and no extremity to descend from, and in + // room version 12 the room ID is its own event ID, so it has to exist + // before the room does. + if err := r.persistCreateEvent(ctx, roomID, string(version), rules, createEvent); err != nil { + return CreateRoomResponse{}, err + } + + for _, step := range r.creationEvents(ctx, req, preset, rules, roomID) { + if _, err := r.send(ctx, step); err != nil { + return CreateRoomResponse{}, fmt.Errorf("creating %s: setting up %s: %w", roomID, step.Type, err) + } + } + + if alias != "" { + if _, err := r.send(ctx, sendRequest{ + RoomID: roomID, + Type: TypeCanonicalAlias, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{"alias": alias}), + }); err != nil { + return CreateRoomResponse{}, fmt.Errorf("creating %s: setting the canonical alias: %w", roomID, err) + } + } + + // Invites come last, so that an invited user who looks immediately + // finds a room that is fully set up. + for _, invitee := range req.Invite { + if err := r.InviteUser(ctx, InviteRequest{ + RoomID: roomID, + Sender: req.Creator, + Target: invitee, + IsDirect: req.IsDirect, + }); err != nil { + return CreateRoomResponse{}, fmt.Errorf("creating %s: inviting %s: %w", roomID, invitee, err) + } + } + + r.log.Info("created room", + "room_id", roomID, "version", version, "creator", req.Creator, "preset", preset) + return CreateRoomResponse{RoomID: roomID, Alias: alias}, nil +} + +// buildCreateEvent builds the m.room.create event and works out the room ID. +// +// From room version 12 the room ID *is* the create event's ID, so the event +// has to be built before the room can be named; in earlier versions the ID +// is minted here and written into the event. The event is checked against +// the authorisation rules with an empty state, which is the one case rule 1 +// covers. +func (r *API) buildCreateEvent(ctx context.Context, req CreateRoomRequest, rules roomversion.Rules) (string, *event.Event, error) { + content, err := r.createContent(req, rules) + if err != nil { + return "", nil, err + } + + roomID := "" + if !rules.RoomIDIsCreateEventID { + if roomID, err = r.generateRoomID(); err != nil { + return "", nil, err + } + } + + builder := event.Builder{ + RoomID: roomID, + Type: event.TypeCreate, + StateKey: statePtr(""), + Sender: req.Creator, + Content: content, + Depth: 1, + } + created, err := builder.Build(r.now(), r.signingKey, rules) + if err != nil { + return "", nil, fmt.Errorf("building the create event: %w", err) + } + + if rules.RoomIDIsCreateEventID { + if roomID, err = event.RoomIDOfCreateEvent(created); err != nil { + return "", nil, err + } + } + + // Rule 1 against an empty state: the version is one we support, the + // creator is well-formed, and in version 12 the room ID matches. + if err := auth.Check(ctx, created, auth.State{}, nil); err != nil { + return "", nil, err + } + return roomID, created, nil +} + +// createContent assembles the m.room.create content. +// +// creation_content from the client is merged in underneath, so a client can +// set m.federate or make a space without being able to overwrite the +// properties the room version defines — room_version and creator are put on +// last for that reason. +func (r *API) createContent(req CreateRoomRequest, rules roomversion.Rules) (json.RawMessage, error) { + content := map[string]json.RawMessage{} + if len(req.CreationContent) > 0 { + if err := json.Unmarshal(req.CreationContent, &content); err != nil { + return nil, fmt.Errorf("%w: creation_content must be a JSON object", ErrInvalidContent) + } + // These are the server's to decide, whatever the client sent. + delete(content, "room_version") + delete(content, "creator") + delete(content, "additional_creators") + } + content["room_version"] = mustMarshal(string(rules.Version)) + if rules.CreateEventRequiresCreator { + content["creator"] = mustMarshal(req.Creator) + } + + raw, err := json.Marshal(content) + if err != nil { + return nil, fmt.Errorf("%w: creation_content cannot be encoded: %v", ErrInvalidContent, err) + } + return raw, nil +} + +// persistCreateEvent writes the create event, which is the room's entire +// state and its only extremity. +// +// The state it is authorised against is empty, which is what an empty +// snapshot records; the state *after* it — the create event alone — persist +// derives from the event itself, as it does for any other state event. +func (r *API) persistCreateEvent(ctx context.Context, roomID, version string, rules roomversion.Rules, created *event.Event) error { + return r.persist(ctx, persistRequest{ + Room: storage.Room{ID: roomID, Version: version}, + Rules: rules, + Event: created, + StateBefore: auth.State{}, + }) +} + +// creationEvents are the events that turn a bare create event into a usable +// room, in the order they have to be sent. +func (r *API) creationEvents(ctx context.Context, req CreateRoomRequest, preset string, rules roomversion.Rules, roomID string) []sendRequest { + steps := []sendRequest{ + // The creator joins before anything else: every event after this + // one has to be sent by a member of the room. + { + RoomID: roomID, + Type: event.TypeMember, + StateKey: statePtr(req.Creator), + Sender: req.Creator, + Content: r.membershipContent(ctx, event.MembershipJoin, req.Creator, "", req.IsDirect), + }, + // Power levels before the join rules, so that the creator is still + // the only one who can set them. + { + RoomID: roomID, + Type: event.TypePowerLevels, + StateKey: statePtr(""), + Sender: req.Creator, + Content: r.powerLevelsContent(req, preset, rules), + }, + { + RoomID: roomID, + Type: event.TypeJoinRules, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{"join_rule": presetJoinRule(preset)}), + }, + { + RoomID: roomID, + Type: TypeHistoryVisibility, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{ + "history_visibility": HistoryVisibilityShared, + }), + }, + { + RoomID: roomID, + Type: TypeGuestAccess, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{ + "guest_access": presetGuestAccess(preset), + }), + }, + } + + // initial_state comes after the server's own setup events and may + // overwrite any of them, which is what lets a client create an + // encrypted or world-readable room in one call. + for _, s := range req.InitialState { + steps = append(steps, sendRequest{ + RoomID: roomID, + Type: s.Type, + StateKey: statePtr(s.StateKey), + Sender: req.Creator, + Content: s.Content, + }) + } + + // name and topic come after initial_state, because the spec gives the + // dedicated parameters precedence over an initial_state event of the + // same type. + if req.Name != "" { + steps = append(steps, sendRequest{ + RoomID: roomID, + Type: TypeName, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{"name": req.Name}), + }) + } + if req.Topic != "" { + steps = append(steps, sendRequest{ + RoomID: roomID, + Type: TypeTopic, + StateKey: statePtr(""), + Sender: req.Creator, + Content: mustMarshal(map[string]any{"topic": req.Topic}), + }) + } + return steps +} + +// powerLevelsContent is the m.room.power_levels event a new room gets. +// +// The levels are the spec's defaults written out rather than left implicit, +// because a client reading the event is entitled to see what the room +// actually does, and because power_level_content_override has to have +// something to override. +func (r *API) powerLevelsContent(req CreateRoomRequest, preset string, rules roomversion.Rules) json.RawMessage { + users := map[string]any{} + if !rules.CreatorsHaveInfinitePower { + // Up to room version 11 the creator's power has to be written down. + // From version 12 it cannot be: naming a creator in this event is + // itself a rejection. + users[req.Creator] = auth.CreatorPowerLevel + } + // trusted_private_chat makes everyone invited an equal, which is the + // whole point of the preset. + if preset == PresetTrustedPrivateChat { + for _, invitee := range req.Invite { + if rules.CreatorsHaveInfinitePower && invitee == req.Creator { + continue + } + users[invitee] = auth.CreatorPowerLevel + } + } + + content := map[string]any{ + "ban": auth.DefaultBan, + "kick": auth.DefaultKick, + "redact": auth.DefaultRedact, + "invite": auth.DefaultInvite, + "events_default": auth.DefaultEventsDefault, + "state_default": auth.DefaultStateDefault, + "users_default": auth.DefaultUsersDefault, + "users": users, + "events": map[string]any{ + // Tombstoning a room replaces it, so it is kept at the level + // the spec reserves for the room's owner even once others have + // been promoted to moderator. + "m.room.tombstone": 100, + "m.room.power_levels": 100, + "m.room.history_visibility": 100, + "m.room.canonical_alias": auth.DefaultStateDefault, + "m.room.avatar": auth.DefaultStateDefault, + TypeEncryption: 100, + "m.room.server_acl": 100, + }, + "notifications": map[string]any{"room": auth.DefaultNotification}, + } + + raw := mustMarshal(content) + if len(req.PowerLevelContentOverride) == 0 { + return raw + } + // A malformed override is left out rather than failing the room: the + // client is told nothing either way, and a room with the default levels + // is recoverable where a room that does not exist is not. The override + // is still authorised like any other power levels event, so it cannot + // be used to give anybody more power than the creator has. + merged, err := mergeJSON(raw, req.PowerLevelContentOverride) + if err != nil { + r.log.Warn("ignoring an unusable power_level_content_override", "error", err) + return raw + } + return merged +} + +// presetFor resolves the preset, defaulting the way the spec does: from the +// directory visibility when the client named no preset. +func presetFor(req CreateRoomRequest) (string, error) { + switch req.Preset { + case PresetPrivateChat, PresetTrustedPrivateChat, PresetPublicChat: + return req.Preset, nil + case "": + if req.Visibility == VisibilityPublic { + return PresetPublicChat, nil + } + return PresetPrivateChat, nil + default: + return "", fmt.Errorf("%w: %q is not a known preset", ErrInvalidContent, req.Preset) + } +} + +func presetJoinRule(preset string) string { + if preset == PresetPublicChat { + return auth.JoinRulePublic + } + return auth.JoinRuleInvite +} + +func presetGuestAccess(preset string) string { + if preset == PresetPublicChat { + return GuestAccessForbidden + } + return GuestAccessCanJoin +} + +// roomIDRandomBytes is how much randomness a room ID of a version that +// still needs one carries. 16 bytes is the same 128 bits as a UUID, encoded +// without padding. +const roomIDRandomBytes = 16 + +// generateRoomID mints a "!opaque:server" room ID, for the room versions +// before 12 where the ID is not derived from the create event. +func (r *API) generateRoomID() (string, error) { + buf := make([]byte, roomIDRandomBytes) + if _, err := rand.Read(buf); err != nil { + return "", fmt.Errorf("generating a room ID: %w", err) + } + roomID := "!" + base64.RawURLEncoding.EncodeToString(buf) + ":" + r.serverName + if err := id.ValidateRoomID(roomID); err != nil { + return "", err + } + return roomID, nil +} + +// statePtr makes a state key addressable, since the empty string is a +// perfectly good one and nil means "not a state event". +func statePtr(s string) *string { return &s } + +// mustMarshal encodes a value this package built itself. +// +// Every caller passes a map or a string, which cannot fail to encode; a +// failure here would be a bug in this file rather than anything a client +// could cause. +func mustMarshal(v any) json.RawMessage { + raw, err := json.Marshal(v) + if err != nil { + panic("roomserver: encoding a value this package built: " + err.Error()) + } + return raw +} + +// mergeJSON overlays the properties of override onto base, at the top level +// only. +// +// A shallow merge is what the spec asks for: power_level_content_override +// replacing the whole "users" map rather than merging into it is the +// documented behaviour, and the one a client setting up a room relies on. +func mergeJSON(base, override json.RawMessage) (json.RawMessage, error) { + var merged, extra map[string]json.RawMessage + if err := json.Unmarshal(base, &merged); err != nil { + return nil, err + } + if err := json.Unmarshal(override, &extra); err != nil { + return nil, fmt.Errorf("%w: the override must be a JSON object", ErrInvalidContent) + } + for key, value := range extra { + merged[key] = value + } + return json.Marshal(merged) +} diff --git a/server/internal/roomserver/directory.go b/server/internal/roomserver/directory.go new file mode 100644 index 0000000..f9c869b --- /dev/null +++ b/server/internal/roomserver/directory.go @@ -0,0 +1,395 @@ +package roomserver + +import ( + "context" + "fmt" + "strconv" + "strings" + + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// MaxAliasLength is the spec's cap on a room alias, sigil and server name +// included. +const MaxAliasLength = id.MaxLength + +// aliasFor builds a full local alias from the localpart a client supplied, +// and checks it. +func (r *API) aliasFor(localpart string) (string, error) { + alias := "#" + localpart + ":" + r.serverName + if err := r.validateLocalAlias(alias); err != nil { + return "", err + } + return alias, nil +} + +// validateLocalAlias checks an alias and insists it belongs to this server. +// +// The grammar is deliberately the loose one the spec gives for an alias — +// anything but a colon, and no control characters — rather than the +// localpart rules an account is held to. An alias is a label a room's +// members chose, and the spec puts no character restrictions on it beyond +// what parsing needs. +func (r *API) validateLocalAlias(alias string) error { + if err := validateAliasGrammar(alias); err != nil { + return err + } + _, serverName, _ := strings.Cut(alias[1:], ":") + if serverName != r.serverName { + return fmt.Errorf("%w: %s is not an alias of this server", ErrForbidden, alias) + } + return nil +} + +// validateAliasGrammar checks an alias of any server. +func validateAliasGrammar(alias string) error { + switch { + case alias == "": + return fmt.Errorf("%w: a room alias must not be empty", ErrInvalidContent) + case alias[0] != '#': + return fmt.Errorf("%w: a room alias must start with '#'", ErrInvalidContent) + case len(alias) > MaxAliasLength: + return fmt.Errorf("%w: a room alias may be at most %d bytes", ErrInvalidContent, MaxAliasLength) + } + + localpart, serverName, found := strings.Cut(alias[1:], ":") + switch { + case !found: + return fmt.Errorf("%w: %s has no server name; expected #alias:server", ErrInvalidContent, alias) + case localpart == "": + return fmt.Errorf("%w: %s has an empty localpart", ErrInvalidContent, alias) + case serverName == "": + return fmt.Errorf("%w: %s has an empty server name", ErrInvalidContent, alias) + } + // A control character in an alias would be invisible in every client + // that shows it, which is exactly what makes it worth refusing. + if strings.ContainsFunc(alias, func(c rune) bool { return c < 0x20 || c == 0x7f }) { + return fmt.Errorf("%w: %s contains a control character", ErrInvalidContent, alias) + } + return nil +} + +// isLocalAlias reports whether an alias belongs to this server. +func (r *API) isLocalAlias(alias string) bool { + if validateAliasGrammar(alias) != nil { + return false + } + _, serverName, _ := strings.Cut(alias[1:], ":") + return serverName == r.serverName +} + +// CreateAlias points a local alias at a room. +// +// The caller has to be in the room. The spec leaves the permission to the +// server, and "a member of the room may give it a name" is the rule that +// matches what an alias is for while keeping a stranger from squatting +// names against rooms they cannot see. +func (r *API) CreateAlias(ctx context.Context, alias, roomID, userID string) error { + if err := r.validateLocalAlias(alias); err != nil { + return err + } + if err := id.ValidateRoomID(roomID); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidContent, err) + } + if _, err := r.store.Room(ctx, roomID); err != nil { + return err + } + if err := r.requireJoined(ctx, roomID, userID); err != nil { + return err + } + return r.store.CreateAlias(ctx, storage.RoomAlias{ + Alias: alias, + RoomID: roomID, + Creator: userID, + CreatedAtMS: r.nowMS(), + }) +} + +// DeleteAlias removes a local alias. +// +// Either the user who created it, or a user who could set the room's +// canonical alias, may delete it — the first because it is theirs, the +// second because the room's moderators have to be able to clean up after a +// member who has left. +func (r *API) DeleteAlias(ctx context.Context, alias, userID string, isAdmin bool) error { + if err := r.validateLocalAlias(alias); err != nil { + return err + } + + stored, err := r.store.Alias(ctx, alias) + if err != nil { + return err + } + if !isAdmin && stored.Creator != userID { + canSet, err := r.CanSendStateEvent(ctx, stored.RoomID, userID, TypeCanonicalAlias) + if err != nil { + return err + } + if !canSet { + return fmt.Errorf("%w: %s may not delete the alias %s", ErrForbidden, userID, alias) + } + } + return r.store.DeleteAlias(ctx, alias) +} + +// ResolvedAlias is what a directory lookup returns. +type ResolvedAlias struct { + RoomID string + + // Servers are the servers known to be in the room, which a client + // passes back as ?server_name= when it joins. With federation off the + // only one is this server. + Servers []string +} + +// ResolveAlias turns a room alias into a room ID. +// +// A remote alias has to be asked of the server that owns it, which needs +// federation; with it off the client is told that rather than being told +// the alias does not exist, because the two call for quite different +// reactions. +func (r *API) ResolveAlias(ctx context.Context, alias string) (ResolvedAlias, error) { + if err := validateAliasGrammar(alias); err != nil { + return ResolvedAlias{}, err + } + if !r.isLocalAlias(alias) { + if !r.federationEnabled() { + return ResolvedAlias{}, fmt.Errorf( + "%w: %s belongs to another server, and resolving a remote alias", + ErrFederationDisabled, alias) + } + return ResolvedAlias{}, fmt.Errorf( + "%w: resolving an alias of another server is not implemented yet", ErrFederationDisabled) + } + + stored, err := r.store.Alias(ctx, alias) + if err != nil { + return ResolvedAlias{}, err + } + return ResolvedAlias{RoomID: stored.RoomID, Servers: []string{r.serverName}}, nil +} + +// RoomAliases lists the local aliases of a room, for a user who may see it. +func (r *API) RoomAliases(ctx context.Context, roomID, userID string) ([]string, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return nil, err + } + if err := r.requireVisibility(ctx, room, rules, userID); err != nil { + return nil, err + } + aliases, err := r.store.AliasesForRoom(ctx, roomID) + if err != nil { + return nil, err + } + if aliases == nil { + aliases = []string{} + } + return aliases, nil +} + +// RoomVisibility reports whether a room is in this server's public +// directory. +func (r *API) RoomVisibility(ctx context.Context, roomID string) (string, error) { + room, err := r.store.Room(ctx, roomID) + if err != nil { + return "", err + } + if room.Published { + return VisibilityPublic, nil + } + return VisibilityPrivate, nil +} + +// SetRoomVisibility publishes a room in this server's directory, or +// withdraws it. +// +// The permission is the one the spec hints at and Synapse settled on: a +// user who could set the room's canonical alias, or a server admin. An +// unpublished room that nobody can find is recoverable; a room published by +// a passer-by is not. +func (r *API) SetRoomVisibility(ctx context.Context, roomID, userID, visibility string, isAdmin bool) error { + switch visibility { + case VisibilityPublic, VisibilityPrivate: + default: + return fmt.Errorf("%w: visibility must be %q or %q", ErrInvalidContent, + VisibilityPublic, VisibilityPrivate) + } + if _, err := r.store.Room(ctx, roomID); err != nil { + return err + } + + if !isAdmin { + canSet, err := r.CanSendStateEvent(ctx, roomID, userID, TypeCanonicalAlias) + if err != nil { + return err + } + if !canSet { + return fmt.Errorf("%w: %s may not change whether %s is published", + ErrForbidden, userID, roomID) + } + } + return r.store.SetPublished(ctx, roomID, visibility == VisibilityPublic) +} + +// PublicRoomsRequest pages and filters the public room directory. +type PublicRoomsRequest struct { + // Limit is how many rooms to return; 0 lets the store decide. + Limit int + + // Since is the pagination token from a previous response. + Since string + + // Search is the generic search term from the filter. + Search string + + // Server is the ?server= parameter: another server's directory, which + // needs federation. + Server string +} + +// PublicRoom is one entry in the public room directory. +type PublicRoom struct { + RoomID string `json:"room_id"` + Name string `json:"name,omitempty"` + Topic string `json:"topic,omitempty"` + CanonicalAlias string `json:"canonical_alias,omitempty"` + AvatarURL string `json:"avatar_url,omitempty"` + NumJoinedMembers int `json:"num_joined_members"` + WorldReadable bool `json:"world_readable"` + GuestCanJoin bool `json:"guest_can_join"` + JoinRule string `json:"join_rule,omitempty"` + RoomType string `json:"room_type,omitempty"` + Aliases []string `json:"aliases,omitempty"` +} + +// PublicRoomsResponse is the directory page. +type PublicRoomsResponse struct { + Chunk []PublicRoom `json:"chunk"` + + // NextBatch and PrevBatch are the tokens for the pages either side, + // and are absent at the ends. + NextBatch string `json:"next_batch,omitempty"` + PrevBatch string `json:"prev_batch,omitempty"` + + TotalRoomCountEstimate int `json:"total_room_count_estimate"` +} + +// PublicRooms returns a page of this server's public room directory. +// +// The pagination token is the offset, written out as a decimal. That is +// enough for a directory ordered by member count and is honest about what +// it is: a token that encoded a cursor would imply a stability the ordering +// does not have, since a room gaining a member moves between pages whatever +// the token says. +func (r *API) PublicRooms(ctx context.Context, req PublicRoomsRequest) (PublicRoomsResponse, error) { + if req.Server != "" && req.Server != r.serverName { + if !r.federationEnabled() { + return PublicRoomsResponse{}, fmt.Errorf( + "%w: reading the room directory of %s", ErrFederationDisabled, req.Server) + } + return PublicRoomsResponse{}, fmt.Errorf( + "%w: reading another server's room directory is not implemented yet", ErrFederationDisabled) + } + + offset, err := parseDirectoryToken(req.Since) + if err != nil { + return PublicRoomsResponse{}, err + } + + summaries, total, err := r.store.PublishedRooms(ctx, storage.RoomFilter{ + Search: req.Search, + Limit: req.Limit, + Offset: offset, + }) + if err != nil { + return PublicRoomsResponse{}, err + } + + resp := PublicRoomsResponse{ + Chunk: make([]PublicRoom, 0, len(summaries)), + TotalRoomCountEstimate: total, + } + for _, summary := range summaries { + aliases, err := r.store.AliasesForRoom(ctx, summary.ID) + if err != nil { + return PublicRoomsResponse{}, err + } + resp.Chunk = append(resp.Chunk, publicRoomOf(summary, aliases)) + } + + if offset > 0 { + resp.PrevBatch = directoryToken(max(offset-len(summaries), 0)) + } + if offset+len(summaries) < total { + resp.NextBatch = directoryToken(offset + len(summaries)) + } + return resp, nil +} + +// publicRoomOf projects a stored room onto a directory entry. +func publicRoomOf(summary storage.RoomSummary, aliases []string) PublicRoom { + return PublicRoom{ + RoomID: summary.ID, + Name: summary.Name, + Topic: summary.Topic, + CanonicalAlias: summary.CanonicalAlias, + AvatarURL: summary.AvatarURL, + NumJoinedMembers: summary.JoinedMembers, + WorldReadable: summary.HistoryVisibility == HistoryVisibilityWorldReadable, + GuestCanJoin: summary.GuestAccess == GuestAccessCanJoin, + JoinRule: summary.JoinRule, + RoomType: summary.RoomType, + Aliases: aliases, + } +} + +// containsString reports whether needle is in haystack. +func containsString(haystack []string, needle string) bool { + for _, s := range haystack { + if s == needle { + return true + } + } + return false +} + +// membershipFilterValues are the memberships a client may filter a member +// list by. +var membershipFilterValues = []string{ + event.MembershipJoin, event.MembershipLeave, + event.MembershipInvite, event.MembershipBan, event.MembershipKnock, +} + +// ValidateMembershipFilter checks a ?membership= or ?not_membership= value. +func ValidateMembershipFilter(value string) error { + if value == "" || containsString(membershipFilterValues, value) { + return nil + } + return fmt.Errorf("%w: %q is not a membership", ErrInvalidContent, value) +} + +// maxDirectoryOffset bounds how far into the directory a pagination token +// may point. The token comes back from a client, so it is input: without a +// bound, "since=99999999999999" is an invitation to make the database count +// past the end of a table. +const maxDirectoryOffset = 1 << 20 + +// directoryToken renders a directory offset as a pagination token. +func directoryToken(offset int) string { + return strconv.Itoa(offset) +} + +// parseDirectoryToken reads a pagination token back. +func parseDirectoryToken(token string) (int, error) { + if token == "" { + return 0, nil + } + offset, err := strconv.Atoi(token) + if err != nil || offset < 0 || offset > maxDirectoryOffset { + return 0, fmt.Errorf("%w: %q is not a pagination token from this server", ErrInvalidContent, token) + } + return offset, nil +} diff --git a/server/internal/roomserver/directory_test.go b/server/internal/roomserver/directory_test.go new file mode 100644 index 0000000..1b856a7 --- /dev/null +++ b/server/internal/roomserver/directory_test.go @@ -0,0 +1,326 @@ +package roomserver_test + +import ( + "errors" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/roomserver" +) + +func TestAliasLifecycle(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + const alias = "#chat:" + testServerName + + if err := f.api.CreateAlias(t.Context(), alias, resp.RoomID, alice); err != nil { + t.Fatalf("CreateAlias: %v", err) + } + resolved, err := f.api.ResolveAlias(t.Context(), alias) + if err != nil { + t.Fatalf("ResolveAlias: %v", err) + } + if resolved.RoomID != resp.RoomID { + t.Errorf("the alias resolves to %q, want %q", resolved.RoomID, resp.RoomID) + } + // A client needs somewhere to join through, which with federation off + // is this server. + if len(resolved.Servers) != 1 || resolved.Servers[0] != testServerName { + t.Errorf("the alias names servers %v, want [%s]", resolved.Servers, testServerName) + } + + aliases, err := f.api.RoomAliases(t.Context(), resp.RoomID, alice) + if err != nil { + t.Fatalf("RoomAliases: %v", err) + } + if len(aliases) != 1 || aliases[0] != alias { + t.Errorf("RoomAliases = %v, want [%s]", aliases, alias) + } + + // The creator of an alias may remove it. + if err := f.api.DeleteAlias(t.Context(), alias, alice, false); err != nil { + t.Fatalf("DeleteAlias: %v", err) + } + if _, err := f.api.ResolveAlias(t.Context(), alias); !errors.Is(err, roomserver.ErrNoSuchAlias) { + t.Errorf("the deleted alias still resolves: %v", err) + } +} + +func TestAliasPermissions(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + // Only a member may name a room, so that a stranger cannot squat names + // against rooms they cannot see. + err := f.api.CreateAlias(t.Context(), "#squat:"+testServerName, resp.RoomID, bob) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("a stranger creating an alias = %v, want ErrForbidden", err) + } + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + const bobsAlias = "#bobs:" + testServerName + if err := f.api.CreateAlias(t.Context(), bobsAlias, resp.RoomID, bob); err != nil { + t.Fatalf("CreateAlias as a member: %v", err) + } + + // carol is nobody here and may not delete it. + if err := f.api.DeleteAlias(t.Context(), bobsAlias, carol, false); !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("a stranger deleting an alias = %v, want ErrForbidden", err) + } + // alice can, because she could set the room's canonical alias — which + // is what lets a room's moderators clean up after a departed member. + if err := f.api.DeleteAlias(t.Context(), bobsAlias, alice, false); err != nil { + t.Errorf("a moderator deleting a member's alias: %v", err) + } + + // An alias of another server is not ours to create or resolve. + err = f.api.CreateAlias(t.Context(), "#elsewhere:other.example", resp.RoomID, alice) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("creating a remote alias = %v, want ErrForbidden", err) + } + _, err = f.api.ResolveAlias(t.Context(), "#elsewhere:other.example") + if !errors.Is(err, roomserver.ErrFederationDisabled) { + t.Errorf("resolving a remote alias = %v, want ErrFederationDisabled", err) + } + + // Malformed aliases are refused before anything is looked up. + for _, bad := range []string{"", "general", "#", "#general", "#:" + testServerName} { + if err := f.api.CreateAlias(t.Context(), bad, resp.RoomID, alice); !errors.Is(err, roomserver.ErrInvalidContent) { + t.Errorf("CreateAlias(%q) = %v, want ErrInvalidContent", bad, err) + } + } +} + +func TestPublicRoomDirectory(t *testing.T) { + f := newFixture(t, nil) + + private := f.createRoom(roomserver.CreateRoomRequest{Name: "Private"}) + public := f.createRoom(roomserver.CreateRoomRequest{ + Preset: roomserver.PresetPublicChat, + Visibility: roomserver.VisibilityPublic, + Name: "Public Room", + Topic: "Anybody may join", + }) + + resp, err := f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{}) + if err != nil { + t.Fatalf("PublicRooms: %v", err) + } + if resp.TotalRoomCountEstimate != 1 || len(resp.Chunk) != 1 { + t.Fatalf("the directory has %d of %d rooms, want 1 of 1", + len(resp.Chunk), resp.TotalRoomCountEstimate) + } + entry := resp.Chunk[0] + if entry.RoomID != public.RoomID { + t.Errorf("the directory lists %q, want %q", entry.RoomID, public.RoomID) + } + if entry.Name != "Public Room" || entry.Topic != "Anybody may join" { + t.Errorf("the directory entry is %+v", entry) + } + if entry.NumJoinedMembers != 1 { + t.Errorf("num_joined_members = %d, want 1", entry.NumJoinedMembers) + } + if entry.WorldReadable { + t.Error("a shared-history room is reported as world-readable") + } + // The public_chat preset makes the room joinable by anyone with an + // account and closed to guests. The two are independent, which is + // exactly why the pairing is worth pinning down. + if entry.JoinRule != "public" { + t.Errorf("join_rule = %q, want public", entry.JoinRule) + } + if entry.GuestCanJoin { + t.Error("a public_chat room reports that guests may join") + } + + // Publishing the private room puts it in, and the two then order by + // member count. + if err := f.api.SetRoomVisibility(t.Context(), private.RoomID, alice, + roomserver.VisibilityPublic, false); err != nil { + t.Fatalf("SetRoomVisibility: %v", err) + } + if visibility, err := f.api.RoomVisibility(t.Context(), private.RoomID); err != nil { + t.Fatalf("RoomVisibility: %v", err) + } else if visibility != roomserver.VisibilityPublic { + t.Errorf("visibility = %q, want public", visibility) + } + + resp, err = f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{}) + if err != nil { + t.Fatalf("PublicRooms: %v", err) + } + if resp.TotalRoomCountEstimate != 2 { + t.Errorf("the directory has %d rooms, want 2", resp.TotalRoomCountEstimate) + } + + // The search looks at the name. + resp, err = f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{Search: "public"}) + if err != nil { + t.Fatalf("PublicRooms with a search: %v", err) + } + if len(resp.Chunk) != 1 || resp.Chunk[0].RoomID != public.RoomID { + t.Errorf("searching for \"public\" returned %+v", resp.Chunk) + } + + // Withdrawing a room takes it back out. + if err := f.api.SetRoomVisibility(t.Context(), public.RoomID, alice, + roomserver.VisibilityPrivate, false); err != nil { + t.Fatalf("SetRoomVisibility: %v", err) + } + resp, err = f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{}) + if err != nil { + t.Fatalf("PublicRooms: %v", err) + } + if resp.TotalRoomCountEstimate != 1 { + t.Errorf("the directory has %d rooms after a withdrawal, want 1", resp.TotalRoomCountEstimate) + } +} + +func TestPublicRoomDirectoryPaging(t *testing.T) { + f := newFixture(t, nil) + for range 5 { + f.createRoom(roomserver.CreateRoomRequest{ + Preset: roomserver.PresetPublicChat, + Visibility: roomserver.VisibilityPublic, + }) + } + + first, err := f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{Limit: 2}) + if err != nil { + t.Fatalf("PublicRooms: %v", err) + } + if len(first.Chunk) != 2 || first.NextBatch == "" || first.PrevBatch != "" { + t.Fatalf("the first page is %d rooms, next=%q prev=%q", + len(first.Chunk), first.NextBatch, first.PrevBatch) + } + + second, err := f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{ + Limit: 2, Since: first.NextBatch, + }) + if err != nil { + t.Fatalf("PublicRooms page two: %v", err) + } + if len(second.Chunk) != 2 || second.PrevBatch == "" { + t.Fatalf("the second page is %d rooms, prev=%q", len(second.Chunk), second.PrevBatch) + } + if second.Chunk[0].RoomID == first.Chunk[0].RoomID { + t.Error("the second page repeats the first") + } + + last, err := f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{ + Limit: 2, Since: second.NextBatch, + }) + if err != nil { + t.Fatalf("PublicRooms page three: %v", err) + } + if len(last.Chunk) != 1 || last.NextBatch != "" { + t.Errorf("the last page is %d rooms, next=%q", len(last.Chunk), last.NextBatch) + } + + // A token this server did not issue is refused rather than clamped: a + // client that made one up should hear about it. + if _, err := f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{ + Since: "not-a-token", + }); !errors.Is(err, roomserver.ErrInvalidContent) { + t.Errorf("an invented pagination token = %v, want ErrInvalidContent", err) + } +} + +func TestPublicRoomDirectoryPermissions(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + + // An ordinary member cannot publish the room: a room published by a + // passer-by cannot be taken back by its moderators. + err := f.api.SetRoomVisibility(t.Context(), resp.RoomID, bob, + roomserver.VisibilityPublic, false) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("a member publishing a room = %v, want ErrForbidden", err) + } + // An admin can, whoever they are. + if err := f.api.SetRoomVisibility(t.Context(), resp.RoomID, bob, + roomserver.VisibilityPublic, true); err != nil { + t.Errorf("an admin publishing a room: %v", err) + } + + // A value that is neither is a bad request, not a silent no-op. + err = f.api.SetRoomVisibility(t.Context(), resp.RoomID, alice, "secret", false) + if !errors.Is(err, roomserver.ErrInvalidContent) { + t.Errorf("an invented visibility = %v, want ErrInvalidContent", err) + } + + // Another server's directory needs federation. + _, err = f.api.PublicRooms(t.Context(), roomserver.PublicRoomsRequest{Server: "other.example"}) + if !errors.Is(err, roomserver.ErrFederationDisabled) { + t.Errorf("reading a remote directory = %v, want ErrFederationDisabled", err) + } +} + +func TestAdminRoomListAndPurge(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Preset: roomserver.PresetPublicChat, + Name: "Doomed", + RoomAliasName: "doomed", + }) + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + if _, err := f.sendMessage(resp.RoomID, alice, "hello", "m1"); err != nil { + t.Fatalf("sending a message: %v", err) + } + + details, err := f.api.RoomDetails(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomDetails: %v", err) + } + if details.Name != "Doomed" || details.JoinedMembers != 2 || details.Members != 2 { + t.Errorf("RoomDetails = %+v", details) + } + if len(details.Aliases) != 1 { + t.Errorf("RoomDetails lists %v aliases, want one", details.Aliases) + } + // A room filled by one server has a single forward extremity; more + // than one would mean it had forked. + if len(details.ForwardExtremities) != 1 { + t.Errorf("the room has %d forward extremities, want 1", len(details.ForwardExtremities)) + } + + stats, err := f.api.Stats(t.Context()) + if err != nil { + t.Fatalf("Stats: %v", err) + } + if stats.Rooms != 1 || stats.Published != 0 || stats.Events == 0 { + t.Errorf("Stats = %+v", stats) + } + + if err := f.api.PurgeRoom(t.Context(), resp.RoomID); err != nil { + t.Fatalf("PurgeRoom: %v", err) + } + if _, err := f.api.RoomDetails(t.Context(), resp.RoomID); !errors.Is(err, roomserver.ErrNoSuchRoom) { + t.Errorf("the purged room is still there: %v", err) + } + // The alias went with it, so the name can be used again. + if _, err := f.api.ResolveAlias(t.Context(), resp.Alias); !errors.Is(err, roomserver.ErrNoSuchAlias) { + t.Errorf("the purged room's alias still resolves: %v", err) + } + if rooms, err := f.api.JoinedRooms(t.Context(), bob); err != nil { + t.Fatalf("JoinedRooms: %v", err) + } else if len(rooms) != 0 { + t.Errorf("bob is still joined to %v after the purge", rooms) + } + if err := f.api.PurgeRoom(t.Context(), resp.RoomID); !errors.Is(err, roomserver.ErrNoSuchRoom) { + t.Errorf("purging twice = %v, want ErrNoSuchRoom", err) + } +} diff --git a/server/internal/roomserver/input.go b/server/internal/roomserver/input.go new file mode 100644 index 0000000..64d44a9 --- /dev/null +++ b/server/internal/roomserver/input.go @@ -0,0 +1,420 @@ +package roomserver + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/matrix/stateres" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// sendRequest is one event this server is about to create. +// +// What the caller supplies is what the event says. Everything that makes it +// a valid link in the room's DAG — prev_events, depth, auth_events, the +// hashes and the signature — is worked out by the pipeline, because only the +// pipeline holds the room lock that makes those answers stable. +type sendRequest struct { + RoomID string + Type string + + // StateKey makes this a state event when it is non-nil. The empty + // string is a valid state key. + StateKey *string + + // Sender is the full user ID sending the event. It must be a user of + // this server: an event is signed by its sender's server, and this + // server can only sign as itself. + Sender string + + Content json.RawMessage + + // Redacts is the target of an m.room.redaction event. The builder puts + // it wherever the room version keeps it. + Redacts string + + // Transaction, when set, records the client transaction ID this event + // came from, so that a retried request does not send a second event. + Transaction *storage.TransactionRef +} + +// send is the input pipeline for an event this server creates. +// +// The stages are the ones PLAN.md names. Validate: the sender is ours and +// the content is something we will sign. Authorise: the event is built +// against the room's state and run through the authorisation rules, with +// exactly the auth_events the selection algorithm asks for. Resolve: the +// state the event is judged against is the state after the room's forward +// extremities, resolved when there is more than one of them. Persist: the +// event, its state snapshot, the new current state and the new extremity go +// down in one transaction. Emit: the sync API is told. +// +// Everything after the lock is taken is deterministic, which is what lets +// two clients sending into the same room at the same moment produce a +// straight chain rather than a fork. +func (r *API) send(ctx context.Context, req sendRequest) (*event.Event, error) { + if !r.isLocal(req.Sender) { + return nil, fmt.Errorf("%w: %s is not a user of this server", ErrForbidden, req.Sender) + } + content, err := validateContent(req.Content) + if err != nil { + return nil, err + } + req.Content = content + + mu := r.lock(req.RoomID) + mu.Lock() + defer mu.Unlock() + + room, err := r.store.Room(ctx, req.RoomID) + if err != nil { + return nil, err + } + rules, err := r.rules(room) + if err != nil { + return nil, err + } + + // A retried request must not send a second event. The check is inside + // the lock so that two copies of the same request racing each other + // cannot both find the transaction unseen. + if req.Transaction != nil { + switch existing, err := r.store.TransactionEvent(ctx, *req.Transaction); { + case err == nil: + stored, err := r.store.Event(ctx, req.RoomID, existing) + if err != nil { + return nil, err + } + return parseEvent(stored, rules) + case !isNotFound(err): + return nil, err + } + } + + st, extremities, replacesWholeState, err := r.roomState(ctx, room, rules) + if err != nil { + return nil, err + } + + authEvents, err := auth.SelectAuthEvents(rules, req.Type, req.Sender, req.StateKey, req.Content, st) + if err != nil { + return nil, fmt.Errorf("%w: %s", ErrRejected, err) + } + + built, err := r.build(req, rules, authEvents, extremities) + if err != nil { + return nil, err + } + + // Rule 3: the auth_events the event declares are well-formed and are + // the ones the selection algorithm asks for. The pipeline chose them + // itself a moment ago, so this can only fail on a bug in that choice — + // which is exactly the bug worth catching before an event is in a room + // and every other server has to agree with it. + if err := auth.CheckAuthEvents(built, authEvents); err != nil { + return nil, err + } + + // The rules are applied against the state the event's own auth_events + // name, which is what every other server in the room will do when it + // sees this event. For an event created here that state is also the + // room's current state, so there is no separate soft-failure check to + // make: the two can only differ for an event that arrives from + // somewhere else, which is Phase 5. + if err := auth.Check(ctx, built, checkState(authEvents, st), r.verifier()); err != nil { + return nil, err + } + + if err := r.persist(ctx, persistRequest{ + Room: room, + Rules: rules, + Event: built, + StateBefore: st, + ReplaceWholeState: replacesWholeState, + Transaction: req.Transaction, + RedactsEventID: req.Redacts, + }); err != nil { + return nil, err + } + return built, nil +} + +// build assembles and signs the event. +func (r *API) build(req sendRequest, rules roomversion.Rules, authEvents, prevEvents []*event.Event) (*event.Event, error) { + builder := event.Builder{ + Type: req.Type, + StateKey: req.StateKey, + Sender: req.Sender, + Content: req.Content, + Redacts: req.Redacts, + AuthEvents: eventIDs(authEvents), + PrevEvents: eventIDs(prevEvents), + Depth: nextDepth(prevEvents), + } + // A room version 12 create event carries no room_id, because the room + // ID is its own event ID. Nothing else in a room may leave it out. + if !rules.RoomIDIsCreateEventID || req.Type != event.TypeCreate { + builder.RoomID = req.RoomID + } + + built, err := builder.Build(r.now(), r.signingKey, rules) + if err != nil { + return nil, fmt.Errorf("building a %s event: %w", req.Type, err) + } + return built, nil +} + +// checkState is the state an event is judged against: the events its +// auth_events name, plus the create event. +// +// The create event is added separately because from room version 12 it is +// not cited in auth_events at all — the room ID *is* its event ID, so +// naming it would be saying the same thing twice. The rules still need it: +// it is what says who the room's creators are and, in that same version, +// what the room's ID should be. In earlier versions it is already among the +// auth events and this changes nothing. +func checkState(authEvents []*event.Event, roomState auth.State) auth.State { + st := auth.NewState(authEvents) + if st.Create() == nil { + if create := roomState.Create(); create != nil { + st[auth.StateKey{Type: event.TypeCreate}] = create + } + } + return st +} + +// eventIDs projects events onto their IDs, in order. +func eventIDs(events []*event.Event) []string { + out := make([]string, 0, len(events)) + for _, e := range events { + out = append(out, e.EventID()) + } + return out +} + +// nextDepth is one more than the greatest depth among the prev_events. +// +// Depth is advisory — it orders events for display and is never used to +// authorise one — so a room that reaches the limit stays there rather than +// failing: refusing to accept any further events would be a worse outcome +// than a flat tail, and the limit is 2^53-1 deep. +func nextDepth(prevEvents []*event.Event) int64 { + var depth int64 + for _, e := range prevEvents { + if e.Depth() > depth { + depth = e.Depth() + } + } + if depth >= event.MaxDepth { + return event.MaxDepth + } + return depth + 1 +} + +// roomState returns the state the next event in a room is authorised +// against, along with the room's forward extremities. +// +// The stored current state is the answer whenever a room has a single +// forward extremity, which is every room this server fills on its own: the +// lock in send makes each new event a child of the one before it. A room +// with several extremities has forked, and the state is then resolved from +// the state after each of them — the third return value says so, because a +// resolved state has to be written out whole: tuples the resolution dropped +// have to go, and an incremental update cannot express that. +// +// Only federation forks a room, so in Phase 3 the second path is unreachable +// from outside. It is here rather than in Phase 5 because the alternative is +// a pipeline that authorises against one arbitrary branch of a fork without +// saying so. +func (r *API) roomState(ctx context.Context, room storage.Room, rules roomversion.Rules) (auth.State, []*event.Event, bool, error) { + storedExtremities, err := r.store.Extremities(ctx, room.ID) + if err != nil { + return nil, nil, false, err + } + extremities, err := parseEvents(storedExtremities, rules) + if err != nil { + return nil, nil, false, err + } + + if len(extremities) <= 1 { + stored, err := r.store.CurrentStateEvents(ctx, room.ID) + if err != nil { + return nil, nil, false, err + } + events, err := parseEvents(stored, rules) + if err != nil { + return nil, nil, false, err + } + return auth.NewState(events), extremities, false, nil + } + + stateSets := make([][]*event.Event, 0, len(extremities)) + for _, e := range extremities { + set, err := r.stateAfter(ctx, room.ID, rules, e) + if err != nil { + return nil, nil, false, err + } + stateSets = append(stateSets, set) + } + resolved, err := stateres.Resolve(ctx, rules, stateSets, + eventProvider{store: r.store, roomID: room.ID, rules: rules}) + if err != nil { + return nil, nil, false, fmt.Errorf("resolving the state of %s: %w", room.ID, err) + } + return resolved, extremities, true, nil +} + +// stateAfter returns the room state as of an event, the event included. +// +// The snapshot stored against an event is the state *before* it; adding the +// event itself when it is a state event is what turns that into the state a +// child of it would see. +func (r *API) stateAfter(ctx context.Context, roomID string, rules roomversion.Rules, e *event.Event) ([]*event.Event, error) { + entries, err := r.store.StateBefore(ctx, roomID, e.EventID()) + if err != nil { + return nil, err + } + ids := make([]string, 0, len(entries)) + for _, entry := range entries { + ids = append(ids, entry.EventID) + } + stored, err := r.store.EventsByID(ctx, roomID, ids) + if err != nil { + return nil, err + } + events, err := parseEvents(stored, rules) + if err != nil { + return nil, err + } + if e.IsState() { + events = append(events, e) + } + return events, nil +} + +// persistRequest is everything the persist stage needs. +type persistRequest struct { + Room storage.Room + Rules roomversion.Rules + Event *event.Event + + // StateBefore is the room state the event was authorised against. + StateBefore auth.State + + // ReplaceWholeState says StateBefore came out of state resolution, so + // the new current state has to be written out in full. + ReplaceWholeState bool + + Transaction *storage.TransactionRef + RedactsEventID string +} + +// persist writes the event and everything its arrival changes, then emits. +func (r *API) persist(ctx context.Context, req persistRequest) error { + e := req.Event + stateKey, isState := e.StateKey() + + write := storage.RoomWrite{ + Event: storage.StoredEvent{ + RoomID: req.Room.ID, + EventID: e.EventID(), + Type: e.Type(), + Sender: e.Sender(), + Depth: e.Depth(), + OriginServerTS: e.OriginServerTS(), + JSON: e.JSON(), + }, + StateBefore: stateEntries(req.StateBefore), + Extremities: []string{e.EventID()}, + Transaction: req.Transaction, + } + if isState { + write.Event.StateKey = &stateKey + } + + // The state after this event. For the ordinary case that is one tuple + // to upsert; after a resolution it is the whole state, which the store + // is told to write as a replacement. + stateAfter := req.StateBefore + if req.ReplaceWholeState { + write.StateDeltaIsWholeState = true + write.StateDelta = stateEntries(req.StateBefore) + } + if isState { + entry := stateEntry(e.Type(), stateKey, e) + if req.ReplaceWholeState { + write.StateDelta = upsertEntry(write.StateDelta, entry) + } else { + write.StateDelta = []storage.StateEntry{entry} + } + stateAfter = withState(req.StateBefore, e.Type(), stateKey, e) + } + + // The denormalised listing columns are recomputed only when this event + // is one of the ones behind them, which keeps a room full of messages + // from rewriting its own row on every line of chat. + if isState && affectsRoomProfile(e.Type()) { + profile := roomProfile(stateAfter) + write.Profile = &profile + } + + if req.RedactsEventID != "" { + if err := r.prepareRedaction(ctx, &write, req); err != nil { + return err + } + } + + ordering, err := r.store.Write(ctx, write) + if err != nil { + return err + } + + r.log.Debug("persisted event", + "room_id", req.Room.ID, "event_id", e.EventID(), "type", e.Type(), + "sender", e.Sender(), "stream_ordering", ordering) + + if r.notifier != nil { + r.notifier.EventPersisted(ctx, Notification{ + RoomID: req.Room.ID, + Event: e, + StreamOrdering: ordering, + StateChanged: isState, + }) + } + return nil +} + +// withState returns st with one piece of state replaced, leaving st alone. +func withState(st auth.State, eventType, stateKey string, e *event.Event) auth.State { + out := make(auth.State, len(st)+1) + for key, value := range st { + out[key] = value + } + out[auth.StateKey{Type: eventType, StateKey: stateKey}] = e + return out +} + +// upsertEntry replaces the matching tuple in entries, or appends it. +func upsertEntry(entries []storage.StateEntry, entry storage.StateEntry) []storage.StateEntry { + for i, existing := range entries { + if existing.Type == entry.Type && existing.StateKey == entry.StateKey { + entries[i] = entry + return entries + } + } + return append(entries, entry) +} + +// isNotFound reports whether err is one of the store's "nothing there" +// sentinels, which several callers treat as an ordinary answer rather than +// as a failure. +func isNotFound(err error) bool { + return errors.Is(err, ErrNoSuchEvent) || + errors.Is(err, ErrNoSuchRoom) || + errors.Is(err, ErrNoSuchAlias) +} diff --git a/server/internal/roomserver/membership.go b/server/internal/roomserver/membership.go new file mode 100644 index 0000000..aa4d1b1 --- /dev/null +++ b/server/internal/roomserver/membership.go @@ -0,0 +1,459 @@ +package roomserver + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// JoinRequest asks to join a room. +type JoinRequest struct { + // RoomID is the room to join. The caller has already resolved an alias. + RoomID string + + // User is the full user ID joining. + User string + + // Reason is an optional note put in the membership event. + Reason string + + // ServerNames are the ?server_name= hints a client sends for a room its + // own server may not know. They are only usable over federation, and + // their presence for a room this server does not have is what turns a + // join into a federated one. + ServerNames []string +} + +// JoinRoom joins a local room. +// +// Joining a room this server does not have is a federated join: the whole +// room has to be fetched from a server that does have it, which is Phase 5. +// While federation is off that is refused with a message saying so, which is +// the behaviour PLAN.md section 3 asks for — a client otherwise sees a bare +// M_FORBIDDEN and the operator goes looking through power levels. +func (r *API) JoinRoom(ctx context.Context, req JoinRequest) (string, error) { + if !r.isLocal(req.User) { + return "", fmt.Errorf("%w: %s is not a user of this server", ErrForbidden, req.User) + } + + switch _, err := r.store.Room(ctx, req.RoomID); { + case errors.Is(err, ErrNoSuchRoom): + if !r.federationEnabled() { + return "", fmt.Errorf("%w: %s is not a room on this server, and joining a remote room", + ErrFederationDisabled, req.RoomID) + } + // With federation on, a room we do not have still has to be + // fetched from a server that does. That is Phase 5. + return "", fmt.Errorf("%w: joining a room this server does not have is not implemented yet", + ErrFederationDisabled) + case err != nil: + return "", err + } + + // Joining a room you are already in is a no-op rather than a second + // event: clients call /join to make sure, and every extra membership + // event would show up in everybody's timeline. + membership, err := r.store.Membership(ctx, req.RoomID, req.User) + if err != nil { + return "", err + } + if membership.Membership == event.MembershipJoin { + return req.RoomID, nil + } + + content, err := r.joinContent(ctx, req) + if err != nil { + return "", err + } + if _, err := r.send(ctx, sendRequest{ + RoomID: req.RoomID, + Type: event.TypeMember, + StateKey: statePtr(req.User), + Sender: req.User, + Content: content, + }); err != nil { + return "", err + } + return req.RoomID, nil +} + +// joinContent builds the m.room.member content of a join, including the +// authorisation a restricted room needs. +func (r *API) joinContent(ctx context.Context, req JoinRequest) (json.RawMessage, error) { + content := r.membershipContentMap(ctx, event.MembershipJoin, req.User, req.Reason, false) + + authorisedVia, err := r.restrictedJoinAuthoriser(ctx, req) + if err != nil { + return nil, err + } + if authorisedVia != "" { + content["join_authorised_via_users_server"] = authorisedVia + } + return mustMarshal(content), nil +} + +// restrictedJoinAuthoriser finds a local user who can authorise a join +// under the restricted join rule, or returns "" when the rule does not +// apply. +// +// The restricted join rule lets membership of another room stand in for an +// invite, but somebody has to attest that the joining user really is in one +// of the allowed rooms. That attestation is a signature from the authorising +// user's server over the membership event, so the authoriser has to be a +// user this server can sign for and who has the power to invite. While +// federation is off every room is local, so this server can always see the +// allowed rooms itself. +func (r *API) restrictedJoinAuthoriser(ctx context.Context, req JoinRequest) (string, error) { + room, err := r.store.Room(ctx, req.RoomID) + if err != nil { + return "", err + } + rules, err := r.rules(room) + if err != nil { + return "", err + } + if !rules.RestrictedJoinRule { + return "", nil + } + + st, err := r.currentState(ctx, room, rules) + if err != nil { + return "", err + } + switch st.JoinRule() { + case auth.JoinRuleRestricted: + case auth.JoinRuleKnockRestricted: + if !rules.KnockRestrictedJoinRule { + return "", nil + } + default: + return "", nil + } + + // An invited user joins on the invite, with no authorisation needed, + // and the rules check the invite first. + if st.Membership(req.User) == event.MembershipInvite { + return "", nil + } + + allowed, err := allowedRooms(st) + if err != nil { + return "", err + } + inAnAllowedRoom := false + for _, allowedRoom := range allowed { + membership, err := r.store.Membership(ctx, allowedRoom, req.User) + if err != nil { + return "", err + } + if membership.Membership == event.MembershipJoin { + inAnAllowedRoom = true + break + } + } + if !inAnAllowedRoom { + return "", fmt.Errorf("%w: %s is not joined to any room that grants entry to %s", + ErrForbidden, req.User, req.RoomID) + } + + return r.localUserWhoCanInvite(st, req.RoomID) +} + +// allowedRooms reads the room IDs an m.room.join_rules event grants entry +// through. +func allowedRooms(st auth.State) ([]string, error) { + joinRules := st.JoinRules() + if joinRules == nil { + return nil, nil + } + var content struct { + Allow []struct { + Type string `json:"type"` + RoomID string `json:"room_id"` + } `json:"allow"` + } + if err := json.Unmarshal(joinRules.Content(), &content); err != nil { + return nil, fmt.Errorf("%w: the room's join rules are unreadable: %v", ErrRejected, err) + } + var out []string + for _, entry := range content.Allow { + // "m.room_membership" is the only rule type the spec defines; an + // unknown one grants nothing rather than everything. + if entry.Type == "m.room_membership" && entry.RoomID != "" { + out = append(out, entry.RoomID) + } + } + return out, nil +} + +// localUserWhoCanInvite picks a joined local user with the power to invite, +// to authorise a restricted join. +// +// Which user it is does not matter to the rules — only that they are in the +// room, can invite, and belong to a server that will sign for them. The +// room's creators come first because they always qualify. +func (r *API) localUserWhoCanInvite(st auth.State, roomID string) (string, error) { + create := st.Create() + if create == nil { + // A room with no create event in its current state is corrupt, and + // nothing here can judge anything about it. + return "", fmt.Errorf("%w: %s has no create event in its state", ErrRejected, roomID) + } + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), create, create.Rules()) + if err != nil { + return "", fmt.Errorf("%w: %v", ErrRejected, err) + } + + creators, err := auth.Creators(create) + if err != nil { + return "", fmt.Errorf("%w: %v", ErrRejected, err) + } + for _, creator := range creators { + if r.isLocal(creator) && st.Membership(creator) == event.MembershipJoin { + return creator, nil + } + } + + for key, member := range st { + if key.Type != event.TypeMember || !r.isLocal(key.StateKey) { + continue + } + if membership, err := member.Membership(); err != nil || membership != event.MembershipJoin { + continue + } + if powerLevels.UserLevel(key.StateKey) >= powerLevels.Invite { + return key.StateKey, nil + } + } + return "", fmt.Errorf("%w: no user of this server in %s can authorise a restricted join", + ErrForbidden, roomID) +} + +// LeaveRoom leaves a room, or rejects an invite to one. +// +// Both are the same event: an m.room.member with membership "leave" sent by +// the user themselves. Whether they were joined, invited or knocking is the +// authorisation rules' business, and a user who was never in the room is +// refused by them. +func (r *API) LeaveRoom(ctx context.Context, roomID, userID, reason string) error { + _, err := r.send(ctx, sendRequest{ + RoomID: roomID, + Type: event.TypeMember, + StateKey: statePtr(userID), + Sender: userID, + Content: r.membershipContent(ctx, event.MembershipLeave, userID, reason, false), + }) + return err +} + +// InviteRequest asks to invite a user to a room. +type InviteRequest struct { + RoomID string + + // Sender is the full user ID doing the inviting. + Sender string + + // Target is the full user ID being invited. + Target string + + Reason string + + // IsDirect marks the invite as a direct chat. + IsDirect bool +} + +// InviteUser invites a user to a room. +// +// Inviting a user of another server means asking that server to +// countersign the invite, which needs federation. With it off the invite is +// refused with a message that says why rather than a bare M_FORBIDDEN — +// PLAN.md section 3. +func (r *API) InviteUser(ctx context.Context, req InviteRequest) error { + if err := validateUserID(req.Target); err != nil { + return err + } + if !r.isLocal(req.Target) { + if !r.federationEnabled() { + return fmt.Errorf("%w: %s is on another server, and inviting a remote user", + ErrFederationDisabled, req.Target) + } + return fmt.Errorf("%w: inviting a user of another server is not implemented yet", + ErrFederationDisabled) + } + + _, err := r.send(ctx, sendRequest{ + RoomID: req.RoomID, + Type: event.TypeMember, + StateKey: statePtr(req.Target), + Sender: req.Sender, + Content: r.membershipContent(ctx, event.MembershipInvite, req.Target, req.Reason, req.IsDirect), + }) + return err +} + +// KickUser removes a user from a room without barring them from returning. +// +// A kick is a membership change to "leave" made by somebody other than the +// user. It works on an invited or knocking user too, which is how an invite +// is withdrawn or a knock refused. +// +// A target who is not in the room at all is refused here rather than by the +// rules, which would accept a "leave" over a "leave" and add an event to +// the room that says nothing — and that every member would then see in +// their timeline. Lifting a ban is /unban, not a kick. +func (r *API) KickUser(ctx context.Context, roomID, sender, target, reason string) error { + if err := validateUserID(target); err != nil { + return err + } + membership, err := r.store.Membership(ctx, roomID, target) + if err != nil { + return err + } + switch membership.Membership { + case event.MembershipJoin, event.MembershipInvite, event.MembershipKnock: + default: + return fmt.Errorf("%w: %s is not in %s", ErrForbidden, target, roomID) + } + + _, err = r.send(ctx, sendRequest{ + RoomID: roomID, + Type: event.TypeMember, + StateKey: statePtr(target), + Sender: sender, + Content: r.membershipContent(ctx, event.MembershipLeave, target, reason, false), + }) + return err +} + +// BanUser bars a user from a room. +func (r *API) BanUser(ctx context.Context, roomID, sender, target, reason string) error { + if err := validateUserID(target); err != nil { + return err + } + _, err := r.send(ctx, sendRequest{ + RoomID: roomID, + Type: event.TypeMember, + StateKey: statePtr(target), + Sender: sender, + Content: r.membershipContent(ctx, event.MembershipBan, target, reason, false), + }) + return err +} + +// UnbanUser lifts a ban, which leaves the user outside the room rather than +// in it. +func (r *API) UnbanUser(ctx context.Context, roomID, sender, target, reason string) error { + if err := validateUserID(target); err != nil { + return err + } + + // Unbanning someone who is not banned is refused here rather than by + // the rules, which would happily accept a "leave" over a "leave" and + // produce an event that says nothing. + membership, err := r.store.Membership(ctx, roomID, target) + if err != nil { + return err + } + if membership.Membership != event.MembershipBan { + return fmt.Errorf("%w: %s is not banned from %s", ErrForbidden, target, roomID) + } + + _, err = r.send(ctx, sendRequest{ + RoomID: roomID, + Type: event.TypeMember, + StateKey: statePtr(target), + Sender: sender, + Content: r.membershipContent(ctx, event.MembershipLeave, target, reason, false), + }) + return err +} + +// ForceLeave makes a user leave a room without any authorisation check, +// for the admin API. +// +// It is still an ordinary membership event, signed and added to the room, +// so the other members see what happened. What it skips is only the +// question of whether the user was allowed to leave — a user stuck in a +// broken room is the case this exists for, and in a broken room the rules +// are exactly what is in the way. +func (r *API) ForceLeave(ctx context.Context, roomID, userID string) error { + membership, err := r.store.Membership(ctx, roomID, userID) + if err != nil { + return err + } + switch membership.Membership { + case event.MembershipJoin, event.MembershipInvite, event.MembershipKnock: + default: + return fmt.Errorf("%w: %s is not in %s", ErrForbidden, userID, roomID) + } + return r.LeaveRoom(ctx, roomID, userID, "Removed by a server administrator") +} + +// membershipContent builds an m.room.member content. +func (r *API) membershipContent(ctx context.Context, membership, userID, reason string, isDirect bool) json.RawMessage { + return mustMarshal(r.membershipContentMap(ctx, membership, userID, reason, isDirect)) +} + +// membershipContentMap is membershipContent before encoding, so that a +// restricted join can add its authorisation to it. +// +// The profile is copied into the event on purpose: a client rendering a +// room's history has to show the name the member had at the time, and the +// membership event is the only record of it. A profile that cannot be read +// is left out rather than failing the membership change — being unable to +// join a room because the display name lookup failed would be the wrong +// trade. +func (r *API) membershipContentMap(ctx context.Context, membership, userID, reason string, isDirect bool) map[string]any { + content := map[string]any{"membership": membership} + if reason != "" { + content["reason"] = reason + } + if isDirect { + content["is_direct"] = true + } + + // A ban or a kick says nothing about the target's profile: the event is + // about the sender's decision, and copying a display name into it would + // only give a moderator a way to write into somebody else's membership. + if membership != event.MembershipJoin && membership != event.MembershipInvite { + return content + } + if r.profiles == nil { + return content + } + localpart, ok := r.localpartOf(userID) + if !ok { + return content + } + account, err := r.profiles.Account(ctx, localpart) + if err != nil { + if !errors.Is(err, storage.ErrNoSuchUser) { + r.log.Warn("could not read a profile for a membership event", + "user_id", userID, "error", err) + } + return content + } + if account.DisplayName != "" { + content["displayname"] = account.DisplayName + } + if account.AvatarURL != "" { + content["avatar_url"] = account.AvatarURL + } + return content +} + +// validateUserID checks a user ID arriving from a client against the +// grammar events use, which is laxer than the one this server's own +// accounts are held to: a room can contain user IDs minted years ago. +func validateUserID(userID string) error { + if err := id.ValidateUserID(userID); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidContent, err) + } + return nil +} diff --git a/server/internal/roomserver/membership_test.go b/server/internal/roomserver/membership_test.go new file mode 100644 index 0000000..4312f9d --- /dev/null +++ b/server/internal/roomserver/membership_test.go @@ -0,0 +1,530 @@ +package roomserver_test + +import ( + "encoding/json" + "errors" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/roomserver" +) + +func TestJoinPublicRoom(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + joined, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, + User: bob, + }) + if err != nil { + t.Fatalf("JoinRoom: %v", err) + } + if joined != resp.RoomID { + t.Errorf("JoinRoom returned %q, want %q", joined, resp.RoomID) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipJoin { + t.Errorf("bob's membership is %q, want join", got) + } + + // A second join is a no-op, not a second event: clients call /join to + // make sure, and every extra membership event lands in everybody's + // timeline. + before, err := f.store.Event(t.Context(), resp.RoomID, f.memberEventID(resp.RoomID, bob)) + if err != nil { + t.Fatalf("Event: %v", err) + } + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("joining twice: %v", err) + } + after, err := f.store.Event(t.Context(), resp.RoomID, f.memberEventID(resp.RoomID, bob)) + if err != nil { + t.Fatalf("Event: %v", err) + } + if after.EventID != before.EventID { + t.Errorf("joining twice produced a second membership event") + } + + // The joined-member count follows. + summary, err := f.store.RoomSummary(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.JoinedMembers != 2 { + t.Errorf("the room has %d joined members, want 2", summary.JoinedMembers) + } +} + +// memberEventID is the event ID of a user's current membership event. +func (f *fixture) memberEventID(roomID, userID string) string { + f.t.Helper() + entry, err := f.store.Membership(f.t.Context(), roomID, userID) + if err != nil { + f.t.Fatalf("Membership: %v", err) + } + return entry.EventID +} + +func TestJoinInviteOnlyRoom(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPrivateChat}) + + // Without an invite the rules refuse the join. The refusal comes from + // them, not from a check in this package. + _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Fatalf("joining an invite-only room = %v, want ErrRejected", err) + } + + if err := f.api.InviteUser(t.Context(), roomserver.InviteRequest{ + RoomID: resp.RoomID, Sender: alice, Target: bob, + }); err != nil { + t.Fatalf("InviteUser: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipInvite { + t.Errorf("bob's membership after the invite is %q, want invite", got) + } + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("joining after an invite: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipJoin { + t.Errorf("bob's membership after joining is %q, want join", got) + } +} + +// TestInviteRequiresPower is the rule a room's moderators rely on. +func TestInviteRequiresPower(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + + // The default invite level is 0, so bob can invite. Raise it and he + // cannot. + if _, err := f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: event.TypePowerLevels, + StateKey: stringPtr(""), + Content: json.RawMessage(`{"invite":50,"users":{"` + alice + `":100}}`), + }); err != nil { + t.Fatalf("raising the invite level: %v", err) + } + + err := f.api.InviteUser(t.Context(), roomserver.InviteRequest{ + RoomID: resp.RoomID, Sender: bob, Target: carol, + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("an invite without the power = %v, want ErrRejected", err) + } +} + +func TestLeaveAndRejectInvite(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + if err := f.api.LeaveRoom(t.Context(), resp.RoomID, bob, "bye"); err != nil { + t.Fatalf("LeaveRoom: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipLeave { + t.Errorf("bob's membership after leaving is %q, want leave", got) + } + + // The same endpoint rejects an invite, which is why it has to work on + // a room the caller was never joined to. + if err := f.api.InviteUser(t.Context(), roomserver.InviteRequest{ + RoomID: resp.RoomID, Sender: alice, Target: carol, + }); err != nil { + t.Fatalf("InviteUser: %v", err) + } + if err := f.api.LeaveRoom(t.Context(), resp.RoomID, carol, "no thanks"); err != nil { + t.Fatalf("rejecting an invite: %v", err) + } + if got := f.membership(resp.RoomID, carol); got != event.MembershipLeave { + t.Errorf("carol's membership after rejecting is %q, want leave", got) + } + + // Leaving a room you were never in is refused by the rules. + err := f.api.LeaveRoom(t.Context(), resp.RoomID, "@dave:"+testServerName, "") + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("leaving a room never joined = %v, want ErrRejected", err) + } +} + +func TestKickBanAndUnban(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + join := func(user string) { + t.Helper() + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: user, + }); err != nil { + t.Fatalf("%s joining: %v", user, err) + } + } + join(bob) + + if err := f.api.KickUser(t.Context(), resp.RoomID, alice, bob, "behave"); err != nil { + t.Fatalf("KickUser: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipLeave { + t.Errorf("bob's membership after the kick is %q, want leave", got) + } + // A kick is not a ban: bob can come back. + join(bob) + + if err := f.api.BanUser(t.Context(), resp.RoomID, alice, bob, "enough"); err != nil { + t.Fatalf("BanUser: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipBan { + t.Errorf("bob's membership after the ban is %q, want ban", got) + } + _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("a banned user joining = %v, want ErrRejected", err) + } + + if err := f.api.UnbanUser(t.Context(), resp.RoomID, alice, bob, ""); err != nil { + t.Fatalf("UnbanUser: %v", err) + } + // Unbanning leaves the user outside the room, not in it. + if got := f.membership(resp.RoomID, bob); got != event.MembershipLeave { + t.Errorf("bob's membership after the unban is %q, want leave", got) + } + join(bob) + + // Unbanning somebody who is not banned says so rather than writing an + // event that means nothing. + err = f.api.UnbanUser(t.Context(), resp.RoomID, alice, bob, "") + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("unbanning a member = %v, want ErrForbidden", err) + } +} + +// TestKickRequiresPower checks that a member cannot remove a peer. +func TestKickRequiresPower(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + for _, user := range []string{bob, carol} { + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: user, + }); err != nil { + t.Fatalf("%s joining: %v", user, err) + } + } + + err := f.api.KickUser(t.Context(), resp.RoomID, bob, carol, "") + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("a kick without the power = %v, want ErrRejected", err) + } +} + +// TestMembershipRemoteUsersNeedFederation is the behaviour PLAN.md section 3 +// asks for: a clear refusal rather than a bare M_FORBIDDEN. +func TestMembershipRemoteUsersNeedFederation(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + err := f.api.InviteUser(t.Context(), roomserver.InviteRequest{ + RoomID: resp.RoomID, Sender: alice, Target: "@eve:elsewhere.example", + }) + if !errors.Is(err, roomserver.ErrFederationDisabled) { + t.Errorf("inviting a remote user = %v, want ErrFederationDisabled", err) + } + + _, err = f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: "!elsewhere:elsewhere.example", User: alice, + }) + if !errors.Is(err, roomserver.ErrFederationDisabled) { + t.Errorf("joining a remote room = %v, want ErrFederationDisabled", err) + } + + // With federation on it is still not implemented, and says so — but it + // says it as a federation problem rather than as a missing room. + g := newFixture(t, func(opts *roomserver.Options) { + opts.FederationEnabled = func() bool { return true } + }) + _, err = g.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: "!elsewhere:elsewhere.example", User: alice, + }) + if !errors.Is(err, roomserver.ErrFederationDisabled) { + t.Errorf("joining a remote room with federation on = %v, want ErrFederationDisabled", err) + } +} + +// TestRestrictedJoin is the one authorisation rule that needs a signature +// checked, and so the one that needs this server to vouch for a join. +func TestRestrictedJoin(t *testing.T) { + f := newFixture(t, nil) + + // The space whose membership grants entry. + space := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + restricted := f.createRoom(roomserver.CreateRoomRequest{ + Version: "11", + InitialState: []roomserver.StateEvent{{ + Type: event.TypeJoinRules, + Content: json.RawMessage(`{"join_rule":"restricted","allow":[` + + `{"type":"m.room_membership","room_id":"` + space.RoomID + `"}]}`), + }}, + }) + + // Not in the space: no entry. + _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: restricted.RoomID, User: bob, + }) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Fatalf("joining a restricted room from outside = %v, want ErrForbidden", err) + } + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: space.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining the space: %v", err) + } + + // In the space: the join is authorised via a local user with the power + // to invite, and the event carries their server's signature. + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: restricted.RoomID, User: bob, + }); err != nil { + t.Fatalf("joining a restricted room from the space: %v", err) + } + if got := f.membership(restricted.RoomID, bob); got != event.MembershipJoin { + t.Errorf("bob's membership is %q, want join", got) + } + + st := f.state(restricted.RoomID) + member := st.Member(bob) + if member == nil { + t.Fatal("bob has no membership event") + } + var content struct { + AuthorisedVia string `json:"join_authorised_via_users_server"` + } + if err := json.Unmarshal(member.Content(), &content); err != nil { + t.Fatalf("the membership content is not an object: %v", err) + } + if content.AuthorisedVia != alice { + t.Errorf("join_authorised_via_users_server = %q, want %q", content.AuthorisedVia, alice) + } +} + +func TestJoinedRoomsAndMembers(t *testing.T) { + f := newFixture(t, nil) + first := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + second := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: first.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + + rooms, err := f.api.JoinedRooms(t.Context(), bob) + if err != nil { + t.Fatalf("JoinedRooms: %v", err) + } + if len(rooms) != 1 || rooms[0] != first.RoomID { + t.Errorf("bob's joined rooms = %v, want [%s]", rooms, first.RoomID) + } + + aliceRooms, err := f.api.JoinedRooms(t.Context(), alice) + if err != nil { + t.Fatalf("JoinedRooms: %v", err) + } + if len(aliceRooms) != 2 { + t.Errorf("alice is joined to %v, want both rooms", aliceRooms) + } + + joined, err := f.api.JoinedMembers(t.Context(), first.RoomID, alice) + if err != nil { + t.Fatalf("JoinedMembers: %v", err) + } + if len(joined) != 2 { + t.Fatalf("JoinedMembers returned %d members, want 2: %+v", len(joined), joined) + } + if joined[bob].DisplayName != "Bob" { + t.Errorf("bob's display name is %q, want Bob", joined[bob].DisplayName) + } + + // /joined_members is for members: a stranger is refused. + if _, err := f.api.JoinedMembers(t.Context(), second.RoomID, bob); !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("JoinedMembers for a stranger = %v, want ErrForbidden", err) + } + + // /members includes the users who have left, unless filtered. + if err := f.api.LeaveRoom(t.Context(), first.RoomID, bob, ""); err != nil { + t.Fatalf("LeaveRoom: %v", err) + } + all, err := f.api.Members(t.Context(), roomserver.MembersRequest{ + RoomID: first.RoomID, User: alice, + }) + if err != nil { + t.Fatalf("Members: %v", err) + } + if len(all) != 2 { + t.Errorf("Members returned %d events, want 2 (one of them a leave)", len(all)) + } + onlyJoined, err := f.api.Members(t.Context(), roomserver.MembersRequest{ + RoomID: first.RoomID, User: alice, Membership: event.MembershipJoin, + }) + if err != nil { + t.Fatalf("Members with a filter: %v", err) + } + if len(onlyJoined) != 1 { + t.Errorf("filtering by join returned %d events, want 1", len(onlyJoined)) + } + notJoined, err := f.api.Members(t.Context(), roomserver.MembersRequest{ + RoomID: first.RoomID, User: alice, NotMembership: event.MembershipJoin, + }) + if err != nil { + t.Fatalf("Members with not_membership: %v", err) + } + if len(notJoined) != 1 { + t.Errorf("not_membership=join returned %d events, want 1", len(notJoined)) + } +} + +// TestStateVisibility is the room-level visibility check the read endpoints +// make: a member sees the room, a stranger does not, and anyone sees a +// world-readable one. +func TestStateVisibility(t *testing.T) { + f := newFixture(t, nil) + private := f.createRoom(roomserver.CreateRoomRequest{}) + + if _, err := f.api.RoomState(t.Context(), private.RoomID, alice); err != nil { + t.Errorf("the creator cannot read their own room's state: %v", err) + } + if _, err := f.api.RoomState(t.Context(), private.RoomID, bob); !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("a stranger reading state = %v, want ErrForbidden", err) + } + + readable := f.createRoom(roomserver.CreateRoomRequest{ + InitialState: []roomserver.StateEvent{{ + Type: event.TypeHistoryVisibility, + Content: json.RawMessage(`{"history_visibility":"world_readable"}`), + }}, + }) + if _, err := f.api.RoomState(t.Context(), readable.RoomID, bob); err != nil { + t.Errorf("a stranger cannot read a world-readable room's state: %v", err) + } +} + +// TestForceLeave is the admin escape hatch: it adds a real membership +// event, so the room's other members see what happened. +func TestForceLeave(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + if err := f.api.ForceLeave(t.Context(), resp.RoomID, bob); err != nil { + t.Fatalf("ForceLeave: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipLeave { + t.Errorf("bob's membership is %q, want leave", got) + } + + // A user who is not in the room has nothing to leave. + if err := f.api.ForceLeave(t.Context(), resp.RoomID, carol); !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("force-leaving a stranger = %v, want ErrForbidden", err) + } +} + +// TestPowerLevelsCannotBeRaisedAboveYourOwn is the rule that keeps a +// moderator from promoting themselves. +func TestPowerLevelsCannotBeRaisedAboveYourOwn(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Version: "11", + Preset: roomserver.PresetPublicChat, + PowerLevelContentOverride: json.RawMessage( + `{"users":{"` + alice + `":100,"` + bob + `":50}}`), + }) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + + _, err := f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: bob, + Type: event.TypePowerLevels, + StateKey: stringPtr(""), + Content: json.RawMessage(`{"users":{"` + alice + `":100,"` + bob + `":100}}`), + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("promoting yourself = %v, want ErrRejected", err) + } + + st := f.state(resp.RoomID) + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), st.Create(), st.Create().Rules()) + if err != nil { + t.Fatalf("ResolvePowerLevels: %v", err) + } + if got := powerLevels.UserLevel(bob); got != 50 { + t.Errorf("bob's power level is %d, want 50", got) + } +} + +// TestKickRequiresTheTargetToBeInTheRoom keeps a kick from adding a leave +// event for somebody who was never there — which the authorisation rules +// would accept, and which every member of the room would then see. +func TestKickRequiresTheTargetToBeInTheRoom(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + err := f.api.KickUser(t.Context(), resp.RoomID, alice, bob, "") + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("kicking a stranger = %v, want ErrForbidden", err) + } + + // An invite can be withdrawn with a kick, which is what the endpoint + // is for as much as removing a member is. + if err := f.api.InviteUser(t.Context(), roomserver.InviteRequest{ + RoomID: resp.RoomID, Sender: alice, Target: bob, + }); err != nil { + t.Fatalf("InviteUser: %v", err) + } + if err := f.api.KickUser(t.Context(), resp.RoomID, alice, bob, ""); err != nil { + t.Fatalf("withdrawing an invite with a kick: %v", err) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipLeave { + t.Errorf("bob's membership is %q, want leave", got) + } + + // Lifting a ban is /unban, not a kick. + if err := f.api.BanUser(t.Context(), resp.RoomID, alice, bob, ""); err != nil { + t.Fatalf("BanUser: %v", err) + } + if err := f.api.KickUser(t.Context(), resp.RoomID, alice, bob, ""); !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("kicking a banned user = %v, want ErrForbidden", err) + } +} diff --git a/server/internal/roomserver/profile.go b/server/internal/roomserver/profile.go new file mode 100644 index 0000000..35272b6 --- /dev/null +++ b/server/internal/roomserver/profile.go @@ -0,0 +1,106 @@ +package roomserver + +import ( + "encoding/json" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// The state event types the Client-Server API gives a room its public face +// with. Only these are in the spec's "stripped state" and in a public room +// directory entry, which is why only these are denormalised. +const ( + TypeName = "m.room.name" + TypeTopic = "m.room.topic" + TypeAvatar = "m.room.avatar" + TypeCanonicalAlias = "m.room.canonical_alias" + TypeGuestAccess = "m.room.guest_access" + TypeEncryption = "m.room.encryption" + TypeMessage = "m.room.message" + TypeHistoryVisibility = event.TypeHistoryVisibility +) + +// History visibility values, in the order the spec lists them. +const ( + HistoryVisibilityWorldReadable = "world_readable" + HistoryVisibilityShared = "shared" + HistoryVisibilityInvited = "invited" + HistoryVisibilityJoined = "joined" +) + +// Guest access values. +const ( + GuestAccessCanJoin = "can_join" + GuestAccessForbidden = "forbidden" +) + +// affectsRoomProfile reports whether an event of this type changes the +// denormalised room listing columns. +func affectsRoomProfile(eventType string) bool { + switch eventType { + case event.TypeCreate, event.TypeJoinRules, TypeName, TypeTopic, + TypeAvatar, TypeCanonicalAlias, TypeHistoryVisibility, TypeGuestAccess: + return true + } + return false +} + +// roomProfile reads the denormalised listing columns out of a room's state. +// +// Every field is taken from the state rather than accumulated, so a profile +// recomputed after any one of these events is the whole truth about the +// room and not a patch that could drift from it. +func roomProfile(st auth.State) storage.RoomProfile { + profile := storage.RoomProfile{ + JoinRule: st.JoinRule(), + // These two have spec defaults for a room that never set them, and + // the defaults are what a client would apply, so they are what the + // stored row should say. + HistoryVisibility: HistoryVisibilityShared, + GuestAccess: GuestAccessForbidden, + } + + profile.Name = stringFromContent(st, TypeName, "", "name") + profile.Topic = stringFromContent(st, TypeTopic, "", "topic") + profile.AvatarURL = stringFromContent(st, TypeAvatar, "", "url") + profile.CanonicalAlias = stringFromContent(st, TypeCanonicalAlias, "", "alias") + profile.RoomType = stringFromContent(st, event.TypeCreate, "", "type") + + if v := stringFromContent(st, TypeHistoryVisibility, "", "history_visibility"); v != "" { + profile.HistoryVisibility = v + } + if v := stringFromContent(st, TypeGuestAccess, "", "guest_access"); v != "" { + profile.GuestAccess = v + } + return profile +} + +// stringFromContent reads one string property out of a state event's +// content, or returns "" when the event, the property or the type is not +// what it should be. +// +// A malformed value is treated as absent on purpose. This is a projection +// for a room listing, not an authorisation input: the right behaviour for a +// room whose name is a number is to show it without a name, not to fail +// every read of the room. +func stringFromContent(st auth.State, eventType, stateKey, property string) string { + e := st.Get(eventType, stateKey) + if e == nil { + return "" + } + var content map[string]json.RawMessage + if err := json.Unmarshal(e.Content(), &content); err != nil { + return "" + } + raw, ok := content[property] + if !ok { + return "" + } + var value string + if err := json.Unmarshal(raw, &value); err != nil { + return "" + } + return value +} diff --git a/server/internal/roomserver/query.go b/server/internal/roomserver/query.go new file mode 100644 index 0000000..2e4736e --- /dev/null +++ b/server/internal/roomserver/query.go @@ -0,0 +1,417 @@ +package roomserver + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// currentState reads a room's current state as the authorisation rules see +// it. +func (r *API) currentState(ctx context.Context, room storage.Room, rules roomversion.Rules) (auth.State, error) { + stored, err := r.store.CurrentStateEvents(ctx, room.ID) + if err != nil { + return nil, err + } + events, err := parseEvents(stored, rules) + if err != nil { + return nil, err + } + return auth.NewState(events), nil +} + +// ClientEvent is an event in the shape the Client-Server API returns it. +// +// It is not the event as it was signed. A client is given the event ID +// explicitly — in every room version since 3 it is a hash, not a property — +// and is not given the hashes, signatures, auth_events or depth, which are +// a server's business. What it does get that the signed event never carried +// is `unsigned`, which is per-server annotation: the age of the event, the +// state it replaced, and the redaction that stripped it. +type ClientEvent struct { + EventID string `json:"event_id"` + RoomID string `json:"room_id"` + Type string `json:"type"` + StateKey *string `json:"state_key,omitempty"` + Sender string `json:"sender"` + Content json.RawMessage `json:"content"` + OriginServerTS int64 `json:"origin_server_ts"` + Unsigned json.RawMessage `json:"unsigned,omitempty"` + + // Redacts is only on an m.room.redaction event, and only in the room + // versions that keep it at the top level. Clients still read it there. + Redacts string `json:"redacts,omitempty"` +} + +// clientEvent projects a stored event into the client shape. +func (r *API) clientEvent(stored storage.StoredEvent, e *event.Event, roomID string) ClientEvent { + out := ClientEvent{ + EventID: e.EventID(), + RoomID: roomID, + Type: e.Type(), + Sender: e.Sender(), + Content: e.Content(), + OriginServerTS: e.OriginServerTS(), + Redacts: e.Redacts(), + } + if stateKey, ok := e.StateKey(); ok { + out.StateKey = &stateKey + } + out.Unsigned = r.unsigned(stored, e) + return out +} + +// unsigned builds the `unsigned` block a client is given. +// +// `age` is computed on the way out rather than stored: it is how long ago +// the event happened as far as this server can tell, and a value written +// down at persist time would be wrong by however long the event sat in the +// database. +func (r *API) unsigned(stored storage.StoredEvent, e *event.Event) json.RawMessage { + block := map[string]json.RawMessage{} + if existing := e.Unsigned(); len(existing) > 0 { + if err := json.Unmarshal(existing, &block); err != nil { + // The unsigned block is this server's own annotation and is + // written as an object everywhere; anything else is a bug here + // rather than something a client sent, and dropping it is + // better than failing the read. + r.log.Warn("stored event has an unreadable unsigned block", "event_id", e.EventID()) + block = map[string]json.RawMessage{} + } + } + if age := r.nowMS() - e.OriginServerTS(); age >= 0 { + block["age"] = mustMarshal(age) + } + if stored.RedactedBy != "" { + block["redacted_because"] = mustMarshal(map[string]any{"event_id": stored.RedactedBy}) + } + if len(block) == 0 { + return nil + } + return mustMarshal(block) +} + +// Event returns one event of a room, if the user may see it. +func (r *API) Event(ctx context.Context, roomID, userID, eventID string) (ClientEvent, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return ClientEvent{}, err + } + if err := r.requireVisibility(ctx, room, rules, userID); err != nil { + return ClientEvent{}, err + } + + stored, err := r.store.Event(ctx, roomID, eventID) + if err != nil { + return ClientEvent{}, err + } + e, err := parseEvent(stored, rules) + if err != nil { + return ClientEvent{}, err + } + return r.clientEvent(stored, e, roomID), nil +} + +// RoomState returns a room's current state in the client shape. +func (r *API) RoomState(ctx context.Context, roomID, userID string) ([]ClientEvent, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return nil, err + } + if err := r.requireVisibility(ctx, room, rules, userID); err != nil { + return nil, err + } + + stored, err := r.store.CurrentStateEvents(ctx, roomID) + if err != nil { + return nil, err + } + out := make([]ClientEvent, 0, len(stored)) + for _, s := range stored { + e, err := parseEvent(s, rules) + if err != nil { + return nil, err + } + out = append(out, r.clientEvent(s, e, roomID)) + } + return out, nil +} + +// StateEventContent returns the content of one piece of a room's current +// state, which is what GET /rooms/{roomID}/state/{type}/{key} answers. +// +// The endpoint returns the content alone, not the event: a quirk of the +// Client-Server API that predates the rest of it, and one clients depend on. +func (r *API) StateEventContent(ctx context.Context, roomID, userID, eventType, stateKey string) (json.RawMessage, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return nil, err + } + if err := r.requireVisibility(ctx, room, rules, userID); err != nil { + return nil, err + } + + stored, err := r.store.CurrentStateEvent(ctx, roomID, eventType, stateKey) + if err != nil { + return nil, err + } + e, err := parseEvent(stored, rules) + if err != nil { + return nil, err + } + return e.Content(), nil +} + +// StateEvent returns one piece of a room's current state as a whole event. +// +// It is what ?format=event asks for on the state endpoint, and the only way +// a client can see who set a piece of state and when. +func (r *API) StateEvent(ctx context.Context, roomID, userID, eventType, stateKey string) (ClientEvent, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return ClientEvent{}, err + } + if err := r.requireVisibility(ctx, room, rules, userID); err != nil { + return ClientEvent{}, err + } + + stored, err := r.store.CurrentStateEvent(ctx, roomID, eventType, stateKey) + if err != nil { + return ClientEvent{}, err + } + e, err := parseEvent(stored, rules) + if err != nil { + return ClientEvent{}, err + } + return r.clientEvent(stored, e, roomID), nil +} + +// MembersRequest filters GET /rooms/{roomID}/members. +type MembersRequest struct { + RoomID string + + // User is the full user ID asking. + User string + + // Membership and NotMembership filter the result; either may be empty. + Membership string + NotMembership string +} + +// Members returns a room's m.room.member events. +func (r *API) Members(ctx context.Context, req MembersRequest) ([]ClientEvent, error) { + room, rules, err := r.roomAndRules(ctx, req.RoomID) + if err != nil { + return nil, err + } + if err := r.requireVisibility(ctx, room, rules, req.User); err != nil { + return nil, err + } + + var filter []string + if req.Membership != "" { + filter = append(filter, req.Membership) + } + stored, err := r.store.Members(ctx, req.RoomID, filter) + if err != nil { + return nil, err + } + + out := make([]ClientEvent, 0, len(stored)) + for _, s := range stored { + e, err := parseEvent(s, rules) + if err != nil { + return nil, err + } + if req.NotMembership != "" { + membership, err := e.Membership() + if err == nil && membership == req.NotMembership { + continue + } + } + out = append(out, r.clientEvent(s, e, req.RoomID)) + } + return out, nil +} + +// RoomMember is one joined member, as /joined_members reports them. +type RoomMember struct { + DisplayName string `json:"display_name,omitempty"` + AvatarURL string `json:"avatar_url,omitempty"` +} + +// JoinedMembers returns the joined members of a room with their profiles. +// +// Unlike /members this one is for members only: the spec says so, and the +// reason is that it exists to let a client that is in a room render it, not +// to let anyone enumerate who is in it. +func (r *API) JoinedMembers(ctx context.Context, roomID, userID string) (map[string]RoomMember, error) { + if err := r.requireJoined(ctx, roomID, userID); err != nil { + return nil, err + } + _, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return nil, err + } + + stored, err := r.store.Members(ctx, roomID, []string{event.MembershipJoin}) + if err != nil { + return nil, err + } + + out := make(map[string]RoomMember, len(stored)) + for _, s := range stored { + e, err := parseEvent(s, rules) + if err != nil { + return nil, err + } + stateKey, ok := e.StateKey() + if !ok { + continue + } + var content struct { + DisplayName string `json:"displayname"` + AvatarURL string `json:"avatar_url"` + } + // A member event with a malformed profile still counts as a member; + // the profile is simply not shown. + _ = json.Unmarshal(e.Content(), &content) + out[stateKey] = RoomMember{DisplayName: content.DisplayName, AvatarURL: content.AvatarURL} + } + return out, nil +} + +// JoinedRooms lists the rooms a user is joined to. +func (r *API) JoinedRooms(ctx context.Context, userID string) ([]string, error) { + rooms, err := r.store.RoomsForUser(ctx, userID, []string{event.MembershipJoin}) + if err != nil { + return nil, err + } + if rooms == nil { + rooms = []string{} + } + return rooms, nil +} + +// Membership returns a user's membership in a room, which is empty for a +// user who was never in it. +func (r *API) Membership(ctx context.Context, roomID, userID string) (string, error) { + entry, err := r.store.Membership(ctx, roomID, userID) + if err != nil { + return "", err + } + return entry.Membership, nil +} + +// roomAndRules reads a room and the rules its events are under. +func (r *API) roomAndRules(ctx context.Context, roomID string) (storage.Room, roomversion.Rules, error) { + room, err := r.store.Room(ctx, roomID) + if err != nil { + return storage.Room{}, roomversion.Rules{}, err + } + rules, err := r.rules(room) + if err != nil { + return storage.Room{}, roomversion.Rules{}, err + } + return room, rules, nil +} + +// requireJoined refuses a caller who is not currently in the room. +func (r *API) requireJoined(ctx context.Context, roomID, userID string) error { + entry, err := r.store.Membership(ctx, roomID, userID) + if err != nil { + return err + } + if entry.Membership != event.MembershipJoin { + return fmt.Errorf("%w: %s is not joined to %s", ErrForbidden, userID, roomID) + } + return nil +} + +// requireVisibility refuses a caller who may not see a room's state. +// +// This is a room-level test, not the per-event history visibility the spec +// defines. A member sees the room; so does anyone at all when the room's +// history is world_readable; a user who has left sees the state as it was +// when they left, which is their own membership event's snapshot and which +// this returns the current state for instead. +// +// That last case is the gap, and it is deliberate. Serving a departed +// member the state as of their leave event means answering every state read +// from a historical snapshot and filtering each event by the visibility in +// force when it was sent — which is the same machinery /messages and +// /context need, and so belongs with them in Phase 4 rather than being +// built twice. Until then a user who has left a room is refused, which errs +// towards showing them too little. +func (r *API) requireVisibility(ctx context.Context, room storage.Room, rules roomversion.Rules, userID string) error { + entry, err := r.store.Membership(ctx, room.ID, userID) + if err != nil { + return err + } + if entry.Membership == event.MembershipJoin { + return nil + } + + summary, err := r.store.RoomSummary(ctx, room.ID) + if err != nil { + return err + } + if summary.HistoryVisibility == HistoryVisibilityWorldReadable { + return nil + } + + // An invited user can see enough of the room to decide, which the spec + // delivers as stripped state on the invite rather than as a state read. + if entry.Membership == event.MembershipInvite { + return fmt.Errorf("%w: %s has been invited to %s but has not joined it", + ErrForbidden, userID, room.ID) + } + return fmt.Errorf("%w: %s is not in %s", ErrForbidden, userID, room.ID) +} + +// CanSendStateEvent reports whether a user may send a state event of a type +// into a room. +// +// It exists for the two endpoints that are *about* a room rather than in it +// — publishing the room in the directory, and deleting one of its aliases — +// where the spec's permission is "could set the canonical alias". Nothing +// that sends an event uses it: an event's permission is decided by the +// authorisation rules when it is sent, and asking twice would be two +// answers to maintain. +func (r *API) CanSendStateEvent(ctx context.Context, roomID, userID, eventType string) (bool, error) { + room, rules, err := r.roomAndRules(ctx, roomID) + if err != nil { + return false, err + } + st, err := r.currentState(ctx, room, rules) + if err != nil { + return false, err + } + if st.Membership(userID) != event.MembershipJoin { + return false, nil + } + create := st.Create() + if create == nil { + return false, fmt.Errorf("%w: %s has no create event in its state", ErrRejected, roomID) + } + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), create, rules) + if err != nil { + return false, fmt.Errorf("%w: %v", ErrRejected, err) + } + return powerLevels.UserLevel(userID) >= powerLevels.SendLevel(eventType, true), nil +} + +// RoomVersion returns a room's version, for the handful of callers that +// need to report it. +func (r *API) RoomVersion(ctx context.Context, roomID string) (string, error) { + room, err := r.store.Room(ctx, roomID) + if err != nil { + return "", err + } + return room.Version, nil +} diff --git a/server/internal/roomserver/redact_test.go b/server/internal/roomserver/redact_test.go new file mode 100644 index 0000000..6502b44 --- /dev/null +++ b/server/internal/roomserver/redact_test.go @@ -0,0 +1,235 @@ +package roomserver_test + +import ( + "encoding/json" + "errors" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// contentOf reads a stored event's content as a map. +func (f *fixture) contentOf(roomID, eventID string) map[string]any { + f.t.Helper() + stored, err := f.store.Event(f.t.Context(), roomID, eventID) + if err != nil { + f.t.Fatalf("Event: %v", err) + } + var body struct { + Content map[string]any `json:"content"` + } + if err := json.Unmarshal(stored.JSON, &body); err != nil { + f.t.Fatalf("the stored event is not an object: %v", err) + } + return body.Content +} + +func TestRedactOwnEvent(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + target, err := f.sendMessage(resp.RoomID, alice, "something regrettable", "m1") + if err != nil { + t.Fatalf("sending the message: %v", err) + } + if body := f.contentOf(resp.RoomID, target); body["body"] == nil { + t.Fatal("the message was stored without its body") + } + + redaction, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, + Sender: alice, + EventID: target, + Reason: "a mistake", + }) + if err != nil { + t.Fatalf("Redact: %v", err) + } + + // The event is stripped in place, and still has its original ID: the + // ID is the reference hash of the redacted form, so redacting an event + // does not rename it. + stored, err := f.store.Event(t.Context(), resp.RoomID, target) + if err != nil { + t.Fatalf("Event: %v", err) + } + if stored.RedactedBy != redaction { + t.Errorf("redacted_by = %q, want %q", stored.RedactedBy, redaction) + } + if body := f.contentOf(resp.RoomID, target); len(body) != 0 { + t.Errorf("the redacted event still has content: %v", body) + } + + // A client reading it back is told why it is empty. + clientEvent, err := f.api.Event(t.Context(), resp.RoomID, alice, target) + if err != nil { + t.Fatalf("Event: %v", err) + } + var unsigned struct { + RedactedBecause struct { + EventID string `json:"event_id"` + } `json:"redacted_because"` + } + if err := json.Unmarshal(clientEvent.Unsigned, &unsigned); err != nil { + t.Fatalf("the unsigned block is not an object: %v", err) + } + if unsigned.RedactedBecause.EventID != redaction { + t.Errorf("unsigned.redacted_because names %q, want %q", + unsigned.RedactedBecause.EventID, redaction) + } +} + +// TestRedactAnotherUsersEvent checks the second half of the spec's test +// under "Handling redactions", which is the one that actually fires on a +// server with no federation: a redaction applies when it comes from the +// same server as the event it redacts, whatever the sender's power level, +// because a server may always take back its own users' events. +// +// Every user here is on one server, so that rule covers every pair of them +// and the power-level branch cannot be reached. Exercising that branch +// needs an event from another server, which is Phase 5; the branch itself +// is tested in internal/matrix/auth. +func TestRedactAnotherUsersEvent(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Preset: roomserver.PresetPublicChat, + // carol is an ordinary member with no redact power, so if the + // redaction applies it is the same-server rule that did it. + PowerLevelContentOverride: json.RawMessage( + `{"users":{"` + alice + `":100},"redact":50}`), + }) + for _, user := range []string{bob, carol} { + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: user, + }); err != nil { + t.Fatalf("%s joining: %v", user, err) + } + } + + target, err := f.sendMessage(resp.RoomID, bob, "bob's message", "m1") + if err != nil { + t.Fatalf("sending the message: %v", err) + } + + redaction, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: carol, EventID: target, + }) + if err != nil { + t.Fatalf("Redact: %v", err) + } + stored, err := f.store.Event(t.Context(), resp.RoomID, target) + if err != nil { + t.Fatalf("Event: %v", err) + } + if stored.RedactedBy != redaction { + t.Errorf("redacted_by = %q, want %q", stored.RedactedBy, redaction) + } + if body := f.contentOf(resp.RoomID, target); len(body) != 0 { + t.Errorf("the redacted event still has content: %v", body) + } + + // A second redaction has nothing left to remove and must not rewrite + // which redaction stripped the event. + if _, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: alice, EventID: target, + }); err != nil { + t.Fatalf("redacting twice: %v", err) + } + stored, err = f.store.Event(t.Context(), resp.RoomID, target) + if err != nil { + t.Fatalf("Event: %v", err) + } + if stored.RedactedBy != redaction { + t.Errorf("a second redaction took over: redacted_by = %q, want %q", + stored.RedactedBy, redaction) + } +} + +// TestRedactStateEventKeepsItsKeys is what the redaction algorithm is for: +// stripping a membership event must not strip the membership, or the room +// would forget who is in it. +func TestRedactStateEventKeepsItsKeys(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + if _, err := f.api.JoinRoom(t.Context(), roomserver.JoinRequest{ + RoomID: resp.RoomID, User: bob, + }); err != nil { + t.Fatalf("bob joining: %v", err) + } + memberEvent := f.memberEventID(resp.RoomID, bob) + + if _, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: alice, EventID: memberEvent, + }); err != nil { + t.Fatalf("Redact: %v", err) + } + + body := f.contentOf(resp.RoomID, memberEvent) + if body["membership"] != event.MembershipJoin { + t.Errorf("the redacted membership event lost its membership: %v", body) + } + // The display name is not protected, so it goes. + if body["displayname"] != nil { + t.Errorf("the redacted membership event kept its display name: %v", body) + } + // Bob is still in the room. + if got := f.membership(resp.RoomID, bob); got != event.MembershipJoin { + t.Errorf("bob's membership after the redaction is %q, want join", got) + } +} + +func TestRedactRefusals(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + // An event that does not exist is a 404, not an event in the room + // that refers to nothing. + _, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: alice, EventID: "$nope", + }) + if !errors.Is(err, roomserver.ErrNoSuchEvent) { + t.Errorf("redacting an unknown event = %v, want ErrNoSuchEvent", err) + } + + // A non-member cannot send a redaction at all. + target, err := f.sendMessage(resp.RoomID, alice, "hello", "m1") + if err != nil { + t.Fatalf("sending the message: %v", err) + } + _, err = f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: bob, EventID: target, + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("a stranger redacting = %v, want ErrRejected", err) + } +} + +func TestRedactIsIdempotentPerTransaction(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + target, err := f.sendMessage(resp.RoomID, alice, "hello", "m1") + if err != nil { + t.Fatalf("sending the message: %v", err) + } + + ref := &storage.TransactionRef{Localpart: "alice", DeviceID: "DEV", TxnID: "r1"} + first, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: alice, EventID: target, Transaction: ref, + }) + if err != nil { + t.Fatalf("first redaction: %v", err) + } + again, err := f.api.Redact(t.Context(), roomserver.RedactRequest{ + RoomID: resp.RoomID, Sender: alice, EventID: target, Transaction: ref, + }) + if err != nil { + t.Fatalf("retried redaction: %v", err) + } + if again != first { + t.Errorf("a retried redaction produced a second event: %q then %q", first, again) + } +} diff --git a/server/internal/roomserver/roomserver.go b/server/internal/roomserver/roomserver.go new file mode 100644 index 0000000..48a2c6e --- /dev/null +++ b/server/internal/roomserver/roomserver.go @@ -0,0 +1,355 @@ +// Package roomserver owns rooms: it is the only thing in Warp that creates +// an event, and the only thing that decides whether one belongs in a room. +// +// Everything here runs through one pipeline — validate, authorise, resolve, +// persist, emit — because every event has to be judged the same way whether +// it came from a client, from `createRoom` or, from Phase 5, from another +// server. The rules themselves are not here: they are in internal/matrix, +// which is pure library code with no storage and no HTTP. This package is +// what connects those rules to a database and to a room's history. +// +// What the Client-Server API handlers get is a small set of operations — +// create a room, change a membership, send an event, redact one, read the +// state — each of which takes user IDs and content and returns an event ID +// or an error. Handlers do no authorisation of their own, and could not: a +// membership change is allowed or refused by the room's power levels and +// join rules, which only this package reads. +package roomserver + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "log/slog" + "sync" + "time" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// Errors callers have to tell apart. +// +// The storage and auth sentinels are re-exported rather than translated, so +// that errors.Is works whichever package a caller reaches for. +var ( + ErrNoSuchRoom = storage.ErrNoSuchRoom + ErrNoSuchEvent = storage.ErrNoSuchEvent + ErrNoSuchAlias = storage.ErrNoSuchAlias + ErrAliasExists = storage.ErrAliasExists + + // ErrRejected means the authorisation rules refused the event. The + // wrapped message names the rule that said no, and is safe to show a + // client: it says why the room would not accept what they sent. + ErrRejected = auth.ErrRejected + + // ErrForbidden is a refusal from a check that sits above the + // authorisation rules — not being in a room whose state you asked for, + // or deleting somebody else's alias. + ErrForbidden = errors.New("forbidden") + + // ErrFederationDisabled is an operation that would need to talk to + // another server while federation is off. + ErrFederationDisabled = errors.New("federation is disabled") + + // ErrInvalidContent is event content this server will not sign: not an + // object, too large, or missing something its type requires. + ErrInvalidContent = errors.New("invalid event content") + + // ErrUnsupportedRoomVersion is a room version Warp cannot take part in. + ErrUnsupportedRoomVersion = roomversion.ErrUnsupportedVersion + + // ErrUnknownRoomVersion is a room version that does not exist. + ErrUnknownRoomVersion = roomversion.ErrUnknownVersion +) + +// Profiles supplies the display name and avatar that a membership event +// carries a copy of. +// +// Membership events duplicate the profile on purpose: a client showing a +// room's history has to render the name the sender had at the time, and the +// member event is the only place that is recorded. *userapi.API satisfies +// this as it stands. +type Profiles interface { + Account(ctx context.Context, localpart string) (storage.Account, error) +} + +// Notification is one persisted event, as the emit stage reports it. +type Notification struct { + RoomID string + + // Event is the event as it was persisted. + Event *event.Event + + // StreamOrdering is the position the event was given in this server's + // event stream. + StreamOrdering int64 + + // StateChanged reports whether the room's current state moved. + StateChanged bool +} + +// Notifier is told about every event this server persists, in the order +// they were persisted. +// +// It is the emit stage of the input pipeline and the seam the sync API +// (PLAN.md phase 4) plugs into: a client long-polling /sync has to be woken +// the moment an event lands, and the input pipeline is the only place that +// knows one has. Phase 3 ships without a notifier, so this is nil and the +// stage is skipped. +type Notifier interface { + EventPersisted(ctx context.Context, n Notification) +} + +// API is the roomserver component. +type API struct { + serverName string + store storage.RoomStore + profiles Profiles + signingKey *signing.KeyPair + notifier Notifier + log *slog.Logger + now func() time.Time + + // federationEnabled reports the live state of the federation switch. + // Several operations are refused outright while it is off, and the + // answer has to be read per request because an admin can flip it. + federationEnabled func() bool + + // defaultRoomVersion is what createRoom uses when the client does not + // ask for one. + defaultRoomVersion roomversion.Version + + // locks serialises the writers of each room. Two events racing into one + // room would each pick up the same forward extremities and both become + // one, forking the room for no reason — and then needing state + // resolution to put it back together. Taking a lock per room instead + // keeps every locally created event a child of the one before it. + locks sync.Map // room ID -> *sync.Mutex +} + +// Options configure the component. +type Options struct { + // ServerName is the Matrix server name. It is the domain part of the + // user IDs this server may send events as, and of the room IDs and + // aliases it mints. + ServerName string + + // Store is the persistence layer. + Store storage.RoomStore + + // Profiles supplies the display name and avatar of a local user. + Profiles Profiles + + // SigningKey signs every event this server creates. + SigningKey *signing.KeyPair + + // DefaultRoomVersion overrides the registry's default for createRoom. + // Empty uses roomversion.Default. + DefaultRoomVersion string + + // FederationEnabled reports the live federation switch. Nil means off, + // which is the safe default for a half-wired server. + FederationEnabled func() bool + + // Notifier receives every persisted event. Nil skips the emit stage. + Notifier Notifier + + Logger *slog.Logger + + // Now is the clock, so that tests can stamp events deterministically. + Now func() time.Time +} + +// New builds the component. +func New(opts Options) (*API, error) { + switch { + case opts.ServerName == "": + return nil, errors.New("roomserver: ServerName must not be empty") + case opts.Store == nil: + return nil, errors.New("roomserver: Store must not be nil") + case opts.SigningKey == nil: + // Without a key nothing here can produce an event, and finding that + // out on the first createRoom rather than at startup would mean a + // server that looks healthy and cannot be used. + return nil, errors.New("roomserver: SigningKey must not be nil") + } + + version := roomversion.Default + if opts.DefaultRoomVersion != "" { + if _, err := roomversion.Get(roomversion.Version(opts.DefaultRoomVersion)); err != nil { + return nil, fmt.Errorf("roomserver: default room version %q: %w", opts.DefaultRoomVersion, err) + } + version = roomversion.Version(opts.DefaultRoomVersion) + } + if opts.FederationEnabled == nil { + opts.FederationEnabled = func() bool { return false } + } + if opts.Logger == nil { + opts.Logger = slog.Default() + } + if opts.Now == nil { + opts.Now = time.Now + } + + return &API{ + serverName: opts.ServerName, + store: opts.Store, + profiles: opts.Profiles, + signingKey: opts.SigningKey, + notifier: opts.Notifier, + log: opts.Logger, + now: opts.Now, + federationEnabled: opts.FederationEnabled, + defaultRoomVersion: version, + }, nil +} + +// ServerName returns the server name this roomserver signs events as. +func (r *API) ServerName() string { return r.serverName } + +// DefaultRoomVersion returns the version createRoom uses by default. +func (r *API) DefaultRoomVersion() string { return string(r.defaultRoomVersion) } + +// nowMS is the current time in the unit events carry. +func (r *API) nowMS() int64 { return r.now().UnixMilli() } + +// lock serialises the writers of one room; see API.locks. +func (r *API) lock(roomID string) *sync.Mutex { + actual, _ := r.locks.LoadOrStore(roomID, &sync.Mutex{}) + mu, ok := actual.(*sync.Mutex) + if !ok { + // Unreachable: nothing else ever stores into this map. + panic("roomserver: room lock map holds something that is not a mutex") + } + return mu +} + +// isLocal reports whether a user ID belongs to this server. +func (r *API) isLocal(userID string) bool { + serverName, err := id.ServerNameOf(userID) + return err == nil && serverName == r.serverName +} + +// localpartOf returns the localpart of a user ID of this server. +func (r *API) localpartOf(userID string) (string, bool) { + localpart, serverName, err := id.SplitUserID(userID) + if err != nil || serverName != r.serverName { + return "", false + } + return localpart, true +} + +// rules returns the room version rules a room's events are read under. +// +// A room whose version is no longer supported is a real possibility — a +// future build could drop one — and it has to fail as a room-level error +// rather than a parse error on every event in it. +func (r *API) rules(room storage.Room) (roomversion.Rules, error) { + rules, err := roomversion.Get(roomversion.Version(room.Version)) + if err != nil { + return roomversion.Rules{}, fmt.Errorf("room %s is version %q: %w", room.ID, room.Version, err) + } + return rules, nil +} + +// parseEvent turns a stored event back into an Event. +// +// A redacted event parses from its redacted form and keeps its original +// event ID, because the ID is the reference hash of the redacted form in +// the first place. +func parseEvent(stored storage.StoredEvent, rules roomversion.Rules) (*event.Event, error) { + e, err := event.Parse(stored.JSON, rules) + if err != nil { + return nil, fmt.Errorf("stored event %s is unreadable: %w", stored.EventID, err) + } + return e, nil +} + +// parseEvents parses a batch, failing on the first unreadable one. +// +// An unreadable stored event is a corrupted database or a bug in this +// server's own encoder, never input: everything in the store was parsed and +// validated on the way in. It must not be skipped quietly, because the +// events being parsed here are a room's state and dropping one would +// silently widen what the room allows. +func parseEvents(stored []storage.StoredEvent, rules roomversion.Rules) ([]*event.Event, error) { + out := make([]*event.Event, 0, len(stored)) + for _, s := range stored { + e, err := parseEvent(s, rules) + if err != nil { + return nil, err + } + out = append(out, e) + } + return out, nil +} + +// stateEntries projects a state map onto the rows the store keeps. +func stateEntries(st auth.State) []storage.StateEntry { + out := make([]storage.StateEntry, 0, len(st)) + for key, e := range st { + out = append(out, stateEntry(key.Type, key.StateKey, e)) + } + return out +} + +// stateEntry projects one piece of state onto a storage row, carrying the +// membership along for an m.room.member event so that the store can index +// it. +func stateEntry(eventType, stateKey string, e *event.Event) storage.StateEntry { + entry := storage.StateEntry{Type: eventType, StateKey: stateKey, EventID: e.EventID()} + if eventType == event.TypeMember { + // A member event whose content has no membership would have been + // refused on the way in, so an error here cannot happen; the empty + // string it leaves behind is also the correct "not a member". + entry.Membership, _ = e.Membership() + } + return entry +} + +// eventProvider lets state resolution read a room's auth chains. +type eventProvider struct { + store storage.RoomStore + roomID string + rules roomversion.Rules +} + +// EventsByID implements stateres.EventProvider. +// +// Events this server does not hold are left out rather than reported as an +// error, which is what the algorithm requires: a rejected or missing auth +// event takes no part in resolution. +func (p eventProvider) EventsByID(ctx context.Context, ids []string) ([]*event.Event, error) { + stored, err := p.store.EventsByID(ctx, p.roomID, ids) + if err != nil { + return nil, err + } + return parseEvents(stored, p.rules) +} + +// validateContent checks event content before this server signs it. +// +// The size limit is the spec's limit on a whole event, applied to the +// content alone: the rest of an event is at most a few hundred bytes, so +// anything bigger than this cannot be made to fit and the client should hear +// about it before the event is built, hashed and signed. +func validateContent(content json.RawMessage) (json.RawMessage, error) { + if len(content) == 0 { + return json.RawMessage("{}"), nil + } + if len(content) > event.MaxEventSize { + return nil, fmt.Errorf("%w: content is %d bytes, the limit for a whole event is %d", + ErrInvalidContent, len(content), event.MaxEventSize) + } + var probe map[string]json.RawMessage + if err := json.Unmarshal(content, &probe); err != nil { + return nil, fmt.Errorf("%w: content must be a JSON object", ErrInvalidContent) + } + return content, nil +} diff --git a/server/internal/roomserver/roomserver_test.go b/server/internal/roomserver/roomserver_test.go new file mode 100644 index 0000000..a7aa5af --- /dev/null +++ b/server/internal/roomserver/roomserver_test.go @@ -0,0 +1,701 @@ +package roomserver_test + +import ( + "context" + "encoding/json" + "errors" + "log/slog" + "path/filepath" + "strconv" + "sync" + "testing" + "time" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" + "github.com/dariuszjaniak/warp/server/internal/roomserver" + "github.com/dariuszjaniak/warp/server/internal/storage" + _ "github.com/dariuszjaniak/warp/server/internal/storage/drivers" +) + +const testServerName = "warp.example" + +const ( + alice = "@alice:" + testServerName + bob = "@bob:" + testServerName + carol = "@carol:" + testServerName +) + +// fixture is a roomserver on a throwaway SQLite database. +// +// It is built from the real event core and the real store rather than from +// fakes, because what these tests are about is whether an event this server +// signs is one the authorisation rules accept — and a fake of either would +// only prove that the pipeline calls what the fake expects. +type fixture struct { + t *testing.T + api *roomserver.API + store storage.RoomStore + db storage.Database +} + +// profiles is a stand-in for the userapi, so that these tests do not need +// an account table to put a display name on a membership event. +type profiles map[string]storage.Account + +func (p profiles) Account(_ context.Context, localpart string) (storage.Account, error) { + account, ok := p[localpart] + if !ok { + return storage.Account{}, storage.ErrNoSuchUser + } + return account, nil +} + +func newFixture(t *testing.T, tweak func(*roomserver.Options)) *fixture { + t.Helper() + + path := filepath.ToSlash(filepath.Join(t.TempDir(), "warp.db")) + db, err := storage.Open(t.Context(), storage.Config{ + Driver: storage.DriverSQLite, + ConnectionString: "file:" + path + "?_pragma=busy_timeout(5000)", + }, slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatalf("open storage: %v", err) + } + t.Cleanup(func() { _ = db.Close() }) + if err := db.Migrate(t.Context()); err != nil { + t.Fatalf("migrate: %v", err) + } + + key, err := signing.Generate("test") + if err != nil { + t.Fatalf("signing.Generate: %v", err) + } + + opts := roomserver.Options{ + ServerName: testServerName, + Store: db.Rooms(), + SigningKey: key, + Profiles: profiles{ + "alice": {Localpart: "alice", DisplayName: "Alice"}, + "bob": {Localpart: "bob", DisplayName: "Bob"}, + "carol": {Localpart: "carol", DisplayName: "Carol"}, + }, + Logger: slog.New(slog.DiscardHandler), + } + if tweak != nil { + tweak(&opts) + } + + api, err := roomserver.New(opts) + if err != nil { + t.Fatalf("roomserver.New: %v", err) + } + return &fixture{t: t, api: api, store: db.Rooms(), db: db} +} + +// createRoom creates a room and fails the test if it cannot. +func (f *fixture) createRoom(req roomserver.CreateRoomRequest) roomserver.CreateRoomResponse { + f.t.Helper() + if req.Creator == "" { + req.Creator = alice + } + resp, err := f.api.CreateRoom(f.t.Context(), req) + if err != nil { + f.t.Fatalf("CreateRoom: %v", err) + } + return resp +} + +// state returns a room's current state as the authorisation rules see it. +func (f *fixture) state(roomID string) auth.State { + f.t.Helper() + room, err := f.store.Room(f.t.Context(), roomID) + if err != nil { + f.t.Fatalf("Room: %v", err) + } + rules, err := roomversion.Get(roomversion.Version(room.Version)) + if err != nil { + f.t.Fatalf("roomversion.Get: %v", err) + } + stored, err := f.store.CurrentStateEvents(f.t.Context(), roomID) + if err != nil { + f.t.Fatalf("CurrentStateEvents: %v", err) + } + events := make([]*event.Event, 0, len(stored)) + for _, s := range stored { + e, err := event.Parse(s.JSON, rules) + if err != nil { + f.t.Fatalf("parsing stored event %s: %v", s.EventID, err) + } + events = append(events, e) + } + return auth.NewState(events) +} + +// membership is a user's membership in a room, or "". +func (f *fixture) membership(roomID, userID string) string { + f.t.Helper() + got, err := f.api.Membership(f.t.Context(), roomID, userID) + if err != nil { + f.t.Fatalf("Membership: %v", err) + } + return got +} + +// sendMessage sends an m.room.message, which is the ordinary case. +func (f *fixture) sendMessage(roomID, sender, body, txnID string) (string, error) { + return f.api.SendEvent(f.t.Context(), roomserver.SendEventRequest{ + RoomID: roomID, + Sender: sender, + Type: "m.room.message", + Content: json.RawMessage(`{"msgtype":"m.text","body":"` + body + `"}`), + Transaction: &storage.TransactionRef{ + Localpart: localpartOf(sender), DeviceID: "DEV", TxnID: txnID, + }, + }) +} + +func localpartOf(userID string) string { + for i := 1; i < len(userID); i++ { + if userID[i] == ':' { + return userID[1:i] + } + } + return userID +} + +// TestCreateRoomBuildsAWorkingRoom is the whole of Phase 3 in one test: if +// the events createRoom sends are not authorised by the rules against each +// other, nothing else in this package can work. +func TestCreateRoomBuildsAWorkingRoom(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Name: "Test Room", + Topic: "A topic", + }) + + if resp.RoomID == "" || resp.RoomID[0] != '!' { + t.Fatalf("createRoom returned room ID %q", resp.RoomID) + } + + st := f.state(resp.RoomID) + for _, want := range []struct{ eventType, stateKey string }{ + {event.TypeCreate, ""}, + {event.TypeMember, alice}, + {event.TypePowerLevels, ""}, + {event.TypeJoinRules, ""}, + {event.TypeHistoryVisibility, ""}, + {roomserver.TypeGuestAccess, ""}, + {roomserver.TypeName, ""}, + {roomserver.TypeTopic, ""}, + } { + if st.Get(want.eventType, want.stateKey) == nil { + t.Errorf("the new room has no %s/%q in its state", want.eventType, want.stateKey) + } + } + + if got := st.Membership(alice); got != event.MembershipJoin { + t.Errorf("the creator's membership is %q, want join", got) + } + if got := st.JoinRule(); got != auth.JoinRuleInvite { + t.Errorf("a private room's join rule is %q, want invite", got) + } + + // The creator's membership event carries their profile, which is what + // lets a client render old messages with the name of the day. + member := st.Member(alice) + if member == nil { + t.Fatal("the creator has no membership event") + } + var content map[string]any + if err := json.Unmarshal(member.Content(), &content); err != nil { + t.Fatalf("the membership content is not an object: %v", err) + } + if content["displayname"] != "Alice" { + t.Errorf("the membership event carries displayname %v, want Alice", content["displayname"]) + } + + // The denormalised listing columns followed the state. + summary, err := f.store.RoomSummary(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.Name != "Test Room" || summary.Topic != "A topic" { + t.Errorf("the stored room profile is %+v", summary.RoomProfile) + } + if summary.JoinedMembers != 1 { + t.Errorf("the new room has %d joined members, want 1", summary.JoinedMembers) + } +} + +// TestCreateRoomVersions checks every supported version end to end. Version +// 12 is the one worth the trouble: its room ID is the create event's own ID +// and its create event has no room_id at all, so a pipeline that assumed +// otherwise would fail here and nowhere else. +func TestCreateRoomVersions(t *testing.T) { + for _, version := range roomversion.Supported() { + t.Run(string(version), func(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Version: string(version)}) + + room, err := f.store.Room(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("Room: %v", err) + } + if room.Version != string(version) { + t.Errorf("the room was created at version %q, want %q", room.Version, version) + } + + rules, err := roomversion.Get(version) + if err != nil { + t.Fatalf("roomversion.Get: %v", err) + } + if rules.RoomIDIsCreateEventID { + stored, err := f.store.CurrentStateEvent(t.Context(), resp.RoomID, event.TypeCreate, "") + if err != nil { + t.Fatalf("CurrentStateEvent: %v", err) + } + if want := "!" + stored.EventID[1:]; resp.RoomID != want { + t.Errorf("room ID %q is not derived from the create event %q", + resp.RoomID, stored.EventID) + } + } + + // Whatever the version, a message has to be sendable: that is + // what proves the auth events the pipeline selected are the + // ones the rules want. + if _, err := f.sendMessage(resp.RoomID, alice, "hello", "m1"); err != nil { + t.Errorf("sending a message into a version %s room: %v", version, err) + } + }) + } +} + +func TestCreateRoomUnknownVersion(t *testing.T) { + f := newFixture(t, nil) + + _, err := f.api.CreateRoom(t.Context(), roomserver.CreateRoomRequest{ + Creator: alice, + Version: "99", + }) + if !errors.Is(err, roomserver.ErrUnknownRoomVersion) { + t.Errorf("createRoom at version 99 = %v, want ErrUnknownRoomVersion", err) + } + + // Versions 1 and 2 exist and are deliberately not supported: their + // event format is a separate piece of work that waits for federation. + _, err = f.api.CreateRoom(t.Context(), roomserver.CreateRoomRequest{ + Creator: alice, + Version: "1", + }) + if !errors.Is(err, roomserver.ErrUnsupportedRoomVersion) { + t.Errorf("createRoom at version 1 = %v, want ErrUnsupportedRoomVersion", err) + } +} + +func TestCreateRoomPresets(t *testing.T) { + tests := []struct { + preset, visibility string + wantJoinRule string + wantGuestAccess string + }{ + {preset: roomserver.PresetPublicChat, wantJoinRule: auth.JoinRulePublic, + wantGuestAccess: roomserver.GuestAccessForbidden}, + {preset: roomserver.PresetPrivateChat, wantJoinRule: auth.JoinRuleInvite, + wantGuestAccess: roomserver.GuestAccessCanJoin}, + {preset: roomserver.PresetTrustedPrivateChat, wantJoinRule: auth.JoinRuleInvite, + wantGuestAccess: roomserver.GuestAccessCanJoin}, + // With no preset the directory visibility decides, as the spec says. + {visibility: roomserver.VisibilityPublic, wantJoinRule: auth.JoinRulePublic, + wantGuestAccess: roomserver.GuestAccessForbidden}, + {visibility: roomserver.VisibilityPrivate, wantJoinRule: auth.JoinRuleInvite, + wantGuestAccess: roomserver.GuestAccessCanJoin}, + } + + for _, tc := range tests { + name := tc.preset + if name == "" { + name = "visibility=" + tc.visibility + } + t.Run(name, func(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Preset: tc.preset, + Visibility: tc.visibility, + }) + + st := f.state(resp.RoomID) + if got := st.JoinRule(); got != tc.wantJoinRule { + t.Errorf("join rule = %q, want %q", got, tc.wantJoinRule) + } + + summary, err := f.store.RoomSummary(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.GuestAccess != tc.wantGuestAccess { + t.Errorf("guest access = %q, want %q", summary.GuestAccess, tc.wantGuestAccess) + } + if want := tc.visibility == roomserver.VisibilityPublic; summary.Published != want { + t.Errorf("published = %v, want %v", summary.Published, want) + } + }) + } +} + +// TestCreateRoomTrustedPrivateChatPromotesInvitees is the one thing the +// preset is for: everybody invited comes in as an equal. +func TestCreateRoomTrustedPrivateChatPromotesInvitees(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + Preset: roomserver.PresetTrustedPrivateChat, + Invite: []string{bob}, + }) + + st := f.state(resp.RoomID) + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), st.Create(), st.Create().Rules()) + if err != nil { + t.Fatalf("ResolvePowerLevels: %v", err) + } + if got := powerLevels.UserLevel(bob); got != auth.CreatorPowerLevel { + t.Errorf("an invitee's power level is %d, want %d", got, auth.CreatorPowerLevel) + } + if got := f.membership(resp.RoomID, bob); got != event.MembershipInvite { + t.Errorf("bob's membership is %q, want invite", got) + } +} + +func TestCreateRoomPowerLevelOverride(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + PowerLevelContentOverride: json.RawMessage(`{"events_default": 25, "invite": 75}`), + }) + + st := f.state(resp.RoomID) + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), st.Create(), st.Create().Rules()) + if err != nil { + t.Fatalf("ResolvePowerLevels: %v", err) + } + if powerLevels.EventsDefault != 25 || powerLevels.Invite != 75 { + t.Errorf("the override was not applied: events_default=%d invite=%d", + powerLevels.EventsDefault, powerLevels.Invite) + } + // The keys the override did not mention keep the defaults, which is + // what makes a shallow merge the right one. + if powerLevels.Ban != auth.DefaultBan { + t.Errorf("ban = %d, want the default %d", powerLevels.Ban, auth.DefaultBan) + } +} + +func TestCreateRoomInitialStateAndAlias(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{ + RoomAliasName: "general", + InitialState: []roomserver.StateEvent{{ + Type: event.TypeHistoryVisibility, + Content: json.RawMessage(`{"history_visibility":"world_readable"}`), + }}, + }) + + if want := "#general:" + testServerName; resp.Alias != want { + t.Errorf("createRoom returned alias %q, want %q", resp.Alias, want) + } + resolved, err := f.api.ResolveAlias(t.Context(), resp.Alias) + if err != nil { + t.Fatalf("ResolveAlias: %v", err) + } + if resolved.RoomID != resp.RoomID { + t.Errorf("the alias resolves to %q, want %q", resolved.RoomID, resp.RoomID) + } + + // initial_state overwrites the server's own setup event, which is what + // makes a world-readable room creatable in one call. + summary, err := f.store.RoomSummary(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.HistoryVisibility != roomserver.HistoryVisibilityWorldReadable { + t.Errorf("history visibility = %q, want world_readable", summary.HistoryVisibility) + } + if summary.CanonicalAlias != resp.Alias { + t.Errorf("canonical alias = %q, want %q", summary.CanonicalAlias, resp.Alias) + } +} + +// TestCreateRoomAliasAlreadyTaken checks the alias is claimed before the +// room is built, so that a clash does not leave a room behind. +func TestCreateRoomAliasAlreadyTaken(t *testing.T) { + f := newFixture(t, nil) + f.createRoom(roomserver.CreateRoomRequest{RoomAliasName: "general"}) + + before, _, err := f.store.ListRooms(t.Context(), storage.RoomFilter{}) + if err != nil { + t.Fatalf("ListRooms: %v", err) + } + + _, err = f.api.CreateRoom(t.Context(), roomserver.CreateRoomRequest{ + Creator: bob, + RoomAliasName: "general", + }) + if !errors.Is(err, roomserver.ErrAliasExists) { + t.Fatalf("createRoom with a taken alias = %v, want ErrAliasExists", err) + } + + after, _, err := f.store.ListRooms(t.Context(), storage.RoomFilter{}) + if err != nil { + t.Fatalf("ListRooms: %v", err) + } + if len(after) != len(before) { + t.Errorf("a failed createRoom left %d extra rooms behind", len(after)-len(before)) + } +} + +func TestSendEventAndStreamOrder(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + first, err := f.sendMessage(resp.RoomID, alice, "one", "m1") + if err != nil { + t.Fatalf("sending the first message: %v", err) + } + second, err := f.sendMessage(resp.RoomID, alice, "two", "m2") + if err != nil { + t.Fatalf("sending the second message: %v", err) + } + + one, err := f.store.Event(t.Context(), resp.RoomID, first) + if err != nil { + t.Fatalf("Event: %v", err) + } + two, err := f.store.Event(t.Context(), resp.RoomID, second) + if err != nil { + t.Fatalf("Event: %v", err) + } + if two.StreamOrdering <= one.StreamOrdering { + t.Errorf("stream ordering did not advance: %d then %d", + one.StreamOrdering, two.StreamOrdering) + } + if two.Depth <= one.Depth { + t.Errorf("depth did not advance: %d then %d", one.Depth, two.Depth) + } + + // The second event descends from the first, so the room stays a chain + // rather than forking. + extremities, err := f.store.Extremities(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("Extremities: %v", err) + } + if len(extremities) != 1 || extremities[0].EventID != second { + t.Errorf("extremities = %v, want just the last event", extremities) + } +} + +// TestSendEventIsIdempotentPerTransaction is what a client relies on when +// its connection drops mid-send. +func TestSendEventIsIdempotentPerTransaction(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + first, err := f.sendMessage(resp.RoomID, alice, "hello", "same") + if err != nil { + t.Fatalf("first send: %v", err) + } + again, err := f.sendMessage(resp.RoomID, alice, "hello", "same") + if err != nil { + t.Fatalf("retried send: %v", err) + } + if again != first { + t.Errorf("a retry produced a second event: %q then %q", first, again) + } +} + +func TestSendEventRefusals(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + // A non-member cannot send into the room: the rules say so, not a + // check in the handler. + _, err := f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: bob, + Type: "m.room.message", + Content: json.RawMessage(`{"msgtype":"m.text","body":"hi"}`), + }) + if !errors.Is(err, roomserver.ErrRejected) { + t.Errorf("a stranger sending a message = %v, want ErrRejected", err) + } + + // A second create event is refused outright, with a message about the + // room rather than about a rule number. + _, err = f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: event.TypeCreate, + StateKey: stringPtr(""), + Content: json.RawMessage(`{"room_version":"11"}`), + }) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("re-creating a room = %v, want ErrForbidden", err) + } + + // A user of another server cannot be sent as: this server can only + // sign as itself. + _, err = f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: "@eve:elsewhere.example", + Type: "m.room.message", + Content: json.RawMessage(`{"msgtype":"m.text","body":"hi"}`), + }) + if !errors.Is(err, roomserver.ErrForbidden) { + t.Errorf("sending as a remote user = %v, want ErrForbidden", err) + } + + // Content that is not an object is refused before anything is signed. + _, err = f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: "m.room.message", + Content: json.RawMessage(`"not an object"`), + }) + if !errors.Is(err, roomserver.ErrInvalidContent) { + t.Errorf("sending a string as content = %v, want ErrInvalidContent", err) + } + + // A room this server does not have is not a room. + _, err = f.sendMessage("!nope:"+testServerName, alice, "hi", "x") + if !errors.Is(err, roomserver.ErrNoSuchRoom) { + t.Errorf("sending into an unknown room = %v, want ErrNoSuchRoom", err) + } +} + +func TestSendStateEvent(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + if _, err := f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: roomserver.TypeName, + StateKey: stringPtr(""), + Content: json.RawMessage(`{"name":"Renamed"}`), + }); err != nil { + t.Fatalf("setting the room name: %v", err) + } + + content, err := f.api.StateEventContent(t.Context(), resp.RoomID, alice, roomserver.TypeName, "") + if err != nil { + t.Fatalf("StateEventContent: %v", err) + } + var body struct { + Name string `json:"name"` + } + if err := json.Unmarshal(content, &body); err != nil { + t.Fatalf("the state content is not an object: %v", err) + } + if body.Name != "Renamed" { + t.Errorf("the room name is %q, want Renamed", body.Name) + } + + // The denormalised listing column followed, which is what the public + // room directory is searched by. + summary, err := f.store.RoomSummary(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.Name != "Renamed" { + t.Errorf("the stored room name is %q, want Renamed", summary.Name) + } + + // A member event through the state endpoint needs a real membership. + _, err = f.api.SendEvent(t.Context(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: event.TypeMember, + StateKey: stringPtr(bob), + Content: json.RawMessage(`{"membership":"nonsense"}`), + }) + if !errors.Is(err, roomserver.ErrInvalidContent) { + t.Errorf("an invented membership = %v, want ErrInvalidContent", err) + } +} + +func stringPtr(s string) *string { return &s } + +// TestClockIsUsedForTimestamps checks that origin_server_ts comes from the +// injected clock, which is what makes a test able to reason about ages. +func TestClockIsUsedForTimestamps(t *testing.T) { + at := time.Date(2026, 3, 1, 12, 0, 0, 0, time.UTC) + f := newFixture(t, func(opts *roomserver.Options) { + opts.Now = func() time.Time { return at } + }) + resp := f.createRoom(roomserver.CreateRoomRequest{}) + + stored, err := f.store.CurrentStateEvent(t.Context(), resp.RoomID, event.TypeCreate, "") + if err != nil { + t.Fatalf("CurrentStateEvent: %v", err) + } + if stored.OriginServerTS != at.UnixMilli() { + t.Errorf("origin_server_ts = %d, want %d", stored.OriginServerTS, at.UnixMilli()) + } +} + +// TestConcurrentSendsStayALinearChain is what the per-room lock exists for. +// +// Without it two events racing into one room would each read the same +// forward extremities, both become one, and fork the room — which would +// then need state resolution to put back together for no reason at all. +func TestConcurrentSendsStayALinearChain(t *testing.T) { + f := newFixture(t, nil) + resp := f.createRoom(roomserver.CreateRoomRequest{Preset: roomserver.PresetPublicChat}) + + const senders = 8 + var wg sync.WaitGroup + errs := make(chan error, senders) + for i := range senders { + wg.Add(1) + go func() { + defer wg.Done() + _, err := f.api.SendEvent(context.Background(), roomserver.SendEventRequest{ + RoomID: resp.RoomID, + Sender: alice, + Type: "m.room.message", + Content: json.RawMessage(`{"msgtype":"m.text","body":"concurrent"}`), + Transaction: &storage.TransactionRef{ + Localpart: "alice", + DeviceID: "DEV", + TxnID: "txn" + strconv.Itoa(i), + }, + }) + errs <- err + }() + } + wg.Wait() + close(errs) + for err := range errs { + if err != nil { + t.Fatalf("a concurrent send failed: %v", err) + } + } + + extremities, err := f.store.Extremities(t.Context(), resp.RoomID) + if err != nil { + t.Fatalf("Extremities: %v", err) + } + if len(extremities) != 1 { + t.Errorf("the room has %d forward extremities, want 1: the sends forked it", + len(extremities)) + } + + // Every send produced an event: the lock serialises them, it does not + // drop any. + stats, err := f.store.RoomStats(t.Context()) + if err != nil { + t.Fatalf("RoomStats: %v", err) + } + if stats.Events < senders { + t.Errorf("the room holds %d events, want at least the %d sent", stats.Events, senders) + } +} diff --git a/server/internal/roomserver/send.go b/server/internal/roomserver/send.go new file mode 100644 index 0000000..ba8c545 --- /dev/null +++ b/server/internal/roomserver/send.go @@ -0,0 +1,231 @@ +package roomserver + +import ( + "context" + "encoding/json" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/auth" + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/roomversion" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// SendEventRequest sends one event into a room on a user's behalf. +type SendEventRequest struct { + RoomID string + + // Sender is the full user ID sending the event. + Sender string + + Type string + + // StateKey makes this a state event when it is non-nil. + StateKey *string + + Content json.RawMessage + + // Transaction, when set, makes the send idempotent: a retry with the + // same transaction ID returns the event the first attempt created. + // State events have no transaction ID in the Client-Server API, which + // is why this is optional. + Transaction *storage.TransactionRef +} + +// SendEvent sends a message or state event into a room. +// +// It does no authorisation of its own, deliberately: whether a user may +// send a given type into a given room is a question about the room's power +// levels and membership, and the one place that is answered correctly is +// the authorisation rules inside the pipeline. A handler-level check here +// would be a second, worse copy of them. +// +// The one thing it does refuse up front is m.room.create. The rules would +// refuse it too — a second create event in a room is rejected — but the +// error a client gets for "you cannot re-create a room" is more use than +// the one for "rule 2 said no". +func (r *API) SendEvent(ctx context.Context, req SendEventRequest) (string, error) { + if req.Type == event.TypeCreate { + return "", fmt.Errorf("%w: a room's m.room.create event is sent by createRoom and cannot be replaced", + ErrForbidden) + } + if req.Type == event.TypeMember { + // A membership change through the state endpoint is legitimate and + // the rules handle it, but the membership value still has to be one + // the rules know: content that is not a membership at all would + // otherwise become an event nobody can interpret. + if err := checkMembershipContent(req.Content); err != nil { + return "", err + } + } + + sent, err := r.send(ctx, sendRequest{ + RoomID: req.RoomID, + Type: req.Type, + StateKey: req.StateKey, + Sender: req.Sender, + Content: req.Content, + Transaction: req.Transaction, + }) + if err != nil { + return "", err + } + return sent.EventID(), nil +} + +// checkMembershipContent rejects an m.room.member content whose membership +// is missing or not one of the five the spec defines. +func checkMembershipContent(content json.RawMessage) error { + var body struct { + Membership string `json:"membership"` + } + if err := json.Unmarshal(content, &body); err != nil { + return fmt.Errorf("%w: an m.room.member content needs a membership", ErrInvalidContent) + } + switch body.Membership { + case event.MembershipJoin, event.MembershipLeave, event.MembershipInvite, + event.MembershipBan, event.MembershipKnock: + return nil + case "": + return fmt.Errorf("%w: an m.room.member content needs a membership", ErrInvalidContent) + default: + return fmt.Errorf("%w: %q is not a membership", ErrInvalidContent, body.Membership) + } +} + +// RedactRequest asks to redact an event. +type RedactRequest struct { + RoomID string + + // Sender is the full user ID sending the redaction. + Sender string + + // EventID is the event to redact. + EventID string + + Reason string + + Transaction *storage.TransactionRef +} + +// Redact redacts an event. +// +// This is two decisions, not one, and the spec keeps them apart. The first +// is whether the m.room.redaction event may be in the room at all, which +// the authorisation rules answer and which is permissive: almost any member +// may send one, because a server receiving a redaction for an event it has +// never seen still has to be able to store it. The second is whether the +// redaction takes effect on its target, which needs both events in hand and +// is auth.ShouldApplyRedaction's job — either the sender has the redact +// level, or the redaction comes from the same server as the event, because +// a server may always take back its own users' events. +// +// Phase 3 only ever has both events, since every event in a room is local. +// A redaction whose target has not arrived is a federation case. +func (r *API) Redact(ctx context.Context, req RedactRequest) (string, error) { + // The room is read first so that a room this server does not have is a + // 404 before anything is built, and so that a room whose version is no + // longer supported says so. + room, err := r.store.Room(ctx, req.RoomID) + if err != nil { + return "", err + } + if _, err := r.rules(room); err != nil { + return "", err + } + + // The target is read before the redaction is built so that redacting + // an event that does not exist is a 404 rather than an event in the + // room that refers to nothing. + if _, err := r.store.Event(ctx, req.RoomID, req.EventID); err != nil { + return "", err + } + + content := map[string]any{} + if req.Reason != "" { + content["reason"] = req.Reason + } + + redaction, err := r.send(ctx, sendRequest{ + RoomID: req.RoomID, + Type: event.TypeRedaction, + Sender: req.Sender, + Content: mustMarshal(content), + Redacts: req.EventID, + Transaction: req.Transaction, + }) + if err != nil { + return "", err + } + return redaction.EventID(), nil +} + +// prepareRedaction decides whether a redaction takes effect and, if it +// does, puts the target's redacted form into the write. +// +// It runs inside persist, within the same transaction as the redaction +// event itself, so that a client which gets an event ID back can always +// read the stripped event. +func (r *API) prepareRedaction(ctx context.Context, write *storage.RoomWrite, req persistRequest) error { + storedTarget, err := r.store.Event(ctx, req.Room.ID, req.RedactsEventID) + if isNotFound(err) { + // Over federation a redaction can arrive before its target, and the + // right answer is to keep it and apply it later. Locally the target + // was read before the redaction was built, so this means it was + // purged in between. + r.log.Info("redaction target is not here", + "room_id", req.Room.ID, "redacts", req.RedactsEventID) + return nil + } else if err != nil { + return err + } + if storedTarget.RedactedBy != "" { + // Already redacted. The event stays as it is: a second redaction + // has nothing left to remove. + return nil + } + + target, err := parseEvent(storedTarget, req.Rules) + if err != nil { + return err + } + applies, err := r.redactionApplies(ctx, req.Room, req.Rules, req.Event, target) + if err != nil { + return err + } + if !applies { + // The event is in the room and has no effect, which is what the + // spec asks for. It is worth a log line: from the client's side a + // redaction that returns an event ID and changes nothing looks + // exactly like one that worked. + r.log.Info("accepted a redaction that does not apply", + "room_id", req.Room.ID, "event_id", req.Event.EventID(), + "redacts", req.RedactsEventID, "sender", req.Event.Sender()) + return nil + } + + redacted, err := target.Redact() + if err != nil { + return fmt.Errorf("redacting %s: %w", req.RedactsEventID, err) + } + write.Redacts = req.RedactsEventID + write.RedactedJSON = redacted.JSON() + return nil +} + +// redactionApplies runs the spec's "should this redaction be applied" test. +func (r *API) redactionApplies(ctx context.Context, room storage.Room, rules roomversion.Rules, redaction, target *event.Event) (bool, error) { + st, err := r.currentState(ctx, room, rules) + if err != nil { + return false, err + } + create := st.Create() + if create == nil { + return false, fmt.Errorf("%w: %s has no create event in its state", ErrRejected, room.ID) + } + powerLevels, err := auth.ResolvePowerLevels(st.PowerLevels(), create, rules) + if err != nil { + return false, fmt.Errorf("%w: %v", ErrRejected, err) + } + return auth.ShouldApplyRedaction(redaction, target, powerLevels), nil +} diff --git a/server/internal/roomserver/verify.go b/server/internal/roomserver/verify.go new file mode 100644 index 0000000..345ae7f --- /dev/null +++ b/server/internal/roomserver/verify.go @@ -0,0 +1,40 @@ +package roomserver + +import ( + "context" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/matrix/event" + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" +) + +// localVerifier checks an event's signature against this server's own key. +// +// One authorisation rule needs somebody else's signature verified: a join +// under the restricted join rule is authorised by a third party named in +// content.join_authorised_via_users_server, and the event must carry a +// signature from *that user's* server. While federation is off the only +// such server that can appear is this one, and its key is already in hand — +// so the rule can be applied in full, locally, without a key fetcher. +// +// Phase 5 replaces this with a verifier that can fetch and cache another +// server's keys. Until then a join authorised via a remote user is refused +// rather than waved through, which is the same answer the federation switch +// gives everywhere else. +type localVerifier struct { + serverName string + key *signing.KeyPair +} + +// verifier returns the Verifier the authorisation rules are given. +func (r *API) verifier() *localVerifier { + return &localVerifier{serverName: r.serverName, key: r.signingKey} +} + +// VerifyEventSignature implements auth.Verifier. +func (v *localVerifier) VerifyEventSignature(_ context.Context, serverName string, e *event.Event) error { + if serverName != v.serverName { + return fmt.Errorf("%w: cannot verify a signature from %s", ErrFederationDisabled, serverName) + } + return e.VerifySignature(v.serverName, v.key.KeyID(), v.key.Public) +} diff --git a/server/internal/setup/routes.go b/server/internal/setup/routes.go index fc9e8b2..77f5f07 100644 --- a/server/internal/setup/routes.go +++ b/server/internal/setup/routes.go @@ -11,10 +11,11 @@ import ( // SpecVersions is what GET /_matrix/client/versions reports. // // Keep it honest: a version belongs here once the endpoints a client -// actually uses from it work. Phase 1 adds accounts, sessions, devices, -// profiles and account data; rooms and /sync are still missing, so the list -// is a promise about the identity half of the API only and grows as -// Phases 3 and 4 land. +// actually uses from it work. Accounts, sessions, devices, profiles, +// account data and — with Phase 3 — rooms all do. What is still missing is +// the event stream: /sync returns an empty placeholder and there is no +// /messages, so a client can create a room and send into it but cannot read +// its history. The list grows when Phase 4 lands. var SpecVersions = []string{"v1.13", "v1.14"} // UnstableFeatures advertises MSC support. Simplified Sliding Sync diff --git a/server/internal/setup/server.go b/server/internal/setup/server.go index f005bae..c4d3c5f 100644 --- a/server/internal/setup/server.go +++ b/server/internal/setup/server.go @@ -21,6 +21,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/httputil" "github.com/dariuszjaniak/warp/server/internal/matrix/signing" "github.com/dariuszjaniak/warp/server/internal/metrics" + "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" "github.com/dariuszjaniak/warp/server/internal/storage" _ "github.com/dariuszjaniak/warp/server/internal/storage/drivers" // link in the storage drivers @@ -39,6 +40,7 @@ type Server struct { signingKey *signing.KeyPair users *userapi.API + rooms *roomserver.API clientAPI *clientapi.ClientAPI adminAPI *adminapi.AdminAPI @@ -142,10 +144,33 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) return nil, err } + // The roomserver needs the signing key: it is the only thing in Warp + // that creates an event, and an event has to be signed. `warp migrate` + // and the Phase 1 tests start without a key, and then start without + // rooms — the C-S room endpoints answer M_UNRECOGNIZED rather than + // pretending to work. + if s.signingKey != nil { + s.rooms, err = roomserver.New(roomserver.Options{ + ServerName: cfg.Global.ServerName, + Store: s.db.Rooms(), + Profiles: s.users, + SigningKey: s.signingKey, + FederationEnabled: settingsService.FederationEnabled, + Logger: log, + }) + if err != nil { + _ = s.closeDB() + return nil, err + } + } else { + log.Warn("started without a signing key: rooms are unavailable") + } + s.clientAPI, err = clientapi.New(clientapi.Options{ ServerName: cfg.Global.ServerName, ClientBaseURL: cfg.ClientBaseURL(), Users: s.users, + Rooms: s.rooms, Settings: settingsService, Federation: s.federation, Registration: cfg.Registration, @@ -161,6 +186,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) s.adminAPI, err = adminapi.New(adminapi.Options{ ServerName: cfg.Global.ServerName, Users: s.users, + Rooms: s.rooms, Settings: settingsService, Federation: s.federation, Database: s.db, @@ -222,6 +248,10 @@ func (s *Server) Database() storage.Database { return s.db } // tests use to create accounts without going through HTTP. func (s *Server) Users() *userapi.API { return s.users } +// Rooms exposes the roomserver component, or nil when the server started +// without a signing key. +func (s *Server) Rooms() *roomserver.API { return s.rooms } + // SigningKey returns the server signing key, or nil when it was skipped. func (s *Server) SigningKey() *signing.KeyPair { return s.signingKey } diff --git a/server/internal/storage/pocketbase/pocketbase.go b/server/internal/storage/pocketbase/pocketbase.go index 96d82a7..bb31295 100644 --- a/server/internal/storage/pocketbase/pocketbase.go +++ b/server/internal/storage/pocketbase/pocketbase.go @@ -54,6 +54,9 @@ func (*Database) Settings() storage.SettingsStore { return notImplementedSetting // Users implements storage.Database. func (*Database) Users() storage.UserStore { return notImplementedUsers{} } +// Rooms implements storage.Database. +func (*Database) Rooms() storage.RoomStore { return notImplementedRooms{} } + // Close implements storage.Database. func (*Database) Close() error { return nil } @@ -163,8 +166,114 @@ func (notImplementedUsers) SetAccountData(context.Context, string, string, strin return storage.ErrNotImplemented } +// notImplementedRooms satisfies storage.RoomStore the same way. It matters +// more here than anywhere else: a room whose create event was accepted and +// not stored would hand a client a room ID that refers to nothing. +type notImplementedRooms struct{} + +func (notImplementedRooms) CreateRoom(context.Context, storage.NewRoom) error { + return storage.ErrNotImplemented +} + +func (notImplementedRooms) Room(context.Context, string) (storage.Room, error) { + return storage.Room{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) Write(context.Context, storage.RoomWrite) (int64, error) { + return 0, storage.ErrNotImplemented +} + +func (notImplementedRooms) Event(context.Context, string, string) (storage.StoredEvent, error) { + return storage.StoredEvent{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) EventsByID(context.Context, string, []string) ([]storage.StoredEvent, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) CurrentState(context.Context, string) ([]storage.StateEntry, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) CurrentStateEvents(context.Context, string) ([]storage.StoredEvent, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) CurrentStateEvent(context.Context, string, string, string) (storage.StoredEvent, error) { + return storage.StoredEvent{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) StateBefore(context.Context, string, string) ([]storage.StateEntry, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) Extremities(context.Context, string) ([]storage.StoredEvent, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) Members(context.Context, string, []string) ([]storage.StoredEvent, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) MembershipEntries(context.Context, string, []string) ([]storage.MembershipEntry, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) Membership(context.Context, string, string) (storage.MembershipEntry, error) { + return storage.MembershipEntry{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) RoomsForUser(context.Context, string, []string) ([]string, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) TransactionEvent(context.Context, storage.TransactionRef) (string, error) { + return "", storage.ErrNotImplemented +} + +func (notImplementedRooms) CreateAlias(context.Context, storage.RoomAlias) error { + return storage.ErrNotImplemented +} + +func (notImplementedRooms) DeleteAlias(context.Context, string) error { + return storage.ErrNotImplemented +} + +func (notImplementedRooms) Alias(context.Context, string) (storage.RoomAlias, error) { + return storage.RoomAlias{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) AliasesForRoom(context.Context, string) ([]string, error) { + return nil, storage.ErrNotImplemented +} + +func (notImplementedRooms) SetPublished(context.Context, string, bool) error { + return storage.ErrNotImplemented +} + +func (notImplementedRooms) PublishedRooms(context.Context, storage.RoomFilter) ([]storage.RoomSummary, int, error) { + return nil, 0, storage.ErrNotImplemented +} + +func (notImplementedRooms) ListRooms(context.Context, storage.RoomFilter) ([]storage.RoomSummary, int, error) { + return nil, 0, storage.ErrNotImplemented +} + +func (notImplementedRooms) RoomSummary(context.Context, string) (storage.RoomSummary, error) { + return storage.RoomSummary{}, storage.ErrNotImplemented +} + +func (notImplementedRooms) PurgeRoom(context.Context, string) error { + return storage.ErrNotImplemented +} + +func (notImplementedRooms) RoomStats(context.Context) (storage.RoomStats, error) { + return storage.RoomStats{}, storage.ErrNotImplemented +} + // Compile-time proof that the stub keeps up with the interfaces. var ( _ storage.Database = (*Database)(nil) _ storage.UserStore = notImplementedUsers{} + _ storage.RoomStore = notImplementedRooms{} ) diff --git a/server/internal/storage/postgres/migrations/0003_roomserver.sql b/server/internal/storage/postgres/migrations/0003_roomserver.sql new file mode 100644 index 0000000..5125542 --- /dev/null +++ b/server/internal/storage/postgres/migrations/0003_roomserver.sql @@ -0,0 +1,165 @@ +-- Rooms, events and room state: the roomserver tables. +-- +-- Events are keyed by their event ID rather than by a numeric surrogate. +-- An event ID is 44 bytes of base64 and appears in prev_events, +-- auth_events, state snapshots and client requests alike, so a surrogate +-- would have to be translated at every one of those boundaries; the +-- indexes below are what make the text key cheap enough. +-- +-- Timestamps are milliseconds since the Unix epoch, as everywhere else in +-- Warp. + +CREATE TABLE IF NOT EXISTS roomserver_rooms ( + room_id TEXT NOT NULL PRIMARY KEY, + -- Fixed at creation. Nothing can read an event without it: the room + -- version decides the event format, the redaction algorithm and the + -- authorisation rules. + room_version TEXT NOT NULL, + creator TEXT NOT NULL, + is_published BOOLEAN NOT NULL DEFAULT FALSE, + created_at_ms BIGINT NOT NULL, + + -- The denormalised room listing columns. Everything below is derivable + -- from the room's current state and is kept here anyway, because the + -- public room directory orders every published room by member count + -- and pages the result: that cannot be done by reading each room's + -- state in turn. The roomserver recomputes them when the state behind + -- them changes; joined_members the store counts for itself. + name TEXT NOT NULL DEFAULT '', + topic TEXT NOT NULL DEFAULT '', + avatar_url TEXT NOT NULL DEFAULT '', + canonical_alias TEXT NOT NULL DEFAULT '', + join_rule TEXT NOT NULL DEFAULT '', + history_visibility TEXT NOT NULL DEFAULT '', + guest_access TEXT NOT NULL DEFAULT '', + room_type TEXT NOT NULL DEFAULT '', + joined_members BIGINT NOT NULL DEFAULT 0 +); + +-- The public room directory's own ordering: published rooms, biggest first. +CREATE INDEX IF NOT EXISTS roomserver_rooms_published_idx + ON roomserver_rooms (is_published, joined_members); + +-- One counter row per stream, incremented inside the write transaction. +-- +-- Neither AUTOINCREMENT nor a sequence would do: SQLite and PostgreSQL +-- spell those differently, and a PostgreSQL sequence is explicitly not +-- transactional, so a rolled-back write would leave a hole in the stream +-- that /sync would have to be taught to skip. An UPDATE on this row takes +-- a row lock in both engines and gives out positions with no gaps. +CREATE TABLE IF NOT EXISTS roomserver_sequences ( + name TEXT NOT NULL PRIMARY KEY, + value BIGINT NOT NULL +); + +INSERT INTO roomserver_sequences (name, value) VALUES ('stream_ordering', 0) + ON CONFLICT (name) DO NOTHING; +INSERT INTO roomserver_sequences (name, value) VALUES ('state_snapshot', 0) + ON CONFLICT (name) DO NOTHING; + +-- A state snapshot is the complete state of a room at one point, stored +-- once and shared by every event it applies to. Consecutive messages in a +-- room whose state is not changing all point at the same snapshot, so the +-- cost is one snapshot per state change rather than one per event. +CREATE TABLE IF NOT EXISTS roomserver_state_snapshots ( + snapshot_id BIGINT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + -- SHA-256 over the sorted (type, state_key, event_id) triples: the + -- identity of the snapshot's contents, which is what lets two + -- identical states share a row. + state_hash TEXT NOT NULL, + UNIQUE (room_id, state_hash) +); + +CREATE TABLE IF NOT EXISTS roomserver_state_entries ( + snapshot_id BIGINT NOT NULL REFERENCES roomserver_state_snapshots (snapshot_id) ON DELETE CASCADE, + event_type TEXT NOT NULL, + state_key TEXT NOT NULL, + event_id TEXT NOT NULL, + PRIMARY KEY (snapshot_id, event_type, state_key) +); + +CREATE TABLE IF NOT EXISTS roomserver_events ( + event_id TEXT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + -- This server's own monotonic position for the event: the order it was + -- persisted in, which is the order clients are shown it in. + stream_ordering BIGINT NOT NULL, + event_type TEXT NOT NULL, + -- NULL for a message event. The empty string is a valid state key, so + -- "no state key" cannot be spelled as ''. + state_key TEXT, + sender TEXT NOT NULL, + depth BIGINT NOT NULL, + origin_server_ts BIGINT NOT NULL, + -- The event as canonical JSON, byte for byte as it was signed. Every + -- other column here is a projection of it, written out to be indexed. + event_json TEXT NOT NULL, + -- The state of the room as of this event's prev_events, 0 for the + -- create event. Keeping it per event is what lets the state after a + -- forward extremity be worked out, which state resolution needs when a + -- room has more than one. + state_before_id BIGINT NOT NULL DEFAULT 0, + -- The redaction that applies to this event, or ''. The JSON above is + -- rewritten to its redacted form at the same time; this records which + -- event did it so that unsigned.redacted_because can be filled in. + redacted_by TEXT NOT NULL DEFAULT '' +); + +CREATE UNIQUE INDEX IF NOT EXISTS roomserver_events_stream_idx + ON roomserver_events (stream_ordering); + +-- Paging a room's timeline, which is what /messages and /sync do. +CREATE INDEX IF NOT EXISTS roomserver_events_room_stream_idx + ON roomserver_events (room_id, stream_ordering); + +-- The forward extremities: the events with no children, and so the +-- prev_events of whatever is sent next. +CREATE TABLE IF NOT EXISTS roomserver_extremities ( + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + event_id TEXT NOT NULL, + PRIMARY KEY (room_id, event_id) +); + +-- The room's resolved current state: the state after its extremities. +-- +-- It is derivable from the events, and is stored anyway because every +-- authorisation decision reads it. The membership column is a denormalised +-- copy of content.membership on m.room.member rows, so that "the rooms +-- @alice is joined to" is an index lookup rather than a scan that parses +-- JSON. +CREATE TABLE IF NOT EXISTS roomserver_current_state ( + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + event_type TEXT NOT NULL, + state_key TEXT NOT NULL, + event_id TEXT NOT NULL, + membership TEXT NOT NULL DEFAULT '', + PRIMARY KEY (room_id, event_type, state_key) +); + +CREATE INDEX IF NOT EXISTS roomserver_current_state_membership_idx + ON roomserver_current_state (event_type, state_key, membership); + +-- Client transaction IDs, so that a retried request returns the event the +-- first attempt created instead of sending a second one. +CREATE TABLE IF NOT EXISTS roomserver_transactions ( + localpart TEXT NOT NULL, + device_id TEXT NOT NULL, + txn_id TEXT NOT NULL, + event_id TEXT NOT NULL, + room_id TEXT NOT NULL, + sent_at_ms BIGINT NOT NULL, + PRIMARY KEY (localpart, device_id, txn_id) +); + +-- Local room aliases. Remote aliases are never stored: they are resolved +-- over federation every time, because the other server may have moved them. +CREATE TABLE IF NOT EXISTS roomserver_aliases ( + alias TEXT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + creator TEXT NOT NULL, + created_at_ms BIGINT NOT NULL +); + +CREATE INDEX IF NOT EXISTS roomserver_aliases_room_idx + ON roomserver_aliases (room_id); diff --git a/server/internal/storage/rooms.go b/server/internal/storage/rooms.go new file mode 100644 index 0000000..6e106c6 --- /dev/null +++ b/server/internal/storage/rooms.go @@ -0,0 +1,357 @@ +package storage + +import ( + "context" + "errors" +) + +// Errors the room store returns for cases a caller has to tell apart. +var ( + // ErrNoSuchRoom is returned for a room this server does not know. + ErrNoSuchRoom = errors.New("no such room") + + // ErrRoomExists is returned by CreateRoom for a room ID already taken. + // It can only happen through a room ID collision, which for a room + // version 12 room would mean two identical create events. + ErrRoomExists = errors.New("room already exists") + + // ErrNoSuchEvent is returned for an event ID the room store does not + // hold. + ErrNoSuchEvent = errors.New("no such event") + + // ErrNoSuchAlias is returned for an unknown room alias. + ErrNoSuchAlias = errors.New("no such room alias") + + // ErrAliasExists is returned by CreateAlias for an alias already + // pointing somewhere. + ErrAliasExists = errors.New("room alias already exists") +) + +// Room is one room this server takes part in. +// +// The version is fixed when the room is created and is what every later read +// needs first: an event cannot even be parsed without knowing the rules it +// was written under. +type Room struct { + ID string + + // Version is the room version identifier, e.g. "11". + Version string + + // Creator is the full user ID that sent the m.room.create event. + Creator string + + // Published reports whether the room is in this server's public room + // directory. + Published bool + + CreatedAtMS int64 +} + +// NewRoom describes a room to create. The create event itself is written by +// the same Write call that appends any other event. +type NewRoom struct { + ID string + Version string + Creator string + Published bool + CreatedAtMS int64 +} + +// RoomProfile is the denormalised view of a room's state that a room +// listing is searched and ordered by. +// +// It is all derivable from the room's current state, and is stored anyway: +// the public room directory sorts every published room by member count and +// pages the result, which cannot be done by reading each room's state in +// turn. The roomserver recomputes it whenever one of the state events behind +// it changes and hands it to Write; the store only keeps it, because +// knowing that m.room.name means a name is the roomserver's business. +// +// JoinedMembers is the exception: the store maintains it itself, by counting +// the current-state rows whose membership is "join". +type RoomProfile struct { + Name string + Topic string + AvatarURL string + CanonicalAlias string + + JoinRule string + HistoryVisibility string + GuestAccess string + + // RoomType is m.room.create's content.type: empty for a normal room, + // "m.space" for a space. + RoomType string +} + +// StateEntry is one piece of room state and the event that holds it. +type StateEntry struct { + Type string + StateKey string + EventID string + + // Membership is the membership value of an m.room.member entry, and + // empty on every other kind of state. It travels with the entry so that + // the store can keep "who is joined to this room" answerable with an + // index instead of by parsing event JSON. + Membership string +} + +// StoredEvent is an event as the room store holds it: the canonical JSON +// plus the columns worth indexing separately. +// +// JSON is the authority. The other fields are projections of it, written out +// so that the store can answer "the members of this room" or "events in +// stream order" without parsing every row. +type StoredEvent struct { + RoomID string + EventID string + Type string + + // StateKey is nil for a non-state event. The empty string is a + // perfectly good state key, which is why this is a pointer. + StateKey *string + + Sender string + Depth int64 + OriginServerTS int64 + + // JSON is the event as canonical JSON, exactly as it was signed. + JSON []byte + + // StreamOrdering is this server's own monotonic position for the event: + // the order it was persisted in, which is the order /sync and /messages + // hand events to clients. + StreamOrdering int64 + + // RedactedBy is the event ID of the redaction that applies to this + // event, or empty. The JSON is rewritten in place when a redaction is + // applied, so a redacted event is already stripped on the way out; this + // field is what lets unsigned.redacted_because be filled in. + RedactedBy string +} + +// IsState reports whether the event is a state event. +func (e StoredEvent) IsState() bool { return e.StateKey != nil } + +// TransactionRef records a client transaction ID against the event it +// produced, so that a retried PUT returns the first event rather than +// sending a second one. +type TransactionRef struct { + Localpart string + DeviceID string + TxnID string +} + +// RoomWrite is one event and everything its arrival changes, written in a +// single transaction. +// +// Persisting an event is never just an insert: it moves the room's forward +// extremities, it may replace a piece of current state, and it may strip an +// earlier event. Splitting that across calls would leave a window in which +// the room's current state disagrees with its events, so the whole lot goes +// down together. +type RoomWrite struct { + // Event is the event being appended. Its StreamOrdering is assigned by + // the store and returned. + Event StoredEvent + + // StateBefore is the room state the event was authorised against: the + // full state of the room as of its prev_events. It is stored as a + // deduplicated snapshot, so appending a message to a quiet room adds no + // state rows at all. + StateBefore []StateEntry + + // StateDelta are the pieces of current state this event changes — for + // an ordinary state event, the single tuple it writes. It is empty for + // an event that changes no state. + StateDelta []StateEntry + + // StateDeltaIsWholeState says that StateDelta is the room's entire new + // current state, and that any tuple not in it must be removed. State + // resolution across a forked room produces such a result; a single + // state event does not, and rewriting the whole of a large room's state + // on every join would make filling a room quadratic. + StateDeltaIsWholeState bool + + // Profile updates the denormalised room listing columns. It is nil when + // this event changes none of them. + Profile *RoomProfile + + // Extremities replaces the room's forward extremities. For a locally + // created event that is the event itself, since it descends from every + // extremity there was. + Extremities []string + + // Transaction, when set, records the client transaction ID this event + // came from. + Transaction *TransactionRef + + // Redacts is the event ID this event redacts, when the redaction is to + // be applied. RedactedJSON is the target's redacted form, which + // replaces the stored JSON. + Redacts string + RedactedJSON []byte +} + +// RoomAlias is a local alias and the room it points at. +type RoomAlias struct { + Alias string + RoomID string + + // Creator is the full user ID that created the alias. + Creator string + + CreatedAtMS int64 +} + +// MembershipEntry is one member of a room. +type MembershipEntry struct { + UserID string + Membership string + EventID string +} + +// RoomFilter narrows and pages a room listing. +type RoomFilter struct { + // Search matches a substring of the room ID, name, topic or canonical + // alias, case-insensitively. Empty matches everything. + Search string + + // Limit caps the returned rows; 0 means the store's own default. + Limit int + + // Offset skips rows, for paging. + Offset int +} + +// RoomSummary is a room as a listing shows it: the room row, its +// denormalised state and its member count. +type RoomSummary struct { + Room + RoomProfile + + // JoinedMembers is how many members of the room are joined. + JoinedMembers int +} + +// RoomStats are the room counts the admin dashboard shows. +type RoomStats struct { + Rooms int + Published int + Events int +} + +// RoomStore persists rooms, their events and their state: the storage half +// of the roomserver component. +// +// Every method takes identifiers the caller has already validated. The store +// interprets nothing: it does not know the authorisation rules, it does not +// know which room versions exist, and it never parses an event. What it +// guarantees is that a room's events, its state and its extremities are only +// ever seen in a consistent set. +type RoomStore interface { + // CreateRoom inserts the room row, returning ErrRoomExists when the ID + // is taken. + CreateRoom(ctx context.Context, room NewRoom) error + + // Room reads one room, returning ErrNoSuchRoom if it is absent. + Room(ctx context.Context, roomID string) (Room, error) + + // Write appends one event and applies everything it changes. It returns + // the stream ordering the event was given. + Write(ctx context.Context, w RoomWrite) (int64, error) + + // Event reads one event of a room, returning ErrNoSuchEvent if it is + // absent. + Event(ctx context.Context, roomID, eventID string) (StoredEvent, error) + + // EventsByID reads events by ID within one room. Unknown IDs are left + // out rather than reported: state resolution needs exactly that + // behaviour for events this server does not have. + EventsByID(ctx context.Context, roomID string, eventIDs []string) ([]StoredEvent, error) + + // CurrentState returns the room's resolved current state. + CurrentState(ctx context.Context, roomID string) ([]StateEntry, error) + + // CurrentStateEvents returns the events behind the room's current + // state, which is what the authorisation rules are run against. + CurrentStateEvents(ctx context.Context, roomID string) ([]StoredEvent, error) + + // CurrentStateEvent returns one piece of current state, or + // ErrNoSuchEvent when the room has no such state. + CurrentStateEvent(ctx context.Context, roomID, eventType, stateKey string) (StoredEvent, error) + + // StateBefore returns the state snapshot recorded against an event: the + // state of the room as of that event's prev_events. + StateBefore(ctx context.Context, roomID, eventID string) ([]StateEntry, error) + + // Extremities returns the room's forward extremities, which are the + // prev_events of the next event to be sent. + Extremities(ctx context.Context, roomID string) ([]StoredEvent, error) + + // Members returns the m.room.member events of a room. memberships + // filters by membership value; empty returns every member event, + // including the users who have left. + Members(ctx context.Context, roomID string, memberships []string) ([]StoredEvent, error) + + // MembershipEntries returns the memberships of a room without the event + // bodies, which is what /joined_members needs. + MembershipEntries(ctx context.Context, roomID string, memberships []string) ([]MembershipEntry, error) + + // Membership returns a user's membership in a room and the event that + // set it. A user who was never in the room gets an empty membership and + // no error: "never joined" and "left" are the same answer to every rule + // that asks. + Membership(ctx context.Context, roomID, userID string) (MembershipEntry, error) + + // RoomsForUser lists the rooms a user has one of the given memberships + // in, ordered by room ID. + RoomsForUser(ctx context.Context, userID string, memberships []string) ([]string, error) + + // TransactionEvent returns the event a client transaction ID already + // produced, or ErrNoSuchEvent when this is the first time it is seen. + TransactionEvent(ctx context.Context, ref TransactionRef) (string, error) + + // CreateAlias points an alias at a room, returning ErrAliasExists when + // the alias is taken. + CreateAlias(ctx context.Context, alias RoomAlias) error + + // DeleteAlias removes an alias, returning ErrNoSuchAlias if it is + // absent. + DeleteAlias(ctx context.Context, alias string) error + + // Alias resolves an alias, returning ErrNoSuchAlias if it is absent. + // The creator comes back with it because the permission check on + // deleting an alias is partly about who made it. + Alias(ctx context.Context, alias string) (RoomAlias, error) + + // AliasesForRoom lists the local aliases pointing at a room, sorted. + AliasesForRoom(ctx context.Context, roomID string) ([]string, error) + + // SetPublished adds or removes a room from the public room directory, + // returning ErrNoSuchRoom for a room that does not exist. + SetPublished(ctx context.Context, roomID string, published bool) error + + // PublishedRooms returns a page of the public room directory ordered by + // descending joined-member count, plus the total number of published + // rooms matching the filter. + PublishedRooms(ctx context.Context, filter RoomFilter) ([]RoomSummary, int, error) + + // ListRooms returns a page of every room for the admin API, plus the + // total matching the filter. + ListRooms(ctx context.Context, filter RoomFilter) ([]RoomSummary, int, error) + + // RoomSummary returns one room with its denormalised state, returning + // ErrNoSuchRoom if it is absent. + RoomSummary(ctx context.Context, roomID string) (RoomSummary, error) + + // PurgeRoom deletes a room and everything belonging to it, returning + // ErrNoSuchRoom for a room that does not exist. It is the admin API's + // last resort and has no undo. + PurgeRoom(ctx context.Context, roomID string) error + + // RoomStats returns the aggregate counts for the admin dashboard. + RoomStats(ctx context.Context) (RoomStats, error) +} diff --git a/server/internal/storage/rooms_test.go b/server/internal/storage/rooms_test.go new file mode 100644 index 0000000..9c7ea7f --- /dev/null +++ b/server/internal/storage/rooms_test.go @@ -0,0 +1,552 @@ +package storage_test + +import ( + "database/sql" + "errors" + "os" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// migratedRoomDB opens a driver, migrates it and empties the roomserver +// tables, which is what every room-store test needs first. +func migratedRoomDB(t *testing.T, driver string) storage.Database { + t.Helper() + db := openTestDB(t, driver) + if err := db.Migrate(t.Context()); err != nil { + t.Fatalf("Migrate: %v", err) + } + resetRoomTables(t, driver) + return db +} + +// resetRoomTables empties the roomserver tables and rewinds the counters. +// +// SQLite gets a fresh file per test. The PostgreSQL tests share one +// database, so without this the second test to create "!room" would collide +// with the first — and only in CI, where PostgreSQL is available. +func resetRoomTables(t *testing.T, driver string) { + t.Helper() + if driver != storage.DriverPostgres { + return + } + + db, err := sql.Open("pgx", os.Getenv("WARP_TEST_POSTGRES_DSN")) + if err != nil { + t.Fatalf("open postgres to reset the room tables: %v", err) + } + defer func() { _ = db.Close() }() + + if _, err := db.ExecContext(t.Context(), `TRUNCATE + roomserver_state_entries, roomserver_state_snapshots, + roomserver_current_state, roomserver_extremities, + roomserver_transactions, roomserver_aliases, + roomserver_events, roomserver_rooms`); err != nil { + t.Fatalf("reset the room tables: %v", err) + } + if _, err := db.ExecContext(t.Context(), + `UPDATE roomserver_sequences SET value = 0`); err != nil { + t.Fatalf("rewind the room counters: %v", err) + } +} + +// sqlDB reaches the underlying pool, for the one assertion that is about +// the schema rather than the interface: that identical state is stored once. +func sqlDB(t *testing.T, db storage.Database) *sql.DB { + t.Helper() + pool, ok := db.(interface{ DB() *sql.DB }) + if !ok { + t.Skipf("driver %s does not expose a database/sql pool", db.Driver()) + } + return pool.DB() +} + +// The room store never parses an event, so these tests feed it event JSON +// that is only plausible. What is under test is the bookkeeping: stream +// ordering, state snapshots, current state and the member counters. +const testRoomID = "!room:warp.example" + +func newTestRoom(id string) storage.NewRoom { + return storage.NewRoom{ + ID: id, + Version: "11", + Creator: "@alice:warp.example", + CreatedAtMS: 1_700_000_000_000, + } +} + +// stateKey returns a pointer to s, for StoredEvent.StateKey. +func stateKey(s string) *string { return &s } + +func newTestEvent(roomID, eventID, eventType string, key *string) storage.StoredEvent { + return storage.StoredEvent{ + RoomID: roomID, + EventID: eventID, + Type: eventType, + StateKey: key, + Sender: "@alice:warp.example", + Depth: 1, + OriginServerTS: 1_700_000_000_000, + JSON: []byte(`{"type":"` + eventType + `"}`), + } +} + +func TestRoomCreateAndRead(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + + if _, err := rooms.Room(ctx, testRoomID); !errors.Is(err, storage.ErrNoSuchRoom) { + t.Errorf("Room of an unknown room = %v, want ErrNoSuchRoom", err) + } + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); !errors.Is(err, storage.ErrRoomExists) { + t.Errorf("CreateRoom twice = %v, want ErrRoomExists", err) + } + + room, err := rooms.Room(ctx, testRoomID) + if err != nil { + t.Fatalf("Room: %v", err) + } + if room.Version != "11" || room.Creator != "@alice:warp.example" || room.Published { + t.Errorf("Room returned %+v", room) + } + }) + } +} + +func TestRoomWriteStreamOrderingAndState(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + + // The create event: no state before it, and it is the room's + // only state and only extremity. + create := newTestEvent(testRoomID, "$create", "m.room.create", stateKey("")) + createState := []storage.StateEntry{ + {Type: "m.room.create", StateKey: "", EventID: "$create"}, + } + ordering, err := rooms.Write(ctx, storage.RoomWrite{ + Event: create, + StateDelta: createState, + Extremities: []string{"$create"}, + }) + if err != nil { + t.Fatalf("Write the create event: %v", err) + } + if ordering != 1 { + t.Errorf("the first event got stream ordering %d, want 1", ordering) + } + + // A join, authorised against the state above. + join := newTestEvent(testRoomID, "$join", "m.room.member", stateKey("@alice:warp.example")) + join.Depth = 2 + ordering, err = rooms.Write(ctx, storage.RoomWrite{ + Event: join, + StateBefore: createState, + StateDelta: []storage.StateEntry{{ + Type: "m.room.member", StateKey: "@alice:warp.example", + EventID: "$join", Membership: "join", + }}, + Extremities: []string{"$join"}, + }) + if err != nil { + t.Fatalf("Write the join: %v", err) + } + if ordering != 2 { + t.Errorf("the second event got stream ordering %d, want 2", ordering) + } + + // A message, which changes no state at all. + message := newTestEvent(testRoomID, "$message", "m.room.message", nil) + message.Depth = 3 + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: message, + StateBefore: append(createState, storage.StateEntry{ + Type: "m.room.member", StateKey: "@alice:warp.example", EventID: "$join", + }), + Extremities: []string{"$message"}, + }); err != nil { + t.Fatalf("Write the message: %v", err) + } + + state, err := rooms.CurrentState(ctx, testRoomID) + if err != nil { + t.Fatalf("CurrentState: %v", err) + } + if len(state) != 2 { + t.Fatalf("current state has %d entries, want 2: %+v", len(state), state) + } + + stored, err := rooms.Event(ctx, testRoomID, "$message") + if err != nil { + t.Fatalf("Event: %v", err) + } + if stored.IsState() { + t.Error("a message event came back as a state event") + } + if stored.StreamOrdering != 3 { + t.Errorf("stream ordering = %d, want 3", stored.StreamOrdering) + } + if _, err := rooms.Event(ctx, testRoomID, "$nope"); !errors.Is(err, storage.ErrNoSuchEvent) { + t.Errorf("Event of an unknown ID = %v, want ErrNoSuchEvent", err) + } + + // The state recorded against the message is the state it was + // authorised against, not the state now. + before, err := rooms.StateBefore(ctx, testRoomID, "$message") + if err != nil { + t.Fatalf("StateBefore: %v", err) + } + if len(before) != 2 { + t.Errorf("the state before the message has %d entries, want 2", len(before)) + } + + extremities, err := rooms.Extremities(ctx, testRoomID) + if err != nil { + t.Fatalf("Extremities: %v", err) + } + if len(extremities) != 1 || extremities[0].EventID != "$message" { + t.Errorf("extremities = %+v, want just $message", extremities) + } + + // The create event is a state event with the empty state key, + // which must not read back as "no state key". + storedCreate, err := rooms.Event(ctx, testRoomID, "$create") + if err != nil { + t.Fatalf("Event of the create event: %v", err) + } + if !storedCreate.IsState() { + t.Error("the create event came back without a state key") + } else if *storedCreate.StateKey != "" { + t.Errorf("the create event state key = %q, want empty", *storedCreate.StateKey) + } + }) + } +} + +// TestRoomStateSnapshotsAreShared is the reason a per-event state snapshot +// is affordable: two events authorised against the same state must not +// store it twice. +func TestRoomStateSnapshotsAreShared(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + db := migratedRoomDB(t, driver) + rooms := db.Rooms() + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + + state := []storage.StateEntry{ + {Type: "m.room.create", StateKey: "", EventID: "$create"}, + } + for _, id := range []string{"$a", "$b", "$c"} { + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: newTestEvent(testRoomID, id, "m.room.message", nil), + StateBefore: state, + Extremities: []string{id}, + }); err != nil { + t.Fatalf("Write %s: %v", id, err) + } + } + + var snapshots int + if err := sqlDB(t, db).QueryRowContext(ctx, + `SELECT COUNT(*) FROM roomserver_state_snapshots`).Scan(&snapshots); err != nil { + t.Fatalf("count snapshots: %v", err) + } + if snapshots != 1 { + t.Errorf("three events sharing one state stored %d snapshots, want 1", snapshots) + } + }) + } +} + +func TestRoomMembershipAndCounts(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + + write := func(eventID, userID, membership string) { + t.Helper() + e := newTestEvent(testRoomID, eventID, "m.room.member", stateKey(userID)) + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: e, + StateDelta: []storage.StateEntry{{ + Type: "m.room.member", StateKey: userID, + EventID: eventID, Membership: membership, + }}, + Extremities: []string{eventID}, + }); err != nil { + t.Fatalf("Write %s: %v", eventID, err) + } + } + write("$alice", "@alice:warp.example", "join") + write("$bob", "@bob:warp.example", "invite") + write("$carol", "@carol:warp.example", "join") + + // A user who was never in the room gets an empty membership + // rather than an error. + absent, err := rooms.Membership(ctx, testRoomID, "@dave:warp.example") + if err != nil { + t.Fatalf("Membership of a stranger: %v", err) + } + if absent.Membership != "" { + t.Errorf("membership of a stranger = %q, want empty", absent.Membership) + } + + alice, err := rooms.Membership(ctx, testRoomID, "@alice:warp.example") + if err != nil { + t.Fatalf("Membership: %v", err) + } + if alice.Membership != "join" || alice.EventID != "$alice" { + t.Errorf("Membership returned %+v", alice) + } + + joined, err := rooms.MembershipEntries(ctx, testRoomID, []string{"join"}) + if err != nil { + t.Fatalf("MembershipEntries: %v", err) + } + if len(joined) != 2 { + t.Errorf("%d joined members, want 2: %+v", len(joined), joined) + } + + all, err := rooms.Members(ctx, testRoomID, nil) + if err != nil { + t.Fatalf("Members: %v", err) + } + if len(all) != 3 { + t.Errorf("%d member events, want 3", len(all)) + } + + // Leaving replaces the entry rather than adding one, and the + // member count follows. + write("$aliceleft", "@alice:warp.example", "leave") + summary, err := rooms.RoomSummary(ctx, testRoomID) + if err != nil { + t.Fatalf("RoomSummary: %v", err) + } + if summary.JoinedMembers != 1 { + t.Errorf("joined members = %d, want 1", summary.JoinedMembers) + } + + forAlice, err := rooms.RoomsForUser(ctx, "@alice:warp.example", []string{"join"}) + if err != nil { + t.Fatalf("RoomsForUser: %v", err) + } + if len(forAlice) != 0 { + t.Errorf("alice is still joined to %v after leaving", forAlice) + } + forCarol, err := rooms.RoomsForUser(ctx, "@carol:warp.example", []string{"join"}) + if err != nil { + t.Fatalf("RoomsForUser: %v", err) + } + if len(forCarol) != 1 || forCarol[0] != testRoomID { + t.Errorf("carol's joined rooms = %v, want [%s]", forCarol, testRoomID) + } + }) + } +} + +func TestRoomRedactionAndTransactions(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + + ref := storage.TransactionRef{Localpart: "alice", DeviceID: "DEV", TxnID: "txn1"} + if _, err := rooms.TransactionEvent(ctx, ref); !errors.Is(err, storage.ErrNoSuchEvent) { + t.Errorf("TransactionEvent of an unseen ID = %v, want ErrNoSuchEvent", err) + } + + target := newTestEvent(testRoomID, "$target", "m.room.message", nil) + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: target, + Extremities: []string{"$target"}, + Transaction: &ref, + }); err != nil { + t.Fatalf("Write the target: %v", err) + } + + got, err := rooms.TransactionEvent(ctx, ref) + if err != nil { + t.Fatalf("TransactionEvent: %v", err) + } + if got != "$target" { + t.Errorf("TransactionEvent = %q, want $target", got) + } + + redaction := newTestEvent(testRoomID, "$redaction", "m.room.redaction", nil) + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: redaction, + Extremities: []string{"$redaction"}, + Redacts: "$target", + RedactedJSON: []byte(`{"type":"m.room.message","content":{}}`), + }); err != nil { + t.Fatalf("Write the redaction: %v", err) + } + + stored, err := rooms.Event(ctx, testRoomID, "$target") + if err != nil { + t.Fatalf("Event: %v", err) + } + if stored.RedactedBy != "$redaction" { + t.Errorf("redacted_by = %q, want $redaction", stored.RedactedBy) + } + if string(stored.JSON) != `{"type":"m.room.message","content":{}}` { + t.Errorf("the stored event was not replaced with its redacted form: %s", stored.JSON) + } + }) + } +} + +func TestRoomAliasesAndDirectory(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil { + t.Fatalf("CreateRoom: %v", err) + } + + const aliasName = "#general:warp.example" + newAlias := storage.RoomAlias{ + Alias: aliasName, + RoomID: testRoomID, + Creator: "@alice:warp.example", + CreatedAtMS: 1, + } + if _, err := rooms.Alias(ctx, aliasName); !errors.Is(err, storage.ErrNoSuchAlias) { + t.Errorf("Alias of an unknown alias = %v, want ErrNoSuchAlias", err) + } + if err := rooms.CreateAlias(ctx, newAlias); err != nil { + t.Fatalf("CreateAlias: %v", err) + } + taken := newAlias + taken.Creator = "@bob:warp.example" + if err := rooms.CreateAlias(ctx, taken); !errors.Is(err, storage.ErrAliasExists) { + t.Errorf("CreateAlias twice = %v, want ErrAliasExists", err) + } + + stored, err := rooms.Alias(ctx, aliasName) + if err != nil { + t.Fatalf("Alias: %v", err) + } + if stored.RoomID != testRoomID || stored.Creator != "@alice:warp.example" { + t.Errorf("Alias returned %+v", stored) + } + aliases, err := rooms.AliasesForRoom(ctx, testRoomID) + if err != nil || len(aliases) != 1 || aliases[0] != aliasName { + t.Fatalf("AliasesForRoom = %v, %v", aliases, err) + } + + if err := rooms.DeleteAlias(ctx, aliasName); err != nil { + t.Fatalf("DeleteAlias: %v", err) + } + if err := rooms.DeleteAlias(ctx, aliasName); !errors.Is(err, storage.ErrNoSuchAlias) { + t.Errorf("DeleteAlias twice = %v, want ErrNoSuchAlias", err) + } + + // The directory is empty until the room is published. + published, total, err := rooms.PublishedRooms(ctx, storage.RoomFilter{}) + if err != nil { + t.Fatalf("PublishedRooms: %v", err) + } + if total != 0 || len(published) != 0 { + t.Errorf("an unpublished room is in the directory: %+v", published) + } + if err := rooms.SetPublished(ctx, testRoomID, true); err != nil { + t.Fatalf("SetPublished: %v", err) + } + if err := rooms.SetPublished(ctx, "!nope:warp.example", true); !errors.Is(err, storage.ErrNoSuchRoom) { + t.Errorf("SetPublished on an unknown room = %v, want ErrNoSuchRoom", err) + } + published, total, err = rooms.PublishedRooms(ctx, storage.RoomFilter{}) + if err != nil { + t.Fatalf("PublishedRooms: %v", err) + } + if total != 1 || len(published) != 1 || published[0].ID != testRoomID { + t.Errorf("PublishedRooms = %+v (total %d)", published, total) + } + }) + } +} + +func TestRoomListingAndPurge(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + rooms := migratedRoomDB(t, driver).Rooms() + + for _, id := range []string{"!a:warp.example", "!b:warp.example"} { + if err := rooms.CreateRoom(ctx, newTestRoom(id)); err != nil { + t.Fatalf("CreateRoom %s: %v", id, err) + } + if _, err := rooms.Write(ctx, storage.RoomWrite{ + Event: newTestEvent(id, "$create"+id, "m.room.create", stateKey("")), + Extremities: []string{"$create" + id}, + Profile: &storage.RoomProfile{Name: "Room " + id, JoinRule: "invite"}, + }); err != nil { + t.Fatalf("Write %s: %v", id, err) + } + } + + listed, total, err := rooms.ListRooms(ctx, storage.RoomFilter{}) + if err != nil { + t.Fatalf("ListRooms: %v", err) + } + if total != 2 || len(listed) != 2 { + t.Fatalf("ListRooms returned %d of %d, want 2 of 2", len(listed), total) + } + if listed[0].Name == "" || listed[0].JoinRule != "invite" { + t.Errorf("the denormalised profile was not stored: %+v", listed[0]) + } + + // The search covers the name as well as the ID. + found, total, err := rooms.ListRooms(ctx, storage.RoomFilter{Search: "room !a"}) + if err != nil { + t.Fatalf("ListRooms with a search: %v", err) + } + if total != 1 || len(found) != 1 || found[0].ID != "!a:warp.example" { + t.Errorf("searching for a name returned %+v (total %d)", found, total) + } + + stats, err := rooms.RoomStats(ctx) + if err != nil { + t.Fatalf("RoomStats: %v", err) + } + if stats.Rooms != 2 || stats.Events != 2 { + t.Errorf("RoomStats = %+v, want 2 rooms and 2 events", stats) + } + + if err := rooms.PurgeRoom(ctx, "!a:warp.example"); err != nil { + t.Fatalf("PurgeRoom: %v", err) + } + if err := rooms.PurgeRoom(ctx, "!a:warp.example"); !errors.Is(err, storage.ErrNoSuchRoom) { + t.Errorf("PurgeRoom twice = %v, want ErrNoSuchRoom", err) + } + if _, err := rooms.Room(ctx, "!a:warp.example"); !errors.Is(err, storage.ErrNoSuchRoom) { + t.Errorf("the purged room is still there: %v", err) + } + if _, err := rooms.Event(ctx, "!b:warp.example", "$create!b:warp.example"); err != nil { + t.Errorf("purging one room disturbed another: %v", err) + } + }) + } +} diff --git a/server/internal/storage/sqlite/migrations/0003_roomserver.sql b/server/internal/storage/sqlite/migrations/0003_roomserver.sql new file mode 100644 index 0000000..d32eed7 --- /dev/null +++ b/server/internal/storage/sqlite/migrations/0003_roomserver.sql @@ -0,0 +1,165 @@ +-- Rooms, events and room state: the roomserver tables. +-- +-- Events are keyed by their event ID rather than by a numeric surrogate. +-- An event ID is 44 bytes of base64 and appears in prev_events, +-- auth_events, state snapshots and client requests alike, so a surrogate +-- would have to be translated at every one of those boundaries; the +-- indexes below are what make the text key cheap enough. +-- +-- Timestamps are milliseconds since the Unix epoch, as everywhere else in +-- Warp. + +CREATE TABLE IF NOT EXISTS roomserver_rooms ( + room_id TEXT NOT NULL PRIMARY KEY, + -- Fixed at creation. Nothing can read an event without it: the room + -- version decides the event format, the redaction algorithm and the + -- authorisation rules. + room_version TEXT NOT NULL, + creator TEXT NOT NULL, + is_published BOOLEAN NOT NULL DEFAULT 0, + created_at_ms BIGINT NOT NULL, + + -- The denormalised room listing columns. Everything below is derivable + -- from the room's current state and is kept here anyway, because the + -- public room directory orders every published room by member count + -- and pages the result: that cannot be done by reading each room's + -- state in turn. The roomserver recomputes them when the state behind + -- them changes; joined_members the store counts for itself. + name TEXT NOT NULL DEFAULT '', + topic TEXT NOT NULL DEFAULT '', + avatar_url TEXT NOT NULL DEFAULT '', + canonical_alias TEXT NOT NULL DEFAULT '', + join_rule TEXT NOT NULL DEFAULT '', + history_visibility TEXT NOT NULL DEFAULT '', + guest_access TEXT NOT NULL DEFAULT '', + room_type TEXT NOT NULL DEFAULT '', + joined_members BIGINT NOT NULL DEFAULT 0 +); + +-- The public room directory's own ordering: published rooms, biggest first. +CREATE INDEX IF NOT EXISTS roomserver_rooms_published_idx + ON roomserver_rooms (is_published, joined_members); + +-- One counter row per stream, incremented inside the write transaction. +-- +-- Neither AUTOINCREMENT nor a sequence would do: SQLite and PostgreSQL +-- spell those differently, and a PostgreSQL sequence is explicitly not +-- transactional, so a rolled-back write would leave a hole in the stream +-- that /sync would have to be taught to skip. An UPDATE on this row takes +-- a row lock in both engines and gives out positions with no gaps. +CREATE TABLE IF NOT EXISTS roomserver_sequences ( + name TEXT NOT NULL PRIMARY KEY, + value BIGINT NOT NULL +); + +INSERT INTO roomserver_sequences (name, value) VALUES ('stream_ordering', 0) + ON CONFLICT (name) DO NOTHING; +INSERT INTO roomserver_sequences (name, value) VALUES ('state_snapshot', 0) + ON CONFLICT (name) DO NOTHING; + +-- A state snapshot is the complete state of a room at one point, stored +-- once and shared by every event it applies to. Consecutive messages in a +-- room whose state is not changing all point at the same snapshot, so the +-- cost is one snapshot per state change rather than one per event. +CREATE TABLE IF NOT EXISTS roomserver_state_snapshots ( + snapshot_id BIGINT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + -- SHA-256 over the sorted (type, state_key, event_id) triples: the + -- identity of the snapshot's contents, which is what lets two + -- identical states share a row. + state_hash TEXT NOT NULL, + UNIQUE (room_id, state_hash) +); + +CREATE TABLE IF NOT EXISTS roomserver_state_entries ( + snapshot_id BIGINT NOT NULL REFERENCES roomserver_state_snapshots (snapshot_id) ON DELETE CASCADE, + event_type TEXT NOT NULL, + state_key TEXT NOT NULL, + event_id TEXT NOT NULL, + PRIMARY KEY (snapshot_id, event_type, state_key) +); + +CREATE TABLE IF NOT EXISTS roomserver_events ( + event_id TEXT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + -- This server's own monotonic position for the event: the order it was + -- persisted in, which is the order clients are shown it in. + stream_ordering BIGINT NOT NULL, + event_type TEXT NOT NULL, + -- NULL for a message event. The empty string is a valid state key, so + -- "no state key" cannot be spelled as ''. + state_key TEXT, + sender TEXT NOT NULL, + depth BIGINT NOT NULL, + origin_server_ts BIGINT NOT NULL, + -- The event as canonical JSON, byte for byte as it was signed. Every + -- other column here is a projection of it, written out to be indexed. + event_json TEXT NOT NULL, + -- The state of the room as of this event's prev_events, 0 for the + -- create event. Keeping it per event is what lets the state after a + -- forward extremity be worked out, which state resolution needs when a + -- room has more than one. + state_before_id BIGINT NOT NULL DEFAULT 0, + -- The redaction that applies to this event, or ''. The JSON above is + -- rewritten to its redacted form at the same time; this records which + -- event did it so that unsigned.redacted_because can be filled in. + redacted_by TEXT NOT NULL DEFAULT '' +); + +CREATE UNIQUE INDEX IF NOT EXISTS roomserver_events_stream_idx + ON roomserver_events (stream_ordering); + +-- Paging a room's timeline, which is what /messages and /sync do. +CREATE INDEX IF NOT EXISTS roomserver_events_room_stream_idx + ON roomserver_events (room_id, stream_ordering); + +-- The forward extremities: the events with no children, and so the +-- prev_events of whatever is sent next. +CREATE TABLE IF NOT EXISTS roomserver_extremities ( + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + event_id TEXT NOT NULL, + PRIMARY KEY (room_id, event_id) +); + +-- The room's resolved current state: the state after its extremities. +-- +-- It is derivable from the events, and is stored anyway because every +-- authorisation decision reads it. The membership column is a denormalised +-- copy of content.membership on m.room.member rows, so that "the rooms +-- @alice is joined to" is an index lookup rather than a scan that parses +-- JSON. +CREATE TABLE IF NOT EXISTS roomserver_current_state ( + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + event_type TEXT NOT NULL, + state_key TEXT NOT NULL, + event_id TEXT NOT NULL, + membership TEXT NOT NULL DEFAULT '', + PRIMARY KEY (room_id, event_type, state_key) +); + +CREATE INDEX IF NOT EXISTS roomserver_current_state_membership_idx + ON roomserver_current_state (event_type, state_key, membership); + +-- Client transaction IDs, so that a retried request returns the event the +-- first attempt created instead of sending a second one. +CREATE TABLE IF NOT EXISTS roomserver_transactions ( + localpart TEXT NOT NULL, + device_id TEXT NOT NULL, + txn_id TEXT NOT NULL, + event_id TEXT NOT NULL, + room_id TEXT NOT NULL, + sent_at_ms BIGINT NOT NULL, + PRIMARY KEY (localpart, device_id, txn_id) +); + +-- Local room aliases. Remote aliases are never stored: they are resolved +-- over federation every time, because the other server may have moved them. +CREATE TABLE IF NOT EXISTS roomserver_aliases ( + alias TEXT NOT NULL PRIMARY KEY, + room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE, + creator TEXT NOT NULL, + created_at_ms BIGINT NOT NULL +); + +CREATE INDEX IF NOT EXISTS roomserver_aliases_room_idx + ON roomserver_aliases (room_id); diff --git a/server/internal/storage/sqlstore/rooms.go b/server/internal/storage/sqlstore/rooms.go new file mode 100644 index 0000000..991ee74 --- /dev/null +++ b/server/internal/storage/sqlstore/rooms.go @@ -0,0 +1,987 @@ +package sqlstore + +import ( + "context" + "crypto/sha256" + "database/sql" + "encoding/base64" + "errors" + "fmt" + "slices" + "strings" + + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// Rooms is the shared SQL implementation of storage.RoomStore. +// +// Like the user store it is written once with `?` placeholders and rebound +// per driver. The statements stay inside the syntax both engines agree on: +// ON CONFLICT ... DO UPDATE with `excluded`, LIMIT/OFFSET, and no JSON +// functions at all — what a room's events mean is the roomserver's +// business, and the only reason this package ever looks inside one is that +// it does not. +type Rooms struct { + db *sql.DB + rebind func(string) string + isUnique func(error) bool +} + +// NewRooms builds a room store. +func NewRooms(db *sql.DB, rebind func(string) string, isUnique func(error) bool) *Rooms { + if rebind == nil { + rebind = func(query string) string { return query } + } + if isUnique == nil { + isUnique = func(error) bool { return false } + } + return &Rooms{db: db, rebind: rebind, isUnique: isUnique} +} + +// Page sizes for a room listing. The public room directory is reachable +// without a token on some deployments, so an unbounded ?limit= there would +// be a way to ask the server to serialise every room it knows. +const ( + defaultRoomPageSize = 100 + maxRoomPageSize = 1000 +) + +// The two counters in roomserver_sequences. +const ( + seqStreamOrdering = "stream_ordering" + seqStateSnapshot = "state_snapshot" +) + +// q rebinds a statement for this driver. +func (s *Rooms) q(query string) string { return s.rebind(query) } + +// tx runs fn in a transaction, rolling back on any error. +func (s *Rooms) tx(ctx context.Context, fn func(*sql.Tx) error) error { + tx, err := s.db.BeginTx(ctx, nil) + if err != nil { + return fmt.Errorf("begin transaction: %w", err) + } + defer func() { _ = tx.Rollback() }() + + if err := fn(tx); err != nil { + return err + } + if err := tx.Commit(); err != nil { + return fmt.Errorf("commit transaction: %w", err) + } + return nil +} + +const insertRoomSQL = ` + INSERT INTO roomserver_rooms (room_id, room_version, creator, is_published, created_at_ms) + VALUES (?, ?, ?, ?, ?)` + +// CreateRoom implements storage.RoomStore. +func (s *Rooms) CreateRoom(ctx context.Context, room storage.NewRoom) error { + _, err := s.db.ExecContext(ctx, s.q(insertRoomSQL), + room.ID, room.Version, room.Creator, room.Published, room.CreatedAtMS) + if err != nil { + if s.isUnique(err) { + return fmt.Errorf("create room %q: %w", room.ID, storage.ErrRoomExists) + } + return fmt.Errorf("create room %q: %w", room.ID, err) + } + return nil +} + +const selectRoomSQL = ` + SELECT room_id, room_version, creator, is_published, created_at_ms + FROM roomserver_rooms WHERE room_id = ?` + +// Room implements storage.RoomStore. +func (s *Rooms) Room(ctx context.Context, roomID string) (storage.Room, error) { + var room storage.Room + err := s.db.QueryRowContext(ctx, s.q(selectRoomSQL), roomID).Scan( + &room.ID, &room.Version, &room.Creator, &room.Published, &room.CreatedAtMS) + switch { + case errors.Is(err, sql.ErrNoRows): + return storage.Room{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom) + case err != nil: + return storage.Room{}, fmt.Errorf("read room %q: %w", roomID, err) + } + return room, nil +} + +// Write implements storage.RoomStore. +// +// The order inside the transaction is the order the pieces depend on each +// other: the stream position first, because the event row needs it; the +// state snapshot next, because the event row points at it; then the event, +// then everything that refers to the event by ID. +func (s *Rooms) Write(ctx context.Context, w storage.RoomWrite) (int64, error) { + var ordering int64 + err := s.tx(ctx, func(tx *sql.Tx) error { + var err error + if ordering, err = s.nextSequence(ctx, tx, seqStreamOrdering); err != nil { + return err + } + + snapshotID, err := s.stateSnapshot(ctx, tx, w.Event.RoomID, w.StateBefore) + if err != nil { + return err + } + + if err := s.insertEvent(ctx, tx, w.Event, ordering, snapshotID); err != nil { + return err + } + if w.Redacts != "" { + if err := s.applyRedaction(ctx, tx, w.Event.RoomID, w.Redacts, + w.Event.EventID, w.RedactedJSON); err != nil { + return err + } + } + if err := s.applyStateDelta(ctx, tx, w); err != nil { + return err + } + if err := s.replaceExtremities(ctx, tx, w.Event.RoomID, w.Extremities); err != nil { + return err + } + if w.Profile != nil { + if err := s.updateProfile(ctx, tx, w.Event.RoomID, *w.Profile); err != nil { + return err + } + } + // The member count is recomputed whenever current state moved, + // because that is the only thing that can change it. + if len(w.StateDelta) > 0 || w.StateDeltaIsWholeState { + if err := s.recountMembers(ctx, tx, w.Event.RoomID); err != nil { + return err + } + } + if w.Transaction != nil { + if err := s.recordTransaction(ctx, tx, *w.Transaction, + w.Event.EventID, w.Event.RoomID, w.Event.OriginServerTS); err != nil { + return err + } + } + return nil + }) + if err != nil { + return 0, fmt.Errorf("write event %s: %w", w.Event.EventID, err) + } + return ordering, nil +} + +const ( + bumpSequenceSQL = `UPDATE roomserver_sequences SET value = value + 1 WHERE name = ?` + selectSequenceSQL = `SELECT value FROM roomserver_sequences WHERE name = ?` +) + +// nextSequence hands out the next value of a counter. +// +// The UPDATE takes a row lock that both engines hold until the transaction +// ends, so two concurrent writers queue rather than collide, and a value +// handed out by a transaction that then rolls back is never used — which is +// what keeps the stream gapless. A database sequence would be faster and +// would not have that property. +func (s *Rooms) nextSequence(ctx context.Context, tx *sql.Tx, name string) (int64, error) { + if _, err := tx.ExecContext(ctx, s.q(bumpSequenceSQL), name); err != nil { + return 0, fmt.Errorf("advance the %s counter: %w", name, err) + } + var value int64 + if err := tx.QueryRowContext(ctx, s.q(selectSequenceSQL), name).Scan(&value); err != nil { + return 0, fmt.Errorf("read the %s counter: %w", name, err) + } + return value, nil +} + +const ( + selectSnapshotSQL = ` + SELECT snapshot_id FROM roomserver_state_snapshots + WHERE room_id = ? AND state_hash = ?` + insertSnapshotSQL = ` + INSERT INTO roomserver_state_snapshots (snapshot_id, room_id, state_hash) + VALUES (?, ?, ?)` + insertStateEntrySQL = ` + INSERT INTO roomserver_state_entries (snapshot_id, event_type, state_key, event_id) + VALUES (?, ?, ?, ?)` +) + +// stateSnapshot stores a state set, re-using the row of an identical one. +// +// Deduplication is what makes a per-event state snapshot affordable: every +// message in a room whose state is not changing resolves to the snapshot +// the previous one already stored, so the table grows with state changes +// rather than with traffic. An empty state set is snapshot 0 and stores +// nothing, which is the create event's case. +func (s *Rooms) stateSnapshot(ctx context.Context, tx *sql.Tx, roomID string, entries []storage.StateEntry) (int64, error) { + if len(entries) == 0 { + return 0, nil + } + hash := stateHash(entries) + + var id int64 + err := tx.QueryRowContext(ctx, s.q(selectSnapshotSQL), roomID, hash).Scan(&id) + switch { + case err == nil: + return id, nil + case !errors.Is(err, sql.ErrNoRows): + return 0, fmt.Errorf("look up state snapshot: %w", err) + } + + if id, err = s.nextSequence(ctx, tx, seqStateSnapshot); err != nil { + return 0, err + } + if _, err := tx.ExecContext(ctx, s.q(insertSnapshotSQL), id, roomID, hash); err != nil { + return 0, fmt.Errorf("insert state snapshot: %w", err) + } + for _, e := range entries { + if _, err := tx.ExecContext(ctx, s.q(insertStateEntrySQL), + id, e.Type, e.StateKey, e.EventID); err != nil { + return 0, fmt.Errorf("insert state entry %s/%s: %w", e.Type, e.StateKey, err) + } + } + return id, nil +} + +// stateHash identifies a state set by its contents. +// +// The entries are sorted first so that the same state always hashes the +// same whatever order it was assembled in, and each field is written with +// its length so that no two different sets can be run together into the +// same bytes. +func stateHash(entries []storage.StateEntry) string { + sorted := slices.Clone(entries) + slices.SortFunc(sorted, func(a, b storage.StateEntry) int { + if c := strings.Compare(a.Type, b.Type); c != 0 { + return c + } + return strings.Compare(a.StateKey, b.StateKey) + }) + + h := sha256.New() + for _, e := range sorted { + for _, field := range [...]string{e.Type, e.StateKey, e.EventID} { + fmt.Fprintf(h, "%d:%s", len(field), field) + } + } + return base64.RawURLEncoding.EncodeToString(h.Sum(nil)) +} + +const insertEventSQL = ` + INSERT INTO roomserver_events ( + event_id, room_id, stream_ordering, event_type, state_key, + sender, depth, origin_server_ts, event_json, state_before_id) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)` + +func (s *Rooms) insertEvent(ctx context.Context, tx *sql.Tx, e storage.StoredEvent, ordering, snapshotID int64) error { + _, err := tx.ExecContext(ctx, s.q(insertEventSQL), + e.EventID, e.RoomID, ordering, e.Type, e.StateKey, + e.Sender, e.Depth, e.OriginServerTS, string(e.JSON), snapshotID) + if err != nil { + if s.isUnique(err) { + // The same event twice is not a failure worth propagating: an + // event ID is a hash of its contents, so the second copy says + // exactly what the first one did. + return nil + } + return fmt.Errorf("insert event: %w", err) + } + return nil +} + +const applyRedactionSQL = ` + UPDATE roomserver_events SET event_json = ?, redacted_by = ? + WHERE room_id = ? AND event_id = ? AND redacted_by = ''` + +// applyRedaction replaces a stored event with its redacted form. +// +// The condition on redacted_by makes a second redaction of the same event a +// no-op rather than a rewrite: the first one already stripped the content, +// and the event may only be redacted once. +func (s *Rooms) applyRedaction(ctx context.Context, tx *sql.Tx, roomID, targetID, redactionID string, redacted []byte) error { + _, err := tx.ExecContext(ctx, s.q(applyRedactionSQL), + string(redacted), redactionID, roomID, targetID) + if err != nil { + return fmt.Errorf("apply redaction to %s: %w", targetID, err) + } + return nil +} + +const ( + upsertCurrentStateSQL = ` + INSERT INTO roomserver_current_state (room_id, event_type, state_key, event_id, membership) + VALUES (?, ?, ?, ?, ?) + ON CONFLICT (room_id, event_type, state_key) DO UPDATE + SET event_id = excluded.event_id, membership = excluded.membership` + deleteCurrentStateSQL = `DELETE FROM roomserver_current_state WHERE room_id = ?` +) + +// applyStateDelta moves the room's current state. +// +// The usual case is a single tuple to upsert. A whole-state write — which +// only state resolution produces — clears the table for the room first, +// because tuples the resolution dropped have to go. +func (s *Rooms) applyStateDelta(ctx context.Context, tx *sql.Tx, w storage.RoomWrite) error { + if len(w.StateDelta) == 0 && !w.StateDeltaIsWholeState { + return nil + } + if w.StateDeltaIsWholeState { + if _, err := tx.ExecContext(ctx, s.q(deleteCurrentStateSQL), w.Event.RoomID); err != nil { + return fmt.Errorf("clear current state: %w", err) + } + } + for _, e := range w.StateDelta { + if _, err := tx.ExecContext(ctx, s.q(upsertCurrentStateSQL), + w.Event.RoomID, e.Type, e.StateKey, e.EventID, e.Membership); err != nil { + return fmt.Errorf("set current state %s/%s: %w", e.Type, e.StateKey, err) + } + } + return nil +} + +const ( + deleteExtremitiesSQL = `DELETE FROM roomserver_extremities WHERE room_id = ?` + insertExtremitySQL = ` + INSERT INTO roomserver_extremities (room_id, event_id) VALUES (?, ?) + ON CONFLICT (room_id, event_id) DO NOTHING` +) + +func (s *Rooms) replaceExtremities(ctx context.Context, tx *sql.Tx, roomID string, extremities []string) error { + if len(extremities) == 0 { + return nil + } + if _, err := tx.ExecContext(ctx, s.q(deleteExtremitiesSQL), roomID); err != nil { + return fmt.Errorf("clear forward extremities: %w", err) + } + for _, id := range extremities { + if _, err := tx.ExecContext(ctx, s.q(insertExtremitySQL), roomID, id); err != nil { + return fmt.Errorf("insert forward extremity %s: %w", id, err) + } + } + return nil +} + +const updateProfileSQL = ` + UPDATE roomserver_rooms SET + name = ?, topic = ?, avatar_url = ?, canonical_alias = ?, + join_rule = ?, history_visibility = ?, guest_access = ?, room_type = ? + WHERE room_id = ?` + +func (s *Rooms) updateProfile(ctx context.Context, tx *sql.Tx, roomID string, p storage.RoomProfile) error { + _, err := tx.ExecContext(ctx, s.q(updateProfileSQL), + p.Name, p.Topic, p.AvatarURL, p.CanonicalAlias, + p.JoinRule, p.HistoryVisibility, p.GuestAccess, p.RoomType, roomID) + if err != nil { + return fmt.Errorf("update room profile: %w", err) + } + return nil +} + +const recountMembersSQL = ` + UPDATE roomserver_rooms SET joined_members = ( + SELECT COUNT(*) FROM roomserver_current_state + WHERE room_id = ? AND membership = 'join' + ) WHERE room_id = ?` + +func (s *Rooms) recountMembers(ctx context.Context, tx *sql.Tx, roomID string) error { + if _, err := tx.ExecContext(ctx, s.q(recountMembersSQL), roomID, roomID); err != nil { + return fmt.Errorf("recount joined members: %w", err) + } + return nil +} + +const insertTransactionSQL = ` + INSERT INTO roomserver_transactions (localpart, device_id, txn_id, event_id, room_id, sent_at_ms) + VALUES (?, ?, ?, ?, ?, ?) + ON CONFLICT (localpart, device_id, txn_id) DO NOTHING` + +func (s *Rooms) recordTransaction(ctx context.Context, tx *sql.Tx, ref storage.TransactionRef, eventID, roomID string, atMS int64) error { + _, err := tx.ExecContext(ctx, s.q(insertTransactionSQL), + ref.Localpart, ref.DeviceID, ref.TxnID, eventID, roomID, atMS) + if err != nil { + return fmt.Errorf("record transaction %q: %w", ref.TxnID, err) + } + return nil +} + +// eventColumns is the select list every event read shares, in the order +// scanEvent expects. +const eventColumns = ` + e.room_id, e.event_id, e.event_type, e.state_key, e.sender, + e.depth, e.origin_server_ts, e.event_json, e.stream_ordering, e.redacted_by` + +// scanEvent reads one row of eventColumns. +func scanEvent(scan func(...any) error) (storage.StoredEvent, error) { + var ( + e storage.StoredEvent + stateKey sql.NullString + body string + ) + err := scan(&e.RoomID, &e.EventID, &e.Type, &stateKey, &e.Sender, + &e.Depth, &e.OriginServerTS, &body, &e.StreamOrdering, &e.RedactedBy) + if err != nil { + return storage.StoredEvent{}, err + } + if stateKey.Valid { + e.StateKey = &stateKey.String + } + e.JSON = []byte(body) + return e, nil +} + +// scanEvents drains a result set of eventColumns rows. +func scanEvents(rows *sql.Rows) ([]storage.StoredEvent, error) { + defer func() { _ = rows.Close() }() + + var out []storage.StoredEvent + for rows.Next() { + e, err := scanEvent(rows.Scan) + if err != nil { + return nil, err + } + out = append(out, e) + } + return out, rows.Err() +} + +var selectEventSQL = ` + SELECT ` + eventColumns + ` + FROM roomserver_events e WHERE e.room_id = ? AND e.event_id = ?` + +// Event implements storage.RoomStore. +func (s *Rooms) Event(ctx context.Context, roomID, eventID string) (storage.StoredEvent, error) { + e, err := scanEvent(s.db.QueryRowContext(ctx, s.q(selectEventSQL), roomID, eventID).Scan) + switch { + case errors.Is(err, sql.ErrNoRows): + return storage.StoredEvent{}, fmt.Errorf("event %q: %w", eventID, storage.ErrNoSuchEvent) + case err != nil: + return storage.StoredEvent{}, fmt.Errorf("read event %q: %w", eventID, err) + } + return e, nil +} + +// EventsByID implements storage.RoomStore. +func (s *Rooms) EventsByID(ctx context.Context, roomID string, eventIDs []string) ([]storage.StoredEvent, error) { + if len(eventIDs) == 0 { + return nil, nil + } + query := `SELECT ` + eventColumns + ` + FROM roomserver_events e + WHERE e.room_id = ? AND e.event_id IN (` + placeholders(len(eventIDs)) + `)` + + args := make([]any, 0, len(eventIDs)+1) + args = append(args, roomID) + for _, id := range eventIDs { + args = append(args, id) + } + + rows, err := s.db.QueryContext(ctx, s.q(query), args...) + if err != nil { + return nil, fmt.Errorf("read events by ID: %w", err) + } + events, err := scanEvents(rows) + if err != nil { + return nil, fmt.Errorf("read events by ID: %w", err) + } + return events, nil +} + +const selectCurrentStateSQL = ` + SELECT event_type, state_key, event_id, membership + FROM roomserver_current_state WHERE room_id = ? + ORDER BY event_type, state_key` + +// CurrentState implements storage.RoomStore. +func (s *Rooms) CurrentState(ctx context.Context, roomID string) ([]storage.StateEntry, error) { + rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateSQL), roomID) + if err != nil { + return nil, fmt.Errorf("read current state of %q: %w", roomID, err) + } + entries, err := scanStateEntries(rows) + if err != nil { + return nil, fmt.Errorf("read current state of %q: %w", roomID, err) + } + return entries, nil +} + +func scanStateEntries(rows *sql.Rows) ([]storage.StateEntry, error) { + defer func() { _ = rows.Close() }() + + var out []storage.StateEntry + for rows.Next() { + var e storage.StateEntry + if err := rows.Scan(&e.Type, &e.StateKey, &e.EventID, &e.Membership); err != nil { + return nil, err + } + out = append(out, e) + } + return out, rows.Err() +} + +var selectCurrentStateEventsSQL = ` + SELECT ` + eventColumns + ` + FROM roomserver_current_state cs + JOIN roomserver_events e ON e.event_id = cs.event_id + WHERE cs.room_id = ? + ORDER BY cs.event_type, cs.state_key` + +// CurrentStateEvents implements storage.RoomStore. +func (s *Rooms) CurrentStateEvents(ctx context.Context, roomID string) ([]storage.StoredEvent, error) { + rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateEventsSQL), roomID) + if err != nil { + return nil, fmt.Errorf("read current state events of %q: %w", roomID, err) + } + events, err := scanEvents(rows) + if err != nil { + return nil, fmt.Errorf("read current state events of %q: %w", roomID, err) + } + return events, nil +} + +var selectCurrentStateEventSQL = ` + SELECT ` + eventColumns + ` + FROM roomserver_current_state cs + JOIN roomserver_events e ON e.event_id = cs.event_id + WHERE cs.room_id = ? AND cs.event_type = ? AND cs.state_key = ?` + +// CurrentStateEvent implements storage.RoomStore. +func (s *Rooms) CurrentStateEvent(ctx context.Context, roomID, eventType, stateKey string) (storage.StoredEvent, error) { + e, err := scanEvent(s.db.QueryRowContext(ctx, + s.q(selectCurrentStateEventSQL), roomID, eventType, stateKey).Scan) + switch { + case errors.Is(err, sql.ErrNoRows): + return storage.StoredEvent{}, fmt.Errorf("state %s/%s of %q: %w", + eventType, stateKey, roomID, storage.ErrNoSuchEvent) + case err != nil: + return storage.StoredEvent{}, fmt.Errorf("read state %s/%s of %q: %w", + eventType, stateKey, roomID, err) + } + return e, nil +} + +const selectStateBeforeSQL = ` + SELECT se.event_type, se.state_key, se.event_id, '' + FROM roomserver_events e + JOIN roomserver_state_entries se ON se.snapshot_id = e.state_before_id + WHERE e.room_id = ? AND e.event_id = ? + ORDER BY se.event_type, se.state_key` + +// StateBefore implements storage.RoomStore. +// +// The membership column is not stored on a snapshot entry: it exists on +// current state to make a membership lookup an index hit, and a historical +// snapshot is only ever read as a whole and then parsed anyway. +func (s *Rooms) StateBefore(ctx context.Context, roomID, eventID string) ([]storage.StateEntry, error) { + rows, err := s.db.QueryContext(ctx, s.q(selectStateBeforeSQL), roomID, eventID) + if err != nil { + return nil, fmt.Errorf("read the state before %q: %w", eventID, err) + } + entries, err := scanStateEntries(rows) + if err != nil { + return nil, fmt.Errorf("read the state before %q: %w", eventID, err) + } + return entries, nil +} + +var selectExtremitiesSQL = ` + SELECT ` + eventColumns + ` + FROM roomserver_extremities x + JOIN roomserver_events e ON e.event_id = x.event_id + WHERE x.room_id = ? + ORDER BY e.stream_ordering` + +// Extremities implements storage.RoomStore. +func (s *Rooms) Extremities(ctx context.Context, roomID string) ([]storage.StoredEvent, error) { + rows, err := s.db.QueryContext(ctx, s.q(selectExtremitiesSQL), roomID) + if err != nil { + return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err) + } + events, err := scanEvents(rows) + if err != nil { + return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err) + } + return events, nil +} + +// Members implements storage.RoomStore. +func (s *Rooms) Members(ctx context.Context, roomID string, memberships []string) ([]storage.StoredEvent, error) { + query := `SELECT ` + eventColumns + ` + FROM roomserver_current_state cs + JOIN roomserver_events e ON e.event_id = cs.event_id + WHERE cs.room_id = ? AND cs.event_type = 'm.room.member'` + args := []any{roomID} + if len(memberships) > 0 { + query += ` AND cs.membership IN (` + placeholders(len(memberships)) + `)` + for _, m := range memberships { + args = append(args, m) + } + } + query += ` ORDER BY cs.state_key` + + rows, err := s.db.QueryContext(ctx, s.q(query), args...) + if err != nil { + return nil, fmt.Errorf("read the members of %q: %w", roomID, err) + } + events, err := scanEvents(rows) + if err != nil { + return nil, fmt.Errorf("read the members of %q: %w", roomID, err) + } + return events, nil +} + +// MembershipEntries implements storage.RoomStore. +func (s *Rooms) MembershipEntries(ctx context.Context, roomID string, memberships []string) ([]storage.MembershipEntry, error) { + query := ` + SELECT state_key, membership, event_id FROM roomserver_current_state + WHERE room_id = ? AND event_type = 'm.room.member'` + args := []any{roomID} + if len(memberships) > 0 { + query += ` AND membership IN (` + placeholders(len(memberships)) + `)` + for _, m := range memberships { + args = append(args, m) + } + } + query += ` ORDER BY state_key` + + rows, err := s.db.QueryContext(ctx, s.q(query), args...) + if err != nil { + return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err) + } + defer func() { _ = rows.Close() }() + + var out []storage.MembershipEntry + for rows.Next() { + var m storage.MembershipEntry + if err := rows.Scan(&m.UserID, &m.Membership, &m.EventID); err != nil { + return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err) + } + out = append(out, m) + } + return out, rows.Err() +} + +const selectMembershipSQL = ` + SELECT state_key, membership, event_id FROM roomserver_current_state + WHERE room_id = ? AND event_type = 'm.room.member' AND state_key = ?` + +// Membership implements storage.RoomStore. +func (s *Rooms) Membership(ctx context.Context, roomID, userID string) (storage.MembershipEntry, error) { + var m storage.MembershipEntry + err := s.db.QueryRowContext(ctx, s.q(selectMembershipSQL), roomID, userID). + Scan(&m.UserID, &m.Membership, &m.EventID) + switch { + case errors.Is(err, sql.ErrNoRows): + // Never a member is not an error: it is the same answer as having + // left, which is what the rules above this ask for. + return storage.MembershipEntry{UserID: userID}, nil + case err != nil: + return storage.MembershipEntry{}, fmt.Errorf("read the membership of %q in %q: %w", userID, roomID, err) + } + return m, nil +} + +// RoomsForUser implements storage.RoomStore. +func (s *Rooms) RoomsForUser(ctx context.Context, userID string, memberships []string) ([]string, error) { + query := ` + SELECT room_id FROM roomserver_current_state + WHERE event_type = 'm.room.member' AND state_key = ?` + args := []any{userID} + if len(memberships) > 0 { + query += ` AND membership IN (` + placeholders(len(memberships)) + `)` + for _, m := range memberships { + args = append(args, m) + } + } + query += ` ORDER BY room_id` + + rows, err := s.db.QueryContext(ctx, s.q(query), args...) + if err != nil { + return nil, fmt.Errorf("read the rooms of %q: %w", userID, err) + } + defer func() { _ = rows.Close() }() + + var out []string + for rows.Next() { + var roomID string + if err := rows.Scan(&roomID); err != nil { + return nil, fmt.Errorf("read the rooms of %q: %w", userID, err) + } + out = append(out, roomID) + } + return out, rows.Err() +} + +const selectTransactionSQL = ` + SELECT event_id FROM roomserver_transactions + WHERE localpart = ? AND device_id = ? AND txn_id = ?` + +// TransactionEvent implements storage.RoomStore. +func (s *Rooms) TransactionEvent(ctx context.Context, ref storage.TransactionRef) (string, error) { + var eventID string + err := s.db.QueryRowContext(ctx, s.q(selectTransactionSQL), + ref.Localpart, ref.DeviceID, ref.TxnID).Scan(&eventID) + switch { + case errors.Is(err, sql.ErrNoRows): + return "", fmt.Errorf("transaction %q: %w", ref.TxnID, storage.ErrNoSuchEvent) + case err != nil: + return "", fmt.Errorf("read transaction %q: %w", ref.TxnID, err) + } + return eventID, nil +} + +const insertAliasSQL = ` + INSERT INTO roomserver_aliases (alias, room_id, creator, created_at_ms) + VALUES (?, ?, ?, ?)` + +// CreateAlias implements storage.RoomStore. +func (s *Rooms) CreateAlias(ctx context.Context, alias storage.RoomAlias) error { + _, err := s.db.ExecContext(ctx, s.q(insertAliasSQL), + alias.Alias, alias.RoomID, alias.Creator, alias.CreatedAtMS) + if err != nil { + if s.isUnique(err) { + return fmt.Errorf("create alias %q: %w", alias.Alias, storage.ErrAliasExists) + } + return fmt.Errorf("create alias %q: %w", alias.Alias, err) + } + return nil +} + +const deleteAliasSQL = `DELETE FROM roomserver_aliases WHERE alias = ?` + +// DeleteAlias implements storage.RoomStore. +func (s *Rooms) DeleteAlias(ctx context.Context, alias string) error { + result, err := s.db.ExecContext(ctx, s.q(deleteAliasSQL), alias) + if err != nil { + return fmt.Errorf("delete alias %q: %w", alias, err) + } + return expectOneRow(result, alias, storage.ErrNoSuchAlias) +} + +const selectAliasSQL = ` + SELECT alias, room_id, creator, created_at_ms + FROM roomserver_aliases WHERE alias = ?` + +// Alias implements storage.RoomStore. +func (s *Rooms) Alias(ctx context.Context, alias string) (storage.RoomAlias, error) { + var out storage.RoomAlias + err := s.db.QueryRowContext(ctx, s.q(selectAliasSQL), alias). + Scan(&out.Alias, &out.RoomID, &out.Creator, &out.CreatedAtMS) + switch { + case errors.Is(err, sql.ErrNoRows): + return storage.RoomAlias{}, fmt.Errorf("alias %q: %w", alias, storage.ErrNoSuchAlias) + case err != nil: + return storage.RoomAlias{}, fmt.Errorf("resolve alias %q: %w", alias, err) + } + return out, nil +} + +const selectRoomAliasesSQL = ` + SELECT alias FROM roomserver_aliases WHERE room_id = ? ORDER BY alias` + +// AliasesForRoom implements storage.RoomStore. +func (s *Rooms) AliasesForRoom(ctx context.Context, roomID string) ([]string, error) { + rows, err := s.db.QueryContext(ctx, s.q(selectRoomAliasesSQL), roomID) + if err != nil { + return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err) + } + defer func() { _ = rows.Close() }() + + var out []string + for rows.Next() { + var alias string + if err := rows.Scan(&alias); err != nil { + return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err) + } + out = append(out, alias) + } + return out, rows.Err() +} + +const setPublishedSQL = `UPDATE roomserver_rooms SET is_published = ? WHERE room_id = ?` + +// SetPublished implements storage.RoomStore. +func (s *Rooms) SetPublished(ctx context.Context, roomID string, published bool) error { + result, err := s.db.ExecContext(ctx, s.q(setPublishedSQL), published, roomID) + if err != nil { + return fmt.Errorf("set the published flag of %q: %w", roomID, err) + } + return expectOneRow(result, roomID, storage.ErrNoSuchRoom) +} + +// roomSummaryColumns is the select list a room listing shares, in the order +// scanRoomSummary expects. +const roomSummaryColumns = ` + room_id, room_version, creator, is_published, created_at_ms, + name, topic, avatar_url, canonical_alias, + join_rule, history_visibility, guest_access, room_type, joined_members` + +func scanRoomSummary(scan func(...any) error) (storage.RoomSummary, error) { + var r storage.RoomSummary + err := scan(&r.ID, &r.Version, &r.Creator, &r.Published, &r.CreatedAtMS, + &r.Name, &r.Topic, &r.AvatarURL, &r.CanonicalAlias, + &r.JoinRule, &r.HistoryVisibility, &r.GuestAccess, &r.RoomType, &r.JoinedMembers) + return r, err +} + +// PublishedRooms implements storage.RoomStore. +func (s *Rooms) PublishedRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) { + return s.listRooms(ctx, filter, true) +} + +// ListRooms implements storage.RoomStore. +func (s *Rooms) ListRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) { + return s.listRooms(ctx, filter, false) +} + +// listRooms is the body of both listings: the same query with and without +// the published condition. +// +// Rooms are ordered by descending member count and then by room ID. The +// second key is not decoration: paging a listing whose order is not total +// can show the same room on two pages and skip another, and ties in member +// count are the common case in a directory of small rooms. +func (s *Rooms) listRooms(ctx context.Context, filter storage.RoomFilter, publishedOnly bool) ([]storage.RoomSummary, int, error) { + where := ` WHERE 1 = 1` + var args []any + if publishedOnly { + where += ` AND is_published = ?` + args = append(args, true) + } + if search := strings.TrimSpace(filter.Search); search != "" { + pattern := "%" + strings.ToLower(search) + "%" + where += ` AND (LOWER(room_id) LIKE ? OR LOWER(name) LIKE ? + OR LOWER(topic) LIKE ? OR LOWER(canonical_alias) LIKE ?)` + args = append(args, pattern, pattern, pattern, pattern) + } + + var total int + if err := s.db.QueryRowContext(ctx, + s.q(`SELECT COUNT(*) FROM roomserver_rooms`+where), args...).Scan(&total); err != nil { + return nil, 0, fmt.Errorf("count rooms: %w", err) + } + + limit, offset := pageBounds(filter.Limit, filter.Offset, defaultRoomPageSize, maxRoomPageSize) + query := `SELECT ` + roomSummaryColumns + ` FROM roomserver_rooms` + where + + ` ORDER BY joined_members DESC, room_id LIMIT ? OFFSET ?` + + rows, err := s.db.QueryContext(ctx, s.q(query), append(args, limit, offset)...) + if err != nil { + return nil, 0, fmt.Errorf("list rooms: %w", err) + } + defer func() { _ = rows.Close() }() + + out := make([]storage.RoomSummary, 0, limit) + for rows.Next() { + r, err := scanRoomSummary(rows.Scan) + if err != nil { + return nil, 0, fmt.Errorf("list rooms: %w", err) + } + out = append(out, r) + } + if err := rows.Err(); err != nil { + return nil, 0, fmt.Errorf("list rooms: %w", err) + } + return out, total, nil +} + +var selectRoomSummarySQL = `SELECT ` + roomSummaryColumns + + ` FROM roomserver_rooms WHERE room_id = ?` + +// RoomSummary implements storage.RoomStore. +func (s *Rooms) RoomSummary(ctx context.Context, roomID string) (storage.RoomSummary, error) { + r, err := scanRoomSummary(s.db.QueryRowContext(ctx, s.q(selectRoomSummarySQL), roomID).Scan) + switch { + case errors.Is(err, sql.ErrNoRows): + return storage.RoomSummary{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom) + case err != nil: + return storage.RoomSummary{}, fmt.Errorf("read room %q: %w", roomID, err) + } + return r, nil +} + +// purgeStatements are run in order, children before parents. +// +// Both engines are given ON DELETE CASCADE on these tables, but SQLite +// enforces foreign keys only when the pragma is on, and the transactions +// table has no key at all because it outlives the room it refers to. The +// explicit order is what makes a purge mean the same thing on both. +var purgeStatements = []string{ + `DELETE FROM roomserver_state_entries WHERE snapshot_id IN ( + SELECT snapshot_id FROM roomserver_state_snapshots WHERE room_id = ?)`, + `DELETE FROM roomserver_state_snapshots WHERE room_id = ?`, + `DELETE FROM roomserver_current_state WHERE room_id = ?`, + `DELETE FROM roomserver_extremities WHERE room_id = ?`, + `DELETE FROM roomserver_transactions WHERE room_id = ?`, + `DELETE FROM roomserver_aliases WHERE room_id = ?`, + `DELETE FROM roomserver_events WHERE room_id = ?`, + `DELETE FROM roomserver_rooms WHERE room_id = ?`, +} + +// PurgeRoom implements storage.RoomStore. +func (s *Rooms) PurgeRoom(ctx context.Context, roomID string) error { + err := s.tx(ctx, func(tx *sql.Tx) error { + for i, stmt := range purgeStatements { + result, err := tx.ExecContext(ctx, s.q(stmt), roomID) + if err != nil { + return err + } + // The last statement is the room row itself: no row there means + // there was no room, and the whole purge rolls back so that the + // caller's 404 is the truth. + if i == len(purgeStatements)-1 { + if err := expectOneRow(result, roomID, storage.ErrNoSuchRoom); err != nil { + return err + } + } + } + return nil + }) + if err != nil { + return fmt.Errorf("purge room %q: %w", roomID, err) + } + return nil +} + +const roomStatsSQL = ` + SELECT + (SELECT COUNT(*) FROM roomserver_rooms), + (SELECT COUNT(*) FROM roomserver_rooms WHERE is_published = ?), + (SELECT COUNT(*) FROM roomserver_events)` + +// RoomStats implements storage.RoomStore. +func (s *Rooms) RoomStats(ctx context.Context) (storage.RoomStats, error) { + var stats storage.RoomStats + err := s.db.QueryRowContext(ctx, s.q(roomStatsSQL), true). + Scan(&stats.Rooms, &stats.Published, &stats.Events) + if err != nil { + return storage.RoomStats{}, fmt.Errorf("read room statistics: %w", err) + } + return stats, nil +} + +// placeholders renders n comma-separated `?` placeholders for an IN clause. +func placeholders(n int) string { + return strings.TrimSuffix(strings.Repeat("?, ", n), ", ") +} + +// pageBounds clamps a caller's limit and offset. +func pageBounds(limit, offset, defaultLimit, maxLimit int) (int, int) { + if limit <= 0 { + limit = defaultLimit + } + if limit > maxLimit { + limit = maxLimit + } + if offset < 0 { + offset = 0 + } + return limit, offset +} + +// expectOneRow turns "the UPDATE or DELETE matched nothing" into notFound. +func expectOneRow(result sql.Result, subject string, notFound error) error { + n, err := result.RowsAffected() + if err != nil { + return fmt.Errorf("count the affected rows: %w", err) + } + if n == 0 { + return fmt.Errorf("%q: %w", subject, notFound) + } + return nil +} diff --git a/server/internal/storage/sqlstore/sqlstore.go b/server/internal/storage/sqlstore/sqlstore.go index 16ec8d8..f918e23 100644 --- a/server/internal/storage/sqlstore/sqlstore.go +++ b/server/internal/storage/sqlstore/sqlstore.go @@ -57,6 +57,7 @@ type Store struct { log *slog.Logger settings *Settings users *Users + rooms *Rooms } // New builds a Store. It does not touch the database. @@ -73,6 +74,7 @@ func New(opts Options) (*Store, error) { log: opts.Logger, settings: NewSettings(opts.DB, opts.Settings), users: NewUsers(opts.DB, opts.Rebind, opts.IsUniqueViolation), + rooms: NewRooms(opts.DB, opts.Rebind, opts.IsUniqueViolation), }, nil } @@ -94,6 +96,9 @@ func (s *Store) Settings() storage.SettingsStore { return s.settings } // Users implements storage.Database. func (s *Store) Users() storage.UserStore { return s.users } +// Rooms implements storage.Database. +func (s *Store) Rooms() storage.RoomStore { return s.rooms } + // Migrate implements storage.Database. func (s *Store) Migrate(ctx context.Context) error { ms, err := s.migrations() diff --git a/server/internal/storage/storage.go b/server/internal/storage/storage.go index d517387..45edac2 100644 --- a/server/internal/storage/storage.go +++ b/server/internal/storage/storage.go @@ -64,6 +64,9 @@ type Database interface { // Users is the userapi store: accounts, devices, tokens, account data. Users() UserStore + // Rooms is the roomserver store: rooms, events, room state, aliases. + Rooms() RoomStore + // Close releases the connection pool. Close() error } diff --git a/server/test/complement/run.sh b/server/test/complement/run.sh index 20de58c..bb9550f 100755 --- a/server/test/complement/run.sh +++ b/server/test/complement/run.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Run Complement against a freshly built Warp image. # -# server/test/complement/run.sh # the Phase 1 test set +# server/test/complement/run.sh # the test set for the phases that are done # server/test/complement/run.sh TestLogin # one test # COMPLEMENT_TESTS='Test.*' server/test/complement/run.sh # @@ -17,10 +17,14 @@ REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" : "${COMPLEMENT_DIR:=${REPO_ROOT}/.complement}" : "${COMPLEMENT_IMAGE:=complement-warp}" -# The Phase 1 target set, from PLAN.md section 7. Add to it as phases land -# rather than switching to the whole suite: a pass count that goes down -# because a later phase is missing is noise. -: "${COMPLEMENT_TESTS:=TestRegistration|TestLogin|TestDeviceManagement}" +# The target set for the phases that are done, from PLAN.md section 7. Add +# to it as phases land rather than switching to the whole suite: a pass +# count that goes down because a later phase is missing is noise. +# +# Phase 3 adds the room tests. TestRoomCreationReportsEventsToMyself and +# anything that pages a room's history are left for Phase 4, which is where +# /sync and /messages arrive. +: "${COMPLEMENT_TESTS:=TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*}" if [[ $# -gt 0 ]]; then COMPLEMENT_TESTS="$(IFS='|'; echo "$*")" diff --git a/warp.yaml.example b/warp.yaml.example index c70ff01..5a87b59 100644 --- a/warp.yaml.example +++ b/warp.yaml.example @@ -99,6 +99,15 @@ client_api: register: burst: 5 interval: 2s + # Creating a room writes a dozen events and joining one writes a + # membership event that everybody in the room then sees, so both are + # deliberately slower than a read. + room_create: + burst: 10 + interval: 1s + join: + burst: 10 + interval: 1s federation: # Off by default: a new install is a private homeserver until you say