Phase 3: rooms, local only
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s

Implements PLAN.md Phase 3: the roomserver, the room half of storage, the
Client-Server room endpoints and the admin room endpoints. A client can now
create a room, invite, join, leave, kick, ban, send messages and state,
redact, name a room with an alias and find one in the public directory.
Nothing talks to another server; every operation that would need to is
refused with a message that says so rather than a bare M_FORBIDDEN.

roomserver: one input pipeline every event goes through, whatever sent it —
validate, authorise, resolve, persist, emit. The rules are not
reimplemented here; internal/matrix is pure library code and this is what
connects it to a database and to a room's history. The Client-Server
handlers do no authorisation of their own, deliberately: whether Alice may
kick Bob is a question about the room's power levels, and the one place
that is answered correctly is the authorisation rules inside the pipeline.

A mutex per room serialises writers. Two events racing into one room would
each read the same forward extremities, both become one, and fork the room
for no reason — which would then need state resolution to put back
together. State resolution is wired in anyway for the multi-extremity
case. It is unreachable from outside in this phase, and the alternative is
a pipeline that authorises against one arbitrary branch of a fork without
saying so.

Room versions 3 to 12 all work and are each tested end to end. Version 12
earns that coverage: its room ID is the create event's own ID, its create
event carries no room_id, and the create event is excluded from every other
event's auth_events — so the state auth.Check is given is the event's
auth_events plus the create event, which that version identifies by room ID
rather than citing.

createRoom sends the spec's events in the spec's order, each authorised
against what the ones before it produced. A failure part-way leaves an
incomplete room, which is the lesser evil: the alternative is holding every
event of a new room in memory and writing them in one transaction, making
the pipeline that authorises them a special case. The one failure a second
client can cause at any time, an alias clash, takes the room row back out.

Redactions stay two decisions, as the spec keeps them. The rules let almost
any redaction into the room; whether it applies to its target is a separate
test, run inside the same transaction as the redaction itself so that a
client holding an event ID can always read the stripped event.

storage: events keyed by event ID rather than by a surrogate, since an
event ID appears in prev_events, auth_events, state snapshots and client
requests alike. Stream ordering comes from a counter row bumped inside the
write transaction, because a PostgreSQL sequence is explicitly not
transactional and a rolled-back write would leave a hole in the stream that
/sync would have to be taught to skip. State snapshots are stored per event
and deduplicated by content hash, so the table grows with state changes
rather than with traffic — and that is what makes "the state after a
forward extremity" answerable at all. The room's listing columns are
denormalised onto its row because the public directory orders every
published room by member count and pages the result, which cannot be done
by reading each room's state in turn.

Federation-disabled behaviour from PLAN.md section 3 is enforced here.
Restricted joins are the exception that works: the rule needs a signature
from the authorising user's server verified, and while federation is off
that server is always this one.

adminapi: room list and detail, purge, force-leave, and room and event
counters on the dashboard. clientapi gains its own rate-limit buckets for
createRoom and for joining, both of which write events rather than read
them.

Tested against the real event core and a real database rather than fakes:
what is under test is whether an event this server signs is one the
authorisation rules accept, and a fake of either would only prove that the
pipeline calls what the fake expects. The storage tests run against both
SQL drivers. docs/rooms.md describes the pipeline, the storage decisions
and the known gaps — chief among them that history visibility is
room-level rather than per-event, and that /sync still reports no rooms,
both of which belong with the event stream in Phase 4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5 committed 2026-10-16 10:34:57 +02:00
1 parent 2fbd5f53ce
commit c96f88e994
45 files changed
+9637 -42

No files matched your search

+987
View File
@@ -0,0 +1,987 @@
package sqlstore
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/base64"
"errors"
"fmt"
"slices"
"strings"
"github.com/dariuszjaniak/warp/server/internal/storage"
)
// Rooms is the shared SQL implementation of storage.RoomStore.
//
// Like the user store it is written once with `?` placeholders and rebound
// per driver. The statements stay inside the syntax both engines agree on:
// ON CONFLICT ... DO UPDATE with `excluded`, LIMIT/OFFSET, and no JSON
// functions at all — what a room's events mean is the roomserver's
// business, and the only reason this package ever looks inside one is that
// it does not.
type Rooms struct {
db *sql.DB
rebind func(string) string
isUnique func(error) bool
}
// NewRooms builds a room store.
func NewRooms(db *sql.DB, rebind func(string) string, isUnique func(error) bool) *Rooms {
if rebind == nil {
rebind = func(query string) string { return query }
}
if isUnique == nil {
isUnique = func(error) bool { return false }
}
return &Rooms{db: db, rebind: rebind, isUnique: isUnique}
}
// Page sizes for a room listing. The public room directory is reachable
// without a token on some deployments, so an unbounded ?limit= there would
// be a way to ask the server to serialise every room it knows.
const (
defaultRoomPageSize = 100
maxRoomPageSize = 1000
)
// The two counters in roomserver_sequences.
const (
seqStreamOrdering = "stream_ordering"
seqStateSnapshot = "state_snapshot"
)
// q rebinds a statement for this driver.
func (s *Rooms) q(query string) string { return s.rebind(query) }
// tx runs fn in a transaction, rolling back on any error.
func (s *Rooms) tx(ctx context.Context, fn func(*sql.Tx) error) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("begin transaction: %w", err)
}
defer func() { _ = tx.Rollback() }()
if err := fn(tx); err != nil {
return err
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("commit transaction: %w", err)
}
return nil
}
const insertRoomSQL = `
INSERT INTO roomserver_rooms (room_id, room_version, creator, is_published, created_at_ms)
VALUES (?, ?, ?, ?, ?)`
// CreateRoom implements storage.RoomStore.
func (s *Rooms) CreateRoom(ctx context.Context, room storage.NewRoom) error {
_, err := s.db.ExecContext(ctx, s.q(insertRoomSQL),
room.ID, room.Version, room.Creator, room.Published, room.CreatedAtMS)
if err != nil {
if s.isUnique(err) {
return fmt.Errorf("create room %q: %w", room.ID, storage.ErrRoomExists)
}
return fmt.Errorf("create room %q: %w", room.ID, err)
}
return nil
}
const selectRoomSQL = `
SELECT room_id, room_version, creator, is_published, created_at_ms
FROM roomserver_rooms WHERE room_id = ?`
// Room implements storage.RoomStore.
func (s *Rooms) Room(ctx context.Context, roomID string) (storage.Room, error) {
var room storage.Room
err := s.db.QueryRowContext(ctx, s.q(selectRoomSQL), roomID).Scan(
&room.ID, &room.Version, &room.Creator, &room.Published, &room.CreatedAtMS)
switch {
case errors.Is(err, sql.ErrNoRows):
return storage.Room{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom)
case err != nil:
return storage.Room{}, fmt.Errorf("read room %q: %w", roomID, err)
}
return room, nil
}
// Write implements storage.RoomStore.
//
// The order inside the transaction is the order the pieces depend on each
// other: the stream position first, because the event row needs it; the
// state snapshot next, because the event row points at it; then the event,
// then everything that refers to the event by ID.
func (s *Rooms) Write(ctx context.Context, w storage.RoomWrite) (int64, error) {
var ordering int64
err := s.tx(ctx, func(tx *sql.Tx) error {
var err error
if ordering, err = s.nextSequence(ctx, tx, seqStreamOrdering); err != nil {
return err
}
snapshotID, err := s.stateSnapshot(ctx, tx, w.Event.RoomID, w.StateBefore)
if err != nil {
return err
}
if err := s.insertEvent(ctx, tx, w.Event, ordering, snapshotID); err != nil {
return err
}
if w.Redacts != "" {
if err := s.applyRedaction(ctx, tx, w.Event.RoomID, w.Redacts,
w.Event.EventID, w.RedactedJSON); err != nil {
return err
}
}
if err := s.applyStateDelta(ctx, tx, w); err != nil {
return err
}
if err := s.replaceExtremities(ctx, tx, w.Event.RoomID, w.Extremities); err != nil {
return err
}
if w.Profile != nil {
if err := s.updateProfile(ctx, tx, w.Event.RoomID, *w.Profile); err != nil {
return err
}
}
// The member count is recomputed whenever current state moved,
// because that is the only thing that can change it.
if len(w.StateDelta) > 0 || w.StateDeltaIsWholeState {
if err := s.recountMembers(ctx, tx, w.Event.RoomID); err != nil {
return err
}
}
if w.Transaction != nil {
if err := s.recordTransaction(ctx, tx, *w.Transaction,
w.Event.EventID, w.Event.RoomID, w.Event.OriginServerTS); err != nil {
return err
}
}
return nil
})
if err != nil {
return 0, fmt.Errorf("write event %s: %w", w.Event.EventID, err)
}
return ordering, nil
}
const (
bumpSequenceSQL = `UPDATE roomserver_sequences SET value = value + 1 WHERE name = ?`
selectSequenceSQL = `SELECT value FROM roomserver_sequences WHERE name = ?`
)
// nextSequence hands out the next value of a counter.
//
// The UPDATE takes a row lock that both engines hold until the transaction
// ends, so two concurrent writers queue rather than collide, and a value
// handed out by a transaction that then rolls back is never used — which is
// what keeps the stream gapless. A database sequence would be faster and
// would not have that property.
func (s *Rooms) nextSequence(ctx context.Context, tx *sql.Tx, name string) (int64, error) {
if _, err := tx.ExecContext(ctx, s.q(bumpSequenceSQL), name); err != nil {
return 0, fmt.Errorf("advance the %s counter: %w", name, err)
}
var value int64
if err := tx.QueryRowContext(ctx, s.q(selectSequenceSQL), name).Scan(&value); err != nil {
return 0, fmt.Errorf("read the %s counter: %w", name, err)
}
return value, nil
}
const (
selectSnapshotSQL = `
SELECT snapshot_id FROM roomserver_state_snapshots
WHERE room_id = ? AND state_hash = ?`
insertSnapshotSQL = `
INSERT INTO roomserver_state_snapshots (snapshot_id, room_id, state_hash)
VALUES (?, ?, ?)`
insertStateEntrySQL = `
INSERT INTO roomserver_state_entries (snapshot_id, event_type, state_key, event_id)
VALUES (?, ?, ?, ?)`
)
// stateSnapshot stores a state set, re-using the row of an identical one.
//
// Deduplication is what makes a per-event state snapshot affordable: every
// message in a room whose state is not changing resolves to the snapshot
// the previous one already stored, so the table grows with state changes
// rather than with traffic. An empty state set is snapshot 0 and stores
// nothing, which is the create event's case.
func (s *Rooms) stateSnapshot(ctx context.Context, tx *sql.Tx, roomID string, entries []storage.StateEntry) (int64, error) {
if len(entries) == 0 {
return 0, nil
}
hash := stateHash(entries)
var id int64
err := tx.QueryRowContext(ctx, s.q(selectSnapshotSQL), roomID, hash).Scan(&id)
switch {
case err == nil:
return id, nil
case !errors.Is(err, sql.ErrNoRows):
return 0, fmt.Errorf("look up state snapshot: %w", err)
}
if id, err = s.nextSequence(ctx, tx, seqStateSnapshot); err != nil {
return 0, err
}
if _, err := tx.ExecContext(ctx, s.q(insertSnapshotSQL), id, roomID, hash); err != nil {
return 0, fmt.Errorf("insert state snapshot: %w", err)
}
for _, e := range entries {
if _, err := tx.ExecContext(ctx, s.q(insertStateEntrySQL),
id, e.Type, e.StateKey, e.EventID); err != nil {
return 0, fmt.Errorf("insert state entry %s/%s: %w", e.Type, e.StateKey, err)
}
}
return id, nil
}
// stateHash identifies a state set by its contents.
//
// The entries are sorted first so that the same state always hashes the
// same whatever order it was assembled in, and each field is written with
// its length so that no two different sets can be run together into the
// same bytes.
func stateHash(entries []storage.StateEntry) string {
sorted := slices.Clone(entries)
slices.SortFunc(sorted, func(a, b storage.StateEntry) int {
if c := strings.Compare(a.Type, b.Type); c != 0 {
return c
}
return strings.Compare(a.StateKey, b.StateKey)
})
h := sha256.New()
for _, e := range sorted {
for _, field := range [...]string{e.Type, e.StateKey, e.EventID} {
fmt.Fprintf(h, "%d:%s", len(field), field)
}
}
return base64.RawURLEncoding.EncodeToString(h.Sum(nil))
}
const insertEventSQL = `
INSERT INTO roomserver_events (
event_id, room_id, stream_ordering, event_type, state_key,
sender, depth, origin_server_ts, event_json, state_before_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
func (s *Rooms) insertEvent(ctx context.Context, tx *sql.Tx, e storage.StoredEvent, ordering, snapshotID int64) error {
_, err := tx.ExecContext(ctx, s.q(insertEventSQL),
e.EventID, e.RoomID, ordering, e.Type, e.StateKey,
e.Sender, e.Depth, e.OriginServerTS, string(e.JSON), snapshotID)
if err != nil {
if s.isUnique(err) {
// The same event twice is not a failure worth propagating: an
// event ID is a hash of its contents, so the second copy says
// exactly what the first one did.
return nil
}
return fmt.Errorf("insert event: %w", err)
}
return nil
}
const applyRedactionSQL = `
UPDATE roomserver_events SET event_json = ?, redacted_by = ?
WHERE room_id = ? AND event_id = ? AND redacted_by = ''`
// applyRedaction replaces a stored event with its redacted form.
//
// The condition on redacted_by makes a second redaction of the same event a
// no-op rather than a rewrite: the first one already stripped the content,
// and the event may only be redacted once.
func (s *Rooms) applyRedaction(ctx context.Context, tx *sql.Tx, roomID, targetID, redactionID string, redacted []byte) error {
_, err := tx.ExecContext(ctx, s.q(applyRedactionSQL),
string(redacted), redactionID, roomID, targetID)
if err != nil {
return fmt.Errorf("apply redaction to %s: %w", targetID, err)
}
return nil
}
const (
upsertCurrentStateSQL = `
INSERT INTO roomserver_current_state (room_id, event_type, state_key, event_id, membership)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT (room_id, event_type, state_key) DO UPDATE
SET event_id = excluded.event_id, membership = excluded.membership`
deleteCurrentStateSQL = `DELETE FROM roomserver_current_state WHERE room_id = ?`
)
// applyStateDelta moves the room's current state.
//
// The usual case is a single tuple to upsert. A whole-state write — which
// only state resolution produces — clears the table for the room first,
// because tuples the resolution dropped have to go.
func (s *Rooms) applyStateDelta(ctx context.Context, tx *sql.Tx, w storage.RoomWrite) error {
if len(w.StateDelta) == 0 && !w.StateDeltaIsWholeState {
return nil
}
if w.StateDeltaIsWholeState {
if _, err := tx.ExecContext(ctx, s.q(deleteCurrentStateSQL), w.Event.RoomID); err != nil {
return fmt.Errorf("clear current state: %w", err)
}
}
for _, e := range w.StateDelta {
if _, err := tx.ExecContext(ctx, s.q(upsertCurrentStateSQL),
w.Event.RoomID, e.Type, e.StateKey, e.EventID, e.Membership); err != nil {
return fmt.Errorf("set current state %s/%s: %w", e.Type, e.StateKey, err)
}
}
return nil
}
const (
deleteExtremitiesSQL = `DELETE FROM roomserver_extremities WHERE room_id = ?`
insertExtremitySQL = `
INSERT INTO roomserver_extremities (room_id, event_id) VALUES (?, ?)
ON CONFLICT (room_id, event_id) DO NOTHING`
)
func (s *Rooms) replaceExtremities(ctx context.Context, tx *sql.Tx, roomID string, extremities []string) error {
if len(extremities) == 0 {
return nil
}
if _, err := tx.ExecContext(ctx, s.q(deleteExtremitiesSQL), roomID); err != nil {
return fmt.Errorf("clear forward extremities: %w", err)
}
for _, id := range extremities {
if _, err := tx.ExecContext(ctx, s.q(insertExtremitySQL), roomID, id); err != nil {
return fmt.Errorf("insert forward extremity %s: %w", id, err)
}
}
return nil
}
const updateProfileSQL = `
UPDATE roomserver_rooms SET
name = ?, topic = ?, avatar_url = ?, canonical_alias = ?,
join_rule = ?, history_visibility = ?, guest_access = ?, room_type = ?
WHERE room_id = ?`
func (s *Rooms) updateProfile(ctx context.Context, tx *sql.Tx, roomID string, p storage.RoomProfile) error {
_, err := tx.ExecContext(ctx, s.q(updateProfileSQL),
p.Name, p.Topic, p.AvatarURL, p.CanonicalAlias,
p.JoinRule, p.HistoryVisibility, p.GuestAccess, p.RoomType, roomID)
if err != nil {
return fmt.Errorf("update room profile: %w", err)
}
return nil
}
const recountMembersSQL = `
UPDATE roomserver_rooms SET joined_members = (
SELECT COUNT(*) FROM roomserver_current_state
WHERE room_id = ? AND membership = 'join'
) WHERE room_id = ?`
func (s *Rooms) recountMembers(ctx context.Context, tx *sql.Tx, roomID string) error {
if _, err := tx.ExecContext(ctx, s.q(recountMembersSQL), roomID, roomID); err != nil {
return fmt.Errorf("recount joined members: %w", err)
}
return nil
}
const insertTransactionSQL = `
INSERT INTO roomserver_transactions (localpart, device_id, txn_id, event_id, room_id, sent_at_ms)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT (localpart, device_id, txn_id) DO NOTHING`
func (s *Rooms) recordTransaction(ctx context.Context, tx *sql.Tx, ref storage.TransactionRef, eventID, roomID string, atMS int64) error {
_, err := tx.ExecContext(ctx, s.q(insertTransactionSQL),
ref.Localpart, ref.DeviceID, ref.TxnID, eventID, roomID, atMS)
if err != nil {
return fmt.Errorf("record transaction %q: %w", ref.TxnID, err)
}
return nil
}
// eventColumns is the select list every event read shares, in the order
// scanEvent expects.
const eventColumns = `
e.room_id, e.event_id, e.event_type, e.state_key, e.sender,
e.depth, e.origin_server_ts, e.event_json, e.stream_ordering, e.redacted_by`
// scanEvent reads one row of eventColumns.
func scanEvent(scan func(...any) error) (storage.StoredEvent, error) {
var (
e storage.StoredEvent
stateKey sql.NullString
body string
)
err := scan(&e.RoomID, &e.EventID, &e.Type, &stateKey, &e.Sender,
&e.Depth, &e.OriginServerTS, &body, &e.StreamOrdering, &e.RedactedBy)
if err != nil {
return storage.StoredEvent{}, err
}
if stateKey.Valid {
e.StateKey = &stateKey.String
}
e.JSON = []byte(body)
return e, nil
}
// scanEvents drains a result set of eventColumns rows.
func scanEvents(rows *sql.Rows) ([]storage.StoredEvent, error) {
defer func() { _ = rows.Close() }()
var out []storage.StoredEvent
for rows.Next() {
e, err := scanEvent(rows.Scan)
if err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
var selectEventSQL = `
SELECT ` + eventColumns + `
FROM roomserver_events e WHERE e.room_id = ? AND e.event_id = ?`
// Event implements storage.RoomStore.
func (s *Rooms) Event(ctx context.Context, roomID, eventID string) (storage.StoredEvent, error) {
e, err := scanEvent(s.db.QueryRowContext(ctx, s.q(selectEventSQL), roomID, eventID).Scan)
switch {
case errors.Is(err, sql.ErrNoRows):
return storage.StoredEvent{}, fmt.Errorf("event %q: %w", eventID, storage.ErrNoSuchEvent)
case err != nil:
return storage.StoredEvent{}, fmt.Errorf("read event %q: %w", eventID, err)
}
return e, nil
}
// EventsByID implements storage.RoomStore.
func (s *Rooms) EventsByID(ctx context.Context, roomID string, eventIDs []string) ([]storage.StoredEvent, error) {
if len(eventIDs) == 0 {
return nil, nil
}
query := `SELECT ` + eventColumns + `
FROM roomserver_events e
WHERE e.room_id = ? AND e.event_id IN (` + placeholders(len(eventIDs)) + `)`
args := make([]any, 0, len(eventIDs)+1)
args = append(args, roomID)
for _, id := range eventIDs {
args = append(args, id)
}
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("read events by ID: %w", err)
}
events, err := scanEvents(rows)
if err != nil {
return nil, fmt.Errorf("read events by ID: %w", err)
}
return events, nil
}
const selectCurrentStateSQL = `
SELECT event_type, state_key, event_id, membership
FROM roomserver_current_state WHERE room_id = ?
ORDER BY event_type, state_key`
// CurrentState implements storage.RoomStore.
func (s *Rooms) CurrentState(ctx context.Context, roomID string) ([]storage.StateEntry, error) {
rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateSQL), roomID)
if err != nil {
return nil, fmt.Errorf("read current state of %q: %w", roomID, err)
}
entries, err := scanStateEntries(rows)
if err != nil {
return nil, fmt.Errorf("read current state of %q: %w", roomID, err)
}
return entries, nil
}
func scanStateEntries(rows *sql.Rows) ([]storage.StateEntry, error) {
defer func() { _ = rows.Close() }()
var out []storage.StateEntry
for rows.Next() {
var e storage.StateEntry
if err := rows.Scan(&e.Type, &e.StateKey, &e.EventID, &e.Membership); err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
var selectCurrentStateEventsSQL = `
SELECT ` + eventColumns + `
FROM roomserver_current_state cs
JOIN roomserver_events e ON e.event_id = cs.event_id
WHERE cs.room_id = ?
ORDER BY cs.event_type, cs.state_key`
// CurrentStateEvents implements storage.RoomStore.
func (s *Rooms) CurrentStateEvents(ctx context.Context, roomID string) ([]storage.StoredEvent, error) {
rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateEventsSQL), roomID)
if err != nil {
return nil, fmt.Errorf("read current state events of %q: %w", roomID, err)
}
events, err := scanEvents(rows)
if err != nil {
return nil, fmt.Errorf("read current state events of %q: %w", roomID, err)
}
return events, nil
}
var selectCurrentStateEventSQL = `
SELECT ` + eventColumns + `
FROM roomserver_current_state cs
JOIN roomserver_events e ON e.event_id = cs.event_id
WHERE cs.room_id = ? AND cs.event_type = ? AND cs.state_key = ?`
// CurrentStateEvent implements storage.RoomStore.
func (s *Rooms) CurrentStateEvent(ctx context.Context, roomID, eventType, stateKey string) (storage.StoredEvent, error) {
e, err := scanEvent(s.db.QueryRowContext(ctx,
s.q(selectCurrentStateEventSQL), roomID, eventType, stateKey).Scan)
switch {
case errors.Is(err, sql.ErrNoRows):
return storage.StoredEvent{}, fmt.Errorf("state %s/%s of %q: %w",
eventType, stateKey, roomID, storage.ErrNoSuchEvent)
case err != nil:
return storage.StoredEvent{}, fmt.Errorf("read state %s/%s of %q: %w",
eventType, stateKey, roomID, err)
}
return e, nil
}
const selectStateBeforeSQL = `
SELECT se.event_type, se.state_key, se.event_id, ''
FROM roomserver_events e
JOIN roomserver_state_entries se ON se.snapshot_id = e.state_before_id
WHERE e.room_id = ? AND e.event_id = ?
ORDER BY se.event_type, se.state_key`
// StateBefore implements storage.RoomStore.
//
// The membership column is not stored on a snapshot entry: it exists on
// current state to make a membership lookup an index hit, and a historical
// snapshot is only ever read as a whole and then parsed anyway.
func (s *Rooms) StateBefore(ctx context.Context, roomID, eventID string) ([]storage.StateEntry, error) {
rows, err := s.db.QueryContext(ctx, s.q(selectStateBeforeSQL), roomID, eventID)
if err != nil {
return nil, fmt.Errorf("read the state before %q: %w", eventID, err)
}
entries, err := scanStateEntries(rows)
if err != nil {
return nil, fmt.Errorf("read the state before %q: %w", eventID, err)
}
return entries, nil
}
var selectExtremitiesSQL = `
SELECT ` + eventColumns + `
FROM roomserver_extremities x
JOIN roomserver_events e ON e.event_id = x.event_id
WHERE x.room_id = ?
ORDER BY e.stream_ordering`
// Extremities implements storage.RoomStore.
func (s *Rooms) Extremities(ctx context.Context, roomID string) ([]storage.StoredEvent, error) {
rows, err := s.db.QueryContext(ctx, s.q(selectExtremitiesSQL), roomID)
if err != nil {
return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err)
}
events, err := scanEvents(rows)
if err != nil {
return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err)
}
return events, nil
}
// Members implements storage.RoomStore.
func (s *Rooms) Members(ctx context.Context, roomID string, memberships []string) ([]storage.StoredEvent, error) {
query := `SELECT ` + eventColumns + `
FROM roomserver_current_state cs
JOIN roomserver_events e ON e.event_id = cs.event_id
WHERE cs.room_id = ? AND cs.event_type = 'm.room.member'`
args := []any{roomID}
if len(memberships) > 0 {
query += ` AND cs.membership IN (` + placeholders(len(memberships)) + `)`
for _, m := range memberships {
args = append(args, m)
}
}
query += ` ORDER BY cs.state_key`
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("read the members of %q: %w", roomID, err)
}
events, err := scanEvents(rows)
if err != nil {
return nil, fmt.Errorf("read the members of %q: %w", roomID, err)
}
return events, nil
}
// MembershipEntries implements storage.RoomStore.
func (s *Rooms) MembershipEntries(ctx context.Context, roomID string, memberships []string) ([]storage.MembershipEntry, error) {
query := `
SELECT state_key, membership, event_id FROM roomserver_current_state
WHERE room_id = ? AND event_type = 'm.room.member'`
args := []any{roomID}
if len(memberships) > 0 {
query += ` AND membership IN (` + placeholders(len(memberships)) + `)`
for _, m := range memberships {
args = append(args, m)
}
}
query += ` ORDER BY state_key`
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err)
}
defer func() { _ = rows.Close() }()
var out []storage.MembershipEntry
for rows.Next() {
var m storage.MembershipEntry
if err := rows.Scan(&m.UserID, &m.Membership, &m.EventID); err != nil {
return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err)
}
out = append(out, m)
}
return out, rows.Err()
}
const selectMembershipSQL = `
SELECT state_key, membership, event_id FROM roomserver_current_state
WHERE room_id = ? AND event_type = 'm.room.member' AND state_key = ?`
// Membership implements storage.RoomStore.
func (s *Rooms) Membership(ctx context.Context, roomID, userID string) (storage.MembershipEntry, error) {
var m storage.MembershipEntry
err := s.db.QueryRowContext(ctx, s.q(selectMembershipSQL), roomID, userID).
Scan(&m.UserID, &m.Membership, &m.EventID)
switch {
case errors.Is(err, sql.ErrNoRows):
// Never a member is not an error: it is the same answer as having
// left, which is what the rules above this ask for.
return storage.MembershipEntry{UserID: userID}, nil
case err != nil:
return storage.MembershipEntry{}, fmt.Errorf("read the membership of %q in %q: %w", userID, roomID, err)
}
return m, nil
}
// RoomsForUser implements storage.RoomStore.
func (s *Rooms) RoomsForUser(ctx context.Context, userID string, memberships []string) ([]string, error) {
query := `
SELECT room_id FROM roomserver_current_state
WHERE event_type = 'm.room.member' AND state_key = ?`
args := []any{userID}
if len(memberships) > 0 {
query += ` AND membership IN (` + placeholders(len(memberships)) + `)`
for _, m := range memberships {
args = append(args, m)
}
}
query += ` ORDER BY room_id`
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
if err != nil {
return nil, fmt.Errorf("read the rooms of %q: %w", userID, err)
}
defer func() { _ = rows.Close() }()
var out []string
for rows.Next() {
var roomID string
if err := rows.Scan(&roomID); err != nil {
return nil, fmt.Errorf("read the rooms of %q: %w", userID, err)
}
out = append(out, roomID)
}
return out, rows.Err()
}
const selectTransactionSQL = `
SELECT event_id FROM roomserver_transactions
WHERE localpart = ? AND device_id = ? AND txn_id = ?`
// TransactionEvent implements storage.RoomStore.
func (s *Rooms) TransactionEvent(ctx context.Context, ref storage.TransactionRef) (string, error) {
var eventID string
err := s.db.QueryRowContext(ctx, s.q(selectTransactionSQL),
ref.Localpart, ref.DeviceID, ref.TxnID).Scan(&eventID)
switch {
case errors.Is(err, sql.ErrNoRows):
return "", fmt.Errorf("transaction %q: %w", ref.TxnID, storage.ErrNoSuchEvent)
case err != nil:
return "", fmt.Errorf("read transaction %q: %w", ref.TxnID, err)
}
return eventID, nil
}
const insertAliasSQL = `
INSERT INTO roomserver_aliases (alias, room_id, creator, created_at_ms)
VALUES (?, ?, ?, ?)`
// CreateAlias implements storage.RoomStore.
func (s *Rooms) CreateAlias(ctx context.Context, alias storage.RoomAlias) error {
_, err := s.db.ExecContext(ctx, s.q(insertAliasSQL),
alias.Alias, alias.RoomID, alias.Creator, alias.CreatedAtMS)
if err != nil {
if s.isUnique(err) {
return fmt.Errorf("create alias %q: %w", alias.Alias, storage.ErrAliasExists)
}
return fmt.Errorf("create alias %q: %w", alias.Alias, err)
}
return nil
}
const deleteAliasSQL = `DELETE FROM roomserver_aliases WHERE alias = ?`
// DeleteAlias implements storage.RoomStore.
func (s *Rooms) DeleteAlias(ctx context.Context, alias string) error {
result, err := s.db.ExecContext(ctx, s.q(deleteAliasSQL), alias)
if err != nil {
return fmt.Errorf("delete alias %q: %w", alias, err)
}
return expectOneRow(result, alias, storage.ErrNoSuchAlias)
}
const selectAliasSQL = `
SELECT alias, room_id, creator, created_at_ms
FROM roomserver_aliases WHERE alias = ?`
// Alias implements storage.RoomStore.
func (s *Rooms) Alias(ctx context.Context, alias string) (storage.RoomAlias, error) {
var out storage.RoomAlias
err := s.db.QueryRowContext(ctx, s.q(selectAliasSQL), alias).
Scan(&out.Alias, &out.RoomID, &out.Creator, &out.CreatedAtMS)
switch {
case errors.Is(err, sql.ErrNoRows):
return storage.RoomAlias{}, fmt.Errorf("alias %q: %w", alias, storage.ErrNoSuchAlias)
case err != nil:
return storage.RoomAlias{}, fmt.Errorf("resolve alias %q: %w", alias, err)
}
return out, nil
}
const selectRoomAliasesSQL = `
SELECT alias FROM roomserver_aliases WHERE room_id = ? ORDER BY alias`
// AliasesForRoom implements storage.RoomStore.
func (s *Rooms) AliasesForRoom(ctx context.Context, roomID string) ([]string, error) {
rows, err := s.db.QueryContext(ctx, s.q(selectRoomAliasesSQL), roomID)
if err != nil {
return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err)
}
defer func() { _ = rows.Close() }()
var out []string
for rows.Next() {
var alias string
if err := rows.Scan(&alias); err != nil {
return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err)
}
out = append(out, alias)
}
return out, rows.Err()
}
const setPublishedSQL = `UPDATE roomserver_rooms SET is_published = ? WHERE room_id = ?`
// SetPublished implements storage.RoomStore.
func (s *Rooms) SetPublished(ctx context.Context, roomID string, published bool) error {
result, err := s.db.ExecContext(ctx, s.q(setPublishedSQL), published, roomID)
if err != nil {
return fmt.Errorf("set the published flag of %q: %w", roomID, err)
}
return expectOneRow(result, roomID, storage.ErrNoSuchRoom)
}
// roomSummaryColumns is the select list a room listing shares, in the order
// scanRoomSummary expects.
const roomSummaryColumns = `
room_id, room_version, creator, is_published, created_at_ms,
name, topic, avatar_url, canonical_alias,
join_rule, history_visibility, guest_access, room_type, joined_members`
func scanRoomSummary(scan func(...any) error) (storage.RoomSummary, error) {
var r storage.RoomSummary
err := scan(&r.ID, &r.Version, &r.Creator, &r.Published, &r.CreatedAtMS,
&r.Name, &r.Topic, &r.AvatarURL, &r.CanonicalAlias,
&r.JoinRule, &r.HistoryVisibility, &r.GuestAccess, &r.RoomType, &r.JoinedMembers)
return r, err
}
// PublishedRooms implements storage.RoomStore.
func (s *Rooms) PublishedRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) {
return s.listRooms(ctx, filter, true)
}
// ListRooms implements storage.RoomStore.
func (s *Rooms) ListRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) {
return s.listRooms(ctx, filter, false)
}
// listRooms is the body of both listings: the same query with and without
// the published condition.
//
// Rooms are ordered by descending member count and then by room ID. The
// second key is not decoration: paging a listing whose order is not total
// can show the same room on two pages and skip another, and ties in member
// count are the common case in a directory of small rooms.
func (s *Rooms) listRooms(ctx context.Context, filter storage.RoomFilter, publishedOnly bool) ([]storage.RoomSummary, int, error) {
where := ` WHERE 1 = 1`
var args []any
if publishedOnly {
where += ` AND is_published = ?`
args = append(args, true)
}
if search := strings.TrimSpace(filter.Search); search != "" {
pattern := "%" + strings.ToLower(search) + "%"
where += ` AND (LOWER(room_id) LIKE ? OR LOWER(name) LIKE ?
OR LOWER(topic) LIKE ? OR LOWER(canonical_alias) LIKE ?)`
args = append(args, pattern, pattern, pattern, pattern)
}
var total int
if err := s.db.QueryRowContext(ctx,
s.q(`SELECT COUNT(*) FROM roomserver_rooms`+where), args...).Scan(&total); err != nil {
return nil, 0, fmt.Errorf("count rooms: %w", err)
}
limit, offset := pageBounds(filter.Limit, filter.Offset, defaultRoomPageSize, maxRoomPageSize)
query := `SELECT ` + roomSummaryColumns + ` FROM roomserver_rooms` + where +
` ORDER BY joined_members DESC, room_id LIMIT ? OFFSET ?`
rows, err := s.db.QueryContext(ctx, s.q(query), append(args, limit, offset)...)
if err != nil {
return nil, 0, fmt.Errorf("list rooms: %w", err)
}
defer func() { _ = rows.Close() }()
out := make([]storage.RoomSummary, 0, limit)
for rows.Next() {
r, err := scanRoomSummary(rows.Scan)
if err != nil {
return nil, 0, fmt.Errorf("list rooms: %w", err)
}
out = append(out, r)
}
if err := rows.Err(); err != nil {
return nil, 0, fmt.Errorf("list rooms: %w", err)
}
return out, total, nil
}
var selectRoomSummarySQL = `SELECT ` + roomSummaryColumns +
` FROM roomserver_rooms WHERE room_id = ?`
// RoomSummary implements storage.RoomStore.
func (s *Rooms) RoomSummary(ctx context.Context, roomID string) (storage.RoomSummary, error) {
r, err := scanRoomSummary(s.db.QueryRowContext(ctx, s.q(selectRoomSummarySQL), roomID).Scan)
switch {
case errors.Is(err, sql.ErrNoRows):
return storage.RoomSummary{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom)
case err != nil:
return storage.RoomSummary{}, fmt.Errorf("read room %q: %w", roomID, err)
}
return r, nil
}
// purgeStatements are run in order, children before parents.
//
// Both engines are given ON DELETE CASCADE on these tables, but SQLite
// enforces foreign keys only when the pragma is on, and the transactions
// table has no key at all because it outlives the room it refers to. The
// explicit order is what makes a purge mean the same thing on both.
var purgeStatements = []string{
`DELETE FROM roomserver_state_entries WHERE snapshot_id IN (
SELECT snapshot_id FROM roomserver_state_snapshots WHERE room_id = ?)`,
`DELETE FROM roomserver_state_snapshots WHERE room_id = ?`,
`DELETE FROM roomserver_current_state WHERE room_id = ?`,
`DELETE FROM roomserver_extremities WHERE room_id = ?`,
`DELETE FROM roomserver_transactions WHERE room_id = ?`,
`DELETE FROM roomserver_aliases WHERE room_id = ?`,
`DELETE FROM roomserver_events WHERE room_id = ?`,
`DELETE FROM roomserver_rooms WHERE room_id = ?`,
}
// PurgeRoom implements storage.RoomStore.
func (s *Rooms) PurgeRoom(ctx context.Context, roomID string) error {
err := s.tx(ctx, func(tx *sql.Tx) error {
for i, stmt := range purgeStatements {
result, err := tx.ExecContext(ctx, s.q(stmt), roomID)
if err != nil {
return err
}
// The last statement is the room row itself: no row there means
// there was no room, and the whole purge rolls back so that the
// caller's 404 is the truth.
if i == len(purgeStatements)-1 {
if err := expectOneRow(result, roomID, storage.ErrNoSuchRoom); err != nil {
return err
}
}
}
return nil
})
if err != nil {
return fmt.Errorf("purge room %q: %w", roomID, err)
}
return nil
}
const roomStatsSQL = `
SELECT
(SELECT COUNT(*) FROM roomserver_rooms),
(SELECT COUNT(*) FROM roomserver_rooms WHERE is_published = ?),
(SELECT COUNT(*) FROM roomserver_events)`
// RoomStats implements storage.RoomStore.
func (s *Rooms) RoomStats(ctx context.Context) (storage.RoomStats, error) {
var stats storage.RoomStats
err := s.db.QueryRowContext(ctx, s.q(roomStatsSQL), true).
Scan(&stats.Rooms, &stats.Published, &stats.Events)
if err != nil {
return storage.RoomStats{}, fmt.Errorf("read room statistics: %w", err)
}
return stats, nil
}
// placeholders renders n comma-separated `?` placeholders for an IN clause.
func placeholders(n int) string {
return strings.TrimSuffix(strings.Repeat("?, ", n), ", ")
}
// pageBounds clamps a caller's limit and offset.
func pageBounds(limit, offset, defaultLimit, maxLimit int) (int, int) {
if limit <= 0 {
limit = defaultLimit
}
if limit > maxLimit {
limit = maxLimit
}
if offset < 0 {
offset = 0
}
return limit, offset
}
// expectOneRow turns "the UPDATE or DELETE matched nothing" into notFound.
func expectOneRow(result sql.Result, subject string, notFound error) error {
n, err := result.RowsAffected()
if err != nil {
return fmt.Errorf("count the affected rows: %w", err)
}
if n == 0 {
return fmt.Errorf("%q: %w", subject, notFound)
}
return nil
}
@@ -57,6 +57,7 @@ type Store struct {
log *slog.Logger
settings *Settings
users *Users
rooms *Rooms
}
// New builds a Store. It does not touch the database.
@@ -73,6 +74,7 @@ func New(opts Options) (*Store, error) {
log: opts.Logger,
settings: NewSettings(opts.DB, opts.Settings),
users: NewUsers(opts.DB, opts.Rebind, opts.IsUniqueViolation),
rooms: NewRooms(opts.DB, opts.Rebind, opts.IsUniqueViolation),
}, nil
}
@@ -94,6 +96,9 @@ func (s *Store) Settings() storage.SettingsStore { return s.settings }
// Users implements storage.Database.
func (s *Store) Users() storage.UserStore { return s.users }
// Rooms implements storage.Database.
func (s *Store) Rooms() storage.RoomStore { return s.rooms }
// Migrate implements storage.Database.
func (s *Store) Migrate(ctx context.Context) error {
ms, err := s.migrations()