Phase 3: rooms, local only
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Implements PLAN.md Phase 3: the roomserver, the room half of storage, the Client-Server room endpoints and the admin room endpoints. A client can now create a room, invite, join, leave, kick, ban, send messages and state, redact, name a room with an alias and find one in the public directory. Nothing talks to another server; every operation that would need to is refused with a message that says so rather than a bare M_FORBIDDEN. roomserver: one input pipeline every event goes through, whatever sent it — validate, authorise, resolve, persist, emit. The rules are not reimplemented here; internal/matrix is pure library code and this is what connects it to a database and to a room's history. The Client-Server handlers do no authorisation of their own, deliberately: whether Alice may kick Bob is a question about the room's power levels, and the one place that is answered correctly is the authorisation rules inside the pipeline. A mutex per room serialises writers. Two events racing into one room would each read the same forward extremities, both become one, and fork the room for no reason — which would then need state resolution to put back together. State resolution is wired in anyway for the multi-extremity case. It is unreachable from outside in this phase, and the alternative is a pipeline that authorises against one arbitrary branch of a fork without saying so. Room versions 3 to 12 all work and are each tested end to end. Version 12 earns that coverage: its room ID is the create event's own ID, its create event carries no room_id, and the create event is excluded from every other event's auth_events — so the state auth.Check is given is the event's auth_events plus the create event, which that version identifies by room ID rather than citing. createRoom sends the spec's events in the spec's order, each authorised against what the ones before it produced. A failure part-way leaves an incomplete room, which is the lesser evil: the alternative is holding every event of a new room in memory and writing them in one transaction, making the pipeline that authorises them a special case. The one failure a second client can cause at any time, an alias clash, takes the room row back out. Redactions stay two decisions, as the spec keeps them. The rules let almost any redaction into the room; whether it applies to its target is a separate test, run inside the same transaction as the redaction itself so that a client holding an event ID can always read the stripped event. storage: events keyed by event ID rather than by a surrogate, since an event ID appears in prev_events, auth_events, state snapshots and client requests alike. Stream ordering comes from a counter row bumped inside the write transaction, because a PostgreSQL sequence is explicitly not transactional and a rolled-back write would leave a hole in the stream that /sync would have to be taught to skip. State snapshots are stored per event and deduplicated by content hash, so the table grows with state changes rather than with traffic — and that is what makes "the state after a forward extremity" answerable at all. The room's listing columns are denormalised onto its row because the public directory orders every published room by member count and pages the result, which cannot be done by reading each room's state in turn. Federation-disabled behaviour from PLAN.md section 3 is enforced here. Restricted joins are the exception that works: the rule needs a signature from the authorising user's server verified, and while federation is off that server is always this one. adminapi: room list and detail, purge, force-leave, and room and event counters on the dashboard. clientapi gains its own rate-limit buckets for createRoom and for joining, both of which write events rather than read them. Tested against the real event core and a real database rather than fakes: what is under test is whether an event this server signs is one the authorisation rules accept, and a fake of either would only prove that the pipeline calls what the fake expects. The storage tests run against both SQL drivers. docs/rooms.md describes the pipeline, the storage decisions and the known gaps — chief among them that history visibility is room-level rather than per-event, and that /sync still reports no rooms, both of which belong with the event stream in Phase 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5 committed
1 parent
2fbd5f53ce
commit
c96f88e994
45 files changed
+9637
-42
No files matched your search
@@ -54,6 +54,9 @@ func (*Database) Settings() storage.SettingsStore { return notImplementedSetting
|
||||
// Users implements storage.Database.
|
||||
func (*Database) Users() storage.UserStore { return notImplementedUsers{} }
|
||||
|
||||
// Rooms implements storage.Database.
|
||||
func (*Database) Rooms() storage.RoomStore { return notImplementedRooms{} }
|
||||
|
||||
// Close implements storage.Database.
|
||||
func (*Database) Close() error { return nil }
|
||||
|
||||
@@ -163,8 +166,114 @@ func (notImplementedUsers) SetAccountData(context.Context, string, string, strin
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
// notImplementedRooms satisfies storage.RoomStore the same way. It matters
|
||||
// more here than anywhere else: a room whose create event was accepted and
|
||||
// not stored would hand a client a room ID that refers to nothing.
|
||||
type notImplementedRooms struct{}
|
||||
|
||||
func (notImplementedRooms) CreateRoom(context.Context, storage.NewRoom) error {
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Room(context.Context, string) (storage.Room, error) {
|
||||
return storage.Room{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Write(context.Context, storage.RoomWrite) (int64, error) {
|
||||
return 0, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Event(context.Context, string, string) (storage.StoredEvent, error) {
|
||||
return storage.StoredEvent{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) EventsByID(context.Context, string, []string) ([]storage.StoredEvent, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) CurrentState(context.Context, string) ([]storage.StateEntry, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) CurrentStateEvents(context.Context, string) ([]storage.StoredEvent, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) CurrentStateEvent(context.Context, string, string, string) (storage.StoredEvent, error) {
|
||||
return storage.StoredEvent{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) StateBefore(context.Context, string, string) ([]storage.StateEntry, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Extremities(context.Context, string) ([]storage.StoredEvent, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Members(context.Context, string, []string) ([]storage.StoredEvent, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) MembershipEntries(context.Context, string, []string) ([]storage.MembershipEntry, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Membership(context.Context, string, string) (storage.MembershipEntry, error) {
|
||||
return storage.MembershipEntry{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) RoomsForUser(context.Context, string, []string) ([]string, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) TransactionEvent(context.Context, storage.TransactionRef) (string, error) {
|
||||
return "", storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) CreateAlias(context.Context, storage.RoomAlias) error {
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) DeleteAlias(context.Context, string) error {
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) Alias(context.Context, string) (storage.RoomAlias, error) {
|
||||
return storage.RoomAlias{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) AliasesForRoom(context.Context, string) ([]string, error) {
|
||||
return nil, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) SetPublished(context.Context, string, bool) error {
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) PublishedRooms(context.Context, storage.RoomFilter) ([]storage.RoomSummary, int, error) {
|
||||
return nil, 0, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) ListRooms(context.Context, storage.RoomFilter) ([]storage.RoomSummary, int, error) {
|
||||
return nil, 0, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) RoomSummary(context.Context, string) (storage.RoomSummary, error) {
|
||||
return storage.RoomSummary{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) PurgeRoom(context.Context, string) error {
|
||||
return storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
func (notImplementedRooms) RoomStats(context.Context) (storage.RoomStats, error) {
|
||||
return storage.RoomStats{}, storage.ErrNotImplemented
|
||||
}
|
||||
|
||||
// Compile-time proof that the stub keeps up with the interfaces.
|
||||
var (
|
||||
_ storage.Database = (*Database)(nil)
|
||||
_ storage.UserStore = notImplementedUsers{}
|
||||
_ storage.RoomStore = notImplementedRooms{}
|
||||
)
|
||||
@@ -0,0 +1,165 @@
|
||||
-- Rooms, events and room state: the roomserver tables.
|
||||
--
|
||||
-- Events are keyed by their event ID rather than by a numeric surrogate.
|
||||
-- An event ID is 44 bytes of base64 and appears in prev_events,
|
||||
-- auth_events, state snapshots and client requests alike, so a surrogate
|
||||
-- would have to be translated at every one of those boundaries; the
|
||||
-- indexes below are what make the text key cheap enough.
|
||||
--
|
||||
-- Timestamps are milliseconds since the Unix epoch, as everywhere else in
|
||||
-- Warp.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_rooms (
|
||||
room_id TEXT NOT NULL PRIMARY KEY,
|
||||
-- Fixed at creation. Nothing can read an event without it: the room
|
||||
-- version decides the event format, the redaction algorithm and the
|
||||
-- authorisation rules.
|
||||
room_version TEXT NOT NULL,
|
||||
creator TEXT NOT NULL,
|
||||
is_published BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at_ms BIGINT NOT NULL,
|
||||
|
||||
-- The denormalised room listing columns. Everything below is derivable
|
||||
-- from the room's current state and is kept here anyway, because the
|
||||
-- public room directory orders every published room by member count
|
||||
-- and pages the result: that cannot be done by reading each room's
|
||||
-- state in turn. The roomserver recomputes them when the state behind
|
||||
-- them changes; joined_members the store counts for itself.
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
topic TEXT NOT NULL DEFAULT '',
|
||||
avatar_url TEXT NOT NULL DEFAULT '',
|
||||
canonical_alias TEXT NOT NULL DEFAULT '',
|
||||
join_rule TEXT NOT NULL DEFAULT '',
|
||||
history_visibility TEXT NOT NULL DEFAULT '',
|
||||
guest_access TEXT NOT NULL DEFAULT '',
|
||||
room_type TEXT NOT NULL DEFAULT '',
|
||||
joined_members BIGINT NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
-- The public room directory's own ordering: published rooms, biggest first.
|
||||
CREATE INDEX IF NOT EXISTS roomserver_rooms_published_idx
|
||||
ON roomserver_rooms (is_published, joined_members);
|
||||
|
||||
-- One counter row per stream, incremented inside the write transaction.
|
||||
--
|
||||
-- Neither AUTOINCREMENT nor a sequence would do: SQLite and PostgreSQL
|
||||
-- spell those differently, and a PostgreSQL sequence is explicitly not
|
||||
-- transactional, so a rolled-back write would leave a hole in the stream
|
||||
-- that /sync would have to be taught to skip. An UPDATE on this row takes
|
||||
-- a row lock in both engines and gives out positions with no gaps.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_sequences (
|
||||
name TEXT NOT NULL PRIMARY KEY,
|
||||
value BIGINT NOT NULL
|
||||
);
|
||||
|
||||
INSERT INTO roomserver_sequences (name, value) VALUES ('stream_ordering', 0)
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
INSERT INTO roomserver_sequences (name, value) VALUES ('state_snapshot', 0)
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
|
||||
-- A state snapshot is the complete state of a room at one point, stored
|
||||
-- once and shared by every event it applies to. Consecutive messages in a
|
||||
-- room whose state is not changing all point at the same snapshot, so the
|
||||
-- cost is one snapshot per state change rather than one per event.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_state_snapshots (
|
||||
snapshot_id BIGINT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
-- SHA-256 over the sorted (type, state_key, event_id) triples: the
|
||||
-- identity of the snapshot's contents, which is what lets two
|
||||
-- identical states share a row.
|
||||
state_hash TEXT NOT NULL,
|
||||
UNIQUE (room_id, state_hash)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_state_entries (
|
||||
snapshot_id BIGINT NOT NULL REFERENCES roomserver_state_snapshots (snapshot_id) ON DELETE CASCADE,
|
||||
event_type TEXT NOT NULL,
|
||||
state_key TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
PRIMARY KEY (snapshot_id, event_type, state_key)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_events (
|
||||
event_id TEXT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
-- This server's own monotonic position for the event: the order it was
|
||||
-- persisted in, which is the order clients are shown it in.
|
||||
stream_ordering BIGINT NOT NULL,
|
||||
event_type TEXT NOT NULL,
|
||||
-- NULL for a message event. The empty string is a valid state key, so
|
||||
-- "no state key" cannot be spelled as ''.
|
||||
state_key TEXT,
|
||||
sender TEXT NOT NULL,
|
||||
depth BIGINT NOT NULL,
|
||||
origin_server_ts BIGINT NOT NULL,
|
||||
-- The event as canonical JSON, byte for byte as it was signed. Every
|
||||
-- other column here is a projection of it, written out to be indexed.
|
||||
event_json TEXT NOT NULL,
|
||||
-- The state of the room as of this event's prev_events, 0 for the
|
||||
-- create event. Keeping it per event is what lets the state after a
|
||||
-- forward extremity be worked out, which state resolution needs when a
|
||||
-- room has more than one.
|
||||
state_before_id BIGINT NOT NULL DEFAULT 0,
|
||||
-- The redaction that applies to this event, or ''. The JSON above is
|
||||
-- rewritten to its redacted form at the same time; this records which
|
||||
-- event did it so that unsigned.redacted_because can be filled in.
|
||||
redacted_by TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS roomserver_events_stream_idx
|
||||
ON roomserver_events (stream_ordering);
|
||||
|
||||
-- Paging a room's timeline, which is what /messages and /sync do.
|
||||
CREATE INDEX IF NOT EXISTS roomserver_events_room_stream_idx
|
||||
ON roomserver_events (room_id, stream_ordering);
|
||||
|
||||
-- The forward extremities: the events with no children, and so the
|
||||
-- prev_events of whatever is sent next.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_extremities (
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
event_id TEXT NOT NULL,
|
||||
PRIMARY KEY (room_id, event_id)
|
||||
);
|
||||
|
||||
-- The room's resolved current state: the state after its extremities.
|
||||
--
|
||||
-- It is derivable from the events, and is stored anyway because every
|
||||
-- authorisation decision reads it. The membership column is a denormalised
|
||||
-- copy of content.membership on m.room.member rows, so that "the rooms
|
||||
-- @alice is joined to" is an index lookup rather than a scan that parses
|
||||
-- JSON.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_current_state (
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
event_type TEXT NOT NULL,
|
||||
state_key TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
membership TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (room_id, event_type, state_key)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS roomserver_current_state_membership_idx
|
||||
ON roomserver_current_state (event_type, state_key, membership);
|
||||
|
||||
-- Client transaction IDs, so that a retried request returns the event the
|
||||
-- first attempt created instead of sending a second one.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_transactions (
|
||||
localpart TEXT NOT NULL,
|
||||
device_id TEXT NOT NULL,
|
||||
txn_id TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
room_id TEXT NOT NULL,
|
||||
sent_at_ms BIGINT NOT NULL,
|
||||
PRIMARY KEY (localpart, device_id, txn_id)
|
||||
);
|
||||
|
||||
-- Local room aliases. Remote aliases are never stored: they are resolved
|
||||
-- over federation every time, because the other server may have moved them.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_aliases (
|
||||
alias TEXT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
creator TEXT NOT NULL,
|
||||
created_at_ms BIGINT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS roomserver_aliases_room_idx
|
||||
ON roomserver_aliases (room_id);
|
||||
@@ -0,0 +1,357 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
)
|
||||
|
||||
// Errors the room store returns for cases a caller has to tell apart.
|
||||
var (
|
||||
// ErrNoSuchRoom is returned for a room this server does not know.
|
||||
ErrNoSuchRoom = errors.New("no such room")
|
||||
|
||||
// ErrRoomExists is returned by CreateRoom for a room ID already taken.
|
||||
// It can only happen through a room ID collision, which for a room
|
||||
// version 12 room would mean two identical create events.
|
||||
ErrRoomExists = errors.New("room already exists")
|
||||
|
||||
// ErrNoSuchEvent is returned for an event ID the room store does not
|
||||
// hold.
|
||||
ErrNoSuchEvent = errors.New("no such event")
|
||||
|
||||
// ErrNoSuchAlias is returned for an unknown room alias.
|
||||
ErrNoSuchAlias = errors.New("no such room alias")
|
||||
|
||||
// ErrAliasExists is returned by CreateAlias for an alias already
|
||||
// pointing somewhere.
|
||||
ErrAliasExists = errors.New("room alias already exists")
|
||||
)
|
||||
|
||||
// Room is one room this server takes part in.
|
||||
//
|
||||
// The version is fixed when the room is created and is what every later read
|
||||
// needs first: an event cannot even be parsed without knowing the rules it
|
||||
// was written under.
|
||||
type Room struct {
|
||||
ID string
|
||||
|
||||
// Version is the room version identifier, e.g. "11".
|
||||
Version string
|
||||
|
||||
// Creator is the full user ID that sent the m.room.create event.
|
||||
Creator string
|
||||
|
||||
// Published reports whether the room is in this server's public room
|
||||
// directory.
|
||||
Published bool
|
||||
|
||||
CreatedAtMS int64
|
||||
}
|
||||
|
||||
// NewRoom describes a room to create. The create event itself is written by
|
||||
// the same Write call that appends any other event.
|
||||
type NewRoom struct {
|
||||
ID string
|
||||
Version string
|
||||
Creator string
|
||||
Published bool
|
||||
CreatedAtMS int64
|
||||
}
|
||||
|
||||
// RoomProfile is the denormalised view of a room's state that a room
|
||||
// listing is searched and ordered by.
|
||||
//
|
||||
// It is all derivable from the room's current state, and is stored anyway:
|
||||
// the public room directory sorts every published room by member count and
|
||||
// pages the result, which cannot be done by reading each room's state in
|
||||
// turn. The roomserver recomputes it whenever one of the state events behind
|
||||
// it changes and hands it to Write; the store only keeps it, because
|
||||
// knowing that m.room.name means a name is the roomserver's business.
|
||||
//
|
||||
// JoinedMembers is the exception: the store maintains it itself, by counting
|
||||
// the current-state rows whose membership is "join".
|
||||
type RoomProfile struct {
|
||||
Name string
|
||||
Topic string
|
||||
AvatarURL string
|
||||
CanonicalAlias string
|
||||
|
||||
JoinRule string
|
||||
HistoryVisibility string
|
||||
GuestAccess string
|
||||
|
||||
// RoomType is m.room.create's content.type: empty for a normal room,
|
||||
// "m.space" for a space.
|
||||
RoomType string
|
||||
}
|
||||
|
||||
// StateEntry is one piece of room state and the event that holds it.
|
||||
type StateEntry struct {
|
||||
Type string
|
||||
StateKey string
|
||||
EventID string
|
||||
|
||||
// Membership is the membership value of an m.room.member entry, and
|
||||
// empty on every other kind of state. It travels with the entry so that
|
||||
// the store can keep "who is joined to this room" answerable with an
|
||||
// index instead of by parsing event JSON.
|
||||
Membership string
|
||||
}
|
||||
|
||||
// StoredEvent is an event as the room store holds it: the canonical JSON
|
||||
// plus the columns worth indexing separately.
|
||||
//
|
||||
// JSON is the authority. The other fields are projections of it, written out
|
||||
// so that the store can answer "the members of this room" or "events in
|
||||
// stream order" without parsing every row.
|
||||
type StoredEvent struct {
|
||||
RoomID string
|
||||
EventID string
|
||||
Type string
|
||||
|
||||
// StateKey is nil for a non-state event. The empty string is a
|
||||
// perfectly good state key, which is why this is a pointer.
|
||||
StateKey *string
|
||||
|
||||
Sender string
|
||||
Depth int64
|
||||
OriginServerTS int64
|
||||
|
||||
// JSON is the event as canonical JSON, exactly as it was signed.
|
||||
JSON []byte
|
||||
|
||||
// StreamOrdering is this server's own monotonic position for the event:
|
||||
// the order it was persisted in, which is the order /sync and /messages
|
||||
// hand events to clients.
|
||||
StreamOrdering int64
|
||||
|
||||
// RedactedBy is the event ID of the redaction that applies to this
|
||||
// event, or empty. The JSON is rewritten in place when a redaction is
|
||||
// applied, so a redacted event is already stripped on the way out; this
|
||||
// field is what lets unsigned.redacted_because be filled in.
|
||||
RedactedBy string
|
||||
}
|
||||
|
||||
// IsState reports whether the event is a state event.
|
||||
func (e StoredEvent) IsState() bool { return e.StateKey != nil }
|
||||
|
||||
// TransactionRef records a client transaction ID against the event it
|
||||
// produced, so that a retried PUT returns the first event rather than
|
||||
// sending a second one.
|
||||
type TransactionRef struct {
|
||||
Localpart string
|
||||
DeviceID string
|
||||
TxnID string
|
||||
}
|
||||
|
||||
// RoomWrite is one event and everything its arrival changes, written in a
|
||||
// single transaction.
|
||||
//
|
||||
// Persisting an event is never just an insert: it moves the room's forward
|
||||
// extremities, it may replace a piece of current state, and it may strip an
|
||||
// earlier event. Splitting that across calls would leave a window in which
|
||||
// the room's current state disagrees with its events, so the whole lot goes
|
||||
// down together.
|
||||
type RoomWrite struct {
|
||||
// Event is the event being appended. Its StreamOrdering is assigned by
|
||||
// the store and returned.
|
||||
Event StoredEvent
|
||||
|
||||
// StateBefore is the room state the event was authorised against: the
|
||||
// full state of the room as of its prev_events. It is stored as a
|
||||
// deduplicated snapshot, so appending a message to a quiet room adds no
|
||||
// state rows at all.
|
||||
StateBefore []StateEntry
|
||||
|
||||
// StateDelta are the pieces of current state this event changes — for
|
||||
// an ordinary state event, the single tuple it writes. It is empty for
|
||||
// an event that changes no state.
|
||||
StateDelta []StateEntry
|
||||
|
||||
// StateDeltaIsWholeState says that StateDelta is the room's entire new
|
||||
// current state, and that any tuple not in it must be removed. State
|
||||
// resolution across a forked room produces such a result; a single
|
||||
// state event does not, and rewriting the whole of a large room's state
|
||||
// on every join would make filling a room quadratic.
|
||||
StateDeltaIsWholeState bool
|
||||
|
||||
// Profile updates the denormalised room listing columns. It is nil when
|
||||
// this event changes none of them.
|
||||
Profile *RoomProfile
|
||||
|
||||
// Extremities replaces the room's forward extremities. For a locally
|
||||
// created event that is the event itself, since it descends from every
|
||||
// extremity there was.
|
||||
Extremities []string
|
||||
|
||||
// Transaction, when set, records the client transaction ID this event
|
||||
// came from.
|
||||
Transaction *TransactionRef
|
||||
|
||||
// Redacts is the event ID this event redacts, when the redaction is to
|
||||
// be applied. RedactedJSON is the target's redacted form, which
|
||||
// replaces the stored JSON.
|
||||
Redacts string
|
||||
RedactedJSON []byte
|
||||
}
|
||||
|
||||
// RoomAlias is a local alias and the room it points at.
|
||||
type RoomAlias struct {
|
||||
Alias string
|
||||
RoomID string
|
||||
|
||||
// Creator is the full user ID that created the alias.
|
||||
Creator string
|
||||
|
||||
CreatedAtMS int64
|
||||
}
|
||||
|
||||
// MembershipEntry is one member of a room.
|
||||
type MembershipEntry struct {
|
||||
UserID string
|
||||
Membership string
|
||||
EventID string
|
||||
}
|
||||
|
||||
// RoomFilter narrows and pages a room listing.
|
||||
type RoomFilter struct {
|
||||
// Search matches a substring of the room ID, name, topic or canonical
|
||||
// alias, case-insensitively. Empty matches everything.
|
||||
Search string
|
||||
|
||||
// Limit caps the returned rows; 0 means the store's own default.
|
||||
Limit int
|
||||
|
||||
// Offset skips rows, for paging.
|
||||
Offset int
|
||||
}
|
||||
|
||||
// RoomSummary is a room as a listing shows it: the room row, its
|
||||
// denormalised state and its member count.
|
||||
type RoomSummary struct {
|
||||
Room
|
||||
RoomProfile
|
||||
|
||||
// JoinedMembers is how many members of the room are joined.
|
||||
JoinedMembers int
|
||||
}
|
||||
|
||||
// RoomStats are the room counts the admin dashboard shows.
|
||||
type RoomStats struct {
|
||||
Rooms int
|
||||
Published int
|
||||
Events int
|
||||
}
|
||||
|
||||
// RoomStore persists rooms, their events and their state: the storage half
|
||||
// of the roomserver component.
|
||||
//
|
||||
// Every method takes identifiers the caller has already validated. The store
|
||||
// interprets nothing: it does not know the authorisation rules, it does not
|
||||
// know which room versions exist, and it never parses an event. What it
|
||||
// guarantees is that a room's events, its state and its extremities are only
|
||||
// ever seen in a consistent set.
|
||||
type RoomStore interface {
|
||||
// CreateRoom inserts the room row, returning ErrRoomExists when the ID
|
||||
// is taken.
|
||||
CreateRoom(ctx context.Context, room NewRoom) error
|
||||
|
||||
// Room reads one room, returning ErrNoSuchRoom if it is absent.
|
||||
Room(ctx context.Context, roomID string) (Room, error)
|
||||
|
||||
// Write appends one event and applies everything it changes. It returns
|
||||
// the stream ordering the event was given.
|
||||
Write(ctx context.Context, w RoomWrite) (int64, error)
|
||||
|
||||
// Event reads one event of a room, returning ErrNoSuchEvent if it is
|
||||
// absent.
|
||||
Event(ctx context.Context, roomID, eventID string) (StoredEvent, error)
|
||||
|
||||
// EventsByID reads events by ID within one room. Unknown IDs are left
|
||||
// out rather than reported: state resolution needs exactly that
|
||||
// behaviour for events this server does not have.
|
||||
EventsByID(ctx context.Context, roomID string, eventIDs []string) ([]StoredEvent, error)
|
||||
|
||||
// CurrentState returns the room's resolved current state.
|
||||
CurrentState(ctx context.Context, roomID string) ([]StateEntry, error)
|
||||
|
||||
// CurrentStateEvents returns the events behind the room's current
|
||||
// state, which is what the authorisation rules are run against.
|
||||
CurrentStateEvents(ctx context.Context, roomID string) ([]StoredEvent, error)
|
||||
|
||||
// CurrentStateEvent returns one piece of current state, or
|
||||
// ErrNoSuchEvent when the room has no such state.
|
||||
CurrentStateEvent(ctx context.Context, roomID, eventType, stateKey string) (StoredEvent, error)
|
||||
|
||||
// StateBefore returns the state snapshot recorded against an event: the
|
||||
// state of the room as of that event's prev_events.
|
||||
StateBefore(ctx context.Context, roomID, eventID string) ([]StateEntry, error)
|
||||
|
||||
// Extremities returns the room's forward extremities, which are the
|
||||
// prev_events of the next event to be sent.
|
||||
Extremities(ctx context.Context, roomID string) ([]StoredEvent, error)
|
||||
|
||||
// Members returns the m.room.member events of a room. memberships
|
||||
// filters by membership value; empty returns every member event,
|
||||
// including the users who have left.
|
||||
Members(ctx context.Context, roomID string, memberships []string) ([]StoredEvent, error)
|
||||
|
||||
// MembershipEntries returns the memberships of a room without the event
|
||||
// bodies, which is what /joined_members needs.
|
||||
MembershipEntries(ctx context.Context, roomID string, memberships []string) ([]MembershipEntry, error)
|
||||
|
||||
// Membership returns a user's membership in a room and the event that
|
||||
// set it. A user who was never in the room gets an empty membership and
|
||||
// no error: "never joined" and "left" are the same answer to every rule
|
||||
// that asks.
|
||||
Membership(ctx context.Context, roomID, userID string) (MembershipEntry, error)
|
||||
|
||||
// RoomsForUser lists the rooms a user has one of the given memberships
|
||||
// in, ordered by room ID.
|
||||
RoomsForUser(ctx context.Context, userID string, memberships []string) ([]string, error)
|
||||
|
||||
// TransactionEvent returns the event a client transaction ID already
|
||||
// produced, or ErrNoSuchEvent when this is the first time it is seen.
|
||||
TransactionEvent(ctx context.Context, ref TransactionRef) (string, error)
|
||||
|
||||
// CreateAlias points an alias at a room, returning ErrAliasExists when
|
||||
// the alias is taken.
|
||||
CreateAlias(ctx context.Context, alias RoomAlias) error
|
||||
|
||||
// DeleteAlias removes an alias, returning ErrNoSuchAlias if it is
|
||||
// absent.
|
||||
DeleteAlias(ctx context.Context, alias string) error
|
||||
|
||||
// Alias resolves an alias, returning ErrNoSuchAlias if it is absent.
|
||||
// The creator comes back with it because the permission check on
|
||||
// deleting an alias is partly about who made it.
|
||||
Alias(ctx context.Context, alias string) (RoomAlias, error)
|
||||
|
||||
// AliasesForRoom lists the local aliases pointing at a room, sorted.
|
||||
AliasesForRoom(ctx context.Context, roomID string) ([]string, error)
|
||||
|
||||
// SetPublished adds or removes a room from the public room directory,
|
||||
// returning ErrNoSuchRoom for a room that does not exist.
|
||||
SetPublished(ctx context.Context, roomID string, published bool) error
|
||||
|
||||
// PublishedRooms returns a page of the public room directory ordered by
|
||||
// descending joined-member count, plus the total number of published
|
||||
// rooms matching the filter.
|
||||
PublishedRooms(ctx context.Context, filter RoomFilter) ([]RoomSummary, int, error)
|
||||
|
||||
// ListRooms returns a page of every room for the admin API, plus the
|
||||
// total matching the filter.
|
||||
ListRooms(ctx context.Context, filter RoomFilter) ([]RoomSummary, int, error)
|
||||
|
||||
// RoomSummary returns one room with its denormalised state, returning
|
||||
// ErrNoSuchRoom if it is absent.
|
||||
RoomSummary(ctx context.Context, roomID string) (RoomSummary, error)
|
||||
|
||||
// PurgeRoom deletes a room and everything belonging to it, returning
|
||||
// ErrNoSuchRoom for a room that does not exist. It is the admin API's
|
||||
// last resort and has no undo.
|
||||
PurgeRoom(ctx context.Context, roomID string) error
|
||||
|
||||
// RoomStats returns the aggregate counts for the admin dashboard.
|
||||
RoomStats(ctx context.Context) (RoomStats, error)
|
||||
}
|
||||
@@ -0,0 +1,552 @@
|
||||
package storage_test
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/dariuszjaniak/warp/server/internal/storage"
|
||||
)
|
||||
|
||||
// migratedRoomDB opens a driver, migrates it and empties the roomserver
|
||||
// tables, which is what every room-store test needs first.
|
||||
func migratedRoomDB(t *testing.T, driver string) storage.Database {
|
||||
t.Helper()
|
||||
db := openTestDB(t, driver)
|
||||
if err := db.Migrate(t.Context()); err != nil {
|
||||
t.Fatalf("Migrate: %v", err)
|
||||
}
|
||||
resetRoomTables(t, driver)
|
||||
return db
|
||||
}
|
||||
|
||||
// resetRoomTables empties the roomserver tables and rewinds the counters.
|
||||
//
|
||||
// SQLite gets a fresh file per test. The PostgreSQL tests share one
|
||||
// database, so without this the second test to create "!room" would collide
|
||||
// with the first — and only in CI, where PostgreSQL is available.
|
||||
func resetRoomTables(t *testing.T, driver string) {
|
||||
t.Helper()
|
||||
if driver != storage.DriverPostgres {
|
||||
return
|
||||
}
|
||||
|
||||
db, err := sql.Open("pgx", os.Getenv("WARP_TEST_POSTGRES_DSN"))
|
||||
if err != nil {
|
||||
t.Fatalf("open postgres to reset the room tables: %v", err)
|
||||
}
|
||||
defer func() { _ = db.Close() }()
|
||||
|
||||
if _, err := db.ExecContext(t.Context(), `TRUNCATE
|
||||
roomserver_state_entries, roomserver_state_snapshots,
|
||||
roomserver_current_state, roomserver_extremities,
|
||||
roomserver_transactions, roomserver_aliases,
|
||||
roomserver_events, roomserver_rooms`); err != nil {
|
||||
t.Fatalf("reset the room tables: %v", err)
|
||||
}
|
||||
if _, err := db.ExecContext(t.Context(),
|
||||
`UPDATE roomserver_sequences SET value = 0`); err != nil {
|
||||
t.Fatalf("rewind the room counters: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// sqlDB reaches the underlying pool, for the one assertion that is about
|
||||
// the schema rather than the interface: that identical state is stored once.
|
||||
func sqlDB(t *testing.T, db storage.Database) *sql.DB {
|
||||
t.Helper()
|
||||
pool, ok := db.(interface{ DB() *sql.DB })
|
||||
if !ok {
|
||||
t.Skipf("driver %s does not expose a database/sql pool", db.Driver())
|
||||
}
|
||||
return pool.DB()
|
||||
}
|
||||
|
||||
// The room store never parses an event, so these tests feed it event JSON
|
||||
// that is only plausible. What is under test is the bookkeeping: stream
|
||||
// ordering, state snapshots, current state and the member counters.
|
||||
const testRoomID = "!room:warp.example"
|
||||
|
||||
func newTestRoom(id string) storage.NewRoom {
|
||||
return storage.NewRoom{
|
||||
ID: id,
|
||||
Version: "11",
|
||||
Creator: "@alice:warp.example",
|
||||
CreatedAtMS: 1_700_000_000_000,
|
||||
}
|
||||
}
|
||||
|
||||
// stateKey returns a pointer to s, for StoredEvent.StateKey.
|
||||
func stateKey(s string) *string { return &s }
|
||||
|
||||
func newTestEvent(roomID, eventID, eventType string, key *string) storage.StoredEvent {
|
||||
return storage.StoredEvent{
|
||||
RoomID: roomID,
|
||||
EventID: eventID,
|
||||
Type: eventType,
|
||||
StateKey: key,
|
||||
Sender: "@alice:warp.example",
|
||||
Depth: 1,
|
||||
OriginServerTS: 1_700_000_000_000,
|
||||
JSON: []byte(`{"type":"` + eventType + `"}`),
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomCreateAndRead(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
|
||||
if _, err := rooms.Room(ctx, testRoomID); !errors.Is(err, storage.ErrNoSuchRoom) {
|
||||
t.Errorf("Room of an unknown room = %v, want ErrNoSuchRoom", err)
|
||||
}
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); !errors.Is(err, storage.ErrRoomExists) {
|
||||
t.Errorf("CreateRoom twice = %v, want ErrRoomExists", err)
|
||||
}
|
||||
|
||||
room, err := rooms.Room(ctx, testRoomID)
|
||||
if err != nil {
|
||||
t.Fatalf("Room: %v", err)
|
||||
}
|
||||
if room.Version != "11" || room.Creator != "@alice:warp.example" || room.Published {
|
||||
t.Errorf("Room returned %+v", room)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomWriteStreamOrderingAndState(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
|
||||
// The create event: no state before it, and it is the room's
|
||||
// only state and only extremity.
|
||||
create := newTestEvent(testRoomID, "$create", "m.room.create", stateKey(""))
|
||||
createState := []storage.StateEntry{
|
||||
{Type: "m.room.create", StateKey: "", EventID: "$create"},
|
||||
}
|
||||
ordering, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: create,
|
||||
StateDelta: createState,
|
||||
Extremities: []string{"$create"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Write the create event: %v", err)
|
||||
}
|
||||
if ordering != 1 {
|
||||
t.Errorf("the first event got stream ordering %d, want 1", ordering)
|
||||
}
|
||||
|
||||
// A join, authorised against the state above.
|
||||
join := newTestEvent(testRoomID, "$join", "m.room.member", stateKey("@alice:warp.example"))
|
||||
join.Depth = 2
|
||||
ordering, err = rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: join,
|
||||
StateBefore: createState,
|
||||
StateDelta: []storage.StateEntry{{
|
||||
Type: "m.room.member", StateKey: "@alice:warp.example",
|
||||
EventID: "$join", Membership: "join",
|
||||
}},
|
||||
Extremities: []string{"$join"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Write the join: %v", err)
|
||||
}
|
||||
if ordering != 2 {
|
||||
t.Errorf("the second event got stream ordering %d, want 2", ordering)
|
||||
}
|
||||
|
||||
// A message, which changes no state at all.
|
||||
message := newTestEvent(testRoomID, "$message", "m.room.message", nil)
|
||||
message.Depth = 3
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: message,
|
||||
StateBefore: append(createState, storage.StateEntry{
|
||||
Type: "m.room.member", StateKey: "@alice:warp.example", EventID: "$join",
|
||||
}),
|
||||
Extremities: []string{"$message"},
|
||||
}); err != nil {
|
||||
t.Fatalf("Write the message: %v", err)
|
||||
}
|
||||
|
||||
state, err := rooms.CurrentState(ctx, testRoomID)
|
||||
if err != nil {
|
||||
t.Fatalf("CurrentState: %v", err)
|
||||
}
|
||||
if len(state) != 2 {
|
||||
t.Fatalf("current state has %d entries, want 2: %+v", len(state), state)
|
||||
}
|
||||
|
||||
stored, err := rooms.Event(ctx, testRoomID, "$message")
|
||||
if err != nil {
|
||||
t.Fatalf("Event: %v", err)
|
||||
}
|
||||
if stored.IsState() {
|
||||
t.Error("a message event came back as a state event")
|
||||
}
|
||||
if stored.StreamOrdering != 3 {
|
||||
t.Errorf("stream ordering = %d, want 3", stored.StreamOrdering)
|
||||
}
|
||||
if _, err := rooms.Event(ctx, testRoomID, "$nope"); !errors.Is(err, storage.ErrNoSuchEvent) {
|
||||
t.Errorf("Event of an unknown ID = %v, want ErrNoSuchEvent", err)
|
||||
}
|
||||
|
||||
// The state recorded against the message is the state it was
|
||||
// authorised against, not the state now.
|
||||
before, err := rooms.StateBefore(ctx, testRoomID, "$message")
|
||||
if err != nil {
|
||||
t.Fatalf("StateBefore: %v", err)
|
||||
}
|
||||
if len(before) != 2 {
|
||||
t.Errorf("the state before the message has %d entries, want 2", len(before))
|
||||
}
|
||||
|
||||
extremities, err := rooms.Extremities(ctx, testRoomID)
|
||||
if err != nil {
|
||||
t.Fatalf("Extremities: %v", err)
|
||||
}
|
||||
if len(extremities) != 1 || extremities[0].EventID != "$message" {
|
||||
t.Errorf("extremities = %+v, want just $message", extremities)
|
||||
}
|
||||
|
||||
// The create event is a state event with the empty state key,
|
||||
// which must not read back as "no state key".
|
||||
storedCreate, err := rooms.Event(ctx, testRoomID, "$create")
|
||||
if err != nil {
|
||||
t.Fatalf("Event of the create event: %v", err)
|
||||
}
|
||||
if !storedCreate.IsState() {
|
||||
t.Error("the create event came back without a state key")
|
||||
} else if *storedCreate.StateKey != "" {
|
||||
t.Errorf("the create event state key = %q, want empty", *storedCreate.StateKey)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestRoomStateSnapshotsAreShared is the reason a per-event state snapshot
|
||||
// is affordable: two events authorised against the same state must not
|
||||
// store it twice.
|
||||
func TestRoomStateSnapshotsAreShared(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
db := migratedRoomDB(t, driver)
|
||||
rooms := db.Rooms()
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
|
||||
state := []storage.StateEntry{
|
||||
{Type: "m.room.create", StateKey: "", EventID: "$create"},
|
||||
}
|
||||
for _, id := range []string{"$a", "$b", "$c"} {
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: newTestEvent(testRoomID, id, "m.room.message", nil),
|
||||
StateBefore: state,
|
||||
Extremities: []string{id},
|
||||
}); err != nil {
|
||||
t.Fatalf("Write %s: %v", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
var snapshots int
|
||||
if err := sqlDB(t, db).QueryRowContext(ctx,
|
||||
`SELECT COUNT(*) FROM roomserver_state_snapshots`).Scan(&snapshots); err != nil {
|
||||
t.Fatalf("count snapshots: %v", err)
|
||||
}
|
||||
if snapshots != 1 {
|
||||
t.Errorf("three events sharing one state stored %d snapshots, want 1", snapshots)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomMembershipAndCounts(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
|
||||
write := func(eventID, userID, membership string) {
|
||||
t.Helper()
|
||||
e := newTestEvent(testRoomID, eventID, "m.room.member", stateKey(userID))
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: e,
|
||||
StateDelta: []storage.StateEntry{{
|
||||
Type: "m.room.member", StateKey: userID,
|
||||
EventID: eventID, Membership: membership,
|
||||
}},
|
||||
Extremities: []string{eventID},
|
||||
}); err != nil {
|
||||
t.Fatalf("Write %s: %v", eventID, err)
|
||||
}
|
||||
}
|
||||
write("$alice", "@alice:warp.example", "join")
|
||||
write("$bob", "@bob:warp.example", "invite")
|
||||
write("$carol", "@carol:warp.example", "join")
|
||||
|
||||
// A user who was never in the room gets an empty membership
|
||||
// rather than an error.
|
||||
absent, err := rooms.Membership(ctx, testRoomID, "@dave:warp.example")
|
||||
if err != nil {
|
||||
t.Fatalf("Membership of a stranger: %v", err)
|
||||
}
|
||||
if absent.Membership != "" {
|
||||
t.Errorf("membership of a stranger = %q, want empty", absent.Membership)
|
||||
}
|
||||
|
||||
alice, err := rooms.Membership(ctx, testRoomID, "@alice:warp.example")
|
||||
if err != nil {
|
||||
t.Fatalf("Membership: %v", err)
|
||||
}
|
||||
if alice.Membership != "join" || alice.EventID != "$alice" {
|
||||
t.Errorf("Membership returned %+v", alice)
|
||||
}
|
||||
|
||||
joined, err := rooms.MembershipEntries(ctx, testRoomID, []string{"join"})
|
||||
if err != nil {
|
||||
t.Fatalf("MembershipEntries: %v", err)
|
||||
}
|
||||
if len(joined) != 2 {
|
||||
t.Errorf("%d joined members, want 2: %+v", len(joined), joined)
|
||||
}
|
||||
|
||||
all, err := rooms.Members(ctx, testRoomID, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("Members: %v", err)
|
||||
}
|
||||
if len(all) != 3 {
|
||||
t.Errorf("%d member events, want 3", len(all))
|
||||
}
|
||||
|
||||
// Leaving replaces the entry rather than adding one, and the
|
||||
// member count follows.
|
||||
write("$aliceleft", "@alice:warp.example", "leave")
|
||||
summary, err := rooms.RoomSummary(ctx, testRoomID)
|
||||
if err != nil {
|
||||
t.Fatalf("RoomSummary: %v", err)
|
||||
}
|
||||
if summary.JoinedMembers != 1 {
|
||||
t.Errorf("joined members = %d, want 1", summary.JoinedMembers)
|
||||
}
|
||||
|
||||
forAlice, err := rooms.RoomsForUser(ctx, "@alice:warp.example", []string{"join"})
|
||||
if err != nil {
|
||||
t.Fatalf("RoomsForUser: %v", err)
|
||||
}
|
||||
if len(forAlice) != 0 {
|
||||
t.Errorf("alice is still joined to %v after leaving", forAlice)
|
||||
}
|
||||
forCarol, err := rooms.RoomsForUser(ctx, "@carol:warp.example", []string{"join"})
|
||||
if err != nil {
|
||||
t.Fatalf("RoomsForUser: %v", err)
|
||||
}
|
||||
if len(forCarol) != 1 || forCarol[0] != testRoomID {
|
||||
t.Errorf("carol's joined rooms = %v, want [%s]", forCarol, testRoomID)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomRedactionAndTransactions(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
|
||||
ref := storage.TransactionRef{Localpart: "alice", DeviceID: "DEV", TxnID: "txn1"}
|
||||
if _, err := rooms.TransactionEvent(ctx, ref); !errors.Is(err, storage.ErrNoSuchEvent) {
|
||||
t.Errorf("TransactionEvent of an unseen ID = %v, want ErrNoSuchEvent", err)
|
||||
}
|
||||
|
||||
target := newTestEvent(testRoomID, "$target", "m.room.message", nil)
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: target,
|
||||
Extremities: []string{"$target"},
|
||||
Transaction: &ref,
|
||||
}); err != nil {
|
||||
t.Fatalf("Write the target: %v", err)
|
||||
}
|
||||
|
||||
got, err := rooms.TransactionEvent(ctx, ref)
|
||||
if err != nil {
|
||||
t.Fatalf("TransactionEvent: %v", err)
|
||||
}
|
||||
if got != "$target" {
|
||||
t.Errorf("TransactionEvent = %q, want $target", got)
|
||||
}
|
||||
|
||||
redaction := newTestEvent(testRoomID, "$redaction", "m.room.redaction", nil)
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: redaction,
|
||||
Extremities: []string{"$redaction"},
|
||||
Redacts: "$target",
|
||||
RedactedJSON: []byte(`{"type":"m.room.message","content":{}}`),
|
||||
}); err != nil {
|
||||
t.Fatalf("Write the redaction: %v", err)
|
||||
}
|
||||
|
||||
stored, err := rooms.Event(ctx, testRoomID, "$target")
|
||||
if err != nil {
|
||||
t.Fatalf("Event: %v", err)
|
||||
}
|
||||
if stored.RedactedBy != "$redaction" {
|
||||
t.Errorf("redacted_by = %q, want $redaction", stored.RedactedBy)
|
||||
}
|
||||
if string(stored.JSON) != `{"type":"m.room.message","content":{}}` {
|
||||
t.Errorf("the stored event was not replaced with its redacted form: %s", stored.JSON)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomAliasesAndDirectory(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(testRoomID)); err != nil {
|
||||
t.Fatalf("CreateRoom: %v", err)
|
||||
}
|
||||
|
||||
const aliasName = "#general:warp.example"
|
||||
newAlias := storage.RoomAlias{
|
||||
Alias: aliasName,
|
||||
RoomID: testRoomID,
|
||||
Creator: "@alice:warp.example",
|
||||
CreatedAtMS: 1,
|
||||
}
|
||||
if _, err := rooms.Alias(ctx, aliasName); !errors.Is(err, storage.ErrNoSuchAlias) {
|
||||
t.Errorf("Alias of an unknown alias = %v, want ErrNoSuchAlias", err)
|
||||
}
|
||||
if err := rooms.CreateAlias(ctx, newAlias); err != nil {
|
||||
t.Fatalf("CreateAlias: %v", err)
|
||||
}
|
||||
taken := newAlias
|
||||
taken.Creator = "@bob:warp.example"
|
||||
if err := rooms.CreateAlias(ctx, taken); !errors.Is(err, storage.ErrAliasExists) {
|
||||
t.Errorf("CreateAlias twice = %v, want ErrAliasExists", err)
|
||||
}
|
||||
|
||||
stored, err := rooms.Alias(ctx, aliasName)
|
||||
if err != nil {
|
||||
t.Fatalf("Alias: %v", err)
|
||||
}
|
||||
if stored.RoomID != testRoomID || stored.Creator != "@alice:warp.example" {
|
||||
t.Errorf("Alias returned %+v", stored)
|
||||
}
|
||||
aliases, err := rooms.AliasesForRoom(ctx, testRoomID)
|
||||
if err != nil || len(aliases) != 1 || aliases[0] != aliasName {
|
||||
t.Fatalf("AliasesForRoom = %v, %v", aliases, err)
|
||||
}
|
||||
|
||||
if err := rooms.DeleteAlias(ctx, aliasName); err != nil {
|
||||
t.Fatalf("DeleteAlias: %v", err)
|
||||
}
|
||||
if err := rooms.DeleteAlias(ctx, aliasName); !errors.Is(err, storage.ErrNoSuchAlias) {
|
||||
t.Errorf("DeleteAlias twice = %v, want ErrNoSuchAlias", err)
|
||||
}
|
||||
|
||||
// The directory is empty until the room is published.
|
||||
published, total, err := rooms.PublishedRooms(ctx, storage.RoomFilter{})
|
||||
if err != nil {
|
||||
t.Fatalf("PublishedRooms: %v", err)
|
||||
}
|
||||
if total != 0 || len(published) != 0 {
|
||||
t.Errorf("an unpublished room is in the directory: %+v", published)
|
||||
}
|
||||
if err := rooms.SetPublished(ctx, testRoomID, true); err != nil {
|
||||
t.Fatalf("SetPublished: %v", err)
|
||||
}
|
||||
if err := rooms.SetPublished(ctx, "!nope:warp.example", true); !errors.Is(err, storage.ErrNoSuchRoom) {
|
||||
t.Errorf("SetPublished on an unknown room = %v, want ErrNoSuchRoom", err)
|
||||
}
|
||||
published, total, err = rooms.PublishedRooms(ctx, storage.RoomFilter{})
|
||||
if err != nil {
|
||||
t.Fatalf("PublishedRooms: %v", err)
|
||||
}
|
||||
if total != 1 || len(published) != 1 || published[0].ID != testRoomID {
|
||||
t.Errorf("PublishedRooms = %+v (total %d)", published, total)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRoomListingAndPurge(t *testing.T) {
|
||||
for _, driver := range testDrivers {
|
||||
t.Run(driver, func(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
rooms := migratedRoomDB(t, driver).Rooms()
|
||||
|
||||
for _, id := range []string{"!a:warp.example", "!b:warp.example"} {
|
||||
if err := rooms.CreateRoom(ctx, newTestRoom(id)); err != nil {
|
||||
t.Fatalf("CreateRoom %s: %v", id, err)
|
||||
}
|
||||
if _, err := rooms.Write(ctx, storage.RoomWrite{
|
||||
Event: newTestEvent(id, "$create"+id, "m.room.create", stateKey("")),
|
||||
Extremities: []string{"$create" + id},
|
||||
Profile: &storage.RoomProfile{Name: "Room " + id, JoinRule: "invite"},
|
||||
}); err != nil {
|
||||
t.Fatalf("Write %s: %v", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
listed, total, err := rooms.ListRooms(ctx, storage.RoomFilter{})
|
||||
if err != nil {
|
||||
t.Fatalf("ListRooms: %v", err)
|
||||
}
|
||||
if total != 2 || len(listed) != 2 {
|
||||
t.Fatalf("ListRooms returned %d of %d, want 2 of 2", len(listed), total)
|
||||
}
|
||||
if listed[0].Name == "" || listed[0].JoinRule != "invite" {
|
||||
t.Errorf("the denormalised profile was not stored: %+v", listed[0])
|
||||
}
|
||||
|
||||
// The search covers the name as well as the ID.
|
||||
found, total, err := rooms.ListRooms(ctx, storage.RoomFilter{Search: "room !a"})
|
||||
if err != nil {
|
||||
t.Fatalf("ListRooms with a search: %v", err)
|
||||
}
|
||||
if total != 1 || len(found) != 1 || found[0].ID != "!a:warp.example" {
|
||||
t.Errorf("searching for a name returned %+v (total %d)", found, total)
|
||||
}
|
||||
|
||||
stats, err := rooms.RoomStats(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("RoomStats: %v", err)
|
||||
}
|
||||
if stats.Rooms != 2 || stats.Events != 2 {
|
||||
t.Errorf("RoomStats = %+v, want 2 rooms and 2 events", stats)
|
||||
}
|
||||
|
||||
if err := rooms.PurgeRoom(ctx, "!a:warp.example"); err != nil {
|
||||
t.Fatalf("PurgeRoom: %v", err)
|
||||
}
|
||||
if err := rooms.PurgeRoom(ctx, "!a:warp.example"); !errors.Is(err, storage.ErrNoSuchRoom) {
|
||||
t.Errorf("PurgeRoom twice = %v, want ErrNoSuchRoom", err)
|
||||
}
|
||||
if _, err := rooms.Room(ctx, "!a:warp.example"); !errors.Is(err, storage.ErrNoSuchRoom) {
|
||||
t.Errorf("the purged room is still there: %v", err)
|
||||
}
|
||||
if _, err := rooms.Event(ctx, "!b:warp.example", "$create!b:warp.example"); err != nil {
|
||||
t.Errorf("purging one room disturbed another: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,165 @@
|
||||
-- Rooms, events and room state: the roomserver tables.
|
||||
--
|
||||
-- Events are keyed by their event ID rather than by a numeric surrogate.
|
||||
-- An event ID is 44 bytes of base64 and appears in prev_events,
|
||||
-- auth_events, state snapshots and client requests alike, so a surrogate
|
||||
-- would have to be translated at every one of those boundaries; the
|
||||
-- indexes below are what make the text key cheap enough.
|
||||
--
|
||||
-- Timestamps are milliseconds since the Unix epoch, as everywhere else in
|
||||
-- Warp.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_rooms (
|
||||
room_id TEXT NOT NULL PRIMARY KEY,
|
||||
-- Fixed at creation. Nothing can read an event without it: the room
|
||||
-- version decides the event format, the redaction algorithm and the
|
||||
-- authorisation rules.
|
||||
room_version TEXT NOT NULL,
|
||||
creator TEXT NOT NULL,
|
||||
is_published BOOLEAN NOT NULL DEFAULT 0,
|
||||
created_at_ms BIGINT NOT NULL,
|
||||
|
||||
-- The denormalised room listing columns. Everything below is derivable
|
||||
-- from the room's current state and is kept here anyway, because the
|
||||
-- public room directory orders every published room by member count
|
||||
-- and pages the result: that cannot be done by reading each room's
|
||||
-- state in turn. The roomserver recomputes them when the state behind
|
||||
-- them changes; joined_members the store counts for itself.
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
topic TEXT NOT NULL DEFAULT '',
|
||||
avatar_url TEXT NOT NULL DEFAULT '',
|
||||
canonical_alias TEXT NOT NULL DEFAULT '',
|
||||
join_rule TEXT NOT NULL DEFAULT '',
|
||||
history_visibility TEXT NOT NULL DEFAULT '',
|
||||
guest_access TEXT NOT NULL DEFAULT '',
|
||||
room_type TEXT NOT NULL DEFAULT '',
|
||||
joined_members BIGINT NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
-- The public room directory's own ordering: published rooms, biggest first.
|
||||
CREATE INDEX IF NOT EXISTS roomserver_rooms_published_idx
|
||||
ON roomserver_rooms (is_published, joined_members);
|
||||
|
||||
-- One counter row per stream, incremented inside the write transaction.
|
||||
--
|
||||
-- Neither AUTOINCREMENT nor a sequence would do: SQLite and PostgreSQL
|
||||
-- spell those differently, and a PostgreSQL sequence is explicitly not
|
||||
-- transactional, so a rolled-back write would leave a hole in the stream
|
||||
-- that /sync would have to be taught to skip. An UPDATE on this row takes
|
||||
-- a row lock in both engines and gives out positions with no gaps.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_sequences (
|
||||
name TEXT NOT NULL PRIMARY KEY,
|
||||
value BIGINT NOT NULL
|
||||
);
|
||||
|
||||
INSERT INTO roomserver_sequences (name, value) VALUES ('stream_ordering', 0)
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
INSERT INTO roomserver_sequences (name, value) VALUES ('state_snapshot', 0)
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
|
||||
-- A state snapshot is the complete state of a room at one point, stored
|
||||
-- once and shared by every event it applies to. Consecutive messages in a
|
||||
-- room whose state is not changing all point at the same snapshot, so the
|
||||
-- cost is one snapshot per state change rather than one per event.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_state_snapshots (
|
||||
snapshot_id BIGINT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
-- SHA-256 over the sorted (type, state_key, event_id) triples: the
|
||||
-- identity of the snapshot's contents, which is what lets two
|
||||
-- identical states share a row.
|
||||
state_hash TEXT NOT NULL,
|
||||
UNIQUE (room_id, state_hash)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_state_entries (
|
||||
snapshot_id BIGINT NOT NULL REFERENCES roomserver_state_snapshots (snapshot_id) ON DELETE CASCADE,
|
||||
event_type TEXT NOT NULL,
|
||||
state_key TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
PRIMARY KEY (snapshot_id, event_type, state_key)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roomserver_events (
|
||||
event_id TEXT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
-- This server's own monotonic position for the event: the order it was
|
||||
-- persisted in, which is the order clients are shown it in.
|
||||
stream_ordering BIGINT NOT NULL,
|
||||
event_type TEXT NOT NULL,
|
||||
-- NULL for a message event. The empty string is a valid state key, so
|
||||
-- "no state key" cannot be spelled as ''.
|
||||
state_key TEXT,
|
||||
sender TEXT NOT NULL,
|
||||
depth BIGINT NOT NULL,
|
||||
origin_server_ts BIGINT NOT NULL,
|
||||
-- The event as canonical JSON, byte for byte as it was signed. Every
|
||||
-- other column here is a projection of it, written out to be indexed.
|
||||
event_json TEXT NOT NULL,
|
||||
-- The state of the room as of this event's prev_events, 0 for the
|
||||
-- create event. Keeping it per event is what lets the state after a
|
||||
-- forward extremity be worked out, which state resolution needs when a
|
||||
-- room has more than one.
|
||||
state_before_id BIGINT NOT NULL DEFAULT 0,
|
||||
-- The redaction that applies to this event, or ''. The JSON above is
|
||||
-- rewritten to its redacted form at the same time; this records which
|
||||
-- event did it so that unsigned.redacted_because can be filled in.
|
||||
redacted_by TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS roomserver_events_stream_idx
|
||||
ON roomserver_events (stream_ordering);
|
||||
|
||||
-- Paging a room's timeline, which is what /messages and /sync do.
|
||||
CREATE INDEX IF NOT EXISTS roomserver_events_room_stream_idx
|
||||
ON roomserver_events (room_id, stream_ordering);
|
||||
|
||||
-- The forward extremities: the events with no children, and so the
|
||||
-- prev_events of whatever is sent next.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_extremities (
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
event_id TEXT NOT NULL,
|
||||
PRIMARY KEY (room_id, event_id)
|
||||
);
|
||||
|
||||
-- The room's resolved current state: the state after its extremities.
|
||||
--
|
||||
-- It is derivable from the events, and is stored anyway because every
|
||||
-- authorisation decision reads it. The membership column is a denormalised
|
||||
-- copy of content.membership on m.room.member rows, so that "the rooms
|
||||
-- @alice is joined to" is an index lookup rather than a scan that parses
|
||||
-- JSON.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_current_state (
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
event_type TEXT NOT NULL,
|
||||
state_key TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
membership TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (room_id, event_type, state_key)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS roomserver_current_state_membership_idx
|
||||
ON roomserver_current_state (event_type, state_key, membership);
|
||||
|
||||
-- Client transaction IDs, so that a retried request returns the event the
|
||||
-- first attempt created instead of sending a second one.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_transactions (
|
||||
localpart TEXT NOT NULL,
|
||||
device_id TEXT NOT NULL,
|
||||
txn_id TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL,
|
||||
room_id TEXT NOT NULL,
|
||||
sent_at_ms BIGINT NOT NULL,
|
||||
PRIMARY KEY (localpart, device_id, txn_id)
|
||||
);
|
||||
|
||||
-- Local room aliases. Remote aliases are never stored: they are resolved
|
||||
-- over federation every time, because the other server may have moved them.
|
||||
CREATE TABLE IF NOT EXISTS roomserver_aliases (
|
||||
alias TEXT NOT NULL PRIMARY KEY,
|
||||
room_id TEXT NOT NULL REFERENCES roomserver_rooms (room_id) ON DELETE CASCADE,
|
||||
creator TEXT NOT NULL,
|
||||
created_at_ms BIGINT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS roomserver_aliases_room_idx
|
||||
ON roomserver_aliases (room_id);
|
||||
@@ -0,0 +1,987 @@
|
||||
package sqlstore
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/dariuszjaniak/warp/server/internal/storage"
|
||||
)
|
||||
|
||||
// Rooms is the shared SQL implementation of storage.RoomStore.
|
||||
//
|
||||
// Like the user store it is written once with `?` placeholders and rebound
|
||||
// per driver. The statements stay inside the syntax both engines agree on:
|
||||
// ON CONFLICT ... DO UPDATE with `excluded`, LIMIT/OFFSET, and no JSON
|
||||
// functions at all — what a room's events mean is the roomserver's
|
||||
// business, and the only reason this package ever looks inside one is that
|
||||
// it does not.
|
||||
type Rooms struct {
|
||||
db *sql.DB
|
||||
rebind func(string) string
|
||||
isUnique func(error) bool
|
||||
}
|
||||
|
||||
// NewRooms builds a room store.
|
||||
func NewRooms(db *sql.DB, rebind func(string) string, isUnique func(error) bool) *Rooms {
|
||||
if rebind == nil {
|
||||
rebind = func(query string) string { return query }
|
||||
}
|
||||
if isUnique == nil {
|
||||
isUnique = func(error) bool { return false }
|
||||
}
|
||||
return &Rooms{db: db, rebind: rebind, isUnique: isUnique}
|
||||
}
|
||||
|
||||
// Page sizes for a room listing. The public room directory is reachable
|
||||
// without a token on some deployments, so an unbounded ?limit= there would
|
||||
// be a way to ask the server to serialise every room it knows.
|
||||
const (
|
||||
defaultRoomPageSize = 100
|
||||
maxRoomPageSize = 1000
|
||||
)
|
||||
|
||||
// The two counters in roomserver_sequences.
|
||||
const (
|
||||
seqStreamOrdering = "stream_ordering"
|
||||
seqStateSnapshot = "state_snapshot"
|
||||
)
|
||||
|
||||
// q rebinds a statement for this driver.
|
||||
func (s *Rooms) q(query string) string { return s.rebind(query) }
|
||||
|
||||
// tx runs fn in a transaction, rolling back on any error.
|
||||
func (s *Rooms) tx(ctx context.Context, fn func(*sql.Tx) error) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin transaction: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback() }()
|
||||
|
||||
if err := fn(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit transaction: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const insertRoomSQL = `
|
||||
INSERT INTO roomserver_rooms (room_id, room_version, creator, is_published, created_at_ms)
|
||||
VALUES (?, ?, ?, ?, ?)`
|
||||
|
||||
// CreateRoom implements storage.RoomStore.
|
||||
func (s *Rooms) CreateRoom(ctx context.Context, room storage.NewRoom) error {
|
||||
_, err := s.db.ExecContext(ctx, s.q(insertRoomSQL),
|
||||
room.ID, room.Version, room.Creator, room.Published, room.CreatedAtMS)
|
||||
if err != nil {
|
||||
if s.isUnique(err) {
|
||||
return fmt.Errorf("create room %q: %w", room.ID, storage.ErrRoomExists)
|
||||
}
|
||||
return fmt.Errorf("create room %q: %w", room.ID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const selectRoomSQL = `
|
||||
SELECT room_id, room_version, creator, is_published, created_at_ms
|
||||
FROM roomserver_rooms WHERE room_id = ?`
|
||||
|
||||
// Room implements storage.RoomStore.
|
||||
func (s *Rooms) Room(ctx context.Context, roomID string) (storage.Room, error) {
|
||||
var room storage.Room
|
||||
err := s.db.QueryRowContext(ctx, s.q(selectRoomSQL), roomID).Scan(
|
||||
&room.ID, &room.Version, &room.Creator, &room.Published, &room.CreatedAtMS)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return storage.Room{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom)
|
||||
case err != nil:
|
||||
return storage.Room{}, fmt.Errorf("read room %q: %w", roomID, err)
|
||||
}
|
||||
return room, nil
|
||||
}
|
||||
|
||||
// Write implements storage.RoomStore.
|
||||
//
|
||||
// The order inside the transaction is the order the pieces depend on each
|
||||
// other: the stream position first, because the event row needs it; the
|
||||
// state snapshot next, because the event row points at it; then the event,
|
||||
// then everything that refers to the event by ID.
|
||||
func (s *Rooms) Write(ctx context.Context, w storage.RoomWrite) (int64, error) {
|
||||
var ordering int64
|
||||
err := s.tx(ctx, func(tx *sql.Tx) error {
|
||||
var err error
|
||||
if ordering, err = s.nextSequence(ctx, tx, seqStreamOrdering); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
snapshotID, err := s.stateSnapshot(ctx, tx, w.Event.RoomID, w.StateBefore)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := s.insertEvent(ctx, tx, w.Event, ordering, snapshotID); err != nil {
|
||||
return err
|
||||
}
|
||||
if w.Redacts != "" {
|
||||
if err := s.applyRedaction(ctx, tx, w.Event.RoomID, w.Redacts,
|
||||
w.Event.EventID, w.RedactedJSON); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := s.applyStateDelta(ctx, tx, w); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.replaceExtremities(ctx, tx, w.Event.RoomID, w.Extremities); err != nil {
|
||||
return err
|
||||
}
|
||||
if w.Profile != nil {
|
||||
if err := s.updateProfile(ctx, tx, w.Event.RoomID, *w.Profile); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// The member count is recomputed whenever current state moved,
|
||||
// because that is the only thing that can change it.
|
||||
if len(w.StateDelta) > 0 || w.StateDeltaIsWholeState {
|
||||
if err := s.recountMembers(ctx, tx, w.Event.RoomID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if w.Transaction != nil {
|
||||
if err := s.recordTransaction(ctx, tx, *w.Transaction,
|
||||
w.Event.EventID, w.Event.RoomID, w.Event.OriginServerTS); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("write event %s: %w", w.Event.EventID, err)
|
||||
}
|
||||
return ordering, nil
|
||||
}
|
||||
|
||||
const (
|
||||
bumpSequenceSQL = `UPDATE roomserver_sequences SET value = value + 1 WHERE name = ?`
|
||||
selectSequenceSQL = `SELECT value FROM roomserver_sequences WHERE name = ?`
|
||||
)
|
||||
|
||||
// nextSequence hands out the next value of a counter.
|
||||
//
|
||||
// The UPDATE takes a row lock that both engines hold until the transaction
|
||||
// ends, so two concurrent writers queue rather than collide, and a value
|
||||
// handed out by a transaction that then rolls back is never used — which is
|
||||
// what keeps the stream gapless. A database sequence would be faster and
|
||||
// would not have that property.
|
||||
func (s *Rooms) nextSequence(ctx context.Context, tx *sql.Tx, name string) (int64, error) {
|
||||
if _, err := tx.ExecContext(ctx, s.q(bumpSequenceSQL), name); err != nil {
|
||||
return 0, fmt.Errorf("advance the %s counter: %w", name, err)
|
||||
}
|
||||
var value int64
|
||||
if err := tx.QueryRowContext(ctx, s.q(selectSequenceSQL), name).Scan(&value); err != nil {
|
||||
return 0, fmt.Errorf("read the %s counter: %w", name, err)
|
||||
}
|
||||
return value, nil
|
||||
}
|
||||
|
||||
const (
|
||||
selectSnapshotSQL = `
|
||||
SELECT snapshot_id FROM roomserver_state_snapshots
|
||||
WHERE room_id = ? AND state_hash = ?`
|
||||
insertSnapshotSQL = `
|
||||
INSERT INTO roomserver_state_snapshots (snapshot_id, room_id, state_hash)
|
||||
VALUES (?, ?, ?)`
|
||||
insertStateEntrySQL = `
|
||||
INSERT INTO roomserver_state_entries (snapshot_id, event_type, state_key, event_id)
|
||||
VALUES (?, ?, ?, ?)`
|
||||
)
|
||||
|
||||
// stateSnapshot stores a state set, re-using the row of an identical one.
|
||||
//
|
||||
// Deduplication is what makes a per-event state snapshot affordable: every
|
||||
// message in a room whose state is not changing resolves to the snapshot
|
||||
// the previous one already stored, so the table grows with state changes
|
||||
// rather than with traffic. An empty state set is snapshot 0 and stores
|
||||
// nothing, which is the create event's case.
|
||||
func (s *Rooms) stateSnapshot(ctx context.Context, tx *sql.Tx, roomID string, entries []storage.StateEntry) (int64, error) {
|
||||
if len(entries) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
hash := stateHash(entries)
|
||||
|
||||
var id int64
|
||||
err := tx.QueryRowContext(ctx, s.q(selectSnapshotSQL), roomID, hash).Scan(&id)
|
||||
switch {
|
||||
case err == nil:
|
||||
return id, nil
|
||||
case !errors.Is(err, sql.ErrNoRows):
|
||||
return 0, fmt.Errorf("look up state snapshot: %w", err)
|
||||
}
|
||||
|
||||
if id, err = s.nextSequence(ctx, tx, seqStateSnapshot); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, s.q(insertSnapshotSQL), id, roomID, hash); err != nil {
|
||||
return 0, fmt.Errorf("insert state snapshot: %w", err)
|
||||
}
|
||||
for _, e := range entries {
|
||||
if _, err := tx.ExecContext(ctx, s.q(insertStateEntrySQL),
|
||||
id, e.Type, e.StateKey, e.EventID); err != nil {
|
||||
return 0, fmt.Errorf("insert state entry %s/%s: %w", e.Type, e.StateKey, err)
|
||||
}
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// stateHash identifies a state set by its contents.
|
||||
//
|
||||
// The entries are sorted first so that the same state always hashes the
|
||||
// same whatever order it was assembled in, and each field is written with
|
||||
// its length so that no two different sets can be run together into the
|
||||
// same bytes.
|
||||
func stateHash(entries []storage.StateEntry) string {
|
||||
sorted := slices.Clone(entries)
|
||||
slices.SortFunc(sorted, func(a, b storage.StateEntry) int {
|
||||
if c := strings.Compare(a.Type, b.Type); c != 0 {
|
||||
return c
|
||||
}
|
||||
return strings.Compare(a.StateKey, b.StateKey)
|
||||
})
|
||||
|
||||
h := sha256.New()
|
||||
for _, e := range sorted {
|
||||
for _, field := range [...]string{e.Type, e.StateKey, e.EventID} {
|
||||
fmt.Fprintf(h, "%d:%s", len(field), field)
|
||||
}
|
||||
}
|
||||
return base64.RawURLEncoding.EncodeToString(h.Sum(nil))
|
||||
}
|
||||
|
||||
const insertEventSQL = `
|
||||
INSERT INTO roomserver_events (
|
||||
event_id, room_id, stream_ordering, event_type, state_key,
|
||||
sender, depth, origin_server_ts, event_json, state_before_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
|
||||
|
||||
func (s *Rooms) insertEvent(ctx context.Context, tx *sql.Tx, e storage.StoredEvent, ordering, snapshotID int64) error {
|
||||
_, err := tx.ExecContext(ctx, s.q(insertEventSQL),
|
||||
e.EventID, e.RoomID, ordering, e.Type, e.StateKey,
|
||||
e.Sender, e.Depth, e.OriginServerTS, string(e.JSON), snapshotID)
|
||||
if err != nil {
|
||||
if s.isUnique(err) {
|
||||
// The same event twice is not a failure worth propagating: an
|
||||
// event ID is a hash of its contents, so the second copy says
|
||||
// exactly what the first one did.
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("insert event: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const applyRedactionSQL = `
|
||||
UPDATE roomserver_events SET event_json = ?, redacted_by = ?
|
||||
WHERE room_id = ? AND event_id = ? AND redacted_by = ''`
|
||||
|
||||
// applyRedaction replaces a stored event with its redacted form.
|
||||
//
|
||||
// The condition on redacted_by makes a second redaction of the same event a
|
||||
// no-op rather than a rewrite: the first one already stripped the content,
|
||||
// and the event may only be redacted once.
|
||||
func (s *Rooms) applyRedaction(ctx context.Context, tx *sql.Tx, roomID, targetID, redactionID string, redacted []byte) error {
|
||||
_, err := tx.ExecContext(ctx, s.q(applyRedactionSQL),
|
||||
string(redacted), redactionID, roomID, targetID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("apply redaction to %s: %w", targetID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const (
|
||||
upsertCurrentStateSQL = `
|
||||
INSERT INTO roomserver_current_state (room_id, event_type, state_key, event_id, membership)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT (room_id, event_type, state_key) DO UPDATE
|
||||
SET event_id = excluded.event_id, membership = excluded.membership`
|
||||
deleteCurrentStateSQL = `DELETE FROM roomserver_current_state WHERE room_id = ?`
|
||||
)
|
||||
|
||||
// applyStateDelta moves the room's current state.
|
||||
//
|
||||
// The usual case is a single tuple to upsert. A whole-state write — which
|
||||
// only state resolution produces — clears the table for the room first,
|
||||
// because tuples the resolution dropped have to go.
|
||||
func (s *Rooms) applyStateDelta(ctx context.Context, tx *sql.Tx, w storage.RoomWrite) error {
|
||||
if len(w.StateDelta) == 0 && !w.StateDeltaIsWholeState {
|
||||
return nil
|
||||
}
|
||||
if w.StateDeltaIsWholeState {
|
||||
if _, err := tx.ExecContext(ctx, s.q(deleteCurrentStateSQL), w.Event.RoomID); err != nil {
|
||||
return fmt.Errorf("clear current state: %w", err)
|
||||
}
|
||||
}
|
||||
for _, e := range w.StateDelta {
|
||||
if _, err := tx.ExecContext(ctx, s.q(upsertCurrentStateSQL),
|
||||
w.Event.RoomID, e.Type, e.StateKey, e.EventID, e.Membership); err != nil {
|
||||
return fmt.Errorf("set current state %s/%s: %w", e.Type, e.StateKey, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const (
|
||||
deleteExtremitiesSQL = `DELETE FROM roomserver_extremities WHERE room_id = ?`
|
||||
insertExtremitySQL = `
|
||||
INSERT INTO roomserver_extremities (room_id, event_id) VALUES (?, ?)
|
||||
ON CONFLICT (room_id, event_id) DO NOTHING`
|
||||
)
|
||||
|
||||
func (s *Rooms) replaceExtremities(ctx context.Context, tx *sql.Tx, roomID string, extremities []string) error {
|
||||
if len(extremities) == 0 {
|
||||
return nil
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, s.q(deleteExtremitiesSQL), roomID); err != nil {
|
||||
return fmt.Errorf("clear forward extremities: %w", err)
|
||||
}
|
||||
for _, id := range extremities {
|
||||
if _, err := tx.ExecContext(ctx, s.q(insertExtremitySQL), roomID, id); err != nil {
|
||||
return fmt.Errorf("insert forward extremity %s: %w", id, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const updateProfileSQL = `
|
||||
UPDATE roomserver_rooms SET
|
||||
name = ?, topic = ?, avatar_url = ?, canonical_alias = ?,
|
||||
join_rule = ?, history_visibility = ?, guest_access = ?, room_type = ?
|
||||
WHERE room_id = ?`
|
||||
|
||||
func (s *Rooms) updateProfile(ctx context.Context, tx *sql.Tx, roomID string, p storage.RoomProfile) error {
|
||||
_, err := tx.ExecContext(ctx, s.q(updateProfileSQL),
|
||||
p.Name, p.Topic, p.AvatarURL, p.CanonicalAlias,
|
||||
p.JoinRule, p.HistoryVisibility, p.GuestAccess, p.RoomType, roomID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("update room profile: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const recountMembersSQL = `
|
||||
UPDATE roomserver_rooms SET joined_members = (
|
||||
SELECT COUNT(*) FROM roomserver_current_state
|
||||
WHERE room_id = ? AND membership = 'join'
|
||||
) WHERE room_id = ?`
|
||||
|
||||
func (s *Rooms) recountMembers(ctx context.Context, tx *sql.Tx, roomID string) error {
|
||||
if _, err := tx.ExecContext(ctx, s.q(recountMembersSQL), roomID, roomID); err != nil {
|
||||
return fmt.Errorf("recount joined members: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const insertTransactionSQL = `
|
||||
INSERT INTO roomserver_transactions (localpart, device_id, txn_id, event_id, room_id, sent_at_ms)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (localpart, device_id, txn_id) DO NOTHING`
|
||||
|
||||
func (s *Rooms) recordTransaction(ctx context.Context, tx *sql.Tx, ref storage.TransactionRef, eventID, roomID string, atMS int64) error {
|
||||
_, err := tx.ExecContext(ctx, s.q(insertTransactionSQL),
|
||||
ref.Localpart, ref.DeviceID, ref.TxnID, eventID, roomID, atMS)
|
||||
if err != nil {
|
||||
return fmt.Errorf("record transaction %q: %w", ref.TxnID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// eventColumns is the select list every event read shares, in the order
|
||||
// scanEvent expects.
|
||||
const eventColumns = `
|
||||
e.room_id, e.event_id, e.event_type, e.state_key, e.sender,
|
||||
e.depth, e.origin_server_ts, e.event_json, e.stream_ordering, e.redacted_by`
|
||||
|
||||
// scanEvent reads one row of eventColumns.
|
||||
func scanEvent(scan func(...any) error) (storage.StoredEvent, error) {
|
||||
var (
|
||||
e storage.StoredEvent
|
||||
stateKey sql.NullString
|
||||
body string
|
||||
)
|
||||
err := scan(&e.RoomID, &e.EventID, &e.Type, &stateKey, &e.Sender,
|
||||
&e.Depth, &e.OriginServerTS, &body, &e.StreamOrdering, &e.RedactedBy)
|
||||
if err != nil {
|
||||
return storage.StoredEvent{}, err
|
||||
}
|
||||
if stateKey.Valid {
|
||||
e.StateKey = &stateKey.String
|
||||
}
|
||||
e.JSON = []byte(body)
|
||||
return e, nil
|
||||
}
|
||||
|
||||
// scanEvents drains a result set of eventColumns rows.
|
||||
func scanEvents(rows *sql.Rows) ([]storage.StoredEvent, error) {
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
var out []storage.StoredEvent
|
||||
for rows.Next() {
|
||||
e, err := scanEvent(rows.Scan)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
var selectEventSQL = `
|
||||
SELECT ` + eventColumns + `
|
||||
FROM roomserver_events e WHERE e.room_id = ? AND e.event_id = ?`
|
||||
|
||||
// Event implements storage.RoomStore.
|
||||
func (s *Rooms) Event(ctx context.Context, roomID, eventID string) (storage.StoredEvent, error) {
|
||||
e, err := scanEvent(s.db.QueryRowContext(ctx, s.q(selectEventSQL), roomID, eventID).Scan)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return storage.StoredEvent{}, fmt.Errorf("event %q: %w", eventID, storage.ErrNoSuchEvent)
|
||||
case err != nil:
|
||||
return storage.StoredEvent{}, fmt.Errorf("read event %q: %w", eventID, err)
|
||||
}
|
||||
return e, nil
|
||||
}
|
||||
|
||||
// EventsByID implements storage.RoomStore.
|
||||
func (s *Rooms) EventsByID(ctx context.Context, roomID string, eventIDs []string) ([]storage.StoredEvent, error) {
|
||||
if len(eventIDs) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
query := `SELECT ` + eventColumns + `
|
||||
FROM roomserver_events e
|
||||
WHERE e.room_id = ? AND e.event_id IN (` + placeholders(len(eventIDs)) + `)`
|
||||
|
||||
args := make([]any, 0, len(eventIDs)+1)
|
||||
args = append(args, roomID)
|
||||
for _, id := range eventIDs {
|
||||
args = append(args, id)
|
||||
}
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read events by ID: %w", err)
|
||||
}
|
||||
events, err := scanEvents(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read events by ID: %w", err)
|
||||
}
|
||||
return events, nil
|
||||
}
|
||||
|
||||
const selectCurrentStateSQL = `
|
||||
SELECT event_type, state_key, event_id, membership
|
||||
FROM roomserver_current_state WHERE room_id = ?
|
||||
ORDER BY event_type, state_key`
|
||||
|
||||
// CurrentState implements storage.RoomStore.
|
||||
func (s *Rooms) CurrentState(ctx context.Context, roomID string) ([]storage.StateEntry, error) {
|
||||
rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateSQL), roomID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read current state of %q: %w", roomID, err)
|
||||
}
|
||||
entries, err := scanStateEntries(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read current state of %q: %w", roomID, err)
|
||||
}
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
func scanStateEntries(rows *sql.Rows) ([]storage.StateEntry, error) {
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
var out []storage.StateEntry
|
||||
for rows.Next() {
|
||||
var e storage.StateEntry
|
||||
if err := rows.Scan(&e.Type, &e.StateKey, &e.EventID, &e.Membership); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
var selectCurrentStateEventsSQL = `
|
||||
SELECT ` + eventColumns + `
|
||||
FROM roomserver_current_state cs
|
||||
JOIN roomserver_events e ON e.event_id = cs.event_id
|
||||
WHERE cs.room_id = ?
|
||||
ORDER BY cs.event_type, cs.state_key`
|
||||
|
||||
// CurrentStateEvents implements storage.RoomStore.
|
||||
func (s *Rooms) CurrentStateEvents(ctx context.Context, roomID string) ([]storage.StoredEvent, error) {
|
||||
rows, err := s.db.QueryContext(ctx, s.q(selectCurrentStateEventsSQL), roomID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read current state events of %q: %w", roomID, err)
|
||||
}
|
||||
events, err := scanEvents(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read current state events of %q: %w", roomID, err)
|
||||
}
|
||||
return events, nil
|
||||
}
|
||||
|
||||
var selectCurrentStateEventSQL = `
|
||||
SELECT ` + eventColumns + `
|
||||
FROM roomserver_current_state cs
|
||||
JOIN roomserver_events e ON e.event_id = cs.event_id
|
||||
WHERE cs.room_id = ? AND cs.event_type = ? AND cs.state_key = ?`
|
||||
|
||||
// CurrentStateEvent implements storage.RoomStore.
|
||||
func (s *Rooms) CurrentStateEvent(ctx context.Context, roomID, eventType, stateKey string) (storage.StoredEvent, error) {
|
||||
e, err := scanEvent(s.db.QueryRowContext(ctx,
|
||||
s.q(selectCurrentStateEventSQL), roomID, eventType, stateKey).Scan)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return storage.StoredEvent{}, fmt.Errorf("state %s/%s of %q: %w",
|
||||
eventType, stateKey, roomID, storage.ErrNoSuchEvent)
|
||||
case err != nil:
|
||||
return storage.StoredEvent{}, fmt.Errorf("read state %s/%s of %q: %w",
|
||||
eventType, stateKey, roomID, err)
|
||||
}
|
||||
return e, nil
|
||||
}
|
||||
|
||||
const selectStateBeforeSQL = `
|
||||
SELECT se.event_type, se.state_key, se.event_id, ''
|
||||
FROM roomserver_events e
|
||||
JOIN roomserver_state_entries se ON se.snapshot_id = e.state_before_id
|
||||
WHERE e.room_id = ? AND e.event_id = ?
|
||||
ORDER BY se.event_type, se.state_key`
|
||||
|
||||
// StateBefore implements storage.RoomStore.
|
||||
//
|
||||
// The membership column is not stored on a snapshot entry: it exists on
|
||||
// current state to make a membership lookup an index hit, and a historical
|
||||
// snapshot is only ever read as a whole and then parsed anyway.
|
||||
func (s *Rooms) StateBefore(ctx context.Context, roomID, eventID string) ([]storage.StateEntry, error) {
|
||||
rows, err := s.db.QueryContext(ctx, s.q(selectStateBeforeSQL), roomID, eventID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the state before %q: %w", eventID, err)
|
||||
}
|
||||
entries, err := scanStateEntries(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the state before %q: %w", eventID, err)
|
||||
}
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
var selectExtremitiesSQL = `
|
||||
SELECT ` + eventColumns + `
|
||||
FROM roomserver_extremities x
|
||||
JOIN roomserver_events e ON e.event_id = x.event_id
|
||||
WHERE x.room_id = ?
|
||||
ORDER BY e.stream_ordering`
|
||||
|
||||
// Extremities implements storage.RoomStore.
|
||||
func (s *Rooms) Extremities(ctx context.Context, roomID string) ([]storage.StoredEvent, error) {
|
||||
rows, err := s.db.QueryContext(ctx, s.q(selectExtremitiesSQL), roomID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err)
|
||||
}
|
||||
events, err := scanEvents(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the forward extremities of %q: %w", roomID, err)
|
||||
}
|
||||
return events, nil
|
||||
}
|
||||
|
||||
// Members implements storage.RoomStore.
|
||||
func (s *Rooms) Members(ctx context.Context, roomID string, memberships []string) ([]storage.StoredEvent, error) {
|
||||
query := `SELECT ` + eventColumns + `
|
||||
FROM roomserver_current_state cs
|
||||
JOIN roomserver_events e ON e.event_id = cs.event_id
|
||||
WHERE cs.room_id = ? AND cs.event_type = 'm.room.member'`
|
||||
args := []any{roomID}
|
||||
if len(memberships) > 0 {
|
||||
query += ` AND cs.membership IN (` + placeholders(len(memberships)) + `)`
|
||||
for _, m := range memberships {
|
||||
args = append(args, m)
|
||||
}
|
||||
}
|
||||
query += ` ORDER BY cs.state_key`
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the members of %q: %w", roomID, err)
|
||||
}
|
||||
events, err := scanEvents(rows)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the members of %q: %w", roomID, err)
|
||||
}
|
||||
return events, nil
|
||||
}
|
||||
|
||||
// MembershipEntries implements storage.RoomStore.
|
||||
func (s *Rooms) MembershipEntries(ctx context.Context, roomID string, memberships []string) ([]storage.MembershipEntry, error) {
|
||||
query := `
|
||||
SELECT state_key, membership, event_id FROM roomserver_current_state
|
||||
WHERE room_id = ? AND event_type = 'm.room.member'`
|
||||
args := []any{roomID}
|
||||
if len(memberships) > 0 {
|
||||
query += ` AND membership IN (` + placeholders(len(memberships)) + `)`
|
||||
for _, m := range memberships {
|
||||
args = append(args, m)
|
||||
}
|
||||
}
|
||||
query += ` ORDER BY state_key`
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
var out []storage.MembershipEntry
|
||||
for rows.Next() {
|
||||
var m storage.MembershipEntry
|
||||
if err := rows.Scan(&m.UserID, &m.Membership, &m.EventID); err != nil {
|
||||
return nil, fmt.Errorf("read the memberships of %q: %w", roomID, err)
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
const selectMembershipSQL = `
|
||||
SELECT state_key, membership, event_id FROM roomserver_current_state
|
||||
WHERE room_id = ? AND event_type = 'm.room.member' AND state_key = ?`
|
||||
|
||||
// Membership implements storage.RoomStore.
|
||||
func (s *Rooms) Membership(ctx context.Context, roomID, userID string) (storage.MembershipEntry, error) {
|
||||
var m storage.MembershipEntry
|
||||
err := s.db.QueryRowContext(ctx, s.q(selectMembershipSQL), roomID, userID).
|
||||
Scan(&m.UserID, &m.Membership, &m.EventID)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
// Never a member is not an error: it is the same answer as having
|
||||
// left, which is what the rules above this ask for.
|
||||
return storage.MembershipEntry{UserID: userID}, nil
|
||||
case err != nil:
|
||||
return storage.MembershipEntry{}, fmt.Errorf("read the membership of %q in %q: %w", userID, roomID, err)
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// RoomsForUser implements storage.RoomStore.
|
||||
func (s *Rooms) RoomsForUser(ctx context.Context, userID string, memberships []string) ([]string, error) {
|
||||
query := `
|
||||
SELECT room_id FROM roomserver_current_state
|
||||
WHERE event_type = 'm.room.member' AND state_key = ?`
|
||||
args := []any{userID}
|
||||
if len(memberships) > 0 {
|
||||
query += ` AND membership IN (` + placeholders(len(memberships)) + `)`
|
||||
for _, m := range memberships {
|
||||
args = append(args, m)
|
||||
}
|
||||
}
|
||||
query += ` ORDER BY room_id`
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, s.q(query), args...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the rooms of %q: %w", userID, err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
var out []string
|
||||
for rows.Next() {
|
||||
var roomID string
|
||||
if err := rows.Scan(&roomID); err != nil {
|
||||
return nil, fmt.Errorf("read the rooms of %q: %w", userID, err)
|
||||
}
|
||||
out = append(out, roomID)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
const selectTransactionSQL = `
|
||||
SELECT event_id FROM roomserver_transactions
|
||||
WHERE localpart = ? AND device_id = ? AND txn_id = ?`
|
||||
|
||||
// TransactionEvent implements storage.RoomStore.
|
||||
func (s *Rooms) TransactionEvent(ctx context.Context, ref storage.TransactionRef) (string, error) {
|
||||
var eventID string
|
||||
err := s.db.QueryRowContext(ctx, s.q(selectTransactionSQL),
|
||||
ref.Localpart, ref.DeviceID, ref.TxnID).Scan(&eventID)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return "", fmt.Errorf("transaction %q: %w", ref.TxnID, storage.ErrNoSuchEvent)
|
||||
case err != nil:
|
||||
return "", fmt.Errorf("read transaction %q: %w", ref.TxnID, err)
|
||||
}
|
||||
return eventID, nil
|
||||
}
|
||||
|
||||
const insertAliasSQL = `
|
||||
INSERT INTO roomserver_aliases (alias, room_id, creator, created_at_ms)
|
||||
VALUES (?, ?, ?, ?)`
|
||||
|
||||
// CreateAlias implements storage.RoomStore.
|
||||
func (s *Rooms) CreateAlias(ctx context.Context, alias storage.RoomAlias) error {
|
||||
_, err := s.db.ExecContext(ctx, s.q(insertAliasSQL),
|
||||
alias.Alias, alias.RoomID, alias.Creator, alias.CreatedAtMS)
|
||||
if err != nil {
|
||||
if s.isUnique(err) {
|
||||
return fmt.Errorf("create alias %q: %w", alias.Alias, storage.ErrAliasExists)
|
||||
}
|
||||
return fmt.Errorf("create alias %q: %w", alias.Alias, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const deleteAliasSQL = `DELETE FROM roomserver_aliases WHERE alias = ?`
|
||||
|
||||
// DeleteAlias implements storage.RoomStore.
|
||||
func (s *Rooms) DeleteAlias(ctx context.Context, alias string) error {
|
||||
result, err := s.db.ExecContext(ctx, s.q(deleteAliasSQL), alias)
|
||||
if err != nil {
|
||||
return fmt.Errorf("delete alias %q: %w", alias, err)
|
||||
}
|
||||
return expectOneRow(result, alias, storage.ErrNoSuchAlias)
|
||||
}
|
||||
|
||||
const selectAliasSQL = `
|
||||
SELECT alias, room_id, creator, created_at_ms
|
||||
FROM roomserver_aliases WHERE alias = ?`
|
||||
|
||||
// Alias implements storage.RoomStore.
|
||||
func (s *Rooms) Alias(ctx context.Context, alias string) (storage.RoomAlias, error) {
|
||||
var out storage.RoomAlias
|
||||
err := s.db.QueryRowContext(ctx, s.q(selectAliasSQL), alias).
|
||||
Scan(&out.Alias, &out.RoomID, &out.Creator, &out.CreatedAtMS)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return storage.RoomAlias{}, fmt.Errorf("alias %q: %w", alias, storage.ErrNoSuchAlias)
|
||||
case err != nil:
|
||||
return storage.RoomAlias{}, fmt.Errorf("resolve alias %q: %w", alias, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
const selectRoomAliasesSQL = `
|
||||
SELECT alias FROM roomserver_aliases WHERE room_id = ? ORDER BY alias`
|
||||
|
||||
// AliasesForRoom implements storage.RoomStore.
|
||||
func (s *Rooms) AliasesForRoom(ctx context.Context, roomID string) ([]string, error) {
|
||||
rows, err := s.db.QueryContext(ctx, s.q(selectRoomAliasesSQL), roomID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
var out []string
|
||||
for rows.Next() {
|
||||
var alias string
|
||||
if err := rows.Scan(&alias); err != nil {
|
||||
return nil, fmt.Errorf("read the aliases of %q: %w", roomID, err)
|
||||
}
|
||||
out = append(out, alias)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
const setPublishedSQL = `UPDATE roomserver_rooms SET is_published = ? WHERE room_id = ?`
|
||||
|
||||
// SetPublished implements storage.RoomStore.
|
||||
func (s *Rooms) SetPublished(ctx context.Context, roomID string, published bool) error {
|
||||
result, err := s.db.ExecContext(ctx, s.q(setPublishedSQL), published, roomID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("set the published flag of %q: %w", roomID, err)
|
||||
}
|
||||
return expectOneRow(result, roomID, storage.ErrNoSuchRoom)
|
||||
}
|
||||
|
||||
// roomSummaryColumns is the select list a room listing shares, in the order
|
||||
// scanRoomSummary expects.
|
||||
const roomSummaryColumns = `
|
||||
room_id, room_version, creator, is_published, created_at_ms,
|
||||
name, topic, avatar_url, canonical_alias,
|
||||
join_rule, history_visibility, guest_access, room_type, joined_members`
|
||||
|
||||
func scanRoomSummary(scan func(...any) error) (storage.RoomSummary, error) {
|
||||
var r storage.RoomSummary
|
||||
err := scan(&r.ID, &r.Version, &r.Creator, &r.Published, &r.CreatedAtMS,
|
||||
&r.Name, &r.Topic, &r.AvatarURL, &r.CanonicalAlias,
|
||||
&r.JoinRule, &r.HistoryVisibility, &r.GuestAccess, &r.RoomType, &r.JoinedMembers)
|
||||
return r, err
|
||||
}
|
||||
|
||||
// PublishedRooms implements storage.RoomStore.
|
||||
func (s *Rooms) PublishedRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) {
|
||||
return s.listRooms(ctx, filter, true)
|
||||
}
|
||||
|
||||
// ListRooms implements storage.RoomStore.
|
||||
func (s *Rooms) ListRooms(ctx context.Context, filter storage.RoomFilter) ([]storage.RoomSummary, int, error) {
|
||||
return s.listRooms(ctx, filter, false)
|
||||
}
|
||||
|
||||
// listRooms is the body of both listings: the same query with and without
|
||||
// the published condition.
|
||||
//
|
||||
// Rooms are ordered by descending member count and then by room ID. The
|
||||
// second key is not decoration: paging a listing whose order is not total
|
||||
// can show the same room on two pages and skip another, and ties in member
|
||||
// count are the common case in a directory of small rooms.
|
||||
func (s *Rooms) listRooms(ctx context.Context, filter storage.RoomFilter, publishedOnly bool) ([]storage.RoomSummary, int, error) {
|
||||
where := ` WHERE 1 = 1`
|
||||
var args []any
|
||||
if publishedOnly {
|
||||
where += ` AND is_published = ?`
|
||||
args = append(args, true)
|
||||
}
|
||||
if search := strings.TrimSpace(filter.Search); search != "" {
|
||||
pattern := "%" + strings.ToLower(search) + "%"
|
||||
where += ` AND (LOWER(room_id) LIKE ? OR LOWER(name) LIKE ?
|
||||
OR LOWER(topic) LIKE ? OR LOWER(canonical_alias) LIKE ?)`
|
||||
args = append(args, pattern, pattern, pattern, pattern)
|
||||
}
|
||||
|
||||
var total int
|
||||
if err := s.db.QueryRowContext(ctx,
|
||||
s.q(`SELECT COUNT(*) FROM roomserver_rooms`+where), args...).Scan(&total); err != nil {
|
||||
return nil, 0, fmt.Errorf("count rooms: %w", err)
|
||||
}
|
||||
|
||||
limit, offset := pageBounds(filter.Limit, filter.Offset, defaultRoomPageSize, maxRoomPageSize)
|
||||
query := `SELECT ` + roomSummaryColumns + ` FROM roomserver_rooms` + where +
|
||||
` ORDER BY joined_members DESC, room_id LIMIT ? OFFSET ?`
|
||||
|
||||
rows, err := s.db.QueryContext(ctx, s.q(query), append(args, limit, offset)...)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("list rooms: %w", err)
|
||||
}
|
||||
defer func() { _ = rows.Close() }()
|
||||
|
||||
out := make([]storage.RoomSummary, 0, limit)
|
||||
for rows.Next() {
|
||||
r, err := scanRoomSummary(rows.Scan)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("list rooms: %w", err)
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, 0, fmt.Errorf("list rooms: %w", err)
|
||||
}
|
||||
return out, total, nil
|
||||
}
|
||||
|
||||
var selectRoomSummarySQL = `SELECT ` + roomSummaryColumns +
|
||||
` FROM roomserver_rooms WHERE room_id = ?`
|
||||
|
||||
// RoomSummary implements storage.RoomStore.
|
||||
func (s *Rooms) RoomSummary(ctx context.Context, roomID string) (storage.RoomSummary, error) {
|
||||
r, err := scanRoomSummary(s.db.QueryRowContext(ctx, s.q(selectRoomSummarySQL), roomID).Scan)
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
return storage.RoomSummary{}, fmt.Errorf("room %q: %w", roomID, storage.ErrNoSuchRoom)
|
||||
case err != nil:
|
||||
return storage.RoomSummary{}, fmt.Errorf("read room %q: %w", roomID, err)
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
// purgeStatements are run in order, children before parents.
|
||||
//
|
||||
// Both engines are given ON DELETE CASCADE on these tables, but SQLite
|
||||
// enforces foreign keys only when the pragma is on, and the transactions
|
||||
// table has no key at all because it outlives the room it refers to. The
|
||||
// explicit order is what makes a purge mean the same thing on both.
|
||||
var purgeStatements = []string{
|
||||
`DELETE FROM roomserver_state_entries WHERE snapshot_id IN (
|
||||
SELECT snapshot_id FROM roomserver_state_snapshots WHERE room_id = ?)`,
|
||||
`DELETE FROM roomserver_state_snapshots WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_current_state WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_extremities WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_transactions WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_aliases WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_events WHERE room_id = ?`,
|
||||
`DELETE FROM roomserver_rooms WHERE room_id = ?`,
|
||||
}
|
||||
|
||||
// PurgeRoom implements storage.RoomStore.
|
||||
func (s *Rooms) PurgeRoom(ctx context.Context, roomID string) error {
|
||||
err := s.tx(ctx, func(tx *sql.Tx) error {
|
||||
for i, stmt := range purgeStatements {
|
||||
result, err := tx.ExecContext(ctx, s.q(stmt), roomID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// The last statement is the room row itself: no row there means
|
||||
// there was no room, and the whole purge rolls back so that the
|
||||
// caller's 404 is the truth.
|
||||
if i == len(purgeStatements)-1 {
|
||||
if err := expectOneRow(result, roomID, storage.ErrNoSuchRoom); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("purge room %q: %w", roomID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const roomStatsSQL = `
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM roomserver_rooms),
|
||||
(SELECT COUNT(*) FROM roomserver_rooms WHERE is_published = ?),
|
||||
(SELECT COUNT(*) FROM roomserver_events)`
|
||||
|
||||
// RoomStats implements storage.RoomStore.
|
||||
func (s *Rooms) RoomStats(ctx context.Context) (storage.RoomStats, error) {
|
||||
var stats storage.RoomStats
|
||||
err := s.db.QueryRowContext(ctx, s.q(roomStatsSQL), true).
|
||||
Scan(&stats.Rooms, &stats.Published, &stats.Events)
|
||||
if err != nil {
|
||||
return storage.RoomStats{}, fmt.Errorf("read room statistics: %w", err)
|
||||
}
|
||||
return stats, nil
|
||||
}
|
||||
|
||||
// placeholders renders n comma-separated `?` placeholders for an IN clause.
|
||||
func placeholders(n int) string {
|
||||
return strings.TrimSuffix(strings.Repeat("?, ", n), ", ")
|
||||
}
|
||||
|
||||
// pageBounds clamps a caller's limit and offset.
|
||||
func pageBounds(limit, offset, defaultLimit, maxLimit int) (int, int) {
|
||||
if limit <= 0 {
|
||||
limit = defaultLimit
|
||||
}
|
||||
if limit > maxLimit {
|
||||
limit = maxLimit
|
||||
}
|
||||
if offset < 0 {
|
||||
offset = 0
|
||||
}
|
||||
return limit, offset
|
||||
}
|
||||
|
||||
// expectOneRow turns "the UPDATE or DELETE matched nothing" into notFound.
|
||||
func expectOneRow(result sql.Result, subject string, notFound error) error {
|
||||
n, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return fmt.Errorf("count the affected rows: %w", err)
|
||||
}
|
||||
if n == 0 {
|
||||
return fmt.Errorf("%q: %w", subject, notFound)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -57,6 +57,7 @@ type Store struct {
|
||||
log *slog.Logger
|
||||
settings *Settings
|
||||
users *Users
|
||||
rooms *Rooms
|
||||
}
|
||||
|
||||
// New builds a Store. It does not touch the database.
|
||||
@@ -73,6 +74,7 @@ func New(opts Options) (*Store, error) {
|
||||
log: opts.Logger,
|
||||
settings: NewSettings(opts.DB, opts.Settings),
|
||||
users: NewUsers(opts.DB, opts.Rebind, opts.IsUniqueViolation),
|
||||
rooms: NewRooms(opts.DB, opts.Rebind, opts.IsUniqueViolation),
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -94,6 +96,9 @@ func (s *Store) Settings() storage.SettingsStore { return s.settings }
|
||||
// Users implements storage.Database.
|
||||
func (s *Store) Users() storage.UserStore { return s.users }
|
||||
|
||||
// Rooms implements storage.Database.
|
||||
func (s *Store) Rooms() storage.RoomStore { return s.rooms }
|
||||
|
||||
// Migrate implements storage.Database.
|
||||
func (s *Store) Migrate(ctx context.Context) error {
|
||||
ms, err := s.migrations()
|
||||
|
||||
@@ -64,6 +64,9 @@ type Database interface {
|
||||
// Users is the userapi store: accounts, devices, tokens, account data.
|
||||
Users() UserStore
|
||||
|
||||
// Rooms is the roomserver store: rooms, events, room state, aliases.
|
||||
Rooms() RoomStore
|
||||
|
||||
// Close releases the connection pool.
|
||||
Close() error
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user