Files
Warp/server/internal/storage/relations.go
T
tajniak81andClaude Opus 5.5 487e0f410b
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Phase 10: hardening and ops
Implements PLAN.md Phase 10.

Simplified Sliding Sync (MSC4186) is served at the unstable
org.matrix.simplified_msc3575 endpoint and advertised in /versions. Lists
are sorted by activity and filtered by DM, invite, encryption, room type,
space and tag; room subscriptions, required_state with wildcards, $ME and
$LAZY, and the to_device, e2ee, account_data, receipts and typing
extensions are supported. Per-connection state is kept in memory and an
unknown pos answers M_UNKNOWN_POS.

Threads and relations: m.relates_to is indexed on roomserver_events
(migration 0012), /relations (with recursion) and /threads are served, and
every event a timeline read returns carries its m.thread, m.replace and
m.reference summaries in unsigned.m.relations. Notifications carry a
thread ID (migration 0013): threaded receipts clear only their thread, and
sync reports unread_thread_notifications when the filter asks.

Spaces: /hierarchy walks a space breadth first, asking other servers about
rooms this one is not in through the new federation hierarchy endpoint;
room_summary describes one room. /search runs on FTS5 (sqlite, pocketbase)
or tsvector (postgres), bounded to what each user may see, with context,
profiles, state, grouping and paging.

The PocketBase driver is real: the sqlite schema and store on an embedded
PocketBase app's database, with media in its filesystem, local or S3
(database.pocketbase.s3). It runs the full storage test suite.

Ops: OpenTelemetry tracing over OTLP/HTTP (tracing.*), with server spans
per route, component spans, and trace propagation on federation, push and
application service calls. New metrics for persisted events, waiting
syncs, sliding sync connections and the database pool, with a Grafana
dashboard. deploy/ adds a Compose setup (PostgreSQL, Prometheus, Grafana,
Jaeger) and a Helm chart; cmd/warp-loadtest loads a server and reports
request and delivery latencies.

The load tester found a long-standing bug, fixed here: on a server where
no account data had been written, every incremental /sync re-sent the push
rules and returned at once, and an all-zero token was taken for an initial
sync, so long polls became busy loops.

The PostgreSQL migrations and search dialect compile but were not run
against a live PostgreSQL in this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 20:21:31 +02:00

190 lines
6.1 KiB
Go

package storage
import "context"
// The relation and search reads of the room store: threads, edits,
// reactions and references (an event's content.m.relates_to), and the
// full-text index /search is answered from.
//
// Both are projections of the event JSON that the roomserver works out and
// hands over with the event, the same as its type and state key: the store
// still never parses an event. They live in a file of their own because
// they are the Phase 10 additions to the room store, and because neither is
// about a room's state or its stream order — they are about which events
// point at which, and which events say what.
// Relation types the server aggregates. Every rel_type is recorded and can
// be listed through /relations; these are the ones whose summaries go into
// an event's unsigned.m.relations.
const (
RelThread = "m.thread"
RelReplace = "m.replace"
RelReference = "m.reference"
RelAnnotate = "m.annotation"
)
// RelationsRequest is one page of the events relating to some parents.
//
// The bounds are a TimelineRequest's: Low exclusive, High inclusive, zero
// unbounded at either end, positions possibly negative inside backfilled
// history.
type RelationsRequest struct {
RoomID string
// ParentIDs are the events whose children are read. It is one event
// for a plain /relations call and the event plus its descendants for a
// recursive one.
ParentIDs []string
// RelType and EventType narrow the children, when set.
RelType string
EventType string
Low int64
High int64
Direction TimelineDirection
// Limit caps the rows returned, and is required.
Limit int
}
// RelationAggregate is what one parent's children of one relation type
// come to, from one sender.
//
// It is per sender because two of the summaries need it: an edit only
// counts when it comes from the original's sender, and a thread reports
// whether the user asking took part. Summing over senders is then the
// roomserver's job.
type RelationAggregate struct {
ParentID string
RelType string
Sender string
// Count is how many children there are.
Count int
// LatestEventID is the newest of them, by stream order.
LatestEventID string
// LatestOrdering is its position.
LatestOrdering int64
}
// ThreadsRequest is one page of a room's threads, newest activity first.
type ThreadsRequest struct {
RoomID string
// ParticipatedBy, when set, keeps only the threads that user started
// or replied in.
ParticipatedBy string
// Before is an exclusive bound on a thread's latest activity: the page
// after one whose last thread was last active at Before. Zero is the
// first page.
Before int64
// High bounds what counts as activity, inclusive, for a user whose
// view of the room ends at their leave. Zero is unbounded.
High int64
// Limit caps the threads returned, and is required.
Limit int
}
// Thread is one thread of a room.
type Thread struct {
RootID string
// LatestOrdering is the position of the newest event in the thread,
// which is what the listing is ordered and paged by.
LatestOrdering int64
}
// SearchOrder is how /search orders its results.
type SearchOrder string
// The two orders the Client-Server API defines.
const (
SearchByRank SearchOrder = "rank"
SearchByRecent SearchOrder = "recent"
)
// SearchRange is the part of one room a user may search: the window of its
// stream their membership and the room's history visibility allow. The
// bounds are a TimelineRequest's.
type SearchRange struct {
RoomID string
Low int64
High int64
}
// SearchRequest is one page of a full-text search.
type SearchRequest struct {
// Term is what the user typed. The store turns it into its engine's
// query syntax; a term with no searchable words matches nothing.
Term string
// Rooms are where to look, each bounded to what the user may see.
Rooms []SearchRange
// Keys narrows which fields match: "content.body", "content.name",
// "content.topic". Empty means all three.
Keys []string
// Filter narrows by type and sender, as a timeline filter does.
Filter TimelineFilter
Order SearchOrder
// Offset skips results, for paging a ranked search. A search by
// recency pages by Before instead, which stays put as events arrive.
Offset int
// Before is an exclusive upper bound on the stream position, for
// paging a search by recency. Zero is the first page.
Before int64
// Limit caps the results returned, and is required.
Limit int
}
// SearchResult is one event a search found.
type SearchResult struct {
Event StoredEvent
// Rank is the engine's relevance score, higher is better. It is only
// comparable within one search.
Rank float64
}
// RelationStore is the part of the room store about relations and search.
// It is embedded in RoomStore; it is a separate interface only so that the
// two halves can be read apart.
type RelationStore interface {
// Relations reads one page of the children of the given parents, in
// stream order in the direction asked for.
Relations(ctx context.Context, req RelationsRequest) ([]StoredEvent, error)
// ChildEventIDs lists the direct children of the given parents,
// narrowed by relation and event type when those are set. It is how a
// recursive /relations walks down a level at a time.
ChildEventIDs(ctx context.Context, roomID string, parentIDs []string, relType, eventType string) ([]string, error)
// RelationAggregates summarises the thread, edit and reference
// children of the given parents, up to and including high (zero for
// no bound).
RelationAggregates(ctx context.Context, roomID string, parentIDs []string, high int64) ([]RelationAggregate, error)
// RelationEventIDs lists the children of one relation type of the
// given parents, oldest first, at most limit of them: the chunk of an
// m.reference summary.
RelationEventIDs(ctx context.Context, roomID string, parentIDs []string, relType string, high int64, limit int) (map[string][]string, error)
// Threads reads one page of a room's threads.
Threads(ctx context.Context, req ThreadsRequest) ([]Thread, error)
// Search reads one page of a full-text search and the total number of
// matches.
Search(ctx context.Context, req SearchRequest) ([]SearchResult, int, error)
}