Phase 10: hardening and ops
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Implements PLAN.md Phase 10. Simplified Sliding Sync (MSC4186) is served at the unstable org.matrix.simplified_msc3575 endpoint and advertised in /versions. Lists are sorted by activity and filtered by DM, invite, encryption, room type, space and tag; room subscriptions, required_state with wildcards, $ME and $LAZY, and the to_device, e2ee, account_data, receipts and typing extensions are supported. Per-connection state is kept in memory and an unknown pos answers M_UNKNOWN_POS. Threads and relations: m.relates_to is indexed on roomserver_events (migration 0012), /relations (with recursion) and /threads are served, and every event a timeline read returns carries its m.thread, m.replace and m.reference summaries in unsigned.m.relations. Notifications carry a thread ID (migration 0013): threaded receipts clear only their thread, and sync reports unread_thread_notifications when the filter asks. Spaces: /hierarchy walks a space breadth first, asking other servers about rooms this one is not in through the new federation hierarchy endpoint; room_summary describes one room. /search runs on FTS5 (sqlite, pocketbase) or tsvector (postgres), bounded to what each user may see, with context, profiles, state, grouping and paging. The PocketBase driver is real: the sqlite schema and store on an embedded PocketBase app's database, with media in its filesystem, local or S3 (database.pocketbase.s3). It runs the full storage test suite. Ops: OpenTelemetry tracing over OTLP/HTTP (tracing.*), with server spans per route, component spans, and trace propagation on federation, push and application service calls. New metrics for persisted events, waiting syncs, sliding sync connections and the database pool, with a Grafana dashboard. deploy/ adds a Compose setup (PostgreSQL, Prometheus, Grafana, Jaeger) and a Helm chart; cmd/warp-loadtest loads a server and reports request and delivery latencies. The load tester found a long-standing bug, fixed here: on a server where no account data had been written, every incremental /sync re-sent the push rules and returned at once, and an all-zero token was taken for an initial sync, so long polls became busy loops. The PostgreSQL migrations and search dialect compile but were not run against a live PostgreSQL in this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a94794b866
commit
487e0f410b
100 files changed
+10528
-893
No files matched your search
+42
-6
@@ -135,7 +135,7 @@ write to the rules, which is what makes the cache safe in one process.
|
||||
|
||||
| Table (migration `0010_push.sql`) | What |
|
||||
|---|---|
|
||||
| `push_notifications` | one row per event that notified a user: the event, its position in the room event stream, the actions, highlight, read |
|
||||
| `push_notifications` | one row per event that notified a user: the event, its position in the room event stream, its thread (`0013_thread_notifications.sql`), the actions, highlight, read |
|
||||
| `push_room_counts` | each user's unread and highlight counts in each room, kept current by the writes, stamped with a notification stream position |
|
||||
| `push_pushers` | pushers |
|
||||
| `push_sequences` | the notification stream's counter |
|
||||
@@ -165,10 +165,46 @@ then recounted from the rows that are still unread rather than decremented,
|
||||
so if anything ever put them out of step, the next receipt puts them right.
|
||||
`/read_markers` goes through the same path.
|
||||
|
||||
Notifications are not tracked per thread (threads are Phase 10). So a
|
||||
receipt counts when it is for the whole room or for its `main` timeline. A
|
||||
receipt inside a thread changes nothing; thread notifications are cleared
|
||||
when the main receipt moves past them.
|
||||
Each notification is filed under the thread its event is in: the root
|
||||
that its `m.relates_to` names when `rel_type` is `m.thread`, or the main
|
||||
timeline for anything else (a thread root itself included). A receipt
|
||||
reads the part of the room its `thread_id` names:
|
||||
|
||||
| `thread_id` | Reads, up to the receipt's event |
|
||||
|---|---|
|
||||
| none | the whole room, every thread included |
|
||||
| `main` | the main timeline only |
|
||||
| a thread root's event ID | that thread only |
|
||||
|
||||
So reading one thread leaves the main timeline and the other threads
|
||||
unread, and a client that knows nothing of threads sends unthreaded
|
||||
receipts and clears the room as it always did. `/read_markers` sends
|
||||
unthreaded receipts. A local user leaving or being banned reads the whole
|
||||
room.
|
||||
|
||||
### Per-thread counts
|
||||
|
||||
`push_room_counts` stays the room's total across every thread, which is
|
||||
what `unread_notifications` reports by default, what sliding sync reports,
|
||||
and what a push's badge counts. A sync whose filter sets
|
||||
`room.timeline.unread_thread_notifications` gets the counts split instead:
|
||||
`unread_notifications` counts the main timeline only, and
|
||||
`unread_thread_notifications` maps each thread root with something unread
|
||||
to its `notification_count` and `highlight_count`. A thread with nothing
|
||||
unread is left out of the map, and a room with no unread threads has no
|
||||
map, both of which the spec allows.
|
||||
|
||||
The per-thread counts are counted from the unread rows of
|
||||
`push_notifications` when a sync asks, rather than kept in a table like
|
||||
the room totals: a room has few unread notifications at a time, they are a
|
||||
range of an index, and a second table of counts would be one more thing
|
||||
every write and receipt had to keep in step. They need no stream of their
|
||||
own either: any change to a thread's counts is a change to its room's,
|
||||
which stamps the room and brings it down the next sync.
|
||||
|
||||
Notifications stored before migration 0013 are filed by the relation the
|
||||
roomserver projected onto their events in 0012; one whose event is gone
|
||||
stays on the main timeline.
|
||||
|
||||
Read notifications are kept for `push.notification_retention` (30 days by
|
||||
default) for `GET /notifications`, then pruned hourly. Unread ones are kept
|
||||
@@ -227,7 +263,7 @@ to again.
|
||||
|
||||
| Thing | Why |
|
||||
|---|---|
|
||||
| `unread_thread_notifications` | needs threads (Phase 10) |
|
||||
| Per-thread counts in sliding sync and in pushes | sliding sync reports each room's total, and a push's badge is the user's total across rooms; neither has anywhere the spec puts a per-thread count |
|
||||
| Skipping senders in `m.ignored_user_list` | ignored users' events still notify; a client hides them, but they count |
|
||||
| Email pushers | needs a mail server; see above |
|
||||
| Durable evaluation | the emit stage is in memory, so events persisted just before a crash are never evaluated. Their messages are still there; only the count and the push are missed. |
|
||||
|
||||
Reference in new issue
Block a user