Warp/docs/push.md

Push and notifications

server/internal/pushrules and server/internal/pushserver: PLAN.md Phase 8. Push rules decide which events notify a user. Every event that notifies somebody is recorded, counted per room as unread, and pushed to the push gateways of that user's devices.

The two packages split along a clear line:

Package	What
pushrules	The rule language: the rule model, the spec's default ruleset, conditions, glob matching, evaluation. Pure code, like internal/matrix: no storage, no HTTP.
pushserver	Everything around it: keeping each user's rules, the pushers, the worker that evaluates events, the counts, the push gateway client.

Push rules

Where they are kept

A user's push rules are their m.push_rules account data. That is where the spec has clients read them, and where every client already takes them from: the account data in its first /sync. A separate table would be a second copy, re-rendered into account data on every change.

The push server is the only writer. PUT /user/{id}/account_data/m.push_rules is refused with 405 M_BAD_JSON, as the spec requires. Changes go through /pushrules.

The stored document is the whole ruleset as a client sees it. When it is read back, only the parts the user owns are kept:

• the rules they created, in their order
• the enabled flag and actions of each default rule

Everything else comes from pushrules.Defaults: which default rules exist, and their conditions. So when a server upgrade adds a default rule (newer spec versions do), every user gets it on their next read. Nobody's account data has to be rewritten. A default rule the stored copy still has but the server has since dropped is ignored.

A user who has never changed a rule has nothing stored. They still have rules: /pushrules/, the account data endpoint and the initial sync all render the defaults for them.

Order

Within a kind, user rules come before the defaults of that kind. The exception is .m.rule.master, which stays ahead of everything, so switching it on still silences everything. This is also where Synapse puts them.

Kind	Order
override	.m.rule.master, user rules, the other defaults
content	user rules, .m.rule.contains_user_name
room, sender	user rules
underride	user rules, the defaults

A new rule with no before or after goes first among the user rules of its kind. A rule replaced with no before or after keeps its place and its enabled flag. before and after must name a user rule of the same kind; a default rule, or one that does not exist, is 404 M_NOT_FOUND.

The defaults

The spec's predefined rules, in the spec's order:

• override: master (off), suppress_notices, invite_for_me, member_event, is_user_mention, contains_display_name, is_room_mention, roomnotif, tombstone, reaction, server_acl, suppress_edits
• content: contains_user_name
• underride: call, encrypted_room_one_to_one, room_one_to_one, message, encrypted

An event with an m.mentions property says outright whom it mentions. For such an event the three legacy rules that guess from the body — contains_display_name, contains_user_name and roomnotif — are skipped, as the spec requires.

Evaluation

Condition	Matches when
event_match	the string at key matches the glob, case-insensitively. * is any run of characters and ? one. The whole value has to match, except for content.body, where the pattern has to match whole words somewhere in it. A number or boolean at the key never matches.
event_property_is	the value at key is exactly value: a string, an integer, a boolean or null
event_property_contains	the array at key contains value
contains_display_name	the body contains the user's display name in the room as whole words. The name is a literal, not a glob, so a user called * is not mentioned by everything.
room_member_count	the joined member count compares with is (2, ==2, <2, >2, <=2, >=2)
sender_notification_permission	the sender's power level is at least notifications.{key} in the room's power levels (default 50)
anything else	never: a rule with a condition the server does not know never matches

Dotted keys follow the spec's escaping: content.m\.mentions.user_ids reaches into a property whose name has a dot in it.

A word boundary is any character that is not a letter, a digit or an underscore, in any script. That is Python's \w, so it matches what Synapse does.

Compiled patterns are cached, up to 4096 of them. Patterns come from users, so the cache is bounded and simply starts over when full.

Notifications

The worker

The push server is a listener on the roomserver's emit stage, like the federation queue, and like the queue it does no work there. The emit stage runs while the room's write lock is held. Evaluating an event needs the room's members, their display names, the power levels, and a ruleset for each member, so the event goes on a list and a worker of the push server's own takes it from there, in order.

For each event the worker:

1. If it is a local user leaving or being banned, marks everything in the room read for them. A count left in a room they are no longer in would stay on their badge for good.
2. Reads the room (roomserver.PushRoomFor): joined members and their display names, the name and canonical alias, the power levels.
3. Evaluates the event for every local member except the sender, and for the target of an invite, who is not a member yet.
4. Records every event that notifies somebody, in one transaction that also moves each (user, room) count.
5. Wakes the syncs of the users it notified.
6. Pushes to their pushers' gateways, if push.enabled is on.

The room is read as it is now, not as it was at the event. The worker runs moments after the event lands, so the two only differ if somebody joined or renamed themselves in between. Reading the state at each event would cost a state resolution per message per member.

Each user's merged ruleset is cached in memory. The push server makes every write to the rules, which is what makes the cache safe in one process.

Counts

Table (migration 0010_push.sql)	What
push_notifications	one row per event that notified a user: the event, its position in the room event stream, its thread (0013_thread_notifications.sql), the actions, highlight, read
push_room_counts	each user's unread and highlight counts in each room, kept current by the writes, stamped with a notification stream position
push_pushers	pushers
push_sequences	the notification stream's counter

The counts are a stream of their own: the seventh position of a sync token. Every write that changes a user's counts in a room stamps the room with a new position in the same transaction. An incremental sync then asks which rooms' counts moved since its token, and reports them even when nothing else happened in them.

That stream is what makes an asynchronous worker safe. A sync can show a message before the worker has counted it. When the worker's write lands, it moves the stream and wakes the sync, and the next sync carries the corrected count. It also covers the case where counts move without any event: a receipt sent from another device.

unread_notifications is reported for every joined room in a response, not only the rooms whose counts moved. A client showing a new message wants the count next to it, whichever sync moved it.

Reading

A read receipt — m.read, or m.read.private, which is private from other users but not from the user's own badge — marks the user's notifications in the room read, up to and including the event it names. The counts are then recounted from the rows that are still unread rather than decremented, so if anything ever put them out of step, the next receipt puts them right. /read_markers goes through the same path.

Each notification is filed under the thread its event is in: the root that its m.relates_to names when rel_type is m.thread, or the main timeline for anything else (a thread root itself included). A receipt reads the part of the room its thread_id names:

thread_id	Reads, up to the receipt's event
none	the whole room, every thread included
main	the main timeline only
a thread root's event ID	that thread only

So reading one thread leaves the main timeline and the other threads unread, and a client that knows nothing of threads sends unthreaded receipts and clears the room as it always did. /read_markers sends unthreaded receipts. A local user leaving or being banned reads the whole room.

Per-thread counts

push_room_counts stays the room's total across every thread, which is what unread_notifications reports by default, what sliding sync reports, and what a push's badge counts. A sync whose filter sets room.timeline.unread_thread_notifications gets the counts split instead: unread_notifications counts the main timeline only, and unread_thread_notifications maps each thread root with something unread to its notification_count and highlight_count. A thread with nothing unread is left out of the map, and a room with no unread threads has no map, both of which the spec allows.

The per-thread counts are counted from the unread rows of push_notifications when a sync asks, rather than kept in a table like the room totals: a room has few unread notifications at a time, they are a range of an index, and a second table of counts would be one more thing every write and receipt had to keep in step. They need no stream of their own either: any change to a thread's counts is a change to its room's, which stamps the room and brings it down the next sync.

Notifications stored before migration 0013 are filed by the relation the roomserver projected onto their events in 0012; one whose event is gone stays on the main timeline.

Read notifications are kept for push.notification_retention (30 days by default) for GET /notifications, then pruned hourly. Unread ones are kept until they are read.

Pushers and the gateway

POST /pushers/set accepts the http kind only. An email pusher would need a mail server and templates, and a client asking for one is told 400 M_INVALID_PARAM rather than given a pusher that never fires.

The gateway URL must be absolute, must have the path /_matrix/push/v1/notify, and must not carry credentials. The spec asks for HTTPS; plain HTTP is also accepted, because the usual deployment runs the gateway (Sygnal, or a Warp-native one later — PLAN.md open question 6) next to the homeserver on a private network.

A pusher remembers the session that registered it. Logging that session out or deleting the device removes the pusher: the userapi's device observer now fans out to the keyserver and the push server. Registering the same app ID and push key for another user removes it from the previous user, unless append is set.

A push carries the event's IDs, type, sender, sender display name, room name and alias, the content (unless push.include_content is off), the user's total unread count, the matched rule's tweaks, and the pusher's data minus its url. prio is high for a highlight or anything that makes a sound, and low otherwise. A pusher with "format": "event_id_only" gets only the IDs, the priority and the counts; the app fetches the event itself.

Pushes go out from four senders fed by a queue of 1024, so a slow gateway delays only its own pushes. A server error, a 429 or a network failure is retried twice more, after 1 s and then 4 s. A 4xx is not retried. When the queue is full, a push is dropped and logged. A push is a best-effort wake-up: the notification is already counted, and the device sees it on its next sync whether or not the push arrived.

A push key the gateway lists in rejected is removed, so it is not pushed to again.

Endpoints

Endpoint	Notes
GET /pushrules/ · GET /pushrules/global/	The ruleset, under global or bare. The forms without the trailing slash are served too.
GET / PUT / DELETE /pushrules/global/{kind}/{ruleId}	PUT takes ?before= / ?after=. IDs starting with . are reserved (400). Deleting a default rule is 404.
GET / PUT /pushrules/global/{kind}/{ruleId}/enabled	default rules included
GET / PUT /pushrules/global/{kind}/{ruleId}/actions	default rules included
GET /pushers · POST /pushers/set	kind: null removes a pusher
GET /notifications	?from=, ?limit= (20 by default, at most 100), ?only=highlight. Newest first. A notification whose event has since been purged, or that the user can no longer see, is left out.
GET /user/{id}/account_data/m.push_rules	the rendered ruleset
PUT /user/{id}/account_data/m.push_rules	405 M_BAD_JSON

What is deliberately missing

Thing	Why
Per-thread counts in sliding sync and in pushes	sliding sync reports each room's total, and a push's badge is the user's total across rooms; neither has anywhere the spec puts a per-thread count
Skipping senders in m.ignored_user_list	ignored users' events still notify; a client hides them, but they count
Email pushers	needs a mail server; see above
Durable evaluation	the emit stage is in memory, so events persisted just before a crash are never evaluated. Their messages are still there; only the count and the push are missed.
A gateway URL deny-list	push gateways commonly sit on private networks, so private addresses cannot simply be refused the way remote media is. An operator who needs it should firewall outbound traffic.
Retries that survive a restart	a push waiting for its retry is lost on shutdown, for the same reason as above