5 Commits

Include renames

Author	SHA1	Message	Date
tajniak81andClaude Opus 5.5	b6f3be6ac4	Web client (warp-web) with E2EE, and two sync fixes ... apps/web: a Matrix client for the browser on Vue 3, Tailwind and matrix-js-sdk, built on @warp/ui. Like the admin panel it is an optional, separate service: warp-web embeds the built app, serves /config.json and proxies only /_matrix/client and /_matrix/media to the homeserver (or, with -public-homeserver, lets the browser call it directly). Sign-in and sign-up with registration tokens, room list, timeline with back-pagination, text, images, video and files over authenticated media, local echo with sent/read/failed states, receipts, typing, invites, creating, joining and leaving rooms, room info, settings, phone layout. End-to-end encryption through the Rust crypto: encrypted messages and files, emoji verification, key backup setup and restore. apps/ui: WRoomItem, WMessageBubble and WComposer from the brand package, and the icons the client needs. syncapi fixes found with the client: - A room joined inside the sync window was sent as the join alone with limited: false, so clients thought they had the whole room. It is now read like an initial sync; a profile change is not mistaken for a join. - A live typing notification was reported on every sync, so every long poll returned at once while anybody typed. Typing now has a position in the sync token and is reported when it changes, including the empty list when the last person stops. Also: Dockerfile, a `web` compose profile, Makefile targets, a CI job with a proxy smoke test, docs. PLAN.md moves out of the repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-18 16:22:11 +02:00
tajniak81andClaude Opus 5.5	b507753172	Phase 8: push and notifications ... Implements PLAN.md Phase 8. internal/pushrules is the rule language: the spec's default ruleset, all six condition kinds, glob and word-boundary matching, intentional mentions switching off the legacy body rules, and rule ordering as Synapse has it. internal/pushserver keeps the rules, pushers and notifications and talks to push gateways. Push rules are the m.push_rules account data, written only by the push server; reads keep the user's own rules and the enabled/actions of each default and take the rest from the current defaults, so new default rules reach every user without a migration. PUT account_data/m.push_rules is refused with 405. The push server listens on the roomserver's emit stage and evaluates each event on its own worker for every local member and an invite's target. Notifications and per-room unread counts are stored in migration 0010; counts are a seventh sync token stream, so /sync reports unread_notifications and picks up count changes made after the event or by a receipt from another device. m.read and m.read.private receipts clear the counts; leaving a room clears it. Pushers (http kind only) push through a bounded sender pool with retries, honour event_id_only, drop push keys the gateway rejects, and go away with their device. GET /notifications pages the history. New config section push: enabled, include_content, gateway_timeout, notification_retention. Account data writes now wake the user's blocked syncs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-17 11:43:36 +02:00
tajniak81andClaude Opus 5.5	fc49f56339	Phase 7: E2EE support ... Implements PLAN.md Phase 7: the server half of end-to-end encryption. Devices upload identity, one-time and fallback keys; other devices query and claim them, across servers too, and every one-time key is handed out exactly once. Cross-signing, signature uploads, device list tracking and server-side key backup complete the set. internal/keyserver is the component. Keys are stored in canonical JSON so that re-uploading the same key is a byte comparison, and signatures uploaded later are kept apart from the key they sign and merged in on read, so a key's own JSON stays what its owner uploaded. Device keys must be for the caller's own device; cross-signing keys must be the uploader's, carry the right usage and, for self- and user-signing keys, be signed by the master key. Signatures are verified against the key as stored. Replacing a master key asks for the password; a first upload or an upload of the keys already stored does not, as the spec allows. A signature of another user's master key is shown only to the user who made it and never federated. Every write that changes what /keys/query answers moves the device list stream in the same transaction, one position per device, and wakes the syncs of everyone sharing a room. The stream is the sixth field of a sync token; older tokens still parse. The syncapi works out device_lists changed and left from the stream and from membership moves in the window, skipping invites and knocks, and reports device_one_time_keys_count and device_unused_fallback_key_types, the latter as [] when empty. GET /keys/changes uses the same logic. Over federation: /user/keys/query, /user/keys/claim and /user/devices/{userId}, and m.device_list_update and m.signing_key_update both ways. Remote keys are not cached; an update about a remote user sharing a room moves them in the stream so local clients re-query. Device display names are not sent to other servers. With federation off, remote users are reported in failures at once. Deleting a device deletes its keys, through a new userapi DeviceObserver. Key backup keeps the better copy of each session by the spec's rule, never reuses a deleted version's number, and answers writes to a stale version with M_WRONG_ROOM_KEYS_VERSION. Storage is migration 0008 for sqlite and postgres, with a PocketBase stub. The Complement target sets gain the key, device list, fallback, cross-signing and backup tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-17 10:34:08 +02:00
tajniak81andClaude Opus 5.5	6aaf554336	Phase 5: federation ... Implements PLAN.md Phase 5: the Server-Server API. Warp joins rooms on other servers and lets their users join its rooms, invites across servers with the invitee's server countersigning, rejects remote invites (locally if the inviter is unreachable), and exchanges events, typing, receipts and to-device messages with other homeservers — all behind the runtime switch, which now starts and stops real outbound traffic. The protocol lives in internal/federation with no knowledge of rooms: X-Matrix signing (fuzzed parser), the spec's discovery algorithm keeping the dial address and the TLS name apart, a signed client that refuses every request while the switch is off or the policy denies the destination, and a key ring that caches other servers' keys for at most a week, with optional notaries. The roomserver takes events from elsewhere through the spec's intake: signatures and hashes (a hash mismatch keeps the redacted form), the rules against the event's own auth events, against the state before it — with /get_missing_events for small gaps and /state_ids for wide ones — and against the current state, where a failure is a soft failure. Events held without their state are outliers: a column on roomserver_events, because the current state of a room joined over federation is almost all outliers, and excluded from every timeline query. Forks are resolved at the room's head. Membership across servers is make_join/send_join, make_leave/send_leave and countersigned invites, in both directions; m.room.server_acl is enforced on everything another server sends. History before a remote join is backfilled when /messages pages past it, stored at negative stream positions. internal/federationapi is the running component: the inbound endpoints, and an outbound queue that persists what it owes each destination, sends it in transactions of 50 PDUs and 100 EDUs, backs off from ten seconds to six hours, survives restarts, and drops events persisted while federation is off rather than delivering a backlog in one burst. Its Module is what the switch starts and stops. Also: a TLS listener for port 8448, trusted key servers, key validity and an outbound CA bundle in the config; per-destination health and backoff reset in the admin API; remote profiles, aliases and directories in the Client-Server API; a Complement federation profile. See docs/federation.md, including its known gaps (no partial-state joins, no key rotation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-16 17:14:20 +02:00
tajniak81andClaude Opus 5	d187ad0d17	Phase 4: sync ... Implements PLAN.md Phase 4: the read model. /sync, /messages and /context, the filters that narrow them, and the streams a response carries that are not room events — typing, receipts, presence and to-device messages. With this a client can be used against Warp, which is milestone M1. The one architectural choice is that the syncapi has no copy of the rooms: it reads them through the roomserver. Dendrite gives its syncapi its own tables and feeds them from the output stream, which is right for a design where the two can be separate processes, and Dendrite's could be. Warp is one process by decision, and there the copy buys nothing and costs the hardest class of bug a homeserver has — a read model that disagrees with the room, where a client is shown a message that was rejected and the only way to find out is to compare two tables. So the roomserver grew a read model (stream.go, roomsync.go, visibility.go) rather than exporting its store: reading an event means parsing it under its room's version, and a second copy of the version registry, the parser and the client projection would be a second place for them to disagree. syncapi: stream tokens naming a position in each of five independent streams, so a client that has seen receipt 7 but only event 4 can say so in one opaque string. A token with fewer positions than this build writes is accepted and the rest read as zero, which is what lets a token stored before a stream existed go on working. The PDU position comes from MAX(stream_ordering) and not from the counter row the other four use: the counter is advanced inside the writing transaction, so a token taken from it would name a position whose event no reader can see yet, and the event would be skipped for good once that transaction committed. Long polling goes through a notifier that wakes by room and by user rather than globally — an event wakes the listeners of its room, a membership event also wakes the user it names, so an invite reaches a client that is not in the room yet. A wake-up is only ever a hint: the listener re-reads the streams and decides for itself what changed, which is what makes it safe for the notifier to hold no state about what each client has seen. The listener registers before the read, so an event landing in between leaves the channel signalled and the sync reads again rather than blocking on a change it just missed. EventPersisted must not block, because it is called with the room's write lock still held. "Something happened" is not "a token moved". A stream can move without this user being concerned by it, and returning on every such move would turn a long poll back into a busy loop. The same trap caught presence twice over: a client says it is still online by syncing, so ?set_presence= makes every sync a presence write, and a stream that moved each time would wake every peer, which would sync, which would move it again. SetPresence therefore reports whether anything a peer would notice actually changed, and a write that only refreshes the last-active timestamp leaves the stream alone. Timeline windows are named by their two ends and a direction, Low exclusive and High inclusive in both directions, rather than by "from" and "to" as the wire format spells them. Each of those would otherwise be the low bound in one direction and the high bound in the other, which is exactly the off-by-one every pagination bug is made of; the handlers translate once and nothing below them thinks about it again. A sync's timeline is read backwards and reported forwards, because a client wants the newest events of the window it missed — a forward read with a limit hands it the oldest and then calls the rest a gap. visibility: history visibility is now a span rather than a yes or no, which is what Phase 3 deferred here. A user who has left a room is served the room as it was when they left — their state reads, their members and their timeline — instead of being refused, and is bounded above at their own membership event so a kicked client cannot go on streaming. Their span is found from their own member events, so a user who joined, left and rejoined sees their latest span. ?at= on /members is honoured for the same reason. What is left is that the span is computed from the room's current history_visibility where the spec judges each event by the one in force when it was sent; that is a state read per event and belongs with open question 1. Typing is held in memory and nowhere else: notifications expire in seconds, are worthless across a restart, and arrive one per keystroke-burst per room, so persisting them would be the busiest table in the database holding the least valuable data in it. Receipts only ever advance, compared on the position of the event acknowledged rather than on arrival, because two of a user's clients read at different speeds and the slower one must not drag the marker backwards. To-device queues are per device and drained by the next sync rather than by the one that delivered them, so a client that crashes in between gets the batch again — the right way round, since a lost key share leaves a message permanently undecryptable and a duplicate is ignored. A wildcard addressee is expanded when sent, not stored, because "every device" means the ones that existed then. Presence is off by default, which answers PLAN.md open question 3. It is the noisiest thing in Matrix and the one genuinely fan-out notification here; a private homeserver gains little from it. With it off the endpoints refuse rather than disappearing, because a client that gets a 404 from /presence concludes the server is too old to have it and stops asking. storage: one migration per driver adding the receipt, to-device, presence and filter tables with a counter row each, and a stream position on the existing account data table rather than a parallel one — there is exactly one current value per (user, room, type), and a second table would be a second thing to keep in step. Account data is reported in an event list but is not an event, so it gets its own response type: a room event with an empty sender and no timestamp would make a client decide what that meant. Deliberately missing, each named in docs/sync.md: notification counts (they need push rules, Phase 8), device lists (the keyserver, Phase 7), anything remote (Phase 5), threads (Phase 10). device_lists is sent as two empty lists because a client reads a missing one as malformed; the notification counts are left out entirely, because a zero there is a claim and an absent field is not. Tested through HTTP rather than against the builder, because what is worth pinning down is the response a client sees — which section a room lands in, whether a token handed back can be handed in again, whether paging terminates and loses nothing. The storage tests run against both SQL drivers; the notifier has its own tests for the concurrent case. A live server was driven end to end, which is what caught the two bugs the unit tests had missed: a backwards /messages page must be in reverse chronological order, and `start` must name where the page began rather than position zero when ?from= was omitted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-10-16 11:55:27 +02:00