3 Commits

Include renames

Author	SHA1	Message	Date
tajniak81andClaude Opus 5.5	487e0f410b	Phase 10: hardening and ops ... Implements PLAN.md Phase 10. Simplified Sliding Sync (MSC4186) is served at the unstable org.matrix.simplified_msc3575 endpoint and advertised in /versions. Lists are sorted by activity and filtered by DM, invite, encryption, room type, space and tag; room subscriptions, required_state with wildcards, $ME and $LAZY, and the to_device, e2ee, account_data, receipts and typing extensions are supported. Per-connection state is kept in memory and an unknown pos answers M_UNKNOWN_POS. Threads and relations: m.relates_to is indexed on roomserver_events (migration 0012), /relations (with recursion) and /threads are served, and every event a timeline read returns carries its m.thread, m.replace and m.reference summaries in unsigned.m.relations. Notifications carry a thread ID (migration 0013): threaded receipts clear only their thread, and sync reports unread_thread_notifications when the filter asks. Spaces: /hierarchy walks a space breadth first, asking other servers about rooms this one is not in through the new federation hierarchy endpoint; room_summary describes one room. /search runs on FTS5 (sqlite, pocketbase) or tsvector (postgres), bounded to what each user may see, with context, profiles, state, grouping and paging. The PocketBase driver is real: the sqlite schema and store on an embedded PocketBase app's database, with media in its filesystem, local or S3 (database.pocketbase.s3). It runs the full storage test suite. Ops: OpenTelemetry tracing over OTLP/HTTP (tracing.*), with server spans per route, component spans, and trace propagation on federation, push and application service calls. New metrics for persisted events, waiting syncs, sliding sync connections and the database pool, with a Grafana dashboard. deploy/ adds a Compose setup (PostgreSQL, Prometheus, Grafana, Jaeger) and a Helm chart; cmd/warp-loadtest loads a server and reports request and delivery latencies. The load tester found a long-standing bug, fixed here: on a server where no account data had been written, every incremental /sync re-sent the push rules and returned at once, and an all-zero token was taken for an initial sync, so long polls became busy loops. The PostgreSQL migrations and search dialect compile but were not run against a live PostgreSQL in this change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-17 20:21:31 +02:00
tajniak81andClaude Opus 5.5	6aaf554336	Phase 5: federation ... Implements PLAN.md Phase 5: the Server-Server API. Warp joins rooms on other servers and lets their users join its rooms, invites across servers with the invitee's server countersigning, rejects remote invites (locally if the inviter is unreachable), and exchanges events, typing, receipts and to-device messages with other homeservers — all behind the runtime switch, which now starts and stops real outbound traffic. The protocol lives in internal/federation with no knowledge of rooms: X-Matrix signing (fuzzed parser), the spec's discovery algorithm keeping the dial address and the TLS name apart, a signed client that refuses every request while the switch is off or the policy denies the destination, and a key ring that caches other servers' keys for at most a week, with optional notaries. The roomserver takes events from elsewhere through the spec's intake: signatures and hashes (a hash mismatch keeps the redacted form), the rules against the event's own auth events, against the state before it — with /get_missing_events for small gaps and /state_ids for wide ones — and against the current state, where a failure is a soft failure. Events held without their state are outliers: a column on roomserver_events, because the current state of a room joined over federation is almost all outliers, and excluded from every timeline query. Forks are resolved at the room's head. Membership across servers is make_join/send_join, make_leave/send_leave and countersigned invites, in both directions; m.room.server_acl is enforced on everything another server sends. History before a remote join is backfilled when /messages pages past it, stored at negative stream positions. internal/federationapi is the running component: the inbound endpoints, and an outbound queue that persists what it owes each destination, sends it in transactions of 50 PDUs and 100 EDUs, backs off from ten seconds to six hours, survives restarts, and drops events persisted while federation is off rather than delivering a backlog in one burst. Its Module is what the switch starts and stops. Also: a TLS listener for port 8448, trusted key servers, key validity and an outbound CA bundle in the config; per-destination health and backoff reset in the admin API; remote profiles, aliases and directories in the Client-Server API; a Complement federation profile. See docs/federation.md, including its known gaps (no partial-state joins, no key rotation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-16 17:14:20 +02:00
tajniak81andClaude Opus 5	c96f88e994	Phase 3: rooms, local only ... Implements PLAN.md Phase 3: the roomserver, the room half of storage, the Client-Server room endpoints and the admin room endpoints. A client can now create a room, invite, join, leave, kick, ban, send messages and state, redact, name a room with an alias and find one in the public directory. Nothing talks to another server; every operation that would need to is refused with a message that says so rather than a bare M_FORBIDDEN. roomserver: one input pipeline every event goes through, whatever sent it — validate, authorise, resolve, persist, emit. The rules are not reimplemented here; internal/matrix is pure library code and this is what connects it to a database and to a room's history. The Client-Server handlers do no authorisation of their own, deliberately: whether Alice may kick Bob is a question about the room's power levels, and the one place that is answered correctly is the authorisation rules inside the pipeline. A mutex per room serialises writers. Two events racing into one room would each read the same forward extremities, both become one, and fork the room for no reason — which would then need state resolution to put back together. State resolution is wired in anyway for the multi-extremity case. It is unreachable from outside in this phase, and the alternative is a pipeline that authorises against one arbitrary branch of a fork without saying so. Room versions 3 to 12 all work and are each tested end to end. Version 12 earns that coverage: its room ID is the create event's own ID, its create event carries no room_id, and the create event is excluded from every other event's auth_events — so the state auth.Check is given is the event's auth_events plus the create event, which that version identifies by room ID rather than citing. createRoom sends the spec's events in the spec's order, each authorised against what the ones before it produced. A failure part-way leaves an incomplete room, which is the lesser evil: the alternative is holding every event of a new room in memory and writing them in one transaction, making the pipeline that authorises them a special case. The one failure a second client can cause at any time, an alias clash, takes the room row back out. Redactions stay two decisions, as the spec keeps them. The rules let almost any redaction into the room; whether it applies to its target is a separate test, run inside the same transaction as the redaction itself so that a client holding an event ID can always read the stripped event. storage: events keyed by event ID rather than by a surrogate, since an event ID appears in prev_events, auth_events, state snapshots and client requests alike. Stream ordering comes from a counter row bumped inside the write transaction, because a PostgreSQL sequence is explicitly not transactional and a rolled-back write would leave a hole in the stream that /sync would have to be taught to skip. State snapshots are stored per event and deduplicated by content hash, so the table grows with state changes rather than with traffic — and that is what makes "the state after a forward extremity" answerable at all. The room's listing columns are denormalised onto its row because the public directory orders every published room by member count and pages the result, which cannot be done by reading each room's state in turn. Federation-disabled behaviour from PLAN.md section 3 is enforced here. Restricted joins are the exception that works: the rule needs a signature from the authorising user's server verified, and while federation is off that server is always this one. adminapi: room list and detail, purge, force-leave, and room and event counters on the dashboard. clientapi gains its own rate-limit buckets for createRoom and for joining, both of which write events rather than read them. Tested against the real event core and a real database rather than fakes: what is under test is whether an event this server signs is one the authorisation rules accept, and a fake of either would only prove that the pipeline calls what the fake expects. The storage tests run against both SQL drivers. docs/rooms.md describes the pipeline, the storage decisions and the known gaps — chief among them that history visibility is room-level rather than per-event, and that /sync still reports no rooms, both of which belong with the event stream in Phase 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-10-16 10:34:57 +02:00