4 Commits
Author SHA1 Message Date
tajniak81andClaude Opus 5.5 12550c4078 Mobile: calls in the background, screen sharing, read receipts per thread
- Calls keep going with the app in the background: a foreground service
  (CallService.kt, driven from lib/matrix/call_service.dart) declares the
  microphone, the camera while it's on and the screen while it's shared.
  Its notification (CallStyle) shows the call's length, opens the call
  and hangs up. Swiping the app away ends the call.
- Screen sharing on Android: consent, then media projection on the
  service, then LiveKit publishes a screen-share track. A shared screen
  replaces the sharer's camera on their tile, shown whole and unmirrored;
  stopping it from Android's controls stops the share.
- Read receipts per thread (MSC3771): an open thread sends receipts with
  thread_id, "Read" under replies comes from receipts in the thread, and
  threads with unread replies get a dot on "N replies" and in the list.

Not tested on a device yet. README updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:22:56 +02:00
tajniak81andClaude Opus 5.5 41f800ab0b Mobile: voice and video calls (MatrixRTC with LiveKit)
Calls the way the web client and Element Call do them, with the same
events, so the three can share a call. The famedly SDK's own group calls
use com.famedly.call.member events, which the others don't read, so this
is written against the matrix-js-sdk wire format instead (lib/matrix/
calls.dart):

- Membership: org.matrix.msc3401.call.member state, key
  _@user:server_DEVICE_m.call, with the legacy session fields the js-sdk
  sends (application m.call, call_id "", membershipID, expires,
  focus_active, foci_preferred). An empty delayed copy (MSC4140) is
  scheduled first and pushed back every 5 s; leaving sends the empty
  state and cancels it.
- Media: a LiveKit token from the JWT service (the call's, else the
  homeserver's rtc/transports or .well-known rtc_foci) for an OpenID
  token; livekit_client connects.
- Encrypted rooms: a random 16-byte key per device, sent to every other
  device in the call over encrypted to-device
  io.element.call.encryption_keys, keys from others applied per LiveKit
  identity (@user:server:DEVICE); per-participant keys, no ratcheting,
  HKDF, like the web client's livekit-client.
- Ringing: a call started in a room of two sends a ring notification
  (MSC4075); the other side gets an incoming-call card and an insistent
  ringtone notification (full screen on a locked phone), declines with
  MSC4310 or answers by joining. The caller gives up after a minute or on
  a decline.
- UI: voice and video buttons in the room header, a call screen (the
  other person large and you in the corner, or a grid; mute, camera, flip,
  leave), a Join banner and a room-list marker for calls going on.

Not tested end to end yet: it needs a LiveKit SFU and JWT service.

README: replies, reactions, threads, spaces, QR verification, sliding
sync and calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 08:43:19 +02:00
tajniak81andClaude Opus 5.5 0fc8ee809d Mobile: QR-code verification
When both sides support it, a verification offers QR codes first: this
device shows its code (in a white quiet zone, whatever the theme) and can
scan the other's with the camera (mobile_scanner; only codes that start
with MATRIX count). The side whose code was scanned confirms the other
saw it succeed. Compare emoji stays available, and is used directly when
either side has no QR support.

Adds the CAMERA permission (camera hardware not required).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 23:16:27 +02:00
tajniak81andClaude Opus 5.5 3356089c13 Android app (apps/mobile) with Flutter
apps/mobile is a Matrix client for Android on the famedly Matrix Dart SDK,
with vodozemac (compiled from Rust during the build) for end-to-end
encryption. It speaks the plain Client-Server API and uses the web
client's design tokens, fonts and symbol.

- Sign in by server name (.well-known), homeserver address or full user ID;
  sign up with the dummy and registration token stages. Sessions survive
  restarts: SQLite in private storage, credentials also in the Keystore,
  both excluded from backups and device transfers.
- Room list with unread and mention counts, invites, search and filters.
  Timeline with grouping, day separators, notices, history on scroll,
  links, images and files (authenticated media). Sending text, photos,
  camera shots and files, with Sending/Sent/Read/Not sent and Retry.
  Typing, receipts, deleting. New rooms and DMs, encrypted by default;
  joining by address; room info with members, invite, rename, topic,
  notification level, leave.
- Encryption: one-step setup behind a recovery key, verifying a new session
  with the recovery key (restoring the backup) or another session, emoji
  verification of people and sessions, reset. Undecryptable messages say
  why. Session list with verify and sign out.
- Notifications while running, and push through UnifiedPush (no Firebase):
  the distributor's endpoint becomes an event_id_only pusher, the gateway
  is discovered, and a push starts the app headless to fetch, decrypt and
  show the event.

Make targets: mobile (release APKs per ABI into bin/), mobile-run,
mobile-test. Kotlin incremental compilation is off (Windows, project and
pub cache on different drives), and Tink's JVM artifact is excluded in
favour of tink-android.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:45:06 +02:00