Desktop: notifications, tray, auto-update, installer; web: sliding sync, voice messages, notifications
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Admin panel (push) Canceled after 0s
CI / Web client (push) Canceled after 0s
CI / Desktop client (ubuntu-24.04) (push) Canceled after 0s
CI / Desktop client (windows-latest) (push) Canceled after 0s

Web client (also what the desktop app runs):

- Simplified Sliding Sync (MSC4186) when the homeserver advertises
  org.matrix.simplified_msc3575. src/matrix/sliding.ts answers the SDK's
  GET /sync with a sliding sync request and rewrites the response into
  the /sync shape, as the Android app does; the SDK's own sliding sync
  mode doesn't sort thread replies into threads. 20 rooms first, then
  every room (up to 1000); to_device, e2ee, account data, receipts and
  typing extensions; M_UNKNOWN_POS starts a new connection. The SDK's
  /members?at=<sync token> drops the token, which here is ours. Settings
  can switch back to /sync.
- Notifications for new messages and incoming calls while Warp isn't in
  front, by the account's push rules, evaluated after decryption. The
  browser's Notification API once turned on in Settings, or the desktop
  app's bridge (window.warpDesktop). One per room; a click opens it.
- Voice messages: record from the composer, sent as m.audio with
  MSC3245's voice marker and MSC1767's duration and waveform (encrypted
  in encrypted rooms); a waveform player in the timeline.
- @warp/ui: a slot for extra composer buttons; bell-off, play and pause
  icons.

Desktop app:

- System notifications through Wails' notifications service, fed by
  /_desktop/bridge.js (/_desktop/notify, JSON only); clicking one shows
  the window and opens the room.
- Tray icon with Open and Quit; closing the window hides it (a one-time
  notification says so), File > Quit ends it. The page title goes to
  the window and the tray tooltip, the unread count to the taskbar or
  Dock badge.
- Auto-update from a Wails update manifest at DESKTOP_UPDATE_URL: a quiet
  check a minute after start and every six hours, a notification when a
  newer version exists, Help > Check for updates opens the update window.
  Optional signature key (DESKTOP_UPDATE_KEY). Linux package installs
  leave updates to the package manager. build/manifest writes the
  manifest.
- Per-user NSIS installer (make desktop-windows-installer), the macOS
  bundle zipped for the updater, a desktop CI job, and a Desktop release
  workflow that builds all three platforms and the manifest.

Tested on Windows against a local homeserver: sliding sync with E2EE,
threads and a server restart; notifications (toast history), close to
tray, relaunch restoring the window; voice messages recorded, sent and
played, encrypted too; the update check finding a newer version. Not
tested: installing an update, the NSIS installer, CI, macOS and Linux.
Code signing needs certificates and isn't done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5.5 committed 2026-10-19 14:01:02 +02:00
1 parent 8ff2730676
commit 829b236efe
33 files changed
+2067 -51

No files matched your search

+9
View File
@@ -5,6 +5,7 @@ import { formatBytes } from '@/lib/format'
import { linkify } from '@/lib/linkify'
import { downloadMedia, mediaUrl, useMedia, type MediaSource } from '@/matrix/media'
import { mediaSource, type MessageItem } from '@/matrix/timeline'
import VoicePlayer from './VoicePlayer.vue'
// What goes inside a bubble: text with links, an image, a player or a file
// card. Message text is never rendered as HTML.
@@ -92,6 +93,14 @@ async function download() {
<span v-else class="flex h-full items-center justify-center"><WSpinner /></span>
</button>
<!-- Voice message -->
<VoicePlayer
v-else-if="item.msgtype === 'm.audio' && media?.mxc && media.voice"
:source="mediaSource(media)!"
:duration-ms="media.duration"
:waveform="media.voice.waveform"
/>
<!-- Video and audio -->
<div v-else-if="(item.msgtype === 'm.video' || item.msgtype === 'm.audio') && media?.mxc" class="flex flex-col gap-2">
<video v-if="playerUrl && item.msgtype === 'm.video'" :src="playerUrl" controls autoplay class="max-h-80 max-w-full rounded-md" />
+75 -2
View File
@@ -5,7 +5,8 @@ import { WBanner, WComposer, WIcon, WIconButton, WSpinner } from '@warp/ui'
import { drafts } from '@/lib/drafts'
import { errorMessage, formatBytes } from '@/lib/format'
import { client, cryptoError } from '@/matrix/client'
import { sendFile, sendText } from '@/matrix/send'
import { sendFile, sendText, sendVoice } from '@/matrix/send'
import { canRecord, formatDuration, VoiceRecorder } from '@/matrix/voice'
import type { MessageItem } from '@/matrix/timeline'
import { useChat } from '@/stores/chat'
import { useToasts } from '@/stores/toasts'
@@ -111,6 +112,58 @@ async function sendFiles(list: FileList | File[]) {
}
}
// Voice messages: the record button replaces typing until it is sent or
// thrown away.
const recordable = canRecord()
const recording = ref(false)
const elapsed = ref(0)
const levels = ref<number[]>([])
let recorder: VoiceRecorder | null = null
async function startRecording() {
if (recording.value || !canWrite.value) return
const rec = new VoiceRecorder()
rec.onTick = (ms, level) => {
elapsed.value = ms
levels.value = [...levels.value.slice(-47), level]
}
rec.onLimit = () => void stopRecording(true)
try {
await rec.start()
} catch (e) {
rec.cancel()
toasts.error(e instanceof DOMException && e.name === 'NotAllowedError' ? 'Allow the microphone to record a voice message.' : `Can’t record: ${errorMessage(e)}`)
return
}
recorder = rec
elapsed.value = 0
levels.value = []
recording.value = true
}
async function stopRecording(send: boolean) {
const rec = recorder
recorder = null
recording.value = false
if (!rec) return
if (!send) {
rec.cancel()
return
}
try {
const voice = await rec.stop()
if (voice.durationMs < 700) return
const to = target()
replyTo.value = null
const id = ++uploadId
uploads.value.push({ id, name: 'voice message', loaded: 0, total: 0 })
await sendVoice(client(), props.room, voice, to).finally(() => (uploads.value = uploads.value.filter((x) => x.id !== id)))
} catch (e) {
toasts.error(`Can’t send the voice message: ${errorMessage(e)}`)
}
}
onBeforeUnmount(() => recorder?.cancel())
function onPaste(e: ClipboardEvent) {
const files = e.clipboardData?.files
if (files && files.length) {
@@ -120,6 +173,11 @@ function onPaste(e: ClipboardEvent) {
}
function onKey(e: KeyboardEvent) {
if (e.key === 'Escape' && recording.value) {
e.stopPropagation()
void stopRecording(false)
return
}
if (e.key === 'Escape' && replyTo.value) {
e.stopPropagation()
replyTo.value = null
@@ -150,7 +208,22 @@ defineExpose({ sendFiles, canWrite, focus: () => composer.value?.focus() })
</p>
<WIconButton icon="close" label="Cancel reply" @click="replyTo = null" />
</div>
<WComposer ref="composer" v-model="text" :placeholder="placeholder" :encrypted="encrypted" @send="send" @attach="sendFiles" />
<div v-if="recording" class="wp-composer items-center" role="group" aria-label="Recording a voice message">
<WIconButton icon="trash" label="Discard the voice message" @click="stopRecording(false)" />
<span class="size-2.5 shrink-0 animate-pulse rounded-full bg-danger" aria-hidden="true" />
<span class="body tabular-nums">{{ formatDuration(elapsed) }}</span>
<div class="flex h-6 min-w-0 flex-1 items-center justify-end gap-[2px] overflow-hidden" aria-hidden="true">
<span v-for="(l, i) in levels" :key="i" class="w-[3px] shrink-0 rounded-full bg-ink-muted" :style="{ height: `${Math.max(12, Math.round(l * 100))}%` }" />
</div>
<button type="button" class="wp-composer-send" aria-label="Send the voice message" @click="stopRecording(true)">
<WIcon name="send" :size="20" />
</button>
</div>
<WComposer v-else ref="composer" v-model="text" :placeholder="placeholder" :encrypted="encrypted" @send="send" @attach="sendFiles">
<template v-if="recordable && !text.trim()" #actions>
<WIconButton icon="mic" label="Record a voice message" @click="startRecording" />
</template>
</WComposer>
</div>
</div>
</template>
+37 -1
View File
@@ -7,7 +7,9 @@ import MxcAvatar from '@/components/MxcAvatar.vue'
import SessionsDialog from '@/components/SessionsDialog.vue'
import { config } from '@/config'
import { errorMessage, localpart } from '@/lib/format'
import { cryptoError } from '@/matrix/client'
import { cryptoError, usingSlidingSync } from '@/matrix/client'
import { setSlidingEnabled, slidingEnabled } from '@/matrix/sliding'
import { useNotifications } from '@/stores/notifications'
import { useProfile } from '@/stores/profile'
import { useSecurity } from '@/stores/security'
import { useSession } from '@/stores/session'
@@ -21,6 +23,15 @@ const toasts = useToasts()
const router = useRouter()
const theme = useTheme()
const security = useSecurity()
const notifications = useNotifications()
// Sliding sync takes effect on the next start of the client.
const sliding = ref(slidingEnabled())
function toggleSliding() {
sliding.value = !sliding.value
setSlidingEnabled(sliding.value)
window.location.reload()
}
const name = ref('')
const savingName = ref(false)
@@ -137,6 +148,25 @@ const themes: { value: ThemeChoice; label: string; icon: IconName }[] = [
</div>
</section>
<section class="flex flex-col gap-2 border-t border-line pt-4">
<p class="body-strong m-0">Notifications</p>
<div class="flex items-start gap-3">
<WIcon :name="notifications.active ? 'bell' : 'bell-off'" :class="['mt-[2px] shrink-0', notifications.active ? 'text-teal-text' : 'text-ink-muted']" />
<div class="min-w-0 flex-1">
<p class="body m-0">{{ notifications.active ? 'Notifications are on' : 'Notifications are off' }}</p>
<p class="small m-0 text-ink-muted">
<template v-if="notifications.permission === 'unsupported'">This browser can’t show notifications.</template>
<template v-else-if="notifications.permission === 'denied'">The browser blocks notifications from this site. Allow them in its site settings.</template>
<template v-else>New messages and calls, when Warp isn’t in front. Your push rules decide which messages.</template>
</p>
</div>
<template v-if="notifications.permission !== 'unsupported' && notifications.permission !== 'denied'">
<WButton v-if="!notifications.active" size="sm" variant="primary" icon="bell" @click="notifications.enable()">Turn on</WButton>
<WButton v-else size="sm" variant="ghost" @click="notifications.enabled = false">Turn off</WButton>
</template>
</div>
</section>
<section class="flex flex-col gap-3 border-t border-line pt-4">
<p class="body-strong m-0">Encryption</p>
<WBanner v-if="cryptoError()" tone="warning" title="End-to-end encryption didn’t start">
@@ -183,6 +213,12 @@ const themes: { value: ThemeChoice; label: string; icon: IconName }[] = [
<p class="body-strong m-0">This session</p>
<p class="small m-0 text-ink-muted">Session ID <span class="mxid text-ink">{{ session.deviceId }}</span></p>
<p class="small m-0 text-ink-muted">Warp web {{ config.version }}</p>
<p class="small m-0 text-ink-muted">
Sync: {{ usingSlidingSync() ? 'Simplified Sliding Sync' : '/sync' }}
<button type="button" class="ml-1 font-semibold text-teal-text hover:underline" @click="toggleSliding">
{{ sliding ? 'Use /sync instead' : 'Use sliding sync when the server has it' }}
</button>
</p>
</section>
<EncryptionDialog v-model:open="encOpen" :mode="encMode" />
+115
View File
@@ -0,0 +1,115 @@
<script setup lang="ts">
import { computed, nextTick, onBeforeUnmount, ref } from 'vue'
import { WIcon, WSpinner } from '@warp/ui'
import { mediaUrl, type MediaSource } from '@/matrix/media'
import { formatDuration } from '@/matrix/voice'
// A voice message: play/pause, its waveform filling in as it plays (click
// to seek), and the time. The file loads on the first play.
const props = defineProps<{ source: MediaSource; durationMs?: number; waveform: number[] }>()
const bars = 40
const heights = computed(() => {
const w = props.waveform
if (!w.length) return Array.from({ length: bars }, () => 0.3)
return Array.from({ length: bars }, (_, i) => {
const v = w[Math.floor((i * w.length) / bars)] ?? 0
return Math.max(0.12, Math.min(1, v / 1024))
})
})
const audio = ref<HTMLAudioElement>()
const src = ref<string>()
const loading = ref(false)
const failed = ref(false)
const playing = ref(false)
const position = ref(0)
const length = ref(props.durationMs ? props.durationMs / 1000 : 0)
const progress = computed(() => (length.value ? Math.min(1, position.value / length.value) : 0))
const shownTime = computed(() => formatDuration((playing.value || position.value ? position.value : length.value) * 1000))
async function load(): Promise<boolean> {
if (src.value) return true
loading.value = true
failed.value = false
try {
src.value = await mediaUrl(props.source)
// Let the <audio> pick up the new src before it is played.
await nextTick()
return true
} catch {
failed.value = true
return false
} finally {
loading.value = false
}
}
async function toggle() {
if (playing.value) {
audio.value?.pause()
return
}
if (!(await load()) || !audio.value) return
await audio.value.play().catch(() => (failed.value = true))
}
async function seek(e: MouseEvent) {
const el = e.currentTarget as HTMLElement
const frac = Math.min(1, Math.max(0, (e.clientX - el.getBoundingClientRect().left) / el.clientWidth))
if (!(await load()) || !audio.value) return
const total = Number.isFinite(audio.value.duration) ? audio.value.duration : length.value
audio.value.currentTime = frac * total
position.value = audio.value.currentTime
}
function onMeta() {
const d = audio.value?.duration
// WebM recordings often report Infinity until played through.
if (d && Number.isFinite(d)) length.value = d
}
function onEnded() {
playing.value = false
position.value = 0
}
onBeforeUnmount(() => audio.value?.pause())
</script>
<template>
<div class="flex min-w-[220px] items-center gap-3">
<button
type="button"
class="flex size-9 shrink-0 items-center justify-center rounded-full border border-current/40 hover:bg-current/10"
:aria-label="playing ? 'Pause voice message' : 'Play voice message'"
@click="toggle"
>
<WSpinner v-if="loading" :size="16" />
<WIcon v-else :name="playing ? 'pause' : 'play'" :size="18" />
</button>
<div class="flex h-8 flex-1 cursor-pointer items-center gap-[2px]" role="presentation" @click="seek">
<span
v-for="(h, i) in heights"
:key="i"
:class="['w-[3px] shrink-0 rounded-full bg-current', i / bars < progress ? 'opacity-100' : 'opacity-35']"
:style="{ height: `${Math.round(h * 100)}%` }"
/>
</div>
<span class="caption w-10 shrink-0 text-right opacity-75 tabular-nums">{{ failed ? '—' : shownTime }}</span>
<audio
ref="audio"
:src="src"
preload="auto"
class="hidden"
@play="playing = true"
@pause="playing = false"
@ended="onEnded"
@timeupdate="position = audio?.currentTime ?? 0"
@loadedmetadata="onMeta"
@durationchange="onMeta"
/>
</div>
</template>
+4
View File
@@ -15,8 +15,10 @@ import SpacePicker from '@/components/SpacePicker.vue'
import StartChatDialog from '@/components/StartChatDialog.vue'
import VerificationDialog from '@/components/VerificationDialog.vue'
import { formatListTime, localpart } from '@/lib/format'
import { client } from '@/matrix/client'
import { useCall } from '@/stores/call'
import { useChat } from '@/stores/chat'
import { useNotifications } from '@/stores/notifications'
import { useProfile } from '@/stores/profile'
import { useSecurity } from '@/stores/security'
import { useSession } from '@/stores/session'
@@ -26,6 +28,7 @@ const chat = useChat()
const call = useCall()
const profile = useProfile()
const security = useSecurity()
const notifications = useNotifications()
const route = useRoute()
const router = useRouter()
@@ -79,6 +82,7 @@ async function start() {
chat.attach()
security.attach()
call.attach()
notifications.attach(client())
profile.load()
}
} catch (e) {
+23
View File
@@ -1,6 +1,7 @@
import { createClient, PendingEventOrdering, type MatrixClient } from 'matrix-js-sdk'
import { config } from '@/config'
import { bindCrypto, cryptoCallbacks, cryptoDatabasePrefix } from './crypto'
import { slidingEnabled, slidingFeature, slidingFetch } from './sliding'
// The one MatrixClient of a signed-in session. It is kept outside Vue's
// reactivity on purpose: the SDK's objects mutate themselves and emit
@@ -35,6 +36,24 @@ function decodeKey(b64: string): Uint8Array {
let current: MatrixClient | null = null
let cryptoFailure = ''
let sliding = false
/** Whether this session syncs with Simplified Sliding Sync (see ./sliding.ts). */
export function usingSlidingSync(): boolean {
return sliding
}
/** Whether the homeserver has Simplified Sliding Sync; false when it can't be asked. */
async function serverHasSliding(baseUrl: string): Promise<boolean> {
try {
const res = await fetch(`${baseUrl.replace(/\/+$/, '')}/_matrix/client/versions`, { credentials: 'omit' })
if (!res.ok) return false
const body = (await res.json()) as { unstable_features?: Record<string, boolean> }
return body.unstable_features?.[slidingFeature] === true
} catch {
return false
}
}
/** The running client. Only call this while signed in. */
export function client(): MatrixClient {
@@ -61,6 +80,8 @@ export function anonymousClient(): MatrixClient {
export async function startClient(creds: Credentials): Promise<MatrixClient> {
stopClient()
sliding = slidingEnabled() && (await serverHasSliding(creds.baseUrl))
console.info(sliding ? 'sync: Simplified Sliding Sync' : 'sync: /sync')
const c = createClient({
baseUrl: creds.baseUrl,
userId: creds.userId,
@@ -75,6 +96,7 @@ export async function startClient(creds: Credentials): Promise<MatrixClient> {
// Emoji (SAS), and QR codes both ways: shown here, or scanned with
// the camera.
verificationMethods: ['m.sas.v1', 'm.qr_code.show.v1', 'm.qr_code.scan.v1', 'm.reciprocate.v1'],
fetchFn: sliding ? slidingFetch(creds.userId) : undefined,
})
current = c
// End-to-end encryption: the Rust crypto stack (matrix-sdk-crypto-wasm),
@@ -110,6 +132,7 @@ export async function startClient(creds: Credentials): Promise<MatrixClient> {
export function stopClient() {
stopHeartbeat()
sliding = false
if (!current) return
bindCrypto(null)
current.stopClient()
+39 -10
View File
@@ -1,6 +1,7 @@
import { EventType, MsgType, RelationType, type MatrixClient, type MatrixEvent, type Room } from 'matrix-js-sdk'
import type { RoomMessageEventContent } from 'matrix-js-sdk/lib/types'
import { encryptAttachment } from './attachments'
import { extensionFor, type Recording } from './voice'
// Sending: text, and files uploaded to the media repository first. Your own
// message appears at once as a local echo with status "sending"; the SDK
@@ -134,16 +135,7 @@ export async function sendFile(
Object.assign(info, { thumbnail_url: up.content_uri, thumbnail_info: thumbInfo })
}
}
let source: { url: string } | { file: object }
if (encrypted) {
const { data, file: keys } = await encryptAttachment(await file.arrayBuffer())
// The name and type stay inside the encrypted message.
const upload = await c.uploadContent(new Blob([data]), { type: 'application/octet-stream', includeFilename: false, progressHandler })
source = { file: { ...keys, url: upload.content_uri } }
} else {
const upload = await c.uploadContent(file, { name: file.name, type: file.type || 'application/octet-stream', progressHandler })
source = { url: upload.content_uri }
}
const source = await upload(c, encrypted, file, file.name, progressHandler)
const content = {
msgtype,
body: file.name,
@@ -153,3 +145,40 @@ export async function sendFile(
} as RoomMessageEventContent
return c.sendMessage(room.roomId, target.threadId ?? null, withTarget(content, target))
}
/** Uploads a file, encrypted first in an encrypted room: the message's url, or its file. */
async function upload(
c: MatrixClient,
encrypted: boolean,
blob: Blob,
name: string,
progressHandler?: (p: { loaded: number; total: number }) => void,
): Promise<{ url: string } | { file: object }> {
if (encrypted) {
const { data, file: keys } = await encryptAttachment(await blob.arrayBuffer())
// The name and type stay inside the encrypted message.
const up = await c.uploadContent(new Blob([data]), { type: 'application/octet-stream', includeFilename: false, progressHandler })
return { file: { ...keys, url: up.content_uri } }
}
const up = await c.uploadContent(blob, { name, type: blob.type || 'application/octet-stream', progressHandler })
return { url: up.content_uri }
}
/**
* Sends a voice message: m.audio marked as voice (MSC3245), with its
* duration and waveform (MSC1767) so it draws before it is downloaded.
*/
export async function sendVoice(c: MatrixClient, room: Room, rec: Recording, target: SendTarget = {}) {
const filename = `Voice message.${extensionFor(rec.mimetype)}`
const source = await upload(c, room.hasEncryptionStateEvent(), rec.blob, filename)
const content = {
msgtype: MsgType.Audio,
body: 'Voice message',
filename,
...source,
info: { mimetype: rec.mimetype, size: rec.blob.size, duration: rec.durationMs },
'org.matrix.msc1767.audio': { duration: rec.durationMs, waveform: rec.waveform },
'org.matrix.msc3245.voice': {},
} as unknown as RoomMessageEventContent
return c.sendMessage(room.roomId, target.threadId ?? null, withTarget(content, target))
}
+297
View File
@@ -0,0 +1,297 @@
// Simplified Sliding Sync (MSC4186), underneath matrix-js-sdk's /sync.
//
// The SDK has a sliding sync mode of its own, but it doesn't sort thread
// replies into their threads, which the room and thread views rely on.
// Instead, the client's fetch answers the SDK's GET /sync with a sliding
// sync request and rewrites the response into the /sync shape. The SDK's
// sync loop, threads, decryption and receipts then work unchanged. The
// Android app does the same (apps/mobile/lib/matrix/sliding_sync.dart).
//
// What it buys is the first sync. A classic initial sync sends every room
// with its full state before anything can be drawn; here the first request
// asks for the 20 most recent rooms with a handful of state events each,
// and the next requests widen the window to every room.
//
// The SDK's sync token becomes `ss1|<pos>|<to-device since>`. None, or a
// classic one, starts a new connection; so does M_UNKNOWN_POS, the server
// saying it has forgotten the connection (after a restart, or idle time).
export const slidingFeature = 'org.matrix.simplified_msc3575'
const firstWindow = 20
const maxWindow = 1000
const connId = 'warp-web'
const tokenPrefix = 'ss1|'
/** The state each room is sent with: what the screens draw, what encryption needs, and calls. */
export const requiredState: string[][] = [
['m.room.create', ''],
['m.room.name', ''],
['m.room.avatar', ''],
['m.room.topic', ''],
['m.room.canonical_alias', ''],
['m.room.encryption', ''],
['m.room.power_levels', ''],
['m.room.join_rules', ''],
['m.room.history_visibility', ''],
['m.room.guest_access', ''],
['m.room.tombstone', ''],
['m.room.pinned_events', ''],
['m.space.child', '*'],
['m.space.parent', '*'],
['m.room.member', '$LAZY'],
['m.room.member', '$ME'],
['org.matrix.msc3401.call.member', '*'],
]
export interface SlidingToken {
pos?: string
toDeviceSince?: string
}
export function parseToken(token: string | null | undefined): SlidingToken {
if (!token?.startsWith(tokenPrefix)) return {}
const [pos, toDevice] = token.slice(tokenPrefix.length).split('|')
return { pos: pos || undefined, toDeviceSince: toDevice || undefined }
}
export function formatToken(t: SlidingToken): string {
return `${tokenPrefix}${t.pos ?? ''}|${t.toDeviceSince ?? ''}`
}
/** The request body: rooms and direct chats over the same window, and the extensions encryption needs. */
export function slidingRequest(window: number, toDeviceSince?: string): Record<string, unknown> {
const ranges = [[0, window - 1]]
return {
conn_id: connId,
lists: {
rooms: { ranges, timeline_limit: 10, required_state: requiredState, filters: { is_dm: false } },
// A direct chat's two members come with it, so its name comes out right.
dms: { ranges, timeline_limit: 10, required_state: [...requiredState, ['m.room.member', '*']], filters: { is_dm: true } },
},
extensions: {
to_device: { enabled: true, ...(toDeviceSince ? { since: toDeviceSince } : {}) },
e2ee: { enabled: true },
account_data: { enabled: true },
receipts: { enabled: true },
typing: { enabled: true },
},
}
}
type Json = Record<string, unknown>
const isObj = (v: unknown): v is Json => typeof v === 'object' && v !== null && !Array.isArray(v)
const list = (v: unknown): unknown[] => (Array.isArray(v) ? v : [])
/** The user's membership as of the last of the events about them. */
function ownMembership(events: unknown[], userId: string): string | undefined {
let m: string | undefined
for (const e of events) {
if (isObj(e) && e.type === 'm.room.member' && e.state_key === userId && isObj(e.content) && typeof e.content.membership === 'string') {
m = e.content.membership
}
}
return m
}
/**
* Rewrites a sliding sync response into the /sync response with the same
* news. A room with invite_state is an invite, one with knock_state a
* knock. Any other room is joined, unless the user's own latest membership
* in it is a leave or ban: sliding sync keeps rooms the user left while the
* connection was open in its lists, which is how a client hears they're gone.
*/
export function slidingToSync(response: Json, userId: string, nextBatch: string): Json {
const join: Record<string, Json> = {}
const invite: Json = {}
const knock: Json = {}
const leave: Record<string, Json> = {}
const joined = (roomId: string) => (join[roomId] ??= {})
const rooms = isObj(response.rooms) ? response.rooms : {}
for (const [roomId, room] of Object.entries(rooms)) {
if (!isObj(room)) continue
if (Array.isArray(room.invite_state)) {
invite[roomId] = { invite_state: { events: room.invite_state } }
continue
}
if (Array.isArray(room.knock_state)) {
knock[roomId] = { knock_state: { events: room.knock_state } }
continue
}
const state = list(room.required_state)
const timeline = list(room.timeline)
const timelineJson: Json = { events: timeline, limited: room.limited === true }
if (typeof room.prev_batch === 'string') timelineJson.prev_batch = room.prev_batch
const own = ownMembership([...state, ...timeline], userId)
if (own === 'leave' || own === 'ban') {
leave[roomId] = { state: { events: state }, timeline: timelineJson }
continue
}
const summary: Json = {}
if (Array.isArray(room.heroes)) {
summary['m.heroes'] = room.heroes.filter((h) => isObj(h) && typeof h.user_id === 'string').map((h) => (h as Json).user_id)
}
if (typeof room.joined_count === 'number') summary['m.joined_member_count'] = room.joined_count
if (typeof room.invited_count === 'number') summary['m.invited_member_count'] = room.invited_count
Object.assign(joined(roomId), {
state: { events: state },
timeline: timelineJson,
summary,
unread_notifications: {
notification_count: typeof room.notification_count === 'number' ? room.notification_count : 0,
highlight_count: typeof room.highlight_count === 'number' ? room.highlight_count : 0,
},
})
}
const out: Json = { next_batch: nextBatch }
const ext = isObj(response.extensions) ? response.extensions : {}
// Typing and receipts are per-room ephemeral events.
for (const name of ['typing', 'receipts']) {
const e = ext[name]
const byRoom = isObj(e) && isObj(e.rooms) ? e.rooms : {}
for (const [roomId, ev] of Object.entries(byRoom)) {
// Sent as an invite, knock or leave: no ephemeral events there.
if (!join[roomId] && rooms[roomId] !== undefined) continue
const r = joined(roomId)
const eph = (r.ephemeral ??= { events: [] }) as { events: unknown[] }
eph.events.push(ev)
}
}
const accountData = ext.account_data
if (isObj(accountData)) {
if (Array.isArray(accountData.global)) out.account_data = { events: accountData.global }
if (isObj(accountData.rooms)) {
for (const [roomId, events] of Object.entries(accountData.rooms)) {
if (leave[roomId]) leave[roomId].account_data = { events }
else if (!(roomId in invite) && !(roomId in knock)) joined(roomId).account_data = { events }
}
}
}
const toDevice = ext.to_device
if (isObj(toDevice) && Array.isArray(toDevice.events)) out.to_device = { events: toDevice.events }
const e2ee = ext.e2ee
if (isObj(e2ee)) {
if (isObj(e2ee.device_lists)) out.device_lists = e2ee.device_lists
if (isObj(e2ee.device_one_time_keys_count)) out.device_one_time_keys_count = e2ee.device_one_time_keys_count
if (Array.isArray(e2ee.device_unused_fallback_key_types)) out.device_unused_fallback_key_types = e2ee.device_unused_fallback_key_types
}
const anyRooms = Object.keys(join).length + Object.keys(invite).length + Object.keys(knock).length + Object.keys(leave).length
if (anyRooms) {
const r: Json = {}
if (Object.keys(join).length) r.join = join
if (Object.keys(invite).length) r.invite = invite
if (Object.keys(knock).length) r.knock = knock
if (Object.keys(leave).length) r.leave = leave
out.rooms = r
}
return out
}
/**
* A fetch for the SDK that answers GET /sync with sliding sync, and passes
* every other request through.
*/
export function slidingFetch(userId: string, base: typeof fetch = globalThis.fetch.bind(globalThis)): typeof fetch {
let window = firstWindow
async function slide(syncUrl: URL, init: RequestInit | undefined, token: SlidingToken): Promise<Response> {
const url = new URL(syncUrl.href.replace(/\/v3\/sync(\?.*)?$/, `/unstable/${slidingFeature}/sync`))
if (token.pos) {
url.searchParams.set('pos', token.pos)
// A new connection answers at once; only an established one waits.
const timeout = syncUrl.searchParams.get('timeout')
if (timeout) url.searchParams.set('timeout', timeout)
}
const presence = syncUrl.searchParams.get('set_presence')
if (presence) url.searchParams.set('set_presence', presence)
const headers = new Headers(init?.headers)
headers.set('Content-Type', 'application/json')
return base(url, {
method: 'POST',
headers,
body: JSON.stringify(slidingRequest(window, token.toDeviceSince)),
signal: init?.signal,
credentials: init?.credentials,
})
}
return async (input, init) => {
const href = input instanceof Request ? input.url : String(input)
const method = (init?.method ?? (input instanceof Request ? input.method : 'GET')).toUpperCase()
let syncUrl: URL
try {
syncUrl = new URL(href)
} catch {
return base(input, init)
}
// The SDK loads a room's members "at" its sync token, which here is
// ours, not the server's: without it the server answers as of now.
const at = syncUrl.searchParams.get('at')
if (at?.startsWith(tokenPrefix)) {
syncUrl.searchParams.delete('at')
return base(syncUrl, init)
}
if (method !== 'GET' || !syncUrl.pathname.endsWith('/_matrix/client/v3/sync')) return base(input, init)
let token = parseToken(syncUrl.searchParams.get('since'))
let res = await slide(syncUrl, init, token)
if (res.status === 400) {
const err = await res.clone().json().catch(() => null)
if (isObj(err) && err.errcode === 'M_UNKNOWN_POS') {
console.info('sliding sync: the connection expired; starting a new one')
window = firstWindow
token = { toDeviceSince: token.toDeviceSince }
res = await slide(syncUrl, init, token)
}
}
// Errors go to the SDK as they are: it knows M_UNKNOWN_TOKEN and backs off on the rest.
if (!res.ok) return res
const body = (await res.json()) as Json
// Widen the window to every room once the first ones are drawn.
if (isObj(body.lists)) {
let count = 0
for (const l of Object.values(body.lists)) if (isObj(l) && typeof l.count === 'number') count = Math.max(count, l.count)
if (count > window) window = Math.min(count, maxWindow)
}
const toDevice = isObj(body.extensions) && isObj(body.extensions.to_device) ? body.extensions.to_device : {}
const next: SlidingToken = {
pos: typeof body.pos === 'string' ? body.pos : undefined,
toDeviceSince: typeof toDevice.next_batch === 'string' ? toDevice.next_batch : token.toDeviceSince,
}
return new Response(JSON.stringify(slidingToSync(body, userId, formatToken(next))), {
status: 200,
headers: { 'Content-Type': 'application/json' },
})
}
}
// Sliding sync is used when the homeserver has it, unless turned off in
// Settings (for a server whose implementation misbehaves).
const offKey = 'warp.web.slidingSync'
export function slidingEnabled(): boolean {
try {
return localStorage.getItem(offKey) !== 'off'
} catch {
return true
}
}
export function setSlidingEnabled(on: boolean) {
try {
if (on) localStorage.removeItem(offKey)
else localStorage.setItem(offKey, 'off')
} catch {
// Not kept: the default (on) holds.
}
}
+12 -1
View File
@@ -37,6 +37,10 @@ export interface MediaInfo {
w?: number
h?: number
filename: string
/** Audio and video length, in ms. */
duration?: number
/** Set on a voice message (MSC3245): its waveform, 0–1024 per point. */
voice?: { waveform: number[] }
}
/** The message a reply answers, quoted above it. */
@@ -315,8 +319,13 @@ function mediaOf(content: RoomMessageEventContent): MediaInfo | undefined {
thumbnail_url?: string
thumbnail_file?: EncryptedFile
thumbnail_info?: { mimetype?: string }
duration?: number
}
'org.matrix.msc1767.audio'?: { duration?: number; waveform?: unknown }
'org.matrix.msc3245.voice'?: object
}
const audio = c['org.matrix.msc1767.audio']
const waveform = Array.isArray(audio?.waveform) ? audio.waveform.filter((n): n is number => typeof n === 'number') : []
return {
mxc: c.url ?? c.file?.url,
thumbMxc: c.info?.thumbnail_url ?? c.info?.thumbnail_file?.url,
@@ -328,6 +337,8 @@ function mediaOf(content: RoomMessageEventContent): MediaInfo | undefined {
w: c.info?.w,
h: c.info?.h,
filename: c.filename || c.body || 'file',
duration: typeof c.info?.duration === 'number' ? c.info.duration : typeof audio?.duration === 'number' ? audio.duration : undefined,
voice: msgtype === MsgType.Audio && c['org.matrix.msc3245.voice'] ? { waveform } : undefined,
}
}
@@ -388,7 +399,7 @@ export function quoteText(ev: MatrixEvent): string {
case MsgType.Video:
return 'Video'
case MsgType.Audio:
return 'Audio'
return content['org.matrix.msc3245.voice'] ? 'Voice message' : 'Audio'
case MsgType.File:
return `File: ${content.body ?? ''}`
}
+135
View File
@@ -0,0 +1,135 @@
// Recording voice messages: the microphone through MediaRecorder, and a
// waveform sampled while it records. A voice message is an m.audio with
// MSC3245's voice marker and MSC1767's duration and waveform, as Element
// and the other clients send it.
export interface Recording {
blob: Blob
/** Without codec parameters: what the message's info.mimetype says. */
mimetype: string
durationMs: number
/** 0–1024 per point, as MSC3246 has it. */
waveform: number[]
}
// Opus in whichever container the engine records: Ogg (Firefox), WebM
// (Chromium, WebView2) or MP4 (Safari, WKWebView).
const candidates = ['audio/ogg;codecs=opus', 'audio/webm;codecs=opus', 'audio/mp4', 'audio/webm']
export function canRecord(): boolean {
return typeof MediaRecorder !== 'undefined' && Boolean(navigator.mediaDevices?.getUserMedia)
}
/** How many points a sent waveform has. */
const waveformPoints = 100
/** The longest voice message; recording stops there. */
export const maxRecordingMs = 10 * 60_000
export class VoiceRecorder {
private stream?: MediaStream
private recorder?: MediaRecorder
private audio?: AudioContext
private analyser?: AnalyserNode
private chunks: Blob[] = []
private samples: number[] = []
private timer = 0
private started = 0
/** Called about ten times a second with the elapsed time and the current level (0–1). */
onTick?: (elapsedMs: number, level: number) => void
/** Called when recording stops by itself at maxRecordingMs. */
onLimit?: () => void
async start() {
this.stream = await navigator.mediaDevices.getUserMedia({ audio: { echoCancellation: true, noiseSuppression: true } })
const mimeType = candidates.find((t) => MediaRecorder.isTypeSupported(t))
this.recorder = new MediaRecorder(this.stream, mimeType ? { mimeType, audioBitsPerSecond: 48_000 } : undefined)
this.chunks = []
this.samples = []
this.recorder.ondataavailable = (e) => e.data.size && this.chunks.push(e.data)
this.audio = new AudioContext()
this.analyser = this.audio.createAnalyser()
this.analyser.fftSize = 1024
this.audio.createMediaStreamSource(this.stream).connect(this.analyser)
const buf = new Float32Array(this.analyser.fftSize)
this.recorder.start(1000)
this.started = performance.now()
this.timer = window.setInterval(() => {
this.analyser!.getFloatTimeDomainData(buf)
let sum = 0
for (const v of buf) sum += v * v
// RMS, lifted so speech fills the range.
const level = Math.min(1, Math.sqrt(sum / buf.length) * 4)
this.samples.push(level)
const elapsed = performance.now() - this.started
this.onTick?.(elapsed, level)
if (elapsed >= maxRecordingMs) this.onLimit?.()
}, 100)
}
/** Stops and returns the recording. */
stop(): Promise<Recording> {
const rec = this.recorder
if (!rec) return Promise.reject(new Error('Not recording'))
const durationMs = Math.round(performance.now() - this.started)
return new Promise((resolve, reject) => {
rec.onstop = () => {
const type = (rec.mimeType || this.chunks[0]?.type || 'audio/webm').split(';')[0]!.trim()
const blob = new Blob(this.chunks, { type })
const waveform = resample(this.samples, waveformPoints)
this.release()
if (!blob.size) reject(new Error('Nothing was recorded'))
else resolve({ blob, mimetype: type, durationMs, waveform })
}
rec.stop()
})
}
/** Stops and throws the recording away. */
cancel() {
if (this.recorder && this.recorder.state !== 'inactive') {
this.recorder.onstop = null
this.recorder.stop()
}
this.release()
}
private release() {
window.clearInterval(this.timer)
this.timer = 0
for (const t of this.stream?.getTracks() ?? []) t.stop()
void this.audio?.close().catch(() => undefined)
this.stream = undefined
this.recorder = undefined
this.audio = undefined
this.analyser = undefined
}
}
/** Averages levels (0–1) down to n points of 0–1024. */
export function resample(levels: number[], n: number): number[] {
if (!levels.length) return []
const out: number[] = []
const count = Math.min(n, levels.length)
for (let i = 0; i < count; i++) {
const from = Math.floor((i * levels.length) / count)
const to = Math.max(from + 1, Math.floor(((i + 1) * levels.length) / count))
let sum = 0
for (let j = from; j < to; j++) sum += levels[j]!
out.push(Math.round((sum / (to - from)) * 1024))
}
return out
}
/** The file extension for a recording's type. */
export function extensionFor(mimetype: string): string {
if (mimetype === 'audio/ogg') return 'ogg'
if (mimetype === 'audio/mp4') return 'm4a'
return 'webm'
}
/** m:ss */
export function formatDuration(ms: number): string {
const s = Math.max(0, Math.round(ms / 1000))
return `${Math.floor(s / 60)}:${String(s % 60).padStart(2, '0')}`
}
+1 -1
View File
@@ -91,7 +91,7 @@ export function describeEvent(ev: MatrixEvent, me: string, room: Room, withSende
text = 'Video'
break
case MsgType.Audio:
text = 'Audio'
text = content['org.matrix.msc3245.voice'] ? 'Voice message' : 'Audio'
break
case MsgType.File:
text = `File: ${content.body ?? ''}`
+186
View File
@@ -0,0 +1,186 @@
import { EventType, MatrixEventEvent, RoomEvent, type MatrixClient, type MatrixEvent, type Room } from 'matrix-js-sdk'
import { defineStore } from 'pinia'
import { computed, ref, watch } from 'vue'
import { router } from '@/router'
import { useCall } from './call'
import { describeEvent } from './chat'
// Notifications for new messages and incoming calls, while the app isn't
// in front of you. Which messages notify is the account's push rules
// (the same ones that decide what your phone is told), evaluated here on
// each live event once it is decrypted.
//
// In the desktop app they are the system's own notifications, through the
// shell's bridge (window.warpDesktop, see apps/desktop); in a browser they
// are the Notification API's, once you allow them.
export interface DesktopNotification {
/** Replaces an earlier notification with the same ID: one per room. */
id: string
title: string
body: string
roomId: string
}
declare global {
interface Window {
/** Added by the desktop app's shell. */
warpDesktop?: { notify(n: DesktopNotification): void }
}
}
/** The desktop app asks the client to open a room (a notification was clicked). */
const openRoomEvent = 'warp-desktop:open-room'
export type NotificationPermissionState = NotificationPermission | 'unsupported' | 'desktop'
const offKey = 'warp.web.notifications'
function savedEnabled(): boolean {
try {
return localStorage.getItem(offKey) !== 'off'
} catch {
return true
}
}
function currentPermission(): NotificationPermissionState {
if (window.warpDesktop) return 'desktop'
if (typeof Notification === 'undefined') return 'unsupported'
return Notification.permission
}
// A message this much older than now is history catching up (after
// sleep, or a room arriving late), not news.
const staleAfterMs = 5 * 60_000
export const useNotifications = defineStore('notifications', () => {
const enabled = ref(savedEnabled())
const permission = ref<NotificationPermissionState>(currentPermission())
watch(enabled, (on) => {
try {
if (on) localStorage.removeItem(offKey)
else localStorage.setItem(offKey, 'off')
} catch {
// Not kept: on again after a reload.
}
})
/** Whether notifications will actually show. */
const active = computed(() => enabled.value && (permission.value === 'granted' || permission.value === 'desktop'))
/** Asks the browser for permission; must run from a click. */
async function enable() {
enabled.value = true
if (permission.value === 'default') {
permission.value = await Notification.requestPermission()
}
}
const shown = new Map<string, Notification>()
const seen = new Set<string>()
let attached: MatrixClient | null = null
let since = 0
function inFront(roomId: string): boolean {
const route = router.currentRoute.value
return document.visibilityState === 'visible' && document.hasFocus() && route.name === 'room' && route.params.roomId === roomId
}
function show(n: DesktopNotification) {
if (!active.value) return
if (window.warpDesktop) {
window.warpDesktop.notify(n)
return
}
shown.get(n.id)?.close()
const note = new Notification(n.title, { body: n.body, tag: n.id, icon: '/favicon.svg' })
note.onclick = () => {
window.focus()
openRoom(n.roomId)
note.close()
}
note.onclose = () => {
if (shown.get(n.id) === note) shown.delete(n.id)
}
shown.set(n.id, note)
}
function openRoom(roomId: string) {
void router.push({ name: 'room', params: { roomId } })
}
function consider(c: MatrixClient, ev: MatrixEvent, room: Room) {
const id = ev.getId()
if (!id || seen.has(id)) return
if (ev.getSender() === c.getUserId()) return
if (ev.getTs() < since || Date.now() - ev.getTs() > staleAfterMs) return
// Calls ring through the call store, below.
const type = ev.getType()
if (type === EventType.CallInvite || type === EventType.RTCNotification) return
// Not decrypted yet: decide once it is.
if (type === EventType.RoomMessageEncrypted && !ev.isDecryptionFailure()) {
ev.once(MatrixEventEvent.Decrypted, () => consider(c, ev, room))
return
}
if (ev.isDecryptionFailure() || ev.isRedacted()) return
seen.add(id)
if (seen.size > 500) seen.delete(seen.values().next().value!)
if (!c.getPushActionsForEvent(ev, true)?.notify) return
if (inFront(room.roomId)) return
const me = c.getUserId() ?? ''
const dms = (c.getAccountData(EventType.Direct)?.getContent() ?? {}) as Record<string, string[]>
const direct = Object.values(dms).some((rooms) => Array.isArray(rooms) && rooms.includes(room.roomId))
const text = describeEvent(ev, me, room, !direct)
if (text === null) return
show({ id: room.roomId, title: room.name, body: text, roomId: room.roomId })
}
function onTimeline(ev: MatrixEvent, room: Room | undefined, toStart: boolean | undefined, removed: boolean, data: { liveEvent?: boolean }) {
if (!attached || !room || toStart || removed || !data?.liveEvent) return
if (!attached.isInitialSyncComplete()) return
consider(attached, ev, room)
}
function onOpenRoom(e: Event) {
const roomId = (e as CustomEvent<string>).detail
if (typeof roomId === 'string' && roomId) openRoom(roomId)
}
/** Starts watching the running client for things to notify about. */
function attach(c: MatrixClient) {
if (attached === c) return
attached = c
since = Date.now() - 30_000
permission.value = currentPermission()
// Thread replies are in their thread's timeline; the room re-emits those.
c.on(RoomEvent.Timeline, onTimeline)
window.addEventListener(openRoomEvent, onOpenRoom)
}
function detach() {
attached?.off(RoomEvent.Timeline, onTimeline)
attached = null
window.removeEventListener(openRoomEvent, onOpenRoom)
for (const n of shown.values()) n.close()
shown.clear()
seen.clear()
}
// An incoming call is news even with the app open in another window.
const call = useCall()
watch(
() => call.incoming,
(inc) => {
if (!inc || !attached || (document.visibilityState === 'visible' && document.hasFocus())) return
show({
id: inc.roomId,
title: inc.roomName,
body: `${inc.callerName}: ${inc.video ? 'Incoming video call' : 'Incoming voice call'}`,
roomId: inc.roomId,
})
},
)
return { enabled, permission, active, enable, attach, detach }
})