Voice and video calls: TURN endpoint and web client calls

Server: GET /voip/turnServer, configured under client_api.turn. A shared
secret (coturn use-auth-secret) gives each user expiring TURN REST API
credentials; static credentials are the fallback; with nothing configured
it answers {} so clients still try a direct connection.

Web client: 1:1 calls (Matrix VoIP via matrix-js-sdk) in rooms with one
other person. Voice and video buttons in the room header, a ringing card
with answer, answer without video and decline, and a call panel that
stays up across rooms: mute, camera, screen sharing, expand, hang up,
ring tones. Call events show in the timeline and room list. Works around
the SDK keeping a failed microphone request forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5.5 committed 2026-10-18 18:04:02 +02:00
1 parent 69ce927b6d
commit 325431a5be
25 files changed
+930 -2

No files matched your search

+8
View File
@@ -22,9 +22,15 @@ export type IconName =
| 'lock'
| 'logout'
| 'megaphone'
| 'maximize'
| 'menu'
| 'mic'
| 'mic-off'
| 'minimize'
| 'moon'
| 'more'
| 'phone'
| 'phone-off'
| 'plug'
| 'plus'
| 'react'
@@ -32,6 +38,7 @@ export type IconName =
| 'refresh'
| 'reply'
| 'room'
| 'screen-share'
| 'search'
| 'send'
| 'server'
@@ -43,6 +50,7 @@ export type IconName =
| 'user'
| 'users'
| 'video'
| 'video-off'
const files = import.meta.glob<string>('./*.svg', { query: '?raw', import: 'default', eager: true })
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>maximize</title><path d="M14 4h6v6"/><path d="M20 4l-6.5 6.5"/><path d="M10 20H4v-6"/><path d="M4 20l6.5-6.5"/></svg>

After

Width:  |  Height:  |  Size: 285 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>mic-off</title><path d="M15 9.5V6a3 3 0 0 0-5.8-1"/><path d="M9 9v2a3 3 0 0 0 4.6 2.5"/><path d="M5.5 11a6.5 6.5 0 0 0 10.6 5"/><path d="M18.5 11a6.4 6.4 0 0 1-.6 2.7"/><path d="M12 17.5V21"/><path d="M4 4l16 16"/></svg>

After

Width:  |  Height:  |  Size: 388 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>mic</title><rect x="9" y="3" width="6" height="11" rx="3"/><path d="M5.5 11a6.5 6.5 0 0 0 13 0"/><path d="M12 17.5V21"/></svg>

After

Width:  |  Height:  |  Size: 294 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>minimize</title><path d="M20 10h-6V4"/><path d="M14 10l6.5-6.5"/><path d="M4 14h6v6"/><path d="M10 14l-6.5 6.5"/></svg>

After

Width:  |  Height:  |  Size: 287 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>phone-off</title><path d="M3.5 12.5a13 13 0 0 1 17 0l-1.7 2.6-3.3-1.3v-2.5a10 10 0 0 0-7 0v2.5l-3.3 1.3z"/></svg>

After

Width:  |  Height:  |  Size: 281 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>phone</title><path d="M5 4h3.5l1.8 4.5-2.3 1.4a11 11 0 0 0 6.1 6.1l1.4-2.3L20 15.5V19a1.5 1.5 0 0 1-1.6 1.5A16.5 16.5 0 0 1 3.5 5.6 1.5 1.5 0 0 1 5 4z"/></svg>

After

Width:  |  Height:  |  Size: 327 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>screen-share</title><rect x="3" y="4.5" width="18" height="12" rx="2"/><path d="M8.5 20h7"/><path d="M12 16.5V20"/><path d="M12 13.5v-6"/><path d="M9.5 10l2.5-2.5 2.5 2.5"/></svg>

After

Width:  |  Height:  |  Size: 347 B

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.75" stroke-linecap="round" stroke-linejoin="round"><title>video-off</title><path d="M10 6.5h3a2.5 2.5 0 0 1 2.5 2.5v3"/><path d="M15.5 15A2.5 2.5 0 0 1 13 17.5H5.5A2.5 2.5 0 0 1 3 15V9a2.5 2.5 0 0 1 2.5-2.5"/><path d="M15.5 10.5l5.5-3v9l-3-1.6"/><path d="M3 3l18 18"/></svg>

After

Width:  |  Height:  |  Size: 384 B

+7 -1
View File
@@ -63,6 +63,12 @@ that one origin.
them asks you to acknowledge it; a reset since you *verified* them says
their sessions are no longer trusted, with Verify again or Withdraw
verification.
- Voice and video calls in a room with one other person (Matrix 1:1 VoIP,
WebRTC between the browsers): ringing, answer with or without video,
mute, camera on and off, screen sharing, and a panel that stays up while
you move between rooms and expands to fill the window. In an encrypted
room the call signalling is encrypted too. Calls outside one network
need a TURN server; see `client_api.turn` in the server configuration.
- Encryption setup in Settings: cross-signing keys, secret storage and a
key backup in one step, behind a recovery key shown once. A new session
verifies itself with the recovery key (or another session) and restores
@@ -166,7 +172,7 @@ page saying the client is missing.
| Path | What |
|---|---|
| `src/matrix/` | everything that touches matrix-js-sdk: the client's lifecycle, the timeline projection, sending, media, encryption (`crypto.ts`, `attachments.ts`) |
| `src/stores/` | session, room list and connection state, encryption and verification (`security.ts`), profile, toasts (Pinia) |
| `src/stores/` | session, room list and connection state, encryption and verification (`security.ts`), calls (`call.ts`), profile, toasts (Pinia) |
| `src/views/` | sign-in, sign-up, home, room |
| `src/components/` | timeline, message body, room info, dialogs |
| `cmd/warp-web/` | the binary |
+204
View File
@@ -0,0 +1,204 @@
<script setup lang="ts">
import type { CallFeed } from 'matrix-js-sdk/lib/webrtc/callFeed'
import { computed, onBeforeUnmount, ref, watch, watchEffect } from 'vue'
import { useRouter } from 'vue-router'
import { WIcon } from '@warp/ui'
import type { IconName } from '@warp/ui'
import MxcAvatar from '@/components/MxcAvatar.vue'
import { formatDuration } from '@/lib/format'
import { useCall } from '@/stores/call'
// The call on screen: a ringing card for an incoming call, else a panel in
// the corner that a video call can expand to fill the window. It sits above
// every view, so a call goes on while you move between rooms.
const call = useCall()
const router = useRouter()
const expanded = ref(false)
function feedVideo(feed: CallFeed | undefined): MediaStream | undefined {
if (!feed || feed.isVideoMuted() || !feed.stream.getVideoTracks().length) return undefined
return feed.stream
}
const view = computed(() => {
void call.version
const c = call.call
if (!c) return null
const remoteCamera = feedVideo(c.remoteUsermediaFeed)
const remoteScreen = c.remoteScreensharingStream
const localCamera = feedVideo(c.localUsermediaFeed)
const localScreen = c.localScreensharingStream
// Their screen beats their camera on the big stage; whatever is left of
// theirs or yours goes in the small picture.
const main = remoteScreen ?? remoteCamera
const pip = remoteScreen && remoteCamera ? remoteCamera : (localScreen ?? localCamera)
return {
main,
pip,
pipMirrored: pip !== undefined && pip === localCamera,
remoteAudio: c.remoteUsermediaStream,
micMuted: c.isMicrophoneMuted(),
cameraOn: Boolean(localCamera),
sharing: c.isScreensharing(),
remoteOnHold: c.isRemoteOnHold(),
localOnHold: c.isLocalOnHold(),
roomId: c.roomId,
}
})
const stage = computed(() => call.phase !== 'ringing' && Boolean(view.value?.main || view.value?.pip || call.isVideo))
const canShare = typeof navigator !== 'undefined' && Boolean(navigator.mediaDevices?.getDisplayMedia)
const status = computed(() => {
const v = view.value
switch (call.phase) {
case 'ringing':
return call.isVideo ? 'Incoming video call' : 'Incoming voice call'
case 'calling':
return 'Calling…'
case 'connecting':
return 'Connecting…'
case 'connected':
if (v?.localOnHold) return 'You’re on hold'
if (v?.remoteOnHold) return 'On hold'
return formatDuration(call.length)
case 'ended':
return call.endReason
default:
return ''
}
})
// Streams go to the media elements by hand: srcObject is not an attribute.
const mainEl = ref<HTMLVideoElement>()
const pipEl = ref<HTMLVideoElement>()
const audioEl = ref<HTMLAudioElement>()
function bind(el: HTMLMediaElement | undefined, stream: MediaStream | undefined) {
if (!el || el.srcObject === (stream ?? null)) return
el.srcObject = stream ?? null
if (stream) void el.play().catch(() => undefined)
}
watchEffect(() => bind(mainEl.value, view.value?.main), { flush: 'post' })
watchEffect(() => bind(pipEl.value, view.value?.pip), { flush: 'post' })
watchEffect(() => bind(audioEl.value, view.value?.remoteAudio), { flush: 'post' })
watch(
() => call.call,
(c) => {
if (!c) expanded.value = false
},
)
function onKey(e: KeyboardEvent) {
if (e.key === 'Escape' && expanded.value) expanded.value = false
}
window.addEventListener('keydown', onKey)
onBeforeUnmount(() => window.removeEventListener('keydown', onKey))
function openRoom() {
const roomId = view.value?.roomId
if (!roomId) return
expanded.value = false
router.push({ name: 'room', params: { roomId } })
}
interface Control {
icon: IconName
label: string
action: () => void
on?: boolean
tone?: 'danger' | 'accept'
}
const controls = computed<Control[]>(() => {
const v = view.value
if (!v) return []
if (call.phase === 'ringing') {
const list: Control[] = [{ icon: 'phone-off', label: 'Decline', action: call.decline, tone: 'danger' }]
if (call.isVideo) list.push({ icon: 'phone', label: 'Answer without video', action: () => call.answer(false) })
list.push({ icon: call.isVideo ? 'video' : 'phone', label: 'Answer', action: () => call.answer(), tone: 'accept' })
return list
}
if (call.phase === 'ended') return [{ icon: 'close', label: 'Close', action: call.hangup }]
const list: Control[] = [
{ icon: v.micMuted ? 'mic-off' : 'mic', label: v.micMuted ? 'Unmute' : 'Mute', action: call.toggleMic, on: v.micMuted },
{ icon: v.cameraOn ? 'video' : 'video-off', label: v.cameraOn ? 'Turn camera off' : 'Turn camera on', action: call.toggleCamera, on: !v.cameraOn && call.isVideo },
]
if (canShare && call.phase === 'connected') {
list.push({ icon: 'screen-share', label: v.sharing ? 'Stop sharing' : 'Share screen', action: call.toggleScreenShare, on: v.sharing })
}
list.push({ icon: 'phone-off', label: 'Hang up', action: call.hangup, tone: 'danger' })
return list
})
function controlClass(c: Control): string {
if (c.tone === 'danger') return 'bg-danger text-[#ffffff] hover:brightness-110'
if (c.tone === 'accept') return 'bg-success text-[#ffffff] hover:brightness-110'
if (c.on) return 'bg-[#eceeee] text-[#111418] hover:bg-[#ffffff]'
return 'bg-[#ffffff1f] text-[#eceeee] hover:bg-[#ffffff33]'
}
</script>
<template>
<section
v-if="call.call && view"
:class="[
'fixed z-40 flex flex-col overflow-hidden bg-[#111418] text-[#eceeee] shadow-float',
expanded ? 'inset-0' : 'right-4 bottom-4 left-4 rounded-lg sm:left-auto sm:w-[360px]',
]"
:aria-label="`Call with ${call.peerName}`"
:role="call.phase === 'ringing' ? 'alertdialog' : 'region'"
>
<audio ref="audioEl" autoplay class="hidden" />
<div v-if="stage" :class="['relative bg-[#000000]', expanded ? 'min-h-0 flex-1' : 'aspect-video']">
<video v-show="view.main" ref="mainEl" autoplay playsinline muted class="h-full w-full object-contain" />
<div v-if="!view.main" class="absolute inset-0 flex items-center justify-center">
<MxcAvatar :name="call.peerName" :mxc="call.peerAvatarMxc" :size="expanded ? 120 : 72" />
</div>
<video
v-show="view.pip"
ref="pipEl"
autoplay
playsinline
muted
:class="['absolute right-3 bottom-3 w-1/4 min-w-24 rounded-md bg-[#000000] object-cover shadow-float', view.pipMirrored && '-scale-x-100']"
/>
<button
v-if="call.isVideo || view.main"
type="button"
class="absolute top-2 right-2 flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-0 bg-[#00000080] text-[#eceeee] hover:bg-[#000000b3]"
:aria-label="expanded ? 'Shrink' : 'Expand'"
:title="expanded ? 'Shrink' : 'Expand'"
@click="expanded = !expanded"
>
<WIcon :name="expanded ? 'minimize' : 'maximize'" :size="18" />
</button>
</div>
<div class="flex items-center gap-3 px-4 pt-4">
<MxcAvatar v-if="!stage" :name="call.peerName" :mxc="call.peerAvatarMxc" :size="48" />
<button type="button" class="min-w-0 flex-1 cursor-pointer border-0 bg-transparent p-0 text-left text-[#eceeee]" @click="openRoom">
<span class="body-strong block truncate">{{ call.peerName }}</span>
<span class="small block truncate text-[#eceeeeb3] tabular-nums" aria-live="polite">{{ status }}</span>
</button>
</div>
<div class="flex items-center justify-center gap-3 px-4 pt-3 pb-4">
<button
v-for="c in controls"
:key="c.label"
type="button"
:class="['flex h-12 w-12 cursor-pointer items-center justify-center rounded-full border-0 transition-colors', controlClass(c)]"
:aria-label="c.label"
:aria-pressed="c.on === undefined || c.tone ? undefined : c.on"
:title="c.label"
@click="c.action()"
>
<WIcon :name="c.icon" :size="22" />
</button>
</div>
</section>
</template>
+5
View File
@@ -2,6 +2,7 @@
import { computed, onMounted, ref, watchEffect } from 'vue'
import { useRoute, useRouter } from 'vue-router'
import { WBanner, WButton, WIcon, WIconButton, WLogo, WSpinner } from '@warp/ui'
import CallView from '@/components/CallView.vue'
import CreateRoomDialog from '@/components/CreateRoomDialog.vue'
import EncryptionDialog from '@/components/EncryptionDialog.vue'
import JoinRoomDialog from '@/components/JoinRoomDialog.vue'
@@ -13,6 +14,7 @@ import SettingsDialog from '@/components/SettingsDialog.vue'
import StartChatDialog from '@/components/StartChatDialog.vue'
import VerificationDialog from '@/components/VerificationDialog.vue'
import { formatListTime, localpart } from '@/lib/format'
import { useCall } from '@/stores/call'
import { useChat } from '@/stores/chat'
import { useProfile } from '@/stores/profile'
import { useSecurity } from '@/stores/security'
@@ -20,6 +22,7 @@ import { useSession } from '@/stores/session'
const session = useSession()
const chat = useChat()
const call = useCall()
const profile = useProfile()
const security = useSecurity()
const route = useRoute()
@@ -40,6 +43,7 @@ async function start() {
if (session.phase === 'ready') {
chat.attach()
security.attach()
call.attach()
profile.load()
}
} catch (e) {
@@ -204,5 +208,6 @@ const ownName = computed(() => profile.displayName || localpart(session.userId))
<SettingsDialog :open="dialog === 'settings'" @update:open="dialog = $event ? 'settings' : null" />
<EncryptionDialog mode="unlock" :open="dialog === 'verify'" @update:open="dialog = $event ? 'verify' : null" />
<VerificationDialog />
<CallView />
</div>
</template>
+9
View File
@@ -43,6 +43,15 @@ export function sameDay(a: number, b: number): boolean {
return startOfDay(a) === startOfDay(b)
}
/** A call's length: "0:42", "12:05", "1:02:09". */
export function formatDuration(seconds: number): string {
const s = Math.max(0, Math.floor(seconds))
const h = Math.floor(s / 3600)
const m = Math.floor((s % 3600) / 60)
const ss = String(s % 60).padStart(2, '0')
return h ? `${h}:${String(m).padStart(2, '0')}:${ss}` : `${m}:${ss}`
}
const units = ['B', 'KB', 'MB', 'GB', 'TB']
export function formatBytes(n: number | undefined): string {
+38 -1
View File
@@ -101,6 +101,36 @@ function statusOf(ev: MatrixEvent): MessageStatus | undefined {
}
}
/** Whether an m.call.invite offers video. */
export function isVideoCallInvite(ev: MatrixEvent): boolean {
const sdp = ev.getContent().offer?.sdp
return typeof sdp === 'string' && sdp.includes('m=video')
}
/** A one-line description of a call event, or null for the signalling in between. */
function describeCall(ev: MatrixEvent, room: Room): string | null {
const sender = nameOf(room, ev.getSender() ?? '')
switch (ev.getType()) {
case EventType.CallInvite:
return `${sender} started a ${isVideoCallInvite(ev) ? 'video' : 'voice'} call`
case EventType.CallReject:
return `${sender} declined the call`
case EventType.CallHangup:
switch (ev.getContent().reason) {
case 'invite_timeout':
return 'The call wasn’t answered'
case 'user_busy':
return `${sender} was busy`
case 'ice_failed':
return 'The call couldn’t connect'
default:
return `${sender} ended the call`
}
default:
return null
}
}
/** A one-line description of a state change, or null to leave it out. */
function describeState(ev: MatrixEvent, room: Room): string | null {
const sender = nameOf(room, ev.getSender() ?? '')
@@ -272,7 +302,14 @@ export function buildTimeline(c: MatrixClient, room: Room): TimelineItem[] {
const isMessage = type === EventType.RoomMessage || type === EventType.Sticker || type === EventType.RoomMessageEncrypted
if (!isMessage) {
if (!ev.isState()) continue
if (!ev.isState()) {
const text = describeCall(ev, room)
if (text) {
items.push({ kind: 'notice', key, ts, text })
prev = null
}
continue
}
const text = describeState(ev, room)
if (text) {
items.push({ kind: 'notice', key, ts, text })
+67
View File
@@ -0,0 +1,67 @@
// The ring of an incoming call and the ringback of an outgoing one, made
// with Web Audio rather than shipped as sound files. One tone plays at a
// time. A browser that hasn't seen the page interacted with yet may keep
// the audio context suspended: the call still shows, just silently.
export type Tone = 'ring' | 'ringback'
// Each pattern repeats: [frequencies played together, seconds on, seconds off].
const patterns: Record<Tone, { freqs: number[]; on: number; off: number }[]> = {
// Two short bursts, a pause: a phone ringing.
ring: [
{ freqs: [440, 480], on: 0.4, off: 0.2 },
{ freqs: [440, 480], on: 0.4, off: 2 },
],
// The long tone you hear while the other side rings.
ringback: [{ freqs: [440, 480], on: 1.5, off: 3 }],
}
let ctx: AudioContext | null = null
let playing: Tone | null = null
let timer = 0
let gain: GainNode | null = null
function context(): AudioContext | null {
try {
ctx ??= new AudioContext()
if (ctx.state === 'suspended') void ctx.resume().catch(() => undefined)
return ctx
} catch {
return null
}
}
export function startTone(tone: Tone) {
if (playing === tone) return
stopTone()
const ac = context()
if (!ac) return
playing = tone
gain = ac.createGain()
gain.gain.value = tone === 'ring' ? 0.12 : 0.06
gain.connect(ac.destination)
const steps = patterns[tone]
let i = 0
const step = () => {
if (playing !== tone || !gain) return
const s = steps[i++ % steps.length]!
const start = ac.currentTime
for (const f of s.freqs) {
const osc = ac.createOscillator()
osc.frequency.value = f
osc.connect(gain)
osc.start(start)
osc.stop(start + s.on)
}
timer = window.setTimeout(step, (s.on + s.off) * 1000)
}
step()
}
export function stopTone() {
playing = null
window.clearTimeout(timer)
timer = 0
gain?.disconnect()
gain = null
}
+340
View File
@@ -0,0 +1,340 @@
import { CallEvent, CallFeedEvent, createNewMatrixCall, type MatrixCall, type MatrixClient } from 'matrix-js-sdk'
import { CallDirection, CallErrorCode, CallParty, CallState, CallType, type CallError } from 'matrix-js-sdk/lib/webrtc/call'
import { CallEventHandlerEvent } from 'matrix-js-sdk/lib/webrtc/callEventHandler'
import type { CallFeed } from 'matrix-js-sdk/lib/webrtc/callFeed'
import { defineStore } from 'pinia'
import { computed, ref, shallowRef } from 'vue'
import { errorMessage } from '@/lib/format'
import { client } from '@/matrix/client'
import { startTone, stopTone } from '@/matrix/tones'
import { useToasts } from '@/stores/toasts'
// Voice and video calls between two people: Matrix's 1:1 VoIP (m.call.*
// events in the room, WebRTC between the browsers), which the SDK runs.
// In an encrypted room the call events are encrypted like any other; the
// media itself is always encrypted by WebRTC (DTLS-SRTP).
//
// One call at a time. A call that comes in while one is going is answered
// "busy", as a phone would.
export type CallPhase = 'ringing' | 'calling' | 'connecting' | 'connected' | 'ended'
/** How long the ended call stays on screen, saying why. */
const endedLingerMs = 2500
export const useCall = defineStore('call', () => {
const call = shallowRef<MatrixCall | null>(null)
/** Bumps on every change the call view shows. */
const version = ref(0)
/** Seconds since the call connected. */
const length = ref(0)
/** Why the call ended, once it has. */
const endReason = ref('')
let attached: MatrixClient | null = null
let clearTimer = 0
/** The call got as far as connecting media, at some point. */
let connected = false
/** The feeds whose mute state is listened to. */
let feeds: CallFeed[] = []
const phase = computed<CallPhase | null>(() => {
void version.value
const c = call.value
if (!c) return null
switch (c.state) {
case CallState.Ringing:
return 'ringing'
case CallState.Fledgling:
case CallState.WaitLocalMedia:
case CallState.CreateOffer:
case CallState.InviteSent:
return c.direction === CallDirection.Inbound ? 'connecting' : 'calling'
case CallState.CreateAnswer:
case CallState.Connecting:
return 'connecting'
case CallState.Connected:
return 'connected'
default:
return 'ended'
}
})
const isVideo = computed(() => {
void version.value
return call.value?.type === CallType.Video
})
const peerName = computed(() => {
void version.value
const c = call.value
if (!c) return ''
const member = c.getOpponentMember()
if (member) return member.name
const room = attached?.getRoom(c.roomId)
return room?.name ?? c.invitee ?? ''
})
const peerAvatarMxc = computed(() => {
void version.value
return call.value?.getOpponentMember()?.getMxcAvatarUrl() ?? undefined
})
function bump() {
version.value++
}
// A feed muting its camera or microphone is not a change in the call's
// feeds, so each feed is listened to as well.
function watchFeeds(c: MatrixCall | null) {
for (const f of feeds) {
f.off(CallFeedEvent.MuteStateChanged, bump)
f.off(CallFeedEvent.NewStream, bump)
}
feeds = c ? c.getFeeds() : []
for (const f of feeds) {
f.on(CallFeedEvent.MuteStateChanged, bump)
f.on(CallFeedEvent.NewStream, bump)
}
}
function onFeeds() {
watchFeeds(call.value)
bump()
}
function tones() {
const p = phase.value
if (p === 'ringing') startTone('ring')
else if (p === 'calling') startTone('ringback')
else stopTone()
}
function onState(state: CallState) {
if (state === CallState.Connected) connected = true
bump()
tones()
}
function onLength(seconds: number) {
length.value = seconds
}
function onError(err: CallError) {
// The hangup that follows says the rest.
if (err.code === CallErrorCode.NoUserMedia) {
forgetFailedMedia()
useToasts().error(isVideo.value ? 'Can’t use your camera or microphone. Check that the browser may use them.' : 'Can’t use your microphone. Check that the browser may use it.')
} else {
useToasts().error(`Call failed: ${errorMessage(err)}`)
}
}
function onHangup(c: MatrixCall) {
if (c !== call.value) return
endReason.value = describeHangup(c, connected)
bump()
stopTone()
window.clearTimeout(clearTimer)
clearTimer = window.setTimeout(() => {
if (call.value === c) release()
}, endedLingerMs)
}
function onReplaced(newCall: MatrixCall) {
// Both of you called at once: the SDK keeps one of the two calls.
show(newCall)
}
function listen(c: MatrixCall, on: boolean) {
const m = on ? 'on' : 'off'
c[m](CallEvent.State, onState)
c[m](CallEvent.FeedsChanged, onFeeds)
c[m](CallEvent.LocalHoldUnhold, bump)
c[m](CallEvent.RemoteHoldUnhold, bump)
c[m](CallEvent.LengthChanged, onLength)
c[m](CallEvent.Error, onError)
c[m](CallEvent.Hangup, onHangup)
c[m](CallEvent.Replaced, onReplaced)
}
function show(c: MatrixCall) {
if (call.value) listen(call.value, false)
window.clearTimeout(clearTimer)
call.value = c
connected = false
length.value = 0
endReason.value = ''
listen(c, true)
watchFeeds(c)
bump()
tones()
}
function release() {
if (call.value) listen(call.value, false)
watchFeeds(null)
window.clearTimeout(clearTimer)
call.value = null
stopTone()
bump()
}
function onIncoming(incoming: MatrixCall) {
if (call.value && phase.value !== 'ended') {
incoming.hangup(CallErrorCode.UserBusy, false)
return
}
show(incoming)
// Hung up (or answered elsewhere) before we did anything.
if (incoming.callHasEnded()) onHangup(incoming)
}
/** Starts listening for calls on the running client. */
function attach() {
const c = client()
if (attached === c) return
if (attached) attached.off(CallEventHandlerEvent.Incoming, onIncoming)
release()
attached = c
c.on(CallEventHandlerEvent.Incoming, onIncoming)
}
/** Whether this browser can make calls at all. */
function supported(): boolean {
return Boolean(attached?.supportsVoip())
}
async function place(roomId: string, invitee: string | undefined, video: boolean) {
if (call.value && phase.value !== 'ended') {
useToasts().error('You’re already in a call.')
return
}
const c = createNewMatrixCall(client(), roomId, { invitee })
if (!c) {
useToasts().error('This browser can’t make calls.')
return
}
show(c)
try {
if (video) await c.placeVideoCall()
else await c.placeVoiceCall()
} catch (e) {
// The SDK has hung up already and said why through CallEvent.Error.
console.error('call: ', e)
}
}
/** Answers the ringing call; a video call with the camera off when `video` is false. */
async function answer(video = true) {
const c = call.value
if (!c || phase.value !== 'ringing') return
stopTone()
try {
await c.answer(true, video && c.type === CallType.Video)
} catch (e) {
console.error('call: ', e)
}
}
function decline() {
const c = call.value
if (!c) return
if (phase.value === 'ringing') c.reject()
else hangup()
}
function hangup() {
const c = call.value
if (!c) return
if (phase.value === 'ended') release()
else c.hangup(CallErrorCode.UserHangup, false)
}
async function toggleMic() {
const c = call.value
if (!c) return
await c.setMicrophoneMuted(!c.isMicrophoneMuted())
bump()
}
async function toggleCamera() {
const c = call.value
if (!c) return
await c.setLocalVideoMuted(!c.isLocalVideoMuted())
bump()
}
async function toggleScreenShare() {
const c = call.value
if (!c) return
try {
await c.setScreensharingEnabled(!c.isScreensharing())
} catch (e) {
useToasts().error(`Can’t share your screen: ${errorMessage(e)}`)
}
bump()
}
function toggleHold() {
const c = call.value
if (!c) return
c.setRemoteOnHold(!c.isRemoteOnHold())
bump()
}
return {
call,
version,
length,
endReason,
phase,
isVideo,
peerName,
peerAvatarMxc,
attach,
supported,
place,
answer,
decline,
hangup,
toggleMic,
toggleCamera,
toggleScreenShare,
toggleHold,
}
})
/**
* The SDK's media handler queues every request for the microphone and
* camera behind the previous one, and never lets go of a request that
* failed: once you said no, every later call would fail the same way until
* the page reloads. Dropping the failed request lets the next call ask again.
*/
function forgetFailedMedia() {
const handler = client().getMediaHandler() as unknown as { getMediaStreamPromise?: Promise<MediaStream> }
handler.getMediaStreamPromise = undefined
}
function describeHangup(c: MatrixCall, connected: boolean): string {
switch (c.hangupReason) {
case CallErrorCode.UserBusy:
return 'Busy'
case CallErrorCode.InviteTimeout:
return c.direction === CallDirection.Inbound ? 'Missed call' : 'No answer'
case CallErrorCode.AnsweredElsewhere:
return 'Answered in another session'
case CallErrorCode.IceFailed:
return 'Couldn’t connect'
case CallErrorCode.NoUserMedia:
return 'No microphone or camera'
case CallErrorCode.Replaced:
return 'Call replaced'
default:
if (!connected) {
const outbound = c.direction === CallDirection.Outbound
if (c.hangupParty === CallParty.Remote) return outbound ? 'Declined' : 'Missed call'
return outbound ? 'Call cancelled' : 'Declined'
}
return 'Call ended'
}
}
+5
View File
@@ -19,6 +19,7 @@ import {
import { defineStore } from 'pinia'
import { computed, ref, shallowRef, watch } from 'vue'
import { client } from '@/matrix/client'
import { isVideoCallInvite } from '@/matrix/timeline'
import { onShieldsChanged } from '@/matrix/trust'
// The room list and the client's connection state, as plain reactive data
@@ -59,6 +60,10 @@ export function describeEvent(ev: MatrixEvent, me: string, room: Room, withSende
// own placeholder text: never show that.
if (ev.isDecryptionFailure()) return withSender ? `${senderName(ev, me, room)}: Can’t decrypt this message` : 'Can’t decrypt this message'
if (type === EventType.RoomMessageEncrypted) return 'Encrypted message'
if (type === EventType.CallInvite) {
const text = isVideoCallInvite(ev) ? 'Video call' : 'Voice call'
return withSender ? `${senderName(ev, me, room)}: ${text}` : text
}
if (type !== EventType.RoomMessage && type !== EventType.Sticker) return null
if (ev.isRelation('m.replace')) return null
const content = ev.getContent()
+21
View File
@@ -11,6 +11,7 @@ import { errorMessage, formatBytes } from '@/lib/format'
import { client, cryptoError } from '@/matrix/client'
import { acceptIdentity, identityWarnings, withdrawVerification, type IdentityWarning } from '@/matrix/crypto'
import { sendFile, sendText } from '@/matrix/send'
import { useCall } from '@/stores/call'
import { useChat } from '@/stores/chat'
import { useSecurity } from '@/stores/security'
import { useToasts } from '@/stores/toasts'
@@ -18,6 +19,7 @@ import { useToasts } from '@/stores/toasts'
const props = defineProps<{ roomId: string }>()
const chat = useChat()
const call = useCall()
const security = useSecurity()
const toasts = useToasts()
const router = useRouter()
@@ -52,6 +54,21 @@ const canSend = computed(() => {
const infoOpen = ref(false)
// Calls are between two people: offered in a room with just you and one
// other person in it.
const callee = computed(() => {
void chat.version
const r = room.value
if (!r || r.getJoinedMemberCount() !== 2) return undefined
return r.getJoinedMembers().find((m) => m.userId !== me)?.userId
})
const canCall = computed(() => Boolean(callee.value) && call.supported())
// One call at a time, wherever it is.
const inCall = computed(() => Boolean(call.call) && call.phase !== 'ended')
function startCall(video: boolean) {
void call.place(props.roomId, callee.value, video)
}
// Someone in an encrypted room whose identity changed: said before you
// write to them, until you acknowledge it.
const warnings = ref<IdentityWarning[]>([])
@@ -277,6 +294,10 @@ const placeholder = computed(() => `Message ${summary.value?.name ?? 'this room'
<span v-if="!encrypted" class="caption hidden items-center gap-1 text-ink-muted sm:flex">
<WIcon name="shield-alert" :size="14" /> Not encrypted
</span>
<template v-if="canCall">
<WIconButton icon="phone" label="Voice call" :disabled="inCall" @click="startCall(false)" />
<WIconButton icon="video" label="Video call" :disabled="inCall" @click="startCall(true)" />
</template>
<WIconButton icon="users" :label="infoOpen ? 'Hide room info' : 'Room info'" :aria-pressed="infoOpen" @click="infoOpen = !infoOpen" />
</header>
+4
View File
@@ -62,6 +62,10 @@ List-valued variables are comma-separated. A single `-` clears a list:
| `client_api.rate_limit.join.burst` / `.interval` | — | `10` / `1s` | The two join endpoints. |
| `client_api.rate_limit.sync.burst` / `.interval` | — | `100` / `20ms` | `/sync` and the writes behind it: typing, receipts, to-device, presence. |
| `client_api.rate_limit.media.burst` / `.interval` | — | `100` / `20ms` | Media downloads and thumbnails: a client opening a room asks for every avatar at once. Uploads are in the default bucket. |
| `client_api.turn.uris` | `WARP_TURN_URIS` | `[]` | `turn:`, `turns:` and `stun:` URIs for calls. Empty = no relay; `/voip/turnServer` answers `{}`. |
| `client_api.turn.shared_secret` | `WARP_TURN_SHARED_SECRET` | — | coturn's `static-auth-secret`: each user gets expiring TURN REST API credentials. |
| `client_api.turn.user_lifetime` | `WARP_TURN_USER_LIFETIME` | `1h` | How long shared-secret credentials last. |
| `client_api.turn.username` / `.password` | `WARP_TURN_USERNAME` / `WARP_TURN_PASSWORD` | — | Static credentials, used only without a shared secret. |
| `sync.presence` | `WARP_SYNC_PRESENCE` | `false` | Off by default; see below. |
| `sync.typing_timeout` | `WARP_SYNC_TYPING_TIMEOUT` | `30s` | How long a typing notification lasts, and the cap on what a client may ask for. |
| `sync.long_poll_timeout` | `WARP_SYNC_LONG_POLL_TIMEOUT` | `30s` | The cap on a `/sync` `?timeout=`. |
+8
View File
@@ -47,6 +47,7 @@ type ClientAPI struct {
remoteProfile RemoteProfiles
regTokens *userapi.RegistrationTokens
appservices *appservice.API
turn config.TURN
log *slog.Logger
now func() time.Time
@@ -141,6 +142,9 @@ type Options struct {
// RateLimit configures the per-endpoint buckets.
RateLimit config.RateLimit
// TURN is the relay handed to clients for calls.
TURN config.TURN
// TrustXForwardedFor takes the client address from the proxy header.
TrustXForwardedFor bool
@@ -200,6 +204,7 @@ func New(opts Options) (*ClientAPI, error) {
remoteProfile: opts.RemoteProfiles,
regTokens: opts.RegistrationTokens,
appservices: opts.AppServices,
turn: opts.TURN,
log: opts.Logger,
now: opts.Now,
uia: newUIAStore(uiaSessionTTL, opts.Now),
@@ -267,6 +272,9 @@ func (c *ClientAPI) RegisterRoutes(r *httputil.Router) {
c.authed(r, "POST "+v1+"/appservice/{appserviceID}/ping", c.limitDefault, c.handleAppServicePing)
c.authed(r, "GET "+v3+"/capabilities", c.limitDefault, c.handleCapabilities)
// Calls: the relay a client sets its peer connection up with.
c.authed(r, "GET "+v3+"/voip/turnServer", c.limitDefault, c.handleTurnServer)
// Devices.
c.authed(r, "GET "+v3+"/devices", c.limitDefault, c.handleGetDevices)
c.authed(r, "GET "+v3+"/devices/{deviceID}", c.limitDefault, c.handleGetDevice)
+60
View File
@@ -0,0 +1,60 @@
package clientapi
import (
"crypto/hmac"
"crypto/sha1" //nolint:gosec // The TURN REST API is defined with HMAC-SHA1; coturn checks exactly that.
"encoding/base64"
"net/http"
"strconv"
"github.com/dariuszjaniak/warp/server/internal/config"
"github.com/dariuszjaniak/warp/server/internal/httputil"
"github.com/dariuszjaniak/warp/server/internal/userapi"
)
type turnServerResponse struct {
URIs []string `json:"uris"`
Username string `json:"username"`
Password string `json:"password"`
TTL int64 `json:"ttl"`
}
// staticTURNTTL is the ttl given with static credentials. They never
// change, so it only sets how often a client asks again.
const staticTURNTTL = 86400
// handleTurnServer implements GET /_matrix/client/v3/voip/turnServer.
//
// With no TURN server configured it answers an empty object, as Synapse
// does: clients take that as "no relay" and still try a direct connection,
// where an error would have some of them refuse to place a call at all.
func (c *ClientAPI) handleTurnServer(w http.ResponseWriter, _ *http.Request, auth userapi.Authenticated) error {
t := c.turn
if len(t.URIs) == 0 || (t.SharedSecret == "" && t.Username == "") {
httputil.WriteJSON(w, http.StatusOK, struct{}{})
return nil
}
if t.SharedSecret != "" {
username, password, ttl := turnRESTCredentials(t, auth.UserID, c.now().Unix())
httputil.WriteJSON(w, http.StatusOK, turnServerResponse{
URIs: t.URIs, Username: username, Password: password, TTL: ttl,
})
return nil
}
httputil.WriteJSON(w, http.StatusOK, turnServerResponse{
URIs: t.URIs, Username: t.Username, Password: t.Password, TTL: staticTURNTTL,
})
return nil
}
// turnRESTCredentials makes TURN REST API credentials: the username is the
// expiry time and the user, and the password is the username signed with
// the secret the TURN server shares, so it can check them without asking
// anyone.
func turnRESTCredentials(t config.TURN, userID string, now int64) (username, password string, ttl int64) {
ttl = int64(t.UserLifetime.Seconds())
username = strconv.FormatInt(now+ttl, 10) + ":" + userID
mac := hmac.New(sha1.New, []byte(t.SharedSecret))
mac.Write([]byte(username))
return username, base64.StdEncoding.EncodeToString(mac.Sum(nil)), ttl
}
+89
View File
@@ -0,0 +1,89 @@
package clientapi_test
import (
"crypto/hmac"
"crypto/sha1" //nolint:gosec // The TURN REST API is HMAC-SHA1.
"encoding/base64"
"net/http"
"strconv"
"strings"
"testing"
"time"
"github.com/dariuszjaniak/warp/server/internal/clientapi"
"github.com/dariuszjaniak/warp/server/internal/config"
)
const turnPath = "/_matrix/client/v3/voip/turnServer"
type turnResponse struct {
URIs []string `json:"uris"`
Username string `json:"username"`
Password string `json:"password"`
TTL int64 `json:"ttl"`
}
func TestTurnServerNeedsAToken(t *testing.T) {
ts := newTestServer(t, nil)
expectError(t, ts.get(turnPath, ""), http.StatusUnauthorized, "M_MISSING_TOKEN")
}
func TestTurnServerUnconfiguredIsEmpty(t *testing.T) {
ts := newTestServer(t, nil)
acc := ts.register("alice", "correct horse battery")
rec := ts.get(turnPath, acc.AccessToken)
expectStatus(t, rec, http.StatusOK)
if body := jsonBody(t, rec); len(body) != 0 {
t.Errorf("body = %v, want an empty object", body)
}
}
func TestTurnServerSharedSecret(t *testing.T) {
const secret = "s3cret"
ts := newTestServer(t, func(o *clientapi.Options) {
o.TURN = config.TURN{
URIs: []string{"turn:turn.warp.example:3478?transport=udp"},
SharedSecret: secret,
UserLifetime: time.Hour,
}
})
acc := ts.register("alice", "correct horse battery")
before := time.Now().Unix()
rec := ts.get(turnPath, acc.AccessToken)
expectStatus(t, rec, http.StatusOK)
got := decode[turnResponse](t, rec)
if len(got.URIs) != 1 || got.TTL != 3600 {
t.Fatalf("response = %+v", got)
}
expiry, user, ok := strings.Cut(got.Username, ":")
if !ok || user != acc.UserID {
t.Fatalf("username = %q, want <expiry>:%s", got.Username, acc.UserID)
}
if n, err := strconv.ParseInt(expiry, 10, 64); err != nil || n < before+3600 || n > before+3600+60 {
t.Errorf("expiry = %q, want about an hour from now", expiry)
}
mac := hmac.New(sha1.New, []byte(secret))
mac.Write([]byte(got.Username))
if want := base64.StdEncoding.EncodeToString(mac.Sum(nil)); got.Password != want {
t.Errorf("password = %q, want %q", got.Password, want)
}
}
func TestTurnServerStatic(t *testing.T) {
ts := newTestServer(t, func(o *clientapi.Options) {
o.TURN = config.TURN{
URIs: []string{"turns:turn.warp.example:5349", "stun:turn.warp.example:3478"},
Username: "warp",
Password: "hunter2",
}
})
acc := ts.register("alice", "correct horse battery")
got := decode[turnResponse](t, ts.get(turnPath, acc.AccessToken))
if len(got.URIs) != 2 || got.Username != "warp" || got.Password != "hunter2" || got.TTL <= 0 {
t.Errorf("response = %+v", got)
}
}
+39
View File
@@ -178,6 +178,36 @@ type ClientAPI struct {
RefreshTokenLifetime time.Duration `yaml:"refresh_token_lifetime" env:"WARP_REFRESH_TOKEN_LIFETIME"`
RateLimit RateLimit `yaml:"rate_limit"`
// TURN is what GET /voip/turnServer hands clients for voice and video
// calls.
TURN TURN `yaml:"turn"`
}
// TURN configures the relay servers clients use to set up calls. With no
// URIs the endpoint answers an empty object, and calls only connect where
// the two browsers can reach each other directly.
//
// Credentials come one of two ways. A shared secret is coturn's
// `use-auth-secret` (the TURN REST API): every user gets a username that
// expires and a password derived from it, and nothing has to be kept in
// step with the TURN server. A static username and password are handed to
// every user as they are, and are only for servers that cannot do the first.
type TURN struct {
// URIs are the turn:, turns: and stun: URIs, e.g.
// "turn:turn.example:3478?transport=udp".
URIs []string `yaml:"uris" env:"WARP_TURN_URIS"`
// SharedSecret is coturn's static-auth-secret.
SharedSecret string `yaml:"shared_secret" env:"WARP_TURN_SHARED_SECRET"`
// UserLifetime is how long credentials made from SharedSecret last.
UserLifetime time.Duration `yaml:"user_lifetime" env:"WARP_TURN_USER_LIFETIME"`
// Username and Password are static credentials, used when there is no
// SharedSecret.
Username string `yaml:"username" env:"WARP_TURN_USERNAME"`
Password string `yaml:"password" env:"WARP_TURN_PASSWORD"`
}
// RateLimit configures the per-endpoint token buckets in front of the C-S
@@ -500,6 +530,7 @@ func Default() *Config {
// Avatars: a client opening a room asks for dozens at once.
Media: RateLimitBucket{Burst: 100, Interval: 20 * time.Millisecond},
},
TURN: TURN{UserLifetime: time.Hour},
},
Sync: Sync{
Presence: false,
@@ -675,6 +706,14 @@ func (c *Config) Validate() error {
}
}
}
for i, uri := range c.ClientAPI.TURN.URIs {
if !strings.HasPrefix(uri, "turn:") && !strings.HasPrefix(uri, "turns:") && !strings.HasPrefix(uri, "stun:") {
bad("client_api.turn.uris[%d]: must be a turn:, turns: or stun: URI, got %q", i, uri)
}
}
if c.ClientAPI.TURN.SharedSecret != "" && c.ClientAPI.TURN.UserLifetime <= 0 {
bad("client_api.turn.user_lifetime: must be positive")
}
if c.Sync.TypingTimeout <= 0 {
bad("sync.typing_timeout: must be positive")
}
+1
View File
@@ -489,6 +489,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error)
RegistrationTokens: s.regTokens,
AppServices: s.appservices,
RateLimit: cfg.ClientAPI.RateLimit,
TURN: cfg.ClientAPI.TURN,
TrustXForwardedFor: cfg.ClientAPI.TrustXForwardedFor,
Logger: log,
})
+17
View File
@@ -103,6 +103,23 @@ client_api:
# 0 means no expiry. WARP_REFRESH_TOKEN_LIFETIME
refresh_token_lifetime: 0s
# Relay servers for voice and video calls, handed to clients by
# GET /voip/turnServer. With no uris, calls only connect where the two
# browsers can reach each other directly (the same network, mostly).
#
# With shared_secret (coturn's `use-auth-secret` + `static-auth-secret`)
# every user gets credentials that expire after user_lifetime. username
# and password are static credentials for servers that cannot do that.
turn:
uris: [] # WARP_TURN_URIS
# - "turn:turn.example.org:3478?transport=udp"
# - "turn:turn.example.org:3478?transport=tcp"
# - "turns:turn.example.org:5349?transport=tcp"
shared_secret: "" # WARP_TURN_SHARED_SECRET
user_lifetime: 1h # WARP_TURN_USER_LIFETIME
username: "" # WARP_TURN_USERNAME
password: "" # WARP_TURN_PASSWORD
# Per-endpoint rate limiting. Each bucket allows `burst` requests back
# to back and returns one slot every `interval`, per client address.
# Login and registration get their own, tighter, buckets because they