Group voice and video calls with MatrixRTC and LiveKit
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Admin panel (push) Canceled after 0s
CI / Web client (push) Canceled after 0s
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Admin panel (push) Canceled after 0s
CI / Web client (push) Canceled after 0s
Every call in the web client now goes through MatrixRTC with a LiveKit
SFU, the model Element Call and Element X use; the legacy 1:1 m.call.*
calls are gone. In a room of two the other person is rung and can answer
or decline; in a bigger room a banner offers to join the call. Calls show
a grid of participant tiles with mute, camera, screen sharing and leave,
and media is end-to-end encrypted in encrypted rooms.
Server:
- MSC4140 delayed events (send/state with a delay, plus the
delayed_events list/restart/cancel/send API), stored and scheduled, so
a member whose client drops off is taken out of the call.
- POST /user/{userId}/openid/request_token, which lk-jwt-service uses to
hand out LiveKit tokens.
- org.matrix.msc4143.rtc_foci in /.well-known/matrix/client, configured
under client_api.rtc.
- Room presets let members send call membership state.
- Migration 0014 for both SQL dialects.
Deploy: LiveKit and lk-jwt-service in compose (calls.yml) and the helm
chart. Docs: docs/calls.md, configuration and deployment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
325431a5be
commit
76dfc98a86
49 files changed
+3849
-432
No files matched your search
+18
-6
@@ -63,12 +63,24 @@ that one origin.
|
||||
them asks you to acknowledge it; a reset since you *verified* them says
|
||||
their sessions are no longer trusted, with Verify again or Withdraw
|
||||
verification.
|
||||
- Voice and video calls in a room with one other person (Matrix 1:1 VoIP,
|
||||
WebRTC between the browsers): ringing, answer with or without video,
|
||||
mute, camera on and off, screen sharing, and a panel that stays up while
|
||||
you move between rooms and expands to fill the window. In an encrypted
|
||||
room the call signalling is encrypted too. Calls outside one network
|
||||
need a TURN server; see `client_api.turn` in the server configuration.
|
||||
- Voice and video calls in any room, all through a LiveKit SFU (MatrixRTC,
|
||||
the Element Call model; the old Matrix 1:1 VoIP is not used). Voice and
|
||||
video call buttons in the room header. In a room with one other person
|
||||
the call rings them: an incoming-call card with ring tone, answer, answer
|
||||
without video and decline, and "Calling…" with a ringback tone for the
|
||||
caller until they answer, decline or a minute passes. A call of two shows
|
||||
the other person large and you in the corner; a bigger one is a grid
|
||||
with a tile per person, their camera or screen, and a ring around
|
||||
whoever is speaking. Mute, camera, screen sharing, leave, and a panel
|
||||
that stays up while you move between rooms. A "Join call" banner while
|
||||
one is going on in a room you're not in it. In an encrypted room the media
|
||||
is end-to-end encrypted with per-participant keys. The room list says
|
||||
when a room has a call going on, and the timeline says when one started
|
||||
and ended. It needs `client_api.rtc` on the server and a LiveKit JWT
|
||||
service; see [docs/calls.md](../../docs/calls.md). warp-web proxies
|
||||
`/.well-known/matrix/client` for this, and `-call-origins`
|
||||
(`WARP_WEB_CALL_ORIGINS`) lets the browser reach the JWT service and
|
||||
LiveKit through the Content-Security-Policy.
|
||||
- Encryption setup in Settings: cross-signing keys, secret storage and a
|
||||
key backup in one step, behind a recovery key shown once. A new session
|
||||
verifies itself with the recovery key (or another session) and restores
|
||||
|
||||
@@ -54,6 +54,8 @@ func run() error {
|
||||
"base URL of the homeserver, proxied for the client (WARP_WEB_HOMESERVER)")
|
||||
public := flag.String("public-homeserver", env("WARP_WEB_PUBLIC_HOMESERVER", ""),
|
||||
"public URL of the homeserver; when set, the browser calls it directly and nothing is proxied (WARP_WEB_PUBLIC_HOMESERVER)")
|
||||
callOrigins := flag.String("call-origins", env("WARP_WEB_CALL_ORIGINS", ""),
|
||||
"space-separated origins of the LiveKit JWT service and SFU the browser may connect to for group calls (WARP_WEB_CALL_ORIGINS)")
|
||||
logFormat := flag.String("log-format", env("WARP_WEB_LOG_FORMAT", "text"), "text or json (WARP_WEB_LOG_FORMAT)")
|
||||
showVersion := flag.Bool("version", false, "print the version and exit")
|
||||
flag.Parse()
|
||||
@@ -74,7 +76,7 @@ func run() error {
|
||||
}
|
||||
slog.SetDefault(logger)
|
||||
|
||||
opts := host.Options{Logger: logger, Version: version}
|
||||
opts := host.Options{Logger: logger, Version: version, CallOrigins: strings.Fields(*callOrigins)}
|
||||
mode, target := "proxy", *homeserver
|
||||
if *public != "" {
|
||||
u, err := parseURL("public-homeserver", *public)
|
||||
|
||||
@@ -41,6 +41,11 @@ type Options struct {
|
||||
Logger *slog.Logger
|
||||
// Transport overrides the proxy's round tripper, for tests.
|
||||
Transport http.RoundTripper
|
||||
// CallOrigins are origins outside the homeserver the browser may
|
||||
// connect to for group calls: the LiveKit JWT service and the LiveKit
|
||||
// SFU's WebSocket, e.g. https://livekit-jwt.example and
|
||||
// wss://livekit.example. They are added to the CSP's connect-src.
|
||||
CallOrigins []string
|
||||
}
|
||||
|
||||
// proxiedPrefixes are the parts of the homeserver a Matrix client uses: the
|
||||
@@ -48,6 +53,10 @@ type Options struct {
|
||||
// are not reachable through warp-web.
|
||||
var proxiedPrefixes = []string{"/_matrix/client/", "/_matrix/media/"}
|
||||
|
||||
// wellKnownClientPath is proxied too: it is where the client finds the
|
||||
// group call backend (org.matrix.msc4143.rtc_foci).
|
||||
const wellKnownClientPath = "/.well-known/matrix/client"
|
||||
|
||||
// Config is the body of /config.json, which the client reads on start.
|
||||
type Config struct {
|
||||
// HomeserverURL is where the client sends Matrix requests; empty
|
||||
@@ -78,6 +87,22 @@ func New(opts Options) (http.Handler, error) {
|
||||
cfg.HomeserverURL = strings.TrimRight(u.String(), "/")
|
||||
connect += " " + u.Scheme + "://" + u.Host
|
||||
}
|
||||
for _, raw := range opts.CallOrigins {
|
||||
u, err := url.Parse(strings.TrimRight(raw, "/"))
|
||||
if err != nil || u.Host == "" || u.Path != "" || u.RawQuery != "" ||
|
||||
(u.Scheme != "http" && u.Scheme != "https" && u.Scheme != "ws" && u.Scheme != "wss") {
|
||||
return nil, errors.New("host: a call origin must be an http(s) or ws(s) origin, got " + raw)
|
||||
}
|
||||
connect += " " + u.Scheme + "://" + u.Host
|
||||
// LiveKit falls back from its WebSocket to an HTTP request on
|
||||
// the same host to find out why a connection failed.
|
||||
switch u.Scheme {
|
||||
case "ws":
|
||||
connect += " http://" + u.Host
|
||||
case "wss":
|
||||
connect += " https://" + u.Host
|
||||
}
|
||||
}
|
||||
body, err := json.Marshal(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -167,6 +192,10 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte("ok\n"))
|
||||
return
|
||||
|
||||
case p == wellKnownClientPath && h.proxy != nil:
|
||||
h.proxy.ServeHTTP(w, r)
|
||||
return
|
||||
|
||||
case strings.HasPrefix(p, "/_matrix/") || strings.HasPrefix(p, "/_warp/") || strings.HasPrefix(p, "/_synapse/"):
|
||||
// A path that cleans to something else (dot segments, doubled
|
||||
// slashes) is refused rather than normalised: what is allowed
|
||||
|
||||
@@ -304,3 +304,35 @@ func TestNewValidatesHomeserver(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestProxiesWellKnownClient(t *testing.T) {
|
||||
up := newUpstream(t)
|
||||
h := newHandler(t, up, true)
|
||||
rec := do(h, http.MethodGet, "/.well-known/matrix/client", nil)
|
||||
if rec.Code != 200 || len(up.hits) != 1 || up.hits[0].URL.Path != "/.well-known/matrix/client" {
|
||||
t.Fatalf("status %d, hits %d", rec.Code, len(up.hits))
|
||||
}
|
||||
// The server delegation document stays private.
|
||||
if rec := do(h, http.MethodGet, "/.well-known/matrix/server", nil); rec.Code == 200 && len(up.hits) > 1 {
|
||||
t.Errorf("/.well-known/matrix/server was proxied")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallOriginsJoinConnectSrc(t *testing.T) {
|
||||
up := newUpstream(t)
|
||||
target, _ := url.Parse(up.srv.URL)
|
||||
h, err := New(Options{Homeserver: target, Client: clientFS,
|
||||
CallOrigins: []string{"https://livekit-jwt.warp.example/", "wss://livekit.warp.example"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rec := do(h, http.MethodGet, "/", nil)
|
||||
want := "connect-src 'self' https://livekit-jwt.warp.example wss://livekit.warp.example https://livekit.warp.example;"
|
||||
if csp := rec.Header().Get("Content-Security-Policy"); !strings.Contains(csp, want) {
|
||||
t.Errorf("CSP %q", csp)
|
||||
}
|
||||
|
||||
if _, err := New(Options{Homeserver: target, CallOrigins: []string{"https://x.example/path"}}); err == nil {
|
||||
t.Error("a call origin with a path was accepted")
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@
|
||||
"dependencies": {
|
||||
"@warp/ui": "workspace:*",
|
||||
"jsqr": "1.4.0",
|
||||
"livekit-client": "2.22.3",
|
||||
"matrix-js-sdk": "^44.0.0",
|
||||
"pinia": "^3.0.3",
|
||||
"qrcode-generator": "1.4.4",
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
<script setup lang="ts">
|
||||
import type { Track } from 'livekit-client'
|
||||
import { computed, onBeforeUnmount, ref, watch } from 'vue'
|
||||
import { WIcon } from '@warp/ui'
|
||||
import MxcAvatar from '@/components/MxcAvatar.vue'
|
||||
import type { Tile } from '@/stores/call'
|
||||
|
||||
// One person in a group call: their screen or camera, else their avatar,
|
||||
// with their name, a muted microphone, and a ring while they speak.
|
||||
|
||||
const props = defineProps<{ tile: Tile; compact?: boolean }>()
|
||||
|
||||
const videoEl = ref<HTMLVideoElement>()
|
||||
const audioEl = ref<HTMLAudioElement>()
|
||||
|
||||
const shown = computed(() => props.tile.screen ?? props.tile.camera)
|
||||
// Your own camera is mirrored, the way a mirror (and every call app) shows you.
|
||||
const mirrored = computed(() => props.tile.local && !props.tile.screen)
|
||||
|
||||
function bind(el: HTMLMediaElement | undefined, track: Track | undefined, old: Track | undefined) {
|
||||
if (!el) return
|
||||
if (old && old !== track) old.detach(el)
|
||||
if (track) track.attach(el)
|
||||
}
|
||||
|
||||
watch([videoEl, shown], ([el, track], [, old]) => bind(el, track, old), { flush: 'post', immediate: true })
|
||||
watch([audioEl, () => props.tile.audio], ([el, track], [, old]) => bind(el, track, old), { flush: 'post', immediate: true })
|
||||
|
||||
onBeforeUnmount(() => {
|
||||
if (videoEl.value) shown.value?.detach(videoEl.value)
|
||||
if (audioEl.value) props.tile.audio?.detach(audioEl.value)
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div
|
||||
:class="[
|
||||
'relative flex min-h-0 items-center justify-center overflow-hidden rounded-md bg-[#1d2126] transition-shadow',
|
||||
tile.speaking ? 'shadow-[0_0_0_3px_#3fb68b]' : '',
|
||||
]"
|
||||
>
|
||||
<audio v-if="tile.audio" ref="audioEl" autoplay class="hidden" />
|
||||
<video
|
||||
v-show="shown"
|
||||
ref="videoEl"
|
||||
autoplay
|
||||
playsinline
|
||||
muted
|
||||
:class="['h-full w-full', tile.screen ? 'object-contain' : 'object-cover', mirrored && '-scale-x-100']"
|
||||
/>
|
||||
<MxcAvatar v-if="!shown" :name="tile.name" :mxc="tile.avatarMxc" :size="compact ? 48 : 88" />
|
||||
<div class="absolute bottom-2 left-2 flex max-w-[calc(100%-1rem)] items-center gap-1 rounded-sm bg-[#000000a6] px-2 py-0.5">
|
||||
<WIcon v-if="tile.micMuted" name="mic-off" :size="14" label="Muted" class="shrink-0 text-[#ff8a80]" />
|
||||
<WIcon v-if="tile.screen" name="screen-share" :size="14" label="Sharing their screen" class="shrink-0" />
|
||||
<span class="caption truncate text-[#eceeee]">{{ tile.local ? `${tile.name} (you)` : tile.name }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
@@ -1,108 +1,74 @@
|
||||
<script setup lang="ts">
|
||||
import type { CallFeed } from 'matrix-js-sdk/lib/webrtc/callFeed'
|
||||
import { computed, onBeforeUnmount, ref, watch, watchEffect } from 'vue'
|
||||
import { computed, onBeforeUnmount, ref, watch } from 'vue'
|
||||
import { useRouter } from 'vue-router'
|
||||
import { WIcon } from '@warp/ui'
|
||||
import { WIcon, WSpinner } from '@warp/ui'
|
||||
import type { IconName } from '@warp/ui'
|
||||
import CallTile from '@/components/CallTile.vue'
|
||||
import MxcAvatar from '@/components/MxcAvatar.vue'
|
||||
import { formatDuration } from '@/lib/format'
|
||||
import { useCall } from '@/stores/call'
|
||||
import { useCall, type Tile } from '@/stores/call'
|
||||
|
||||
// The call on screen: a ringing card for an incoming call, else a panel in
|
||||
// the corner that a video call can expand to fill the window. It sits above
|
||||
// every view, so a call goes on while you move between rooms.
|
||||
// The call on screen: a ringing card for a call coming in, else the call
|
||||
// you're in. A call of two shows the other person large and you in the
|
||||
// corner; a bigger one is a grid. The panel shrinks to the corner so the
|
||||
// call goes on while you read and write in your rooms, and sits above
|
||||
// every view.
|
||||
|
||||
const call = useCall()
|
||||
const router = useRouter()
|
||||
const expanded = ref(false)
|
||||
|
||||
function feedVideo(feed: CallFeed | undefined): MediaStream | undefined {
|
||||
if (!feed || feed.isVideoMuted() || !feed.stream.getVideoTracks().length) return undefined
|
||||
return feed.stream
|
||||
}
|
||||
|
||||
const view = computed(() => {
|
||||
void call.version
|
||||
const c = call.call
|
||||
if (!c) return null
|
||||
const remoteCamera = feedVideo(c.remoteUsermediaFeed)
|
||||
const remoteScreen = c.remoteScreensharingStream
|
||||
const localCamera = feedVideo(c.localUsermediaFeed)
|
||||
const localScreen = c.localScreensharingStream
|
||||
// Their screen beats their camera on the big stage; whatever is left of
|
||||
// theirs or yours goes in the small picture.
|
||||
const main = remoteScreen ?? remoteCamera
|
||||
const pip = remoteScreen && remoteCamera ? remoteCamera : (localScreen ?? localCamera)
|
||||
return {
|
||||
main,
|
||||
pip,
|
||||
pipMirrored: pip !== undefined && pip === localCamera,
|
||||
remoteAudio: c.remoteUsermediaStream,
|
||||
micMuted: c.isMicrophoneMuted(),
|
||||
cameraOn: Boolean(localCamera),
|
||||
sharing: c.isScreensharing(),
|
||||
remoteOnHold: c.isRemoteOnHold(),
|
||||
localOnHold: c.isLocalOnHold(),
|
||||
roomId: c.roomId,
|
||||
}
|
||||
})
|
||||
|
||||
const stage = computed(() => call.phase !== 'ringing' && Boolean(view.value?.main || view.value?.pip || call.isVideo))
|
||||
const canShare = typeof navigator !== 'undefined' && Boolean(navigator.mediaDevices?.getDisplayMedia)
|
||||
|
||||
const status = computed(() => {
|
||||
const v = view.value
|
||||
switch (call.phase) {
|
||||
case 'ringing':
|
||||
return call.isVideo ? 'Incoming video call' : 'Incoming voice call'
|
||||
case 'calling':
|
||||
return 'Calling…'
|
||||
case 'connecting':
|
||||
return 'Connecting…'
|
||||
case 'connected':
|
||||
if (v?.localOnHold) return 'You’re on hold'
|
||||
if (v?.remoteOnHold) return 'On hold'
|
||||
return formatDuration(call.length)
|
||||
case 'ended':
|
||||
return call.endReason
|
||||
default:
|
||||
return ''
|
||||
}
|
||||
})
|
||||
|
||||
// Streams go to the media elements by hand: srcObject is not an attribute.
|
||||
const mainEl = ref<HTMLVideoElement>()
|
||||
const pipEl = ref<HTMLVideoElement>()
|
||||
const audioEl = ref<HTMLAudioElement>()
|
||||
|
||||
function bind(el: HTMLMediaElement | undefined, stream: MediaStream | undefined) {
|
||||
if (!el || el.srcObject === (stream ?? null)) return
|
||||
el.srcObject = stream ?? null
|
||||
if (stream) void el.play().catch(() => undefined)
|
||||
}
|
||||
watchEffect(() => bind(mainEl.value, view.value?.main), { flush: 'post' })
|
||||
watchEffect(() => bind(pipEl.value, view.value?.pip), { flush: 'post' })
|
||||
watchEffect(() => bind(audioEl.value, view.value?.remoteAudio), { flush: 'post' })
|
||||
const expanded = ref(true)
|
||||
|
||||
watch(
|
||||
() => call.call,
|
||||
(c) => {
|
||||
if (!c) expanded.value = false
|
||||
() => call.roomId,
|
||||
(rid) => {
|
||||
// Video calls and group calls open full size; a voice call between
|
||||
// two people starts as the panel in the corner.
|
||||
if (rid) expanded.value = call.video || !call.callee
|
||||
},
|
||||
)
|
||||
|
||||
function onKey(e: KeyboardEvent) {
|
||||
if (e.key === 'Escape' && expanded.value) expanded.value = false
|
||||
}
|
||||
window.addEventListener('keydown', onKey)
|
||||
onBeforeUnmount(() => window.removeEventListener('keydown', onKey))
|
||||
const remotes = computed(() => call.tiles.filter((t) => !t.local))
|
||||
const self = computed(() => call.tiles.find((t) => t.local))
|
||||
/** A call of two: the other person on the stage, you in the corner. */
|
||||
const pair = computed(() => remotes.value.length === 1)
|
||||
/** Ringing them, nobody here yet. */
|
||||
const calling = computed(() => call.ringingOut && remotes.value.length === 0)
|
||||
|
||||
function openRoom() {
|
||||
const roomId = view.value?.roomId
|
||||
if (!roomId) return
|
||||
expanded.value = false
|
||||
router.push({ name: 'room', params: { roomId } })
|
||||
}
|
||||
// Whoever shares their screen takes the stage; otherwise the person
|
||||
// speaking comes first, so the panel in the corner shows them.
|
||||
const ordered = computed(() => {
|
||||
const rank = (t: Tile) => (t.screen ? 0 : t.speaking && !t.local ? 1 : t.local ? 3 : 2)
|
||||
return [...call.tiles].sort((a, b) => rank(a) - rank(b))
|
||||
})
|
||||
const shown = computed(() => (expanded.value ? ordered.value : ordered.value.slice(0, 1)))
|
||||
// The rest still have to be heard while the panel shows only one.
|
||||
const hidden = computed(() => (expanded.value ? [] : ordered.value.slice(1).filter((t) => t.audio)))
|
||||
|
||||
const columns = computed(() => {
|
||||
const n = shown.value.length
|
||||
if (n <= 1) return 'grid-cols-1'
|
||||
if (n <= 4) return 'grid-cols-1 sm:grid-cols-2'
|
||||
if (n <= 9) return 'grid-cols-2 lg:grid-cols-3'
|
||||
return 'grid-cols-2 md:grid-cols-3 xl:grid-cols-4'
|
||||
})
|
||||
|
||||
const status = computed(() => {
|
||||
switch (call.phase) {
|
||||
case 'joining':
|
||||
return calling.value ? 'Calling…' : 'Joining…'
|
||||
case 'reconnecting':
|
||||
return 'Reconnecting…'
|
||||
default: {
|
||||
if (calling.value) return 'Calling…'
|
||||
const time = formatDuration(call.length)
|
||||
if (call.callee) return time
|
||||
const n = call.tiles.length
|
||||
return n === 1 ? `Only you so far · ${time}` : `${n} people · ${time}`
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
const canShare = typeof navigator !== 'undefined' && Boolean(navigator.mediaDevices?.getDisplayMedia)
|
||||
|
||||
interface Control {
|
||||
icon: IconName
|
||||
@@ -113,23 +79,28 @@ interface Control {
|
||||
}
|
||||
|
||||
const controls = computed<Control[]>(() => {
|
||||
const v = view.value
|
||||
if (!v) return []
|
||||
if (call.phase === 'ringing') {
|
||||
const list: Control[] = [{ icon: 'phone-off', label: 'Decline', action: call.decline, tone: 'danger' }]
|
||||
if (call.isVideo) list.push({ icon: 'phone', label: 'Answer without video', action: () => call.answer(false) })
|
||||
list.push({ icon: call.isVideo ? 'video' : 'phone', label: 'Answer', action: () => call.answer(), tone: 'accept' })
|
||||
return list
|
||||
}
|
||||
if (call.phase === 'ended') return [{ icon: 'close', label: 'Close', action: call.hangup }]
|
||||
const list: Control[] = [
|
||||
{ icon: v.micMuted ? 'mic-off' : 'mic', label: v.micMuted ? 'Unmute' : 'Mute', action: call.toggleMic, on: v.micMuted },
|
||||
{ icon: v.cameraOn ? 'video' : 'video-off', label: v.cameraOn ? 'Turn camera off' : 'Turn camera on', action: call.toggleCamera, on: !v.cameraOn && call.isVideo },
|
||||
{ icon: call.micOn ? 'mic' : 'mic-off', label: call.micOn ? 'Mute' : 'Unmute', action: () => call.toggle('mic'), on: !call.micOn },
|
||||
{
|
||||
icon: call.cameraOn ? 'video' : 'video-off',
|
||||
label: call.cameraOn ? 'Turn camera off' : 'Turn camera on',
|
||||
action: () => call.toggle('camera'),
|
||||
on: !call.cameraOn,
|
||||
},
|
||||
]
|
||||
if (canShare && call.phase === 'connected') {
|
||||
list.push({ icon: 'screen-share', label: v.sharing ? 'Stop sharing' : 'Share screen', action: call.toggleScreenShare, on: v.sharing })
|
||||
list.push({ icon: 'screen-share', label: call.sharing ? 'Stop sharing' : 'Share screen', action: () => call.toggle('screen'), on: call.sharing })
|
||||
}
|
||||
list.push({ icon: 'phone-off', label: 'Hang up', action: call.hangup, tone: 'danger' })
|
||||
list.push({ icon: 'phone-off', label: calling.value ? 'Cancel call' : 'Leave call', action: () => void call.leave(), tone: 'danger' })
|
||||
return list
|
||||
})
|
||||
|
||||
const ringControls = computed<Control[]>(() => {
|
||||
const c = call.incoming
|
||||
if (!c) return []
|
||||
const list: Control[] = [{ icon: 'phone-off', label: 'Decline', action: () => void call.decline(), tone: 'danger' }]
|
||||
if (c.video) list.push({ icon: 'phone', label: 'Answer without video', action: () => void call.answer(false) })
|
||||
list.push({ icon: c.video ? 'video' : 'phone', label: 'Answer', action: () => void call.answer(), tone: 'accept' })
|
||||
return list
|
||||
})
|
||||
|
||||
@@ -139,37 +110,70 @@ function controlClass(c: Control): string {
|
||||
if (c.on) return 'bg-[#eceeee] text-[#111418] hover:bg-[#ffffff]'
|
||||
return 'bg-[#ffffff1f] text-[#eceeee] hover:bg-[#ffffff33]'
|
||||
}
|
||||
|
||||
function openRoom() {
|
||||
const roomId = call.roomId
|
||||
if (!roomId) return
|
||||
expanded.value = false
|
||||
router.push({ name: 'room', params: { roomId } })
|
||||
}
|
||||
|
||||
function onKey(e: KeyboardEvent) {
|
||||
if (e.key === 'Escape' && expanded.value) expanded.value = false
|
||||
}
|
||||
window.addEventListener('keydown', onKey)
|
||||
onBeforeUnmount(() => window.removeEventListener('keydown', onKey))
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section
|
||||
v-if="call.call && view"
|
||||
v-if="call.incoming && !call.joined"
|
||||
class="fixed right-4 bottom-4 left-4 z-40 flex flex-col overflow-hidden rounded-lg bg-[#111418] text-[#eceeee] shadow-float sm:left-auto sm:w-[360px]"
|
||||
:aria-label="`Call from ${call.incoming.callerName}`"
|
||||
role="alertdialog"
|
||||
>
|
||||
<div class="flex items-center gap-3 px-4 pt-4">
|
||||
<MxcAvatar :name="call.incoming.callerName" :mxc="call.incoming.callerAvatarMxc" :size="48" />
|
||||
<div class="min-w-0 flex-1">
|
||||
<span class="body-strong block truncate">{{ call.incoming.callerName }}</span>
|
||||
<span class="small block truncate text-[#eceeeeb3]">{{ call.incoming.video ? 'Incoming video call' : 'Incoming voice call' }}</span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center justify-center gap-3 px-4 pt-3 pb-4">
|
||||
<button
|
||||
v-for="c in ringControls"
|
||||
:key="c.label"
|
||||
type="button"
|
||||
:class="['flex h-12 w-12 cursor-pointer items-center justify-center rounded-full border-0 transition-colors', controlClass(c)]"
|
||||
:aria-label="c.label"
|
||||
:title="c.label"
|
||||
@click="c.action()"
|
||||
>
|
||||
<WIcon :name="c.icon" :size="22" />
|
||||
</button>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section
|
||||
v-if="call.roomId"
|
||||
:class="[
|
||||
'fixed z-40 flex flex-col overflow-hidden bg-[#111418] text-[#eceeee] shadow-float',
|
||||
expanded ? 'inset-0' : 'right-4 bottom-4 left-4 rounded-lg sm:left-auto sm:w-[360px]',
|
||||
]"
|
||||
:aria-label="`Call with ${call.peerName}`"
|
||||
:role="call.phase === 'ringing' ? 'alertdialog' : 'region'"
|
||||
:aria-label="`Call: ${call.title}`"
|
||||
role="region"
|
||||
>
|
||||
<audio ref="audioEl" autoplay class="hidden" />
|
||||
|
||||
<div v-if="stage" :class="['relative bg-[#000000]', expanded ? 'min-h-0 flex-1' : 'aspect-video']">
|
||||
<video v-show="view.main" ref="mainEl" autoplay playsinline muted class="h-full w-full object-contain" />
|
||||
<div v-if="!view.main" class="absolute inset-0 flex items-center justify-center">
|
||||
<MxcAvatar :name="call.peerName" :mxc="call.peerAvatarMxc" :size="expanded ? 120 : 72" />
|
||||
</div>
|
||||
<video
|
||||
v-show="view.pip"
|
||||
ref="pipEl"
|
||||
autoplay
|
||||
playsinline
|
||||
muted
|
||||
:class="['absolute right-3 bottom-3 w-1/4 min-w-24 rounded-md bg-[#000000] object-cover shadow-float', view.pipMirrored && '-scale-x-100']"
|
||||
/>
|
||||
<div class="flex items-center gap-3 px-4 pt-3">
|
||||
<button type="button" class="min-w-0 flex-1 cursor-pointer border-0 bg-transparent p-0 text-left text-[#eceeee]" @click="openRoom">
|
||||
<span class="body-strong flex items-center gap-2 truncate">
|
||||
{{ call.title }}
|
||||
<WIcon v-if="call.encrypted" name="lock" :size="14" label="End-to-end encrypted" class="shrink-0 text-[#3fb68b]" />
|
||||
</span>
|
||||
<span class="small block truncate text-[#eceeeeb3] tabular-nums" aria-live="polite">{{ status }}</span>
|
||||
</button>
|
||||
<button
|
||||
v-if="call.isVideo || view.main"
|
||||
type="button"
|
||||
class="absolute top-2 right-2 flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-0 bg-[#00000080] text-[#eceeee] hover:bg-[#000000b3]"
|
||||
class="flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-0 bg-[#ffffff1f] text-[#eceeee] hover:bg-[#ffffff33]"
|
||||
:aria-label="expanded ? 'Shrink' : 'Expand'"
|
||||
:title="expanded ? 'Shrink' : 'Expand'"
|
||||
@click="expanded = !expanded"
|
||||
@@ -178,12 +182,34 @@ function controlClass(c: Control): string {
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="flex items-center gap-3 px-4 pt-4">
|
||||
<MxcAvatar v-if="!stage" :name="call.peerName" :mxc="call.peerAvatarMxc" :size="48" />
|
||||
<button type="button" class="min-w-0 flex-1 cursor-pointer border-0 bg-transparent p-0 text-left text-[#eceeee]" @click="openRoom">
|
||||
<span class="body-strong block truncate">{{ call.peerName }}</span>
|
||||
<span class="small block truncate text-[#eceeeeb3] tabular-nums" aria-live="polite">{{ status }}</span>
|
||||
</button>
|
||||
<div :class="['relative px-4 pt-3', expanded ? 'min-h-0 flex-1' : '']">
|
||||
<!-- Joining, or ringing someone who hasn't come yet. -->
|
||||
<div
|
||||
v-if="call.phase === 'joining' || calling"
|
||||
:class="['flex flex-col items-center justify-center gap-4', expanded ? 'h-full' : 'aspect-video']"
|
||||
>
|
||||
<MxcAvatar v-if="call.callee" :name="call.callee.name" :mxc="call.callee.avatarMxc" :size="expanded ? 120 : 72" />
|
||||
<WSpinner v-else :size="32" />
|
||||
</div>
|
||||
|
||||
<!-- Two of you: them on the stage, you in the corner. -->
|
||||
<div v-else-if="pair && remotes[0]" :class="['relative', expanded ? 'h-full' : 'aspect-video']">
|
||||
<CallTile :tile="remotes[0]" :compact="!expanded" class="h-full w-full" />
|
||||
<CallTile
|
||||
v-if="self"
|
||||
:tile="self"
|
||||
compact
|
||||
class="absolute right-3 bottom-3 aspect-video w-1/4 min-w-28 shadow-float"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<!-- More: a grid, or whoever is speaking when shrunk. -->
|
||||
<template v-else>
|
||||
<div :class="['grid gap-2', expanded ? `h-full auto-rows-fr ${columns}` : 'aspect-video grid-cols-1']">
|
||||
<CallTile v-for="t in shown" :key="t.id" :tile="t" :compact="!expanded" />
|
||||
</div>
|
||||
<CallTile v-for="t in hidden" :key="t.id" :tile="t" class="hidden" />
|
||||
</template>
|
||||
</div>
|
||||
|
||||
<div class="flex items-center justify-center gap-3 px-4 pt-3 pb-4">
|
||||
@@ -193,7 +219,7 @@ function controlClass(c: Control): string {
|
||||
type="button"
|
||||
:class="['flex h-12 w-12 cursor-pointer items-center justify-center rounded-full border-0 transition-colors', controlClass(c)]"
|
||||
:aria-label="c.label"
|
||||
:aria-pressed="c.on === undefined || c.tone ? undefined : c.on"
|
||||
:aria-pressed="c.tone ? undefined : c.on"
|
||||
:title="c.label"
|
||||
@click="c.action()"
|
||||
>
|
||||
|
||||
@@ -83,6 +83,13 @@ watchEffect(() => {
|
||||
document.title = `${n ? `(${n}) ` : ''}${name ? `${name} · ` : ''}Warp`
|
||||
})
|
||||
|
||||
// A room with a call going on says so in the list, instead of its last message.
|
||||
function callPreview(roomId: string): string {
|
||||
if (call.roomId === roomId) return 'You’re in the call'
|
||||
const n = call.inCall(roomId)
|
||||
return n === 1 ? 'Call in progress: 1 person' : `Call in progress: ${n} people`
|
||||
}
|
||||
|
||||
const ownName = computed(() => profile.displayName || localpart(session.userId))
|
||||
</script>
|
||||
|
||||
@@ -166,7 +173,7 @@ const ownName = computed(() => profile.displayName || localpart(session.userId))
|
||||
:key="r.roomId"
|
||||
:name="r.name"
|
||||
:avatar-mxc="r.avatarMxc"
|
||||
:preview="r.preview"
|
||||
:preview="call.inCall(r.roomId) ? callPreview(r.roomId) : r.preview"
|
||||
:time="formatListTime(r.ts)"
|
||||
:unread="r.unread"
|
||||
:mention="r.highlight > 0"
|
||||
|
||||
@@ -68,6 +68,9 @@ export async function startClient(creds: Credentials): Promise<MatrixClient> {
|
||||
accessToken: creds.accessToken,
|
||||
timelineSupport: true,
|
||||
useAuthorizationHeader: true,
|
||||
// Calls are MatrixRTC with LiveKit (stores/call.ts); the SDK's own
|
||||
// 1:1 VoIP, and its polling for TURN servers, stay off.
|
||||
disableVoip: true,
|
||||
cryptoCallbacks,
|
||||
// Emoji (SAS), and QR codes both ways: shown here, or scanned with
|
||||
// the camera.
|
||||
|
||||
@@ -114,6 +114,7 @@ function describeCall(ev: MatrixEvent, room: Room): string | null {
|
||||
case EventType.CallInvite:
|
||||
return `${sender} started a ${isVideoCallInvite(ev) ? 'video' : 'voice'} call`
|
||||
case EventType.CallReject:
|
||||
case EventType.RTCDecline:
|
||||
return `${sender} declined the call`
|
||||
case EventType.CallHangup:
|
||||
switch (ev.getContent().reason) {
|
||||
@@ -131,6 +132,47 @@ function describeCall(ev: MatrixEvent, room: Room): string | null {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a call.member state event (MatrixRTC) says its device is in the
|
||||
* call. Leaving is an empty event; the legacy form lists memberships.
|
||||
*/
|
||||
export function isActiveCallMember(content: Record<string, unknown>): boolean {
|
||||
if (Object.keys(content).length === 0) return false
|
||||
if (Array.isArray(content.memberships)) return content.memberships.length > 0
|
||||
return true
|
||||
}
|
||||
|
||||
/**
|
||||
* Follows who is in a room's group call along the timeline, to say when
|
||||
* it started and ended. Each device in the call has a call.member state
|
||||
* event of its own; a member refreshing theirs says nothing.
|
||||
*/
|
||||
class CallTracker {
|
||||
private readonly inCall = new Set<string>()
|
||||
|
||||
constructor(room: Room) {
|
||||
const start = room.getLiveTimeline().getState(Direction.Backward)
|
||||
for (const ev of start?.getStateEvents(EventType.GroupCallMemberPrefix) ?? []) {
|
||||
if (isActiveCallMember(ev.getContent())) this.inCall.add(ev.getStateKey() ?? '')
|
||||
}
|
||||
}
|
||||
|
||||
describe(ev: MatrixEvent, room: Room): string | null {
|
||||
const key = ev.getStateKey() ?? ''
|
||||
const was = this.inCall.has(key)
|
||||
const now = isActiveCallMember(ev.getContent())
|
||||
if (was === now) return null
|
||||
const sender = nameOf(room, ev.getSender() ?? '')
|
||||
if (now) {
|
||||
const first = this.inCall.size === 0
|
||||
this.inCall.add(key)
|
||||
return first ? `${sender} started a call` : `${sender} joined the call`
|
||||
}
|
||||
this.inCall.delete(key)
|
||||
return this.inCall.size === 0 ? 'The call ended' : `${sender} left the call`
|
||||
}
|
||||
}
|
||||
|
||||
/** A one-line description of a state change, or null to leave it out. */
|
||||
function describeState(ev: MatrixEvent, room: Room): string | null {
|
||||
const sender = nameOf(room, ev.getSender() ?? '')
|
||||
@@ -285,6 +327,7 @@ export function buildTimeline(c: MatrixClient, room: Room): TimelineItem[] {
|
||||
const items: TimelineItem[] = []
|
||||
let lastTs = 0
|
||||
let prev: MessageItem | null = null
|
||||
const calls = new CallTracker(room)
|
||||
|
||||
for (const ev of events) {
|
||||
const type = ev.getType()
|
||||
@@ -310,7 +353,7 @@ export function buildTimeline(c: MatrixClient, room: Room): TimelineItem[] {
|
||||
}
|
||||
continue
|
||||
}
|
||||
const text = describeState(ev, room)
|
||||
const text = type === EventType.GroupCallMemberPrefix ? calls.describe(ev, room) : describeState(ev, room)
|
||||
if (text) {
|
||||
items.push({ kind: 'notice', key, ts, text })
|
||||
prev = null
|
||||
|
||||
+616
-264
@@ -1,7 +1,26 @@
|
||||
import { CallEvent, CallFeedEvent, createNewMatrixCall, type MatrixCall, type MatrixClient } from 'matrix-js-sdk'
|
||||
import { CallDirection, CallErrorCode, CallParty, CallState, CallType, type CallError } from 'matrix-js-sdk/lib/webrtc/call'
|
||||
import { CallEventHandlerEvent } from 'matrix-js-sdk/lib/webrtc/callEventHandler'
|
||||
import type { CallFeed } from 'matrix-js-sdk/lib/webrtc/callFeed'
|
||||
import {
|
||||
ClientEvent,
|
||||
EventType,
|
||||
MatrixEventEvent,
|
||||
RelationType,
|
||||
RoomEvent as MatrixRoomEvent,
|
||||
SyncState,
|
||||
type IRoomTimelineData,
|
||||
type MatrixClient,
|
||||
type MatrixEvent,
|
||||
type Room as MatrixRoom,
|
||||
} from 'matrix-js-sdk'
|
||||
import { MatrixRTCSessionEvent, MatrixRTCSessionManagerEvents, type MatrixRTCSession } from 'matrix-js-sdk/lib/matrixrtc'
|
||||
import {
|
||||
BaseKeyProvider,
|
||||
createKeyMaterialFromBuffer,
|
||||
DisconnectReason,
|
||||
isE2EESupported,
|
||||
Room,
|
||||
RoomEvent,
|
||||
Track,
|
||||
type Participant,
|
||||
} from 'livekit-client'
|
||||
import { defineStore } from 'pinia'
|
||||
import { computed, ref, shallowRef } from 'vue'
|
||||
import { errorMessage } from '@/lib/format'
|
||||
@@ -9,332 +28,665 @@ import { client } from '@/matrix/client'
|
||||
import { startTone, stopTone } from '@/matrix/tones'
|
||||
import { useToasts } from '@/stores/toasts'
|
||||
|
||||
// Voice and video calls between two people: Matrix's 1:1 VoIP (m.call.*
|
||||
// events in the room, WebRTC between the browsers), which the SDK runs.
|
||||
// In an encrypted room the call events are encrypted like any other; the
|
||||
// media itself is always encrypted by WebRTC (DTLS-SRTP).
|
||||
// Voice and video calls, all of them: MatrixRTC (MSC4143) with a LiveKit
|
||||
// SFU, the way Element Call does them. A call between two people is the
|
||||
// same thing as a call between ten, with a ring on top.
|
||||
//
|
||||
// One call at a time. A call that comes in while one is going is answered
|
||||
// "busy", as a phone would.
|
||||
// Who is in a room's call is Matrix state (one
|
||||
// org.matrix.msc3401.call.member event per device), which the SDK's
|
||||
// MatrixRTC session keeps; the media goes through LiveKit, whose token the
|
||||
// LiveKit JWT service hands out for an OpenID token from the homeserver.
|
||||
// The session also schedules a delayed "left" with the homeserver (MSC4140)
|
||||
// and keeps pushing it back, so a member whose browser crashes or loses
|
||||
// its connection drops out of the call on its own.
|
||||
//
|
||||
// Starting a call in a room with one other person rings them: the first
|
||||
// member sends an org.matrix.msc4075.rtc.notification, and the other side
|
||||
// answers by joining or declines with org.matrix.msc4310.rtc.decline.
|
||||
//
|
||||
// In an encrypted room the media is end-to-end encrypted too: each member
|
||||
// makes its own key, shares it with the others over encrypted to-device
|
||||
// messages, and LiveKit encrypts every frame with it (encryptMedia).
|
||||
//
|
||||
// One call at a time.
|
||||
|
||||
export type CallPhase = 'ringing' | 'calling' | 'connecting' | 'connected' | 'ended'
|
||||
/** A person (well, a device) in the call. */
|
||||
export interface Tile {
|
||||
/** The LiveKit identity: "@user:server:DEVICE". */
|
||||
id: string
|
||||
userId: string
|
||||
name: string
|
||||
avatarMxc?: string
|
||||
local: boolean
|
||||
camera?: Track
|
||||
screen?: Track
|
||||
/** A remote microphone, to play. Your own is never played back. */
|
||||
audio?: Track
|
||||
micMuted: boolean
|
||||
speaking: boolean
|
||||
}
|
||||
|
||||
/** How long the ended call stays on screen, saying why. */
|
||||
const endedLingerMs = 2500
|
||||
export type CallPhase = 'joining' | 'connected' | 'reconnecting'
|
||||
|
||||
/** A call ringing for you. */
|
||||
export interface IncomingCall {
|
||||
roomId: string
|
||||
/** The notification event, which a decline refers to. */
|
||||
eventId: string
|
||||
callerId: string
|
||||
callerName: string
|
||||
callerAvatarMxc?: string
|
||||
roomName: string
|
||||
video: boolean
|
||||
/** When the ring stops on its own (ms since the epoch). */
|
||||
expiresAt: number
|
||||
}
|
||||
|
||||
/** The MatrixRTC transport this client publishes: LiveKit via the JWT service. */
|
||||
interface LivekitTransport {
|
||||
type: 'livekit'
|
||||
livekit_service_url: string
|
||||
livekit_alias: string
|
||||
[key: string]: unknown
|
||||
}
|
||||
|
||||
/** How long a call rings, unless the notification says less. */
|
||||
const ringLifetimeMs = 60_000
|
||||
/** The most a received notification may ring for, whatever it says. */
|
||||
const maxRingMs = 120_000
|
||||
|
||||
/**
|
||||
* Feeds the per-participant keys MatrixRTC hands out to LiveKit's frame
|
||||
* encryption. Each key belongs to one participant and index; there is no
|
||||
* shared key and no ratcheting, the same settings as Element Call, so the
|
||||
* two can be in one call.
|
||||
*/
|
||||
class MatrixKeyProvider extends BaseKeyProvider {
|
||||
constructor() {
|
||||
super({ ratchetWindowSize: 0, keyringSize: 256 })
|
||||
}
|
||||
|
||||
async setParticipantKey(key: Uint8Array, index: number, identity: string) {
|
||||
const material = await createKeyMaterialFromBuffer(key.buffer.slice(key.byteOffset, key.byteOffset + key.byteLength) as ArrayBuffer)
|
||||
this.onSetEncryptionKey(material, identity, index)
|
||||
}
|
||||
}
|
||||
|
||||
/** The user of a LiveKit identity lk-jwt-service made: "@alice:example.org:DEVICE". */
|
||||
function identityUser(identity: string): string {
|
||||
// The device is after the last colon; a server name may itself have a port.
|
||||
const i = identity.lastIndexOf(':')
|
||||
return i > 0 ? identity.slice(0, i) : identity
|
||||
}
|
||||
|
||||
/** How many people (not devices) are in a session. */
|
||||
function countMembers(session: MatrixRTCSession): number {
|
||||
return new Set(session.memberships.map((m) => m.userId)).size
|
||||
}
|
||||
|
||||
/** The other person in a room of two, or undefined. */
|
||||
function otherOfTwo(room: MatrixRoom, me: string): string | undefined {
|
||||
if (room.getJoinedMemberCount() !== 2) return undefined
|
||||
return room.getJoinedMembers().find((m) => m.userId !== me)?.userId
|
||||
}
|
||||
|
||||
export const useCall = defineStore('call', () => {
|
||||
const call = shallowRef<MatrixCall | null>(null)
|
||||
/** The room whose call this client is in. */
|
||||
const roomId = ref<string | null>(null)
|
||||
const phase = ref<CallPhase | null>(null)
|
||||
/** Bumps on every change the call view shows. */
|
||||
const version = ref(0)
|
||||
/** Seconds since the call connected. */
|
||||
/** How many people are in each room's call, for every room with one. */
|
||||
const active = ref<Record<string, number>>({})
|
||||
const lk = shallowRef<Room | null>(null)
|
||||
/** Media is end-to-end encrypted. */
|
||||
const encrypted = ref(false)
|
||||
/** Joined with the camera on. */
|
||||
const video = ref(false)
|
||||
/** Seconds since you joined. */
|
||||
const length = ref(0)
|
||||
/** Why the call ended, once it has. */
|
||||
const endReason = ref('')
|
||||
/** We rang someone and they haven't come yet. */
|
||||
const ringingOut = ref(false)
|
||||
/** A call ringing for you. */
|
||||
const incoming = ref<IncomingCall | null>(null)
|
||||
|
||||
let attached: MatrixClient | null = null
|
||||
let clearTimer = 0
|
||||
/** The call got as far as connecting media, at some point. */
|
||||
let connected = false
|
||||
/** The feeds whose mute state is listened to. */
|
||||
let feeds: CallFeed[] = []
|
||||
|
||||
const phase = computed<CallPhase | null>(() => {
|
||||
void version.value
|
||||
const c = call.value
|
||||
if (!c) return null
|
||||
switch (c.state) {
|
||||
case CallState.Ringing:
|
||||
return 'ringing'
|
||||
case CallState.Fledgling:
|
||||
case CallState.WaitLocalMedia:
|
||||
case CallState.CreateOffer:
|
||||
case CallState.InviteSent:
|
||||
return c.direction === CallDirection.Inbound ? 'connecting' : 'calling'
|
||||
case CallState.CreateAnswer:
|
||||
case CallState.Connecting:
|
||||
return 'connecting'
|
||||
case CallState.Connected:
|
||||
return 'connected'
|
||||
default:
|
||||
return 'ended'
|
||||
}
|
||||
})
|
||||
|
||||
const isVideo = computed(() => {
|
||||
void version.value
|
||||
return call.value?.type === CallType.Video
|
||||
})
|
||||
|
||||
const peerName = computed(() => {
|
||||
void version.value
|
||||
const c = call.value
|
||||
if (!c) return ''
|
||||
const member = c.getOpponentMember()
|
||||
if (member) return member.name
|
||||
const room = attached?.getRoom(c.roomId)
|
||||
return room?.name ?? c.invitee ?? ''
|
||||
})
|
||||
|
||||
const peerAvatarMxc = computed(() => {
|
||||
void version.value
|
||||
return call.value?.getOpponentMember()?.getMxcAvatarUrl() ?? undefined
|
||||
})
|
||||
let session: MatrixRTCSession | null = null
|
||||
let keyProvider: MatrixKeyProvider | null = null
|
||||
let lengthTimer = 0
|
||||
let ringOutTimer = 0
|
||||
let ringInTimer = 0
|
||||
/** The ring notification we sent, which a decline refers to. */
|
||||
let sentNotification: string | null = null
|
||||
const watched = new Map<string, MatrixRTCSession>()
|
||||
|
||||
function bump() {
|
||||
version.value++
|
||||
}
|
||||
|
||||
// A feed muting its camera or microphone is not a change in the call's
|
||||
// feeds, so each feed is listened to as well.
|
||||
function watchFeeds(c: MatrixCall | null) {
|
||||
for (const f of feeds) {
|
||||
f.off(CallFeedEvent.MuteStateChanged, bump)
|
||||
f.off(CallFeedEvent.NewStream, bump)
|
||||
}
|
||||
feeds = c ? c.getFeeds() : []
|
||||
for (const f of feeds) {
|
||||
f.on(CallFeedEvent.MuteStateChanged, bump)
|
||||
f.on(CallFeedEvent.NewStream, bump)
|
||||
}
|
||||
const joined = computed(() => roomId.value !== null)
|
||||
|
||||
/** Whether this browser can take part in calls at all. */
|
||||
function supported(): boolean {
|
||||
return typeof window !== 'undefined' && 'RTCPeerConnection' in window
|
||||
}
|
||||
|
||||
function onFeeds() {
|
||||
watchFeeds(call.value)
|
||||
bump()
|
||||
}
|
||||
// --- Tones --------------------------------------------------------------
|
||||
|
||||
function tones() {
|
||||
const p = phase.value
|
||||
if (p === 'ringing') startTone('ring')
|
||||
else if (p === 'calling') startTone('ringback')
|
||||
if (incoming.value && !joined.value) startTone('ring')
|
||||
else if (ringingOut.value) startTone('ringback')
|
||||
else stopTone()
|
||||
}
|
||||
|
||||
function onState(state: CallState) {
|
||||
if (state === CallState.Connected) connected = true
|
||||
// --- Who is in which room's call ----------------------------------------
|
||||
|
||||
function refresh(rid: string) {
|
||||
const s = watched.get(rid)
|
||||
const n = s ? countMembers(s) : 0
|
||||
const next = { ...active.value }
|
||||
if (n > 0) next[rid] = n
|
||||
else delete next[rid]
|
||||
active.value = next
|
||||
// The caller gave up, or someone else answered for you.
|
||||
if (incoming.value?.roomId === rid && n === 0) clearIncoming()
|
||||
// Someone came: stop ringing them.
|
||||
if (rid === roomId.value && ringingOut.value && n > 1) stopRingingOut()
|
||||
}
|
||||
|
||||
function watchSession(rid: string, s: MatrixRTCSession) {
|
||||
if (watched.get(rid) === s) return
|
||||
watched.get(rid)?.off(MatrixRTCSessionEvent.MembershipsChanged, onMemberships)
|
||||
watched.set(rid, s)
|
||||
s.on(MatrixRTCSessionEvent.MembershipsChanged, onMemberships)
|
||||
refresh(rid)
|
||||
// Memberships are parsed asynchronously: a call already going on when
|
||||
// the room loads only shows up once that is done.
|
||||
void s.initialMembershipCalculated.then(() => {
|
||||
refresh(rid)
|
||||
bump()
|
||||
})
|
||||
}
|
||||
|
||||
/** Follows the call of every room the client knows. */
|
||||
function watchRoom(room: MatrixRoom) {
|
||||
if (!attached) return
|
||||
watchSession(room.roomId, attached.matrixRTC.getRoomSession(room))
|
||||
}
|
||||
|
||||
function watchAllRooms() {
|
||||
for (const r of attached?.getRooms() ?? []) watchRoom(r)
|
||||
}
|
||||
|
||||
function onSync(state: SyncState) {
|
||||
if (state === SyncState.Prepared) watchAllRooms()
|
||||
}
|
||||
|
||||
function onMemberships() {
|
||||
for (const rid of watched.keys()) refresh(rid)
|
||||
bump()
|
||||
tones()
|
||||
}
|
||||
|
||||
function onLength(seconds: number) {
|
||||
length.value = seconds
|
||||
function onSessionStarted(rid: string, s: MatrixRTCSession) {
|
||||
watchSession(rid, s)
|
||||
}
|
||||
|
||||
function onError(err: CallError) {
|
||||
// The hangup that follows says the rest.
|
||||
if (err.code === CallErrorCode.NoUserMedia) {
|
||||
forgetFailedMedia()
|
||||
useToasts().error(isVideo.value ? 'Can’t use your camera or microphone. Check that the browser may use them.' : 'Can’t use your microphone. Check that the browser may use it.')
|
||||
} else {
|
||||
useToasts().error(`Call failed: ${errorMessage(err)}`)
|
||||
function onSessionEnded(rid: string) {
|
||||
refresh(rid)
|
||||
}
|
||||
|
||||
// --- Ringing ------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* A ring notification or a decline arriving live. A ring counts when it
|
||||
* is someone else's, still within its lifetime, mentions you (or the
|
||||
* room), and you aren't in that call already.
|
||||
*/
|
||||
function onTimeline(ev: MatrixEvent, room: MatrixRoom | undefined, toStart: boolean | undefined, _removed: boolean, data: IRoomTimelineData) {
|
||||
if (toStart || !data?.liveEvent || !room) return
|
||||
// In an encrypted room this is still ciphertext; onDecrypted has it.
|
||||
if (ev.isEncrypted() && ev.isBeingDecrypted()) return
|
||||
onCallEvent(ev, room)
|
||||
}
|
||||
|
||||
/** An encrypted ring or decline, once it can be read: only recent ones count. */
|
||||
function onDecrypted(ev: MatrixEvent) {
|
||||
if (ev.isDecryptionFailure() || Date.now() - ev.getTs() > maxRingMs) return
|
||||
const room = attached?.getRoom(ev.getRoomId() ?? '')
|
||||
if (room) onCallEvent(ev, room)
|
||||
}
|
||||
|
||||
/** Event IDs already acted on, so a ring isn't taken twice. */
|
||||
const seen = new Set<string>()
|
||||
|
||||
function onCallEvent(ev: MatrixEvent, room: MatrixRoom) {
|
||||
if (!attached) return
|
||||
const type = ev.getType()
|
||||
if (type !== EventType.RTCNotification && type !== EventType.RTCDecline) return
|
||||
const id = ev.getId()
|
||||
if (id) {
|
||||
if (seen.has(id)) return
|
||||
seen.add(id)
|
||||
}
|
||||
}
|
||||
|
||||
function onHangup(c: MatrixCall) {
|
||||
if (c !== call.value) return
|
||||
endReason.value = describeHangup(c, connected)
|
||||
bump()
|
||||
stopTone()
|
||||
window.clearTimeout(clearTimer)
|
||||
clearTimer = window.setTimeout(() => {
|
||||
if (call.value === c) release()
|
||||
}, endedLingerMs)
|
||||
}
|
||||
|
||||
function onReplaced(newCall: MatrixCall) {
|
||||
// Both of you called at once: the SDK keeps one of the two calls.
|
||||
show(newCall)
|
||||
}
|
||||
|
||||
function listen(c: MatrixCall, on: boolean) {
|
||||
const m = on ? 'on' : 'off'
|
||||
c[m](CallEvent.State, onState)
|
||||
c[m](CallEvent.FeedsChanged, onFeeds)
|
||||
c[m](CallEvent.LocalHoldUnhold, bump)
|
||||
c[m](CallEvent.RemoteHoldUnhold, bump)
|
||||
c[m](CallEvent.LengthChanged, onLength)
|
||||
c[m](CallEvent.Error, onError)
|
||||
c[m](CallEvent.Hangup, onHangup)
|
||||
c[m](CallEvent.Replaced, onReplaced)
|
||||
}
|
||||
|
||||
function show(c: MatrixCall) {
|
||||
if (call.value) listen(call.value, false)
|
||||
window.clearTimeout(clearTimer)
|
||||
call.value = c
|
||||
connected = false
|
||||
length.value = 0
|
||||
endReason.value = ''
|
||||
listen(c, true)
|
||||
watchFeeds(c)
|
||||
bump()
|
||||
tones()
|
||||
}
|
||||
|
||||
function release() {
|
||||
if (call.value) listen(call.value, false)
|
||||
watchFeeds(null)
|
||||
window.clearTimeout(clearTimer)
|
||||
call.value = null
|
||||
stopTone()
|
||||
bump()
|
||||
}
|
||||
|
||||
function onIncoming(incoming: MatrixCall) {
|
||||
if (call.value && phase.value !== 'ended') {
|
||||
incoming.hangup(CallErrorCode.UserBusy, false)
|
||||
const me = attached.getUserId() ?? ''
|
||||
if (type === EventType.RTCNotification) {
|
||||
if (ev.getSender() === me || roomId.value === room.roomId) return
|
||||
const content = ev.getContent()
|
||||
const mentions = content['m.mentions'] as { user_ids?: string[]; room?: boolean } | undefined
|
||||
if (!mentions?.room && !mentions?.user_ids?.includes(me)) return
|
||||
// A ring for a call of two; in a bigger room the banner says enough.
|
||||
if (content.notification_type !== 'ring' && !otherOfTwo(room, me)) return
|
||||
const sentAt = typeof content.sender_ts === 'number' ? Math.min(content.sender_ts, ev.getTs()) : ev.getTs()
|
||||
const lifetime = Math.min(typeof content.lifetime === 'number' ? content.lifetime : ringLifetimeMs, maxRingMs)
|
||||
const expiresAt = sentAt + lifetime
|
||||
if (expiresAt <= Date.now()) return
|
||||
if (joined.value) return // Busy: you see the banner when you're free.
|
||||
const caller = room.getMember(ev.getSender() ?? '')
|
||||
incoming.value = {
|
||||
roomId: room.roomId,
|
||||
eventId: ev.getId() ?? '',
|
||||
callerId: ev.getSender() ?? '',
|
||||
callerName: caller?.name ?? ev.getSender() ?? '',
|
||||
callerAvatarMxc: caller?.getMxcAvatarUrl() ?? undefined,
|
||||
roomName: room.name,
|
||||
video: content['m.call.intent'] !== 'audio',
|
||||
expiresAt,
|
||||
}
|
||||
window.clearTimeout(ringInTimer)
|
||||
ringInTimer = window.setTimeout(clearIncoming, expiresAt - Date.now())
|
||||
tones()
|
||||
return
|
||||
}
|
||||
show(incoming)
|
||||
// Hung up (or answered elsewhere) before we did anything.
|
||||
if (incoming.callHasEnded()) onHangup(incoming)
|
||||
if (type === EventType.RTCDecline) {
|
||||
const target = ev.getContent()['m.relates_to']?.event_id
|
||||
if (!target) return
|
||||
// You declined on another of your devices: stop ringing here.
|
||||
if (ev.getSender() === me && incoming.value?.eventId === target) {
|
||||
clearIncoming()
|
||||
return
|
||||
}
|
||||
// The one person you rang said no.
|
||||
if (target === sentNotification && ringingOut.value && room.roomId === roomId.value) {
|
||||
useToasts().info(`${room.getMember(ev.getSender() ?? '')?.name ?? 'They'} declined the call.`)
|
||||
void leave()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function clearIncoming() {
|
||||
window.clearTimeout(ringInTimer)
|
||||
incoming.value = null
|
||||
tones()
|
||||
}
|
||||
|
||||
function stopRingingOut() {
|
||||
window.clearTimeout(ringOutTimer)
|
||||
ringingOut.value = false
|
||||
tones()
|
||||
}
|
||||
|
||||
function onNotificationSent(sent: { event_id: string; lifetime?: number }) {
|
||||
sentNotification = sent.event_id
|
||||
}
|
||||
|
||||
/** Answers the call ringing for you. */
|
||||
async function answer(withVideo = true) {
|
||||
const call = incoming.value
|
||||
if (!call) return
|
||||
clearIncoming()
|
||||
await join(call.roomId, withVideo && call.video)
|
||||
}
|
||||
|
||||
/** Declines the call ringing for you. The caller stops ringing when it's the two of you. */
|
||||
async function decline() {
|
||||
const call = incoming.value
|
||||
if (!call) return
|
||||
clearIncoming()
|
||||
try {
|
||||
await client().sendEvent(call.roomId, EventType.RTCDecline, {
|
||||
'm.relates_to': { rel_type: RelationType.Reference, event_id: call.eventId },
|
||||
})
|
||||
} catch (e) {
|
||||
console.error('call: decline: ', e)
|
||||
}
|
||||
}
|
||||
|
||||
/** Starts listening for calls on the running client. */
|
||||
function attach() {
|
||||
const c = client()
|
||||
if (attached === c) return
|
||||
if (attached) attached.off(CallEventHandlerEvent.Incoming, onIncoming)
|
||||
release()
|
||||
if (attached) {
|
||||
attached.matrixRTC.off(MatrixRTCSessionManagerEvents.SessionStarted, onSessionStarted)
|
||||
attached.matrixRTC.off(MatrixRTCSessionManagerEvents.SessionEnded, onSessionEnded)
|
||||
attached.off(ClientEvent.Room, watchRoom)
|
||||
attached.off(ClientEvent.Sync, onSync)
|
||||
attached.off(MatrixRoomEvent.Timeline, onTimeline)
|
||||
attached.off(MatrixEventEvent.Decrypted, onDecrypted)
|
||||
}
|
||||
for (const s of watched.values()) s.off(MatrixRTCSessionEvent.MembershipsChanged, onMemberships)
|
||||
watched.clear()
|
||||
active.value = {}
|
||||
clearIncoming()
|
||||
void leave()
|
||||
attached = c
|
||||
c.on(CallEventHandlerEvent.Incoming, onIncoming)
|
||||
c.matrixRTC.on(MatrixRTCSessionManagerEvents.SessionStarted, onSessionStarted)
|
||||
c.matrixRTC.on(MatrixRTCSessionManagerEvents.SessionEnded, onSessionEnded)
|
||||
c.on(ClientEvent.Room, watchRoom)
|
||||
c.on(ClientEvent.Sync, onSync)
|
||||
c.on(MatrixRoomEvent.Timeline, onTimeline)
|
||||
c.on(MatrixEventEvent.Decrypted, onDecrypted)
|
||||
watchAllRooms()
|
||||
}
|
||||
|
||||
/** Whether this browser can make calls at all. */
|
||||
function supported(): boolean {
|
||||
return Boolean(attached?.supportsVoip())
|
||||
}
|
||||
// --- Joining ------------------------------------------------------------
|
||||
|
||||
async function place(roomId: string, invitee: string | undefined, video: boolean) {
|
||||
if (call.value && phase.value !== 'ended') {
|
||||
useToasts().error('You’re already in a call.')
|
||||
return
|
||||
/** The LiveKit JWT service to use: the one the call already uses, else this server's. */
|
||||
async function chooseTransport(s: MatrixRTCSession, rid: string): Promise<LivekitTransport> {
|
||||
const oldest = s.memberships[0]
|
||||
const theirs = oldest?.getTransport(oldest)
|
||||
if (theirs && theirs.type === 'livekit' && typeof theirs.livekit_service_url === 'string') {
|
||||
return { type: 'livekit', livekit_service_url: theirs.livekit_service_url, livekit_alias: rid }
|
||||
}
|
||||
const c = createNewMatrixCall(client(), roomId, { invitee })
|
||||
if (!c) {
|
||||
useToasts().error('This browser can’t make calls.')
|
||||
return
|
||||
}
|
||||
show(c)
|
||||
const c = client()
|
||||
// The homeserver lists its backends (MSC4143); a server that predates
|
||||
// the endpoint has them in its discovery document instead.
|
||||
let foci: unknown
|
||||
try {
|
||||
if (video) await c.placeVideoCall()
|
||||
else await c.placeVoiceCall()
|
||||
} catch (e) {
|
||||
// The SDK has hung up already and said why through CallEvent.Error.
|
||||
console.error('call: ', e)
|
||||
foci = await c._unstable_getRTCTransports()
|
||||
} catch {
|
||||
const res = await fetch(`${c.baseUrl.replace(/\/$/, '')}/.well-known/matrix/client`)
|
||||
if (!res.ok) throw new Error('This server doesn’t offer calls.')
|
||||
const doc = (await res.json()) as Record<string, unknown>
|
||||
foci = doc['org.matrix.msc4143.rtc_foci']
|
||||
}
|
||||
const focus = Array.isArray(foci)
|
||||
? (foci as Record<string, unknown>[]).find((f) => f?.type === 'livekit' && typeof f.livekit_service_url === 'string')
|
||||
: undefined
|
||||
if (!focus) throw new Error('This server doesn’t offer calls.')
|
||||
return { type: 'livekit', livekit_service_url: String(focus.livekit_service_url), livekit_alias: rid }
|
||||
}
|
||||
|
||||
/** Answers the ringing call; a video call with the camera off when `video` is false. */
|
||||
async function answer(video = true) {
|
||||
const c = call.value
|
||||
if (!c || phase.value !== 'ringing') return
|
||||
stopTone()
|
||||
/** A LiveKit URL and token, from the JWT service, for an OpenID token. */
|
||||
async function livekitToken(transport: LivekitTransport, rid: string): Promise<{ url: string; jwt: string }> {
|
||||
const c = client()
|
||||
const openid = await c.getOpenIdToken()
|
||||
const res = await fetch(`${transport.livekit_service_url.replace(/\/$/, '')}/sfu/get`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ room: rid, openid_token: openid, device_id: c.getDeviceId() }),
|
||||
})
|
||||
if (!res.ok) throw new Error(`The call service refused to let you in (${res.status}).`)
|
||||
const body = (await res.json()) as { url?: string; jwt?: string }
|
||||
if (!body.url || !body.jwt) throw new Error('The call service gave an unusable answer.')
|
||||
return { url: body.url, jwt: body.jwt }
|
||||
}
|
||||
|
||||
function listen(r: Room) {
|
||||
r.on(RoomEvent.ParticipantConnected, bump)
|
||||
.on(RoomEvent.ParticipantDisconnected, bump)
|
||||
.on(RoomEvent.TrackSubscribed, bump)
|
||||
.on(RoomEvent.TrackUnsubscribed, bump)
|
||||
.on(RoomEvent.TrackMuted, bump)
|
||||
.on(RoomEvent.TrackUnmuted, bump)
|
||||
.on(RoomEvent.LocalTrackPublished, bump)
|
||||
.on(RoomEvent.LocalTrackUnpublished, bump)
|
||||
.on(RoomEvent.ActiveSpeakersChanged, bump)
|
||||
.on(RoomEvent.ParticipantEncryptionStatusChanged, bump)
|
||||
.on(RoomEvent.Reconnecting, () => {
|
||||
phase.value = 'reconnecting'
|
||||
})
|
||||
.on(RoomEvent.Reconnected, () => {
|
||||
phase.value = 'connected'
|
||||
})
|
||||
.on(RoomEvent.Disconnected, (reason?: DisconnectReason) => {
|
||||
if (lk.value !== r) return
|
||||
if (reason !== DisconnectReason.CLIENT_INITIATED) useToasts().error('You were disconnected from the call.')
|
||||
void leave()
|
||||
})
|
||||
}
|
||||
|
||||
function onEncryptionKey(key: Uint8Array, index: number, _membership: unknown, backendIdentity: string) {
|
||||
void keyProvider?.setParticipantKey(key, index, backendIdentity)
|
||||
}
|
||||
|
||||
function onMembershipError(e: unknown) {
|
||||
useToasts().error(`The call stopped working: ${errorMessage(e)}`)
|
||||
void leave()
|
||||
}
|
||||
|
||||
/**
|
||||
* Joins the call in a room, or starts one. Starting a call in a room of
|
||||
* two rings the other person.
|
||||
*/
|
||||
async function join(rid: string, withVideo: boolean) {
|
||||
if (roomId.value === rid) return
|
||||
if (roomId.value) await leave()
|
||||
const c = client()
|
||||
const room = c.getRoom(rid)
|
||||
if (!room) return
|
||||
if (incoming.value?.roomId === rid) clearIncoming()
|
||||
|
||||
roomId.value = rid
|
||||
phase.value = 'joining'
|
||||
video.value = withVideo
|
||||
length.value = 0
|
||||
sentNotification = null
|
||||
bump()
|
||||
const s = c.matrixRTC.getRoomSession(room)
|
||||
watchSession(rid, s)
|
||||
session = s
|
||||
const e2ee = room.hasEncryptionStateEvent()
|
||||
encrypted.value = e2ee
|
||||
const me = c.getUserId()!
|
||||
const ringing = countMembers(s) === 0 && otherOfTwo(room, me) !== undefined
|
||||
try {
|
||||
await c.answer(true, video && c.type === CallType.Video)
|
||||
if (e2ee && !isE2EESupported()) {
|
||||
throw new Error('This browser can’t encrypt call media, and this room is end-to-end encrypted.')
|
||||
}
|
||||
const transport = await chooseTransport(s, rid)
|
||||
const { url, jwt } = await livekitToken(transport, rid)
|
||||
if (roomId.value !== rid) return
|
||||
|
||||
keyProvider = e2ee ? new MatrixKeyProvider() : null
|
||||
const r = new Room({
|
||||
adaptiveStream: true,
|
||||
dynacast: true,
|
||||
e2ee: keyProvider
|
||||
? { keyProvider, worker: new Worker(new URL('livekit-client/e2ee-worker', import.meta.url), { type: 'module' }) }
|
||||
: undefined,
|
||||
})
|
||||
lk.value = r
|
||||
listen(r)
|
||||
if (keyProvider) await r.setE2EEEnabled(true)
|
||||
|
||||
// The Matrix side: announce ourselves (and schedule our own delayed
|
||||
// leave), ring the other person if we're the first, and in an
|
||||
// encrypted room start sharing media keys.
|
||||
const deviceId = c.getDeviceId()!
|
||||
s.on(MatrixRTCSessionEvent.EncryptionKeyChanged, onEncryptionKey)
|
||||
s.on(MatrixRTCSessionEvent.MembershipManagerError, onMembershipError)
|
||||
s.on(MatrixRTCSessionEvent.DidSendCallNotification, onNotificationSent)
|
||||
s.join({ userId: me, deviceId, memberId: `${me}:${deviceId}` }, [transport], {
|
||||
manageMediaKeys: e2ee,
|
||||
callIntent: withVideo ? 'video' : 'audio',
|
||||
notificationType: ringing ? 'ring' : countMembers(s) === 0 ? 'notification' : undefined,
|
||||
notificationLifetimeMs: ringLifetimeMs,
|
||||
})
|
||||
if (e2ee) s.reemitEncryptionKeys()
|
||||
if (ringing) {
|
||||
ringingOut.value = true
|
||||
ringOutTimer = window.setTimeout(() => {
|
||||
if (!ringingOut.value || roomId.value !== rid) return
|
||||
useToasts().info('No answer.')
|
||||
void leave()
|
||||
}, ringLifetimeMs)
|
||||
tones()
|
||||
}
|
||||
|
||||
await r.connect(url, jwt)
|
||||
if (roomId.value !== rid) {
|
||||
await r.disconnect()
|
||||
return
|
||||
}
|
||||
phase.value = 'connected'
|
||||
window.clearInterval(lengthTimer)
|
||||
const since = Date.now()
|
||||
lengthTimer = window.setInterval(() => (length.value = Math.floor((Date.now() - since) / 1000)), 1000)
|
||||
bump()
|
||||
try {
|
||||
await r.localParticipant.setMicrophoneEnabled(true)
|
||||
} catch (e) {
|
||||
useToasts().error(`Can’t use your microphone: ${errorMessage(e)}`)
|
||||
}
|
||||
if (withVideo) {
|
||||
try {
|
||||
await r.localParticipant.setCameraEnabled(true)
|
||||
} catch (e) {
|
||||
useToasts().error(`Can’t use your camera: ${errorMessage(e)}`)
|
||||
}
|
||||
}
|
||||
// Browsers only play audio after a gesture; joining was one.
|
||||
await r.startAudio().catch(() => undefined)
|
||||
bump()
|
||||
} catch (e) {
|
||||
console.error('call: ', e)
|
||||
useToasts().error(errorMessage(e))
|
||||
await leave()
|
||||
}
|
||||
}
|
||||
|
||||
function decline() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
if (phase.value === 'ringing') c.reject()
|
||||
else hangup()
|
||||
}
|
||||
|
||||
function hangup() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
if (phase.value === 'ended') release()
|
||||
else c.hangup(CallErrorCode.UserHangup, false)
|
||||
}
|
||||
|
||||
async function toggleMic() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
await c.setMicrophoneMuted(!c.isMicrophoneMuted())
|
||||
/** Leaves the call this client is in. */
|
||||
async function leave() {
|
||||
const s = session
|
||||
const r = lk.value
|
||||
session = null
|
||||
lk.value = null
|
||||
keyProvider = null
|
||||
sentNotification = null
|
||||
window.clearInterval(lengthTimer)
|
||||
stopRingingOut()
|
||||
const rid = roomId.value
|
||||
roomId.value = null
|
||||
phase.value = null
|
||||
bump()
|
||||
if (s) {
|
||||
s.off(MatrixRTCSessionEvent.EncryptionKeyChanged, onEncryptionKey)
|
||||
s.off(MatrixRTCSessionEvent.MembershipManagerError, onMembershipError)
|
||||
s.off(MatrixRTCSessionEvent.DidSendCallNotification, onNotificationSent)
|
||||
}
|
||||
await Promise.allSettled([r?.disconnect(), s?.leave(5000)])
|
||||
if (rid) refresh(rid)
|
||||
}
|
||||
|
||||
async function toggleCamera() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
await c.setLocalVideoMuted(!c.isLocalVideoMuted())
|
||||
bump()
|
||||
}
|
||||
// --- What the view shows ------------------------------------------------
|
||||
|
||||
async function toggleScreenShare() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
const tiles = computed<Tile[]>(() => {
|
||||
void version.value
|
||||
const r = lk.value
|
||||
const rid = roomId.value
|
||||
if (!r || !rid) return []
|
||||
const room = attached?.getRoom(rid)
|
||||
const speakers = new Set(r.activeSpeakers.map((p) => p.identity))
|
||||
const tile = (p: Participant, local: boolean): Tile => {
|
||||
const userId = identityUser(p.identity)
|
||||
const member = room?.getMember(userId)
|
||||
const cam = p.getTrackPublication(Track.Source.Camera)
|
||||
const screen = p.getTrackPublication(Track.Source.ScreenShare)
|
||||
const mic = p.getTrackPublication(Track.Source.Microphone)
|
||||
return {
|
||||
id: p.identity,
|
||||
userId,
|
||||
name: member?.name ?? userId,
|
||||
avatarMxc: member?.getMxcAvatarUrl() ?? undefined,
|
||||
local,
|
||||
camera: cam?.track && !cam.isMuted ? cam.track : undefined,
|
||||
screen: screen?.track && !screen.isMuted ? screen.track : undefined,
|
||||
audio: !local && mic?.track ? mic.track : undefined,
|
||||
micMuted: !mic || mic.isMuted,
|
||||
speaking: speakers.has(p.identity),
|
||||
}
|
||||
}
|
||||
return [tile(r.localParticipant, true), ...[...r.remoteParticipants.values()].map((p) => tile(p, false))]
|
||||
})
|
||||
|
||||
/** The room's name, or for a call of two, the other person's. */
|
||||
const title = computed(() => {
|
||||
void version.value
|
||||
const rid = roomId.value
|
||||
const room = rid ? attached?.getRoom(rid) : undefined
|
||||
if (!room) return ''
|
||||
const other = otherOfTwo(room, attached?.getUserId() ?? '')
|
||||
return other ? (room.getMember(other)?.name ?? room.name) : room.name
|
||||
})
|
||||
|
||||
/** The person you're calling, for the "ringing" screen of a call of two. */
|
||||
const callee = computed(() => {
|
||||
void version.value
|
||||
const rid = roomId.value
|
||||
const room = rid ? attached?.getRoom(rid) : undefined
|
||||
const other = room ? otherOfTwo(room, attached?.getUserId() ?? '') : undefined
|
||||
const member = other ? room?.getMember(other) : undefined
|
||||
return member ? { name: member.name, avatarMxc: member.getMxcAvatarUrl() ?? undefined } : null
|
||||
})
|
||||
|
||||
const micOn = computed(() => {
|
||||
void version.value
|
||||
return lk.value?.localParticipant.isMicrophoneEnabled ?? false
|
||||
})
|
||||
const cameraOn = computed(() => {
|
||||
void version.value
|
||||
return lk.value?.localParticipant.isCameraEnabled ?? false
|
||||
})
|
||||
const sharing = computed(() => {
|
||||
void version.value
|
||||
return lk.value?.localParticipant.isScreenShareEnabled ?? false
|
||||
})
|
||||
|
||||
async function toggle(what: 'mic' | 'camera' | 'screen') {
|
||||
const p = lk.value?.localParticipant
|
||||
if (!p) return
|
||||
try {
|
||||
await c.setScreensharingEnabled(!c.isScreensharing())
|
||||
if (what === 'mic') await p.setMicrophoneEnabled(!p.isMicrophoneEnabled)
|
||||
else if (what === 'camera') await p.setCameraEnabled(!p.isCameraEnabled)
|
||||
else await p.setScreenShareEnabled(!p.isScreenShareEnabled, { audio: true })
|
||||
} catch (e) {
|
||||
useToasts().error(`Can’t share your screen: ${errorMessage(e)}`)
|
||||
// Cancelling the browser's screen picker is not an error worth a toast.
|
||||
if (!(e instanceof DOMException && e.name === 'NotAllowedError' && what === 'screen')) {
|
||||
useToasts().error(errorMessage(e))
|
||||
}
|
||||
}
|
||||
bump()
|
||||
}
|
||||
|
||||
function toggleHold() {
|
||||
const c = call.value
|
||||
if (!c) return
|
||||
c.setRemoteOnHold(!c.isRemoteOnHold())
|
||||
bump()
|
||||
/** How many people are in a room's call; 0 when there is none. */
|
||||
function inCall(rid: string): number {
|
||||
return active.value[rid] ?? 0
|
||||
}
|
||||
|
||||
return {
|
||||
call,
|
||||
version,
|
||||
length,
|
||||
endReason,
|
||||
roomId,
|
||||
phase,
|
||||
isVideo,
|
||||
peerName,
|
||||
peerAvatarMxc,
|
||||
attach,
|
||||
version,
|
||||
active,
|
||||
joined,
|
||||
encrypted,
|
||||
video,
|
||||
length,
|
||||
ringingOut,
|
||||
incoming,
|
||||
tiles,
|
||||
title,
|
||||
callee,
|
||||
micOn,
|
||||
cameraOn,
|
||||
sharing,
|
||||
supported,
|
||||
place,
|
||||
attach,
|
||||
join,
|
||||
leave,
|
||||
answer,
|
||||
decline,
|
||||
hangup,
|
||||
toggleMic,
|
||||
toggleCamera,
|
||||
toggleScreenShare,
|
||||
toggleHold,
|
||||
toggle,
|
||||
inCall,
|
||||
}
|
||||
})
|
||||
|
||||
/**
|
||||
* The SDK's media handler queues every request for the microphone and
|
||||
* camera behind the previous one, and never lets go of a request that
|
||||
* failed: once you said no, every later call would fail the same way until
|
||||
* the page reloads. Dropping the failed request lets the next call ask again.
|
||||
*/
|
||||
function forgetFailedMedia() {
|
||||
const handler = client().getMediaHandler() as unknown as { getMediaStreamPromise?: Promise<MediaStream> }
|
||||
handler.getMediaStreamPromise = undefined
|
||||
}
|
||||
|
||||
function describeHangup(c: MatrixCall, connected: boolean): string {
|
||||
switch (c.hangupReason) {
|
||||
case CallErrorCode.UserBusy:
|
||||
return 'Busy'
|
||||
case CallErrorCode.InviteTimeout:
|
||||
return c.direction === CallDirection.Inbound ? 'Missed call' : 'No answer'
|
||||
case CallErrorCode.AnsweredElsewhere:
|
||||
return 'Answered in another session'
|
||||
case CallErrorCode.IceFailed:
|
||||
return 'Couldn’t connect'
|
||||
case CallErrorCode.NoUserMedia:
|
||||
return 'No microphone or camera'
|
||||
case CallErrorCode.Replaced:
|
||||
return 'Call replaced'
|
||||
default:
|
||||
if (!connected) {
|
||||
const outbound = c.direction === CallDirection.Outbound
|
||||
if (c.hangupParty === CallParty.Remote) return outbound ? 'Declined' : 'Missed call'
|
||||
return outbound ? 'Call cancelled' : 'Declined'
|
||||
}
|
||||
return 'Call ended'
|
||||
}
|
||||
}
|
||||
@@ -60,8 +60,9 @@ export function describeEvent(ev: MatrixEvent, me: string, room: Room, withSende
|
||||
// own placeholder text: never show that.
|
||||
if (ev.isDecryptionFailure()) return withSender ? `${senderName(ev, me, room)}: Can’t decrypt this message` : 'Can’t decrypt this message'
|
||||
if (type === EventType.RoomMessageEncrypted) return 'Encrypted message'
|
||||
if (type === EventType.CallInvite) {
|
||||
const text = isVideoCallInvite(ev) ? 'Video call' : 'Voice call'
|
||||
if (type === EventType.CallInvite || type === EventType.RTCNotification) {
|
||||
const video = type === EventType.CallInvite ? isVideoCallInvite(ev) : ev.getContent()['m.call.intent'] !== 'audio'
|
||||
const text = video ? 'Video call' : 'Voice call'
|
||||
return withSender ? `${senderName(ev, me, room)}: ${text}` : text
|
||||
}
|
||||
if (type !== EventType.RoomMessage && type !== EventType.Sticker) return null
|
||||
|
||||
@@ -54,19 +54,25 @@ const canSend = computed(() => {
|
||||
|
||||
const infoOpen = ref(false)
|
||||
|
||||
// Calls are between two people: offered in a room with just you and one
|
||||
// other person in it.
|
||||
const callee = computed(() => {
|
||||
// Calls, in any room with someone else in it: starting one in a room of
|
||||
// two rings the other person. One call at a time, wherever it is.
|
||||
const canCall = computed(() => {
|
||||
void chat.version
|
||||
const r = room.value
|
||||
if (!r || r.getJoinedMemberCount() !== 2) return undefined
|
||||
return r.getJoinedMembers().find((m) => m.userId !== me)?.userId
|
||||
return Boolean(r) && membership.value === KnownMembership.Join && call.supported() && (r?.getJoinedMemberCount() ?? 0) > 1
|
||||
})
|
||||
const canCall = computed(() => Boolean(callee.value) && call.supported())
|
||||
// One call at a time, wherever it is.
|
||||
const inCall = computed(() => Boolean(call.call) && call.phase !== 'ended')
|
||||
function startCall(video: boolean) {
|
||||
void call.place(props.roomId, callee.value, video)
|
||||
const inCall = computed(() => call.joined)
|
||||
// A call going on here that you're not in.
|
||||
const callHere = computed(() => call.inCall(props.roomId))
|
||||
const inThisCall = computed(() => call.roomId === props.roomId)
|
||||
const callPeople = computed(() => {
|
||||
void call.version
|
||||
void chat.version
|
||||
const n = callHere.value
|
||||
return n === 1 ? '1 person is' : `${n} people are`
|
||||
})
|
||||
function joinCall(video: boolean) {
|
||||
void call.join(props.roomId, video)
|
||||
}
|
||||
|
||||
// Someone in an encrypted room whose identity changed: said before you
|
||||
@@ -294,13 +300,23 @@ const placeholder = computed(() => `Message ${summary.value?.name ?? 'this room'
|
||||
<span v-if="!encrypted" class="caption hidden items-center gap-1 text-ink-muted sm:flex">
|
||||
<WIcon name="shield-alert" :size="14" /> Not encrypted
|
||||
</span>
|
||||
<template v-if="canCall">
|
||||
<WIconButton icon="phone" label="Voice call" :disabled="inCall" @click="startCall(false)" />
|
||||
<WIconButton icon="video" label="Video call" :disabled="inCall" @click="startCall(true)" />
|
||||
<template v-if="canCall && !callHere">
|
||||
<WIconButton icon="phone" label="Voice call" :disabled="inCall" @click="joinCall(false)" />
|
||||
<WIconButton icon="video" label="Video call" :disabled="inCall" @click="joinCall(true)" />
|
||||
</template>
|
||||
<WIconButton icon="users" :label="infoOpen ? 'Hide room info' : 'Room info'" :aria-pressed="infoOpen" @click="infoOpen = !infoOpen" />
|
||||
</header>
|
||||
|
||||
<div v-if="callHere && canCall && !inThisCall" class="shrink-0 border-b border-line bg-surface px-4 py-2">
|
||||
<WBanner tone="info" icon="video" title="A call is going on in this room">
|
||||
<div class="flex flex-wrap items-center gap-2">
|
||||
<span class="min-w-0 flex-1">{{ callPeople }} in it.</span>
|
||||
<WButton size="sm" :disabled="inCall" @click="joinCall(false)">Join with audio</WButton>
|
||||
<WButton size="sm" variant="primary" :disabled="inCall" @click="joinCall(true)">Join call</WButton>
|
||||
</div>
|
||||
</WBanner>
|
||||
</div>
|
||||
|
||||
<RoomTimeline :room="room" :is-direct="summary?.isDirect ?? false" />
|
||||
|
||||
<div class="shrink-0 bg-surface-sunken px-4 pt-2 pb-4">
|
||||
|
||||
@@ -26,6 +26,8 @@ export default defineConfig({
|
||||
proxy: {
|
||||
'/_matrix/client': { target: homeserver, changeOrigin: true },
|
||||
'/_matrix/media': { target: homeserver, changeOrigin: true },
|
||||
// Where the client finds the group call backend (rtc_foci).
|
||||
'/.well-known/matrix/client': { target: homeserver, changeOrigin: true },
|
||||
},
|
||||
},
|
||||
})
|
||||
Reference in new issue
Block a user