Phase 6: media
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s

Implements PLAN.md Phase 6: the media repository. Clients upload files,
reserve an mxc:// URI and upload to it later, download and ask for
thumbnails, all through the authenticated media endpoints of Matrix 1.11;
other servers download this server's media through the federation media
endpoints, and media uploaded elsewhere is fetched over federation on
first use and kept.

internal/mediaapi is the component. Content goes into a BlobStore named
by its SHA-256, so the same file uploaded twice is stored once and the
hash doubles as the ETag; the metadata that maps an mxc:// URI onto a
blob is in media_repository and media_thumbnails. The local-filesystem
store writes to a temporary file and renames it into place once hashed,
so nothing ever refers to half a file. A storage driver can bring its own
store through storage.BlobProvider, which is where PocketBase's
filesystem abstraction (local or S3) plugs in with Phase 10.

Reserved uploads belong to the user who made them, take one upload,
lapse after a day, and are capped per user; a download of one waits for
its content, woken in memory the moment it arrives, for as long as
timeout_ms allows.

Thumbnails come from a fixed set of configured sizes, so a thousand
requested sizes cannot mean a thousand resizes. They are made on first
request and kept, never enlarge, honour a JPEG's EXIF orientation, and
point at the original when it already fits. The pixel count is checked
from the header before anything is decoded, at most four resizes run at
once, and concurrent requests for the same thumbnail share one.

Remote media is fetched through the signed client: the multipart/mixed
answer of Matrix 1.11, with a fallback to the legacy download endpoint
for servers that answer M_UNRECOGNIZED, and Location redirects followed
by a dialer that refuses loopback, private and link-local addresses.
Concurrent requests share one fetch, detached from the request that
started it. With federation off nothing is fetched, and what was cached
before is still served.

Downloads carry a sandboxing CSP, nosniff, inline Content-Disposition
only for the types the spec lists as safe, and private immutable caching.
The unauthenticated legacy download endpoints are off by default behind
media.legacy_unauthenticated_downloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
tajniak81andClaude Opus 5.5 committed 2026-10-16 21:26:49 +02:00
1 parent 6aaf554336
commit 92836f8dcd
40 files changed
+4729 -63

No files matched your search

+11 -1
View File
@@ -53,6 +53,7 @@ List-valued variables are comma-separated. A single `-` clears a list:
| `client_api.rate_limit.room_create.burst` / `.interval` | — | `10` / `1s` | `/createRoom`, which writes a dozen events per call. |
| `client_api.rate_limit.join.burst` / `.interval` | — | `10` / `1s` | The two join endpoints. |
| `client_api.rate_limit.sync.burst` / `.interval` | — | `100` / `20ms` | `/sync` and the writes behind it: typing, receipts, to-device, presence. |
| `client_api.rate_limit.media.burst` / `.interval` | — | `100` / `20ms` | Media downloads and thumbnails: a client opening a room asks for every avatar at once. Uploads are in the default bucket. |
| `sync.presence` | `WARP_SYNC_PRESENCE` | `false` | Off by default; see below. |
| `sync.typing_timeout` | `WARP_SYNC_TYPING_TIMEOUT` | `30s` | How long a typing notification lasts, and the cap on what a client may ask for. |
| `sync.long_poll_timeout` | `WARP_SYNC_LONG_POLL_TIMEOUT` | `30s` | The cap on a `/sync` `?timeout=`. |
@@ -63,6 +64,13 @@ List-valued variables are comma-separated. A single `-` clears a list:
| `federation.trusted_key_servers` | `WARP_FEDERATION_TRUSTED_KEY_SERVERS` | `[]` | Notaries asked for a server's key when it cannot be reached itself, e.g. `matrix.org`. Empty = only ever ask the server. |
| `federation.key_validity_period` | `WARP_FEDERATION_KEY_VALIDITY_PERIOD` | `24h` | How long other servers may cache this server's key. At most `168h`. |
| `federation.tls_ca_file` | `WARP_FEDERATION_TLS_CA_FILE` | empty | Extra CA certificates (PEM) for outbound federation, on top of the system roots. |
| `media.base_path` | `WARP_MEDIA_BASE_PATH` | `./media_store` | Where media content is kept. Created on the first upload. |
| `media.max_upload_bytes` | `WARP_MEDIA_MAX_UPLOAD_BYTES` | `52428800` | The largest upload (50 MiB), and the largest file fetched from another server. Reported to clients as `m.upload.size`. |
| `media.max_thumbnail_pixels` | `WARP_MEDIA_MAX_THUMBNAIL_PIXELS` | `32000000` | The largest image, in pixels, decoded to make a thumbnail. |
| `media.thumbnail_sizes` | — | five sizes, see [media.md](media.md) | The thumbnails this server makes, as `{width, height, method}`. Empty = no thumbnails. |
| `media.pending_upload_expiry` | `WARP_MEDIA_PENDING_UPLOAD_EXPIRY` | `24h` | How long an ID reserved with `POST /media/v1/create` waits for its content. |
| `media.max_pending_uploads` | `WARP_MEDIA_MAX_PENDING_UPLOADS` | `10` | How many of those one user may hold. |
| `media.legacy_unauthenticated_downloads` | `WARP_MEDIA_LEGACY_UNAUTHENTICATED_DOWNLOADS` | `false` | Also serve the deprecated `/_matrix/media/v3/download` and `/thumbnail`, which need no token. Only for clients older than Matrix 1.11. |
| `registration.enabled` | `WARP_REGISTRATION_ENABLED` | `false` | |
| `registration.allow_runtime_toggle` | `WARP_REGISTRATION_ALLOW_RUNTIME_TOGGLE` | `true` | |
| `registration.tokens` | `WARP_REGISTRATION_TOKENS` | `[]` | Non-empty = sign-up also needs one of these, as the `m.login.registration_token` stage. |
@@ -164,7 +172,9 @@ While `federation.enabled` is false:
time it hears from this one. What was already queued when the switch
went off stays queued, and goes out when it comes back on.
- Client-Server endpoints that would need a remote server answer
`M_FORBIDDEN` with a message saying federation is disabled.
`M_FORBIDDEN` with a message saying federation is disabled. That
includes downloading media from another server that is not already
cached here; media fetched before the switch went off is still served.
- Events are still signed with the server key, and room and event IDs are
unchanged. A room created while federation is off federates normally
once it is switched on.