tajniak81andClaude Opus 5.5
a94794b866
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Implements PLAN.md Phase 9. internal/appservice reads registration files (appservice.config_files), matches namespaces the way Synapse does (anchored at the start), and delivers events to each service as transactions read from the roomserver's event stream at a cursor kept in appservice_state. A transaction in flight is recorded, so after a restart it is resent under the same ID; failures back off from 2s to 5m, and a successful ping cuts the backoff short. In the Client-Server API an as_token acts as the service's sender or, with ?user_id=, as any existing user in its namespace. Services register and log in their users with m.login.application_service; exclusive user and alias namespaces are refused to everybody else with M_EXCLUSIVE. Unknown aliases and users in a namespace are asked of the service first. The ping endpoint reports M_URL_NOT_SET, M_BAD_STATUS, M_CONNECTION_FAILED and M_CONNECTION_TIMEOUT. Services' own requests, and their users' when rate_limited is false, skip the rate limiter. Users can change their password (POST /account/password) and deactivate their account (POST /account/deactivate), both behind the password. Deactivation revokes tokens, deletes devices, optionally erases the profile and leaves every room through a new userapi.AccountObserver. The admin API is complete: user detail, the admin flag, deactivation and reactivation, password resets, devices, media quotas (media.max_user_storage_bytes plus per-user overrides), registration tokens with use limits and expiry (registration.require_token), server notices (internal/servernotices, server_notices.*), an editor for the federation allow/deny lists persisted in server_settings, a delivery view of the application services, and an audit log of every change. Migration 0011 adds the registration token, server notice room, media quota, audit log and appservice cursor tables. Policy lists are now returned sorted. The Complement entrypoint loads the registrations Complement mounts in /complement/appservice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
240 lines
12 KiB
Markdown
240 lines
12 KiB
Markdown
# Media
|
||
|
||
PLAN.md Phase 6: the media repository. Clients upload files and get an
|
||
`mxc://` URI back to put in an event; anybody in the room downloads them,
|
||
or asks for a thumbnail; and media uploaded to another server is fetched
|
||
from it, over federation, the first time somebody here wants it.
|
||
|
||
## Where things live
|
||
|
||
| Package | What |
|
||
|---|---|
|
||
| `internal/mediaapi` | The component: uploads, reserved uploads, downloads, thumbnails, the cache of other servers' media, and the HTTP helpers both APIs share — headers, parameters, error mapping. |
|
||
| `internal/clientapi/media.go` | The Client-Server endpoints. |
|
||
| `internal/federationapi/media.go` | The two endpoints other servers download through. |
|
||
| `internal/federation/media.go` | Downloading from other servers: the multipart answer, redirects, the fallback for servers older than Matrix 1.11. |
|
||
| `internal/storage` | `MediaStore`, the metadata (`media_repository`, `media_thumbnails`), and `BlobStore`, the content. |
|
||
| `internal/storage/filestore` | The local-filesystem `BlobStore`. |
|
||
|
||
## Metadata and content
|
||
|
||
The database holds what an `mxc://` URI means — who uploaded it, its
|
||
content type and file name, its size — and the SHA-256 of its content. The
|
||
content itself is in a `BlobStore`, named by that hash:
|
||
|
||
```
|
||
media_store/
|
||
ab/cd/abcd0123… one file per distinct content
|
||
tmp/ uploads in progress
|
||
```
|
||
|
||
Naming content by its hash gives three things at once. The same file
|
||
uploaded twice — a meme going round, the same avatar set on two accounts —
|
||
is stored once. A thumbnail that turns out to be the original (an image
|
||
already smaller than the size asked for) is a second row pointing at the
|
||
same file rather than a copy. And the hash is the `ETag`, so a client that
|
||
has the file already gets `304 Not Modified`.
|
||
|
||
An upload is written to `tmp/`, hashed as it is written, and renamed into
|
||
place only when it is complete: a reader never sees half a file, and a
|
||
crash leaves a temporary file and nothing else. Nothing in the database
|
||
refers to content until it is all there.
|
||
|
||
The blob store is chosen by the storage driver. SQLite and PostgreSQL use
|
||
the local filesystem under `media.base_path`. A driver that brings its own
|
||
file storage implements `storage.BlobProvider`, and PocketBase will — its
|
||
filesystem abstraction is local disk or S3 (PLAN.md section 4) — when that
|
||
driver is built in Phase 10.
|
||
|
||
## Endpoints
|
||
|
||
| | |
|
||
|---|---|
|
||
| `POST /_matrix/media/v3/upload` | upload; `?filename=` names it |
|
||
| `POST /_matrix/media/v1/create` | reserve an `mxc://` URI to upload to later |
|
||
| `PUT /_matrix/media/v3/upload/{server}/{id}` | upload to a reserved URI |
|
||
| `GET /_matrix/client/v1/media/config` (and `/_matrix/media/v3/config`) | `m.upload.size` |
|
||
| `GET /_matrix/client/v1/media/download/{server}/{id}[/{fileName}]` | download; the file name in the path overrides the stored one |
|
||
| `GET /_matrix/client/v1/media/thumbnail/{server}/{id}` | thumbnail: `width`, `height`, `method`, `animated` |
|
||
| `GET /_matrix/federation/v1/media/download/{id}` | another server downloading this server's media |
|
||
| `GET /_matrix/federation/v1/media/thumbnail/{id}` | and its thumbnails |
|
||
|
||
Every download endpoint takes `?timeout_ms=`: how long to wait for the
|
||
content of a reserved URI that has not been uploaded yet (20 seconds by
|
||
default, two minutes at most). Past it the answer is `504
|
||
M_NOT_YET_UPLOADED`.
|
||
|
||
### Authenticated media only
|
||
|
||
Matrix 1.11 moved downloads under `/_matrix/client/v1/media`, behind an
|
||
access token, because the old endpoints let anybody who learned an
|
||
`mxc://` URI read the file — no account, no room membership. Warp serves
|
||
only the new ones by default. The old unauthenticated
|
||
`/_matrix/media/v3/download` and `/thumbnail` come back with
|
||
`media.legacy_unauthenticated_downloads: true`, for clients too old to know
|
||
better; with it they also honour `allow_remote=false`.
|
||
|
||
`GET /_matrix/client/versions` lists v1.11 and later, which is what tells a
|
||
client to use the authenticated endpoints. The access token may come in the
|
||
`Authorization` header or as `?access_token=`, which is what a web client
|
||
needs for an `<img src>`.
|
||
|
||
An access token proves an account, not membership: like every other server,
|
||
Warp lets any logged-in user download any media whose URI they know.
|
||
Tying media to the rooms it was sent in is MSC3911, not yet in the spec.
|
||
|
||
### What a download is served with
|
||
|
||
- **`Content-Disposition`**: `inline` only for the types the spec lists as
|
||
safe to show in a browser — images, audio, video, plain text, CSS, JSON —
|
||
and `attachment` for everything else. An uploaded HTML page is a
|
||
download, never a page on the homeserver's origin. The file name is
|
||
RFC 6266-encoded when it is not ASCII.
|
||
- **`Content-Security-Policy: sandbox; default-src 'none'; …`** so that
|
||
content a browser is tricked into rendering anyway runs no script.
|
||
- **`X-Content-Type-Options: nosniff`** so a `text/plain` upload is not
|
||
sniffed into HTML.
|
||
- **`Cross-Origin-Resource-Policy: cross-origin`** so a web client on
|
||
another origin can embed the images.
|
||
- **`Cache-Control: private, max-age=31536000, immutable`**: the content of
|
||
an `mxc://` URI never changes, but the endpoint needs a token, so no
|
||
shared cache may keep it.
|
||
- **`ETag`**, the content's hash, and `Range` support, from
|
||
`http.ServeContent`.
|
||
|
||
The uploaded `Content-Type` is kept as sent when it parses, and replaced by
|
||
`application/octet-stream` when it does not.
|
||
|
||
## Reserved uploads
|
||
|
||
`POST /media/v1/create` hands out an `mxc://` URI before there is anything
|
||
behind it, so a client can send the event that refers to a large file
|
||
while the file is still uploading. The reservation belongs to the user who
|
||
made it, lasts `media.pending_upload_expiry` (a day), and takes exactly one
|
||
upload: a second is `409 M_CANNOT_OVERWRITE_MEDIA`, an upload by anybody
|
||
else `403 M_FORBIDDEN`. A user may hold `media.max_pending_uploads` (ten)
|
||
at once; past that, `429 M_LIMIT_EXCEEDED`.
|
||
|
||
A download of a reserved URI waits for its content — in memory, woken by
|
||
the upload the moment it completes — for as long as its `timeout_ms`
|
||
allows. Lapsed reservations are swept whenever a new one is made, which is
|
||
the only time their number matters.
|
||
|
||
## Thumbnails
|
||
|
||
A client asks for a thumbnail of a size; the server answers with one of its
|
||
own sizes, `media.thumbnail_sizes`:
|
||
|
||
| width × height | method | for |
|
||
|---|---|---|
|
||
| 32 × 32 | crop | avatars in lists |
|
||
| 96 × 96 | crop | avatars |
|
||
| 320 × 240 | scale | images in the timeline |
|
||
| 640 × 480 | scale | |
|
||
| 800 × 600 | scale | |
|
||
|
||
The answer is the smallest size of the method asked for that is at least
|
||
as large in both dimensions, or the largest if none is. A method with no
|
||
configured sizes is answered from the other. Fixing the sizes is what makes
|
||
thumbnails cacheable at all: a client asking for 33 × 33 and another
|
||
asking for 34 × 34 share one, and nobody can make the server resize the
|
||
same image a thousand times by asking for a thousand sizes.
|
||
|
||
- **`scale`** fits the image inside the box and keeps its aspect ratio.
|
||
- **`crop`** fills the box exactly, cutting the longer side down equally
|
||
at both ends.
|
||
- Neither enlarges. An image smaller than the box is cropped at its own
|
||
size; one that already fits a `scale` box is served as it is — the
|
||
thumbnail row points at the original's content.
|
||
|
||
Thumbnails are made on first request and kept. JPEG, PNG, GIF and WebP are
|
||
decoded (WebP through `golang.org/x/image`); a JPEG thumbnail is a JPEG,
|
||
anything else a PNG, which keeps transparency. A JPEG's EXIF orientation is
|
||
honoured, so a phone's portrait photo is not a sideways thumbnail. GIFs are
|
||
always re-encoded, as a still image: Warp makes no animated thumbnails, and
|
||
`animated=true` gets a still one, which the spec allows.
|
||
|
||
What could hurt the server is bounded. The image header is read first, and
|
||
anything larger than `media.max_thumbnail_pixels` (32 megapixels) is
|
||
refused before a pixel is decoded — a 100 KiB PNG can claim to be 30000 ×
|
||
30000, and decoding costs four bytes a pixel whatever the file size. At
|
||
most four thumbnails are made at once, and concurrent requests for the same
|
||
one share one resize.
|
||
|
||
Content that cannot be thumbnailed — a PDF, a video, an image that does not
|
||
decode — is `400 M_UNKNOWN`.
|
||
|
||
## Other servers' media
|
||
|
||
Media whose `mxc://` server is another server is fetched from it on first
|
||
request and kept in the same tables and blob store as local media, under
|
||
its own server name. An `mxc://` URI names immutable content, so the copy
|
||
never goes stale.
|
||
|
||
Fetching uses the signed federation client:
|
||
|
||
1. `GET /_matrix/federation/v1/media/download/{id}`, whose answer is
|
||
`multipart/mixed`: a JSON part (an empty object, for now) and then
|
||
either the content or a part with a `Location` header saying where the
|
||
content is.
|
||
2. A server older than Matrix 1.11 answers that with `404 M_UNRECOGNIZED`,
|
||
and is then asked through the deprecated
|
||
`GET /_matrix/media/v3/download/{server}/{id}?allow_remote=false` —
|
||
what every server before 1.11 served media to other servers through.
|
||
A plain `404 M_NOT_FOUND` is not retried: that server knows the
|
||
endpoint and lacks the media.
|
||
|
||
A `Location` is an ordinary web address, often a CDN, fetched without
|
||
signing. It is the one place another server can make this one fetch an
|
||
arbitrary URL and hand the answer to a client, so the dialer checks the
|
||
address it actually connects to — after DNS, on every redirect — and
|
||
refuses loopback, private, link-local and multicast addresses. A remote
|
||
server cannot use it to read this server's internal network.
|
||
|
||
Concurrent requests for the same remote media share one fetch, which runs
|
||
detached from the request that started it: a client that gives up does not
|
||
waste the download for the others, and what was fetched is kept. The
|
||
content is held to `media.max_upload_bytes`, like an upload.
|
||
|
||
Thumbnails of remote media come from the cached original when there is
|
||
one; otherwise the remote server is asked for a thumbnail of the
|
||
configured size that answers the request, and that is cached under that
|
||
size. The original is not downloaded just to make a thumbnail.
|
||
|
||
Another server is served this server's own media only. It is never made to
|
||
fetch from a third server on another's behalf.
|
||
|
||
With federation off, nothing is fetched: media that was cached before is
|
||
still served, and anything else from another server is `403 M_FORBIDDEN`
|
||
with a message saying federation is disabled.
|
||
|
||
## Errors
|
||
|
||
| | |
|
||
|---|---|
|
||
| `404 M_NOT_FOUND` | no such media, here or on its server |
|
||
| `413 M_TOO_LARGE` | larger than `media.max_upload_bytes`, uploaded or fetched; or over the uploader's quota |
|
||
| `504 M_NOT_YET_UPLOADED` | reserved, not uploaded within `timeout_ms` |
|
||
| `409 M_CANNOT_OVERWRITE_MEDIA` | a reserved URI that already has content |
|
||
| `403 M_FORBIDDEN` | someone else's reservation; or remote media with federation off |
|
||
| `429 M_LIMIT_EXCEEDED` | too many reservations |
|
||
| `400 M_UNKNOWN` | cannot make a thumbnail of this |
|
||
| `502 M_UNKNOWN` | the media's server failed to provide it |
|
||
|
||
## Known gaps
|
||
|
||
- **URL previews** (`/media/v3/preview_url`) are not implemented and answer
|
||
`M_UNRECOGNIZED`. Fetching arbitrary URLs for clients needs the same
|
||
network guard as media redirects plus an HTML parser, and is its own
|
||
piece of work.
|
||
- **No deletion yet.** Nothing removes media: no admin purge, and no
|
||
retention for the remote cache. Content-addressed storage means a purge
|
||
has to check that no other row still uses a blob before deleting it.
|
||
Per-user quotas are in (Phase 9): `media.max_user_storage_bytes`, and a
|
||
quota per user through the admin API. An upload over the quota is
|
||
`413 M_TOO_LARGE`. See [admin-api.md](admin-api.md).
|
||
- **No quarantine**, which belongs with the same admin work.
|
||
- **No animated thumbnails**; `animated=true` gets a still image.
|
||
- **No S3** until the PocketBase driver brings its filesystem abstraction;
|
||
the `BlobProvider` seam is where it plugs in.
|