diff --git a/Dockerfile b/Dockerfile index 9e869ef..f3a77da 100644 --- a/Dockerfile +++ b/Dockerfile @@ -44,6 +44,7 @@ ENV WARP_LISTEN_ADDRESS=":8008" \ WARP_PRIVATE_KEY="/data/signing.key" \ WARP_DB_DRIVER="sqlite" \ WARP_DB_CONNECTION_STRING="file:/data/warp.db?_pragma=journal_mode(WAL)&_pragma=busy_timeout(10000)&_pragma=foreign_keys(1)" \ + WARP_MEDIA_BASE_PATH="/data/media" \ WARP_LOG_FORMAT="json" EXPOSE 8008 diff --git a/README.md b/README.md index 2847701..8d47fb8 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,20 @@ what exists today. ## Status +**Phase 6 — media. Complete.** Clients can upload files, reserve an +`mxc://` URI and upload to it later, download and ask for thumbnails, all +through the authenticated media endpoints of Matrix 1.11. Content is +stored once per distinct file, named by its SHA-256, on the local +filesystem; thumbnails are made on first request from a fixed set of +sizes, the right way up for a phone's photos, with the pixel count checked +before anything is decoded. Media from other servers is fetched over +federation on first use — with the fallback older servers need, and +redirects that cannot be pointed into a private network — and cached, so +it is still there with federation switched off. Every download carries the +headers that keep an uploaded file from running as a page on the +homeserver's origin. Known gaps — no URL previews, no deletion or quotas +until Phase 9 — are listed in [docs/media.md](docs/media.md). + **Phase 5 — federation. Complete.** Warp talks to other homeservers. It joins rooms on other servers and lets their users join its rooms, invites across servers with the invitee's server countersigning, rejects remote @@ -87,6 +101,8 @@ Client-Server API: | `POST /rooms/{id}/receipt/{type}/{event}` · `/read_markers` | read receipts, public and private, and the read marker | | `PUT /sendToDevice/{type}/{txn}` | device-to-device messaging, for E2EE in Phase 7 | | `GET`/`PUT /presence/{user}/status` | presence — **only while `sync.presence` is on**, and it is off by default | +| `POST /_matrix/media/v3/upload` · `POST /_matrix/media/v1/create` · `PUT /_matrix/media/v3/upload/{server}/{id}` | uploads, directly or to a reserved URI | +| `GET /_matrix/client/v1/media/download/…` · `/thumbnail/…` · `/config` | downloads and thumbnails, local and from other servers; see [docs/media.md](docs/media.md) | | `GET /_matrix/client/versions` · `/.well-known/matrix/client` | discovery | | `/.well-known/matrix/server` | server discovery — **only while federation is on** | | `/_matrix/federation/*` · `/_matrix/key/*` | the Server-Server API — **only while federation is on**; see [docs/federation.md](docs/federation.md) | @@ -110,8 +126,9 @@ profiles. `make complement` runs the account, room and stream tests with federation off — `TestRegistration`, `TestLogin`, `TestDeviceManagement`, the Phase 3 room tests and the Phase 4 `TestSync*`, `TestMessages*`, `TestContext*`, `TestFilter*`, `TestTyping*`, `TestReceipt*` and -`TestToDevice*`. `make complement-federation` runs the Phase 5 -Server-Server tests with federation on, over TLS with Complement's CA. +`TestToDevice*`, and the Phase 6 media tests. `make complement-federation` +runs the Phase 5 Server-Server tests with federation on, over TLS with +Complement's CA. ## Quick start @@ -235,6 +252,7 @@ server/ the homeserver (Go module) settings/ runtime settings service (the switches) userapi/ accounts, passwords, devices, tokens, account data roomserver/ rooms: the input pipeline (docs/rooms.md) + mediaapi/ uploads, downloads, thumbnails, remote media (docs/media.md) clientapi/ the Client-Server API handlers adminapi/ /_warp/admin/v1 setup/ wiring: storage, routes, graceful shutdown @@ -249,6 +267,7 @@ server/ the homeserver (Go module) storage/ interfaces + driver registry migrations/ embedded migration runner sqlstore/ shared database/sql implementation + filestore/ media content on the local filesystem sqlite/ postgres/ pocketbase/ test/complement/ the Complement homeserver image and runner apps/ clients: ui, admin, web, desktop, mobile (Phase 1+) diff --git a/docs/README.md b/docs/README.md index d372734..abfa0d6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,6 +8,7 @@ | [rooms.md](rooms.md) | `internal/roomserver`: the input pipeline, room storage, known gaps | | [sync.md](sync.md) | `internal/syncapi`: stream tokens, long polling, filters, the ephemeral streams | | [federation.md](federation.md) | the Server-Server API: discovery, keys, taking in events, joins across servers, the outbound queue, the switch | +| [media.md](media.md) | `internal/mediaapi`: uploads, downloads, thumbnails, other servers' media, where the files go | | [admin-api.md](admin-api.md) | `/_warp/admin/v1`: the operator API behind the admin panel | The design and roadmap are in [PLAN.md](../PLAN.md) at the repository root. diff --git a/docs/configuration.md b/docs/configuration.md index 113e05f..59b0712 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -53,6 +53,7 @@ List-valued variables are comma-separated. A single `-` clears a list: | `client_api.rate_limit.room_create.burst` / `.interval` | — | `10` / `1s` | `/createRoom`, which writes a dozen events per call. | | `client_api.rate_limit.join.burst` / `.interval` | — | `10` / `1s` | The two join endpoints. | | `client_api.rate_limit.sync.burst` / `.interval` | — | `100` / `20ms` | `/sync` and the writes behind it: typing, receipts, to-device, presence. | +| `client_api.rate_limit.media.burst` / `.interval` | — | `100` / `20ms` | Media downloads and thumbnails: a client opening a room asks for every avatar at once. Uploads are in the default bucket. | | `sync.presence` | `WARP_SYNC_PRESENCE` | `false` | Off by default; see below. | | `sync.typing_timeout` | `WARP_SYNC_TYPING_TIMEOUT` | `30s` | How long a typing notification lasts, and the cap on what a client may ask for. | | `sync.long_poll_timeout` | `WARP_SYNC_LONG_POLL_TIMEOUT` | `30s` | The cap on a `/sync` `?timeout=`. | @@ -63,6 +64,13 @@ List-valued variables are comma-separated. A single `-` clears a list: | `federation.trusted_key_servers` | `WARP_FEDERATION_TRUSTED_KEY_SERVERS` | `[]` | Notaries asked for a server's key when it cannot be reached itself, e.g. `matrix.org`. Empty = only ever ask the server. | | `federation.key_validity_period` | `WARP_FEDERATION_KEY_VALIDITY_PERIOD` | `24h` | How long other servers may cache this server's key. At most `168h`. | | `federation.tls_ca_file` | `WARP_FEDERATION_TLS_CA_FILE` | empty | Extra CA certificates (PEM) for outbound federation, on top of the system roots. | +| `media.base_path` | `WARP_MEDIA_BASE_PATH` | `./media_store` | Where media content is kept. Created on the first upload. | +| `media.max_upload_bytes` | `WARP_MEDIA_MAX_UPLOAD_BYTES` | `52428800` | The largest upload (50 MiB), and the largest file fetched from another server. Reported to clients as `m.upload.size`. | +| `media.max_thumbnail_pixels` | `WARP_MEDIA_MAX_THUMBNAIL_PIXELS` | `32000000` | The largest image, in pixels, decoded to make a thumbnail. | +| `media.thumbnail_sizes` | — | five sizes, see [media.md](media.md) | The thumbnails this server makes, as `{width, height, method}`. Empty = no thumbnails. | +| `media.pending_upload_expiry` | `WARP_MEDIA_PENDING_UPLOAD_EXPIRY` | `24h` | How long an ID reserved with `POST /media/v1/create` waits for its content. | +| `media.max_pending_uploads` | `WARP_MEDIA_MAX_PENDING_UPLOADS` | `10` | How many of those one user may hold. | +| `media.legacy_unauthenticated_downloads` | `WARP_MEDIA_LEGACY_UNAUTHENTICATED_DOWNLOADS` | `false` | Also serve the deprecated `/_matrix/media/v3/download` and `/thumbnail`, which need no token. Only for clients older than Matrix 1.11. | | `registration.enabled` | `WARP_REGISTRATION_ENABLED` | `false` | | | `registration.allow_runtime_toggle` | `WARP_REGISTRATION_ALLOW_RUNTIME_TOGGLE` | `true` | | | `registration.tokens` | `WARP_REGISTRATION_TOKENS` | `[]` | Non-empty = sign-up also needs one of these, as the `m.login.registration_token` stage. | @@ -164,7 +172,9 @@ While `federation.enabled` is false: time it hears from this one. What was already queued when the switch went off stays queued, and goes out when it comes back on. - Client-Server endpoints that would need a remote server answer - `M_FORBIDDEN` with a message saying federation is disabled. + `M_FORBIDDEN` with a message saying federation is disabled. That + includes downloading media from another server that is not already + cached here; media fetched before the switch went off is still served. - Events are still signed with the server key, and room and event IDs are unchanged. A room created while federation is off federates normally once it is switched on. diff --git a/docs/media.md b/docs/media.md new file mode 100644 index 0000000..263be6f --- /dev/null +++ b/docs/media.md @@ -0,0 +1,238 @@ +# Media + +PLAN.md Phase 6: the media repository. Clients upload files and get an +`mxc://` URI back to put in an event; anybody in the room downloads them, +or asks for a thumbnail; and media uploaded to another server is fetched +from it, over federation, the first time somebody here wants it. + +## Where things live + +| Package | What | +|---|---| +| `internal/mediaapi` | The component: uploads, reserved uploads, downloads, thumbnails, the cache of other servers' media, and the HTTP helpers both APIs share — headers, parameters, error mapping. | +| `internal/clientapi/media.go` | The Client-Server endpoints. | +| `internal/federationapi/media.go` | The two endpoints other servers download through. | +| `internal/federation/media.go` | Downloading from other servers: the multipart answer, redirects, the fallback for servers older than Matrix 1.11. | +| `internal/storage` | `MediaStore`, the metadata (`media_repository`, `media_thumbnails`), and `BlobStore`, the content. | +| `internal/storage/filestore` | The local-filesystem `BlobStore`. | + +## Metadata and content + +The database holds what an `mxc://` URI means — who uploaded it, its +content type and file name, its size — and the SHA-256 of its content. The +content itself is in a `BlobStore`, named by that hash: + +``` +media_store/ + ab/cd/abcd0123… one file per distinct content + tmp/ uploads in progress +``` + +Naming content by its hash gives three things at once. The same file +uploaded twice — a meme going round, the same avatar set on two accounts — +is stored once. A thumbnail that turns out to be the original (an image +already smaller than the size asked for) is a second row pointing at the +same file rather than a copy. And the hash is the `ETag`, so a client that +has the file already gets `304 Not Modified`. + +An upload is written to `tmp/`, hashed as it is written, and renamed into +place only when it is complete: a reader never sees half a file, and a +crash leaves a temporary file and nothing else. Nothing in the database +refers to content until it is all there. + +The blob store is chosen by the storage driver. SQLite and PostgreSQL use +the local filesystem under `media.base_path`. A driver that brings its own +file storage implements `storage.BlobProvider`, and PocketBase will — its +filesystem abstraction is local disk or S3 (PLAN.md section 4) — when that +driver is built in Phase 10. + +## Endpoints + +| | | +|---|---| +| `POST /_matrix/media/v3/upload` | upload; `?filename=` names it | +| `POST /_matrix/media/v1/create` | reserve an `mxc://` URI to upload to later | +| `PUT /_matrix/media/v3/upload/{server}/{id}` | upload to a reserved URI | +| `GET /_matrix/client/v1/media/config` (and `/_matrix/media/v3/config`) | `m.upload.size` | +| `GET /_matrix/client/v1/media/download/{server}/{id}[/{fileName}]` | download; the file name in the path overrides the stored one | +| `GET /_matrix/client/v1/media/thumbnail/{server}/{id}` | thumbnail: `width`, `height`, `method`, `animated` | +| `GET /_matrix/federation/v1/media/download/{id}` | another server downloading this server's media | +| `GET /_matrix/federation/v1/media/thumbnail/{id}` | and its thumbnails | + +Every download endpoint takes `?timeout_ms=`: how long to wait for the +content of a reserved URI that has not been uploaded yet (20 seconds by +default, two minutes at most). Past it the answer is `504 +M_NOT_YET_UPLOADED`. + +### Authenticated media only + +Matrix 1.11 moved downloads under `/_matrix/client/v1/media`, behind an +access token, because the old endpoints let anybody who learned an +`mxc://` URI read the file — no account, no room membership. Warp serves +only the new ones by default. The old unauthenticated +`/_matrix/media/v3/download` and `/thumbnail` come back with +`media.legacy_unauthenticated_downloads: true`, for clients too old to know +better; with it they also honour `allow_remote=false`. + +`GET /_matrix/client/versions` lists v1.11 and later, which is what tells a +client to use the authenticated endpoints. The access token may come in the +`Authorization` header or as `?access_token=`, which is what a web client +needs for an ``. + +An access token proves an account, not membership: like every other server, +Warp lets any logged-in user download any media whose URI they know. +Tying media to the rooms it was sent in is MSC3911, not yet in the spec. + +### What a download is served with + +- **`Content-Disposition`**: `inline` only for the types the spec lists as + safe to show in a browser — images, audio, video, plain text, CSS, JSON — + and `attachment` for everything else. An uploaded HTML page is a + download, never a page on the homeserver's origin. The file name is + RFC 6266-encoded when it is not ASCII. +- **`Content-Security-Policy: sandbox; default-src 'none'; …`** so that + content a browser is tricked into rendering anyway runs no script. +- **`X-Content-Type-Options: nosniff`** so a `text/plain` upload is not + sniffed into HTML. +- **`Cross-Origin-Resource-Policy: cross-origin`** so a web client on + another origin can embed the images. +- **`Cache-Control: private, max-age=31536000, immutable`**: the content of + an `mxc://` URI never changes, but the endpoint needs a token, so no + shared cache may keep it. +- **`ETag`**, the content's hash, and `Range` support, from + `http.ServeContent`. + +The uploaded `Content-Type` is kept as sent when it parses, and replaced by +`application/octet-stream` when it does not. + +## Reserved uploads + +`POST /media/v1/create` hands out an `mxc://` URI before there is anything +behind it, so a client can send the event that refers to a large file +while the file is still uploading. The reservation belongs to the user who +made it, lasts `media.pending_upload_expiry` (a day), and takes exactly one +upload: a second is `409 M_CANNOT_OVERWRITE_MEDIA`, an upload by anybody +else `403 M_FORBIDDEN`. A user may hold `media.max_pending_uploads` (ten) +at once; past that, `429 M_LIMIT_EXCEEDED`. + +A download of a reserved URI waits for its content — in memory, woken by +the upload the moment it completes — for as long as its `timeout_ms` +allows. Lapsed reservations are swept whenever a new one is made, which is +the only time their number matters. + +## Thumbnails + +A client asks for a thumbnail of a size; the server answers with one of its +own sizes, `media.thumbnail_sizes`: + +| width × height | method | for | +|---|---|---| +| 32 × 32 | crop | avatars in lists | +| 96 × 96 | crop | avatars | +| 320 × 240 | scale | images in the timeline | +| 640 × 480 | scale | | +| 800 × 600 | scale | | + +The answer is the smallest size of the method asked for that is at least +as large in both dimensions, or the largest if none is. A method with no +configured sizes is answered from the other. Fixing the sizes is what makes +thumbnails cacheable at all: a client asking for 33 × 33 and another +asking for 34 × 34 share one, and nobody can make the server resize the +same image a thousand times by asking for a thousand sizes. + +- **`scale`** fits the image inside the box and keeps its aspect ratio. +- **`crop`** fills the box exactly, cutting the longer side down equally + at both ends. +- Neither enlarges. An image smaller than the box is cropped at its own + size; one that already fits a `scale` box is served as it is — the + thumbnail row points at the original's content. + +Thumbnails are made on first request and kept. JPEG, PNG, GIF and WebP are +decoded (WebP through `golang.org/x/image`); a JPEG thumbnail is a JPEG, +anything else a PNG, which keeps transparency. A JPEG's EXIF orientation is +honoured, so a phone's portrait photo is not a sideways thumbnail. GIFs are +always re-encoded, as a still image: Warp makes no animated thumbnails, and +`animated=true` gets a still one, which the spec allows. + +What could hurt the server is bounded. The image header is read first, and +anything larger than `media.max_thumbnail_pixels` (32 megapixels) is +refused before a pixel is decoded — a 100 KiB PNG can claim to be 30000 × +30000, and decoding costs four bytes a pixel whatever the file size. At +most four thumbnails are made at once, and concurrent requests for the same +one share one resize. + +Content that cannot be thumbnailed — a PDF, a video, an image that does not +decode — is `400 M_UNKNOWN`. + +## Other servers' media + +Media whose `mxc://` server is another server is fetched from it on first +request and kept in the same tables and blob store as local media, under +its own server name. An `mxc://` URI names immutable content, so the copy +never goes stale. + +Fetching uses the signed federation client: + +1. `GET /_matrix/federation/v1/media/download/{id}`, whose answer is + `multipart/mixed`: a JSON part (an empty object, for now) and then + either the content or a part with a `Location` header saying where the + content is. +2. A server older than Matrix 1.11 answers that with `404 M_UNRECOGNIZED`, + and is then asked through the deprecated + `GET /_matrix/media/v3/download/{server}/{id}?allow_remote=false` — + what every server before 1.11 served media to other servers through. + A plain `404 M_NOT_FOUND` is not retried: that server knows the + endpoint and lacks the media. + +A `Location` is an ordinary web address, often a CDN, fetched without +signing. It is the one place another server can make this one fetch an +arbitrary URL and hand the answer to a client, so the dialer checks the +address it actually connects to — after DNS, on every redirect — and +refuses loopback, private, link-local and multicast addresses. A remote +server cannot use it to read this server's internal network. + +Concurrent requests for the same remote media share one fetch, which runs +detached from the request that started it: a client that gives up does not +waste the download for the others, and what was fetched is kept. The +content is held to `media.max_upload_bytes`, like an upload. + +Thumbnails of remote media come from the cached original when there is +one; otherwise the remote server is asked for a thumbnail of the +configured size that answers the request, and that is cached under that +size. The original is not downloaded just to make a thumbnail. + +Another server is served this server's own media only. It is never made to +fetch from a third server on another's behalf. + +With federation off, nothing is fetched: media that was cached before is +still served, and anything else from another server is `403 M_FORBIDDEN` +with a message saying federation is disabled. + +## Errors + +| | | +|---|---| +| `404 M_NOT_FOUND` | no such media, here or on its server | +| `413 M_TOO_LARGE` | larger than `media.max_upload_bytes`, uploaded or fetched | +| `504 M_NOT_YET_UPLOADED` | reserved, not uploaded within `timeout_ms` | +| `409 M_CANNOT_OVERWRITE_MEDIA` | a reserved URI that already has content | +| `403 M_FORBIDDEN` | someone else's reservation; or remote media with federation off | +| `429 M_LIMIT_EXCEEDED` | too many reservations | +| `400 M_UNKNOWN` | cannot make a thumbnail of this | +| `502 M_UNKNOWN` | the media's server failed to provide it | + +## Known gaps + +- **URL previews** (`/media/v3/preview_url`) are not implemented and answer + `M_UNRECOGNIZED`. Fetching arbitrary URLs for clients needs the same + network guard as media redirects plus an HTML parser, and is its own + piece of work. +- **No deletion or quotas yet.** Nothing removes media: no admin purge, no + retention for the remote cache, no per-user quota. Those are the media + parts of Phase 9's admin work, and content-addressed storage means a + purge has to check that no other row still uses a blob before deleting + it. +- **No quarantine**, which belongs with the same admin work. +- **No animated thumbnails**; `animated=true` gets a still image. +- **No S3** until the PocketBase driver brings its filesystem abstraction; + the `BlobProvider` seam is where it plugs in. diff --git a/server/go.mod b/server/go.mod index 10b8ce4..d525794 100644 --- a/server/go.mod +++ b/server/go.mod @@ -6,6 +6,8 @@ require ( github.com/jackc/pgx/v5 v5.11.0 github.com/prometheus/client_golang v1.25.0 golang.org/x/crypto v0.58.0 + golang.org/x/image v0.46.0 + golang.org/x/sync v0.24.0 golang.org/x/term v0.47.0 gopkg.in/yaml.v3 v3.0.1 modernc.org/sqlite v1.60.1 @@ -28,7 +30,6 @@ require ( github.com/prometheus/procfs v0.22.0 // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/rogpeppe/go-internal v1.16.0 // indirect - golang.org/x/sync v0.24.0 // indirect golang.org/x/sys v0.49.0 // indirect golang.org/x/text v0.43.0 // indirect google.golang.org/protobuf v1.36.12 // indirect diff --git a/server/go.sum b/server/go.sum index a11e156..e13ad3c 100644 --- a/server/go.sum +++ b/server/go.sum @@ -62,6 +62,8 @@ go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54= golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w= +golang.org/x/image v0.46.0 h1:b1+oYj0Jbp6K5MDT4i4/eZpYlk3V8SJhhDKh6LBHAyQ= +golang.org/x/image v0.46.0/go.mod h1:3B3W05VGVQyuXucLINLjXKrqISASfi4Xj+iCVkLMwew= golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c= golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o= golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA= diff --git a/server/internal/clientapi/clientapi.go b/server/internal/clientapi/clientapi.go index 2a2e5e3..20f2502 100644 --- a/server/internal/clientapi/clientapi.go +++ b/server/internal/clientapi/clientapi.go @@ -20,6 +20,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/config" "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" "github.com/dariuszjaniak/warp/server/internal/ratelimit" "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" @@ -34,6 +35,8 @@ type ClientAPI struct { users *userapi.API rooms *roomserver.API sync *syncapi.API + media *mediaapi.API + legacyMedia bool settings *settings.Service federation *federation.Gate remoteProfile RemoteProfiles @@ -57,6 +60,10 @@ type ClientAPI struct { // loop — the one place where rate limiting makes the load worse rather // than better. limitSync *ratelimit.Limiter + + // limitMedia covers downloads and thumbnails, which a client asks for + // in bursts: every avatar in a room list at once. + limitMedia *ratelimit.Limiter } // Options configure the component. @@ -85,6 +92,14 @@ type Options struct { // pretending there is a stream to read. Sync *syncapi.API + // Media is the media repository. Without it the media endpoints + // answer M_UNRECOGNIZED. + Media *mediaapi.API + + // LegacyMediaDownloads serves the deprecated unauthenticated download + // and thumbnail endpoints too. + LegacyMediaDownloads bool + // Settings is the runtime-settings service; the registration switch is // read from it on every registration attempt. Settings *settings.Service @@ -138,6 +153,8 @@ func New(opts Options) (*ClientAPI, error) { users: opts.Users, rooms: opts.Rooms, sync: opts.Sync, + media: opts.Media, + legacyMedia: opts.LegacyMediaDownloads, settings: opts.Settings, federation: opts.Federation, remoteProfile: opts.RemoteProfiles, @@ -159,11 +176,12 @@ func New(opts Options) (*ClientAPI, error) { c.limitRoomCreate = ratelimit.New(bucket(opts.RateLimit.RoomCreate)) c.limitJoin = ratelimit.New(bucket(opts.RateLimit.Join)) c.limitSync = ratelimit.New(bucket(opts.RateLimit.Sync)) + c.limitMedia = ratelimit.New(bucket(opts.RateLimit.Media)) } else { c.limitDefault, c.limitLogin, c.limitRegister = ratelimit.Disabled(), ratelimit.Disabled(), ratelimit.Disabled() c.limitRoomCreate, c.limitJoin = ratelimit.Disabled(), ratelimit.Disabled() - c.limitSync = ratelimit.Disabled() + c.limitSync, c.limitMedia = ratelimit.Disabled(), ratelimit.Disabled() } return c, nil } @@ -229,6 +247,7 @@ func (c *ClientAPI) RegisterRoutes(r *httputil.Router) { c.registerRoomRoutes(r) c.registerSyncRoutes(r) + c.registerMediaRoutes(r) } // registerSyncRoutes adds the endpoints the syncapi serves, if there is diff --git a/server/internal/clientapi/media.go b/server/internal/clientapi/media.go new file mode 100644 index 0000000..0413bf0 --- /dev/null +++ b/server/internal/clientapi/media.go @@ -0,0 +1,187 @@ +package clientapi + +import ( + "net/http" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" + "github.com/dariuszjaniak/warp/server/internal/userapi" +) + +// The media endpoints. Uploads keep their Matrix 1.0 paths under +// /_matrix/media; downloads and thumbnails are the authenticated +// endpoints of Matrix 1.11 under /_matrix/client/v1/media. The +// unauthenticated downloads they replaced are only served when +// media.legacy_unauthenticated_downloads is on. +const ( + mediaV1 = "/_matrix/media/v1" + mediaV3 = "/_matrix/media/v3" +) + +// registerMediaRoutes adds the media endpoints, if there is a media +// repository. +func (c *ClientAPI) registerMediaRoutes(r *httputil.Router) { + if c.media == nil { + return + } + + // Uploads take the default bucket: they are rare, and each one is + // up to media.max_upload_bytes of disk. + c.authed(r, "POST "+mediaV3+"/upload", c.limitDefault, c.handleUpload) + c.authed(r, "POST "+mediaV1+"/create", c.limitDefault, c.handleCreateMedia) + c.authed(r, "PUT "+mediaV3+"/upload/{serverName}/{mediaID}", c.limitDefault, c.handleUploadTo) + + c.authed(r, "GET "+v1+"/media/config", c.limitMedia, c.handleMediaConfig) + c.authed(r, "GET "+mediaV3+"/config", c.limitMedia, c.handleMediaConfig) + + c.authed(r, "GET "+v1+"/media/download/{serverName}/{mediaID}", c.limitMedia, c.handleDownload) + c.authed(r, "GET "+v1+"/media/download/{serverName}/{mediaID}/{fileName}", c.limitMedia, c.handleDownload) + c.authed(r, "GET "+v1+"/media/thumbnail/{serverName}/{mediaID}", c.limitMedia, c.handleThumbnail) + + if c.legacyMedia { + c.public(r, "GET "+mediaV3+"/download/{serverName}/{mediaID}", c.limitMedia, c.handleLegacyDownload) + c.public(r, "GET "+mediaV3+"/download/{serverName}/{mediaID}/{fileName}", c.limitMedia, c.handleLegacyDownload) + c.public(r, "GET "+mediaV3+"/thumbnail/{serverName}/{mediaID}", c.limitMedia, c.handleLegacyThumbnail) + } +} + +type uploadResponse struct { + ContentURI string `json:"content_uri"` +} + +// handleUpload implements POST /_matrix/media/v3/upload. +func (c *ClientAPI) handleUpload(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + if err := c.checkUploadSize(r); err != nil { + return err + } + mxc, err := c.media.Upload(r.Context(), upload(r, auth)) + if err != nil { + return mediaapi.HTTPError(err) + } + httputil.WriteJSON(w, http.StatusOK, uploadResponse{ContentURI: mxc}) + return nil +} + +type createMediaResponse struct { + ContentURI string `json:"content_uri"` + UnusedExpiresAt int64 `json:"unused_expires_at"` +} + +// handleCreateMedia implements POST /_matrix/media/v1/create: an mxc:// +// URI now, its content later. A client uses it to put a message with an +// attachment in the timeline before the attachment has finished +// uploading. +func (c *ClientAPI) handleCreateMedia(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + mxc, expires, err := c.media.Create(r.Context(), auth.UserID) + if err != nil { + return mediaapi.HTTPError(err) + } + httputil.WriteJSON(w, http.StatusOK, createMediaResponse{ContentURI: mxc, UnusedExpiresAt: expires}) + return nil +} + +// handleUploadTo implements PUT /_matrix/media/v3/upload/{serverName}/{mediaId}. +func (c *ClientAPI) handleUploadTo(w http.ResponseWriter, r *http.Request, auth userapi.Authenticated) error { + if err := c.checkUploadSize(r); err != nil { + return err + } + err := c.media.UploadTo(r.Context(), r.PathValue("serverName"), r.PathValue("mediaID"), upload(r, auth)) + if err != nil { + return mediaapi.HTTPError(err) + } + httputil.WriteJSON(w, http.StatusOK, struct{}{}) + return nil +} + +// checkUploadSize refuses an upload that says up front it is too large, +// before a byte of it is read. One that does not say is cut off by the +// blob store at the same limit. +func (c *ClientAPI) checkUploadSize(r *http.Request) error { + if r.ContentLength > c.media.MaxUploadBytes() { + return mediaapi.HTTPError(mediaapi.ErrTooLarge) + } + return nil +} + +func upload(r *http.Request, auth userapi.Authenticated) mediaapi.Upload { + return mediaapi.Upload{ + UserID: auth.UserID, + ContentType: r.Header.Get("Content-Type"), + FileName: r.URL.Query().Get("filename"), + Body: r.Body, + } +} + +type mediaConfigResponse struct { + UploadSize int64 `json:"m.upload.size"` +} + +// handleMediaConfig implements GET /_matrix/client/v1/media/config and its +// deprecated twin under /_matrix/media/v3. +func (c *ClientAPI) handleMediaConfig(w http.ResponseWriter, _ *http.Request, _ userapi.Authenticated) error { + httputil.WriteJSON(w, http.StatusOK, mediaConfigResponse{UploadSize: c.media.MaxUploadBytes()}) + return nil +} + +// handleDownload implements GET /_matrix/client/v1/media/download/... +func (c *ClientAPI) handleDownload(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error { + return c.serveDownload(w, r, false) +} + +// handleThumbnail implements GET /_matrix/client/v1/media/thumbnail/... +func (c *ClientAPI) handleThumbnail(w http.ResponseWriter, r *http.Request, _ userapi.Authenticated) error { + return c.serveThumbnail(w, r, false) +} + +// handleLegacyDownload implements the deprecated, unauthenticated +// GET /_matrix/media/v3/download/..., honouring its allow_remote=false. +func (c *ClientAPI) handleLegacyDownload(w http.ResponseWriter, r *http.Request) error { + return c.serveDownload(w, r, r.URL.Query().Get("allow_remote") == "false") +} + +// handleLegacyThumbnail implements the deprecated, unauthenticated +// GET /_matrix/media/v3/thumbnail/.... +func (c *ClientAPI) handleLegacyThumbnail(w http.ResponseWriter, r *http.Request) error { + return c.serveThumbnail(w, r, r.URL.Query().Get("allow_remote") == "false") +} + +func (c *ClientAPI) serveDownload(w http.ResponseWriter, r *http.Request, localOnly bool) error { + wait, err := mediaapi.WaitParam(r) + if err != nil { + return err + } + content, err := c.media.Download(r.Context(), r.PathValue("serverName"), r.PathValue("mediaID"), + mediaapi.FetchOptions{Wait: wait, LocalOnly: localOnly}) + if err != nil { + return mediaapi.HTTPError(err) + } + defer func() { _ = content.Body.Close() }() + + // A file name in the path overrides the one it was uploaded with: + // that is what the path segment is for. + name := content.FileName + if override := r.PathValue("fileName"); override != "" { + name = override + } + mediaapi.ServeContent(w, r, content, name) + return nil +} + +func (c *ClientAPI) serveThumbnail(w http.ResponseWriter, r *http.Request, localOnly bool) error { + req, err := mediaapi.ThumbnailParams(r) + if err != nil { + return err + } + wait, err := mediaapi.WaitParam(r) + if err != nil { + return err + } + content, err := c.media.Thumbnail(r.Context(), r.PathValue("serverName"), r.PathValue("mediaID"), req, + mediaapi.FetchOptions{Wait: wait, LocalOnly: localOnly}) + if err != nil { + return mediaapi.HTTPError(err) + } + defer func() { _ = content.Body.Close() }() + mediaapi.ServeContent(w, r, content, mediaapi.ThumbnailFileName(content.ContentType)) + return nil +} diff --git a/server/internal/config/config.go b/server/internal/config/config.go index 3c1e2a5..e473040 100644 --- a/server/internal/config/config.go +++ b/server/internal/config/config.go @@ -24,6 +24,7 @@ type Config struct { ClientAPI ClientAPI `yaml:"client_api"` Sync Sync `yaml:"sync"` Federation Federation `yaml:"federation"` + Media Media `yaml:"media"` Registration Registration `yaml:"registration"` Logging Logging `yaml:"logging"` Metrics Metrics `yaml:"metrics"` @@ -165,6 +166,11 @@ type RateLimit struct { // long poll that is refused leaves it reconnecting in a loop, which is // the one case where rate limiting makes the load worse. Sync RateLimitBucket `yaml:"sync"` + + // Media covers downloads and thumbnails. A client opening a room asks + // for every member's avatar at once, so it is as loose as Sync. + // Uploads stay in the default bucket. + Media RateLimitBucket `yaml:"media"` } // RateLimitBucket is one token bucket: Burst requests may arrive back to @@ -229,6 +235,56 @@ type Federation struct { TLSCAFile string `yaml:"tls_ca_file" env:"WARP_FEDERATION_TLS_CA_FILE"` } +// Media configures the media repository: uploads, downloads, thumbnails +// and the cache of other servers' media. +type Media struct { + // BasePath is the directory media content is stored under. It is + // created on the first upload. A storage driver with its own file + // storage (PocketBase, when it lands) ignores it. + BasePath string `yaml:"base_path" env:"WARP_MEDIA_BASE_PATH"` + + // MaxUploadBytes caps one upload, and one file fetched from another + // server. It is what /media/config reports as m.upload.size. + MaxUploadBytes int64 `yaml:"max_upload_bytes" env:"WARP_MEDIA_MAX_UPLOAD_BYTES"` + + // MaxThumbnailPixels is the largest image, in pixels, that will be + // decoded to make a thumbnail. Decoding allocates four bytes a pixel + // whatever the file size, and a 100 KiB PNG can claim to be 30000 × + // 30000. + MaxThumbnailPixels int64 `yaml:"max_thumbnail_pixels" env:"WARP_MEDIA_MAX_THUMBNAIL_PIXELS"` + + // ThumbnailSizes are the thumbnails this server makes. A request is + // answered with the smallest configured size that is at least as big + // as the one asked for, so arbitrary sizes cannot be used to make the + // server resize the same image a thousand times. + ThumbnailSizes []ThumbnailSize `yaml:"thumbnail_sizes"` + + // PendingUploadExpiry is how long an ID from POST /media/v1/create + // waits for its content before it is thrown away. + PendingUploadExpiry time.Duration `yaml:"pending_upload_expiry" env:"WARP_MEDIA_PENDING_UPLOAD_EXPIRY"` + + // MaxPendingUploads caps the IDs one user may hold without content. + MaxPendingUploads int `yaml:"max_pending_uploads" env:"WARP_MEDIA_MAX_PENDING_UPLOADS"` + + // LegacyUnauthenticatedDownloads keeps the deprecated + // /_matrix/media/v3/download and /thumbnail endpoints, which need no + // access token. Off by default: anybody who learns an mxc:// URI can + // read the file through them, which is why Matrix 1.11 replaced them + // with authenticated endpoints. Turn it on only for clients too old + // to know those. + LegacyUnauthenticatedDownloads bool `yaml:"legacy_unauthenticated_downloads" env:"WARP_MEDIA_LEGACY_UNAUTHENTICATED_DOWNLOADS"` +} + +// ThumbnailSize is one configured thumbnail. +type ThumbnailSize struct { + Width int `yaml:"width"` + Height int `yaml:"height"` + + // Method is "crop" (fill the box exactly, cutting the edges off) or + // "scale" (fit inside the box, keeping the aspect ratio). + Method string `yaml:"method"` +} + // Registration controls open registration. type Registration struct { Enabled bool `yaml:"enabled" env:"WARP_REGISTRATION_ENABLED"` @@ -317,6 +373,8 @@ func Default() *Config { // Loose enough that a client in a long poll with typing // and receipts on top never notices it. Sync: RateLimitBucket{Burst: 100, Interval: 20 * time.Millisecond}, + // Avatars: a client opening a room asks for dozens at once. + Media: RateLimitBucket{Burst: 100, Interval: 20 * time.Millisecond}, }, }, Sync: Sync{ @@ -329,6 +387,16 @@ func Default() *Config { AllowRuntimeToggle: true, KeyValidityPeriod: 24 * time.Hour, }, + Media: Media{ + BasePath: "./media_store", + MaxUploadBytes: 50 << 20, + MaxThumbnailPixels: 32_000_000, + ThumbnailSizes: DefaultThumbnailSizes(), + // The spec's own example, and long enough for a client that + // reserves an ID before a slow upload. + PendingUploadExpiry: 24 * time.Hour, + MaxPendingUploads: 10, + }, Registration: Registration{ Enabled: false, AllowRuntimeToggle: true, @@ -338,6 +406,19 @@ func Default() *Config { } } +// DefaultThumbnailSizes are the thumbnails made when the config names +// none: two square crops for avatars, three scaled sizes for images in the +// timeline. They are the sizes Element and most other clients ask for. +func DefaultThumbnailSizes() []ThumbnailSize { + return []ThumbnailSize{ + {Width: 32, Height: 32, Method: "crop"}, + {Width: 96, Height: 96, Method: "crop"}, + {Width: 320, Height: 240, Method: "scale"}, + {Width: 640, Height: 480, Method: "scale"}, + {Width: 800, Height: 600, Method: "scale"}, + } +} + // ClientBaseURL returns the URL advertised to clients, falling back to // https:// when not configured. func (c *Config) ClientBaseURL() string { @@ -430,6 +511,7 @@ func (c *Config) Validate() error { {"room_create", c.ClientAPI.RateLimit.RoomCreate}, {"join", c.ClientAPI.RateLimit.Join}, {"sync", c.ClientAPI.RateLimit.Sync}, + {"media", c.ClientAPI.RateLimit.Media}, } { if b.bucket.Burst < 1 { bad("client_api.rate_limit.%s.burst: must be at least 1", b.name) @@ -446,6 +528,30 @@ func (c *Config) Validate() error { bad("sync.long_poll_timeout: must be positive") } + if c.Media.BasePath == "" { + bad("media.base_path: must not be empty") + } + if c.Media.MaxUploadBytes < 1 { + bad("media.max_upload_bytes: must be positive") + } + if c.Media.MaxThumbnailPixels < 1 { + bad("media.max_thumbnail_pixels: must be positive") + } + for i, size := range c.Media.ThumbnailSizes { + if size.Width < 1 || size.Height < 1 || size.Width > 4096 || size.Height > 4096 { + bad("media.thumbnail_sizes[%d]: width and height must be between 1 and 4096", i) + } + if size.Method != "crop" && size.Method != "scale" { + bad("media.thumbnail_sizes[%d].method: must be crop or scale, got %q", i, size.Method) + } + } + if c.Media.PendingUploadExpiry <= 0 { + bad("media.pending_upload_expiry: must be positive") + } + if c.Media.MaxPendingUploads < 1 { + bad("media.max_pending_uploads: must be at least 1") + } + for i, token := range c.Registration.Tokens { if strings.TrimSpace(token) == "" { bad("registration.tokens[%d]: must not be empty", i) diff --git a/server/internal/config/config_test.go b/server/internal/config/config_test.go index caef13a..21efc84 100644 --- a/server/internal/config/config_test.go +++ b/server/internal/config/config_test.go @@ -3,6 +3,7 @@ package config import ( "os" "path/filepath" + "reflect" "strings" "testing" "time" @@ -171,4 +172,55 @@ func TestExampleConfigIsValid(t *testing.T) { if cfg.ClientAPI.RateLimit.Sync.Burst == 0 { t.Error("the shipped example must set the sync rate-limit bucket") } + // The example documents the defaults, so it has to be them. + if !reflect.DeepEqual(cfg.Media, Default().Media) { + t.Errorf("example media section = %+v, want the defaults %+v", cfg.Media, Default().Media) + } + if cfg.ClientAPI.RateLimit.Media != Default().ClientAPI.RateLimit.Media { + t.Error("the shipped example must set the media rate-limit bucket to its default") + } +} + +func TestMediaValidation(t *testing.T) { + for name, tweak := range map[string]func(*Media){ + "empty base path": func(m *Media) { m.BasePath = "" }, + "zero upload size": func(m *Media) { m.MaxUploadBytes = 0 }, + "zero pixels": func(m *Media) { m.MaxThumbnailPixels = 0 }, + "unknown method": func(m *Media) { m.ThumbnailSizes = []ThumbnailSize{{Width: 1, Height: 1, Method: "stretch"}} }, + "zero-sized thumbnail": func(m *Media) { m.ThumbnailSizes = []ThumbnailSize{{Width: 0, Height: 10, Method: "crop"}} }, + "huge thumbnail": func(m *Media) { m.ThumbnailSizes = []ThumbnailSize{{Width: 5000, Height: 10, Method: "scale"}} }, + "no pending expiry": func(m *Media) { m.PendingUploadExpiry = 0 }, + "no pending uploads": func(m *Media) { m.MaxPendingUploads = 0 }, + } { + t.Run(name, func(t *testing.T) { + cfg := Default() + cfg.Global.ServerName = "warp.example" + tweak(&cfg.Media) + if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "media.") { + t.Errorf("Validate = %v, want a media error", err) + } + }) + } + + cfg := Default() + cfg.Global.ServerName = "warp.example" + cfg.Media.ThumbnailSizes = nil + if err := cfg.Validate(); err != nil { + t.Errorf("no thumbnail sizes at all is valid (thumbnails are then refused): %v", err) + } +} + +func TestMediaFromTheEnvironment(t *testing.T) { + cfg := Default() + env := map[string]string{ + "WARP_MEDIA_BASE_PATH": "/srv/media", + "WARP_MEDIA_MAX_UPLOAD_BYTES": "1024", + "WARP_MEDIA_LEGACY_UNAUTHENTICATED_DOWNLOADS": "true", + } + if err := ApplyEnv(cfg, func(k string) string { return env[k] }); err != nil { + t.Fatalf("ApplyEnv: %v", err) + } + if cfg.Media.BasePath != "/srv/media" || cfg.Media.MaxUploadBytes != 1024 || !cfg.Media.LegacyUnauthenticatedDownloads { + t.Errorf("media = %+v", cfg.Media) + } } diff --git a/server/internal/federation/client.go b/server/internal/federation/client.go index 7a0331d..08e995c 100644 --- a/server/internal/federation/client.go +++ b/server/internal/federation/client.go @@ -95,6 +95,13 @@ type ClientOptions struct { // Policy filters destinations. Nil allows any. Policy *Policy + + // AllowPrivateMediaRedirects lets a media download follow another + // server's redirect to a loopback, private or link-local address. + // Only tests, whose servers are all on loopback, set it: anywhere + // else it would let another server point this one at its own + // internal network and read the answer back through a client. + AllowPrivateMediaRedirects bool } // Client sends signed Server-Server API requests. @@ -106,6 +113,11 @@ type Client struct { enabled func() bool policy *Policy userAgent string + + // mediaHTTP follows the redirects a media download may answer with. + // It is not the federation transport: the URL is an ordinary web + // address, often a CDN, and nothing is signed. + mediaHTTP *http.Client } // dialTargetKey carries the resolved addresses of a request down to the @@ -166,6 +178,7 @@ func NewClient(opts ClientOptions) *Client { enabled: opts.Enabled, policy: opts.Policy, userAgent: buildinfo.UserAgent(), + mediaHTTP: newMediaRedirectClient(tlsConfig, opts.AllowPrivateMediaRedirects), } if c.resolve == nil { // The well-known document is fetched with ordinary HTTPS and the @@ -210,12 +223,47 @@ type request struct { // do performs a request and decodes a JSON answer into out (which may be // nil). func (c *Client) do(ctx context.Context, req request, out any) error { - if err := c.check(req.destination); err != nil { + resp, cancel, err := c.send(ctx, req) + if err != nil { return err } + defer cancel() + defer func() { _ = resp.Body.Close() }() + + limit := req.maxBytes + if limit == 0 { + limit = defaultMaxResponse + } + body, err := io.ReadAll(io.LimitReader(resp.Body, limit+1)) + if err != nil { + return fmt.Errorf("%s %s%s: read response: %w", req.method, req.destination, req.path, err) + } + if int64(len(body)) > limit { + return fmt.Errorf("%s %s%s: response is larger than %d bytes", req.method, req.destination, req.path, limit) + } + + if resp.StatusCode < 200 || resp.StatusCode > 299 { + return httpError(resp.StatusCode, body) + } + if out == nil { + return nil + } + if err := json.Unmarshal(body, out); err != nil { + return fmt.Errorf("%s %s%s: decode response: %w", req.method, req.destination, req.path, err) + } + return nil +} + +// send performs a request and hands back the response with its body +// unread, for the callers that stream it. The caller closes the body and +// then calls cancel, which ends the request's deadline. +func (c *Client) send(ctx context.Context, req request) (*http.Response, context.CancelFunc, error) { + if err := c.check(req.destination); err != nil { + return nil, nil, err + } resolved, err := c.resolve(ctx, req.destination) if err != nil { - return fmt.Errorf("resolve %s: %w", req.destination, err) + return nil, nil, fmt.Errorf("resolve %s: %w", req.destination, err) } timeout := req.timeout @@ -223,13 +271,13 @@ func (c *Client) do(ctx context.Context, req request, out any) error { timeout = defaultRequestTimeout } ctx, cancel := context.WithTimeout(ctx, timeout) - defer cancel() ctx = context.WithValue(ctx, dialTargetKey{}, resolved.Addresses) var content []byte if req.body != nil { if content, err = json.Marshal(req.body); err != nil { - return fmt.Errorf("encode request: %w", err) + cancel() + return nil, nil, fmt.Errorf("encode request: %w", err) } } @@ -239,7 +287,8 @@ func (c *Client) do(ctx context.Context, req request, out any) error { } httpReq, err := http.NewRequestWithContext(ctx, req.method, target, bytes.NewReader(content)) if err != nil { - return fmt.Errorf("build request: %w", err) + cancel() + return nil, nil, fmt.Errorf("build request: %w", err) } if content == nil { httpReq.Body = http.NoBody @@ -256,47 +305,32 @@ func (c *Client) do(ctx context.Context, req request, out any) error { x, err := SignRequest(c.key, c.serverName, req.destination, req.method, httpReq.URL.RequestURI(), content) if err != nil { - return fmt.Errorf("sign request: %w", err) + cancel() + return nil, nil, fmt.Errorf("sign request: %w", err) } httpReq.Header.Set("Authorization", x.String()) } resp, err := c.http.Do(httpReq) if err != nil { - return fmt.Errorf("%s %s%s: %w", req.method, req.destination, req.path, err) + cancel() + return nil, nil, fmt.Errorf("%s %s%s: %w", req.method, req.destination, req.path, err) } - defer func() { _ = resp.Body.Close() }() + return resp, cancel, nil +} - limit := req.maxBytes - if limit == 0 { - limit = defaultMaxResponse +// httpError turns a non-2xx answer into an *HTTPError, with the Matrix +// error envelope if the body was one. +func httpError(status int, body []byte) *HTTPError { + httpErr := &HTTPError{Status: status} + var envelope struct { + Code string `json:"errcode"` + Message string `json:"error"` } - body, err := io.ReadAll(io.LimitReader(resp.Body, limit+1)) - if err != nil { - return fmt.Errorf("%s %s%s: read response: %w", req.method, req.destination, req.path, err) + if json.Unmarshal(body, &envelope) == nil { + httpErr.Code, httpErr.Message = envelope.Code, envelope.Message } - if int64(len(body)) > limit { - return fmt.Errorf("%s %s%s: response is larger than %d bytes", req.method, req.destination, req.path, limit) - } - - if resp.StatusCode < 200 || resp.StatusCode > 299 { - httpErr := &HTTPError{Status: resp.StatusCode} - var envelope struct { - Code string `json:"errcode"` - Message string `json:"error"` - } - if json.Unmarshal(body, &envelope) == nil { - httpErr.Code, httpErr.Message = envelope.Code, envelope.Message - } - return httpErr - } - if out == nil { - return nil - } - if err := json.Unmarshal(body, out); err != nil { - return fmt.Errorf("%s %s%s: decode response: %w", req.method, req.destination, req.path, err) - } - return nil + return httpErr } // esc escapes one path segment. Room, event and user IDs carry sigils and diff --git a/server/internal/federation/media.go b/server/internal/federation/media.go new file mode 100644 index 0000000..c01d7aa --- /dev/null +++ b/server/internal/federation/media.go @@ -0,0 +1,269 @@ +package federation + +import ( + "context" + "crypto/tls" + "errors" + "fmt" + "io" + "mime" + "mime/multipart" + "net" + "net/http" + "net/url" + "strconv" + "syscall" + "time" +) + +// MediaResponse is media fetched from another server. The caller reads +// Body and closes it. +type MediaResponse struct { + ContentType string + ContentDisposition string + Body io.ReadCloser +} + +// ThumbnailRequest is the size of thumbnail asked of another server. +type ThumbnailRequest struct { + Width int + Height int + Method string + Animated bool +} + +// mediaTimeout bounds a media download, body included: fifty megabytes +// over a slow link is minutes, not seconds. +const mediaTimeout = 10 * time.Minute + +// maxMediaMetadata caps the JSON part of a multipart media answer, which +// the spec defines as an empty object for now. +const maxMediaMetadata = 64 << 10 + +// errCodeUnrecognized is the answer of a server that predates an endpoint. +const errCodeUnrecognized = "M_UNRECOGNIZED" + +// DownloadMedia fetches media from the server it was uploaded to. +// +// It asks the authenticated endpoint of Matrix 1.11 first. A server older +// than that answers 404 M_UNRECOGNIZED, and is then asked through the +// deprecated Client-Server download endpoint with allow_remote=false, +// which is what every server before 1.11 served media to other servers +// through. +func (c *Client) DownloadMedia(ctx context.Context, origin, mediaID string, wait time.Duration) (MediaResponse, error) { + query := url.Values{} + if wait > 0 { + query.Set("timeout_ms", strconv.FormatInt(wait.Milliseconds(), 10)) + } + legacy := url.Values{"allow_remote": {"false"}} + if wait > 0 { + legacy.Set("timeout_ms", strconv.FormatInt(wait.Milliseconds(), 10)) + } + return c.fetchMedia(ctx, origin, + "/_matrix/federation/v1/media/download/"+esc(mediaID), query, + "/_matrix/media/v3/download/"+esc(origin)+"/"+esc(mediaID), legacy) +} + +// MediaThumbnail fetches a thumbnail from the server the media was +// uploaded to, with the same fallback as DownloadMedia. +func (c *Client) MediaThumbnail(ctx context.Context, origin, mediaID string, req ThumbnailRequest, wait time.Duration) (MediaResponse, error) { + query := url.Values{ + "width": {strconv.Itoa(req.Width)}, + "height": {strconv.Itoa(req.Height)}, + "method": {req.Method}, + } + if req.Animated { + query.Set("animated", "true") + } + if wait > 0 { + query.Set("timeout_ms", strconv.FormatInt(wait.Milliseconds(), 10)) + } + legacy := url.Values{} + for k, v := range query { + legacy[k] = v + } + legacy.Set("allow_remote", "false") + return c.fetchMedia(ctx, origin, + "/_matrix/federation/v1/media/thumbnail/"+esc(mediaID), query, + "/_matrix/media/v3/thumbnail/"+esc(origin)+"/"+esc(mediaID), legacy) +} + +// fetchMedia asks the federation endpoint, and the legacy one if the +// server does not know the federation one. +func (c *Client) fetchMedia(ctx context.Context, origin, path string, query url.Values, legacyPath string, legacyQuery url.Values) (MediaResponse, error) { + resp, cancel, err := c.send(ctx, request{ + method: http.MethodGet, destination: origin, signed: true, + path: path, query: query, timeout: mediaTimeout, + }) + if err != nil { + return MediaResponse{}, err + } + if resp.StatusCode == http.StatusOK { + media, err := c.readMultipart(ctx, resp, cancel) + if err != nil { + return MediaResponse{}, fmt.Errorf("GET %s%s: %w", origin, path, err) + } + return media, nil + } + + failure := readFailure(resp, cancel) + if failure.Status != http.StatusNotFound || failure.Code != errCodeUnrecognized { + return MediaResponse{}, failure + } + + resp, cancel, err = c.send(ctx, request{ + method: http.MethodGet, destination: origin, + path: legacyPath, query: legacyQuery, timeout: mediaTimeout, + }) + if err != nil { + return MediaResponse{}, err + } + if resp.StatusCode != http.StatusOK { + return MediaResponse{}, readFailure(resp, cancel) + } + return MediaResponse{ + ContentType: resp.Header.Get("Content-Type"), + ContentDisposition: resp.Header.Get("Content-Disposition"), + Body: &closer{Reader: resp.Body, close: closeAll(resp.Body, cancel)}, + }, nil +} + +// readFailure reads a non-2xx answer and releases the request. +func readFailure(resp *http.Response, cancel context.CancelFunc) *HTTPError { + body, _ := io.ReadAll(io.LimitReader(resp.Body, maxMediaMetadata)) + _ = resp.Body.Close() + cancel() + return httpError(resp.StatusCode, body) +} + +// readMultipart takes apart the multipart/mixed answer of the federation +// media endpoints: a JSON part, which is metadata and empty for now, then +// either the content itself or a part with a Location header saying where +// to fetch it from. +func (c *Client) readMultipart(ctx context.Context, resp *http.Response, cancel context.CancelFunc) (MediaResponse, error) { + release := closeAll(resp.Body, cancel) + mediaType, params, err := mime.ParseMediaType(resp.Header.Get("Content-Type")) + if err != nil || mediaType != "multipart/mixed" || params["boundary"] == "" { + _ = release() + return MediaResponse{}, fmt.Errorf("answer is %q, not multipart/mixed", resp.Header.Get("Content-Type")) + } + parts := multipart.NewReader(resp.Body, params["boundary"]) + + metadata, err := parts.NextPart() + if err != nil { + _ = release() + return MediaResponse{}, fmt.Errorf("read the metadata part: %w", err) + } + if _, err := io.Copy(io.Discard, io.LimitReader(metadata, maxMediaMetadata)); err != nil { + _ = release() + return MediaResponse{}, fmt.Errorf("read the metadata part: %w", err) + } + + content, err := parts.NextPart() + if err != nil { + _ = release() + return MediaResponse{}, fmt.Errorf("read the content part: %w", err) + } + if location := content.Header.Get("Location"); location != "" { + _ = release() + return c.followMediaRedirect(ctx, location) + } + return MediaResponse{ + ContentType: content.Header.Get("Content-Type"), + ContentDisposition: content.Header.Get("Content-Disposition"), + Body: &closer{Reader: content, close: release}, + }, nil +} + +// followMediaRedirect fetches content another server said is elsewhere. +func (c *Client) followMediaRedirect(ctx context.Context, location string) (MediaResponse, error) { + target, err := url.Parse(location) + if err != nil || (target.Scheme != "https" && target.Scheme != "http") || target.Host == "" { + return MediaResponse{}, fmt.Errorf("media redirect to %q is not a web address", location) + } + ctx, cancel := context.WithTimeout(ctx, mediaTimeout) + req, err := http.NewRequestWithContext(ctx, http.MethodGet, target.String(), http.NoBody) + if err != nil { + cancel() + return MediaResponse{}, fmt.Errorf("build the media redirect request: %w", err) + } + req.Header.Set("User-Agent", c.userAgent) + resp, err := c.mediaHTTP.Do(req) + if err != nil { + cancel() + return MediaResponse{}, fmt.Errorf("follow the media redirect: %w", err) + } + if resp.StatusCode != http.StatusOK { + failure := readFailure(resp, cancel) + return MediaResponse{}, fmt.Errorf("follow the media redirect: %w", failure) + } + return MediaResponse{ + ContentType: resp.Header.Get("Content-Type"), + ContentDisposition: resp.Header.Get("Content-Disposition"), + Body: &closer{Reader: resp.Body, close: closeAll(resp.Body, cancel)}, + }, nil +} + +// errPrivateAddress refuses a media redirect into a private network. +var errPrivateAddress = errors.New("refusing to fetch media from a private address") + +// newMediaRedirectClient builds the client media redirects are followed +// with. Its dialer checks the address it actually connects to — after DNS, +// on every redirect — so a name that resolves into a private network is +// refused as surely as an IP literal. +func newMediaRedirectClient(tlsConfig *tls.Config, allowPrivate bool) *http.Client { + dialer := &net.Dialer{Timeout: 30 * time.Second, KeepAlive: 30 * time.Second} + if !allowPrivate { + dialer.Control = func(_, address string, _ syscall.RawConn) error { + host, _, err := net.SplitHostPort(address) + if err != nil { + return err + } + ip := net.ParseIP(host) + if ip == nil || !isPublic(ip) { + return fmt.Errorf("%w: %s", errPrivateAddress, host) + } + return nil + } + } + return &http.Client{ + Transport: &http.Transport{ + Proxy: nil, + DialContext: dialer.DialContext, + TLSClientConfig: tlsConfig, + TLSHandshakeTimeout: 15 * time.Second, + MaxIdleConns: 20, + IdleConnTimeout: 90 * time.Second, + }, + CheckRedirect: func(_ *http.Request, via []*http.Request) error { + if len(via) >= 5 { + return errors.New("too many media redirects") + } + return nil + }, + } +} + +// isPublic reports an address on the public internet. +func isPublic(ip net.IP) bool { + return !ip.IsLoopback() && !ip.IsPrivate() && !ip.IsUnspecified() && + !ip.IsLinkLocalUnicast() && !ip.IsLinkLocalMulticast() && + !ip.IsInterfaceLocalMulticast() && !ip.IsMulticast() +} + +// closer is a reader whose Close releases more than the reader. +type closer struct { + io.Reader + close func() error +} + +func (c *closer) Close() error { return c.close() } + +// closeAll closes a response body and ends its request's deadline. +func closeAll(body io.Closer, cancel context.CancelFunc) func() error { + return func() error { + err := body.Close() + cancel() + return err + } +} diff --git a/server/internal/federation/media_test.go b/server/internal/federation/media_test.go new file mode 100644 index 0000000..a8e2ef5 --- /dev/null +++ b/server/internal/federation/media_test.go @@ -0,0 +1,204 @@ +package federation + +import ( + "context" + "errors" + "io" + "mime/multipart" + "net" + "net/http" + "net/http/httptest" + "net/textproto" + "strings" + "testing" + "time" + + "github.com/dariuszjaniak/warp/server/internal/matrix/signing" +) + +// mediaServer is another server's media endpoints, as a test wants them. +func mediaServer(t *testing.T, mux *http.ServeMux) *httptest.Server { + t.Helper() + srv := httptest.NewTLSServer(mux) + t.Cleanup(srv.Close) + return srv +} + +func mediaClient(t *testing.T, srv *httptest.Server, allowPrivate bool) *Client { + t.Helper() + key, err := signing.Generate("") + if err != nil { + t.Fatal(err) + } + return NewClient(ClientOptions{ + ServerName: "warp.example", + Key: key, + TLSConfig: srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone(), + Enabled: func() bool { return true }, + Resolve: func(context.Context, string) (Resolved, error) { + _, port, _ := net.SplitHostPort(srv.Listener.Addr().String()) + return Resolved{Addresses: []string{srv.Listener.Addr().String()}, Host: "example.com:" + port}, nil + }, + AllowPrivateMediaRedirects: allowPrivate, + }) +} + +// writeMultipart answers the way the federation media endpoints do. +func writeMultipart(w http.ResponseWriter, contentHeader textproto.MIMEHeader, content string) { + mw := multipart.NewWriter(w) + w.Header().Set("Content-Type", "multipart/mixed; boundary="+mw.Boundary()) + meta, _ := mw.CreatePart(textproto.MIMEHeader{"Content-Type": {"application/json"}}) + _, _ = meta.Write([]byte("{}")) + part, _ := mw.CreatePart(contentHeader) + _, _ = part.Write([]byte(content)) + _ = mw.Close() +} + +func readMedia(t *testing.T, m MediaResponse) string { + t.Helper() + defer func() { _ = m.Body.Close() }() + data, err := io.ReadAll(m.Body) + if err != nil { + t.Fatal(err) + } + return string(data) +} + +func TestDownloadMediaMultipart(t *testing.T) { + mux := http.NewServeMux() + mux.HandleFunc("GET /_matrix/federation/v1/media/download/{mediaID}", func(w http.ResponseWriter, r *http.Request) { + if !strings.HasPrefix(r.Header.Get("Authorization"), "X-Matrix ") { + http.Error(w, "unsigned", http.StatusUnauthorized) + return + } + if r.URL.Query().Get("timeout_ms") != "5000" { + http.Error(w, "no timeout", http.StatusBadRequest) + return + } + writeMultipart(w, textproto.MIMEHeader{ + "Content-Type": {"text/plain"}, + "Content-Disposition": {`inline; filename="a.txt"`}, + }, "content of "+r.PathValue("mediaID")) + }) + srv := mediaServer(t, mux) + c := mediaClient(t, srv, false) + + m, err := c.DownloadMedia(t.Context(), "remote.example", "abc", 5*time.Second) + if err != nil { + t.Fatalf("DownloadMedia: %v", err) + } + if m.ContentType != "text/plain" || m.ContentDisposition != `inline; filename="a.txt"` { + t.Errorf("headers = %q, %q", m.ContentType, m.ContentDisposition) + } + if got := readMedia(t, m); got != "content of abc" { + t.Errorf("content = %q", got) + } +} + +func TestDownloadMediaFallsBackToTheLegacyEndpoint(t *testing.T) { + mux := http.NewServeMux() + mux.HandleFunc("/_matrix/federation/", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusNotFound) + _, _ = w.Write([]byte(`{"errcode":"M_UNRECOGNIZED","error":"Unrecognized request"}`)) + }) + mux.HandleFunc("GET /_matrix/media/v3/download/{origin}/{mediaID}", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("allow_remote") != "false" { + http.Error(w, "would fetch from elsewhere", http.StatusBadRequest) + return + } + w.Header().Set("Content-Type", "image/png") + _, _ = w.Write([]byte("legacy " + r.PathValue("origin") + "/" + r.PathValue("mediaID"))) + }) + srv := mediaServer(t, mux) + c := mediaClient(t, srv, false) + + m, err := c.DownloadMedia(t.Context(), "old.example", "xyz", 0) + if err != nil { + t.Fatalf("DownloadMedia: %v", err) + } + if got := readMedia(t, m); got != "legacy old.example/xyz" || m.ContentType != "image/png" { + t.Errorf("legacy download = %q (%s)", got, m.ContentType) + } +} + +func TestDownloadMediaNotFoundIsNotRetried(t *testing.T) { + legacyAsked := false + mux := http.NewServeMux() + mux.HandleFunc("/_matrix/federation/", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusNotFound) + _, _ = w.Write([]byte(`{"errcode":"M_NOT_FOUND","error":"no such media"}`)) + }) + mux.HandleFunc("/_matrix/media/", func(w http.ResponseWriter, _ *http.Request) { legacyAsked = true }) + c := mediaClient(t, mediaServer(t, mux), false) + + _, err := c.DownloadMedia(t.Context(), "remote.example", "gone", 0) + if !IsNotFound(err) { + t.Fatalf("DownloadMedia = %v, want a 404", err) + } + if legacyAsked { + t.Error("a server that knows the endpoint and lacks the media was asked again") + } +} + +func TestMediaThumbnailRedirect(t *testing.T) { + var target string + mux := http.NewServeMux() + mux.HandleFunc("GET /_matrix/federation/v1/media/thumbnail/{mediaID}", func(w http.ResponseWriter, r *http.Request) { + q := r.URL.Query() + if q.Get("width") != "96" || q.Get("height") != "96" || q.Get("method") != "crop" { + http.Error(w, "bad size", http.StatusBadRequest) + return + } + writeMultipart(w, textproto.MIMEHeader{"Location": {target}}, "") + }) + mux.HandleFunc("GET /cdn/thumb", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "image/jpeg") + _, _ = w.Write([]byte("from the cdn")) + }) + srv := mediaServer(t, mux) + target = srv.URL + "/cdn/thumb" + req := ThumbnailRequest{Width: 96, Height: 96, Method: "crop"} + + m, err := mediaClient(t, srv, true).MediaThumbnail(t.Context(), "remote.example", "abc", req, 0) + if err != nil { + t.Fatalf("MediaThumbnail: %v", err) + } + if got := readMedia(t, m); got != "from the cdn" || m.ContentType != "image/jpeg" { + t.Errorf("redirected thumbnail = %q (%s)", got, m.ContentType) + } + + // The same redirect into a private address is refused when private + // redirects are not allowed — which is everywhere but tests. + _, err = mediaClient(t, srv, false).MediaThumbnail(t.Context(), "remote.example", "abc", req, 0) + if !errors.Is(err, errPrivateAddress) { + t.Errorf("redirect to loopback = %v, want errPrivateAddress", err) + } +} + +func TestMediaNeedsFederation(t *testing.T) { + c := NewClient(ClientOptions{ServerName: "warp.example"}) + if _, err := c.DownloadMedia(t.Context(), "remote.example", "abc", 0); !errors.Is(err, ErrDisabled) { + t.Errorf("DownloadMedia with federation off = %v, want ErrDisabled", err) + } +} + +func TestIsPublic(t *testing.T) { + for addr, want := range map[string]bool{ + "8.8.8.8": true, + "2001:4860::8888": true, + "127.0.0.1": false, + "10.1.2.3": false, + "192.168.0.1": false, + "169.254.169.254": false, + "::1": false, + "fe80::1": false, + "fd00::1": false, + "0.0.0.0": false, + } { + if got := isPublic(net.ParseIP(addr)); got != want { + t.Errorf("isPublic(%s) = %v, want %v", addr, got, want) + } + } +} diff --git a/server/internal/federationapi/media.go b/server/internal/federationapi/media.go new file mode 100644 index 0000000..5883a1d --- /dev/null +++ b/server/internal/federationapi/media.go @@ -0,0 +1,97 @@ +package federationapi + +import ( + "io" + "mime/multipart" + "net/http" + "net/textproto" + + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" +) + +// The media endpoints of Matrix 1.11: how another server downloads media +// uploaded here. They serve local media only — a server asking for a +// third server's media asks that server — and answer in multipart/mixed, +// a JSON part for metadata (an empty object, for now) and then the +// content. + +func (s *Server) registerMediaRoutes(authed func(string, authedHandler)) { + if s.media == nil { + return + } + const v1 = "/_matrix/federation/v1" + authed("GET "+v1+"/media/download/{mediaID}", s.handleMediaDownload) + authed("GET "+v1+"/media/thumbnail/{mediaID}", s.handleMediaThumbnail) +} + +func (s *Server) handleMediaDownload(w http.ResponseWriter, r *http.Request, _ string, _ []byte) error { + wait, err := mediaapi.WaitParam(r) + if err != nil { + return err + } + content, err := s.media.Download(r.Context(), s.serverName, r.PathValue("mediaID"), + mediaapi.FetchOptions{Wait: wait, LocalOnly: true}) + if err != nil { + return mediaapi.HTTPError(err) + } + defer func() { _ = content.Body.Close() }() + writeMultipart(w, r, content, mediaapi.ContentDisposition(content.ContentType, content.FileName)) + return nil +} + +func (s *Server) handleMediaThumbnail(w http.ResponseWriter, r *http.Request, _ string, _ []byte) error { + req, err := mediaapi.ThumbnailParams(r) + if err != nil { + return err + } + wait, err := mediaapi.WaitParam(r) + if err != nil { + return err + } + content, err := s.media.Thumbnail(r.Context(), s.serverName, r.PathValue("mediaID"), req, + mediaapi.FetchOptions{Wait: wait, LocalOnly: true}) + if err != nil { + return mediaapi.HTTPError(err) + } + defer func() { _ = content.Body.Close() }() + writeMultipart(w, r, content, + mediaapi.ContentDisposition(content.ContentType, mediaapi.ThumbnailFileName(content.ContentType))) + return nil +} + +// writeMultipart sends content the way the federation media endpoints do. +// +// The status goes out first, so a failure after it can only cut the +// answer short — which the other server sees as a broken multipart body — +// and is only logged. +func writeMultipart(w http.ResponseWriter, r *http.Request, content mediaapi.Content, disposition string) { + mw := multipart.NewWriter(w) + w.Header().Set("Content-Type", "multipart/mixed; boundary="+mw.Boundary()) + mediaapi.SetSecurityHeaders(w.Header()) + w.WriteHeader(http.StatusOK) + + err := func() error { + metadata, err := mw.CreatePart(textproto.MIMEHeader{"Content-Type": {"application/json"}}) + if err != nil { + return err + } + if _, err := metadata.Write([]byte("{}")); err != nil { + return err + } + part, err := mw.CreatePart(textproto.MIMEHeader{ + "Content-Type": {content.ContentType}, + "Content-Disposition": {disposition}, + }) + if err != nil { + return err + } + if _, err := io.Copy(part, content.Body); err != nil { + return err + } + return mw.Close() + }() + if err != nil { + httputil.LoggerFrom(r.Context()).Debug("sending media over federation failed", "error", err) + } +} diff --git a/server/internal/federationapi/server.go b/server/internal/federationapi/server.go index 34302a9..6e0a382 100644 --- a/server/internal/federationapi/server.go +++ b/server/internal/federationapi/server.go @@ -16,6 +16,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/federation" "github.com/dariuszjaniak/warp/server/internal/httputil" "github.com/dariuszjaniak/warp/server/internal/matrix/id" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/storage" "github.com/dariuszjaniak/warp/server/internal/syncapi" @@ -55,8 +56,13 @@ type ServerOptions struct { Ephemeral EphemeralSink Profiles Profiles Store storage.FederationStore - Logger *slog.Logger - Now func() time.Time + + // Media serves this server's media to others. Nil leaves the media + // endpoints unregistered. + Media *mediaapi.API + + Logger *slog.Logger + Now func() time.Time } // Server is the inbound Server-Server API. @@ -68,6 +74,7 @@ type Server struct { ephemeral EphemeralSink profiles Profiles store storage.FederationStore + media *mediaapi.API log *slog.Logger now func() time.Time } @@ -94,6 +101,7 @@ func NewServer(opts ServerOptions) (*Server, error) { ephemeral: opts.Ephemeral, profiles: opts.Profiles, store: opts.Store, + media: opts.Media, log: opts.Logger, now: opts.Now, }, nil @@ -142,6 +150,8 @@ func (s *Server) RegisterRoutes(r *httputil.Router) { authed("GET "+v1+"/query/profile", s.handleQueryProfile) authed("GET "+v1+"/publicRooms", s.handlePublicRoomsGet) authed("POST "+v1+"/publicRooms", s.handlePublicRoomsPost) + + s.registerMediaRoutes(authed) } // authenticated checks X-Matrix and the allow/deny lists before the diff --git a/server/internal/httputil/errors.go b/server/internal/httputil/errors.go index 95c8193..52d1364 100644 --- a/server/internal/httputil/errors.go +++ b/server/internal/httputil/errors.go @@ -35,6 +35,10 @@ const ( ErrCodeUserDeactivated = "M_USER_DEACTIVATED" ErrCodeExclusive = "M_EXCLUSIVE" ErrCodeWeakPassword = "M_WEAK_PASSWORD" + + // Media codes, added with the mediaapi in Phase 6. + ErrCodeNotYetUploaded = "M_NOT_YET_UPLOADED" + ErrCodeCannotOverwrite = "M_CANNOT_OVERWRITE_MEDIA" ) // MatrixError is the standard Matrix error envelope. Every non-2xx response diff --git a/server/internal/mediaapi/http.go b/server/internal/mediaapi/http.go new file mode 100644 index 0000000..a5d1e0c --- /dev/null +++ b/server/internal/mediaapi/http.go @@ -0,0 +1,204 @@ +package mediaapi + +import ( + "errors" + "mime" + "net/http" + "strconv" + "strings" + "time" + + "github.com/dariuszjaniak/warp/server/internal/httputil" +) + +// inlineTypes are the content types a browser may show in place. They are +// the list the spec gives (Matrix 1.12, "Content-Disposition"): images, +// audio and video a browser cannot be made to run as a page, plain text +// and a few data formats. Anything else is sent as an attachment, so that +// an uploaded HTML file cannot run as a page on this server's origin. +var inlineTypes = map[string]bool{ + "text/css": true, + "text/plain": true, + "text/csv": true, + "application/json": true, + "application/ld+json": true, + "image/jpeg": true, + "image/gif": true, + "image/png": true, + "image/apng": true, + "image/webp": true, + "image/avif": true, + "video/mp4": true, + "video/webm": true, + "video/ogg": true, + "video/quicktime": true, + "audio/mp4": true, + "audio/webm": true, + "audio/aac": true, + "audio/mpeg": true, + "audio/ogg": true, + "audio/wave": true, + "audio/wav": true, + "audio/x-wav": true, + "audio/x-pn-wav": true, + "audio/flac": true, + "audio/x-flac": true, +} + +// ContentDisposition is the Content-Disposition media is served with: +// inline for the types a browser can show safely, an attachment for the +// rest, with the file name encoded the RFC 6266 way when it is not ASCII. +func ContentDisposition(contentType, fileName string) string { + disposition := "attachment" + if mediaType, _, err := mime.ParseMediaType(contentType); err == nil && inlineTypes[mediaType] { + disposition = "inline" + } + if fileName == "" { + return disposition + } + if formatted := mime.FormatMediaType(disposition, map[string]string{"filename": fileName}); formatted != "" { + return formatted + } + // FormatMediaType refuses names it cannot encode; serving without + // one is better than failing the download. + return disposition +} + +// SetSecurityHeaders adds the headers every media response carries. +// +// Media is content anybody could have uploaded, served from the +// homeserver's own origin. The sandboxing CSP keeps it from running +// script there even if a browser is tricked into rendering it as a page; +// nosniff stops a browser from deciding a "text/plain" upload is HTML; +// and the cross-origin resource policy lets a web client on another +// origin embed the images it is shown. +func SetSecurityHeaders(h http.Header) { + h.Set("Content-Security-Policy", + "sandbox; default-src 'none'; script-src 'none'; plugin-types application/pdf; "+ + "style-src 'unsafe-inline'; media-src 'self'; object-src 'self';") + h.Set("X-Content-Type-Options", "nosniff") + h.Set("Cross-Origin-Resource-Policy", "cross-origin") +} + +// ServeContent sends media to a client: the security headers, a +// Content-Disposition naming fileName, an ETag from the content's hash and +// a cache lifetime of a year, since the content of an mxc:// URI never +// changes. Range requests and If-None-Match are handled by +// http.ServeContent. +// +// The cache is private: the endpoints need an access token, and a shared +// cache would hand the file to whoever asked next without one. +func ServeContent(w http.ResponseWriter, r *http.Request, content Content, fileName string) { + h := w.Header() + h.Set("Content-Type", content.ContentType) + h.Set("Content-Disposition", ContentDisposition(content.ContentType, fileName)) + h.Set("Cache-Control", "private, max-age=31536000, immutable") + h.Set("ETag", `"`+content.SHA256+`"`) + SetSecurityHeaders(h) + http.ServeContent(w, r, "", time.Time{}, content.Body) +} + +// ThumbnailFileName names a thumbnail for its Content-Disposition. +func ThumbnailFileName(contentType string) string { + switch { + case strings.HasPrefix(contentType, "image/jpeg"): + return "thumbnail.jpg" + case strings.HasPrefix(contentType, "image/png"): + return "thumbnail.png" + case strings.HasPrefix(contentType, "image/gif"): + return "thumbnail.gif" + case strings.HasPrefix(contentType, "image/webp"): + return "thumbnail.webp" + } + return "thumbnail" +} + +// ThumbnailParams reads the query of a thumbnail request, which the +// client and federation endpoints share. +func ThumbnailParams(r *http.Request) (ThumbnailRequest, error) { + q := r.URL.Query() + width, err := positiveInt(q.Get("width"), "width") + if err != nil { + return ThumbnailRequest{}, err + } + height, err := positiveInt(q.Get("height"), "height") + if err != nil { + return ThumbnailRequest{}, err + } + method := q.Get("method") + switch method { + case "": + method = MethodScale + case MethodCrop, MethodScale: + default: + return ThumbnailRequest{}, httputil.InvalidParam("method must be crop or scale") + } + animated := false + switch q.Get("animated") { + case "", "false": + case "true": + animated = true + default: + return ThumbnailRequest{}, httputil.InvalidParam("animated must be true or false") + } + return ThumbnailRequest{Width: width, Height: height, Method: method, Animated: animated}, nil +} + +// positiveInt reads a required positive integer parameter. +func positiveInt(raw, name string) (int, error) { + if raw == "" { + return 0, httputil.MissingParam("%s is required", name) + } + n, err := strconv.Atoi(raw) + if err != nil || n < 1 { + return 0, httputil.InvalidParam("%s must be a positive integer", name) + } + return n, nil +} + +// defaultMediaWait is how long a download waits for the content of a +// reserved media ID when the request does not say: the spec's default. +const defaultMediaWait = 20 * time.Second + +// WaitParam reads ?timeout_ms=, the time a download may wait for content +// that has been reserved and not yet uploaded. +func WaitParam(r *http.Request) (time.Duration, error) { + raw := r.URL.Query().Get("timeout_ms") + if raw == "" { + return defaultMediaWait, nil + } + ms, err := strconv.ParseInt(raw, 10, 64) + if err != nil || ms < 0 { + return 0, httputil.InvalidParam("timeout_ms must be a non-negative integer") + } + return min(time.Duration(ms)*time.Millisecond, MaxWait), nil +} + +// HTTPError maps an error of this package onto the Matrix error the +// client and federation endpoints answer with; the errors mean the same +// to a client as to a server. Anything else is passed through. +func HTTPError(err error) error { + switch { + case err == nil: + return nil + case errors.Is(err, ErrNotFound): + return httputil.NotFound("Media not found") + case errors.Is(err, ErrTooLarge): + return httputil.NewError(http.StatusRequestEntityTooLarge, httputil.ErrCodeTooLarge, "Media is larger than this server allows") + case errors.Is(err, ErrNotYetUploaded): + return httputil.NewError(http.StatusGatewayTimeout, httputil.ErrCodeNotYetUploaded, "Media has not been uploaded yet") + case errors.Is(err, ErrCannotOverwrite): + return httputil.NewError(http.StatusConflict, httputil.ErrCodeCannotOverwrite, "Media has already been uploaded") + case errors.Is(err, ErrNotYours): + return httputil.Forbidden("This media ID was reserved by another user") + case errors.Is(err, ErrTooManyPending): + return httputil.NewError(http.StatusTooManyRequests, httputil.ErrCodeLimitExceeded, "Too many uploads are pending") + case errors.Is(err, ErrCannotThumbnail): + return httputil.Unknown(http.StatusBadRequest, "Cannot generate a thumbnail of this media") + case errors.Is(err, ErrFederationDisabled): + return httputil.Forbidden("Federation is disabled on this server; fetching media from another server is not possible") + case errors.Is(err, ErrRemote): + return httputil.NewError(http.StatusBadGateway, httputil.ErrCodeUnknown, "The server the media is on did not provide it") + } + return err +} diff --git a/server/internal/mediaapi/mediaapi.go b/server/internal/mediaapi/mediaapi.go new file mode 100644 index 0000000..f3571cd --- /dev/null +++ b/server/internal/mediaapi/mediaapi.go @@ -0,0 +1,502 @@ +// Package mediaapi is the media repository: uploads, downloads, +// thumbnails, and the cache of other servers' media. +// +// Content lives in a storage.BlobStore, named by its SHA-256; this package +// keeps the metadata that maps an mxc:// URI onto a blob, in the +// storage.MediaStore. The HTTP surface is elsewhere — the Client-Server +// endpoints in clientapi, the federation endpoints in federationapi — and +// both come here for everything but the response format. +// +// Media from other servers is fetched on first use, through the +// federation client, and kept: an mxc:// URI names immutable content, so a +// copy never goes stale. With federation off the client refuses to fetch, +// and only what was cached before is served. +package mediaapi + +import ( + "context" + "crypto/rand" + "errors" + "fmt" + "io" + "log/slog" + "mime" + "strings" + "sync" + "time" + + "golang.org/x/sync/singleflight" + + "github.com/dariuszjaniak/warp/server/internal/config" + "github.com/dariuszjaniak/warp/server/internal/federation" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// Errors the API returns. The HTTP layers map them onto Matrix errors. +var ( + // ErrNotFound means there is no such media, or it cannot be had. + ErrNotFound = errors.New("media not found") + + // ErrTooLarge means content is larger than media.max_upload_bytes. + ErrTooLarge = errors.New("media is too large") + + // ErrNotYetUploaded means the media ID was reserved and its content + // has not arrived within the time the request was willing to wait. + ErrNotYetUploaded = errors.New("media has not been uploaded yet") + + // ErrCannotOverwrite means the content of a reserved ID was already + // uploaded. + ErrCannotOverwrite = errors.New("media content has already been uploaded") + + // ErrNotYours means somebody other than the user who reserved an ID + // tried to upload its content. + ErrNotYours = errors.New("media ID was reserved by another user") + + // ErrTooManyPending means the user holds as many reserved IDs as they + // may. + ErrTooManyPending = errors.New("too many pending uploads") + + // ErrCannotThumbnail means the media is not an image this server can + // make a thumbnail of. + ErrCannotThumbnail = errors.New("cannot make a thumbnail of this media") + + // ErrFederationDisabled means the media is on another server and + // federation is off, or that server is not allowed. + ErrFederationDisabled = errors.New("media is on another server and federation with it is off") + + // ErrRemote means the server the media is on failed to provide it. + ErrRemote = errors.New("the server the media is on did not provide it") +) + +// Remote fetches media from other servers. *federation.Client satisfies +// it. +type Remote interface { + DownloadMedia(ctx context.Context, origin, mediaID string, wait time.Duration) (federation.MediaResponse, error) + MediaThumbnail(ctx context.Context, origin, mediaID string, req federation.ThumbnailRequest, wait time.Duration) (federation.MediaResponse, error) +} + +// Options configure the API. +type Options struct { + // ServerName is this server's name: the origin of every upload. + ServerName string + + Store storage.MediaStore + Blobs storage.BlobStore + + // Remote fetches other servers' media. Nil means only local and + // already cached media can be served. + Remote Remote + + Config config.Media + Logger *slog.Logger + + // Now is the clock, so tests can expire reservations without waiting. + Now func() time.Time +} + +// API is the media repository. +type API struct { + serverName string + store storage.MediaStore + blobs storage.BlobStore + cfg config.Media + log *slog.Logger + now func() time.Time + + remoteMu sync.RWMutex + remote Remote + + // waiters wakes downloads blocked on a reserved ID. + waiters *waiters + + // fetches makes concurrent requests for the same remote media, or + // the same thumbnail, share one fetch or one resize. + fetches singleflight.Group + + // resizing bounds how many thumbnails are made at once. Decoding an + // image is the most memory a single request can cost this server. + resizing chan struct{} +} + +// New builds the API. +func New(opts Options) (*API, error) { + switch { + case opts.ServerName == "": + return nil, errors.New("mediaapi: ServerName must not be empty") + case opts.Store == nil, opts.Blobs == nil: + return nil, errors.New("mediaapi: Store and Blobs are required") + case opts.Config.MaxUploadBytes < 1: + return nil, errors.New("mediaapi: Config.MaxUploadBytes must be positive") + } + if opts.Logger == nil { + opts.Logger = slog.Default() + } + if opts.Now == nil { + opts.Now = time.Now + } + if opts.Config.PendingUploadExpiry <= 0 { + opts.Config.PendingUploadExpiry = 24 * time.Hour + } + if opts.Config.MaxPendingUploads < 1 { + opts.Config.MaxPendingUploads = 1 + } + if opts.Config.MaxThumbnailPixels < 1 { + opts.Config.MaxThumbnailPixels = config.Default().Media.MaxThumbnailPixels + } + return &API{ + serverName: opts.ServerName, + store: opts.Store, + blobs: opts.Blobs, + cfg: opts.Config, + log: opts.Logger, + now: opts.Now, + waiters: newWaiters(), + resizing: make(chan struct{}, 4), + remote: opts.Remote, + }, nil +} + +// SetRemote hands the API the client it fetches other servers' media +// with. The server builds the media repository before federation, which +// needs a signing key it may not have; until this is called only local +// and already cached media is served. +func (a *API) SetRemote(remote Remote) { + a.remoteMu.Lock() + defer a.remoteMu.Unlock() + a.remote = remote +} + +// remoteClient reads the remote, or nil. +func (a *API) remoteClient() Remote { + a.remoteMu.RLock() + defer a.remoteMu.RUnlock() + return a.remote +} + +// MaxUploadBytes is the largest upload, for /media/config. +func (a *API) MaxUploadBytes() int64 { return a.cfg.MaxUploadBytes } + +// ServerName is the origin of this server's media. +func (a *API) ServerName() string { return a.serverName } + +func (a *API) nowMS() int64 { return a.now().UnixMilli() } + +// Upload is content arriving from a client. +type Upload struct { + UserID string + + // ContentType is the request's Content-Type; empty becomes + // application/octet-stream. + ContentType string + + // FileName is the ?filename= of the request, or "". + FileName string + + Body io.Reader +} + +// mediaIDLength is long enough that an ID cannot be guessed: 24 +// characters of 62 is 142 bits. +const mediaIDLength = 24 + +const mediaIDAlphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789" + +// newMediaID makes a random media ID. +func newMediaID() string { + // Bytes of 248 and over are thrown away rather than folded in with a + // modulo, which would make the first eight letters more likely than + // the rest. 248 is the largest multiple of 62 a byte holds. + const accept = 256 - 256%len(mediaIDAlphabet) + out := make([]byte, 0, mediaIDLength) + buf := make([]byte, mediaIDLength) + for len(out) < mediaIDLength { + _, _ = rand.Read(buf) + for _, c := range buf { + if int(c) < accept && len(out) < mediaIDLength { + out = append(out, mediaIDAlphabet[int(c)%len(mediaIDAlphabet)]) + } + } + } + return string(out) +} + +// MXC builds an mxc:// URI. +func MXC(origin, mediaID string) string { return "mxc://" + origin + "/" + mediaID } + +// Upload stores new content and returns its mxc:// URI. +func (a *API) Upload(ctx context.Context, up Upload) (string, error) { + content, err := a.storeContent(ctx, up) + if err != nil { + return "", err + } + m := storage.Media{ + Origin: a.serverName, + ContentType: content.ContentType, + UploadName: content.UploadName, + Size: content.Size, + SHA256: content.SHA256, + UserID: up.UserID, + CreatedAtMS: a.nowMS(), + } + // A collision of 142 random bits does not happen; the loop is for the + // day the generator is replaced by something that can. + for range 3 { + m.MediaID = newMediaID() + err = a.store.CreateMedia(ctx, m) + if !errors.Is(err, storage.ErrMediaExists) { + break + } + } + if err != nil { + return "", err + } + a.log.Debug("media uploaded", "media_id", m.MediaID, "user_id", up.UserID, + "content_type", m.ContentType, "bytes", m.Size) + return MXC(m.Origin, m.MediaID), nil +} + +// Create reserves a media ID whose content is uploaded later, returning +// its mxc:// URI and when the reservation lapses. +func (a *API) Create(ctx context.Context, userID string) (string, int64, error) { + now := a.nowMS() + // The sweep rides on reservations rather than a timer: expired rows + // only matter to the count below, and nothing else reads them. + if n, err := a.store.DeleteExpiredUploads(ctx, now); err != nil { + a.log.Warn("could not delete expired media reservations", "error", err) + } else if n > 0 { + a.log.Debug("deleted expired media reservations", "count", n) + } + + pending, err := a.store.PendingUploads(ctx, userID, now) + if err != nil { + return "", 0, err + } + if pending >= a.cfg.MaxPendingUploads { + return "", 0, ErrTooManyPending + } + m := storage.Media{ + Origin: a.serverName, + MediaID: newMediaID(), + UserID: userID, + CreatedAtMS: now, + Pending: true, + ExpiresAtMS: now + a.cfg.PendingUploadExpiry.Milliseconds(), + } + if err := a.store.CreateMedia(ctx, m); err != nil { + return "", 0, err + } + return MXC(m.Origin, m.MediaID), m.ExpiresAtMS, nil +} + +// UploadTo stores the content of a reserved ID. +func (a *API) UploadTo(ctx context.Context, origin, mediaID string, up Upload) error { + if origin != a.serverName { + return ErrNotFound + } + m, err := a.store.Media(ctx, origin, mediaID) + switch { + case errors.Is(err, storage.ErrNoSuchMedia): + return ErrNotFound + case err != nil: + return err + case m.UserID != up.UserID: + return ErrNotYours + case !m.Pending: + return ErrCannotOverwrite + case m.ExpiresAtMS <= a.nowMS(): + return ErrNotFound + } + + content, err := a.storeContent(ctx, up) + if err != nil { + return err + } + err = a.store.CompleteUpload(ctx, origin, mediaID, content) + switch { + case errors.Is(err, storage.ErrMediaUploaded): + return ErrCannotOverwrite + case errors.Is(err, storage.ErrNoSuchMedia): + return ErrNotFound + case err != nil: + return err + } + a.waiters.wake(origin, mediaID) + return nil +} + +// storeContent writes an upload's body to the blob store. +func (a *API) storeContent(ctx context.Context, up Upload) (storage.MediaContent, error) { + blob, err := a.blobs.Put(ctx, up.Body, a.cfg.MaxUploadBytes) + switch { + case errors.Is(err, storage.ErrBlobTooLarge): + return storage.MediaContent{}, ErrTooLarge + case err != nil: + return storage.MediaContent{}, err + } + return storage.MediaContent{ + ContentType: normaliseContentType(up.ContentType), + UploadName: up.FileName, + Size: blob.Size, + SHA256: blob.SHA256, + }, nil +} + +// normaliseContentType keeps a Content-Type that parses and replaces one +// that does not. What is stored is what downloads are served with, and a +// header that does not parse is one every client would guess at. +func normaliseContentType(raw string) string { + if strings.TrimSpace(raw) == "" { + return "application/octet-stream" + } + mediaType, params, err := mime.ParseMediaType(raw) + if err != nil { + return "application/octet-stream" + } + return mime.FormatMediaType(mediaType, params) +} + +// Content is media ready to be sent: the body and what to say about it. +type Content struct { + ContentType string + FileName string + Size int64 + + // SHA256 is the content's hash, which makes a perfect ETag. + SHA256 string + + Body io.ReadSeekCloser +} + +// FetchOptions are the parts of a download request that are not the +// media's name. +type FetchOptions struct { + // Wait is how long to wait for a reserved ID's content. Zero does not + // wait. + Wait time.Duration + + // LocalOnly serves only media uploaded here: what another server is + // asking for, which must not make this one fetch from a third. + LocalOnly bool +} + +// MaxWait caps what a request may ask to wait for content. +const MaxWait = 2 * time.Minute + +// Download returns media's content. +func (a *API) Download(ctx context.Context, origin, mediaID string, opts FetchOptions) (Content, error) { + if !validServerName(origin) || !ValidMediaID(mediaID) { + return Content{}, ErrNotFound + } + if origin == a.serverName { + m, err := a.localMedia(ctx, mediaID, opts.Wait) + if err != nil { + return Content{}, err + } + return a.open(ctx, m.ContentType, m.UploadName, m.Size, m.SHA256) + } + if opts.LocalOnly { + return Content{}, ErrNotFound + } + m, err := a.remoteMedia(ctx, origin, mediaID, opts.Wait) + if err != nil { + return Content{}, err + } + return a.open(ctx, m.ContentType, m.UploadName, m.Size, m.SHA256) +} + +// localMedia reads media uploaded here, waiting for a reserved ID's +// content if asked to. +func (a *API) localMedia(ctx context.Context, mediaID string, wait time.Duration) (storage.Media, error) { + wait = min(max(wait, 0), MaxWait) + deadline := a.now().Add(wait) + for { + // Subscribed before the read, so that an upload completing + // between the read and the wait is not missed. + woken, cancel := a.waiters.subscribe(a.serverName, mediaID) + m, err := a.store.Media(ctx, a.serverName, mediaID) + switch { + case errors.Is(err, storage.ErrNoSuchMedia): + cancel() + return storage.Media{}, ErrNotFound + case err != nil: + cancel() + return storage.Media{}, err + case !m.Pending: + cancel() + return m, nil + case m.ExpiresAtMS <= a.nowMS(): + cancel() + return storage.Media{}, ErrNotFound + } + + remaining := deadline.Sub(a.now()) + if remaining <= 0 { + cancel() + return storage.Media{}, ErrNotYetUploaded + } + timer := time.NewTimer(remaining) + select { + case <-woken: + case <-timer.C: + case <-ctx.Done(): + } + timer.Stop() + cancel() + if ctx.Err() != nil { + return storage.Media{}, ctx.Err() + } + } +} + +// open opens a blob as Content. +func (a *API) open(ctx context.Context, contentType, fileName string, size int64, sha string) (Content, error) { + body, err := a.blobs.Open(ctx, sha) + if errors.Is(err, storage.ErrNoSuchBlob) { + // The row is there and the file is not: somebody cleaned the + // media directory by hand. Say so in the log; the client is told + // the media is gone, which it is. + a.log.Error("media content is missing from the blob store", "sha256", sha) + return Content{}, ErrNotFound + } + if err != nil { + return Content{}, err + } + return Content{ContentType: contentType, FileName: fileName, Size: size, SHA256: sha, Body: body}, nil +} + +// ValidMediaID reports a media ID of the grammar the spec gives: +// letters, digits, underscore and hyphen. Anything else is refused before +// it reaches a database or another server. +func ValidMediaID(id string) bool { + if id == "" || len(id) > 255 { + return false + } + for _, c := range id { + switch { + case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == '_', c == '-': + default: + return false + } + } + return true +} + +// validServerName is a cheap sanity check on the server name of an +// mxc:// URI: a host, an optional port, an IPv6 literal in brackets. +func validServerName(name string) bool { + if name == "" || len(name) > 255 { + return false + } + for _, c := range name { + switch { + case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9': + case c == '.', c == '-', c == ':', c == '[', c == ']': + default: + return false + } + } + return true +} + +// String names media for logs. +func mediaName(origin, mediaID string) string { return fmt.Sprintf("%s/%s", origin, mediaID) } diff --git a/server/internal/mediaapi/mediaapi_test.go b/server/internal/mediaapi/mediaapi_test.go new file mode 100644 index 0000000..d6ecaa0 --- /dev/null +++ b/server/internal/mediaapi/mediaapi_test.go @@ -0,0 +1,740 @@ +package mediaapi + +import ( + "bytes" + "context" + "errors" + "image" + "image/color" + "image/gif" + "image/jpeg" + "image/png" + "io" + "log/slog" + "net/http" + "path/filepath" + "strings" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/dariuszjaniak/warp/server/internal/config" + "github.com/dariuszjaniak/warp/server/internal/federation" + "github.com/dariuszjaniak/warp/server/internal/storage" + _ "github.com/dariuszjaniak/warp/server/internal/storage/drivers" + "github.com/dariuszjaniak/warp/server/internal/storage/filestore" +) + +const ( + serverName = "warp.example" + alice = "@alice:warp.example" + bob = "@bob:warp.example" +) + +// fakeRemote stands in for the federation client. +type fakeRemote struct { + mu sync.Mutex + media map[string]federation.MediaResponse + content map[string][]byte + thumbs map[string][]byte + err error + downloads atomic.Int32 + thumbAsks atomic.Int32 + // gate, when set, holds every download until it is closed. + gate chan struct{} +} + +func (f *fakeRemote) DownloadMedia(_ context.Context, origin, mediaID string, _ time.Duration) (federation.MediaResponse, error) { + f.downloads.Add(1) + if f.gate != nil { + <-f.gate + } + if f.err != nil { + return federation.MediaResponse{}, f.err + } + f.mu.Lock() + defer f.mu.Unlock() + resp, ok := f.media[origin+"/"+mediaID] + if !ok { + return federation.MediaResponse{}, &federation.HTTPError{Status: http.StatusNotFound, Code: "M_NOT_FOUND"} + } + resp.Body = io.NopCloser(bytes.NewReader(f.content[origin+"/"+mediaID])) + return resp, nil +} + +func (f *fakeRemote) MediaThumbnail(_ context.Context, origin, mediaID string, req federation.ThumbnailRequest, _ time.Duration) (federation.MediaResponse, error) { + f.thumbAsks.Add(1) + f.mu.Lock() + defer f.mu.Unlock() + data, ok := f.thumbs[origin+"/"+mediaID] + if !ok { + return federation.MediaResponse{}, &federation.HTTPError{Status: http.StatusNotFound, Code: "M_NOT_FOUND"} + } + return federation.MediaResponse{ContentType: "image/png", Body: io.NopCloser(bytes.NewReader(data))}, nil +} + +type testAPI struct { + *API + store storage.MediaStore + remote *fakeRemote + now *atomic.Int64 +} + +func newTestAPI(t *testing.T, tweak func(*config.Media)) *testAPI { + t.Helper() + dir := t.TempDir() + db, err := storage.Open(t.Context(), storage.Config{ + Driver: storage.DriverSQLite, + ConnectionString: "file:" + filepath.ToSlash(filepath.Join(dir, "warp.db")) + "?_pragma=busy_timeout(5000)", + MaxOpenConns: 4, + }, slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = db.Close() }) + if err := db.Migrate(t.Context()); err != nil { + t.Fatal(err) + } + blobs, err := filestore.New(filepath.Join(dir, "media")) + if err != nil { + t.Fatal(err) + } + + cfg := config.Default().Media + if tweak != nil { + tweak(&cfg) + } + now := &atomic.Int64{} + now.Store(time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC).UnixMilli()) + remote := &fakeRemote{media: map[string]federation.MediaResponse{}, content: map[string][]byte{}, thumbs: map[string][]byte{}} + api, err := New(Options{ + ServerName: serverName, + Store: db.Media(), + Blobs: blobs, + Remote: remote, + Config: cfg, + Logger: slog.New(slog.DiscardHandler), + Now: func() time.Time { return time.UnixMilli(now.Load()) }, + }) + if err != nil { + t.Fatal(err) + } + return &testAPI{API: api, store: db.Media(), remote: remote, now: now} +} + +func readAll(t *testing.T, c Content) []byte { + t.Helper() + defer func() { _ = c.Body.Close() }() + data, err := io.ReadAll(c.Body) + if err != nil { + t.Fatal(err) + } + return data +} + +func mediaIDOf(t *testing.T, mxc string) string { + t.Helper() + rest, ok := strings.CutPrefix(mxc, "mxc://"+serverName+"/") + if !ok || !ValidMediaID(rest) { + t.Fatalf("bad mxc URI %q", mxc) + } + return rest +} + +func TestUploadAndDownload(t *testing.T) { + api := newTestAPI(t, nil) + ctx := t.Context() + + mxc, err := api.Upload(ctx, Upload{ + UserID: alice, ContentType: "text/plain; charset=utf-8", FileName: "hello.txt", + Body: strings.NewReader("hello, world"), + }) + if err != nil { + t.Fatalf("Upload: %v", err) + } + id := mediaIDOf(t, mxc) + + c, err := api.Download(ctx, serverName, id, FetchOptions{}) + if err != nil { + t.Fatalf("Download: %v", err) + } + if got := readAll(t, c); string(got) != "hello, world" { + t.Errorf("content = %q", got) + } + if c.ContentType != "text/plain; charset=utf-8" || c.FileName != "hello.txt" || c.Size != 12 || len(c.SHA256) != 64 { + t.Errorf("content metadata = %+v", c) + } + + for _, bad := range []struct{ origin, id string }{ + {serverName, "nope"}, + {serverName, "../etc"}, + {"bad/server", id}, + } { + if _, err := api.Download(ctx, bad.origin, bad.id, FetchOptions{}); !errors.Is(err, ErrNotFound) { + t.Errorf("Download(%s, %s) = %v, want ErrNotFound", bad.origin, bad.id, err) + } + } +} + +func TestUploadNormalisesContentType(t *testing.T) { + for raw, want := range map[string]string{ + "": "application/octet-stream", + "not a type;;": "application/octet-stream", + "Image/PNG": "image/png", + "text/plain; charset=x": "text/plain; charset=x", + } { + if got := normaliseContentType(raw); got != want { + t.Errorf("normaliseContentType(%q) = %q, want %q", raw, got, want) + } + } +} + +func TestUploadTooLarge(t *testing.T) { + api := newTestAPI(t, func(m *config.Media) { m.MaxUploadBytes = 10 }) + _, err := api.Upload(t.Context(), Upload{UserID: alice, Body: strings.NewReader("eleven bytes")}) + if !errors.Is(err, ErrTooLarge) { + t.Fatalf("Upload = %v, want ErrTooLarge", err) + } +} + +func TestMediaIDs(t *testing.T) { + seen := map[string]bool{} + for range 1000 { + id := newMediaID() + if len(id) != mediaIDLength || !ValidMediaID(id) { + t.Fatalf("bad media ID %q", id) + } + if seen[id] { + t.Fatalf("media ID %q generated twice", id) + } + seen[id] = true + } +} + +func TestCreateAndUploadTo(t *testing.T) { + api := newTestAPI(t, func(m *config.Media) { m.MaxPendingUploads = 2 }) + ctx := t.Context() + + mxc, expires, err := api.Create(ctx, alice) + if err != nil { + t.Fatalf("Create: %v", err) + } + if want := api.now.Load() + (24 * time.Hour).Milliseconds(); expires != want { + t.Errorf("expires = %d, want %d", expires, want) + } + id := mediaIDOf(t, mxc) + + // Not there yet: a download that does not wait is told so. + if _, err := api.Download(ctx, serverName, id, FetchOptions{}); !errors.Is(err, ErrNotYetUploaded) { + t.Errorf("Download of pending media = %v, want ErrNotYetUploaded", err) + } + + up := func(user, body string) error { + return api.UploadTo(ctx, serverName, id, Upload{UserID: user, ContentType: "text/plain", Body: strings.NewReader(body)}) + } + if err := up(bob, "bob's"); !errors.Is(err, ErrNotYours) { + t.Errorf("UploadTo by another user = %v, want ErrNotYours", err) + } + if err := up(alice, "alice's"); err != nil { + t.Fatalf("UploadTo: %v", err) + } + if err := up(alice, "again"); !errors.Is(err, ErrCannotOverwrite) { + t.Errorf("second UploadTo = %v, want ErrCannotOverwrite", err) + } + c, err := api.Download(ctx, serverName, id, FetchOptions{}) + if err != nil { + t.Fatalf("Download: %v", err) + } + if got := readAll(t, c); string(got) != "alice's" { + t.Errorf("content = %q", got) + } + + // The limit counts reservations without content. + if _, _, err := api.Create(ctx, alice); err != nil { + t.Fatal(err) + } + if _, _, err := api.Create(ctx, alice); err != nil { + t.Fatal(err) + } + if _, _, err := api.Create(ctx, alice); !errors.Is(err, ErrTooManyPending) { + t.Errorf("third pending Create = %v, want ErrTooManyPending", err) + } + if _, _, err := api.Create(ctx, bob); err != nil { + t.Errorf("another user's Create: %v", err) + } + + // A day later the reservations have lapsed: they no longer count, + // and their IDs no longer take content. + lapsed, _, _ := api.Create(ctx, bob) + api.now.Add((25 * time.Hour).Milliseconds()) + err = api.UploadTo(ctx, serverName, mediaIDOf(t, lapsed), Upload{UserID: bob, Body: strings.NewReader("late")}) + if !errors.Is(err, ErrNotFound) { + t.Errorf("UploadTo a lapsed ID = %v, want ErrNotFound", err) + } + if _, _, err := api.Create(ctx, alice); err != nil { + t.Errorf("Create after the reservations lapsed: %v", err) + } +} + +func TestDownloadWaitsForPendingContent(t *testing.T) { + api := newTestAPI(t, nil) + ctx := t.Context() + // The wait uses the real clock for its timer; only the deadline is + // computed from the API's clock, which this test leaves running. + api.API.now = time.Now + + mxc, _, err := api.Create(ctx, alice) + if err != nil { + t.Fatal(err) + } + id := mediaIDOf(t, mxc) + + done := make(chan error, 1) + go func() { + c, err := api.Download(ctx, serverName, id, FetchOptions{Wait: 10 * time.Second}) + if err == nil { + _ = c.Body.Close() + } + done <- err + }() + time.Sleep(50 * time.Millisecond) + if err := api.UploadTo(ctx, serverName, id, Upload{UserID: alice, Body: strings.NewReader("now")}); err != nil { + t.Fatal(err) + } + select { + case err := <-done: + if err != nil { + t.Fatalf("waiting Download = %v", err) + } + case <-time.After(5 * time.Second): + t.Fatal("the upload did not wake the waiting download") + } + + // A wait that runs out is ErrNotYetUploaded. + other, _, _ := api.Create(ctx, alice) + start := time.Now() + _, err = api.Download(ctx, serverName, mediaIDOf(t, other), FetchOptions{Wait: 100 * time.Millisecond}) + if !errors.Is(err, ErrNotYetUploaded) { + t.Errorf("Download after the wait = %v, want ErrNotYetUploaded", err) + } + if time.Since(start) < 100*time.Millisecond { + t.Error("Download did not wait") + } +} + +func TestRemoteMediaIsFetchedOnceAndCached(t *testing.T) { + api := newTestAPI(t, nil) + ctx := t.Context() + api.remote.media["remote.example/abc"] = federation.MediaResponse{ + ContentType: "image/png", ContentDisposition: `inline; filename="dir/cat.png"`, + } + api.remote.content["remote.example/abc"] = []byte("png bytes") + api.remote.gate = make(chan struct{}) + + const callers = 5 + var wg sync.WaitGroup + errs := make(chan error, callers) + for range callers { + wg.Go(func() { + c, err := api.Download(ctx, "remote.example", "abc", FetchOptions{}) + if err != nil { + errs <- err + return + } + if got := readAll(t, c); string(got) != "png bytes" { + errs <- errors.New("wrong content: " + string(got)) + } + if c.FileName != "cat.png" { + errs <- errors.New("file name kept its directory: " + c.FileName) + } + }) + } + time.Sleep(50 * time.Millisecond) + close(api.remote.gate) + wg.Wait() + close(errs) + for err := range errs { + t.Error(err) + } + if n := api.remote.downloads.Load(); n != 1 { + t.Errorf("%d downloads for %d concurrent requests, want 1", n, callers) + } + + // Cached: served without asking again, even with federation off. + api.remote.err = federation.ErrDisabled + c, err := api.Download(ctx, "remote.example", "abc", FetchOptions{}) + if err != nil { + t.Fatalf("cached Download: %v", err) + } + _ = c.Body.Close() + if n := api.remote.downloads.Load(); n != 1 { + t.Errorf("cached media was fetched again (%d downloads)", n) + } + + // Another server asking for it is not served a third server's media. + if _, err := api.Download(ctx, "remote.example", "abc", FetchOptions{LocalOnly: true}); !errors.Is(err, ErrNotFound) { + t.Errorf("LocalOnly Download of remote media = %v, want ErrNotFound", err) + } +} + +func TestRemoteErrors(t *testing.T) { + for name, tc := range map[string]struct { + err error + want error + }{ + "federation off": {federation.ErrDisabled, ErrFederationDisabled}, + "not allowed": {federation.ErrNotAllowed, ErrFederationDisabled}, + "not found": {&federation.HTTPError{Status: 404, Code: "M_NOT_FOUND"}, ErrNotFound}, + "not uploaded": {&federation.HTTPError{Status: 504, Code: "M_NOT_YET_UPLOADED"}, ErrNotYetUploaded}, + "too large": {&federation.HTTPError{Status: 413, Code: "M_TOO_LARGE"}, ErrTooLarge}, + "server error": {&federation.HTTPError{Status: 500}, ErrRemote}, + "network failure": {errors.New("connection refused"), ErrRemote}, + } { + t.Run(name, func(t *testing.T) { + api := newTestAPI(t, nil) + api.remote.err = tc.err + _, err := api.Download(t.Context(), "remote.example", "abc", FetchOptions{}) + if !errors.Is(err, tc.want) { + t.Errorf("Download = %v, want %v", err, tc.want) + } + }) + } + + api := newTestAPI(t, nil) + api.remote.content["remote.example/big"] = bytes.Repeat([]byte("x"), 100) + api.remote.media["remote.example/big"] = federation.MediaResponse{ContentType: "text/plain"} + api.cfg.MaxUploadBytes = 10 + if _, err := api.Download(t.Context(), "remote.example", "big", FetchOptions{}); !errors.Is(err, ErrTooLarge) { + t.Errorf("Download of oversized remote media = %v, want ErrTooLarge", err) + } +} + +// --- thumbnails --------------------------------------------------------------- + +func testImage(w, h int) *image.RGBA { + img := image.NewRGBA(image.Rect(0, 0, w, h)) + for y := range h { + for x := range w { + img.Set(x, y, color.RGBA{uint8(x), uint8(y), 128, 255}) + } + } + return img +} + +func encodePNG(t *testing.T, img image.Image) []byte { + t.Helper() + var buf bytes.Buffer + if err := png.Encode(&buf, img); err != nil { + t.Fatal(err) + } + return buf.Bytes() +} + +func uploadBytes(t *testing.T, api *testAPI, contentType string, data []byte) string { + t.Helper() + mxc, err := api.Upload(t.Context(), Upload{UserID: alice, ContentType: contentType, Body: bytes.NewReader(data)}) + if err != nil { + t.Fatal(err) + } + return mediaIDOf(t, mxc) +} + +func decodeThumbnail(t *testing.T, c Content) image.Image { + t.Helper() + img, _, err := image.Decode(bytes.NewReader(readAll(t, c))) + if err != nil { + t.Fatalf("thumbnail does not decode: %v", err) + } + return img +} + +func TestThumbnails(t *testing.T) { + api := newTestAPI(t, nil) + ctx := t.Context() + id := uploadBytes(t, api, "image/png", encodePNG(t, testImage(1000, 500))) + + for _, tc := range []struct { + req ThumbnailRequest + wantW, wantH int + }{ + // The smallest crop size that covers 50 × 50 is 96 × 96. + {ThumbnailRequest{Width: 50, Height: 50, Method: MethodCrop}, 96, 96}, + {ThumbnailRequest{Width: 20, Height: 20, Method: MethodCrop}, 32, 32}, + // 320 × 240 scaled from 2:1 keeps the ratio: 320 × 160. + {ThumbnailRequest{Width: 300, Height: 200, Method: MethodScale}, 320, 160}, + // Larger than every size: the largest, 800 × 600 → 800 × 400. + {ThumbnailRequest{Width: 2000, Height: 2000, Method: MethodScale}, 800, 400}, + } { + c, err := api.Thumbnail(ctx, serverName, id, tc.req, FetchOptions{}) + if err != nil { + t.Fatalf("Thumbnail(%+v): %v", tc.req, err) + } + if c.ContentType != "image/png" { + t.Errorf("Thumbnail(%+v) is %s, want image/png", tc.req, c.ContentType) + } + b := decodeThumbnail(t, c).Bounds() + if b.Dx() != tc.wantW || b.Dy() != tc.wantH { + t.Errorf("Thumbnail(%+v) is %d × %d, want %d × %d", tc.req, b.Dx(), b.Dy(), tc.wantW, tc.wantH) + } + } + + // The second ask for a size is answered from the stored thumbnail. + first, _ := api.store.Thumbnail(ctx, serverName, id, 96, 96, MethodCrop) + c, err := api.Thumbnail(ctx, serverName, id, ThumbnailRequest{Width: 96, Height: 96, Method: MethodCrop}, FetchOptions{}) + if err != nil { + t.Fatal(err) + } + _ = c.Body.Close() + if c.SHA256 != first.SHA256 { + t.Error("the stored thumbnail was not reused") + } +} + +func TestThumbnailOfSmallImageIsTheOriginal(t *testing.T) { + api := newTestAPI(t, nil) + original := encodePNG(t, testImage(100, 50)) + id := uploadBytes(t, api, "image/png", original) + + c, err := api.Thumbnail(t.Context(), serverName, id, ThumbnailRequest{Width: 640, Height: 480, Method: MethodScale}, FetchOptions{}) + if err != nil { + t.Fatal(err) + } + if got := readAll(t, c); !bytes.Equal(got, original) { + t.Error("a thumbnail larger than the image was not the image itself") + } + + // A crop of a small image is never enlarged: 100 × 50 cropped to a + // square is 50 × 50. + c, err = api.Thumbnail(t.Context(), serverName, id, ThumbnailRequest{Width: 96, Height: 96, Method: MethodCrop}, FetchOptions{}) + if err != nil { + t.Fatal(err) + } + if b := decodeThumbnail(t, c).Bounds(); b.Dx() != 50 || b.Dy() != 50 { + t.Errorf("crop of a small image is %d × %d, want 50 × 50", b.Dx(), b.Dy()) + } +} + +func TestThumbnailOfGIFIsStill(t *testing.T) { + api := newTestAPI(t, nil) + pal := image.NewPaletted(image.Rect(0, 0, 10, 10), []color.Color{color.Black, color.White}) + var buf bytes.Buffer + if err := gif.EncodeAll(&buf, &gif.GIF{Image: []*image.Paletted{pal, pal}, Delay: []int{10, 10}}); err != nil { + t.Fatal(err) + } + id := uploadBytes(t, api, "image/gif", buf.Bytes()) + c, err := api.Thumbnail(t.Context(), serverName, id, ThumbnailRequest{Width: 32, Height: 32, Method: MethodScale, Animated: true}, FetchOptions{}) + if err != nil { + t.Fatal(err) + } + _ = c.Body.Close() + if c.ContentType != "image/png" { + t.Errorf("thumbnail of a GIF is %s, want a still image/png", c.ContentType) + } +} + +func TestThumbnailRefusals(t *testing.T) { + api := newTestAPI(t, func(m *config.Media) { m.MaxThumbnailPixels = 100 * 100 }) + ctx := t.Context() + req := ThumbnailRequest{Width: 32, Height: 32, Method: MethodCrop} + + text := uploadBytes(t, api, "text/plain", []byte("not an image")) + if _, err := api.Thumbnail(ctx, serverName, text, req, FetchOptions{}); !errors.Is(err, ErrCannotThumbnail) { + t.Errorf("thumbnail of text = %v, want ErrCannotThumbnail", err) + } + liar := uploadBytes(t, api, "image/png", []byte("not a png")) + if _, err := api.Thumbnail(ctx, serverName, liar, req, FetchOptions{}); !errors.Is(err, ErrCannotThumbnail) { + t.Errorf("thumbnail of a broken PNG = %v, want ErrCannotThumbnail", err) + } + huge := uploadBytes(t, api, "image/png", encodePNG(t, testImage(200, 200))) + if _, err := api.Thumbnail(ctx, serverName, huge, req, FetchOptions{}); !errors.Is(err, ErrCannotThumbnail) { + t.Errorf("thumbnail past the pixel limit = %v, want ErrCannotThumbnail", err) + } + + none := newTestAPI(t, func(m *config.Media) { m.ThumbnailSizes = nil }) + id := uploadBytes(t, none, "image/png", encodePNG(t, testImage(10, 10))) + if _, err := none.Thumbnail(ctx, serverName, id, req, FetchOptions{}); !errors.Is(err, ErrCannotThumbnail) { + t.Errorf("thumbnail with no sizes configured = %v, want ErrCannotThumbnail", err) + } +} + +func TestRemoteThumbnail(t *testing.T) { + api := newTestAPI(t, nil) + ctx := t.Context() + thumb := encodePNG(t, testImage(96, 96)) + api.remote.thumbs["remote.example/pic"] = thumb + + req := ThumbnailRequest{Width: 90, Height: 90, Method: MethodCrop} + for range 2 { + c, err := api.Thumbnail(ctx, "remote.example", "pic", req, FetchOptions{}) + if err != nil { + t.Fatalf("Thumbnail: %v", err) + } + if got := readAll(t, c); !bytes.Equal(got, thumb) { + t.Error("remote thumbnail content differs") + } + } + if n := api.remote.thumbAsks.Load(); n != 1 { + t.Errorf("remote thumbnail fetched %d times, want once", n) + } + if n := api.remote.downloads.Load(); n != 0 { + t.Errorf("the original was downloaded %d times to make a thumbnail", n) + } +} + +func TestPickSize(t *testing.T) { + api := newTestAPI(t, nil) + for _, tc := range []struct { + req ThumbnailRequest + want thumbnailSize + }{ + {ThumbnailRequest{Width: 1, Height: 1, Method: MethodCrop}, thumbnailSize{Width: 32, Height: 32, Method: MethodCrop}}, + {ThumbnailRequest{Width: 33, Height: 10, Method: MethodCrop}, thumbnailSize{Width: 96, Height: 96, Method: MethodCrop}}, + {ThumbnailRequest{Width: 500, Height: 500, Method: MethodCrop}, thumbnailSize{Width: 96, Height: 96, Method: MethodCrop}}, + {ThumbnailRequest{Width: 640, Height: 100, Method: MethodScale}, thumbnailSize{Width: 640, Height: 480, Method: MethodScale}}, + {ThumbnailRequest{Width: 641, Height: 100, Method: MethodScale}, thumbnailSize{Width: 800, Height: 600, Method: MethodScale}}, + } { + got, ok := api.pickSize(tc.req) + if !ok || got != tc.want { + t.Errorf("pickSize(%+v) = %+v, want %+v", tc.req, got, tc.want) + } + } + + scaleOnly := newTestAPI(t, func(m *config.Media) { + m.ThumbnailSizes = []config.ThumbnailSize{{Width: 100, Height: 100, Method: MethodScale}} + }) + if got, ok := scaleOnly.pickSize(ThumbnailRequest{Width: 32, Height: 32, Method: MethodCrop}); !ok || got.Method != MethodScale { + t.Errorf("crop with only scale sizes = %+v, %v; want the scale size", got, ok) + } +} + +func TestFit(t *testing.T) { + for _, tc := range []struct { + srcW, srcH, boxW, boxH int + method string + outW, outH int + crop image.Rectangle + }{ + {1000, 500, 96, 96, MethodCrop, 96, 96, image.Rect(250, 0, 750, 500)}, + {500, 1000, 96, 96, MethodCrop, 96, 96, image.Rect(0, 250, 500, 750)}, + {1000, 500, 320, 240, MethodScale, 320, 160, image.Rect(0, 0, 1000, 500)}, + {50, 20, 320, 240, MethodScale, 50, 20, image.Rect(0, 0, 50, 20)}, + {40, 80, 96, 96, MethodCrop, 40, 40, image.Rect(0, 20, 40, 60)}, + } { + outW, outH, crop := fit(tc.srcW, tc.srcH, tc.boxW, tc.boxH, tc.method) + if outW != tc.outW || outH != tc.outH || crop != tc.crop { + t.Errorf("fit(%d×%d into %d×%d %s) = %d×%d from %v, want %d×%d from %v", + tc.srcW, tc.srcH, tc.boxW, tc.boxH, tc.method, outW, outH, crop, tc.outW, tc.outH, tc.crop) + } + } +} + +// jpegWithOrientation makes a JPEG with an EXIF orientation tag, the way a +// phone camera writes a portrait photo: landscape pixels, and a tag saying +// which way is up. +func jpegWithOrientation(t *testing.T, img image.Image, orientation int, bigEndian bool) []byte { + t.Helper() + var plain bytes.Buffer + if err := jpeg.Encode(&plain, img, nil); err != nil { + t.Fatal(err) + } + + var tiff []byte + if bigEndian { + tiff = []byte{'M', 'M', 0, 42, 0, 0, 0, 8, 0, 1, 0x01, 0x12, 0, 3, 0, 0, 0, 1, 0, byte(orientation), 0, 0, 0, 0, 0, 0} + } else { + tiff = []byte{'I', 'I', 42, 0, 8, 0, 0, 0, 1, 0, 0x12, 0x01, 3, 0, 1, 0, 0, 0, byte(orientation), 0, 0, 0, 0, 0, 0, 0} + } + payload := append([]byte("Exif\x00\x00"), tiff...) + length := len(payload) + 2 + app1 := append([]byte{0xFF, 0xE1, byte(length >> 8), byte(length)}, payload...) + + data := plain.Bytes() + out := append([]byte{}, data[:2]...) + out = append(out, app1...) + return append(out, data[2:]...) +} + +func TestJPEGOrientation(t *testing.T) { + img := testImage(40, 20) + for _, bigEndian := range []bool{false, true} { + for o := 1; o <= 8; o++ { + data := jpegWithOrientation(t, img, o, bigEndian) + if got := jpegOrientation(bytes.NewReader(data)); got != o { + t.Errorf("orientation %d (big endian %v) read as %d", o, bigEndian, got) + } + } + } + if got := jpegOrientation(strings.NewReader("not a jpeg")); got != 1 { + t.Errorf("orientation of garbage = %d, want 1", got) + } +} + +func TestThumbnailIsTurnedUpright(t *testing.T) { + api := newTestAPI(t, nil) + // Stored 1000 × 500, tagged "a quarter turn clockwise": upright it is + // 500 × 1000, portrait. + id := uploadBytes(t, api, "image/jpeg", jpegWithOrientation(t, testImage(1000, 500), 6, false)) + c, err := api.Thumbnail(t.Context(), serverName, id, ThumbnailRequest{Width: 320, Height: 240, Method: MethodScale}, FetchOptions{}) + if err != nil { + t.Fatal(err) + } + if c.ContentType != "image/jpeg" { + t.Errorf("thumbnail of a JPEG is %s", c.ContentType) + } + if b := decodeThumbnail(t, c).Bounds(); b.Dx() != 120 || b.Dy() != 240 { + t.Errorf("upright thumbnail is %d × %d, want 120 × 240", b.Dx(), b.Dy()) + } +} + +func TestOrient(t *testing.T) { + // A 2 × 1 image, red then blue; orientation 6 turns it a quarter + // clockwise, so red ends up on top. + img := image.NewRGBA(image.Rect(0, 0, 2, 1)) + red, blue := color.RGBA{255, 0, 0, 255}, color.RGBA{0, 0, 255, 255} + img.SetRGBA(0, 0, red) + img.SetRGBA(1, 0, blue) + + out := orient(img, 6).(*image.RGBA) + if out.Bounds().Dx() != 1 || out.Bounds().Dy() != 2 { + t.Fatalf("turned image is %v", out.Bounds()) + } + if out.RGBAAt(0, 0) != red || out.RGBAAt(0, 1) != blue { + t.Errorf("turned pixels are %v, %v", out.RGBAAt(0, 0), out.RGBAAt(0, 1)) + } + out = orient(img, 8).(*image.RGBA) + if out.RGBAAt(0, 0) != blue || out.RGBAAt(0, 1) != red { + t.Errorf("anticlockwise pixels are %v, %v", out.RGBAAt(0, 0), out.RGBAAt(0, 1)) + } +} + +func TestContentDisposition(t *testing.T) { + for _, tc := range []struct { + contentType, name, want string + }{ + {"image/png", "", "inline"}, + {"image/png; foo=bar", "cat.png", "inline; filename=cat.png"}, + {"text/html", "page.html", "attachment; filename=page.html"}, + {"image/svg+xml", "", "attachment"}, + {"application/octet-stream", "a b.bin", `attachment; filename="a b.bin"`}, + {"text/plain", "zażółć.txt", "inline; filename*=utf-8''za%C5%BC%C3%B3%C5%82%C4%87.txt"}, + } { + if got := ContentDisposition(tc.contentType, tc.name); got != tc.want { + t.Errorf("ContentDisposition(%q, %q) = %q, want %q", tc.contentType, tc.name, got, tc.want) + } + } +} + +func TestFileNameFrom(t *testing.T) { + for in, want := range map[string]string{ + "": "", + `attachment; filename="a.txt"`: "a.txt", + `inline; filename="../../etc/passwd"`: "passwd", + `inline; filename*=utf-8''%C5%BC.png`: "ż.png", + "this is not a disposition header at all": "", + } { + if got := fileNameFrom(in); got != want { + t.Errorf("fileNameFrom(%q) = %q, want %q", in, got, want) + } + } +} diff --git a/server/internal/mediaapi/remote.go b/server/internal/mediaapi/remote.go new file mode 100644 index 0000000..4dfc60f --- /dev/null +++ b/server/internal/mediaapi/remote.go @@ -0,0 +1,181 @@ +package mediaapi + +import ( + "context" + "errors" + "fmt" + "mime" + "net/http" + "strings" + "time" + + "github.com/dariuszjaniak/warp/server/internal/federation" + "github.com/dariuszjaniak/warp/server/internal/httputil" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// fetchTimeout bounds one fetch from another server, waiting included. +// It is detached from the request that started it: other requests may be +// waiting on the same fetch, and what it fetches is kept either way. +const fetchTimeout = 10 * time.Minute + +// remoteMedia returns another server's media from the cache, fetching it +// first if it is not there. +func (a *API) remoteMedia(ctx context.Context, origin, mediaID string, wait time.Duration) (storage.Media, error) { + m, err := a.store.Media(ctx, origin, mediaID) + if err == nil { + return m, nil + } + if !errors.Is(err, storage.ErrNoSuchMedia) { + return storage.Media{}, err + } + remote := a.remoteClient() + if remote == nil { + return storage.Media{}, ErrNotFound + } + + wait = min(max(wait, 0), MaxWait) + v, err := a.shared(ctx, "download\x00"+mediaName(origin, mediaID), func(ctx context.Context) (any, error) { + return a.fetchRemote(ctx, remote, origin, mediaID, wait) + }) + if err != nil { + return storage.Media{}, err + } + return v.(storage.Media), nil +} + +// shared runs fn once for every concurrent caller with the same key, on a +// context of its own, and lets each caller stop waiting when its own +// request ends. +func (a *API) shared(ctx context.Context, key string, fn func(context.Context) (any, error)) (any, error) { + result := a.fetches.DoChan(key, func() (any, error) { + fctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), fetchTimeout) + defer cancel() + return fn(fctx) + }) + select { + case r := <-result: + return r.Val, r.Err + case <-ctx.Done(): + return nil, ctx.Err() + } +} + +// fetchRemote downloads media from its server and caches it. +func (a *API) fetchRemote(ctx context.Context, remote Remote, origin, mediaID string, wait time.Duration) (storage.Media, error) { + resp, err := remote.DownloadMedia(ctx, origin, mediaID, wait) + if err != nil { + return storage.Media{}, a.remoteError(origin, mediaID, err) + } + defer func() { _ = resp.Body.Close() }() + + blob, err := a.blobs.Put(ctx, resp.Body, a.cfg.MaxUploadBytes) + switch { + case errors.Is(err, storage.ErrBlobTooLarge): + return storage.Media{}, ErrTooLarge + case err != nil: + a.log.Warn("could not fetch remote media", "media", mediaName(origin, mediaID), "error", err) + return storage.Media{}, fmt.Errorf("%w: %v", ErrRemote, err) + } + + m := storage.Media{ + Origin: origin, + MediaID: mediaID, + ContentType: normaliseContentType(resp.ContentType), + UploadName: fileNameFrom(resp.ContentDisposition), + Size: blob.Size, + SHA256: blob.SHA256, + CreatedAtMS: a.nowMS(), + } + err = a.store.CreateMedia(ctx, m) + if errors.Is(err, storage.ErrMediaExists) { + // Fetched twice — by another process, or before a restart. The + // copy already recorded is the same content. + return a.store.Media(ctx, origin, mediaID) + } + if err != nil { + return storage.Media{}, err + } + a.log.Debug("fetched remote media", "media", mediaName(origin, mediaID), + "content_type", m.ContentType, "bytes", m.Size) + return m, nil +} + +// fetchRemoteThumbnail asks the media's server for a thumbnail of one of +// the configured sizes, and caches it under that size. +// +// The size asked for is this server's, not the client's: the cache is +// keyed by it, and a client asking for 33 × 33 and another for 34 × 34 +// share one thumbnail rather than causing two fetches. +func (a *API) fetchRemoteThumbnail(ctx context.Context, remote Remote, origin, mediaID string, size thumbnailSize, wait time.Duration) (storage.Thumbnail, error) { + resp, err := remote.MediaThumbnail(ctx, origin, mediaID, federation.ThumbnailRequest{ + Width: size.Width, Height: size.Height, Method: size.Method, + }, wait) + if err != nil { + var httpErr *federation.HTTPError + if errors.As(err, &httpErr) && httpErr.Status == http.StatusBadRequest { + return storage.Thumbnail{}, ErrCannotThumbnail + } + return storage.Thumbnail{}, a.remoteError(origin, mediaID, err) + } + defer func() { _ = resp.Body.Close() }() + + contentType := normaliseContentType(resp.ContentType) + if !strings.HasPrefix(contentType, "image/") { + // A thumbnail that is not an image is not one, whatever the + // other server meant by it. + return storage.Thumbnail{}, fmt.Errorf("%w: thumbnail of type %q", ErrRemote, contentType) + } + blob, err := a.blobs.Put(ctx, resp.Body, a.cfg.MaxUploadBytes) + switch { + case errors.Is(err, storage.ErrBlobTooLarge): + return storage.Thumbnail{}, ErrTooLarge + case err != nil: + return storage.Thumbnail{}, fmt.Errorf("%w: %v", ErrRemote, err) + } + t := storage.Thumbnail{ + Origin: origin, MediaID: mediaID, + Width: size.Width, Height: size.Height, Method: size.Method, + ContentType: contentType, Size: blob.Size, SHA256: blob.SHA256, + CreatedAtMS: a.nowMS(), + } + if err := a.store.StoreThumbnail(ctx, t); err != nil { + return storage.Thumbnail{}, err + } + return t, nil +} + +// remoteError maps a failed fetch onto the API's errors. +func (a *API) remoteError(origin, mediaID string, err error) error { + var httpErr *federation.HTTPError + switch { + case errors.Is(err, federation.ErrDisabled), errors.Is(err, federation.ErrNotAllowed): + return ErrFederationDisabled + case errors.As(err, &httpErr) && httpErr.Status == http.StatusNotFound: + return ErrNotFound + case errors.As(err, &httpErr) && httpErr.Code == httputil.ErrCodeNotYetUploaded: + return ErrNotYetUploaded + case errors.As(err, &httpErr) && (httpErr.Status == http.StatusRequestEntityTooLarge || httpErr.Code == httputil.ErrCodeTooLarge): + return ErrTooLarge + } + a.log.Warn("could not fetch remote media", "media", mediaName(origin, mediaID), "error", err) + return fmt.Errorf("%w: %v", ErrRemote, err) +} + +// fileNameFrom reads the file name out of a Content-Disposition header. +func fileNameFrom(disposition string) string { + if disposition == "" { + return "" + } + _, params, err := mime.ParseMediaType(disposition) + if err != nil { + return "" + } + name := params["filename"] + // A file name is a name, not a path: keep only the last element of + // one a server sent with directories in it. + if i := strings.LastIndexAny(name, `/\`); i >= 0 { + name = name[i+1:] + } + return name +} diff --git a/server/internal/mediaapi/thumbnail.go b/server/internal/mediaapi/thumbnail.go new file mode 100644 index 0000000..e1a4311 --- /dev/null +++ b/server/internal/mediaapi/thumbnail.go @@ -0,0 +1,432 @@ +package mediaapi + +import ( + "bytes" + "context" + "errors" + "fmt" + "image" + _ "image/gif" // register the GIF decoder with image.Decode + "image/jpeg" + "image/png" + "io" + "math" + "mime" + "time" + + "golang.org/x/image/draw" + _ "golang.org/x/image/webp" // register the WebP decoder with image.Decode + + "github.com/dariuszjaniak/warp/server/internal/config" + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// The two thumbnail methods. +const ( + MethodCrop = "crop" + MethodScale = "scale" +) + +// ThumbnailRequest is the thumbnail a client asked for. +type ThumbnailRequest struct { + Width int + Height int + + // Method is MethodCrop or MethodScale. + Method string + + // Animated asks for an animated thumbnail of animated media. Warp + // makes still thumbnails only, which the spec allows: a server that + // cannot animate one returns a still image. + Animated bool +} + +type thumbnailSize = config.ThumbnailSize + +// Thumbnail returns a thumbnail of media. +func (a *API) Thumbnail(ctx context.Context, origin, mediaID string, req ThumbnailRequest, opts FetchOptions) (Content, error) { + if !validServerName(origin) || !ValidMediaID(mediaID) { + return Content{}, ErrNotFound + } + size, ok := a.pickSize(req) + if !ok { + return Content{}, ErrCannotThumbnail + } + + if origin == a.serverName { + m, err := a.localMedia(ctx, mediaID, opts.Wait) + if err != nil { + return Content{}, err + } + return a.thumbnailOf(ctx, m, size) + } + if opts.LocalOnly { + return Content{}, ErrNotFound + } + + if t, err := a.store.Thumbnail(ctx, origin, mediaID, size.Width, size.Height, size.Method); err == nil { + return a.open(ctx, t.ContentType, "", t.Size, t.SHA256) + } else if !errors.Is(err, storage.ErrNoSuchMedia) { + return Content{}, err + } + // The original is already here: make the thumbnail rather than + // fetch one. + if m, err := a.store.Media(ctx, origin, mediaID); err == nil && !m.Pending { + return a.thumbnailOf(ctx, m, size) + } else if err != nil && !errors.Is(err, storage.ErrNoSuchMedia) { + return Content{}, err + } + remote := a.remoteClient() + if remote == nil { + return Content{}, ErrNotFound + } + + wait := min(max(opts.Wait, 0), MaxWait) + key := fmt.Sprintf("thumbnail\x00%s\x00%dx%d\x00%s", mediaName(origin, mediaID), size.Width, size.Height, size.Method) + v, err := a.shared(ctx, key, func(ctx context.Context) (any, error) { + return a.fetchRemoteThumbnail(ctx, remote, origin, mediaID, size, wait) + }) + if err != nil { + return Content{}, err + } + t := v.(storage.Thumbnail) + return a.open(ctx, t.ContentType, "", t.Size, t.SHA256) +} + +// pickSize chooses the configured size that answers a request: the +// smallest of the requested method that is at least as large in both +// dimensions, or failing that the largest. A method with no configured +// sizes is answered from the other. +func (a *API) pickSize(req ThumbnailRequest) (thumbnailSize, bool) { + var candidates []thumbnailSize + for _, s := range a.cfg.ThumbnailSizes { + if s.Method == req.Method { + candidates = append(candidates, s) + } + } + if len(candidates) == 0 { + candidates = a.cfg.ThumbnailSizes + } + if len(candidates) == 0 { + return thumbnailSize{}, false + } + + area := func(s thumbnailSize) int { return s.Width * s.Height } + var best, largest thumbnailSize + found := false + for _, s := range candidates { + if area(s) > area(largest) { + largest = s + } + if s.Width >= req.Width && s.Height >= req.Height && (!found || area(s) < area(best)) { + best, found = s, true + } + } + if !found { + return largest, true + } + return best, true +} + +// thumbnailable lists the content types that are decoded for thumbnails. +var thumbnailable = map[string]bool{ + "image/jpeg": true, + "image/png": true, + "image/gif": true, + "image/webp": true, +} + +// thumbnailOf returns the thumbnail of media held here, making it on first +// use. +func (a *API) thumbnailOf(ctx context.Context, m storage.Media, size thumbnailSize) (Content, error) { + t, err := a.store.Thumbnail(ctx, m.Origin, m.MediaID, size.Width, size.Height, size.Method) + if err == nil { + return a.open(ctx, t.ContentType, "", t.Size, t.SHA256) + } + if !errors.Is(err, storage.ErrNoSuchMedia) { + return Content{}, err + } + mediaType, _, _ := mime.ParseMediaType(m.ContentType) + if !thumbnailable[mediaType] { + return Content{}, ErrCannotThumbnail + } + + key := fmt.Sprintf("resize\x00%s\x00%dx%d\x00%s", mediaName(m.Origin, m.MediaID), size.Width, size.Height, size.Method) + v, err := a.shared(ctx, key, func(ctx context.Context) (any, error) { + return a.makeThumbnail(ctx, m, size) + }) + if err != nil { + return Content{}, err + } + t = v.(storage.Thumbnail) + return a.open(ctx, t.ContentType, "", t.Size, t.SHA256) +} + +// makeThumbnail decodes media, resizes it and stores the result. +func (a *API) makeThumbnail(ctx context.Context, m storage.Media, size thumbnailSize) (storage.Thumbnail, error) { + select { + case a.resizing <- struct{}{}: + defer func() { <-a.resizing }() + case <-ctx.Done(): + return storage.Thumbnail{}, ctx.Err() + } + + src, err := a.blobs.Open(ctx, m.SHA256) + if err != nil { + return storage.Thumbnail{}, err + } + defer func() { _ = src.Close() }() + + started := time.Now() + thumb, err := a.resize(src, m, size) + if err != nil { + return storage.Thumbnail{}, err + } + + t := storage.Thumbnail{ + Origin: m.Origin, MediaID: m.MediaID, + Width: size.Width, Height: size.Height, Method: size.Method, + CreatedAtMS: a.nowMS(), + } + if thumb == nil { + // The original is already small enough, and is its own thumbnail. + // The row points at the original's blob, so it is neither decoded + // again nor stored twice. + t.ContentType, t.Size, t.SHA256 = m.ContentType, m.Size, m.SHA256 + } else { + blob, err := a.blobs.Put(ctx, bytes.NewReader(thumb.data), int64(len(thumb.data))) + if err != nil { + return storage.Thumbnail{}, err + } + t.ContentType, t.Size, t.SHA256 = thumb.contentType, blob.Size, blob.SHA256 + } + if err := a.store.StoreThumbnail(ctx, t); err != nil { + return storage.Thumbnail{}, err + } + a.log.Debug("made a thumbnail", "media", mediaName(m.Origin, m.MediaID), + "size", fmt.Sprintf("%dx%d", size.Width, size.Height), "method", size.Method, + "bytes", t.Size, "duration", time.Since(started)) + return t, nil +} + +// encoded is a finished thumbnail. +type encoded struct { + contentType string + data []byte +} + +// resize makes the thumbnail of an image. It returns nil, and no error, +// when the original already fits and can be served as it is. +func (a *API) resize(src io.ReadSeeker, m storage.Media, size thumbnailSize) (*encoded, error) { + // The header first: the pixel count is checked before a single pixel + // is decoded. + header, format, err := image.DecodeConfig(src) + if err != nil { + return nil, ErrCannotThumbnail + } + if int64(header.Width)*int64(header.Height) > a.cfg.MaxThumbnailPixels { + return nil, fmt.Errorf("%w: the image is %d × %d", ErrCannotThumbnail, header.Width, header.Height) + } + + orientation := 1 + if format == "jpeg" { + if _, err := src.Seek(0, io.SeekStart); err != nil { + return nil, err + } + orientation = jpegOrientation(io.LimitReader(src, 256<<10)) + } + + // Everything below works on the image as stored. An EXIF rotation by + // a quarter turn swaps the box, the crop is centred either way, and + // the small result is turned upright at the end. + boxW, boxH := size.Width, size.Height + if orientation >= 5 { + boxW, boxH = boxH, boxW + } + srcW, srcH := header.Width, header.Height + outW, outH, crop := fit(srcW, srcH, boxW, boxH, size.Method) + + // A GIF is always re-encoded: it may be animated, and a still + // thumbnail was asked for. + if outW == srcW && outH == srcH && crop == image.Rect(0, 0, srcW, srcH) && orientation == 1 && format != "gif" { + return nil, nil + } + + if _, err := src.Seek(0, io.SeekStart); err != nil { + return nil, err + } + img, _, err := image.Decode(src) + if err != nil { + return nil, ErrCannotThumbnail + } + // The decoded bounds need not start at the origin. + crop = crop.Add(img.Bounds().Min) + + dst := image.NewRGBA(image.Rect(0, 0, outW, outH)) + draw.CatmullRom.Scale(dst, dst.Bounds(), img, crop, draw.Src, nil) + out := orient(dst, orientation) + + var buf bytes.Buffer + if format == "jpeg" { + if err := jpeg.Encode(&buf, out, &jpeg.Options{Quality: 85}); err != nil { + return nil, err + } + return &encoded{contentType: "image/jpeg", data: buf.Bytes()}, nil + } + // PNG for everything else: the source may have transparency, and a + // thumbnail is small enough that PNG's size does not matter. + if err := png.Encode(&buf, out); err != nil { + return nil, err + } + return &encoded{contentType: "image/png", data: buf.Bytes()}, nil +} + +// fit works out a thumbnail's size and the part of the source it is made +// from. Neither method enlarges: a source smaller than the box gives a +// thumbnail no larger than itself. +// +// "scale" keeps all of the source and its aspect ratio, fitting inside the +// box. "crop" keeps the box's aspect ratio, cutting the source's longer +// side down equally on both ends. +func fit(srcW, srcH, boxW, boxH int, method string) (outW, outH int, crop image.Rectangle) { + full := image.Rect(0, 0, srcW, srcH) + if method == MethodCrop { + ratio := math.Min(1, math.Min(float64(srcW)/float64(boxW), float64(srcH)/float64(boxH))) + outW = max(1, int(math.Round(float64(boxW)*ratio))) + outH = max(1, int(math.Round(float64(boxH)*ratio))) + + // The largest centred rectangle of the box's aspect ratio. + cropW, cropH := srcW, srcH + if srcW*boxH > srcH*boxW { + cropW = max(1, int(math.Round(float64(srcH)*float64(boxW)/float64(boxH)))) + } else { + cropH = max(1, int(math.Round(float64(srcW)*float64(boxH)/float64(boxW)))) + } + x0, y0 := (srcW-cropW)/2, (srcH-cropH)/2 + return outW, outH, image.Rect(x0, y0, x0+cropW, y0+cropH) + } + + ratio := math.Min(1, math.Min(float64(boxW)/float64(srcW), float64(boxH)/float64(srcH))) + outW = max(1, int(math.Round(float64(srcW)*ratio))) + outH = max(1, int(math.Round(float64(srcH)*ratio))) + return outW, outH, full +} + +// orient turns an image upright according to its EXIF orientation: +// +// 1 as stored 5 transposed +// 2 mirrored 6 a quarter turn clockwise +// 3 half a turn 7 transversed +// 4 flipped 8 a quarter turn anticlockwise +// +// It runs on the finished thumbnail, which is small. +func orient(img *image.RGBA, orientation int) image.Image { + if orientation < 2 || orientation > 8 { + return img + } + b := img.Bounds() + w, h := b.Dx(), b.Dy() + outW, outH := w, h + if orientation >= 5 { + outW, outH = h, w + } + out := image.NewRGBA(image.Rect(0, 0, outW, outH)) + for y := range h { + for x := range w { + var dx, dy int + switch orientation { + case 2: + dx, dy = w-1-x, y + case 3: + dx, dy = w-1-x, h-1-y + case 4: + dx, dy = x, h-1-y + case 5: + dx, dy = y, x + case 6: + dx, dy = h-1-y, x + case 7: + dx, dy = h-1-y, w-1-x + case 8: + dx, dy = y, w-1-x + } + out.SetRGBA(dx, dy, img.RGBAAt(b.Min.X+x, b.Min.Y+y)) + } + } + return out +} + +// jpegOrientation reads the EXIF orientation tag of a JPEG, or 1 when it +// has none. It walks the markers up to the start of the image data and +// gives up quietly on anything it does not understand: a wrong guess here +// is a sideways thumbnail, never a failure. +func jpegOrientation(r io.Reader) int { + data, err := io.ReadAll(r) + if err != nil || len(data) < 4 || data[0] != 0xFF || data[1] != 0xD8 { + return 1 + } + pos := 2 + for pos+4 <= len(data) { + if data[pos] != 0xFF { + return 1 + } + marker := data[pos+1] + if marker == 0xD8 || (marker >= 0xD0 && marker <= 0xD7) || marker == 0x01 || marker == 0xFF { + pos++ + continue + } + if marker == 0xDA || marker == 0xD9 { // start of scan, end of image + return 1 + } + length := int(data[pos+2])<<8 | int(data[pos+3]) + if length < 2 || pos+2+length > len(data) { + return 1 + } + segment := data[pos+4 : pos+2+length] + if marker == 0xE1 && len(segment) > 6 && string(segment[:6]) == "Exif\x00\x00" { + return exifOrientation(segment[6:]) + } + pos += 2 + length + } + return 1 +} + +// exifOrientation finds tag 0x0112 in the first IFD of a TIFF structure. +func exifOrientation(tiff []byte) int { + if len(tiff) < 8 { + return 1 + } + var u16 func([]byte) int + var u32 func([]byte) int + switch string(tiff[:2]) { + case "II": + u16 = func(b []byte) int { return int(b[0]) | int(b[1])<<8 } + u32 = func(b []byte) int { return int(b[0]) | int(b[1])<<8 | int(b[2])<<16 | int(b[3])<<24 } + case "MM": + u16 = func(b []byte) int { return int(b[0])<<8 | int(b[1]) } + u32 = func(b []byte) int { return int(b[0])<<24 | int(b[1])<<16 | int(b[2])<<8 | int(b[3]) } + default: + return 1 + } + if u16(tiff[2:4]) != 42 { + return 1 + } + ifd := u32(tiff[4:8]) + if ifd < 8 || ifd+2 > len(tiff) { + return 1 + } + entries := u16(tiff[ifd : ifd+2]) + for i := range entries { + entry := ifd + 2 + i*12 + if entry+12 > len(tiff) { + return 1 + } + if u16(tiff[entry:entry+2]) == 0x0112 { + if o := u16(tiff[entry+8 : entry+10]); o >= 1 && o <= 8 { + return o + } + return 1 + } + } + return 1 +} diff --git a/server/internal/mediaapi/waiters.go b/server/internal/mediaapi/waiters.go new file mode 100644 index 0000000..835a7eb --- /dev/null +++ b/server/internal/mediaapi/waiters.go @@ -0,0 +1,57 @@ +package mediaapi + +import "sync" + +// waiters wakes the downloads blocked on a reserved media ID when its +// content arrives. +// +// It is in memory, which is enough for one process: the upload and the +// download that waits for it are both requests to this server. +type waiters struct { + mu sync.Mutex + next uint64 + subs map[string]map[uint64]chan struct{} +} + +func newWaiters() *waiters { + return &waiters{subs: map[string]map[uint64]chan struct{}{}} +} + +// subscribe returns a channel closed when the media's content arrives, and +// a function to call when no longer waiting. +func (w *waiters) subscribe(origin, mediaID string) (<-chan struct{}, func()) { + key := origin + "/" + mediaID + ch := make(chan struct{}) + + w.mu.Lock() + defer w.mu.Unlock() + w.next++ + id := w.next + if w.subs[key] == nil { + w.subs[key] = map[uint64]chan struct{}{} + } + w.subs[key][id] = ch + + return ch, func() { + w.mu.Lock() + defer w.mu.Unlock() + if subs := w.subs[key]; subs != nil { + delete(subs, id) + if len(subs) == 0 { + delete(w.subs, key) + } + } + } +} + +// wake releases everybody waiting for the media. +func (w *waiters) wake(origin, mediaID string) { + key := origin + "/" + mediaID + + w.mu.Lock() + defer w.mu.Unlock() + for _, ch := range w.subs[key] { + close(ch) + } + delete(w.subs, key) +} diff --git a/server/internal/setup/media.go b/server/internal/setup/media.go new file mode 100644 index 0000000..46a51c2 --- /dev/null +++ b/server/internal/setup/media.go @@ -0,0 +1,34 @@ +package setup + +import ( + "log/slog" + + "github.com/dariuszjaniak/warp/server/internal/config" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" + "github.com/dariuszjaniak/warp/server/internal/storage" + "github.com/dariuszjaniak/warp/server/internal/storage/filestore" +) + +// newMedia builds the media repository. Its content goes to the storage +// driver's own file storage if it has one, and under media.base_path if +// not. +func newMedia(cfg *config.Config, db storage.Database, log *slog.Logger) (*mediaapi.API, error) { + var blobs storage.BlobStore + if provider, ok := db.(storage.BlobProvider); ok { + blobs = provider.Blobs() + } else { + store, err := filestore.New(cfg.Media.BasePath) + if err != nil { + return nil, err + } + blobs = store + log.Debug("media is stored on the local filesystem", "path", store.Root()) + } + return mediaapi.New(mediaapi.Options{ + ServerName: cfg.Global.ServerName, + Store: db.Media(), + Blobs: blobs, + Config: cfg.Media, + Logger: log, + }) +} diff --git a/server/internal/setup/phase5_test.go b/server/internal/setup/phase5_test.go index f361b9b..2e8715d 100644 --- a/server/internal/setup/phase5_test.go +++ b/server/internal/setup/phase5_test.go @@ -103,6 +103,7 @@ func newFederation(t *testing.T, names ...string) map[string]*fedNode { cfg.Registration.Enabled = true cfg.ClientAPI.RateLimit.Enabled = false cfg.Federation.Enabled = true + cfg.Media.BasePath = filepath.Join(t.TempDir(), name+"-media") if err := cfg.Validate(); err != nil { t.Fatalf("config: %v", err) } diff --git a/server/internal/setup/phase6_test.go b/server/internal/setup/phase6_test.go new file mode 100644 index 0000000..46a5e77 --- /dev/null +++ b/server/internal/setup/phase6_test.go @@ -0,0 +1,330 @@ +package setup_test + +import ( + "bytes" + "image" + "image/color" + "image/png" + "net/http" + "net/http/httptest" + "path/filepath" + "strings" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/config" + "github.com/dariuszjaniak/warp/server/internal/setup" +) + +// newMediaServer is a server with its media in a temporary directory. +func newMediaServer(t *testing.T, tweak func(*config.Config)) *setup.Server { + t.Helper() + dir := t.TempDir() + return newTestServer(t, func(cfg *config.Config) { + cfg.Registration.Enabled = true + cfg.ClientAPI.RateLimit.Enabled = false + cfg.Media.BasePath = filepath.Join(dir, "media") + if tweak != nil { + tweak(cfg) + } + }) +} + +// send is a request with a raw body. +func send(t *testing.T, srv *setup.Server, method, path, contentType string, body []byte, token string, header http.Header) *httptest.ResponseRecorder { + t.Helper() + req := httptest.NewRequest(method, path, bytes.NewReader(body)) + if contentType != "" { + req.Header.Set("Content-Type", contentType) + } + if token != "" { + req.Header.Set("Authorization", "Bearer "+token) + } + for k, v := range header { + req.Header[k] = v + } + rec := httptest.NewRecorder() + srv.Handler().ServeHTTP(rec, req) + return rec +} + +// uploadThrough uploads content and returns the server name and media ID +// of its mxc:// URI. +func uploadThrough(t *testing.T, srv *setup.Server, token, contentType, fileName string, body []byte) (string, string) { + t.Helper() + path := "/_matrix/media/v3/upload" + if fileName != "" { + path += "?filename=" + fileName + } + rec := send(t, srv, http.MethodPost, path, contentType, body, token, nil) + if rec.Code != http.StatusOK { + t.Fatalf("upload: %d %s", rec.Code, rec.Body.String()) + } + return splitMXC(t, decode[map[string]string](t, rec)["content_uri"]) +} + +func splitMXC(t *testing.T, mxc string) (string, string) { + t.Helper() + rest, ok := strings.CutPrefix(mxc, "mxc://") + server, id, found := strings.Cut(rest, "/") + if !ok || !found { + t.Fatalf("bad mxc URI %q", mxc) + } + return server, id +} + +func pngOf(t *testing.T, w, h int) []byte { + t.Helper() + img := image.NewRGBA(image.Rect(0, 0, w, h)) + for y := range h { + for x := range w { + img.Set(x, y, color.RGBA{uint8(x), uint8(y), 200, 255}) + } + } + var buf bytes.Buffer + if err := png.Encode(&buf, img); err != nil { + t.Fatal(err) + } + return buf.Bytes() +} + +func TestMediaIsWiredUp(t *testing.T) { + srv := newMediaServer(t, nil) + token := registerThrough(t, srv, "alice") + + rec := getWithToken(t, srv, "/_matrix/client/v1/media/config", token) + if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"m.upload.size":52428800`) { + t.Fatalf("media config: %d %s", rec.Code, rec.Body.String()) + } + + server, id := uploadThrough(t, srv, token, "text/plain", "hello.txt", []byte("hello, media")) + if server != "warp.example" { + t.Errorf("uploaded to %q", server) + } + + download := "/_matrix/client/v1/media/download/" + server + "/" + id + rec = getWithToken(t, srv, download, token) + if rec.Code != http.StatusOK || rec.Body.String() != "hello, media" { + t.Fatalf("download: %d %q", rec.Code, rec.Body.String()) + } + for header, want := range map[string]string{ + "Content-Type": "text/plain", + "Content-Disposition": "inline; filename=hello.txt", + "X-Content-Type-Options": "nosniff", + "Cache-Control": "private, max-age=31536000, immutable", + } { + if got := rec.Header().Get(header); got != want { + t.Errorf("%s = %q, want %q", header, got, want) + } + } + if csp := rec.Header().Get("Content-Security-Policy"); !strings.HasPrefix(csp, "sandbox;") { + t.Errorf("Content-Security-Policy = %q", csp) + } + + // The authenticated endpoints need a token. + if rec := get(t, srv, http.MethodGet, download); rec.Code != http.StatusUnauthorized { + t.Errorf("download without a token: %d", rec.Code) + } + // The deprecated unauthenticated endpoint is off by default. + if rec := get(t, srv, http.MethodGet, "/_matrix/media/v3/download/"+server+"/"+id); rec.Code != http.StatusNotFound || + !strings.Contains(rec.Body.String(), "M_UNRECOGNIZED") { + t.Errorf("legacy download: %d %s", rec.Code, rec.Body.String()) + } + + // A file name in the path renames the download. + rec = getWithToken(t, srv, download+"/other.txt", token) + if got := rec.Header().Get("Content-Disposition"); got != "inline; filename=other.txt" { + t.Errorf("renamed download: Content-Disposition = %q", got) + } + + // Ranges and conditional requests come with http.ServeContent. + rec = send(t, srv, http.MethodGet, download, "", nil, token, http.Header{"Range": {"bytes=0-4"}}) + if rec.Code != http.StatusPartialContent || rec.Body.String() != "hello" { + t.Errorf("range: %d %q", rec.Code, rec.Body.String()) + } + etag := getWithToken(t, srv, download, token).Header().Get("ETag") + rec = send(t, srv, http.MethodGet, download, "", nil, token, http.Header{"If-None-Match": {etag}}) + if rec.Code != http.StatusNotModified { + t.Errorf("If-None-Match: %d", rec.Code) + } + + // An HTML upload is served as an attachment, never as a page. + _, page := uploadThrough(t, srv, token, "text/html", "page.html", []byte("")) + rec = getWithToken(t, srv, "/_matrix/client/v1/media/download/warp.example/"+page, token) + if got := rec.Header().Get("Content-Disposition"); !strings.HasPrefix(got, "attachment") { + t.Errorf("HTML Content-Disposition = %q", got) + } + + rec = getWithToken(t, srv, "/_matrix/client/v1/media/download/warp.example/doesnotexist", token) + if rec.Code != http.StatusNotFound || !strings.Contains(rec.Body.String(), "M_NOT_FOUND") { + t.Errorf("unknown media: %d %s", rec.Code, rec.Body.String()) + } +} + +func TestMediaThumbnailsThroughTheAPI(t *testing.T) { + srv := newMediaServer(t, nil) + token := registerThrough(t, srv, "alice") + server, id := uploadThrough(t, srv, token, "image/png", "", pngOf(t, 400, 300)) + + thumb := "/_matrix/client/v1/media/thumbnail/" + server + "/" + id + rec := getWithToken(t, srv, thumb+"?width=64&height=64&method=crop", token) + if rec.Code != http.StatusOK || rec.Header().Get("Content-Type") != "image/png" { + t.Fatalf("thumbnail: %d %s %s", rec.Code, rec.Header().Get("Content-Type"), rec.Body.String()) + } + img, err := png.Decode(rec.Body) + if err != nil { + t.Fatalf("thumbnail does not decode: %v", err) + } + if b := img.Bounds(); b.Dx() != 96 || b.Dy() != 96 { + t.Errorf("thumbnail is %v, want 96 × 96", b) + } + + for query, want := range map[string]string{ + "": "M_MISSING_PARAM", + "?width=0&height=10": "M_INVALID_PARAM", + "?width=10&height=10&method=zap": "M_INVALID_PARAM", + } { + rec := getWithToken(t, srv, thumb+query, token) + if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), want) { + t.Errorf("thumbnail%s: %d %s, want %s", query, rec.Code, rec.Body.String(), want) + } + } + + _, text := uploadThrough(t, srv, token, "text/plain", "", []byte("words")) + rec = getWithToken(t, srv, "/_matrix/client/v1/media/thumbnail/warp.example/"+text+"?width=32&height=32", token) + if rec.Code != http.StatusBadRequest { + t.Errorf("thumbnail of text: %d %s", rec.Code, rec.Body.String()) + } +} + +func TestAsyncUpload(t *testing.T) { + srv := newMediaServer(t, nil) + alice := registerThrough(t, srv, "alice") + bob := registerThrough(t, srv, "bob") + + rec := post(t, srv, "/_matrix/media/v1/create", map[string]any{}, alice) + if rec.Code != http.StatusOK { + t.Fatalf("create: %d %s", rec.Code, rec.Body.String()) + } + created := decode[map[string]any](t, rec) + mxc, _ := created["content_uri"].(string) + if _, ok := created["unused_expires_at"].(float64); !ok { + t.Errorf("create response lacks unused_expires_at: %v", created) + } + server, id := splitMXC(t, mxc) + + download := "/_matrix/client/v1/media/download/" + server + "/" + id + rec = getWithToken(t, srv, download+"?timeout_ms=0", alice) + if rec.Code != http.StatusGatewayTimeout || !strings.Contains(rec.Body.String(), "M_NOT_YET_UPLOADED") { + t.Errorf("download before upload: %d %s", rec.Code, rec.Body.String()) + } + + put := "/_matrix/media/v3/upload/" + server + "/" + id + if rec := send(t, srv, http.MethodPut, put, "text/plain", []byte("bob's"), bob, nil); rec.Code != http.StatusForbidden { + t.Errorf("upload to another user's ID: %d %s", rec.Code, rec.Body.String()) + } + if rec := send(t, srv, http.MethodPut, put, "text/plain", []byte("later"), alice, nil); rec.Code != http.StatusOK { + t.Fatalf("upload: %d %s", rec.Code, rec.Body.String()) + } + rec = send(t, srv, http.MethodPut, put, "text/plain", []byte("again"), alice, nil) + if rec.Code != http.StatusConflict || !strings.Contains(rec.Body.String(), "M_CANNOT_OVERWRITE_MEDIA") { + t.Errorf("second upload: %d %s", rec.Code, rec.Body.String()) + } + if rec := getWithToken(t, srv, download, bob); rec.Code != http.StatusOK || rec.Body.String() != "later" { + t.Errorf("download: %d %q", rec.Code, rec.Body.String()) + } +} + +func TestUploadLimit(t *testing.T) { + srv := newMediaServer(t, func(cfg *config.Config) { cfg.Media.MaxUploadBytes = 8 }) + token := registerThrough(t, srv, "alice") + + rec := send(t, srv, http.MethodPost, "/_matrix/media/v3/upload", "text/plain", []byte("more than eight"), token, nil) + if rec.Code != http.StatusRequestEntityTooLarge || !strings.Contains(rec.Body.String(), "M_TOO_LARGE") { + t.Errorf("oversized upload: %d %s", rec.Code, rec.Body.String()) + } + + // Without a Content-Length the blob store cuts it off instead. + req := httptest.NewRequest(http.MethodPost, "/_matrix/media/v3/upload", strings.NewReader("more than eight")) + req.ContentLength = -1 + req.Header.Set("Authorization", "Bearer "+token) + rec = httptest.NewRecorder() + srv.Handler().ServeHTTP(rec, req) + if rec.Code != http.StatusRequestEntityTooLarge { + t.Errorf("oversized chunked upload: %d %s", rec.Code, rec.Body.String()) + } +} + +func TestLegacyMediaDownloads(t *testing.T) { + srv := newMediaServer(t, func(cfg *config.Config) { cfg.Media.LegacyUnauthenticatedDownloads = true }) + token := registerThrough(t, srv, "alice") + server, id := uploadThrough(t, srv, token, "image/png", "", pngOf(t, 10, 10)) + + if rec := get(t, srv, http.MethodGet, "/_matrix/media/v3/download/"+server+"/"+id); rec.Code != http.StatusOK { + t.Errorf("legacy download: %d %s", rec.Code, rec.Body.String()) + } + if rec := get(t, srv, http.MethodGet, "/_matrix/media/v3/thumbnail/"+server+"/"+id+"?width=32&height=32"); rec.Code != http.StatusOK { + t.Errorf("legacy thumbnail: %d %s", rec.Code, rec.Body.String()) + } +} + +func TestMediaOverFederation(t *testing.T) { + if testing.Short() { + t.Skip("starts two servers") + } + nodes := newFederation(t, "hs-a.example", "hs-b.example") + a, b := nodes["hs-a.example"].srv, nodes["hs-b.example"].srv + alice := registerThrough(t, a, "alice") + bob := registerThrough(t, b, "bob") + + server, id := uploadThrough(t, a, alice, "image/png", "cat.png", pngOf(t, 300, 200)) + + // Bob's server fetches Alice's upload from her server. + download := "/_matrix/client/v1/media/download/" + server + "/" + id + rec := getWithToken(t, b, download, bob) + if rec.Code != http.StatusOK || rec.Header().Get("Content-Type") != "image/png" { + t.Fatalf("remote download: %d %s", rec.Code, rec.Body.String()) + } + if got := rec.Header().Get("Content-Disposition"); got != "inline; filename=cat.png" { + t.Errorf("remote download kept no file name: %q", got) + } + if _, err := png.Decode(rec.Body); err != nil { + t.Errorf("remote download is not the PNG: %v", err) + } + + // A thumbnail of media that is cached is made locally; of media that + // is not, it is fetched. + _, other := uploadThrough(t, a, alice, "image/png", "", pngOf(t, 500, 500)) + rec = getWithToken(t, b, "/_matrix/client/v1/media/thumbnail/"+server+"/"+other+"?width=32&height=32&method=crop", bob) + if rec.Code != http.StatusOK { + t.Fatalf("remote thumbnail: %d %s", rec.Code, rec.Body.String()) + } + if img, err := png.Decode(rec.Body); err != nil || img.Bounds().Dx() != 32 { + t.Errorf("remote thumbnail: %v, %v", img, err) + } + + // Unknown media on the other server is a 404 here too. + rec = getWithToken(t, b, "/_matrix/client/v1/media/download/"+server+"/doesnotexist", bob) + if rec.Code != http.StatusNotFound { + t.Errorf("unknown remote media: %d %s", rec.Code, rec.Body.String()) + } + + // With B's federation off, what B has cached is still served and + // nothing else is fetched. + if err := b.Settings().SetFederationEnabled(t.Context(), false); err != nil { + t.Fatal(err) + } + if rec := getWithToken(t, b, download, bob); rec.Code != http.StatusOK { + t.Errorf("cached remote download with federation off: %d %s", rec.Code, rec.Body.String()) + } + _, uncached := uploadThrough(t, a, alice, "text/plain", "", []byte("never fetched")) + rec = getWithToken(t, b, "/_matrix/client/v1/media/download/"+server+"/"+uncached, bob) + if rec.Code != http.StatusForbidden || !strings.Contains(rec.Body.String(), "Federation is disabled") { + t.Errorf("uncached remote download with federation off: %d %s", rec.Code, rec.Body.String()) + } + + // Another server is only served this server's own media, and only + // over federation. + if rec := get(t, a, http.MethodGet, "/_matrix/federation/v1/media/download/"+id); rec.Code != http.StatusUnauthorized { + t.Errorf("unsigned federation download: %d %s", rec.Code, rec.Body.String()) + } +} diff --git a/server/internal/setup/routes.go b/server/internal/setup/routes.go index e43fd8c..0d47ddf 100644 --- a/server/internal/setup/routes.go +++ b/server/internal/setup/routes.go @@ -12,12 +12,14 @@ import ( // // Keep it honest: a version belongs here once the endpoints a client // actually uses from it work. Accounts, sessions, devices, profiles, -// account data, rooms and — with Phase 4 — the event stream all do, which -// is enough for a client to be used. What is still missing is end-to-end -// encryption (Phase 7), media (Phase 6) and push (Phase 8); a client -// discovers each of those from the endpoint it needs rather than from this -// list, which is why their absence does not make a version listed here a -// lie. +// account data, rooms, the event stream and — with Phase 6 — media all +// do, which is enough for a client to be used. Advertising v1.11 or later +// is also what makes a client use the authenticated media endpoints, the +// only download endpoints Warp serves by default. What is still missing +// is end-to-end encryption (Phase 7) and push (Phase 8); a client +// discovers each of those from the endpoint it needs rather than from +// this list, which is why their absence does not make a version listed +// here a lie. var SpecVersions = []string{"v1.13", "v1.14"} // UnstableFeatures advertises MSC support. Simplified Sliding Sync diff --git a/server/internal/setup/server.go b/server/internal/setup/server.go index 5f7bf72..769c14e 100644 --- a/server/internal/setup/server.go +++ b/server/internal/setup/server.go @@ -22,6 +22,7 @@ import ( "github.com/dariuszjaniak/warp/server/internal/federationapi" "github.com/dariuszjaniak/warp/server/internal/httputil" "github.com/dariuszjaniak/warp/server/internal/matrix/signing" + "github.com/dariuszjaniak/warp/server/internal/mediaapi" "github.com/dariuszjaniak/warp/server/internal/metrics" "github.com/dariuszjaniak/warp/server/internal/roomserver" "github.com/dariuszjaniak/warp/server/internal/settings" @@ -45,6 +46,7 @@ type Server struct { users *userapi.API rooms *roomserver.API sync *syncapi.API + media *mediaapi.API clientAPI *clientapi.ClientAPI adminAPI *adminapi.AdminAPI @@ -186,6 +188,15 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) return nil, err } + // The media repository needs no key: uploads and downloads of local + // media are not signed. Fetching other servers' media is, and is + // handed the federation client below once there is one. + s.media, err = newMedia(cfg, s.db, log) + if err != nil { + _ = s.closeDB() + return nil, err + } + // The roomserver needs the signing key: it is the only thing in Warp // that creates an event, and an event has to be signed. `warp migrate` // and the Phase 1 tests start without a key, and then start without @@ -224,6 +235,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) Enabled: settingsService.FederationEnabled, Policy: s.federation.Policy(), }) + s.media.SetRemote(s.fedClient) s.keyRing = federation.NewKeyRing(federation.KeyRingOptions{ ServerName: cfg.Global.ServerName, Key: s.signingKey, @@ -286,6 +298,7 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) Ephemeral: s.sync, Profiles: s.users, Store: s.db.Federation(), + Media: s.media, Logger: log, }) if err != nil { @@ -307,18 +320,20 @@ func New(ctx context.Context, cfg *config.Config, opts Options) (*Server, error) } s.clientAPI, err = clientapi.New(clientapi.Options{ - ServerName: cfg.Global.ServerName, - ClientBaseURL: cfg.ClientBaseURL(), - Users: s.users, - Rooms: s.rooms, - Sync: s.sync, - Settings: settingsService, - Federation: s.federation, - RemoteProfiles: remoteProfiles, - Registration: cfg.Registration, - RateLimit: cfg.ClientAPI.RateLimit, - TrustXForwardedFor: cfg.ClientAPI.TrustXForwardedFor, - Logger: log, + ServerName: cfg.Global.ServerName, + ClientBaseURL: cfg.ClientBaseURL(), + Users: s.users, + Rooms: s.rooms, + Sync: s.sync, + Media: s.media, + LegacyMediaDownloads: cfg.Media.LegacyUnauthenticatedDownloads, + Settings: settingsService, + Federation: s.federation, + RemoteProfiles: remoteProfiles, + Registration: cfg.Registration, + RateLimit: cfg.ClientAPI.RateLimit, + TrustXForwardedFor: cfg.ClientAPI.TrustXForwardedFor, + Logger: log, }) if err != nil { _ = s.closeDB() @@ -399,6 +414,9 @@ func (s *Server) Rooms() *roomserver.API { return s.rooms } // without a signing key. func (s *Server) Sync() *syncapi.API { return s.sync } +// Media exposes the media repository. +func (s *Server) Media() *mediaapi.API { return s.media } + // SigningKey returns the server signing key, or nil when it was skipped. func (s *Server) SigningKey() *signing.KeyPair { return s.signingKey } diff --git a/server/internal/storage/filestore/filestore.go b/server/internal/storage/filestore/filestore.go new file mode 100644 index 0000000..7cc427d --- /dev/null +++ b/server/internal/storage/filestore/filestore.go @@ -0,0 +1,159 @@ +// Package filestore is the local-filesystem storage.BlobStore: media +// content as files under one directory, each named by its SHA-256. +// +// A blob lives at /ab/cd/abcd…, two levels of fan-out so that no +// directory grows past a few thousand entries. Content is written to +// /tmp first and renamed into place once its hash is known, so a +// reader never sees half a file and a crash leaves nothing behind but a +// temporary file. +package filestore + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// Store is a BlobStore rooted at a directory. +type Store struct { + root string +} + +// New returns a store rooted at dir. Nothing is created until the first +// Put: a server that never stores media leaves no directory behind. +func New(dir string) (*Store, error) { + if dir == "" { + return nil, errors.New("filestore: directory must not be empty") + } + abs, err := filepath.Abs(dir) + if err != nil { + return nil, fmt.Errorf("filestore: %w", err) + } + return &Store{root: abs}, nil +} + +// Root is the directory the store writes under. +func (s *Store) Root() string { return s.root } + +// path is where a blob lives. It refuses anything that is not a SHA-256 in +// lower-case hex, which is also what keeps a hash read from a database row +// from naming a file outside the store. +func (s *Store) path(sum string) (string, bool) { + if len(sum) != sha256.Size*2 { + return "", false + } + for _, c := range sum { + if (c < '0' || c > '9') && (c < 'a' || c > 'f') { + return "", false + } + } + return filepath.Join(s.root, sum[0:2], sum[2:4], sum), true +} + +// Put implements storage.BlobStore. +func (s *Store) Put(ctx context.Context, r io.Reader, limit int64) (storage.BlobInfo, error) { + tmpDir := filepath.Join(s.root, "tmp") + if err := os.MkdirAll(tmpDir, 0o750); err != nil { + return storage.BlobInfo{}, fmt.Errorf("create the media directory: %w", err) + } + tmp, err := os.CreateTemp(tmpDir, "upload-*") + if err != nil { + return storage.BlobInfo{}, fmt.Errorf("create a temporary file: %w", err) + } + committed := false + defer func() { + if !committed { + _ = tmp.Close() + _ = os.Remove(tmp.Name()) + } + }() + + hash := sha256.New() + n, err := io.Copy(io.MultiWriter(tmp, hash), &ctxReader{ctx: ctx, r: io.LimitReader(r, limit+1)}) + if err != nil { + return storage.BlobInfo{}, fmt.Errorf("store content: %w", err) + } + if n > limit { + return storage.BlobInfo{}, storage.ErrBlobTooLarge + } + if err := tmp.Sync(); err != nil { + return storage.BlobInfo{}, fmt.Errorf("store content: %w", err) + } + if err := tmp.Close(); err != nil { + return storage.BlobInfo{}, fmt.Errorf("store content: %w", err) + } + + info := storage.BlobInfo{SHA256: hex.EncodeToString(hash.Sum(nil)), Size: n} + final, _ := s.path(info.SHA256) + if _, err := os.Stat(final); err == nil { + // The same content is already stored: the copy just written is + // the same bytes, and is dropped. + return info, nil + } + if err := os.MkdirAll(filepath.Dir(final), 0o750); err != nil { + return storage.BlobInfo{}, fmt.Errorf("create the media directory: %w", err) + } + if err := os.Rename(tmp.Name(), final); err != nil { + // Another Put of the same content may have got there first, + // which on Windows makes the rename fail rather than replace. + if _, statErr := os.Stat(final); statErr == nil { + return info, nil + } + return storage.BlobInfo{}, fmt.Errorf("store content: %w", err) + } + committed = true + return info, nil +} + +// Open implements storage.BlobStore. +func (s *Store) Open(_ context.Context, sum string) (io.ReadSeekCloser, error) { + p, ok := s.path(sum) + if !ok { + return nil, storage.ErrNoSuchBlob + } + f, err := os.Open(p) + if errors.Is(err, fs.ErrNotExist) { + return nil, storage.ErrNoSuchBlob + } + if err != nil { + return nil, fmt.Errorf("open media content: %w", err) + } + return f, nil +} + +// Delete implements storage.BlobStore. +func (s *Store) Delete(_ context.Context, sum string) error { + p, ok := s.path(sum) + if !ok { + // Not a hash, so not the name of anything stored. + return nil + } + if err := os.Remove(p); err != nil && !errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("delete media content: %w", err) + } + return nil +} + +// ctxReader stops a copy when its context ends: an upload from a client +// that has gone away should not keep writing to disk. +type ctxReader struct { + ctx context.Context + r io.Reader +} + +func (c *ctxReader) Read(p []byte) (int, error) { + if err := c.ctx.Err(); err != nil { + return 0, err + } + return c.r.Read(p) +} + +var _ storage.BlobStore = (*Store)(nil) diff --git a/server/internal/storage/media.go b/server/internal/storage/media.go new file mode 100644 index 0000000..afa84da --- /dev/null +++ b/server/internal/storage/media.go @@ -0,0 +1,149 @@ +package storage + +import ( + "context" + "errors" + "io" +) + +// Errors of the media store. +var ( + // ErrNoSuchMedia means no media, or no thumbnail, is stored under the + // given name. + ErrNoSuchMedia = errors.New("no such media") + + // ErrMediaExists is CreateMedia's answer for a media ID that is taken. + ErrMediaExists = errors.New("media already exists") + + // ErrMediaUploaded is CompleteUpload's answer for media whose content + // has already been uploaded: content, once stored, never changes. + ErrMediaUploaded = errors.New("media content already uploaded") +) + +// Media is the metadata of one piece of content: uploaded here, or fetched +// from the server it was uploaded to and kept. +type Media struct { + // Origin is the server name in the mxc:// URI: this server for an + // upload, another for cached remote media. + Origin string + MediaID string + + // ContentType is the Content-Type the content was uploaded with, + // parameters included. + ContentType string + + // UploadName is the file name the uploader gave, or "". + UploadName string + + // Size is the content's length in bytes. + Size int64 + + // SHA256 names the content in the blob store: its SHA-256, in + // lower-case hex. Empty while the media is pending. + SHA256 string + + // UserID is the uploader; "" for remote media. + UserID string + + CreatedAtMS int64 + + // Pending is set for an ID handed out by /create whose content has + // not been uploaded yet. Pending media expires at ExpiresAtMS. + Pending bool + ExpiresAtMS int64 +} + +// MediaContent is what CompleteUpload fills in on pending media. +type MediaContent struct { + ContentType string + UploadName string + Size int64 + SHA256 string +} + +// Thumbnail is one generated (or, for remote media, fetched) thumbnail. +type Thumbnail struct { + Origin string + MediaID string + + // Width, Height and Method are the size the thumbnail was made for — + // one of the configured sizes — not the size of the image itself, + // which for "scale" keeps the original's aspect ratio. + Width int + Height int + Method string + + ContentType string + Size int64 + SHA256 string + CreatedAtMS int64 +} + +// MediaStore is the mediaapi store: the metadata of local and cached +// remote media and of their thumbnails. The content itself is in a +// BlobStore. +type MediaStore interface { + // CreateMedia records media: an upload, a pending upload or a remote + // fetch. A taken (origin, media ID) is ErrMediaExists. + CreateMedia(ctx context.Context, m Media) error + + // Media reads one media record, pending or not. + Media(ctx context.Context, origin, mediaID string) (Media, error) + + // CompleteUpload stores the content of pending media and clears its + // pending flag. It is ErrNoSuchMedia for an unknown ID and + // ErrMediaUploaded for media that is not pending. + CompleteUpload(ctx context.Context, origin, mediaID string, content MediaContent) error + + // PendingUploads counts a user's pending media that has not expired + // by nowMS. + PendingUploads(ctx context.Context, userID string, nowMS int64) (int, error) + + // DeleteExpiredUploads removes pending media that expired before + // nowMS, reporting how many. + DeleteExpiredUploads(ctx context.Context, nowMS int64) (int, error) + + // Thumbnail reads the thumbnail made for one size. + Thumbnail(ctx context.Context, origin, mediaID string, width, height int, method string) (Thumbnail, error) + + // StoreThumbnail records a thumbnail, replacing one stored for the + // same size. + StoreThumbnail(ctx context.Context, t Thumbnail) error +} + +// ErrNoSuchBlob means the blob store holds nothing under a hash. +var ErrNoSuchBlob = errors.New("no such blob") + +// ErrBlobTooLarge means content passed the limit it was stored with. +var ErrBlobTooLarge = errors.New("content is larger than allowed") + +// BlobInfo describes stored content. +type BlobInfo struct { + // SHA256 is the lower-case hex SHA-256 of the content, and its name in + // the store. + SHA256 string + Size int64 +} + +// BlobStore keeps media content, addressed by its SHA-256. The same bytes +// uploaded twice are stored once. +type BlobStore interface { + // Put stores what r yields, up to limit bytes; more is + // ErrBlobTooLarge and stores nothing. + Put(ctx context.Context, r io.Reader, limit int64) (BlobInfo, error) + + // Open reads stored content. + Open(ctx context.Context, sha256 string) (io.ReadSeekCloser, error) + + // Delete removes stored content. Deleting what is not there is not an + // error. + Delete(ctx context.Context, sha256 string) error +} + +// BlobProvider is implemented by a storage driver that brings its own +// BlobStore. PocketBase will (PLAN.md section 4: its filesystem +// abstraction puts media on local disk or S3); a driver that does not is +// given the local filesystem store. +type BlobProvider interface { + Blobs() BlobStore +} diff --git a/server/internal/storage/media_test.go b/server/internal/storage/media_test.go new file mode 100644 index 0000000..6880ef3 --- /dev/null +++ b/server/internal/storage/media_test.go @@ -0,0 +1,225 @@ +package storage_test + +import ( + "bytes" + "crypto/sha256" + "database/sql" + "encoding/hex" + "errors" + "io" + "os" + "strings" + "testing" + + "github.com/dariuszjaniak/warp/server/internal/storage" + "github.com/dariuszjaniak/warp/server/internal/storage/filestore" +) + +// migratedMediaDB opens a driver with the media tables empty. +func migratedMediaDB(t *testing.T, driver string) storage.Database { + t.Helper() + db := openTestDB(t, driver) + if err := db.Migrate(t.Context()); err != nil { + t.Fatalf("Migrate: %v", err) + } + if driver == storage.DriverPostgres { + pool, err := sql.Open("pgx", os.Getenv("WARP_TEST_POSTGRES_DSN")) + if err != nil { + t.Fatalf("open postgres to reset the media tables: %v", err) + } + defer func() { _ = pool.Close() }() + if _, err := pool.ExecContext(t.Context(), `TRUNCATE media_repository, media_thumbnails`); err != nil { + t.Fatalf("reset the media tables: %v", err) + } + } + return db +} + +func TestMediaUploadAndRead(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + media := migratedMediaDB(t, driver).Media() + + m := storage.Media{ + Origin: "warp.example", MediaID: "abc", ContentType: "image/png", + UploadName: "cat.png", Size: 42, SHA256: strings.Repeat("a", 64), + UserID: "@alice:warp.example", CreatedAtMS: 1000, + } + if err := media.CreateMedia(ctx, m); err != nil { + t.Fatalf("CreateMedia: %v", err) + } + if err := media.CreateMedia(ctx, m); !errors.Is(err, storage.ErrMediaExists) { + t.Fatalf("CreateMedia twice = %v, want ErrMediaExists", err) + } + got, err := media.Media(ctx, "warp.example", "abc") + if err != nil { + t.Fatalf("Media: %v", err) + } + if got != m { + t.Errorf("Media = %+v, want %+v", got, m) + } + if _, err := media.Media(ctx, "other.example", "abc"); !errors.Is(err, storage.ErrNoSuchMedia) { + t.Errorf("Media of another origin = %v, want ErrNoSuchMedia", err) + } + + // Content, once stored, does not change. + err = media.CompleteUpload(ctx, "warp.example", "abc", storage.MediaContent{SHA256: strings.Repeat("b", 64)}) + if !errors.Is(err, storage.ErrMediaUploaded) { + t.Errorf("CompleteUpload of uploaded media = %v, want ErrMediaUploaded", err) + } + err = media.CompleteUpload(ctx, "warp.example", "nope", storage.MediaContent{}) + if !errors.Is(err, storage.ErrNoSuchMedia) { + t.Errorf("CompleteUpload of unknown media = %v, want ErrNoSuchMedia", err) + } + }) + } +} + +func TestMediaPendingUploads(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + media := migratedMediaDB(t, driver).Media() + + const alice = "@alice:warp.example" + for _, p := range []struct { + id string + expires int64 + }{{"p1", 5000}, {"p2", 5000}, {"old", 1500}} { + if err := media.CreateMedia(ctx, storage.Media{ + Origin: "warp.example", MediaID: p.id, UserID: alice, + CreatedAtMS: 1000, Pending: true, ExpiresAtMS: p.expires, + }); err != nil { + t.Fatalf("CreateMedia(%s): %v", p.id, err) + } + } + + n, err := media.PendingUploads(ctx, alice, 2000) + if err != nil { + t.Fatalf("PendingUploads: %v", err) + } + if n != 2 { + t.Errorf("PendingUploads = %d, want 2 (one has expired)", n) + } + + content := storage.MediaContent{ + ContentType: "text/plain", UploadName: "a.txt", Size: 3, SHA256: strings.Repeat("c", 64), + } + if err := media.CompleteUpload(ctx, "warp.example", "p1", content); err != nil { + t.Fatalf("CompleteUpload: %v", err) + } + got, err := media.Media(ctx, "warp.example", "p1") + if err != nil { + t.Fatalf("Media: %v", err) + } + if got.Pending || got.ExpiresAtMS != 0 || got.SHA256 != content.SHA256 || got.Size != 3 { + t.Errorf("completed media = %+v", got) + } + if err := media.CompleteUpload(ctx, "warp.example", "p1", content); !errors.Is(err, storage.ErrMediaUploaded) { + t.Errorf("second CompleteUpload = %v, want ErrMediaUploaded", err) + } + + deleted, err := media.DeleteExpiredUploads(ctx, 2000) + if err != nil { + t.Fatalf("DeleteExpiredUploads: %v", err) + } + if deleted != 1 { + t.Errorf("DeleteExpiredUploads = %d, want 1", deleted) + } + if _, err := media.Media(ctx, "warp.example", "old"); !errors.Is(err, storage.ErrNoSuchMedia) { + t.Errorf("expired upload still there: %v", err) + } + if _, err := media.Media(ctx, "warp.example", "p1"); err != nil { + t.Errorf("a completed upload was swept: %v", err) + } + }) + } +} + +func TestMediaThumbnails(t *testing.T) { + for _, driver := range testDrivers { + t.Run(driver, func(t *testing.T) { + ctx := t.Context() + media := migratedMediaDB(t, driver).Media() + + th := storage.Thumbnail{ + Origin: "remote.example", MediaID: "xyz", Width: 96, Height: 96, Method: "crop", + ContentType: "image/jpeg", Size: 10, SHA256: strings.Repeat("d", 64), CreatedAtMS: 7, + } + if _, err := media.Thumbnail(ctx, "remote.example", "xyz", 96, 96, "crop"); !errors.Is(err, storage.ErrNoSuchMedia) { + t.Fatalf("Thumbnail before storing = %v, want ErrNoSuchMedia", err) + } + if err := media.StoreThumbnail(ctx, th); err != nil { + t.Fatalf("StoreThumbnail: %v", err) + } + th.SHA256, th.Size = strings.Repeat("e", 64), 11 + if err := media.StoreThumbnail(ctx, th); err != nil { + t.Fatalf("StoreThumbnail again: %v", err) + } + got, err := media.Thumbnail(ctx, "remote.example", "xyz", 96, 96, "crop") + if err != nil { + t.Fatalf("Thumbnail: %v", err) + } + if got != th { + t.Errorf("Thumbnail = %+v, want %+v", got, th) + } + if _, err := media.Thumbnail(ctx, "remote.example", "xyz", 96, 96, "scale"); !errors.Is(err, storage.ErrNoSuchMedia) { + t.Errorf("Thumbnail of another method = %v, want ErrNoSuchMedia", err) + } + }) + } +} + +func TestFileStore(t *testing.T) { + ctx := t.Context() + store, err := filestore.New(t.TempDir()) + if err != nil { + t.Fatal(err) + } + + content := []byte("hello, media") + sum := sha256.Sum256(content) + info, err := store.Put(ctx, bytes.NewReader(content), 100) + if err != nil { + t.Fatalf("Put: %v", err) + } + if info.SHA256 != hex.EncodeToString(sum[:]) || info.Size != int64(len(content)) { + t.Fatalf("Put = %+v", info) + } + // The same content again is the same blob. + again, err := store.Put(ctx, bytes.NewReader(content), 100) + if err != nil || again != info { + t.Fatalf("Put twice = %+v, %v", again, err) + } + + f, err := store.Open(ctx, info.SHA256) + if err != nil { + t.Fatalf("Open: %v", err) + } + got, _ := io.ReadAll(f) + _ = f.Close() + if !bytes.Equal(got, content) { + t.Errorf("Open read %q", got) + } + + if _, err := store.Put(ctx, bytes.NewReader(content), 5); !errors.Is(err, storage.ErrBlobTooLarge) { + t.Errorf("Put past the limit = %v, want ErrBlobTooLarge", err) + } + // A hash that is not one cannot name a file, inside the store or out. + for _, bad := range []string{"../../etc/passwd", strings.Repeat("G", 64), ""} { + if _, err := store.Open(ctx, bad); !errors.Is(err, storage.ErrNoSuchBlob) { + t.Errorf("Open(%q) = %v, want ErrNoSuchBlob", bad, err) + } + } + + if err := store.Delete(ctx, info.SHA256); err != nil { + t.Fatalf("Delete: %v", err) + } + if _, err := store.Open(ctx, info.SHA256); !errors.Is(err, storage.ErrNoSuchBlob) { + t.Errorf("Open after Delete = %v, want ErrNoSuchBlob", err) + } + if err := store.Delete(ctx, info.SHA256); err != nil { + t.Errorf("Delete twice = %v", err) + } +} diff --git a/server/internal/storage/pocketbase/pocketbase.go b/server/internal/storage/pocketbase/pocketbase.go index 56c0a5d..5dac29c 100644 --- a/server/internal/storage/pocketbase/pocketbase.go +++ b/server/internal/storage/pocketbase/pocketbase.go @@ -63,6 +63,9 @@ func (*Database) Sync() storage.SyncStore { return notImplementedSync{} } // Federation implements storage.Database. func (*Database) Federation() storage.FederationStore { return notImplementedFederation{} } +// Media implements storage.Database. +func (*Database) Media() storage.MediaStore { return notImplementedMedia{} } + // Close implements storage.Database. func (*Database) Close() error { return nil } @@ -431,6 +434,39 @@ func (notImplementedSync) Filter(context.Context, string, string) ([]byte, error return nil, storage.ErrNotImplemented } +// notImplementedMedia satisfies storage.MediaStore the same way. An upload +// whose metadata was not stored would hand the client an mxc:// URI that +// refers to nothing. +type notImplementedMedia struct{} + +func (notImplementedMedia) CreateMedia(context.Context, storage.Media) error { + return storage.ErrNotImplemented +} + +func (notImplementedMedia) Media(context.Context, string, string) (storage.Media, error) { + return storage.Media{}, storage.ErrNotImplemented +} + +func (notImplementedMedia) CompleteUpload(context.Context, string, string, storage.MediaContent) error { + return storage.ErrNotImplemented +} + +func (notImplementedMedia) PendingUploads(context.Context, string, int64) (int, error) { + return 0, storage.ErrNotImplemented +} + +func (notImplementedMedia) DeleteExpiredUploads(context.Context, int64) (int, error) { + return 0, storage.ErrNotImplemented +} + +func (notImplementedMedia) Thumbnail(context.Context, string, string, int, int, string) (storage.Thumbnail, error) { + return storage.Thumbnail{}, storage.ErrNotImplemented +} + +func (notImplementedMedia) StoreThumbnail(context.Context, storage.Thumbnail) error { + return storage.ErrNotImplemented +} + // Compile-time proof that the stub keeps up with the interfaces. var ( _ storage.Database = (*Database)(nil) @@ -438,4 +474,5 @@ var ( _ storage.RoomStore = notImplementedRooms{} _ storage.SyncStore = notImplementedSync{} _ storage.FederationStore = notImplementedFederation{} + _ storage.MediaStore = notImplementedMedia{} ) diff --git a/server/internal/storage/postgres/migrations/0007_media.sql b/server/internal/storage/postgres/migrations/0007_media.sql new file mode 100644 index 0000000..e724404 --- /dev/null +++ b/server/internal/storage/postgres/migrations/0007_media.sql @@ -0,0 +1,56 @@ +-- Media: the metadata of content uploaded here and of content fetched from +-- other servers and kept, and of the thumbnails made from either. +-- +-- The content itself is not in the database. It is in the blob store — +-- files under media.base_path by default — named by its SHA-256, so the +-- same bytes uploaded twice are stored once and a row here only says which +-- blob an mxc:// URI refers to. +-- +-- Timestamps are milliseconds since the Unix epoch, as everywhere else in +-- Warp. + +CREATE TABLE IF NOT EXISTS media_repository ( + -- The server name of the mxc:// URI: this server for an upload, + -- another for cached remote media. + media_origin TEXT NOT NULL, + media_id TEXT NOT NULL, + -- As uploaded, parameters included: "text/plain; charset=utf-8". + content_type TEXT NOT NULL DEFAULT '', + upload_name TEXT NOT NULL DEFAULT '', + file_size BIGINT NOT NULL DEFAULT 0, + -- Lower-case hex SHA-256 of the content; '' while pending. + sha256 TEXT NOT NULL DEFAULT '', + -- The uploader; '' for remote media. + user_id TEXT NOT NULL DEFAULT '', + created_at_ms BIGINT NOT NULL, + -- An ID handed out by POST /_matrix/media/v1/create whose content has + -- not been uploaded yet. It is deleted unused at expires_at_ms. + pending BOOLEAN NOT NULL DEFAULT FALSE, + expires_at_ms BIGINT NOT NULL DEFAULT 0, + PRIMARY KEY (media_origin, media_id) +); + +-- Counting a user's pending uploads, and sweeping the expired ones. +CREATE INDEX IF NOT EXISTS media_repository_pending_idx + ON media_repository (pending, user_id, expires_at_ms); + +-- Thumbnails, one per configured size that has been asked for. +-- +-- There is no foreign key to media_repository: the thumbnail of remote +-- media is fetched from its server on its own, and the original it was +-- made from may never have been fetched here at all. +CREATE TABLE IF NOT EXISTS media_thumbnails ( + media_origin TEXT NOT NULL, + media_id TEXT NOT NULL, + -- The configured size the thumbnail answers, not the size of the + -- image: a "scale" thumbnail keeps the original's aspect ratio. + width INTEGER NOT NULL, + height INTEGER NOT NULL, + -- 'crop' or 'scale'. + method TEXT NOT NULL, + content_type TEXT NOT NULL, + file_size BIGINT NOT NULL, + sha256 TEXT NOT NULL, + created_at_ms BIGINT NOT NULL, + PRIMARY KEY (media_origin, media_id, width, height, method) +); diff --git a/server/internal/storage/sqlite/migrations/0007_media.sql b/server/internal/storage/sqlite/migrations/0007_media.sql new file mode 100644 index 0000000..25b29f2 --- /dev/null +++ b/server/internal/storage/sqlite/migrations/0007_media.sql @@ -0,0 +1,56 @@ +-- Media: the metadata of content uploaded here and of content fetched from +-- other servers and kept, and of the thumbnails made from either. +-- +-- The content itself is not in the database. It is in the blob store — +-- files under media.base_path by default — named by its SHA-256, so the +-- same bytes uploaded twice are stored once and a row here only says which +-- blob an mxc:// URI refers to. +-- +-- Timestamps are milliseconds since the Unix epoch, as everywhere else in +-- Warp. + +CREATE TABLE IF NOT EXISTS media_repository ( + -- The server name of the mxc:// URI: this server for an upload, + -- another for cached remote media. + media_origin TEXT NOT NULL, + media_id TEXT NOT NULL, + -- As uploaded, parameters included: "text/plain; charset=utf-8". + content_type TEXT NOT NULL DEFAULT '', + upload_name TEXT NOT NULL DEFAULT '', + file_size BIGINT NOT NULL DEFAULT 0, + -- Lower-case hex SHA-256 of the content; '' while pending. + sha256 TEXT NOT NULL DEFAULT '', + -- The uploader; '' for remote media. + user_id TEXT NOT NULL DEFAULT '', + created_at_ms BIGINT NOT NULL, + -- An ID handed out by POST /_matrix/media/v1/create whose content has + -- not been uploaded yet. It is deleted unused at expires_at_ms. + pending BOOLEAN NOT NULL DEFAULT 0, + expires_at_ms BIGINT NOT NULL DEFAULT 0, + PRIMARY KEY (media_origin, media_id) +); + +-- Counting a user's pending uploads, and sweeping the expired ones. +CREATE INDEX IF NOT EXISTS media_repository_pending_idx + ON media_repository (pending, user_id, expires_at_ms); + +-- Thumbnails, one per configured size that has been asked for. +-- +-- There is no foreign key to media_repository: the thumbnail of remote +-- media is fetched from its server on its own, and the original it was +-- made from may never have been fetched here at all. +CREATE TABLE IF NOT EXISTS media_thumbnails ( + media_origin TEXT NOT NULL, + media_id TEXT NOT NULL, + -- The configured size the thumbnail answers, not the size of the + -- image: a "scale" thumbnail keeps the original's aspect ratio. + width INTEGER NOT NULL, + height INTEGER NOT NULL, + -- 'crop' or 'scale'. + method TEXT NOT NULL, + content_type TEXT NOT NULL, + file_size BIGINT NOT NULL, + sha256 TEXT NOT NULL, + created_at_ms BIGINT NOT NULL, + PRIMARY KEY (media_origin, media_id, width, height, method) +); diff --git a/server/internal/storage/sqlstore/media.go b/server/internal/storage/sqlstore/media.go new file mode 100644 index 0000000..3eec569 --- /dev/null +++ b/server/internal/storage/sqlstore/media.go @@ -0,0 +1,174 @@ +package sqlstore + +import ( + "context" + "database/sql" + "errors" + "fmt" + + "github.com/dariuszjaniak/warp/server/internal/storage" +) + +// Media is the shared SQL implementation of storage.MediaStore. +type Media struct { + db *sql.DB + rebind func(string) string + isUnique func(error) bool +} + +// NewMedia builds a media store. +func NewMedia(db *sql.DB, rebind func(string) string, isUnique func(error) bool) *Media { + if rebind == nil { + rebind = func(query string) string { return query } + } + if isUnique == nil { + isUnique = func(error) bool { return false } + } + return &Media{db: db, rebind: rebind, isUnique: isUnique} +} + +func (s *Media) q(query string) string { return s.rebind(query) } + +const insertMediaSQL = ` + INSERT INTO media_repository + (media_origin, media_id, content_type, upload_name, file_size, sha256, + user_id, created_at_ms, pending, expires_at_ms) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)` + +// CreateMedia implements storage.MediaStore. +func (s *Media) CreateMedia(ctx context.Context, m storage.Media) error { + _, err := s.db.ExecContext(ctx, s.q(insertMediaSQL), + m.Origin, m.MediaID, m.ContentType, m.UploadName, m.Size, m.SHA256, + m.UserID, m.CreatedAtMS, m.Pending, m.ExpiresAtMS) + switch { + case err == nil: + return nil + case s.isUnique(err): + return storage.ErrMediaExists + default: + return fmt.Errorf("store media %s/%s: %w", m.Origin, m.MediaID, err) + } +} + +const selectMediaSQL = ` + SELECT media_origin, media_id, content_type, upload_name, file_size, sha256, + user_id, created_at_ms, pending, expires_at_ms + FROM media_repository WHERE media_origin = ? AND media_id = ?` + +// Media implements storage.MediaStore. +func (s *Media) Media(ctx context.Context, origin, mediaID string) (storage.Media, error) { + var m storage.Media + err := s.db.QueryRowContext(ctx, s.q(selectMediaSQL), origin, mediaID).Scan( + &m.Origin, &m.MediaID, &m.ContentType, &m.UploadName, &m.Size, &m.SHA256, + &m.UserID, &m.CreatedAtMS, &m.Pending, &m.ExpiresAtMS) + switch { + case err == nil: + return m, nil + case errors.Is(err, sql.ErrNoRows): + return storage.Media{}, storage.ErrNoSuchMedia + default: + return storage.Media{}, fmt.Errorf("read media %s/%s: %w", origin, mediaID, err) + } +} + +// completeUploadSQL only touches pending media, so that two uploads racing +// for the same ID cannot both win: the second matches no row. +const completeUploadSQL = ` + UPDATE media_repository + SET content_type = ?, upload_name = ?, file_size = ?, sha256 = ?, + pending = ?, expires_at_ms = 0 + WHERE media_origin = ? AND media_id = ? AND pending = ?` + +// CompleteUpload implements storage.MediaStore. +func (s *Media) CompleteUpload(ctx context.Context, origin, mediaID string, c storage.MediaContent) error { + res, err := s.db.ExecContext(ctx, s.q(completeUploadSQL), + c.ContentType, c.UploadName, c.Size, c.SHA256, false, origin, mediaID, true) + if err != nil { + return fmt.Errorf("complete upload %s/%s: %w", origin, mediaID, err) + } + n, err := res.RowsAffected() + if err != nil { + return fmt.Errorf("complete upload %s/%s: %w", origin, mediaID, err) + } + if n == 1 { + return nil + } + // Nothing pending matched: either there is no such media, or its + // content is already there. + if _, err := s.Media(ctx, origin, mediaID); err != nil { + return err + } + return storage.ErrMediaUploaded +} + +const countPendingSQL = ` + SELECT COUNT(*) FROM media_repository + WHERE pending = ? AND user_id = ? AND expires_at_ms > ?` + +// PendingUploads implements storage.MediaStore. +func (s *Media) PendingUploads(ctx context.Context, userID string, nowMS int64) (int, error) { + var n int + if err := s.db.QueryRowContext(ctx, s.q(countPendingSQL), true, userID, nowMS).Scan(&n); err != nil { + return 0, fmt.Errorf("count the pending uploads of %s: %w", userID, err) + } + return n, nil +} + +const deleteExpiredSQL = ` + DELETE FROM media_repository WHERE pending = ? AND expires_at_ms <= ?` + +// DeleteExpiredUploads implements storage.MediaStore. +func (s *Media) DeleteExpiredUploads(ctx context.Context, nowMS int64) (int, error) { + res, err := s.db.ExecContext(ctx, s.q(deleteExpiredSQL), true, nowMS) + if err != nil { + return 0, fmt.Errorf("delete expired uploads: %w", err) + } + n, err := res.RowsAffected() + if err != nil { + return 0, fmt.Errorf("delete expired uploads: %w", err) + } + return int(n), nil +} + +const selectThumbnailSQL = ` + SELECT media_origin, media_id, width, height, method, content_type, + file_size, sha256, created_at_ms + FROM media_thumbnails + WHERE media_origin = ? AND media_id = ? AND width = ? AND height = ? AND method = ?` + +// Thumbnail implements storage.MediaStore. +func (s *Media) Thumbnail(ctx context.Context, origin, mediaID string, width, height int, method string) (storage.Thumbnail, error) { + var t storage.Thumbnail + err := s.db.QueryRowContext(ctx, s.q(selectThumbnailSQL), origin, mediaID, width, height, method).Scan( + &t.Origin, &t.MediaID, &t.Width, &t.Height, &t.Method, &t.ContentType, + &t.Size, &t.SHA256, &t.CreatedAtMS) + switch { + case err == nil: + return t, nil + case errors.Is(err, sql.ErrNoRows): + return storage.Thumbnail{}, storage.ErrNoSuchMedia + default: + return storage.Thumbnail{}, fmt.Errorf("read a thumbnail of %s/%s: %w", origin, mediaID, err) + } +} + +const upsertThumbnailSQL = ` + INSERT INTO media_thumbnails + (media_origin, media_id, width, height, method, content_type, + file_size, sha256, created_at_ms) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT (media_origin, media_id, width, height, method) DO UPDATE SET + content_type = excluded.content_type, + file_size = excluded.file_size, + sha256 = excluded.sha256, + created_at_ms = excluded.created_at_ms` + +// StoreThumbnail implements storage.MediaStore. +func (s *Media) StoreThumbnail(ctx context.Context, t storage.Thumbnail) error { + if _, err := s.db.ExecContext(ctx, s.q(upsertThumbnailSQL), + t.Origin, t.MediaID, t.Width, t.Height, t.Method, t.ContentType, + t.Size, t.SHA256, t.CreatedAtMS); err != nil { + return fmt.Errorf("store a thumbnail of %s/%s: %w", t.Origin, t.MediaID, err) + } + return nil +} diff --git a/server/internal/storage/sqlstore/sqlstore.go b/server/internal/storage/sqlstore/sqlstore.go index 32003ec..7f9cce1 100644 --- a/server/internal/storage/sqlstore/sqlstore.go +++ b/server/internal/storage/sqlstore/sqlstore.go @@ -60,6 +60,7 @@ type Store struct { rooms *Rooms sync *Sync fed *Federation + media *Media } // New builds a Store. It does not touch the database. @@ -79,6 +80,7 @@ func New(opts Options) (*Store, error) { rooms: NewRooms(opts.DB, opts.Rebind, opts.IsUniqueViolation), sync: NewSync(opts.DB, opts.Rebind, opts.IsUniqueViolation), fed: NewFederation(opts.DB, opts.Rebind), + media: NewMedia(opts.DB, opts.Rebind, opts.IsUniqueViolation), }, nil } @@ -109,6 +111,9 @@ func (s *Store) Sync() storage.SyncStore { return s.sync } // Federation implements storage.Database. func (s *Store) Federation() storage.FederationStore { return s.fed } +// Media implements storage.Database. +func (s *Store) Media() storage.MediaStore { return s.media } + // Migrate implements storage.Database. func (s *Store) Migrate(ctx context.Context) error { ms, err := s.migrations() diff --git a/server/internal/storage/storage.go b/server/internal/storage/storage.go index d0f1743..6128887 100644 --- a/server/internal/storage/storage.go +++ b/server/internal/storage/storage.go @@ -75,6 +75,10 @@ type Database interface { // outbound queue and its backoff, and the inbound transaction log. Federation() FederationStore + // Media is the mediaapi store: the metadata of uploaded and cached + // remote media, and of their thumbnails. + Media() MediaStore + // Close releases the connection pool. Close() error } diff --git a/server/test/complement/entrypoint.sh b/server/test/complement/entrypoint.sh index de6656c..05798cb 100755 --- a/server/test/complement/entrypoint.sh +++ b/server/test/complement/entrypoint.sh @@ -17,6 +17,7 @@ export WARP_PRIVATE_KEY="$DATA_DIR/signing.key" export WARP_DB_DRIVER=sqlite export WARP_DB_CONNECTION_STRING="file:$DATA_DIR/warp.db?_pragma=journal_mode(WAL)&_pragma=busy_timeout(10000)&_pragma=foreign_keys(1)" export WARP_DB_AUTO_MIGRATE=true +export WARP_MEDIA_BASE_PATH="$DATA_DIR/media" export WARP_LISTEN_ADDRESS=":8008" diff --git a/server/test/complement/run.sh b/server/test/complement/run.sh index 24ddde3..4c72bcc 100755 --- a/server/test/complement/run.sh +++ b/server/test/complement/run.sh @@ -27,8 +27,11 @@ REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" # stream tests: /sync in all its forms, /messages, /context, filters, # typing, receipts and to-device messaging. What is still out is anything # needing a later phase — the notification counts of the push rules -# (Phase 8) and the device lists of the keyserver (Phase 7). -CSAPI_TESTS='TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData' +# (Phase 8) and the device lists of the keyserver (Phase 7). Phase 6 added +# the media tests: uploads, async uploads, downloads with their file names +# and thumbnails. The remote-media tests are in the federation set, which +# TestRemote.* already covers. +CSAPI_TESTS='TestMedia.*|TestContent.*|TestLocalPngThumbnail|TestAsyncUpload.*|TestUploadLimit|TestRegistration|TestLogin|TestDeviceManagement|TestRoomCreate|TestRoomState|TestRoomAlias|TestRoomForget|TestMembership.*|TestRedaction.*|TestJoinRoom.*|TestLeaveRoom.*|TestKickRoom.*|TestBanRoom.*|TestPublicRoom.*|TestSync.*|TestMessages.*|TestContext.*|TestFilter.*|TestTyping.*|TestReceipt.*|TestToDevice.*|TestRoomCreationReportsEventsToMyself|TestLazyLoading.*|TestWriteMDirectAccountData' # federation: Phase 5. Joins, leaves and invites across servers, keys, # transactions, missing events, server ACLs and the queries. Partial-state diff --git a/warp.yaml.example b/warp.yaml.example index 95f2ece..ec76274 100644 --- a/warp.yaml.example +++ b/warp.yaml.example @@ -125,6 +125,12 @@ client_api: sync: burst: 100 interval: 20ms + # Media downloads and thumbnails. A client opening a room asks for + # every member's avatar at once, so this is as loose as sync. Uploads + # are in the default bucket. + media: + burst: 100 + interval: 20ms sync: # Presence — who is online, and the status message they set. Off by @@ -178,6 +184,42 @@ federation: # private federation with an internal CA. WARP_FEDERATION_TLS_CA_FILE tls_ca_file: "" +media: + # Where uploaded files, and files fetched from other servers, are kept. + # Created on the first upload. WARP_MEDIA_BASE_PATH + base_path: ./media_store + + # The largest upload, and the largest file fetched from another server + # (50 MiB). Clients are told this in /media/config. + # WARP_MEDIA_MAX_UPLOAD_BYTES + max_upload_bytes: 52428800 + + # The largest image, in pixels, decoded to make a thumbnail. Decoding + # costs four bytes a pixel however small the file is. + # WARP_MEDIA_MAX_THUMBNAIL_PIXELS + max_thumbnail_pixels: 32000000 + + # The thumbnails this server makes. A request gets the smallest size here + # that is at least as large as what it asked for. "crop" fills the box + # exactly; "scale" fits inside it and keeps the aspect ratio. + thumbnail_sizes: + - {width: 32, height: 32, method: crop} + - {width: 96, height: 96, method: crop} + - {width: 320, height: 240, method: scale} + - {width: 640, height: 480, method: scale} + - {width: 800, height: 600, method: scale} + + # How long an ID reserved with POST /media/v1/create waits for its + # content, and how many of those one user may hold. + pending_upload_expiry: 24h # WARP_MEDIA_PENDING_UPLOAD_EXPIRY + max_pending_uploads: 10 # WARP_MEDIA_MAX_PENDING_UPLOADS + + # Keep the deprecated /_matrix/media/v3/download and /thumbnail + # endpoints, which need no access token: anybody who learns an mxc:// + # URI can read the file through them. Only for clients older than + # Matrix 1.11. WARP_MEDIA_LEGACY_UNAUTHENTICATED_DOWNLOADS + legacy_unauthenticated_downloads: false + registration: # Off by default. Make the first account with # `warp create-account --username you --admin`, then turn registration