docs: add Image Gateway page for weed image

chrislusf committed 2026-10-05 04:50:46 -07:00
1 parent 873fbfb9ac
commit cd1f3dfa31
2 files changed
+161

No files matched your search

+160
@@ -0,0 +1,160 @@
# On-demand image gateway
`weed image` is an optional public image endpoint. Original images remain in
SeaweedFS. On a cache miss, a separate imgproxy service resizes and encodes the
image; results enter a bounded in-process cache. Subsequent requests recheck
anonymous source access before reusing results. Cache eviction and process
restarts do not affect stored objects. No S3 objects, Filer entries, or
application file records are created for processed images.
```mermaid
flowchart LR
Browser --> CDN
CDN --> Gateway[weed image]
Gateway -->|Check source access and revision on every request| S3[SeaweedFS S3]
Gateway -->|Cache miss| imgproxy
imgproxy -->|Anonymous source read| S3
Gateway --> Cache[Bounded memory cache]
```
The gateway runs on a separate port and does not change existing `weed s3`,
Filer, or Volume services.
## Request interface
```text
/image.png?x-oss-process=image/resize,w_640/quality,Q_85/format,webp
/image.png?x-oss-process=image/resize,w_240,h_240,m_lfit,limit_1/format,webp
/image.png?x-oss-process=image/format,jpg/quality,Q_85
```
This implements a limited subset of OSS image processing parameters, rather
than full Aliyun OSS compatibility:
- `resize` accepts `w`, `h`, `m_lfit`, and `limit_1`. It preserves aspect ratio
and never enlarges the source. One dimension determines the other; two
dimensions specify a bounding box. Any unspecified axis uses `maxDimension`
as its bound, including format-only requests; neither output axis can exceed it.
- `quality,Q_1` through `quality,Q_100` specify absolute quality, defaulting to
85. Aliyun's relative quality `q` has no equivalent here and returns 400.
- `format` accepts `jpg`, `jpeg`, `png`, and `webp`, defaulting to WebP.
- Repeated parameters, unknown operations, cropping, watermarks, animation
processing, automatic format negotiation, and other OSS operations are
unsupported. PNG output is lossless; quality mainly affects JPEG/WebP.
Animated sources use imgproxy's default first-frame behavior.
- A single `versionId` may select a specific source version. Other query
parameters are rejected.
- GET, HEAD, Range, and conditional requests apply to the returned
representation. Processed images have independent SHA-256 ETags and byte
lengths; validators do not depend on the source modification date. Without a
processing parameter the gateway reads the original image, including single
byte ranges.
- Original GET and HEAD responses reject unsupported or malformed media types,
including HTML and all `image/*+xml` subtypes. Non-XML image types and
octet-stream objects retain their original `Content-Type`, including
parameters. This prevents uploaded executable documents from being served
under the gateway's origin. A 304 may omit `Content-Type`; an explicit
unsupported type is still rejected so conditional responses cannot
reclassify previously cached bytes.
## Running the gateway
Start a separate imgproxy service, then run:
```sh
weed image \
-source=http://s3:8333/public-bucket \
-imgproxy=http://imgproxy:8080 \
-ip.bind=0.0.0.0 -port=8334 \
-cacheCapacityMB=64 -concurrency=8 \
-maxSourceMB=25 -maxResultMB=10 -maxDimension=4096 -timeout=15s
```
## Options
| Option | Default | Description |
|--------|---------|-------------|
| `-source` | (required) | Fixed anonymous S3 HTTP(S) source URL, optionally with a bucket path |
| `-imgproxy` | (required) | Separate imgproxy HTTP(S) service URL |
| `-ip.bind` | 127.0.0.1 | Listen address |
| `-port` | 8334 | HTTP port |
| `-concurrency` | 8 | Maximum concurrent requests and encoding jobs; excess requests return 429 |
| `-maxDimension` | 4096 | Maximum output dimension |
| `-cacheCapacityMB` | 64 | Processed image memory cache in MiB; 0 disables caching |
| `-maxSourceMB` | 25 | Maximum source image size in MiB |
| `-maxResultMB` | 10 | Maximum processed image size in MiB |
| `-timeout` | 15s | Separate timeout budgets for source metadata, shared encoding, and client writes |
`source` fixes the source HTTP(S) URL, optionally including a bucket path or a
bucket domain pointing to S3. With `http://s3:8333/public-bucket`, a client
request for `/a/b.png` reads `http://s3:8333/public-bucket/a/b.png`. Both
backend URLs must omit credentials, queries, and fragments. The imgproxy URL
must be a root URL without a path prefix. Dot segments and backslashes,
including repeatedly escaped forms, are rejected to prevent backend path
normalization from escaping a fixed bucket prefix. The gateway and imgproxy
must reach the same source.
## imgproxy configuration
Configure imgproxy limits and isolate the encoder with container or process
resource limits. Suggested imgproxy settings:
```text
IMGPROXY_WORKERS=2
IMGPROXY_REQUESTS_QUEUE_SIZE=8
IMGPROXY_MAX_SRC_FILE_SIZE=26214400
IMGPROXY_MAX_SRC_RESOLUTION=25
IMGPROXY_MAX_RESULT_DIMENSION=4096
IMGPROXY_MAX_ANIMATION_FRAMES=1
IMGPROXY_MAX_REDIRECTS=0
IMGPROXY_ALLOWED_PROCESSING_OPTIONS=rs,q,f
IMGPROXY_ALLOWED_SOURCES=http://s3:8333/public-bucket/
```
These variables apply to imgproxy 3.x. In imgproxy 4.x, some source security
limits move to separate source configuration; configure equivalent
restrictions according to that version's documentation. File size limits do
not replace pixel and memory limits. The gateway does not decode images and
cannot enforce encoder-side limits.
Set hexadecimal `IMGPROXY_KEY` and `IMGPROXY_SALT` in both services to sign
backend requests. The gateway supports one key/salt pair with full SHA-256
signatures; imgproxy must use the default 32-byte signature length. Unsigned
requests are only appropriate on an isolated trusted network. Keep signing
material out of command lines, public configuration, and logs.
## Access and caching
The endpoint only accepts anonymous public reads. It rejects client
Authorization, S3 signature parameters, and `x-amz-*` request headers. Browser
cookies are ignored and never forwarded to backends. The gateway does not
create administrative credentials or signed source URLs for private objects.
Use the existing S3 endpoint for private images.
Every derived request performs an anonymous source HEAD, including cache hits,
HEAD, and 304 requests. The encoder uses anonymous GET, so the source must
apply the same public-read policy to HEAD and GET. A source 403 or 404 rejects
derived requests even if cached bytes remain. Detected source changes return
409.
Concurrent misses for the same result share one encoding job. A cancelled
waiter does not cancel work needed by other waiters. Source metadata checks,
shared encoding, and client writes each receive a full timeout budget. Errors
are never cached. The cache is limited by bytes and 1024 entries, defaults to
64 MiB, and can be disabled with `-cacheCapacityMB=0`. Each instance has its
own cache, empty after restart.
## CDN notes
Responses default to `Cache-Control: no-cache`, allowing downstream storage
with mandatory revalidation so source access is checked on every request. A
CDN must preserve `x-oss-process` and `versionId` and include the complete
query in its cache key. If you explicitly set a CDN TTL for permanently public
immutable objects, CDN hits bypass the gateway. Revocation or deletion then
requires a CDN purge, otherwise access changes only take effect after TTL
expiry. Errors use `no-store`.
Configure HTTPS, CORS, and external rate limits in the existing reverse proxy
or CDN. Route only public image GET/HEAD requests to this gateway. Uploads,
listings, signed reads, and other S3 operations continue using the existing
S3 endpoint.
+1
@@ -105,6 +105,7 @@
* [[S3 API Audit log]]
* [[S3 Nginx Proxy]]
* [[Docker Compose for S3]]
* [[Image Gateway]]
### S3 Table Bucket
* [[S3 Table Bucket]]