mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-09 21:38:19 +01:00
docs: add Image Gateway page for weed image
1 parent
873fbfb9ac
commit
cd1f3dfa31
2 files changed
+161
No files matched your search
+160
@@ -0,0 +1,160 @@
|
||||
# On-demand image gateway
|
||||
|
||||
`weed image` is an optional public image endpoint. Original images remain in
|
||||
SeaweedFS. On a cache miss, a separate imgproxy service resizes and encodes the
|
||||
image; results enter a bounded in-process cache. Subsequent requests recheck
|
||||
anonymous source access before reusing results. Cache eviction and process
|
||||
restarts do not affect stored objects. No S3 objects, Filer entries, or
|
||||
application file records are created for processed images.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Browser --> CDN
|
||||
CDN --> Gateway[weed image]
|
||||
Gateway -->|Check source access and revision on every request| S3[SeaweedFS S3]
|
||||
Gateway -->|Cache miss| imgproxy
|
||||
imgproxy -->|Anonymous source read| S3
|
||||
Gateway --> Cache[Bounded memory cache]
|
||||
```
|
||||
|
||||
The gateway runs on a separate port and does not change existing `weed s3`,
|
||||
Filer, or Volume services.
|
||||
|
||||
## Request interface
|
||||
|
||||
```text
|
||||
/image.png?x-oss-process=image/resize,w_640/quality,Q_85/format,webp
|
||||
/image.png?x-oss-process=image/resize,w_240,h_240,m_lfit,limit_1/format,webp
|
||||
/image.png?x-oss-process=image/format,jpg/quality,Q_85
|
||||
```
|
||||
|
||||
This implements a limited subset of OSS image processing parameters, rather
|
||||
than full Aliyun OSS compatibility:
|
||||
|
||||
- `resize` accepts `w`, `h`, `m_lfit`, and `limit_1`. It preserves aspect ratio
|
||||
and never enlarges the source. One dimension determines the other; two
|
||||
dimensions specify a bounding box. Any unspecified axis uses `maxDimension`
|
||||
as its bound, including format-only requests; neither output axis can exceed it.
|
||||
- `quality,Q_1` through `quality,Q_100` specify absolute quality, defaulting to
|
||||
85. Aliyun's relative quality `q` has no equivalent here and returns 400.
|
||||
- `format` accepts `jpg`, `jpeg`, `png`, and `webp`, defaulting to WebP.
|
||||
- Repeated parameters, unknown operations, cropping, watermarks, animation
|
||||
processing, automatic format negotiation, and other OSS operations are
|
||||
unsupported. PNG output is lossless; quality mainly affects JPEG/WebP.
|
||||
Animated sources use imgproxy's default first-frame behavior.
|
||||
- A single `versionId` may select a specific source version. Other query
|
||||
parameters are rejected.
|
||||
- GET, HEAD, Range, and conditional requests apply to the returned
|
||||
representation. Processed images have independent SHA-256 ETags and byte
|
||||
lengths; validators do not depend on the source modification date. Without a
|
||||
processing parameter the gateway reads the original image, including single
|
||||
byte ranges.
|
||||
- Original GET and HEAD responses reject unsupported or malformed media types,
|
||||
including HTML and all `image/*+xml` subtypes. Non-XML image types and
|
||||
octet-stream objects retain their original `Content-Type`, including
|
||||
parameters. This prevents uploaded executable documents from being served
|
||||
under the gateway's origin. A 304 may omit `Content-Type`; an explicit
|
||||
unsupported type is still rejected so conditional responses cannot
|
||||
reclassify previously cached bytes.
|
||||
|
||||
## Running the gateway
|
||||
|
||||
Start a separate imgproxy service, then run:
|
||||
|
||||
```sh
|
||||
weed image \
|
||||
-source=http://s3:8333/public-bucket \
|
||||
-imgproxy=http://imgproxy:8080 \
|
||||
-ip.bind=0.0.0.0 -port=8334 \
|
||||
-cacheCapacityMB=64 -concurrency=8 \
|
||||
-maxSourceMB=25 -maxResultMB=10 -maxDimension=4096 -timeout=15s
|
||||
```
|
||||
|
||||
## Options
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `-source` | (required) | Fixed anonymous S3 HTTP(S) source URL, optionally with a bucket path |
|
||||
| `-imgproxy` | (required) | Separate imgproxy HTTP(S) service URL |
|
||||
| `-ip.bind` | 127.0.0.1 | Listen address |
|
||||
| `-port` | 8334 | HTTP port |
|
||||
| `-concurrency` | 8 | Maximum concurrent requests and encoding jobs; excess requests return 429 |
|
||||
| `-maxDimension` | 4096 | Maximum output dimension |
|
||||
| `-cacheCapacityMB` | 64 | Processed image memory cache in MiB; 0 disables caching |
|
||||
| `-maxSourceMB` | 25 | Maximum source image size in MiB |
|
||||
| `-maxResultMB` | 10 | Maximum processed image size in MiB |
|
||||
| `-timeout` | 15s | Separate timeout budgets for source metadata, shared encoding, and client writes |
|
||||
|
||||
`source` fixes the source HTTP(S) URL, optionally including a bucket path or a
|
||||
bucket domain pointing to S3. With `http://s3:8333/public-bucket`, a client
|
||||
request for `/a/b.png` reads `http://s3:8333/public-bucket/a/b.png`. Both
|
||||
backend URLs must omit credentials, queries, and fragments. The imgproxy URL
|
||||
must be a root URL without a path prefix. Dot segments and backslashes,
|
||||
including repeatedly escaped forms, are rejected to prevent backend path
|
||||
normalization from escaping a fixed bucket prefix. The gateway and imgproxy
|
||||
must reach the same source.
|
||||
|
||||
## imgproxy configuration
|
||||
|
||||
Configure imgproxy limits and isolate the encoder with container or process
|
||||
resource limits. Suggested imgproxy settings:
|
||||
|
||||
```text
|
||||
IMGPROXY_WORKERS=2
|
||||
IMGPROXY_REQUESTS_QUEUE_SIZE=8
|
||||
IMGPROXY_MAX_SRC_FILE_SIZE=26214400
|
||||
IMGPROXY_MAX_SRC_RESOLUTION=25
|
||||
IMGPROXY_MAX_RESULT_DIMENSION=4096
|
||||
IMGPROXY_MAX_ANIMATION_FRAMES=1
|
||||
IMGPROXY_MAX_REDIRECTS=0
|
||||
IMGPROXY_ALLOWED_PROCESSING_OPTIONS=rs,q,f
|
||||
IMGPROXY_ALLOWED_SOURCES=http://s3:8333/public-bucket/
|
||||
```
|
||||
|
||||
These variables apply to imgproxy 3.x. In imgproxy 4.x, some source security
|
||||
limits move to separate source configuration; configure equivalent
|
||||
restrictions according to that version's documentation. File size limits do
|
||||
not replace pixel and memory limits. The gateway does not decode images and
|
||||
cannot enforce encoder-side limits.
|
||||
|
||||
Set hexadecimal `IMGPROXY_KEY` and `IMGPROXY_SALT` in both services to sign
|
||||
backend requests. The gateway supports one key/salt pair with full SHA-256
|
||||
signatures; imgproxy must use the default 32-byte signature length. Unsigned
|
||||
requests are only appropriate on an isolated trusted network. Keep signing
|
||||
material out of command lines, public configuration, and logs.
|
||||
|
||||
## Access and caching
|
||||
|
||||
The endpoint only accepts anonymous public reads. It rejects client
|
||||
Authorization, S3 signature parameters, and `x-amz-*` request headers. Browser
|
||||
cookies are ignored and never forwarded to backends. The gateway does not
|
||||
create administrative credentials or signed source URLs for private objects.
|
||||
Use the existing S3 endpoint for private images.
|
||||
|
||||
Every derived request performs an anonymous source HEAD, including cache hits,
|
||||
HEAD, and 304 requests. The encoder uses anonymous GET, so the source must
|
||||
apply the same public-read policy to HEAD and GET. A source 403 or 404 rejects
|
||||
derived requests even if cached bytes remain. Detected source changes return
|
||||
409.
|
||||
|
||||
Concurrent misses for the same result share one encoding job. A cancelled
|
||||
waiter does not cancel work needed by other waiters. Source metadata checks,
|
||||
shared encoding, and client writes each receive a full timeout budget. Errors
|
||||
are never cached. The cache is limited by bytes and 1024 entries, defaults to
|
||||
64 MiB, and can be disabled with `-cacheCapacityMB=0`. Each instance has its
|
||||
own cache, empty after restart.
|
||||
|
||||
## CDN notes
|
||||
|
||||
Responses default to `Cache-Control: no-cache`, allowing downstream storage
|
||||
with mandatory revalidation so source access is checked on every request. A
|
||||
CDN must preserve `x-oss-process` and `versionId` and include the complete
|
||||
query in its cache key. If you explicitly set a CDN TTL for permanently public
|
||||
immutable objects, CDN hits bypass the gateway. Revocation or deletion then
|
||||
requires a CDN purge, otherwise access changes only take effect after TTL
|
||||
expiry. Errors use `no-store`.
|
||||
|
||||
Configure HTTPS, CORS, and external rate limits in the existing reverse proxy
|
||||
or CDN. Route only public image GET/HEAD requests to this gateway. Uploads,
|
||||
listings, signed reads, and other S3 operations continue using the existing
|
||||
S3 endpoint.
|
||||
+1
@@ -105,6 +105,7 @@
|
||||
* [[S3 API Audit log]]
|
||||
* [[S3 Nginx Proxy]]
|
||||
* [[Docker Compose for S3]]
|
||||
* [[Image Gateway]]
|
||||
|
||||
### S3 Table Bucket
|
||||
* [[S3 Table Bucket]]
|
||||
|
||||
Reference in new issue
Block a user