tajniak81andClaude Opus 5
fe1e314df9
`weed server -s3` was never one thing. Master, volume, filer and the S3 gateway ran as four goroutines under one process, on one volume, sharing one fate. Splitting them into four containers changes nothing a client can see — the same bucket answers on the same port — but it makes three things possible that were not: the SeaweedFS admin UI, which wants a cluster to look at; a restart or an upgrade of one role without the others; and, eventually, a second volume server somewhere else. A fifth container carries the panel itself. The single-process files stay exactly as they were. These are `.split.` twins beside them, four in all, one per folder per shape, each with the .env example of the same name that both READMEs already promise. Identities are the part that could not simply be copied across. SeaweedFS picks its credentials from one source, in order: an -s3.config file, the filer's IAM store, then AWS_ACCESS_KEY_ID and its secret — and a higher source replaces a lower one rather than adding to it. The existing files use the env pair, which is fine precisely because nothing else writes identities there. Hand somebody a panel that can, and the first user they create lands in the filer's store, the store outranks the environment, and PocketBase's key stops existing — with the first failed upload as the notification. So the gateway here is started with no config file and no AWS_* at all, and the init container seeds PocketBase's identity into the filer's store instead: the same store the panel writes. One source of truth, PocketBase's key sitting in Object Store → Users beside every other, keys minted there picked up without a restart, and a rotated PB_S3_SECRET re-applied in place on the next boot rather than added as a second identity. That seeding is now allowed to fail. The bucket-create it grew out of was best-effort — `|| true`, on the reasoning that the API Server's own S3 check would report a gateway that was genuinely unreachable. That reasoning does not survive the change: a gateway whose IAM store is empty does not refuse anyone, it serves everyone, and the bucket would be wide open rather than unreachable. So the step ends by grepping the configuration back for the access key, the gateway waits on it completing successfully, and a seed that did not land stops the stack instead of opening it. The prod files publish the gateway and the panel, both on loopback, and nothing else. Port 8080 on the volume server hands out file content by file id with no authentication of any kind — the S3 credentials have no bearing on it — so publishing it would publish every attachment in the stack, and the panel shows what that port and the master's would. The panel's own password is required rather than defaulted, because weed serves it with authentication switched off entirely when it is empty, and a page that mints bucket credentials is the bucket. It is passed as WEED_ADMIN_PASSWORD rather than a flag so it stays off the process command line, and SEAWEED_ADMIN_BIND is the knob a remote host needs, named after PB_BIND and API_BIND for the same reason. Master, volume and filer share one /data mount rather than taking three of their own. That is precisely the layout `weed server -dir=/data` writes — the master's raft state, the volume's .dat and .idx, the filer's filerldb2, no two of them naming the same file — so a stack can move between the single-process file and its twin in either direction with nothing to migrate. A second volume server would need its own, and the files say so where somebody would go looking. The dev files map the volume server to 8081 on the host: 8080 there is already the API Server, and in the all-in-one it is the API Server inside the image. Unexercised: written on a machine without Docker, so none of the four has been brought up. Every flag, health path and env name was read out of the pinned 4.45 source rather than recalled — -mdir, -volumeSizeLimitMB, -defaultStoreDir, -max, admin's -master and -dataDir and WEED_ADMIN_*, the filer's and gateway's /healthz, the panel's unauthenticated /health — and the four files were parsed, interpolated against their examples, and checked for duplicate host ports. A `docker compose config` on the target host is still the first thing to run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
331 lines
16 KiB
YAML
331 lines
16 KiB
YAML
name: drivervault-aio
|
|
|
|
# Production all-in-one, with SeaweedFS split into its four roles — pulls the
|
|
# prebuilt image from the registry instead of building. Self-contained: one
|
|
# file, no overlays. Everything an operator needs to set lives in .env.
|
|
#
|
|
# 1. cp .env.prod.seaweedfs.split.example .env (then edit it)
|
|
# 2. docker compose -f docker-compose.prod.seaweedfs.split.yml pull
|
|
# 3. docker compose -f docker-compose.prod.seaweedfs.split.yml up -d
|
|
#
|
|
# This is docker-compose.prod.seaweedfs.yml with the storage layer taken apart.
|
|
# `weed server -s3` runs master, volume, filer and gateway as goroutines in one
|
|
# process; here each is its own container, plus the SeaweedFS admin UI. What
|
|
# that buys:
|
|
#
|
|
# • the admin UI (weed admin) — a cluster view, and Object Store → Users,
|
|
# where S3 identities are created and revoked without touching a file;
|
|
# • per-role restart, upgrade and Prometheus metrics;
|
|
# • room to add a second volume server later, on this host or another.
|
|
#
|
|
# What it costs: five containers beside the all-in-one instead of one, five
|
|
# healthchecks to keep the boot order honest, and one more port worth binding
|
|
# carefully. If none of the above is wanted, use
|
|
# docker-compose.prod.seaweedfs.yml — the S3 behaviour is identical.
|
|
#
|
|
# None of them run inside the all-in-one image, for the same reason the single
|
|
# gateway does not: keeping the object store in that image, on the volume the
|
|
# files are being moved off, would defeat the point and would mean rebuilding.
|
|
#
|
|
# The on-disk layout is deliberately the same as the single-process file's:
|
|
# master, volume and filer share one /data mount, exactly as `weed server -dir`
|
|
# lays it out (master raft state, volume .dat/.idx, the filer's filerldb2/ — no
|
|
# filename overlap). So the two files are interchangeable on the same
|
|
# SEAWEED_DATA, with no migration either way. A *second* volume server would
|
|
# need its own.
|
|
#
|
|
# Only the S3 gateway and the admin UI publish a host port, both on loopback,
|
|
# the way the single-gateway file publishes the S3 port. Master, volume and
|
|
# filer are reachable over the compose network, through the admin UI, or with
|
|
# `docker compose exec` — the volume server in particular serves file content by
|
|
# id with no authentication at all, so it has no business on a public interface.
|
|
#
|
|
# Before turning this on for a stack that already has uploads: PocketBase does
|
|
# NOT copy existing files into the bucket. See README.md.
|
|
#
|
|
# On first boot PocketBase upserts the superuser from PB_ADMIN_*, and the API
|
|
# Server creates any missing collections and the DriverVault super-admin from
|
|
# DRIVERVAULT_SUPERADMIN_*. Both steps are idempotent.
|
|
|
|
services:
|
|
# --- SeaweedFS: master -----------------------------------------------------
|
|
# Keeps the volume/topology metadata and hands out file ids. -ip is the name
|
|
# the other roles are told to reach it by, so it must be the service name and
|
|
# not the container IP the process would otherwise detect.
|
|
seaweedfs-master:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-master
|
|
restart: unless-stopped
|
|
command: >
|
|
master -ip=seaweedfs-master -ip.bind=0.0.0.0 -mdir=/data
|
|
-volumeSizeLimitMB=1024 -metricsPort=9324
|
|
volumes:
|
|
# Named volume by default; set SEAWEED_DATA to a host path in .env for a
|
|
# bind mount, exactly as PB_DATA works. Back it up alongside PB_DATA —
|
|
# from here on the attachments live here, not in the database volume.
|
|
- "${SEAWEED_DATA:-seaweed_data}:/data"
|
|
# No published port: the master UI is one of the pages the admin UI serves.
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:9333/cluster/status || exit 1"]
|
|
interval: 10s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 15s
|
|
|
|
# --- SeaweedFS: volume server ---------------------------------------------
|
|
# Where the bytes actually land. -max=0 lets it size itself from free disk
|
|
# rather than the default cap of 8 volumes.
|
|
seaweedfs-volume:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-volume
|
|
restart: unless-stopped
|
|
command: >
|
|
volume -master=seaweedfs-master:9333 -ip=seaweedfs-volume -ip.bind=0.0.0.0
|
|
-port=8080 -dir=/data -max=0 -metricsPort=9325
|
|
depends_on:
|
|
seaweedfs-master:
|
|
condition: service_healthy
|
|
volumes:
|
|
- "${SEAWEED_DATA:-seaweed_data}:/data"
|
|
# No published port, and this one is not an oversight: 8080 serves file
|
|
# content by file id with NO authentication — the S3 credentials do not
|
|
# apply to it. Publishing it would publish every attachment in the stack.
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8080/healthz || exit 1"]
|
|
interval: 10s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 15s
|
|
|
|
# --- SeaweedFS: filer ------------------------------------------------------
|
|
# Gives the flat volume store a directory tree — buckets, object keys — and
|
|
# holds the S3 identities the admin UI writes. -defaultStoreDir is where its
|
|
# embedded leveldb goes; without it that would be the container's working
|
|
# directory, and the identities would not survive a recreate.
|
|
seaweedfs-filer:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-filer
|
|
restart: unless-stopped
|
|
command: >
|
|
filer -master=seaweedfs-master:9333 -ip=seaweedfs-filer -ip.bind=0.0.0.0
|
|
-port=8888 -defaultStoreDir=/data -metricsPort=9326
|
|
depends_on:
|
|
seaweedfs-volume:
|
|
condition: service_healthy
|
|
volumes:
|
|
- "${SEAWEED_DATA:-seaweed_data}:/data"
|
|
# No published port. The filer's gRPC side (8888 + 10000) carries the IAM
|
|
# service that mints S3 credentials; keep both ends of it on the compose
|
|
# network.
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8888/healthz || exit 1"]
|
|
interval: 10s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 15s
|
|
|
|
# --- SeaweedFS: bucket and identity seed ----------------------------------
|
|
# Runs once and exits, before the gateway starts. Two jobs:
|
|
#
|
|
# 1. create the bucket — PocketBase never issues a CreateBucket of its own;
|
|
# 2. write PocketBase's S3 identity into the filer's IAM store.
|
|
#
|
|
# (2) is why this stack does not set AWS_ACCESS_KEY_ID on the gateway, the way
|
|
# docker-compose.prod.seaweedfs.yml does. Those env vars are the *lowest*
|
|
# priority credential source in SeaweedFS: they are read only while the filer's
|
|
# store is empty, so the first identity added in the admin UI would silently
|
|
# displace them and lock PocketBase out. Seeding the store the admin UI itself
|
|
# writes leaves one source of truth, and the key PocketBase uses appears under
|
|
# Object Store → Users like any other.
|
|
#
|
|
# Both commands update in place, so every later boot re-applies the values from
|
|
# .env and changes nothing else — which is also how a rotated PB_S3_SECRET
|
|
# reaches the gateway.
|
|
#
|
|
# The closing grep is the gate: an empty IAM store means the gateway would come
|
|
# up in its allow-anyone default, so this fails loudly instead and the gateway
|
|
# below never starts. No `|| true` here, deliberately.
|
|
seaweedfs-init:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-init
|
|
restart: "no"
|
|
depends_on:
|
|
seaweedfs-filer:
|
|
condition: service_healthy
|
|
environment:
|
|
# Passed as env and expanded by the shell inside the container, so the
|
|
# secret stays out of the container's argv.
|
|
PB_S3_BUCKET: "${PB_S3_BUCKET:-drivervault}"
|
|
PB_S3_ACCESS_KEY: "${PB_S3_ACCESS_KEY:?set PB_S3_ACCESS_KEY in .env}"
|
|
PB_S3_SECRET: "${PB_S3_SECRET:?set PB_S3_SECRET in .env}"
|
|
entrypoint: ["/bin/sh", "-c"]
|
|
command:
|
|
- |
|
|
set -e
|
|
printf '%s\n' \
|
|
"s3.bucket.create -name $$PB_S3_BUCKET" \
|
|
"s3.configure -user drivervault -access_key $$PB_S3_ACCESS_KEY -secret_key $$PB_S3_SECRET -actions Admin -apply" \
|
|
| weed shell -master=seaweedfs-master:9333 -filer=seaweedfs-filer:8888
|
|
echo "s3.configure" \
|
|
| weed shell -master=seaweedfs-master:9333 -filer=seaweedfs-filer:8888 \
|
|
| grep -q "$$PB_S3_ACCESS_KEY"
|
|
|
|
# --- SeaweedFS: S3 gateway -------------------------------------------------
|
|
# The endpoint PocketBase talks to. No -config file: with only -filer given,
|
|
# credentials come from the filer's IAM store, which is what lets the admin UI
|
|
# add and revoke identities without a restart. A config file would take
|
|
# priority over that store and make the admin UI's users inert.
|
|
seaweedfs-s3:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-s3
|
|
restart: unless-stopped
|
|
command: >
|
|
s3 -filer=seaweedfs-filer:8888 -ip.bind=0.0.0.0 -port=8333
|
|
-metricsPort=9327
|
|
depends_on:
|
|
seaweedfs-filer:
|
|
condition: service_healthy
|
|
# Never serve before an identity exists — see seaweedfs-init above.
|
|
seaweedfs-init:
|
|
condition: service_completed_successfully
|
|
ports:
|
|
# Loopback only: the stack reaches the gateway over the compose network,
|
|
# so this is here for `aws s3 ls --endpoint-url http://127.0.0.1:8333` and
|
|
# nothing else. Set SEAWEED_S3_BIND=0.0.0.0 to expose it, and mean it.
|
|
- "${SEAWEED_S3_BIND:-127.0.0.1}:${SEAWEED_S3_PORT:-8333}:8333"
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8333/healthz || exit 1"]
|
|
interval: 10s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 15s
|
|
|
|
# --- SeaweedFS: admin UI ---------------------------------------------------
|
|
# Cluster topology, volumes, buckets, maintenance tasks, and Object Store →
|
|
# Users, where S3 access keys are minted and revoked. It finds the filer
|
|
# through the master, so -master is all it needs.
|
|
#
|
|
# Bound to loopback by default — the same call SEAWEED_S3_BIND makes above,
|
|
# for the same reason: this is storage plumbing, not one of the app's own
|
|
# panels. On a remote host that means unreachable, so set
|
|
# SEAWEED_ADMIN_BIND=0.0.0.0 and put it behind a reverse proxy.
|
|
#
|
|
# An unauthenticated panel that can mint credentials for the bucket *is* the
|
|
# bucket, so the password is required rather than defaulted — weed leaves auth
|
|
# off entirely when it is empty. It is read from WEED_ADMIN_* rather than a
|
|
# flag, which keeps it off the process command line. -dataDir persists the
|
|
# session key and the maintenance-task settings.
|
|
seaweedfs-admin:
|
|
image: "${SEAWEED_IMAGE:-chrislusf/seaweedfs:4.45}"
|
|
container_name: drivervault-aio-seaweedfs-admin
|
|
restart: unless-stopped
|
|
command: >
|
|
admin -port=23646 -master=seaweedfs-master:9333 -dataDir=/data
|
|
-metricsPort=9328
|
|
depends_on:
|
|
seaweedfs-master:
|
|
condition: service_healthy
|
|
environment:
|
|
WEED_ADMIN_USER: "${SEAWEED_ADMIN_USER:-admin}"
|
|
WEED_ADMIN_PASSWORD: "${SEAWEED_ADMIN_PASSWORD:?set SEAWEED_ADMIN_PASSWORD in .env}"
|
|
# Optional view-only login. weed ignores it unless the admin password
|
|
# above is set, which it is.
|
|
WEED_ADMIN_READONLY_USER: "${SEAWEED_ADMIN_READONLY_USER:-}"
|
|
WEED_ADMIN_READONLY_PASSWORD: "${SEAWEED_ADMIN_READONLY_PASSWORD:-}"
|
|
volumes:
|
|
# Its own small volume: session key and maintenance state, no object data.
|
|
- "${SEAWEED_ADMIN_DATA:-seaweed_admin}:/data"
|
|
ports:
|
|
- "${SEAWEED_ADMIN_BIND:-127.0.0.1}:${SEAWEED_ADMIN_PORT:-23646}:23646"
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:23646/health || exit 1"]
|
|
interval: 10s
|
|
timeout: 3s
|
|
retries: 12
|
|
start_period: 15s
|
|
|
|
drivervault:
|
|
image: "${AIO_IMAGE:-10.2.1.10:5500/admin/drivervault-aio:latest}"
|
|
container_name: drivervault-aio
|
|
restart: unless-stopped
|
|
depends_on:
|
|
# PocketBase — inside this container — is the process that reads and
|
|
# writes the objects, so the gateway has to be serving first, and the
|
|
# bucket has to exist before the bootstrap points PocketBase at it.
|
|
seaweedfs-s3:
|
|
condition: service_healthy
|
|
seaweedfs-init:
|
|
condition: service_completed_successfully
|
|
environment:
|
|
# Superuser (also used by the API Server to authenticate to PocketBase).
|
|
PB_ADMIN_EMAIL: "${PB_ADMIN_EMAIL:?set PB_ADMIN_EMAIL in .env}"
|
|
PB_ADMIN_PASSWORD: "${PB_ADMIN_PASSWORD:?set PB_ADMIN_PASSWORD in .env}"
|
|
# Match CORS to the web origin (only used if a browser calls the API directly).
|
|
CORS_ALLOW_ORIGINS: "${CORS_ALLOW_ORIGINS:-http://localhost:8090}"
|
|
# Probed by the panel status page. nginx serves the Web App on port 80
|
|
# inside this container, so plain localhost:8090 would never answer.
|
|
# Override WEBAPP_URL in .env to make a change from the panel's Web App
|
|
# screen permanent; the panel alone only holds it for the container's life.
|
|
WEBAPP_URL: "${WEBAPP_URL:-http://127.0.0.1:80}"
|
|
# Schema + super-admin bootstrap (idempotent). Leave this ON: a release can
|
|
# add collections or fields the server needs, and a stack that skips the
|
|
# bootstrap never gets them. The API Server self-heals exactly one thing —
|
|
# app_settings, the collection holding the plugin settings, which it
|
|
# creates on demand because it cannot serve the plugin panel without it.
|
|
# Every other schema change still depends on this flag. Turn it off only
|
|
# for a database you know already matches the release.
|
|
PB_BOOTSTRAP: "${PB_BOOTSTRAP:-true}"
|
|
DRIVERVAULT_SUPERADMIN_EMAIL: "${DRIVERVAULT_SUPERADMIN_EMAIL:-}"
|
|
DRIVERVAULT_SUPERADMIN_PASSWORD: "${DRIVERVAULT_SUPERADMIN_PASSWORD:-}"
|
|
DRIVERVAULT_SUPERADMIN_NAME: "${DRIVERVAULT_SUPERADMIN_NAME:-Administrator}"
|
|
# OCPP charger control (Anker Solix). This image serves plain HTTP, so a
|
|
# charger can only connect when TLS is terminated in front of it (set
|
|
# OCPP_PUBLIC_URL to the public wss:// base) — or, on a trusted network,
|
|
# with OCPP_REQUIRE_TLS=false.
|
|
OCPP_REQUIRE_TLS: "${OCPP_REQUIRE_TLS:-true}"
|
|
OCPP_PUBLIC_URL: "${OCPP_PUBLIC_URL:-}"
|
|
# --- File storage --------------------------------------------------
|
|
# supervisord passes these through to the API Server, whose bootstrap
|
|
# writes them into PocketBase's
|
|
# settings on every boot, idempotently. Only record files move — scans,
|
|
# receipts, invoices, part photos. The database and PocketBase's own
|
|
# backups stay on PB_DATA.
|
|
PB_S3_ENABLED: "true"
|
|
PB_S3_BUCKET: "${PB_S3_BUCKET:-drivervault}"
|
|
# The gateway's service name: a server-to-server call inside the compose
|
|
# network.
|
|
PB_S3_ENDPOINT: "http://seaweedfs-s3:8333"
|
|
# SeaweedFS ignores the region; PocketBase insists on having one.
|
|
PB_S3_REGION: "${PB_S3_REGION:-us-east-1}"
|
|
PB_S3_ACCESS_KEY: "${PB_S3_ACCESS_KEY}"
|
|
PB_S3_SECRET: "${PB_S3_SECRET}"
|
|
# Path style, because a self-hosted gateway has no per-bucket DNS.
|
|
PB_S3_FORCE_PATH_STYLE: "true"
|
|
ports:
|
|
- "${WEB_PORT:-8090}:80" # Web App
|
|
- "${PB_PORT:-8070}:8070" # PocketBase admin UI / API
|
|
- "${API_PORT:-8080}:8080" # API Server + panel (root /) + /ocpp/{serial}
|
|
volumes:
|
|
# The only volume — named by default; set PB_DATA to a host path in .env
|
|
# for a bind mount. The API Server keeps no state on disk, so everything
|
|
# it owns (plugin settings included) is in here.
|
|
- "${PB_DATA:-pb_data}:/pb/pb_data"
|
|
healthcheck:
|
|
# All three processes must answer. Declared here as well as in the image so
|
|
# the check is visible, and works against an older pulled image.
|
|
test: ["CMD-SHELL", "wget -qO- http://127.0.0.1:8070/api/health >/dev/null && wget -qO- http://127.0.0.1:8080/healthz >/dev/null && wget -qO- http://127.0.0.1:80/healthz >/dev/null || exit 1"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
retries: 3
|
|
start_period: 60s
|
|
|
|
volumes:
|
|
pb_data:
|
|
# Shared by master, volume and filer — the same layout `weed server -dir`
|
|
# writes, so this file and docker-compose.prod.seaweedfs.yml can swap places
|
|
# on it.
|
|
seaweed_data:
|
|
# The admin UI's own session key and maintenance-task state. Small, and no
|
|
# part of the object store.
|
|
seaweed_admin:
|