CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
CI / Admin panel (push) Canceled after 0s
CI / Web client (push) Canceled after 0s
CI / Desktop client (ubuntu-24.04) (push) Canceled after 0s
CI / Desktop client (windows-latest) (push) Canceled after 0s
The pocketbase driver no longer embeds PocketBase: Warp keeps its data as PocketBase collections on a server of its own and reaches it only through the REST API, the way the postgres driver reaches PostgreSQL. database.connection_string is the server's URL, and Warp signs in as the superuser in WARP_PB_SUPERUSER_EMAIL / WARP_PB_SUPERUSER_PASSWORD (or a WARP_PB_SUPERUSER_FILE of the two). Every SQL table is a superuser-only collection of the same name and columns, created by warp migrate. Reads go through PocketBase's SQL endpoint with the SQL drivers' queries, via a read-only database/sql driver that inlines arguments, moves long ones into warp_params and pages past 1000 rows. Writes are record operations in batch requests under one write lock in the Warp process. Media is a protected file field in warp_media, local or in the S3 bucket Warp writes into PocketBase's settings. Search matches word prefixes with LIKE. The storage tests run against an in-process PocketBase and pass as on SQLite. Docker: the PocketBase stacks get a pocketbase service running tajniak81/pocketbase-docker-root (WARP_PB_IMAGE), with a pb-data volume and its admin UI on loopback; the AIO images run that PocketBase binary as a process beside the homeserver, making up a superuser when none is set. The .env examples, READMEs, docs and warp.yaml.example follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
253 lines
10 KiB
Bash
253 lines
10 KiB
Bash
#!/bin/sh
|
|
# warp-aio: runs the homeserver, the admin panel and the web client in one
|
|
# container, with whatever the image adds: PostgreSQL, or SeaweedFS as the
|
|
# media store.
|
|
#
|
|
# If any process exits, the others are stopped and the container exits, so
|
|
# the restart policy brings the whole set back. SIGTERM stops them in the
|
|
# reverse of the order they started: the clients, the homeserver, then
|
|
# SeaweedFS and PostgreSQL underneath it.
|
|
#
|
|
# With arguments, runs them instead: `docker run warp-aio warp version`.
|
|
#
|
|
# WARP_AIO_ADMIN=false no admin panel
|
|
# WARP_AIO_WEB=false no web client
|
|
# WARP_AIO_EMBEDDED_POSTGRES=true PostgreSQL in the container, data in
|
|
# $PGDATA (/data/postgres); the default
|
|
# in warp-aio-postgresql, the only image
|
|
# that has PostgreSQL
|
|
# WARP_AIO_POCKETBASE=true PocketBase in the container, on port
|
|
# 8070, its pb_data in /data/pb_data; set
|
|
# by the warp-aio-pocketbase* images, the
|
|
# only ones that have it
|
|
# WARP_AIO_SEAWEEDFS=server SeaweedFS as one `weed server -s3`
|
|
# WARP_AIO_SEAWEEDFS=split SeaweedFS as master, volume, filer, S3
|
|
# gateway and admin UI, one process each;
|
|
# set by the warp-aio-pocketbase-seaweedfs
|
|
# images, the only ones with SeaweedFS
|
|
|
|
set -eu
|
|
|
|
if [ "$#" -gt 0 ]; then
|
|
exec "$@"
|
|
fi
|
|
|
|
log() { echo "warp-aio: $*" >&2; }
|
|
|
|
# "name:pid" of every process started, newest first, so shutdown walks it in
|
|
# reverse start order.
|
|
procs=""
|
|
|
|
# run NAME COMMAND...: start COMMAND in the background and track it.
|
|
run() {
|
|
name=$1
|
|
shift
|
|
"$@" &
|
|
procs="$name:$! $procs"
|
|
}
|
|
|
|
shutdown() {
|
|
trap '' TERM INT
|
|
for entry in $procs; do
|
|
name=${entry%%:*}
|
|
pid=${entry#*:}
|
|
# SIGINT is PostgreSQL's fast shutdown: it does not wait for
|
|
# clients, and everything that used it is gone by now.
|
|
if [ "$name" = postgres ]; then sig=INT; else sig=TERM; fi
|
|
kill -"$sig" "$pid" 2>/dev/null || true
|
|
wait "$pid" 2>/dev/null || true
|
|
done
|
|
exit "${1:-0}"
|
|
}
|
|
trap 'log "stopping"; shutdown 0' TERM INT
|
|
|
|
# check: stop everything if any process has exited.
|
|
check() {
|
|
for entry in $procs; do
|
|
if ! kill -0 "${entry#*:}" 2>/dev/null; then
|
|
log "${entry%%:*} exited; stopping the rest"
|
|
shutdown 1
|
|
fi
|
|
done
|
|
}
|
|
|
|
# wait_http NAME URL: wait until URL answers, as long as nothing has died.
|
|
wait_http() {
|
|
until curl -fsS -o /dev/null "$2" 2>/dev/null; do
|
|
check
|
|
log "waiting for $1"
|
|
sleep 1
|
|
done
|
|
}
|
|
|
|
if [ "${WARP_AIO_EMBEDDED_POSTGRES:-false}" = true ]; then
|
|
if ! command -v postgres >/dev/null; then
|
|
log "WARP_AIO_EMBEDDED_POSTGRES=true, but this image has no PostgreSQL; use warp-aio-postgresql"
|
|
exit 1
|
|
fi
|
|
export PGDATA="${PGDATA:-/data/postgres}"
|
|
if [ ! -s "$PGDATA/PG_VERSION" ]; then
|
|
log "initialising PostgreSQL in $PGDATA"
|
|
# Trust is safe here: PostgreSQL listens on no TCP address, only on
|
|
# a socket inside this container.
|
|
initdb -D "$PGDATA" -U warp --auth=trust --encoding=UTF8 --locale=C >/dev/null
|
|
fi
|
|
run postgres postgres -D "$PGDATA" -c listen_addresses='' -c unix_socket_directories=/run/postgresql
|
|
until pg_isready -q -h /run/postgresql -U warp; do
|
|
check
|
|
sleep 1
|
|
done
|
|
if [ "$(psql -h /run/postgresql -U warp -d postgres -tAc "SELECT 1 FROM pg_database WHERE datname = 'warp'")" != 1 ]; then
|
|
createdb -h /run/postgresql -U warp warp
|
|
fi
|
|
fi
|
|
|
|
# SeaweedFS keeps everything under /seaweed/data, in the layout `weed server
|
|
# -dir` writes, which the split roles share as well: switching between the
|
|
# two images needs no migration. Every role advertises 127.0.0.1; the volume
|
|
# server serves file content by id with no authentication at all, so only
|
|
# the S3 gateway and the admin UI are meant to be published. The volume
|
|
# server is on 8081 because 8080 is the web client.
|
|
seaweedfs="${WARP_AIO_SEAWEEDFS:-}"
|
|
if [ -n "$seaweedfs" ]; then
|
|
if ! command -v weed >/dev/null; then
|
|
log "WARP_AIO_SEAWEEDFS=$seaweedfs, but this image has no SeaweedFS; use warp-aio-pocketbase-seaweedfs"
|
|
exit 1
|
|
fi
|
|
# One pair of credentials: the gateway's identity and what the
|
|
# homeserver signs its requests with. An empty pair would leave the
|
|
# gateway in its allow-anyone default, so refuse instead.
|
|
if [ -z "${WARP_PB_S3_ACCESS_KEY:-}" ] || [ -z "${WARP_PB_S3_SECRET:-}" ]; then
|
|
log "WARP_PB_S3_ACCESS_KEY and WARP_PB_S3_SECRET are required; refusing to start an unauthenticated S3 gateway"
|
|
exit 1
|
|
fi
|
|
bucket="${WARP_PB_S3_BUCKET:-warp}"
|
|
mkdir -p /seaweed/data
|
|
fi
|
|
|
|
case "$seaweedfs" in
|
|
"") ;;
|
|
server)
|
|
# Master, volume, filer and gateway as goroutines of one process. With
|
|
# no -s3.config, the gateway takes its single identity from
|
|
# AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, given to it alone.
|
|
run seaweedfs env AWS_ACCESS_KEY_ID="$WARP_PB_S3_ACCESS_KEY" AWS_SECRET_ACCESS_KEY="$WARP_PB_S3_SECRET" \
|
|
weed server -dir=/seaweed/data -ip=127.0.0.1 -ip.bind=0.0.0.0 \
|
|
-volume.port=8081 -volume.max=0 -master.volumeSizeLimitMB=1024 -s3 -s3.port=8333
|
|
wait_http "SeaweedFS S3 gateway" http://127.0.0.1:8333/healthz
|
|
# The homeserver never creates its bucket. Creating one that exists is
|
|
# a no-op.
|
|
echo "s3.bucket.create -name $bucket" | weed shell -master=127.0.0.1:9333 >/dev/null || true
|
|
;;
|
|
split)
|
|
# The admin UI can mint credentials for the bucket, and weed serves it
|
|
# with no authentication when the password is empty.
|
|
if [ -z "${WEED_ADMIN_PASSWORD:-}" ]; then
|
|
log "WEED_ADMIN_PASSWORD is empty; refusing to serve an unauthenticated SeaweedFS admin UI"
|
|
exit 1
|
|
fi
|
|
mkdir -p /seaweed/admin
|
|
run seaweedfs-master weed master -ip=127.0.0.1 -ip.bind=127.0.0.1 -port=9333 \
|
|
-mdir=/seaweed/data -volumeSizeLimitMB=1024 -metricsPort=9324
|
|
wait_http "SeaweedFS master" http://127.0.0.1:9333/cluster/status
|
|
run seaweedfs-volume weed volume -master=127.0.0.1:9333 -ip=127.0.0.1 -ip.bind=127.0.0.1 \
|
|
-port=8081 -dir=/seaweed/data -max=0 -metricsPort=9325
|
|
wait_http "SeaweedFS volume server" http://127.0.0.1:8081/healthz
|
|
run seaweedfs-filer weed filer -master=127.0.0.1:9333 -ip=127.0.0.1 -ip.bind=127.0.0.1 \
|
|
-port=8888 -defaultStoreDir=/seaweed/data -metricsPort=9326
|
|
wait_http "SeaweedFS filer" http://127.0.0.1:8888/healthz
|
|
|
|
# The bucket, and the homeserver's identity in the filer's IAM store,
|
|
# both re-applied on every boot; that is how a rotated secret lands.
|
|
# The store is the one the admin UI writes, so the gateway runs with no
|
|
# -config and no AWS_* variables: those are lower-priority sources that
|
|
# the first identity added in the admin UI would silently displace.
|
|
shell() { weed shell -master=127.0.0.1:9333 -filer=127.0.0.1:8888; }
|
|
printf '%s\n' \
|
|
"s3.bucket.create -name $bucket" \
|
|
"s3.configure -user warp -access_key $WARP_PB_S3_ACCESS_KEY -secret_key $WARP_PB_S3_SECRET -actions Admin -apply" |
|
|
shell >/dev/null || true
|
|
# An empty IAM store would leave the gateway open to anyone.
|
|
if ! echo "s3.configure" | shell 2>/dev/null | grep -q "$WARP_PB_S3_ACCESS_KEY"; then
|
|
log "the homeserver's S3 identity is not in the filer's IAM store; refusing to start the gateway"
|
|
shutdown 1
|
|
fi
|
|
|
|
run seaweedfs-s3 weed s3 -filer=127.0.0.1:8888 -ip.bind=0.0.0.0 -port=8333 -metricsPort=9327
|
|
run seaweedfs-admin weed admin -port=23646 -master=127.0.0.1:9333 -dataDir=/seaweed/admin -metricsPort=9328
|
|
wait_http "SeaweedFS S3 gateway" http://127.0.0.1:8333/healthz
|
|
;;
|
|
*)
|
|
log "WARP_AIO_SEAWEEDFS must be server, split or empty, not $seaweedfs"
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
# An S3 bucket elsewhere: the homeserver starts without checking it, and
|
|
# would only fail on the first upload. Wait for the endpoint to answer at
|
|
# all (any HTTP status will do: without credentials it may well refuse).
|
|
if [ "${WARP_PB_S3_ENABLED:-false}" = true ] && [ -z "$seaweedfs" ] && [ -n "${WARP_PB_S3_ENDPOINT:-}" ]; then
|
|
tries=0
|
|
until curl -s -o /dev/null --max-time 5 "$WARP_PB_S3_ENDPOINT"; do
|
|
tries=$((tries + 1))
|
|
if [ "$tries" -ge 30 ]; then
|
|
log "S3 endpoint $WARP_PB_S3_ENDPOINT does not answer; stopping"
|
|
shutdown 1
|
|
fi
|
|
log "waiting for S3 endpoint $WARP_PB_S3_ENDPOINT"
|
|
sleep 1
|
|
done
|
|
fi
|
|
|
|
# PocketBase is the homeserver's database server. Its superuser is the one
|
|
# the homeserver signs in as; without one in the environment, the first
|
|
# start makes one up and keeps it in the volume, so a bare `docker run`
|
|
# works and the admin UI can still be signed in to. Every start sets the
|
|
# password again, which is how a changed one lands.
|
|
if [ "${WARP_AIO_POCKETBASE:-false}" = true ]; then
|
|
if ! command -v pocketbase >/dev/null; then
|
|
log "WARP_AIO_POCKETBASE=true, but this image has no PocketBase; use warp-aio-pocketbase"
|
|
exit 1
|
|
fi
|
|
# The homeserver reads the same file through WARP_PB_SUPERUSER_FILE, so
|
|
# that `docker exec ... warp create-account` finds the account too.
|
|
superuser="${WARP_PB_SUPERUSER_FILE:-/data/pocketbase-superuser.env}"
|
|
if [ -z "${WARP_PB_SUPERUSER_EMAIL:-}" ] || [ -z "${WARP_PB_SUPERUSER_PASSWORD:-}" ]; then
|
|
if [ ! -f "$superuser" ]; then
|
|
log "no WARP_PB_SUPERUSER_EMAIL and WARP_PB_SUPERUSER_PASSWORD; making up a superuser, kept in $superuser"
|
|
umask 077
|
|
printf 'WARP_PB_SUPERUSER_EMAIL=warp@warp.local\nWARP_PB_SUPERUSER_PASSWORD=%s\n' \
|
|
"$(head -c 24 /dev/urandom | base64 | tr -d '/+=')" >"$superuser"
|
|
fi
|
|
set -a
|
|
. "$superuser"
|
|
set +a
|
|
fi
|
|
pocketbase superuser upsert "$WARP_PB_SUPERUSER_EMAIL" "$WARP_PB_SUPERUSER_PASSWORD" \
|
|
--dir=/data/pb_data >/dev/null
|
|
run pocketbase pocketbase serve --http=0.0.0.0:8070 --dir=/data/pb_data
|
|
wait_http PocketBase http://127.0.0.1:8070/api/health
|
|
fi
|
|
|
|
# The signing key is created on first start and kept in the volume: losing
|
|
# it means other servers no longer recognise this one.
|
|
if [ ! -f "$WARP_PRIVATE_KEY" ]; then
|
|
warp generate-keys --output "$WARP_PRIVATE_KEY"
|
|
fi
|
|
|
|
run warp warp serve
|
|
if [ "${WARP_AIO_ADMIN:-true}" = true ]; then
|
|
run warp-admin warp-admin
|
|
fi
|
|
if [ "${WARP_AIO_WEB:-true}" = true ]; then
|
|
run warp-web warp-web
|
|
fi
|
|
|
|
# Busybox sh has no `wait -n`, so check on the processes every second. The
|
|
# sleep runs in the background so a signal is handled at once.
|
|
while :; do
|
|
check
|
|
sleep 1 &
|
|
wait $! || true
|
|
done
|