mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-17 03:50:51 +02:00
* mini: quieter startup with a docker-compose-style progress board Replaces noisy startup/shutdown logs with a single in-place progress table on a TTY (or one line per state change off-TTY). Each component renders as `pending -> starting -> ready` during startup and `stopping -> stopped` during shutdown, with elapsed time on transition. Also folds in a few cleanups uncovered while making this readable: - route the admin.go startup prints through glog so quietMiniLogs() filters them under mini but standalone weed admin still shows them - generate a dev SSE-S3 KEK + passphrase on first run via WEED_S3_SSE_KEK and WEED_S3_SSE_KEK_PASSPHRASE env vars (viper.Set has a nested-key conflict between s3.sse.kek and s3.sse.kek.passphrase); persisted under the data folder so restarts reuse the same key - demote worker/master gRPC Recv 'context canceled' to V(1); those are the normal shutdown signal, not Errors/Warnings - drop the 'Optimized Settings' block and the 'credentials loaded from environment variables' message from the welcome banner - only show the credentials setup hints when no S3 identities exist (new s3api.HasAnyIdentity accessor backed by an atomic.Bool) - use S3_BUCKET in the credentials hint so it pairs with AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY - reorder running-services list to master / volume / filer / webdav / s3 / iceberg / admin * mini: refuse in-memory-only SSE-S3 dev keys; surface admin serve errors loadOrCreateMiniHexSecret returns "" when os.WriteFile fails, so SSE-S3 won't encrypt data under a KEK that the next restart can't reproduce (which would orphan whatever was written this run). The caller already treats "" as "skip setting WEED_S3_SSE_* env vars", so SSE-S3 and IAM just stay disabled for this run. startAdminServer's serve goroutine used to only log ListenAndServe failures, so a bind error left the caller blocked on ctx.Done() with no listener. Forward the error through a buffered channel and select on it alongside ctx.Done(). * ci(s3-proxy-signature): match weed mini's new progress-board ready line The readiness probe grepped for "S3 (gateway|service).*(started|ready)", which matched weed mini's old "S3 service is ready at ..." line. Mini now emits " S3 ready (Xs)" from its progress board, so the old pattern misses and the test timed out at the 30-second wait. Widen the alternation to also accept "S3\s+ready". The curl HEAD fallback already covers any remaining cases.
125 lines
4.0 KiB
YAML
125 lines
4.0 KiB
YAML
name: "S3 Proxy Signature Tests"
|
|
|
|
on:
|
|
push:
|
|
branches: [ master ]
|
|
pull_request:
|
|
branches: [ master ]
|
|
|
|
concurrency:
|
|
group: ${{ github.head_ref || github.ref }}/s3-proxy-signature-tests
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
proxy-signature-tests:
|
|
name: S3 Proxy Signature Verification Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 15
|
|
steps:
|
|
- name: Check out code into the Go module directory
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go 1.x
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v4
|
|
|
|
- name: Build SeaweedFS binary for Linux
|
|
run: |
|
|
set -x
|
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -buildvcs=false -v -o test/s3/proxy_signature/weed ./weed
|
|
|
|
- name: Run S3 Proxy Signature Tests
|
|
timeout-minutes: 10
|
|
working-directory: test/s3/proxy_signature
|
|
run: |
|
|
set -x
|
|
echo "Starting Docker Compose services..."
|
|
docker compose up -d --build
|
|
|
|
# Check if containers are running
|
|
echo "Checking container status..."
|
|
docker compose ps
|
|
|
|
# Wait for services to be ready
|
|
echo "Waiting for nginx proxy to be ready..."
|
|
PROXY_READY=0
|
|
for i in $(seq 1 30); do
|
|
if curl -s http://localhost:9000/ > /dev/null 2>&1; then
|
|
echo "Proxy is ready"
|
|
PROXY_READY=1
|
|
break
|
|
fi
|
|
echo "Waiting for proxy... ($i/30)"
|
|
sleep 1
|
|
done
|
|
if [ $PROXY_READY -eq 0 ]; then
|
|
echo "ERROR: Proxy failed to become ready after 30 seconds"
|
|
echo "Docker compose logs:"
|
|
docker compose logs --no-color || true
|
|
exit 1
|
|
fi
|
|
|
|
# Wait for SeaweedFS to be ready
|
|
echo "Waiting for SeaweedFS S3 gateway to be ready via proxy..."
|
|
S3_READY=0
|
|
for i in $(seq 1 30); do
|
|
# Check logs first for the readiness line. weed mini's progress
|
|
# board prints " S3 ready (Xs)"; older builds and the
|
|
# standalone S3 binary log "S3 (gateway|service) ... ready".
|
|
if docker compose logs seaweedfs 2>&1 | grep -qE "S3 (gateway|service).*(started|ready)|S3[[:space:]]+ready"; then
|
|
echo "SeaweedFS S3 gateway is ready"
|
|
S3_READY=1
|
|
break
|
|
fi
|
|
# Fallback: check headers via proxy (which is already ready)
|
|
if curl -s -I http://localhost:9000/ | grep -qi "SeaweedFS"; then
|
|
echo "SeaweedFS S3 gateway is responding via proxy"
|
|
S3_READY=1
|
|
break
|
|
fi
|
|
echo "Waiting for S3 gateway... ($i/30)"
|
|
sleep 1
|
|
done
|
|
if [ $S3_READY -eq 0 ]; then
|
|
echo "ERROR: SeaweedFS S3 gateway failed to become ready after 30 seconds"
|
|
echo "Latest seaweedfs logs:"
|
|
docker compose logs --no-color --tail 20 seaweedfs || true
|
|
exit 1
|
|
fi
|
|
|
|
# Run the test script inside AWS CLI container
|
|
echo "Running test script..."
|
|
docker run --rm --network host \
|
|
--entrypoint bash \
|
|
amazon/aws-cli:latest \
|
|
-c "$(cat test.sh)"
|
|
|
|
TEST_RESULT=$?
|
|
|
|
# Cleanup
|
|
docker compose down
|
|
|
|
exit $TEST_RESULT
|
|
|
|
- name: Cleanup on failure
|
|
if: failure()
|
|
working-directory: test/s3/proxy_signature
|
|
run: |
|
|
echo "Cleaning up Docker containers..."
|
|
ls -al weed || true
|
|
ldd weed || true
|
|
echo "Docker compose logs:"
|
|
docker compose logs --no-color || true
|
|
echo "Container status before cleanup:"
|
|
docker ps -a
|
|
echo "Stopping services..."
|
|
docker compose down || true
|