Files
seaweedfs/.github/workflows/s3-proxy-signature-tests.yml
T
Chris Lu 0c2576c3d0 ci: route Docker Hub pulls through a mirror to cut registry timeouts (#9904)
* ci(s3tables): route Docker Hub pulls through mirror, drop unused buildx

The integration jobs set up docker/setup-buildx-action only to docker
pull/run images; the buildx bootstrap pulls moby/buildkit from
registry-1.docker.io, which times out and fails the whole job before any
test runs. These jobs never docker build with buildx, so the setup is
pure overhead and an extra registry dependency.

Replace it with a daemon registry-mirror pointing at mirror.gcr.io (a
pull-through cache for Docker Hub) and retry the pre-pulls a few times.
That removes the buildkit pull entirely and routes the rest through the
cache, with graceful fallback to Docker Hub on a miss.

* ci: route Docker Hub through mirror in remaining docker test workflows

Same registry-1.docker.io timeout fix for the other integration jobs.

s3-spark only docker pulls/runs an image, so drop the vestigial buildx
setup and pull through the mirror with retries, matching s3-tables.

kafka-quicktest, s3-proxy-signature, e2e and postgres build/compose and
genuinely need buildx (e2e/postgres export a local layer cache, which the
default driver can't), so keep it and just configure the mirror first —
that way even the moby/buildkit bootstrap pull is served from the cache.

Left samba/pjdfstest alone: they build-push to a local registry and pull
from localhost, so buildx is required and there's no Docker Hub runtime
pull to mirror.
2026-06-09 17:12:42 -07:00

130 lines
4.2 KiB
YAML

name: "S3 Proxy Signature Tests"
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
concurrency:
group: ${{ github.head_ref || github.ref }}/s3-proxy-signature-tests
cancel-in-progress: true
permissions:
contents: read
jobs:
proxy-signature-tests:
name: S3 Proxy Signature Verification Tests
runs-on: ubuntu-22.04
timeout-minutes: 15
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v6
- name: Set up Go 1.x
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Configure Docker Hub mirror
run: |
echo '{"registry-mirrors": ["https://mirror.gcr.io"]}' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build SeaweedFS binary for Linux
run: |
set -x
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -buildvcs=false -v -o test/s3/proxy_signature/weed ./weed
- name: Run S3 Proxy Signature Tests
timeout-minutes: 10
working-directory: test/s3/proxy_signature
run: |
set -x
echo "Starting Docker Compose services..."
docker compose up -d --build
# Check if containers are running
echo "Checking container status..."
docker compose ps
# Wait for services to be ready
echo "Waiting for nginx proxy to be ready..."
PROXY_READY=0
for i in $(seq 1 30); do
if curl -s http://localhost:9000/ > /dev/null 2>&1; then
echo "Proxy is ready"
PROXY_READY=1
break
fi
echo "Waiting for proxy... ($i/30)"
sleep 1
done
if [ $PROXY_READY -eq 0 ]; then
echo "ERROR: Proxy failed to become ready after 30 seconds"
echo "Docker compose logs:"
docker compose logs --no-color || true
exit 1
fi
# Wait for SeaweedFS to be ready
echo "Waiting for SeaweedFS S3 gateway to be ready via proxy..."
S3_READY=0
for i in $(seq 1 30); do
# Check logs first for the readiness line. weed mini's progress
# board prints " S3 ready (Xs)"; older builds and the
# standalone S3 binary log "S3 (gateway|service) ... ready".
if docker compose logs seaweedfs 2>&1 | grep -qE "S3 (gateway|service).*(started|ready)|S3[[:space:]]+ready"; then
echo "SeaweedFS S3 gateway is ready"
S3_READY=1
break
fi
# Fallback: check headers via proxy (which is already ready)
if curl -s -I http://localhost:9000/ | grep -qi "SeaweedFS"; then
echo "SeaweedFS S3 gateway is responding via proxy"
S3_READY=1
break
fi
echo "Waiting for S3 gateway... ($i/30)"
sleep 1
done
if [ $S3_READY -eq 0 ]; then
echo "ERROR: SeaweedFS S3 gateway failed to become ready after 30 seconds"
echo "Latest seaweedfs logs:"
docker compose logs --no-color --tail 20 seaweedfs || true
exit 1
fi
# Run the test script inside AWS CLI container
echo "Running test script..."
docker run --rm --network host \
--entrypoint bash \
amazon/aws-cli:latest \
-c "$(cat test.sh)"
TEST_RESULT=$?
# Cleanup
docker compose down
exit $TEST_RESULT
- name: Cleanup on failure
if: failure()
working-directory: test/s3/proxy_signature
run: |
echo "Cleaning up Docker containers..."
ls -al weed || true
ldd weed || true
echo "Docker compose logs:"
docker compose logs --no-color || true
echo "Container status before cleanup:"
docker ps -a
echo "Stopping services..."
docker compose down || true