Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-25 06:20:42 +01:00
Code Issues Packages Projects Releases Wiki Activity

ci: clean up stale FUSE mounts before running tests (#12020)

Browse Source 
* ci(s3tables): tolerate stale FUSE mounts in disk diagnostics

* ci(s3tables): bound df diagnostics with timeout, explain once

A stale FUSE mount can also hang statfs, not just fail it, so wrap df in
timeout 15; one comment at the first site covers all sixteen.

* ci: tolerate stale FUSE mounts in df diagnostics elsewhere

The same self-hosted runners serve the other workflow files; a bare
df -h there can stall or fail a job the same way.

* ci: unmount disconnected FUSE test mounts before jobs start

* ci: skip FUSE cleanup when the variant checkout is skipped

* ci: reclaim the crashed run's test dir once its stale FUSE mount is detached

* ci: stop tracking python bytecode for the FUSE cleanup action

The helper's __pycache__ slipped into the tree; ignore it so local
test runs cannot re-add it.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run the FUSE cleanup action's unit tests on action changes

test_cleanup.py was never wired into CI, so a regression in cleanup.py
would ship silently. A path-filtered workflow runs it on every change
under .github/actions/.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: match every FUSE test mount layout in the cleanup action

The candidates list only matched seaweedfs_fuse_test_*/mount, so stale
mounts left by fuse_failover and fuse_p2p runs (mountN leaves under
seaweedfs_fuse_* dirs) and the posix-compliance workflow's fixed
/tmp/seaweedfs/mount were never reclaimed and could still wedge df.

* ci: run the FUSE mount cleanup in the fuse_failover and fuse_p2p workflows

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Chris LuandDevin authored and GitHub committed 2026-10-24 07:07:55 -07:00
1 parent 18f9d9a0b6
commit 34c7bd03ad
19 files changed
+401 -32
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/action-tests.yml
+30
View File
@@ -0,0 +1,30 @@
name: "actions: unit tests"
on:
push:
branches: [ master ]
paths:
- '.github/actions/**'
- '.github/workflows/action-tests.yml'
pull_request:
branches: [ master ]
paths:
- '.github/actions/**'
- '.github/workflows/action-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
cleanup-stale-fuse-mounts:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Run cleanup action unit tests
run: python3 .github/actions/cleanup-stale-fuse-mounts/test_cleanup.py
.github/workflows/container_latest.yml
+5 -2
View File
@@ -170,10 +170,13 @@ jobs:
uses: actions/checkout@v7
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Free Disk Space
run: |
echo "Available disk space before cleanup:"
df -h
timeout 15 df -h || true
# Remove pre-installed tools
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
# Clean package managers
@@ -185,7 +188,7 @@ jobs:
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
echo "Available disk space after cleanup:"
df -h
timeout 15 df -h || true
- name: Configure variant
id: config
.github/workflows/container_release_unified.yml
+5 -1
View File
@@ -180,6 +180,10 @@ jobs:
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Download pre-built Rust binaries
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/download-artifact@v8
@@ -219,7 +223,7 @@ jobs:
sudo docker system prune -af --volumes
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
df -h
timeout 15 df -h || true
- name: Docker meta
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
.github/workflows/fuse-failover.yml
+3
View File
@@ -42,6 +42,9 @@ jobs:
with:
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
.github/workflows/fuse-p2p-integration.yml
+3
View File
@@ -41,6 +41,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
.github/workflows/posix-compliance.yml
+4 -1
View File
@@ -58,6 +58,9 @@ jobs:
with:
fetch-depth: 1
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v7
with:
@@ -393,7 +396,7 @@ jobs:
echo "" >> reports/debug/system_info.txt
echo "Disk Space" >> reports/debug/system_info.txt
df -h >> reports/debug/system_info.txt
timeout 15 df -h >> reports/debug/system_info.txt || true
echo "" >> reports/debug/system_info.txt
echo "Memory Information" >> reports/debug/system_info.txt
.github/workflows/s3-filer-group-tests.yml
+4 -1
View File
@@ -32,6 +32,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -51,7 +54,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Filer Group Tests ==="
# Run tests with automatic server management
.github/workflows/s3-go-tests.yml
+16 -4
View File
@@ -35,6 +35,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -54,7 +57,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h . /tmp
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
@@ -201,6 +204,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -220,7 +226,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h . /tmp
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
@@ -339,6 +345,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -358,7 +367,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h . /tmp
timeout 15 df -h || true
echo "=== Starting Tests ==="
make test-with-server
@@ -398,6 +407,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -417,7 +429,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h . /tmp
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
.github/workflows/s3-iam-tests.yml
+4 -1
View File
@@ -91,6 +91,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -111,7 +114,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 IAM Integration Tests (${{ matrix.test-type }}) ==="
# Set WEED_BINARY to use the installed version
.github/workflows/s3-keycloak-tests.yml
+4 -1
View File
@@ -37,6 +37,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -57,7 +60,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Keycloak Integration Tests ==="
# Set WEED_BINARY to use the installed version
.github/workflows/s3-policy-tests.yml
+8 -2
View File
@@ -89,6 +89,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -108,7 +111,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Policy Variables Integration Tests ==="
@@ -199,6 +202,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -218,7 +224,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Policy Enforcement Tests (${{ matrix.test-case }}) ==="
.github/workflows/s3-spark-tests.yml
+4 -1
View File
@@ -27,6 +27,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -57,7 +60,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Spark Integration Tests ==="
go test -v -timeout 30m . 2>&1 | tee test-output.log || {
.github/workflows/s3-sse-tests.yml
+4 -1
View File
@@ -46,6 +46,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -65,7 +68,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting SSE Tests ==="
# Run tests with automatic server management
.github/workflows/s3-tables-tests.yml
+65 -16
View File
@@ -30,6 +30,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -53,7 +56,8 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
# A stale FUSE mount can hang or fail df; keep diagnostics best effort.
timeout 15 df -h || true
echo "=== Starting S3 Tables Tests ==="
# Run S3 Tables integration tests
@@ -91,6 +95,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -115,7 +122,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Iceberg Catalog Tests ==="
# Run Iceberg catalog integration tests using Makefile (handles build)
@@ -153,6 +160,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -186,7 +196,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Trino Iceberg Catalog Tests ==="
# Run Trino + Iceberg catalog integration tests
@@ -224,6 +234,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -258,7 +271,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Dremio Iceberg Catalog Tests ==="
@@ -298,6 +311,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -332,7 +348,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Doris Iceberg Catalog Tests ==="
@@ -382,6 +398,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -418,7 +437,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting ClickHouse Iceberg Catalog Tests ==="
@@ -470,6 +489,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -507,7 +529,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting OLake Iceberg Catalog Tests ==="
@@ -570,6 +592,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -602,7 +627,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Polaris Tests ==="
go test -v -timeout 20m ./test/s3tables/polaris 2>&1 | tee test/s3tables/polaris/test-output.log || {
@@ -639,6 +664,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -672,7 +700,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Spark Iceberg Catalog Tests ==="
# Run Spark + Iceberg catalog integration tests
@@ -710,6 +738,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -744,7 +775,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting RisingWave Catalog Tests ==="
# Run RisingWave catalog integration tests
@@ -782,6 +813,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -840,7 +874,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting STS Integration Tests ==="
# Run STS integration tests
@@ -878,6 +912,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -901,7 +938,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Lakekeeper Integration Tests ==="
# Run Lakekeeper integration tests
@@ -939,6 +976,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -973,7 +1013,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Unity Catalog Tests ==="
@@ -1013,6 +1053,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -1046,7 +1089,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting LanceDB Namespace Tests ==="
@@ -1157,6 +1200,9 @@ jobs:
# so do not leave a token in the checkout for it to pick up.
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -1191,7 +1237,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting DuckDB Lance Tests ==="
@@ -1233,6 +1279,9 @@ jobs:
# so do not leave a token in the checkout for it to pick up.
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -1266,7 +1315,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Spark Lance Namespace Tests ==="
.github/workflows/tus-tests.yml
+4 -1
View File
@@ -31,6 +31,9 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
@@ -50,7 +53,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting TUS Tests ==="
# Run tests with automatic server management