imagetools create writes the descriptor it pushed with --metadata-file
(buildx 0.32+, the runners ship 0.36), so the digest no longer comes from
re-resolving the tag even within the same step.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
The job only assembles and signs manifests, so nothing there needs the
source_ref checkout; the local signing action now comes from the same
revision as the workflow file that calls it.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
crane copy and the signature both resolved the tag, which another
publisher could move between the two steps. The index digest is read once,
right after it is created, and the Docker Hub copy and both signatures use
it. The manual latest rebuild gets the same treatment. The actions in these
jobs are pinned to commits, crane to v0.22.0 by checksum, and the sparse
checkout no longer keeps the token.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
These run with registry credentials and the OIDC token that signs under
the repository's identity, so a retargeted tag upstream must not be able
to reach them.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
The merge job signs each variant's multi-arch index on GHCR and Docker Hub
once the tag exists, recursively so the platform images are covered too.
latest re-tags the same manifest and inherits the signature.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
Keyless, by digest, with a verification pass against the calling workflow's
own identity right after signing. Signatures use the .sig tag layout rather
than the OCI-referrer bundle cosign 3 writes by default, since that is what
the verifiers people run today read. Dependabot is pointed at the action so
the cosign-installer pin keeps moving.
Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
-dir=/data fails on macOS, where the root filesystem is read-only, and
on any Linux box without root; -dir=./data is created on the spot.
go install of the weed package is refused because go.mod carries
replace directives, so the install script is the shortcut instead.
Claude-Session: https://claude.ai/code/session_014apMEkkquAtAYp89paTkAT
The Helm values put filer metadata on a claim through filer.data, which
is what the chart reads; enablePVC was rendering a hostPath. Claims use
the cluster default storage class instead of local-path. The AWS CLI
test carries its own credentials, the compose download includes the
Prometheus config the compose file mounts, and the disk-read claim is
per blob, since large files are chunked.
Claude-Session: https://claude.ai/code/session_014apMEkkquAtAYp89paTkAT
Drives the full path: AssumeRoleWithWebIdentity embeds the claims in the
session JWT, AuthenticateJWT restores them, and AuthorizeAction evaluates
ForAnyValue:StringEquals on jwt:groups / jwt:roles per bucket. The mock
OIDC provider now carries token claims through and surfaces roles,
matching the real provider.
The owner index read the raw filer entry off BucketConfig, which no
longer carries one. Cache Crtime alongside IdentityId, and pass the
owner id rather than the entry through the ListBuckets visibility
checks so the scan and granted-bucket paths share one implementation.
An empty or truncated tasks/*.pb file unmarshals into a TaskStateFile
with a nil Task, and protobufToMaintenanceTask dereferenced it
immediately, panicking the whole admin process on startup. Guard the
nil case so the loader logs a warning and skips the bad file.
Drop max-parallel so the 13 per-platform builds run together instead of two
waves of 8 (rocksdb was queuing behind the cap and starting ~8 min late).
Keep cache-to mode=max for rocksdb: its RocksDB static_lib compile is
sha-independent, so it caches across releases and stops being the ~16-min
long-pole that gates the merge fan-in. go-build variants stay mode=min.
Add path filters to workflows that fired on every PR/push regardless
of the diff: CodeQL, go build, the e2e/EC/vacuum/TLS/plugin-worker
integration suites, the Kafka and Postgres gateways, the S3 suites
(Ceph s3tests, s3-go, s3-tables, proxy-signature, https, example,
filer-group), TUS, and the dev binary/container builds. Each scopes
to its subsystem under weed/, its test dir, go.mod/go.sum, and the
workflow file, so docs-, helm-, terraform-, rust- or java-only
changes no longer trigger a full compile-and-test fleet.
Mini auto-sizes a few large volume slots, but the master pre-grows 7
volumes per new collection. Under a filer group each S3 bucket is its
own collection, so the first buckets claimed every slot and later
writes failed to assign a volume. Cap mini's volume_growth copy counts
to 1.
TopicExists and getTopicConfFromCache negative-cached a topic for the full
30s TTL whenever a filer lookup failed for any reason, including timeouts.
A topic created earlier then looked gone until the TTL expired, and the
metadata auto-create path couldn't heal it (CreateTopic rejects an
already-persisted conf), so producers saw UNKNOWN_TOPIC_OR_PARTITION.
Only negative-cache on a definitive ErrNotFound; let transient errors fall
through and retry against the filer.
Random allocation could pick 33646 = admin.port (23646) + GrpcPortOffset.
weed mini reserves that as Admin's gRPC port even when the test only
overrides Master/Filer/S3/Iceberg, so the explicit Filer flag failed
with "reserved for gRPC calculation" and TestRisingWaveIcebergCatalog
flaked. Pre-seed the reserved set with every mini default HTTP port
plus its +10000 offset so a random pick (or its own gRPC offset) cannot
land on a service the caller left at its default.
The maintenance scanner tries to plan EC destinations for every
eligible volume, so clusters that can't place EC logged a warning per
volume every cycle. The min-node gate already skips clusters with fewer
nodes than parity shards; demote the rest to V(2).
thrift v0.23.0 uses math.MaxUint32 as an untyped int constant in
lib/go/thrift/framed_transport.go:206, which overflows int on 32-bit
targets (openbsd/arm, linux/arm, freebsd/arm, netbsd/arm) and breaks
the release binary builds.
Dispatch Tick / Cursor Checkpoint Tick / Engine Refresh / Bootstrap Re-walk
are internal terms — operators tuning the form had to read the descriptions
to guess what each field meant. Renames the visible labels and the section
blurb; underlying field names are unchanged so stored configs still load.
archive.ubuntu.com from GitHub-hosted runners has been Ign:/retrying for
~60s per package, eating the Start SeaweedFS step's 10-min budget before
apt-get install finishes. The host already uses azure.archive.ubuntu.com;
do the same inside Dockerfile.e2e and drop the Retries=5 amplifier.
Also rotate /tmp/.buildx-cache-new over /tmp/.buildx-cache so the apt
layer actually survives across runs, and bump the step to 15 minutes
as a safety margin.
The mini cluster's admin/plugin worker keeps creating files under
admin/plugin/job_types/ for ~1s after subtests finish, while the
previous Stop() only cancelled an unobserved context and slept 500ms.
t.TempDir()'s registered RemoveAll then raced the worker and
intermittently failed with "directory not empty" (CI run 25352039081).
Manage the data dir manually so it is removed only after the mini
goroutine has exited, and wire MiniClusterCtx so cancel actually drains
master/volume/filer/admin/s3/webdav.
The onRetry hook in TestOffsetManagement/ConsumerGroupResumption only fires
after defer reader.Close(), so every dump shows the post-LeaveGroup Empty
state — useless for diagnosing why the second consumer hangs in the join
cycle. Add an onTick callback fired every 1.5s while the reader is still
joined so we can see PreparingRebalance / CompletingRebalance churn,
leader, and member assignments during the 20s attempt window.
Replace the old "server -s3" form with a parallel docker run that
uses the same env vars (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,
S3_BUCKET) as the binary weed mini quick start. Drop the explicit
"mini" subcommand since it is the default CMD in the image.
Remove the "Quick Start with Single Binary" section which duplicated
the binary-download instructions and pushed users toward "weed server".
Fold the go install alternative and the volume-scaling tip into the
weed mini section so there is one canonical Quick Start path.
Surface the table-buckets / Iceberg REST Catalog story in the
introduction and add a Data Lakehouse Features section so readers
landing on the README see that SeaweedFS replaces both the object
store and the metastore in an Iceberg stack.
Without this, when "weed mini" fails to bind its master port within the
45s startup timeout the only diagnostic is the timeout message itself,
making flakes impossible to root-cause. The sibling catalog, catalog_dremio,
and catalog_trino packages already wire stdout/stderr through; bring
catalog_spark in line.
The metadata-action used type=ref,event=tag, which produces no tag on
workflow_dispatch, causing build-push to fail with
"tag is needed when pushing to registry". Add a release_tag input and
build the tag from a RELEASE_TAG env, mirroring container_release_unified.yml.
The WORM suite creates one bucket per test, each backed by ~3 reserved
volumes on the data node. With ~30 tests and the default `weed mini`
volume cap, the data node runs out of slots midway through the run and
every PutObject after that fails with InternalError.
Hook a sweep of every test-prefix bucket into the create helpers so a
panicked or interrupted prior test cannot leak buckets into the next.
The 45m suite alarm fires on TestWritebackCacheStressSmallFiles with no
output from the test, since t.Logf is buffered until the test completes
and the alarm panic skips that flush. Add streaming stderr progress, an
explicit verify-phase budget that t.Fatalf's with a goroutine dump on
overrun, and per-retry/per-failure logging so the next hang shows which
file(s) the mount could not read back.
COMPLIANCE-mode retention leaves objects that BypassGovernanceRetention cannot
clear, so the test's DeleteBucket keeps returning BucketNotEmpty and the
underlying SeaweedFS collection (with its 7 reserved volumes) leaks. After a
few leaks on the single-node `weed mini` server, the master logs "Not enough
data nodes found" and every subsequent PutObject 500s, timing the suite out.
Call the master's /col/delete admin endpoint from deleteBucket so the
collection's volumes are reclaimed even when S3-level cleanup is blocked.
Two more spots that broke under a subdirectory deployment:
- plugin.templ pluginRequest() called fetch(url) with relative API
paths from 14+ callers; wrap once inside the helper so they all
honor window.__BASE_PATH__.
- plugin_lane.templ generated <a href="/plugin/configuration?job=...">
with an absolute path; wrap with basePath() so the link stays
inside the deployment prefix.
Follow-up to a6adf530c.
Plugin lane page fetches API endpoints with raw absolute URLs, breaking
deployments under a subdirectory. Wrap the fetch URL with basePath() so
window.__BASE_PATH__ is honored, matching other admin pages.
Addresses https://github.com/seaweedfs/seaweedfs/issues/9240
PR #9230 attempt 1 hit an intermittent
TestConcurrentFileOperations/ConcurrentReadWrite failure where stat
returned ENOENT for a path all writers had just succeeded against, and
the captured mount.log carried no signal about which layer dropped the
entry because the relevant lookup logged at V(4).
Two diagnostic-only changes (no behavior change on the happy path):
- weed/mount/weedfs.go: in lookupEntry, when filer GetEntry returns
ErrNotFound for a path whose inode is still tracked locally with no
in-flight create or flush, log Warningf with inode + dirtyHandle +
pendingFlush + localCache + dirCached. This surfaces layer-by-layer
state at the moment of the suspicious ENOENT.
- test/fuse_integration/framework_test.go: on AssertFileExists failure,
dump five 100ms-spaced stat retries, a parent ReadDir, and a direct
O_RDONLY open before failing. Triangulates kernel dentry caching vs
mount lookup vs filer state.
TestPosixFileLocking/ConcurrentLockContention failed in CI (run
24857323067) with ENOENT when re-opening the file after all 8 workers
had successfully written and closed. The 20s openWithRetry budget was
exhausted, pointing at a real but unproven metaCache/parent-cache
coherence issue in the mount under bursts of concurrent Release.
Test: hold the initial fd open for the whole subtest; use it for the
post-workers Sync() and the verification read. Workers still exercise
the concurrent-flock invariant and per-record write correctness; the
re-open path is no longer load-bearing. On Eventually failure, dump
ReadDir of the parent, Stat, and a fresh O_RDONLY open so a future
recurrence has state to debug from. Drop the darwin-only ENOENT
t.Skip branches that hid this same flake.
Mount: in weedfs.lookupEntry, when returning ENOENT from the
"parent cached but child missing" branch, log at Warningf instead of
V(4) when the kernel is still tracking this path's inode. That
combination is the smoking-gun signal for cache drift and is rare
enough in normal use not to spam the log.
The memory credential store backs LoadConfiguration with a map, so the
identity order is not stable across a save/load round trip. Indexing
Identities[1] intermittently pointed at the owner identity and produced
a spurious credential leak.
Marp-based markdown deck walking through the three-layer production
topology (masters, volumes, filers + DB, S3 gateways, admin + workers)
plus an erasure-coding note. Uses the object-store-layout diagram on the
overview slide. Makefile renders PDF/HTML/PPTX via marp-cli.
TestPosixFileLocking/FcntlReleaseOnClose was flaky because the
subprocess spawned by startLockHolder occasionally saw ENOENT when
opening a file the parent had just created on the FUSE mount. Retry on
ENOENT (matching the existing openWithRetry pattern used in
testConcurrentLockContention) so the subprocess waits for the mount's
dentry state to propagate before reporting the lock acquire as failed.
AllocateMiniPorts(1) reserved masterPort and masterPort+GrpcPortOffset
by holding listeners open, but closed them on return. The subsequent
AllocatePorts call bound 127.0.0.1:0, so the OS could immediately reuse
the just-released mini gRPC port as a volume port — causing the volume
server to fail at bind time with "address already in use".
Introduce AllocatePortSet(miniCount, regularCount) that holds every
listener open until the full set is chosen, and route the five volume
test cluster builders through it.
createTableBucket ran `weed shell` via exec.Command with no deadline.
When the shell's first command retries on a transient master connection
blip, the trailing `exit` on stdin never gets processed and the
subprocess blocks until the outer 20m `go test` timeout fires — the
surfacing symptom is a flaky 20m panic with no diagnostic output.
Wrap the invocation in exec.CommandContext with a 30s timeout, matching
the existing pattern in test/s3tables/catalog_risingwave/setup_test.go.
The random host port allocated by MustFreeMiniPorts was released before
docker run bound it, occasionally losing the race to another process and
failing with "address already in use". The sidecar already reaches
RisingWave via shared netns (--network container:...), so the host -p
mapping and the corresponding WaitForPort check were unused.
ceph/s3-tests pins lxml without an upper bound. When pip picks a release
whose prebuilt wheel isn't published for Python 3.9 on the runner, it
falls back to sdist and fails without libxml2-dev / libxslt1-dev.
freePort allocated in [20000, 55535], which overlaps the Linux ephemeral
range (32768-60999). The kernel could reuse the chosen port for an
outbound connection between the test closing its listener and weed mini
re-checking availability, causing:
Port allocation failed: port N for Filer (specified by flag
filer.port) is not available on 0.0.0.0 and cannot be used
Narrow the range to [20000, 32000] to stay below the ephemeral floor,
and pass -ip.bind=127.0.0.1 so mini's pre-check runs on the same IP the
test actually reserved the port on.
Multiple master pushes within the same minute produced identical BUILD_TIME
values, causing concurrent workflow runs to race on identically-named release
assets. Upload retries hit 422 already_exists and failed the build. Adding a
concurrency group with cancel-in-progress ensures only the latest dev build
runs at a time, which is fine since only the latest dev artifacts matter.
Trigger the helm release workflow automatically on tag pushes so each
software release also publishes the chart to gh-pages and the OCI
registry at ghcr.io/seaweedfs. workflow_dispatch is kept as a manual
fallback.
Refs #6296
Both workflows trigger on push to master and race to delete assets
from the same dev release. When one deletes assets the other is also
trying to delete, the "Not Found" error fails the cleanup job and
skips all downstream build jobs.
Add continue-on-error to both cleanup steps since the error is
harmless — build steps already use overwrite: true.
Chris Lu