Compare commits

...
Author SHA1 Message Date
github-actions[bot] 153ab668c9 4.51 2026-10-25 19:39:07 +00:00
Chris Lu 6dd6ad512c iam/oidc: drop single-use web-identity tokens and the jti replay cache (#12065)
* iam/oidc: drop single-use web-identity tokens and the jti replay cache

Reverts the single-use semantics added in #11936 (and the #12017
refactor): AssumeRoleWithWebIdentity consumed each web-identity
token's jti into a bounded replay cache, so a second exchange of the
same token was rejected until exp + skew.

AWS STS accepts a web-identity token repeatedly until it expires,
and Kubernetes workload identity depends on that: the kubelet keeps
a projected ServiceAccount token until ~80% of its lifetime, so pod
restarts, co-located containers sharing the token file, repeated CLI
invocations, and pre-rotation SDK refreshes all re-present the same
jti and were refused on 4.50.

The replay machinery also carried a real maintenance cost — a
bounded concurrent cache, fingerprint-keyed entries for jti-less
tokens, adopt/merge handoffs so provider rebuilds could not reopen
consumed tokens, and provider-reuse bookkeeping in the refresh path
that existed only to preserve that state. Removing it restores AWS
semantics and deletes the whole apparatus.

Refs #12063. Alternative to the per-provider opt-out in #12064.

* iam: reuse unchanged OIDC providers on refresh

refreshOIDCProvidersFrom rebuilt every managed provider on any store
change, so an unrelated write discarded each provider's JWKS cache and
an unreachable issuer could then fail exchanges the cached keys would
have allowed. Keep a per-record provider cache keyed on the record
content fingerprint and only rebuild when the record changes.

* iam/oidc: require exp in ValidateToken

Dropping the single-use check also removed the implicit bound on
expiry-less tokens: a signed web-identity token without exp could
mint STS sessions indefinitely. Require the exp claim, matching AWS
AssumeRoleWithWebIdentity which rejects tokens without one.

* iam/oidc: bound idle connections on the provider transport

A provider replaced on refresh drops its http.Transport without a
close; give it an IdleConnTimeout so pooled connections and their
goroutines cannot linger.

* s3api: cover web-identity token reuse over HTTP

Sign one token and exchange that exact string twice, directly covering
the Kubernetes projected-token reuse this change restores.

* test(iam): refresh reuses an unchanged record's provider

Covers the provider-reuse cache: an unrelated record's refresh keeps a
provider's JWKS state, a changed record rebuilds it.
2026-10-25 12:04:59 -07:00
515380c349 s3: answer GET and HEAD of a delete marker as S3 does (#12051)
* s3: answer GET and HEAD of a delete marker as S3 does

A delete marker as the current version now returns 404 NoSuchKey with
x-amz-delete-marker: true and the marker's x-amz-version-id. Asking for the
marker by versionId returns 405 MethodNotAllowed with the same headers and
Last-Modified, instead of 404 NoSuchKey.

* s3: check the tag-conditioned read grant before answering a delete marker

The marker branch returned before the entry-based policy recheck the rest
of the read path performs, so an s3:ExistingObjectTag-conditioned Deny did
not apply to the delete-marker response and its version headers.

* s3: send Allow: DELETE with the delete-marker 405

RFC 9110 15.5.6 requires a 405 answer to list the methods the resource
supports; S3 sends allow: DELETE for a delete marker named by versionId.

* s3: keep delete-marker headers on conditional-read misses

If-Match or If-Unmodified-Since against a current delete marker short-
circuited to a bare NoSuchKey. Carry the resolved marker through the
conditional result so the 404 keeps x-amz-delete-marker and
x-amz-version-id, matching the unconditional read.

* s3: exercise delete-marker responses through GET and HEAD handlers

* s3: read marker version id from the entry before the caller's guess

A delete marker written outside the normal PUT path can miss the
version-id attribute while the v_<id> file name still names it; the
new x-amz-version-id header would then report "null". Reuse
versionIdFromEntry so both the handler and conditional-miss paths
emit the real id.

* s3: answer 405 for versionId=null on a suspended-versioning marker

The null marker lives in .versions while getSpecificObjectVersion only
checks the plain path, so GET/HEAD ?versionId=null answered
NoSuchVersion. On that miss, look up the null marker and answer it
like any other marker named by versionId.

* s3: propagate null-marker lookup failures instead of answering 404

A transient filer error reading .versions/v_null was swallowed as
"no marker", so GET/HEAD answered NoSuchVersion for a live read
failure. NotFound and absent entries still mean no marker; anything
else is InternalError now.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
2026-10-25 12:02:19 -07:00
Chris LuandDevin 9620e7ebfe filer: auto-enable per-path commit records when a peer has its own store (#12062)
* filer: enable per-path commit records only when a peer has its own store

Since seaweedfs#11698 every metadata change writes a per-path commit
record into the filer store (an MpTs key, plus an MpDl key on deletes),
and the records stay after the path is deleted. They order the events a
filer replays from peer filers that keep their own store, since replay
only runs when peerSignature != f.Signature.

Filers that all share one store (one Redis, MySQL, TiKV, ...) never
replay each other's events, so the records are never read, yet each
change still costs one or two extra store writes and the key count grows
with every path ever changed — unbounded growth for in-memory stores.

This turns the records off by default and latches them on the first time
a peer reports a different store signature — no option needed. The latch
is persisted in the store (MpRep) so it survives restarts and converges
filers sharing a store even before they discover the peer themselves.
While off, a filer keeps an MpOff marker so enabling later bumps the
records' generation (MpGen, folded into each record key) past the
unrecorded window: a record older than a change made while off cannot
let an older replayed peer event win. Ordering for unrecorded changes
falls back to the entry's mtime, the same as entries written before the
records existed.

Supersedes the manual option approach in seaweedfs#12060.

* filer: keep delete tombstones while records off, harden enable path

Review on the first commit:

- A delete committed while the records were off left no entry and no
  usable tombstone: after enabling, an older replayed create could
  resurrect the path. Deletes and rename sources are now always recorded
  under the generation-0 MpDl key — they are never fenced by a
  generation bump, and they cannot go stale because every delete is
  recorded. Creates/updates still write nothing while off.

- bumpPathCommitGen could regress MpGen under concurrent bumpers on a
  shared store. It now loops: each pass writes only a generation higher
  than any it has seen, so concurrent bumps converge upward.

- doSubscribeToOneFiler proceeded into replay when
  enablePathCommitRecords failed, running the whole session without
  ordering. It now aborts the subscribe so the reconnect retries with
  the records on.

* filer: bind pending commit-record repairs to their generation

A failed MpTs write kept in metaTsPending could land after a shared-store
generation bump, writing a pre-bump timestamp as a current-generation
record and overriding newer unrecorded state. Each pending change record
now carries the generation it failed under: repairs write it only while
that generation is current, and ordering reads ignore it otherwise.
Pending delete records need no binding — MpDl keys are never fenced.

* filer: never move the local records' generation backward

The converge loop in bumpPathCommitGen seeded from zero and both it and
syncPathCommitGen stored unconditionally: a stale writer could leave the
stored generation below what this filer already adopted, and the next
sync or bump then walked the local value back — orphaning records it
wrote under the higher generation. The bump now seeds from the local
generation and every store only moves it forward.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: name a records generation after the off marker it consumes

Two filers consuming the same off marker each counted one past the
stored generation, so the second could store N+2 after the first
stored N+1 and write records under a key the other could not read
until the next check. The bump now takes the marker's own timestamp —
filers racing on one marker land on one value — a marker older than
the stored generation is cleaned as a leftover, and a stored value
that already passed the bump is adopted instead of counted past. A
newly enabling filer whose marker was already consumed names its own
off window with the current time, so its unrecorded period is still
fenced.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: tell a fresh off marker from one the generation already consumed

A marker whose timestamp reads at or below the stored generation was
dropped as leftover, but a new off period can also read that old — the
off filer's clock lagging, or a host clock moving back — and dropping
it kept records from before that period live, so stale replay could
win.

The stored generation now names the marker it was bumped for: a marker
the store still shows is consumed unless it is that one, however old
it reads, and only the consumed marker itself is cleared without a
bump. The generation trails in the value, so a whole-slice read still
yields it on stores written before the pair was stored. Enabling
fences the filer's own off window with the current time instead of a
marker a sharer may already have consumed.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: cover racing off-marker consumers settling on one generation

* filer: adopt the stored generation, not the consumed marker it names

A filer syncing with no off marker present read the value's first field —
the consumed marker's timestamp — so a restart after a lagging-clock off
period under-counted the generation and kept records the bump dropped.

* filer: re-check the off marker before consuming it

A sync whose marker read ran behind a later off period's consume cycle
still bumped for its stale read: the stored generation already covered a
different marker, so the bump overwrote the consumed name, counted the
generation past records a newer consumption meant to keep, and cleared
whatever marker was live by then. Each pass now re-reads the marker and
only bumps or clears while it is still the one the pass saw.

* filer: keep a failed generation read from resetting the records' generation

The final generation read ignored errors and then stored whatever gen
held — possibly zero — so a transient store failure could hide records
the current generation wrote. Unexpected read errors now return before
the local generation changes.

Also cover a sync whose marker read runs behind another consumer's
delete: it must adopt the stored generation, not bump for the stale read.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: fail a bump that cannot verify the stored generation

A generation read that failed mid-bump was treated as absent: the
write could land below a generation a racing bumper had stored, or
re-store a marker that bumper had already consumed. Read failures
now abort the pass like every other store error on the path.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: cover generation reads that fail instead of returning a value

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: carry the commit timestamp on the entry itself

The per-path MpTs record only exists because a commit timestamp had
nowhere else to live. Stamping commit_ts_ns on the entry before its
store write - and reusing that stamp as the emitted event's TsNs -
makes the entry its own commit record: replay and bootstrap traversal
order on it with zero extra writes for creates and updates.

Deletes still need a durable key since a removed entry cannot carry a
timestamp, so MpDl tombstones stay (one write per delete, growth
bounded by delete history). Entries predating the field fall back to
mtime, matching what peers and old records already assume.

This replaces the MpTs/MpOff/MpGen/MpRep machinery entirely: shared-
store filers write nothing extra, and dedicated-store replication
keeps last-writer-wins ordering plus tombstone coverage for deletes
and ancestor drops.

* filer: keep a directory update replay from tombstoning its subtree

A same-path update wrote a delete record before the delete-and-reinsert,
so a directory metadata change read as a subtree delete: a later peer
event for a child older than the update was then skipped even though
nobody deleted the children, leaving files missing on this filer. Only
file updates take the tombstone now — the re-inserted directory entry's
own commit stamp orders the path itself.

* filer: order unstamped entries by their recorded commit before mtime

Entries written before stamping existed have no CommitTsNs, so the
ordering check fell back to the client-settable mtime and dropped the
MpTs record those versions wrote: an entry committed at event time 300
with mtime 100 lost to a delayed peer event at 200 after upgrade. Read
the legacy record for unstamped entries in localCommitTsNs, CommitTsNs,
and the traverse stamp (which also stopped falling back to mtime,
regressing entry_ts_ns to 0).

* filer: release queued rename event stamps when the rename fails

Each moved entry carried an in-flight stamp its queued event would
adopt at emit time. When the move succeeded but a later child move or
the commit failed, the handler rolled back without emitting the queued
events or releasing their stamps, pinning the delivered and flushed
watermarks below the aborted rename forever. Release every queued
stamp on both failure exits in both rename handlers.

* filer: apply a directory update replay with an in-place upsert

Deleting the stored entry before inserting its replacement left a
window where the directory had no entry and no ordering record — a
tombstone cannot stand in for it without reading as a subtree delete
that suppresses the children's own records. A stale peer event accepted
in that gap could replay an older update or delete and sweep the
directory's children. Upsert the replacement instead: the entry keeps
its recorded commit order across the whole update, and a directory has
no chunks or hard links for a delete to release.

* filer: update an existing directory entry on replay instead of upserting

InsertEntry is not an upsert on SQL stores configured without upsert
support — PostgreSQL with enableUpsert=false issues a plain INSERT and
its duplicate-key error does not match the MySQL-only fallback, so a
replay updating a present directory failed on every retry. Pick
UpdateEntry when the entry exists and InsertEntry when it is absent;
UpdateEntry touches no row that is not there.

* filer: detect a shared-store delete racing a replayed directory update

UpdateEntry reports success even when the row was deleted between the
find and the update — the UPDATE changes zero rows and the replayed
change is dropped while the subscription offset advances. Re-read the
entry after the update and retry the write, re-checking the recorded
commit order each pass so a newer peer change wins instead of being
resurrected over.

* filer: declare commit_ts_ns in field-number order

Field 6 sits between extended and hard_link_id; the declaration now
reads in the same order. Wire format unchanged. Also syncs the Java
client's checked-in copy of filer.proto, which the pb Makefile copies
on gen.

* filer: option to skip delete tombstones entirely

filer.options.deleteTombstones=false stops writing MpDl records on
deletes - a single filer or shared-store cluster never replays peer
events, so the tombstones are dead weight. The tradeoff is explicit:
if a dedicated-store peer does replicate, a stale event can resurrect
a deleted path.

* filer: drop delete tombstones entirely

Deletes no longer write MpDl records, and the pending-repair machinery
they needed is gone with them: per-mutation metadata overhead is now
zero for every store topology. Ordering information lives only on the
entry itself via commit_ts_ns.

The accepted tradeoff: a dedicated-store peer replaying an event older
than a delete can resurrect the path, and a kept directory no longer
fences stale child creates. Recorded tombstones and MpTs records left
by older builds are still read, so upgrades keep the protections they
already have.

* filer: stop reading legacy per-path records too

Drops the MpTs/MpDl read path along with the writes: meta_path_ts.go
keeps only the entry-field readers (CommitTsNs, EntryCommitTsNs,
localCommitTsNs). Entries written before stamping existed fall back to
mtime, and tombstones or commit records older builds left behind no
longer fence replayed events.

* filer: trim commit timestamp comments

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-25 11:37:06 -07:00
Chris Lu 53834117a6 mini: lift the post-mint allocation hold on first inventory report (#12061)
* mini: lift the post-mint allocation hold on first inventory report

DeferAllocation armed a fixed warmup window whenever the leader minted a
fresh topology id, so the first write on a brand-new weed mini blocked
for the whole window even though no prior cluster state could exist.
The window still has to cover a rebuilt master's veteran servers, which
report on their first post-reconnect heartbeat.

Mini has exactly one possible reporter — the embedded volume server,
whose first heartbeat carries the full inventory — so hold allocation
only until that report is applied. The bound raise and topology-id
freeze checks run while the heartbeat is processed, so a wiped master
state still raises the bound or trips the freeze before the hold lifts.

Standalone weed master and weed server keep the fixed window: volume
servers outside the process can still be on their way.

* master: apply queued topology-id reports before opening the allocation gate

SetTopologyInitPending(false) cleared the pending flag before replaying
topology ids stashed while the local id was still being minted. A
NextVolumeId landing in the drain window saw no gate and no freeze yet.
The post-mint allocation hold used to cover the gap by delaying the
winner past the freeze; once the hold can release on first inventory,
the gate must stay closed until the queued checks have run.

* mini: gate the first-report release on single-master mode

releaseAllocOnFirstReport assumes the embedded volume server is the only
possible reporter of prior cluster state. A mini started with
-master.peers joins other masters, whose own volume servers can still be
on their way when the first report lands, so keep the regular
quiet-pulse hold there.

* master: check reported topology id on empty full reports

A HasNoVolumes heartbeat carries the same LastSeenTopologyId a populated
list does, but the mismatch freeze only ran for non-empty reports. An
EC-only veteran could therefore release the post-mint allocation hold
through NoteInventoryApplied while its stale topology id never reached
CheckReportedTopologyId.

* master: drain the topology-id stash inside the init-gate lock

CheckReportedTopologyId only held reportedTopologyIdsLock while stashing,
so a report in flight during SetTopologyInitPending(false) could land its
freeze after topologyInitPending cleared, letting a NextVolumeId slip
through the gap. The whole check now runs under the lock, and the gate is
released inside the same critical section as the drain.

* mini, master: pin and enforce the single volume-server reporter

isSingleMasterMode only checks master peers; SendHeartbeat still accepted
any volume server, so a late foreign report could carry prior cluster
state after the first-report release dropped the hold. Mini now pins the
embedded volume server's address and the master refuses registration from
any other reporter while ReleaseAllocOnFirstReport is set.

* master: admit the volume-server reporter before applying heartbeat state

* test: cover SendHeartbeat admission ordering

A refused reporter must not freeze growth or raise the id bound, and the
pinned reporter's stale topology id still must.

* mini: re-derive the single-reporter release on every run

The package-level master options survive in-process reruns, so a
multi-master mini following a single-master one must not inherit the
first-report release or a stale reporter pin.

* master: admit addressless first heartbeats under the reporter pin

An unregistered heartbeat with no Ip skipped the single-reporter
admission check but still reached the seen-bound raise and the
topology-id freeze before the delta-heartbeat skip. Admission now runs
for every unregistered heartbeat; an empty address can never equal the
pin.
2026-10-25 09:25:39 -07:00
Chris LuandDevin 88b91cccba command: no extra localhost listener when -ip.bind is specified (#12055)
* util: add listenOnLocalhost parameter to NewIpAndLocalListeners

Nil keeps the current behavior of also binding localhost when the host
is a specific address; all callers pass nil for now.

* command: add -ip.bind.localhost flag to opt out of the extra localhost listener

Binding a command to a specific -ip.bind address also opened a second
listener on localhost for every port, with no way to turn it off. Each
command that calls NewIpAndLocalListeners (master, master.follower,
filer, s3, iam, volume, sftp, mq.broker, mq.agent) now accepts
-ip.bind.localhost=false to bind only the requested address. Embedded
services get prefixed flags (s3.ip.bind.localhost, sftp.ip.bind.localhost,
iam.ip.bind.localhost), and the command-level flag also disables them so
a single switch turns off every extra listener the command opens.

* mini: only require all-interface port availability while the loopback listener is on

With -ip.bind.localhost=false each service binds just -ip.bind, so a port
held on another address no longer collides; the all-interface precheck
would reject or relocate usable ports. WebDAV, the Admin UI, and the
Admin worker gRPC (which binds -admin.worker.ip, not -ip.bind) never open
a loopback companion listener, so their checks now use the address they
actually bind.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mq.broker: bind the gRPC listener to -ip.bind instead of all interfaces

The listener used an empty host so -ip was ignored for binding and the
localhost companion was never created, leaving -ip.bind.localhost with
nothing to control. Give the broker the same -ip/-ip.bind pair the other
servers use; embedded mode binds to -ip.bind like its peers.

* server: add -sftp.ip.bind.localhost and -iam.ip.bind.localhost switches

weed filer already exposes the prefixed opt-outs for its embedded sftp
and iam; give weed server the same pair so each extra listener can be
disabled individually while -ip.bind.localhost=false still turns them
all off.

* mq.broker: keep the default all-interface bind unless localhost is off

An unset -ip.bind used to mean the standalone broker listened on every
interface; defaulting it to -ip silently cut off multi-homed clients. Only
fall back to -ip when -ip.bind.localhost=false, where a specific bind is
needed for the opt-out to take effect.

* mini: validate the Admin HTTP port on the address it will bind

The port check always used -ip.bind while startMiniAdminWithWorker falls
back to loopback when Admin has no credential, so a loopback-held port
passed validation and killed the admin at startup.

* mq.broker: keep the all-interface default bind when -ip.bind is unset

-ip only becomes the fallback once -ip.bind.localhost=false needs a
specific address for the listener.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mini: validate the Admin HTTP port on its effective bind IP

When the Admin listener falls back to loopback (no credential or mTLS
configured), the pre-flight port check now probes that same address
instead of -ip.bind.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util: keep NewIpAndLocalListeners source-compatible for 3-arg callers

* command: drop the -ip.bind.localhost flag; an explicit -ip.bind drops the extra listener

Instead of an opt-out flag, the loopback companion listener now simply
follows whether the bind address was chosen: it is created only while
-ip.bind is unset and the bind address falls back to -ip. Passing
-ip.bind=<addr> binds each port to exactly that address.

Each service options struct carries a listenOnLocalhost *bool that
embedders (weed server, weed mini, embedded s3/sftp/iam in weed filer)
set from the parent's -ip.bind state, captured before the bind flag is
repointed or filled in by the -ip fallback; standalone commands derive
it from their own flag before the fallback runs.

iam and mq.agent gain a real -ip.bind flag so they have a way to pin a
single address. mq.broker keeps its new -ip.bind flag but drops the
-ip.bind.localhost fallback hack: empty still listens on all interfaces.

* server: default embedded mq.agent brokers to the broker's bind address

With an explicit -ip.bind the embedded broker no longer listens on
localhost, so an unset -mq.agent.brokers defaulting to localhost:17777
could not reach it. Default to <serverBindIp>:<mq.broker.port> (the
address the embedded broker binds), falling back to localhost when the
bind address is a wildcard. An explicit -mq.agent.brokers still wins.

Also fix the NewIpAndLocalListeners skip message to say localhost.

* server: bind embedded mq.agent to serverBindIp

The embedded agent's ipBind was never set, so startQueueAgent fell back
to the advertised -ip address and an explicit -ip.bind did not pin the
agent listener.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-25 06:52:16 -07:00
fefac028ec feat: send EC shards in parallel per destination disk (#12050)
* feat: send EC shards in parallel per destination disk

One stream per destination disk. max_shard_transfers unset means no global cap, 0 sends shards sequentially, and N is a global cap that still keeps one stream per disk. The value is persisted through TaskPolicy. The maintenance form is a checkbox plus Max transfers because unset and 0 differ. The key is intentionally omitted from GetConfigSpec.

* ec shard distribution: treat negative max_shard_transfers as sequential

The config layer (normalizedMaxShardTransfers) clamps negative values to 0,
which selects sequential distribution. shardTransferCap diverged: a negative
cap that reached ShardDistributionOptions directly would have selected
uncapped parallel mode, the most aggressive setting. Align the options layer
with the same normalization so invalid input degrades to the conservative
mode.

* ec: serialize shard transfers to a server when a target leaves disk choice to the receiver

disk_id 0 is auto-placement on the receiving volume server, so an
auto-placed stream can land on a disk another queued stream is pinned
to. Grouping by (destNode, disk_id) alone let both run at once; collapse
all of that server's transfers under the destNode key instead.

* worker: clamp max_shard_transfers to the int32 wire range

ReadIntConfig returned the 0 fallback for values above MaxInt32, turning
a large positive cap into sequential mode. Read int64 and clamp into
range instead.

* worker: drop unused shard-transfer form helper

The maintenance form renders and collects max_shard_transfers in
plugin.templ JavaScript; the Go copy had no callers.

* worker: bound max_shard_transfers conversion under an in-range guard

---------

Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-25 06:37:26 -07:00
Jay HemnaniandChris Lu 0c9872f842 filer, volume: ignore Range when If-Range no longer matches (#12052)
* filer, volume: ignore Range when If-Range no longer matches

A GET with Range and a stale If-Range validator got 206 with the requested
bytes of the current file. RFC 9110 13.1.5: send the whole content instead,
so a resumed download restarts rather than mixing two versions.

* volume(rs): drop Range when If-Range no longer matches

Mirror the Go fix for the Rust volume server: evaluate If-Range against the
response ETag (strong comparison) or Last-Modified once the needle's
validators are known, then drop the range on a stale validator so the whole
current representation is sent. A dropped range on the RangeFromSource
strategy falls back to a full stream of the data-file region; buffered and
chunk-manifest paths already take Option<&str>.

* volume(rs): fail If-Range when the header value is not readable

* volume(rs): accept all HTTP-date formats in If-Range like Go

* server: cover alternate HTTP-date formats in If-Range test

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-25 06:36:40 -07:00
Chris LuandDevin da1d7444b8 master: shorten the first-write hold after a fresh topology id (#12059)
* master: release the post-mint allocation hold after a quiet pulse

DeferAllocation armed a fixed warmup window whenever the leader minted a
fresh topology id, so the first write on a brand-new cluster blocked for
the whole window even though no prior state could exist. The window
still has to cover a rebuilt master's veteran servers, which report on
their first post-reconnect heartbeat.

Track the last heartbeat that could carry prior cluster state — a server
joining, held volumes or shards, a nonzero id bound, or a foreign
topology id — and lift the hold one pulse after it goes quiet, capped by
the original window.

* master: make the post-mint allocation hold configurable

master.volume_growth.allocation_deferral overrides the warmup window a
leader holds volume id allocation for after minting a fresh topology
id. Unset keeps the existing window; "0s" disables the hold for setups
that know the cluster is fresh (test harnesses, single-node labs); any
other duration replaces the cap while the quiet-pulse early release
still applies.

* master: re-check the allocation deadline while it is held

NextVolumeId slept for the whole computed delay, so a heartbeat landing
mid-sleep that pushed the quiet deadline out was ignored — the pending
allocation could still run ahead of a straggler's report. Wait in short
slices, re-evaluating the deadline and the growth freeze each round.

* master: count only ids above the issued bound as prior-state evidence

Heartbeat echoes of volumes and shards this master allocated during the
hold refreshed the quiet window, so each growth allocation re-extended
the deferral until the hard cap. Evidence now requires an id beyond the
bound this leader has issued, a higher reported bound, or a foreign
topology id; a joining server (dn == nil) still counts once.

* master: reject an unparseable allocation_deferral instead of disabling the hold

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: do not count a skipped heartbeat as a joining server

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: assert the allocation hold extends past the original quiet deadline

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: release the post-mint hold after two quiet pulses

One pulse proved short for a veteran that reconnects on a staggered
restart: with only five seconds of quiet, allocation could resume and
reuse ids before a slower server landed its first heartbeat. Two pulses
still halves the fresh-cluster first-write stall versus the full warmup
cap, and known-fresh setups keep the "0s" override.

* master: pick once more when assign times out as growth concludes

The pick loop's final attempt can predate the growth it triggered
concluding: growth registers its volumes in the layout before clearing
the pending flag, so a timeout landing on that boundary used to shed a
hard 406 over volumes that had just arrived. Pick once more after the
loop in both the http and grpc assign paths before deciding how to
shed.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-25 06:17:42 -07:00
Chris LuandDevin d15396fc4d s3tables: scope the anonymous identity's grant on the table catalogs (#12056)
* s3tables: scope the resolved anonymous identity instead of defaulting to allow

Catalog authorization fell back to the open default whenever the request
resolved to the configured anonymous identity, ignoring its grant. The
fallback now applies only when no principal was resolved (zero-config),
so anonymous requests are checked against the identity's configured
actions like any other principal. The S3 Tables API and the Iceberg and
Lance catalogs share this handler, so all three are covered.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: authenticate the catalog clients in the s3tables suites

Anonymous calls into the Lance and Iceberg catalogs are now scoped by
the anonymous identity's grant, so the suites that used to rely on the
anonymous pass authenticate like their real deployments do: x-api-key
for the Lance probes and REST drivers, client-credentials OAuth for
pyiceberg.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* worker(lance): authenticate namespace calls with the storage credential

Anonymous catalog calls are now scoped by the anonymous grant, so a
worker that holds an S3 credential sends it as x-api-key on the
namespace requests instead of relying on the anonymous pass.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: authenticate the pushdown catalog connection in catalog_lancedb

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* lance: follow catalog redirects only within the same host

reqwest forwards custom headers on a follow, so a redirect to another
host would carry the x-api-key credential to a server it was never
meant for. Keep same-host hops for proxies that normalize the path or
upgrade the scheme.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* lance: warn when the catalog api key travels in cleartext

The x-api-key pair is a real storage credential: over plain http to a
non-loopback host the network can read it. Warn at client build so the
operator sees it; https and loopback stay quiet, and the key still
sends since an http catalog is a valid deployment.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* lance: require the same origin on catalog redirects

A same-host hop can still downgrade https to http and put the api key on
the wire; matching scheme, host, and port keeps the credential on the
listener it was issued for.

* lance: bound catalog redirect hops so loops fail fast

The custom redirect policy drops reqwest's default 10-hop limit, so a
catalog that redirects back to itself would hold each call open until
the request timeout. Keep the bound alongside the origin check.

* lance: gofmt the redirect helpers

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-25 06:13:22 -07:00
Chris LuandDevin 39bdbc18db admin: keep read-only sessions out of the filer /etc tree (#12057)
* admin: keep read-only sessions out of the filer /etc tree

The file browser read routes are open to any authenticated session, so a
read-only login could download or view arbitrary filer paths. The filer
stores the credential and cluster configuration under /etc (identity JSON
with plaintext S3 secret keys, SSE-S3 keys, remote storage config), which
let a view-only session recover live secrets that the user API redacts.

Gate the read paths on the session role: read-only sessions get 403 for
/api/files/download, /view, /properties, /metadata on /etc paths, the
browser page redirects away, and recursive metadata exports skip the /etc
subtree for read-only sessions.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* admin: only reject real .. path components in file browser paths

path.Clean already resolves interior parent references, so any '..'
surviving in the cleaned path can only be a leading escape from a
relative input or part of a valid name. Matching on the substring
rejected legitimate entries like /buckets/reports/report..csv.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* admin: cover the read-only /etc export exclusion with a test

Pull the excluded-prefix selection into exportExcludedPrefixes so the
role-dependent /etc skip is exercised directly — the cleanReadPath
tests alone would still pass if the exclusion were dropped.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-25 06:06:21 -07:00
Chris Lu 403d51631b s3api: stop echoing stored tags and internal attributes on GET/HEAD (#12053)
* s3api: stop echoing stored tags and internal attributes on GET/HEAD

setResponseHeaders copied every entry.Extended key into the response,
only filtering xattr- and x-seaweedfs-. Stored object tags
(X-Amz-Tagging-*) and the gateway's own attributes (Seaweed-X-Amz-Acl,
-Owner, -Version-Id, -Object-Lock-Mode, ...) therefore leaked to any
caller holding s3:GetObject, and to anonymous readers on public-read
objects, without s3:GetObjectTagging or READ_ACP.

Tag count, user metadata, storage class and the real S3 headers
(x-amz-version-id, x-amz-object-lock-*) are unaffected.

* s3api: match stored tag and internal attribute keys case-insensitively
2026-10-25 06:05:06 -07:00
Chris Lu 52d91008ac ci: bump hosted runners to ubuntu-24.04
GitHub is retiring ubuntu-22.04; move all 94 runs-on pins across 40 workflows to ubuntu-24.04. rust_binaries_release: the arm64 cross step was jammy-era — noble keeps apt sources in deb822 ubuntu.sources (the sources.list edit was a silent no-op that left amd64 repos open to arm64 and 404'd) and ports suites must say noble. Pin deb822 stanzas to amd64 and add noble ports entries.
2026-10-24 23:07:28 -07:00
github-actions[bot] ec8f979350 4.50 2026-10-25 03:09:42 +00:00
Chris LuandDevin 1a0485a43d test: bound trino container memory + dump logs when it exits before ready (#12044)
* test: dump trino container state and logs when it exits before ready

When the Trino container dies during startup, waitForTrino only
reported the docker exec error ("container is not running"), which
gives no hint whether the JVM was OOM-killed, the image broke, or the
config is bad. Capture docker inspect exit state and the container
logs in the failure message so runner-side flakes are diagnosable
from the test output alone.

* test: cap trino container memory at 4g in catalog tests

Trino's jvm.config sizes the heap at 80% of host RAM. On a loaded
self-hosted runner the JVM attempts to claim most of system memory
at boot and the container is killed ~15s after docker run — the
flake seen in CI today. Bounding the container to 4g gives the JVM
a fixed ~3.2g ceiling (observed usage is ~750mb), immune to host
memory pressure.

* test: keep docker errors in trino container diagnostics

Report docker inspect/logs failures instead of dropping them, and
include the last docker exec output in the container-dead fatal so a
missing container still explains what happened.

* test: detect trino container exit in readiness fallback

When the polling loop times out while the container still appears
alive, the trino --version fallback may itself fail because the
container has since exited. Re-check its output for the
exited-container signature and route through the diagnostic dump.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: dump trino container diagnostics on readiness timeout too

The alive-but-never-ready timeout path reported only the last docker
exec error, losing the container state and logs that the dead-container
path already captures.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 20:00:06 -07:00
Chris LuandDevin 8c1a0a0cc2 filer: fix prefixed listing when the resume point sorts below the prefix (#12049)
* test: prefixed directory listing with start below the prefix

A listing constrained to prefix "item-b" and resumed from "item" must
return item-b.versions. The seek-based stores return an empty page
because the iterator lands before the prefix range and breaks on the
first non-matching name.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: clamp prefixed-listing seek to the prefix range

Seek-based stores (leveldb, leveldb2, leveldb3, rocksdb, tikv, etcd)
started the scan at startFileName and stopped at the first key outside
the prefix. A startFileName below the prefix landed the iterator on a
non-matching sibling first, so the listing returned an empty page even
though matching entries followed. etcd had the mirror problem: a range
beginning below the prefix end leaked non-matching entries. Begin the
seek at max(startFileName, prefix), which is what foundationdb and the
sql stores already do.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer(hbase): honor startFileName lower bound in prefixed listings

The scan starts at the prefix but only skipped the exact startFileName,
so entries sorting before it were returned again on resume.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: keep paging a prefixed listing while matches can still follow

The fallback for stores without native prefix support stopped paging as
soon as the last scanned name reached the prefix, dropping prefix
matches that continue on the next page.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: return the last delivered name from the prefixed-listing fallback

The fallback for stores without native prefix support pages raw entries
and filters client-side. Keep the scan cursor and the returned cursor
separate: resuming must continue after the last delivered match, not the
last scanned name, or matches still inside the current raw page are
skipped on the next page. Also keep paging while the scan cursor is
inside or below the prefix range instead of stopping at the first name
that reaches the prefix.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer(hbase): start prefixed-listing scans at the resume key

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 19:59:19 -07:00
github-actions[bot] 0c32eab48e docs: regenerate star history chart 2026-10-25 00:48:29 +00:00
Chris Lu 85470416cb volume: overlap the local needle write with the replica fan-out (#10074)
* volume: overlap the local needle write with the replica fan-out

ReplicatedWrite wrote the needle to local disk and then fanned it out to
the replicas serially, so a replicated write cost local + replicas. The
needle is already fully buffered, so the two can run at once. Snapshot
the fields the fan-out reads first (the local write stamps the volume
TTL and updates needle flags, so the replica goroutine must not touch
the shared needle), then run the local write and the replica fan-out
concurrently and ack only after both. Where local disk and replica
network are separate resources the write latency drops to max(local,
replicas) instead of their sum; co-located on one disk the two writes
contend, so this helps distributed placements, not single-host tests.

* volume: return the local isUnchanged from the write goroutine

* volume: trim overlap-write comments

* volume-rs: overlap the replica fan-out with the local needle write

* volume: test the overlapped local write and replica fan-out

* volume: decode replica pairs only when replicas exist

A single-copy or incoming replicate write never uses the decoded pair map, so a malformed Pairs blob only burned an error counter and a log line. Gate the decode on replicaCount.

* volume: release the stalled replica on test failure paths

An early t.Fatal left the httptest handler blocked on releaseReplica, so the deferred Close hung instead of reporting the failure. Release idempotently ahead of the server close.

* test: mismatched If-None-Match serves the object over If-Modified-Since

The precedence case asserted the old If-Modified-Since-first ordering and
expected 304. The volume servers now evaluate If-None-Match first, so a
non-matching tag must return 200 with the body even when If-Modified-Since
would allow 304 (RFC 9110 13.1.3).
2026-10-24 15:16:16 -07:00
Chris LuandDevin e442eda88e s3: authorize POST policy uploads through the shared object check (#12047)
* s3: authorize POST policy uploads through the shared object check

doesPolicySignatureMatch gated a verified POST Object signature on the
identity's legacy Write action, which cannot represent attached IAM
policies, STS session policies, or key-scoped grants. Users authorized
only by an attached policy (e.g. s3:PutObject on bucket/*) were denied
with AccessDenied even though the equivalent PUT succeeds.

The handler already runs AuthorizeObjectWrite against the real object
key after the form's x-amz-* fields are promoted to headers, so the
coarse check only shadowed the full evaluation. Signature verification
now just returns the identity; authorization stays in one place.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: POST Object authorizes attached-policy identities, still denies others

Cover the signature check and AuthorizeObjectWrite for an identity whose
only write grant comes from an attached IAM policy, plus the deny leg
for an identity whose attached policy omits s3:PutObject.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 15:09:34 -07:00
Chris Lu 216c1032e6 test: mismatched If-None-Match serves the object over If-Modified-Since (#12045)
The precedence case asserted the old If-Modified-Since-first ordering and
expected 304. The volume servers now evaluate If-None-Match first, so a
non-matching tag must return 200 with the body even when If-Modified-Since
would allow 304 (RFC 9110 13.1.3).
2026-10-24 15:06:19 -07:00
03f773f087 redis2: pipeline entry reads when listing a directory (#12042)
ListDirectoryEntries fetched every member with its own GET, so a page of
1024 entries cost 1024 round trips. Fetch the values in pipelined batches
of 128 as the loop reaches them and walk the replies in order. Each reply
keeps its own error, so missing and expired members are cleaned up and
skipped exactly as before, and an unreadable member still ends the
listing at that name. Batching keeps a caller that lists with a huge
limit and stops after a few entries from reading the whole directory.

Co-authored-by: Patrick Schmidt <1310072+Woellchen@users.noreply.github.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
2026-10-24 15:04:37 -07:00
Junker der Provinz f9dac64374 helm: remove filer.redirectOnRead, which weed filer does not accept (#12040)
weed filer lost -redirectOnRead in 9b3109a5d (1.59), which also removed
the chart value, but 2bed15f1d added it back. Setting it makes the filer
exit with "flag provided but not defined".
2026-10-24 10:45:57 -07:00
7b43dec665 shell: run volume.configure.replication in parallel per volume server (#12041)
* shell: run volume.configure.replication in parallel per volume server

Volume servers were configured one after another, and each server's
result overwrote the previous error, so a failure on one server was
lost when a later server succeeded. Run one job per volume server,
bounded by -maxParallelization like the other shell commands, and
return the errors from all servers.

VolumeConfigure unmounts and remounts the volume, so replicas of the
same volume on different servers take turns instead of all going
offline at once.

Co-authored-by: NyaMisty <5344431+NyaMisty@users.noreply.github.com>

* shell: test that volume.configure.replication overlaps servers up to the limit

* shell: assert every server ran in the parallel configure-replication test

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: NyaMisty <5344431+NyaMisty@users.noreply.github.com>
Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 10:29:38 -07:00
Chris LuandDevin 95d8a709e8 ci: run s3-go-tests on remote_storage changes (#12043)
The S3 Remote Cache tests exercise the remote-storage S3 client, but the
path filter did not include weed/remote_storage, so a tagging change
there can regress the suite without running it on the PR.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 09:42:17 -07:00
Chris LuandDevin e432b75822 remote_storage/s3: send only real object tags to the remote (#12039)
* remote_storage/s3: send only real object tags to the remote

WriteFile and toTagging copied every entry.Extended attribute into the
remote object's tag set, including Seaweed-internal keys such as
Seaweed-X-Amz-Acl whose values are not legal S3 tag values. Remotes
that validate x-amz-tagging now reject these uploads with InvalidTag.

Only X-Amz-Tagging-* attributes are S3 tags: filter to that prefix,
strip it, and encode through s3TaggingHeader so keys and values are
properly escaped.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* remote_storage/s3: skip remote tagging call when the tag set is unchanged

UpdateFileMetadata fired PutObjectTagging on any Extended diff, but most
attribute changes are not tag changes. Compare the effective tag sets and
return early when they match.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* remote_storage/s3: assert WriteFile errors in tag-header tests

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 09:41:01 -07:00
81c90cb446 filer.backup: retry a failed checkpoint read instead of replaying from the epoch (#12036)
* filer.backup: retry a failed checkpoint read instead of replaying from the epoch

Fixes #12035

* test: reset sink.local.directory in backup offset test cleanup

The new test sets sink.local.directory on the shared viper but only
reset the enabled flag, leaking a key that points at a deleted
t.TempDir() to later tests in the package.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: cover the retried attempt re-reading the checkpoint

The fake filer now fails only its first KvGet, and the test runs a
second doFilerBackup attempt asserting the checkpoint is re-read and
the attempt proceeds past the offset stage. LookupDirectoryEntry is
stubbed so the SSE-S3 KEK probe returns a miss instead of retrying
10x2s.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 08:22:29 -07:00
Chris LuandDevin 4c2c0ff21c s3: fix wildcard matcher skipping '*' targets in policy patterns (#12034)
* s3: check pattern '*' before literal equality in matchWildcardString

When the target string carried a '*' at the offset of the pattern's '*',
the literal-equality case consumed it as a one-character match and no
backtrack point was recorded, so the comparison returned false. A Deny
Resource carrying a policy variable, or an Allow carrying NotResource,
could then be sidestepped by an object key containing '*' at the aligned
offset.

* test: cover literal asterisks in wildcard match targets

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 08:20:22 -07:00
Chris Lu 1d2d9dfed2 test(fuse): raise mini/filer and mount readiness waits to 90s (#12037)
Same startup-timing flake class as the Makefile waits: volume/filer
readiness at 30s is marginal on loaded runners — observed failing on a
github-hosted runner too (TestPOSIXCompliance +
TestExternalPOSIXSuites). The process-death early return still reports
real startup failures quickly.
2026-10-24 08:13:36 -07:00
c4d743a69f s3: make ListObjectsV2 continuation tokens round-trip XML-unsafe keys (#11996)
* s3: keep ListObjectsV2 continuation tokens round-tripping XML-unsafe keys

A page ending on a key XML 1.0 cannot carry (for example one containing
\x01) produced a NextContinuationToken from that raw key. The XML
encoder replaced the control characters with U+FFFD, so a client resuming
at the token restarted after the mangled name and skipped every key
sorting between it and the real one. encoding-type=url encoded the keys
but not the token, so it hit the same skip.

Encode the token as base64url behind a prefix when the key is not valid
XML text; keys that survive XML unchanged keep the bare key as the token,
so their tokens are byte-identical to before. The handler decodes the
prefix back to the raw key before listing.

Fixes #11990

* s3: keep undecodable base64: list tokens resuming at the raw key

* s3: round-trip ListObjectsV1 markers over XML-unsafe keys

* s3: round-trip ListObjectVersions key markers over XML-unsafe keys

* s3: round-trip ListMultipartUploads key markers over XML-unsafe keys

* s3: decode only tokens whose key could not have stayed bare

* s3: document and pin the encoded-wins rule for ambiguous list tokens

A token like base64:AQ reads as both a legacy bare key and the encoded form of an XML-unsafe key. The encoded reading already won in code; state why and cover it so the choice is not reverted as a false simplification.

* s3: keep request-marker echoes in the response's wire encoding

The decoded marker/key-marker is a raw key; echoing it writes XML-unsafe
bytes straight into the response, where the encoder turns them into
U+FFFD. Re-encode each echo through encodeListContinuationToken so the
client reads back the token it sent, and apply urlPathEscape for V1 when
encoding-type=url is active, matching the listed keys.

* s3: wire-encode the V1 NextMarker echo like the request marker

* s3: pin resuming an XML-unsafe key past a sibling directory

* s3: wire-encode ListMultipartUploads markers and keys under encoding-type=url

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: unescape url-encoded marker echoes on list resume

Under encoding-type=url the V1 and ListMultipartUploads responses emit
marker echoes through urlPathEscape, and clients pass them back verbatim.
The incoming marker was used as the resume position without undoing that
escaping, so resuming after a url-unsafe-but-XML-safe key like "a b"
(marker "a%20b") restarted the listing at the literal escaped text.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: assert the full decoded V1 sequence in the marker round-trip test

The test only checked the page count, so a regression that skipped one
key while repeating another still passed. Requesting encoding-type=url
keeps the XML-unsafe keys legible for the decoded comparison and
exercises resuming with the escaped marker echo.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: only unescape markers in the canonical urlPathEscape form

A url-encoded request marker like a%41 cannot be encoder output, since
urlPathEscape never emits %41 for 'A'. Unescaping it anyway resumed at
aA and skipped every key sorting between a%41 and aA, including a%42.
Require the unescape to re-escape to the received marker so only
verbatim echoes decode; anything else stays the raw key the client sent.

* s3: pin a raw-key marker that looks like an escape sequence

* s3: cover the marker-echo decode guard with unit cases

* s3: wire-encode the ListObjectsV2 StartAfter echo under encoding-type=url

AWS encodes Delimiter, Prefix, Key, and StartAfter in V2 responses when
url encoding is requested, but StartAfter was echoed raw. The echo now
takes urlPathEscape and the incoming start-after param runs through
decodeMarkerEcho, so a client resuming from the emitted form lands on
the raw key.

* Merge master and reconcile ListMultipartUploads marker encoding

Resolve the overlap with the merged encoding-type=url pass: keys and
markers now go through the single post-pass so nothing is url-escaped
twice. Markers are still token-encoded when the response is not
url-encoded, and the key-marker decode checks the query-escaped wire
form the endpoint actually emits.

* s3: echo the decoded start-after under encoding-type=url

The escaped echo of the raw input is not a fixpoint: a client resubmitting
the echoed StartAfter decoded it to the literal escape text and resumed at
the wrong position.

* s3: keep url-mode multipart key-marker decodes off the token path

Under encoding-type=url the emitted KeyMarker/NextKeyMarker are the
url-escaped raw key, never a base64-prefixed token, so the request decode
must unescape only. Running the token decode afterward misread a literal
"base64:..." key as an encoded token and resumed at the decoded bytes,
skipping the keys in between.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:18:10 -07:00
fcd28031e7 mq/kafka: wait for a filer to register instead of panicking on an empty list (#12019)
* mq/kafka: wait for a filer to register instead of panicking on an empty list

NewSeaweedMQBrokerHandler queried the masters for filers once, ~2 s after
start, and handed an empty list to NewFilerClientAccessor, which panics. A
fresh filer registers only after its empty-membership confirm window
(10 s), so a gateway started with a fresh cluster crashed (Kafka Quick
Test, kafka-gateway exit 2). Retry while the list is empty for up to 60 s,
then return an error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* mq/kafka: wait out a transient-empty broker listing too

A fresh SMQ broker registers with the masters on the same announce
delay as a filer, so the one-shot broker lookup could fail startup the
same way. Generalize waitForFilers into waitForClusterNodes and bound
both discoveries by the same window; the empty-broker check becomes
unreachable and drops away.

* mq/kafka: wait out the broker health check like node discovery

A broker announces to the masters before it has acquired its balancer
lock, so the gateway's three-attempt probe could still fail fast on a
fresh cluster. Retry FindBrokerLeader for the same window the caller
waits out empty listings.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:09:43 -07:00
Chris LuandDevin 34c7bd03ad ci: clean up stale FUSE mounts before running tests (#12020)
* ci(s3tables): tolerate stale FUSE mounts in disk diagnostics

* ci(s3tables): bound df diagnostics with timeout, explain once

A stale FUSE mount can also hang statfs, not just fail it, so wrap df in
timeout 15; one comment at the first site covers all sixteen.

* ci: tolerate stale FUSE mounts in df diagnostics elsewhere

The same self-hosted runners serve the other workflow files; a bare
df -h there can stall or fail a job the same way.

* ci: unmount disconnected FUSE test mounts before jobs start

* ci: skip FUSE cleanup when the variant checkout is skipped

* ci: reclaim the crashed run's test dir once its stale FUSE mount is detached

* ci: stop tracking python bytecode for the FUSE cleanup action

The helper's __pycache__ slipped into the tree; ignore it so local
test runs cannot re-add it.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run the FUSE cleanup action's unit tests on action changes

test_cleanup.py was never wired into CI, so a regression in cleanup.py
would ship silently. A path-filtered workflow runs it on every change
under .github/actions/.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: match every FUSE test mount layout in the cleanup action

The candidates list only matched seaweedfs_fuse_test_*/mount, so stale
mounts left by fuse_failover and fuse_p2p runs (mountN leaves under
seaweedfs_fuse_* dirs) and the posix-compliance workflow's fixed
/tmp/seaweedfs/mount were never reclaimed and could still wedge df.

* ci: run the FUSE mount cleanup in the fuse_failover and fuse_p2p workflows

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:07:55 -07:00
18f9d9a0b6 feat: Free Volume Deadband (M) for EC shard placement (#12029)
* feat: Free Volume Deadband (M) for EC Place nodes and racks

|ΔfreeSlots| ≤ M is a tie; remainder is liveShards. M=0 stays strict
freeSlots. M is free volume slots (× dataShards). pickNodeInRackEligible
and pickRackEligible share the rule. Worker plugin config
free_slots_deadband, default 0. Not on the write path.

* fix: anchor EC freeSlots deadband on class maxFree

Pairwise |Δfree|≤M vs the current winner is not transitive: 120→110→100
at M=10 slots picked 100. After evenness, rank by maxFree, keep
|free-maxFree|≤M, then min liveShards. Shared by node and rack. M=0 is
still only maxFree then live.

* worker: add free_slots_deadband to ErasureCodingTaskConfig

* erasure_coding: persist free_slots_deadband through TaskPolicy

ApplyMaintenanceConfigFromToml accepted the key but the admin save path
round-trips config through ToTaskPolicy, so the value was dropped on
reload; serialize it like the other EC settings.

* erasure_coding: bound free_slots_deadband at its declared maximum

The TOML loader and the plugin worker config only clamped negatives, so
a value above the spec's 100000 could reach the int32 TaskPolicy field
and wrap; cap at 100000 on both entry paths and declare MaxValue on the
plugin field.

* erasure_coding: document free_slots_deadband in scaffold admin.toml

---------

Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
2026-10-24 07:06:46 -07:00
52e05ccda4 s3: accept a self-copy that changes the storage class (#12031)
* s3: accept a self-copy that changes the storage class or encryption

AWS changes an object's storage class or encryption by copying the
object onto itself with the new x-amz-storage-class or SSE headers, and
only rejects a self-copy that changes none of metadata, storage class,
website redirect location or encryption. CopyObject rejected every
self-copy without REPLACE metadata or tagging directives as
InvalidRequest, so these requests failed.

* s3: reject an invalid x-amz-storage-class on CopyObject

A self-copy with a bad storage class used to fail as InvalidRequest;
once such copies are admitted the value must be validated like PutObject
does, or it would be stored as opaque metadata.

* test: cover the invalid storage class rejection on self-copy

* s3: only a storage class change admits a self-copy

Admitting encryption changes on a self-copy is unsafe: the metadata-only
path compares only KMS key IDs, so a new encryption context or bucket-key
setting is advertised over the old data key, and SSE-KMS key rotation
onto the same entry keeps the old data key.

Count only a requested storage class that differs from the object's
(STANDARD when none is stored) as a change, so a self-copy to the
current class or with only encryption headers is rejected as before.
Such a self-copy updates the object in place and has to keep its
encryption; one that would re-encrypt it or uses SSE-KMS is rejected
too.

---------

Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
2026-10-24 07:06:20 -07:00
Chris LuandDevin 514c0c1e99 topology: stop a stale duplicate-vid report from re-quarantining the survivor (#12032)
* topology: drop a duplicate-vid report whose own copy is already gone

flagDuplicateVolumeId runs on a goroutine after registration, so by the
time it executes the reporter's copy may have been unregistered (a
finished move drops the losing copy). The surviving copies are then
legitimate, yet the stale report still quarantined them and stored a
phantom (node, vid) key for the departed reporter. Skip the marks when
the reporter no longer serves the volume under its own layout key.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(topology): cover a stale duplicate report against a lone survivor

The existing tests only flag the conflict before the losing copy is
unregistered. Run the delayed duplicate check after the reporter's copy
is already gone and assert nothing is quarantined and the survivor stays
writable, so removing the stale-report guard fails the test outright
instead of needing a lucky schedule.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(topology): cover the enforceQuarantine-driven release retry

A failed writable push re-arms the record, and the next volume report
drains that retry through enforceQuarantine itself - a different entry
point than the pulse-driven pass the other tests exercise.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:05:46 -07:00
8841eb6c70 volume: check If-None-Match before If-Modified-Since and match ETag lists (#12027)
* filer: match If-Match and If-None-Match against "*" and ETag lists

checkPreconditions compared the whole header as one ETag, so "*", a
list of ETags and a weak W/ tag in If-None-Match never matched
(RFC 9110 13.1.1, 13.1.2).

* volume: check If-None-Match before If-Modified-Since and match ETag lists

A volume GET answered 304 from If-Modified-Since even when If-None-Match
named a different ETag, and compared If-None-Match as one exact string,
so "*", lists and weak tags never matched (RFC 9110 13.1.2, 13.2.2).

* filer: split etag lists only outside quoted tags

A comma inside a quoted opaque tag is a legal etagc, not a list
separator; strings.Split cut such tags into fragments that
CanonicalizeETag could turn into a false match.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: match repeated If-Match and If-None-Match header lines

Header.Get reads only the first field line, so a match on a later
repeated header was missed; join all field values like any other
comma-separated list.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: match repeated If-None-Match header lines

Header.Get reads only the first field line; join all field values so a
match on a later repeated header is seen. Also covers a comma inside a
quoted opaque tag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: send the etag on a 304 like the filer does

RFC 9110 15.4.5 lists ETag among the fields a 304 must carry when a 200
would send it; SetEtag now runs before the notModified check.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: mirror the conditional-GET fixes in the rust server

not_modified_response now checks If-None-Match before If-Modified-Since,
matches "*", weak tags, quoted commas and repeated field lines through
the same list rules, and sends the etag on a 304.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: treat an empty If-None-Match like the Go server does

* filer: ignore empty elements in etag precondition lists

* volume: ignore empty elements in etag lists like the filer

* filer: let a present-but-empty precondition line suppress the date check

* volume: gate If-Modified-Since on a present If-None-Match line

* volume: gate If-Modified-Since on header presence like the Go server

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: chrislusf <chris.lu@gmail.com>
2026-10-24 07:05:29 -07:00
Chris LuandDevin 59cccc788c master: read the writable-volumes cap from master.toml (#12009)
* master: read the writable-volumes cap from master.toml

-maxWritableVolumes becomes the master.max_writable_volumes config key,
so the new master limit is set the same way as other master options
instead of a flag wired into every command that embeds a master.

* master: name the writable cap master.max_writable_volumes_per_layout

The cap applies per volume layout (collection + replica placement + ttl +
disk type), so the key names its scope instead of reading as a global cap.

* master: keep an embedder's MaxWritableVolumes when the key is unset

NewMasterServer overwrote option.MaxWritableVolumes with the viper
value, which reads as 0 whenever master.toml omits the key, silently
dropping a cap a Go caller supplied through MasterOption. Seed the
default from the option instead so the field survives an absent key;
an explicit config entry still wins.

* master: drop stray nested-checkout gitlinks

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:04:45 -07:00
9e7fd2eb90 filer: match If-Match and If-None-Match against "*" and ETag lists (#12026)
* filer: match If-Match and If-None-Match against "*" and ETag lists

checkPreconditions compared the whole header as one ETag, so "*", a
list of ETags and a weak W/ tag in If-None-Match never matched
(RFC 9110 13.1.1, 13.1.2).

* filer: split etag lists only outside quoted tags

A comma inside a quoted opaque tag is a legal etagc, not a list
separator; strings.Split cut such tags into fragments that
CanonicalizeETag could turn into a false match.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: match repeated If-Match and If-None-Match header lines

Header.Get reads only the first field line, so a match on a later
repeated header was missed; join all field values like any other
comma-separated list.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer: ignore empty elements in etag precondition lists

* filer: let a present-but-empty precondition line suppress the date check

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: chrislusf <chris.lu@gmail.com>
2026-10-24 07:04:41 -07:00
Chris LuandDevin d6a68468ad volume: keep a below-minimum fsync batch open for stragglers (#12016)
* storage: hold a below-minimum fsync batch open for stragglers

When the durable-write queue drained, the worker flushed whatever it
held, so low-concurrency fsync writes collapsed to one sync per
request -- a seek plus a rotation on rotational media no matter the
payload. Once the queue empties with the batch still below the
configured minimum bytes and requests, the batch now stays open up to
SyncBatchOption.Wait, armed once per batch so the added latency stays
bounded. DiskLocation carries the option and defaults it; Store exposes
a setter the server calls once at startup.

* volume server: expose the fsync batch window as -fsync* flags

-fsyncMinBatchBytes, -fsyncMinBatchRequests and -fsyncBatchWait carry
the operator's batching floor into every disk's SyncBatchOption.
-fsyncBatchWait=0 restores the previous flush-on-drain behavior.

* server/mini: register the fsync batch window flags

weed server and weed mini build their own VolumeServerOptions and left
the three new fsync flag pointers nil, so startVolumeServer panicked
dereferencing them at startup.

* storage: bound the fsync batch window by Wait while the queue refills

Once the straggler timer was armed, a steady trickle of queued writes
kept taking the len(requests)>0 continue path, so the first write could
wait past Wait until the size or count cap flushed the batch. Track the
armed deadline and flush once it passes even with requests buffered.

* storage: sync the straggler test on the worker taking the first write

The fixed sleep let both writes queue together (passing under the old
flush-on-drain behavior) or the straggler arrive after the window closed
(flaky failure). Poll the drained queue instead so the second write can
only land inside the armed window or a later batch.

* volume server: default the fsync batch window when its options are unset

weed server and weed mini build VolumeServerOptions without the fsync
flag pointers, and outside embedders can too. Resolve the batch window
from the defaults so startVolumeServer never dereferences nil options.

* storage: drive the batch-window test through an injectable timer

The window is armed by the worker now, so the straggler provably joins
an open batch and expiry is fired by hand instead of racing a sleep.

* seaweed-volume: accept -fsyncMinBatch/-fsyncBatchWait flags

Same flags and defaults as weed volume so the Rust server's durable
write batching can be tuned identically.

* seaweed-volume: hold a below-minimum fsync batch open for stragglers

Mirrors the Go worker's batchWait: once the queue drains under the
minimum batch, the window stays open for stragglers and an expired
window still bounds the batch while the queue refills. Paused-time
coverage replaces any sleep-based assertion.

* seaweed-volume: sync the straggler test on the armed batch window

The test fired both writes together, so the straggler could already be
queued when the worker drained — a regression that drops mid-window
writes from the open batch would still pass. Wait until the worker
arms the window before sending the last write (Go's test polls the
drained channel for the same guarantee).

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* seaweed-volume: reject fsyncMinBatchBytes that does not fit usize

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:04:05 -07:00
b0e6f7a096 s3: allow up to 50 tags in a bucket tag set (#12030)
* s3: allow up to 50 tags in a bucket tag set

PutBucketTagging validated the tag set with the object tag rules, which
allow 10 tags. AWS allows a bucket up to 50 tags, so tag sets of 11 to 50
tags were rejected with InvalidTag. Validate bucket tags against 50 and
keep the limit of 10 for objects and S3 Tables.

A tag set that repeats a key is rejected with InvalidTag, as AWS does,
instead of keeping one of the values, so 51 tags with a repeated key no
longer pass as 50.

* s3: reject duplicate tag keys in PutObjectTagging

The tag set collapses to a map, so a repeated key was silently kept once.\nMatch the bucket tagging check and answer InvalidTag instead.

* s3: reject empty tag keys

AWS requires a tag key to be at least one character; the shared\nvalidator only checked the 128-char upper bound.

* s3: reject duplicate tag keys in the tagging header parser

parseTagsHeader (the x-amz-tagging path used by CopyObject with
TaggingDirective=REPLACE) silently kept the last value when a key
repeated, while the PutObject and multipart header paths and the new
XML checks return InvalidTag for the same input. A repeated key after
URL decoding now fails the same way.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: validate x-amz-tagging tags on PutObject before storing the entry

The header path only rejected duplicate keys, so an upload could store
more than ten tags or keys and values that fail S3 tag rules. Run the
parsed set through ValidateTags like the XML tagging handlers do.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: validate x-amz-tagging tags on multipart initiate

ParseS3Metadata stored header tags with no tag-set validation, the same
gap putToFiler had. Multipart uploads now reject oversize or malformed
tag sets with InvalidTag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: count tag length in characters, not bytes

* s3: validate object tags before uploading chunks

* s3: reuse parseTagsHeader and drop raw header from warnings

---------

Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:04:01 -07:00
de02158212 filer: let NewChunkStreamReaderFromFiler fall back to the master on a cache miss (#11964)
refreshUrls invalidates every cached location of a volume after a failed
read and re-looks it up through the reader's lookup function. With the
cache-only masterClient.LookupFileId, one failed read to one replica left
the volume missing from the filer's cache until restart, so metadata-log
replay skipped it. Use GetLookupFileIdFunction like the other read paths.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-24 07:03:47 -07:00
Chris LuandDevin 8844d9823f iam/oidc: wrap the jti replay cache in its own type (#12017)
* iam/oidc: wrap the jti replay cache in its own type

The replay-protection state lived as ten fields and nine methods spread
across OIDCProvider. jtiReplayCache now owns the store, count, bound,
sweep scheduling, and cleanup worker; the provider keeps a single jti
field and delegates configure, consume, stop, adopt, and merge.

* iam/oidc: lazily allocate the jti replay cache

A zero-value OIDCProvider left jti nil, so Initialize, single-use
validation, Shutdown, and the replay-state handoffs panicked on a nil
cache. Allocate the cache through a sync.Once-guarded getter instead.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:03:42 -07:00
Chris LuandDevin 3e5aa8c3b3 topology: mark replica quarantined before recording the pair (#11983)
* topology: serialize quarantine records with their layout marks

A quarantine publishes a (node, vid) record and marks the replica read-only at the layout level; release and heartbeat flag updates must observe that pair as one state. Store-then-mark let a concurrent release delete the record before the mark landed, stranding the surviving replica read-only; mark-then-store let a heartbeat clear the mark before the record was visible, returning a conflicting replica to the writable list.

quarantineLock now covers every record<->mark pair: flagDuplicateVolumeId's quarantine, releaseResolvedQuarantine's undo, the isQuarantined-gated marks on registration, and the quarantine-folded read-only flag writes on heartbeat paths. Remote server pushes stay outside the lock; a push that lands after the record was released is undone, and a failed release push re-adds the record for a later retry.

The release test now waits for in-flight async flags to settle instead of asserting on a single read.

* topology: fold quarantine into RegisterVolume's flag rebuild

RegisterVolume rewrites every replica's location read-only flag from what
its node last reported, and reported flags deliberately exclude quarantine.
Any registration of a shared vid -- a reconnected replica, a new copy, the
self-heal and delta-heartbeat re-register paths -- therefore cleared the
quarantine mark on every replica in the location, while the gate afterwards
only re-marked the registering node. Two call sites had no gate at all.

RegisterVolume now takes an isQuarantined predicate and evaluates it under
accessLock for every replica it touches, so the mark is never dropped in
the first place: a release unmarks after the rebuild, and a quarantine
mark re-applies on the next rebuild, with no window between.

* topology: keep quarantine when a vacuum commit re-adds a replica

SetVolumeAvailable re-adds the replica into the location list; a re-added
slot starts with a cleared read-only flag, so a quarantined replica came
back flag-free and the next ensureCorrectWritables could return a
conflicting volume to the writable list. Pass the quarantine state in and
set the flag back under the same accessLock, with the caller holding
quarantineLock so release cannot interleave between the check and the mark.

* topology: merge quarantine records instead of overwriting them

enforceQuarantine wrote back the state copy it loaded before its remote
push, so a concurrent writer's enforced=true was lost: the server stayed
read-only while the record said unenforced, and a later release skipped
the writable push it owed. The write-back now reloads the record under
quarantineLock and merges -- enforced wins from either side, reported is
taken from the newer pre-enforcement observation only while no push has
landed yet.

All three compensating writable pushes had the same hole in reverse: a
failed un-quarantine left the server read-only with no record to retry
from, and release's restore blindly overwrote any record a fresh flag
pass had created in between. They now go through restoreQuarantineRecord,
which adds the record only when absent and otherwise sets enforced on
the current one, preserving its reported flag.

* topology: hold quarantineLock over registration's quarantine check-and-mark

RegisterVolume reads the quarantine record and marks the replica
read-only under accessLock, but the pair ran without quarantineLock.
A release could push writable, watch the record get re-read, and then
delete it: the replica stayed read-only with no record left to clear,
and unchanged writable heartbeats skip SetReplicaReadOnlyFlag so the
mark never repaired. Holding quarantineLock around the check-and-mark
in all three production callers serializes it against the release
clear-and-delete.

* topology: serialize remote quarantine pushes behind a record recheck

releaseResolvedQuarantine deletes the record under quarantineLock and
sends the writable request after unlock. A conflict flagged while that
request is in flight creates a fresh record and marks the replica
read-only again, and the older writable request can still land last,
leaving the server writable under an active quarantine. The reverse
flap strands a stale read-only mark the same way.

Route every QuarantineVolumeFn call through pushQuarantine, which
holds quarantinePushLock across the remote call and re-checks the
record under quarantineLock right before sending: a read-only push
requires the pair to exist, a writable push requires it to be gone.
A skipped push means the decision already moved on, so the final
server state always follows the newest record.

* topology: serialize quarantine pushes per replica, not globally

A single quarantinePushLock held across the remote mark call let one slow
volume server (up to the 10s client timeout) delay pushes for unrelated
replicas. Keying the lock by (node, vid) keeps pushes to one replica
ordered in record-decision order while pushes to others run in parallel.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* topology: restore the layout read-only mark with a released record

A failed writable push re-armed the quarantine record but left the
replica marked writable in the layout, so PickForWrite could route to a
replica the server still holds read-only.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* topology: wait out async duplicate checks in the release-retry test

TestFailedReleasePushRestoresQuarantineMark asserts layout marks around a
release pass, but the two RegisterVolumeLayout calls each start a
background flagDuplicateVolumeId that can still be mid-flight. A late
mark could land after the final release and fail the writable assertion
even though the code behaved correctly. Track the async passes in a
WaitGroup and drain them before the explicit flag call so the rest of
the test is deterministic.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* topology: keep quarantine when a replica's layout key is unreadable

* topology: keep a failed writable push's restore ordered under the push lock

* topology: retry the quarantine release on every duplicate check

* ci: isolate Java and S3 test servers on persistent runners

* ci: drop the duplicated -ip flags in the lifecycle test server

The bind flags were already on the command; the isolation pass added a
second pair.

* topology: retry a failed quarantine writable push on the next report

* topology: retry pending quarantine releases off the leader pulse

A failed writable push re-arms the quarantine record with a retry that only
enforceQuarantine fires, and that runs only for volumes a heartbeat carries.
Once the server has reported the replica read-only it drops out of the delta
reports, and restoreQuarantineRecord keeps the stored flag matching the
server's so the digests agree and no full list is requested - the pending
release then never retries and the idle volume stays read-only forever. Run
the same release pass from the leader's pulse loop so a pending retry no
longer waits on a report.

* topology: exercise the pulse-driven quarantine release retry

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:00:45 -07:00
5700aa5e91 s3: evaluate bucket CORS for path-style OPTIONS on the bucket root (#12028)
* s3: evaluate bucket CORS for path-style OPTIONS on the bucket root

A path-style preflight for /bucket (no trailing slash) missed the bucket
OPTIONS route, whose template is /{bucket}/, and was answered by the global
OPTIONS handler from the global AllowedOrigins instead of the bucket CORS
configuration. Register the bucket OPTIONS route without a path matcher, and
keep OPTIONS for the health probe paths on the service-level handler ahead of
each path-style router.

Fixes #12025

* test: drop the issue reference from the bucket-root preflight test comment

---------

Co-authored-by: Wleeaf <179125857+wleeaf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 06:58:59 -07:00
Chris LuandDevin 9281ed8007 master: never auto-admit raft peers on a single-node configuration (#12021)
* master: never auto-admit raft peers on a single-node configuration

A master started without cluster peers (peers=none, or no peers so the
list holds only itself) declares a one-node cluster, yet any foreign
master that registered over KeepConnected was admitted as a second
voter. The resulting two-voter cluster cannot reach quorum when the
foreign peer is unreachable, wedging a previously healthy single
master. Gate raft admission on having been configured with cluster
peers; masters started with a peer list keep the announce-to-join
flow used to grow the cluster.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: source SingleMaster from -peers=none, not the peer count

A master started with an empty peer list ends up with a one-peer map
containing itself, and NewMasterServer inferred declared single-master
mode from that count. The master still mints a raft cluster through the
bootstrap path, but AdmitRaftPeer then ignores every later joiner, so a
deployment that grows from one master to several can never form a
quorum. isSingleMasterMode already spells out the declared case
(-peers=none); populate MasterOption.SingleMaster from it instead.

* master: cover clustered raft admission with a real in-memory leader

The previous clustered test had no raft leader and asserted nothing, so
an AdmitRaftPeer that always returned early still passed. Bootstrap a
single-node hashicorp raft over the in-memory transport, wait for
leadership, and assert the leader reports and admits a missing peer.

* master: return the raft admission result and assert committed join

AdmitRaftPeer ran AddVoter on a detached goroutine, so the only observable
state was an in-flight marker deleted on success and failure alike, and the
clustered test could never tell whether the join worked. Admission now
hands back a result channel; the test runs a connected second in-memory
node and asserts the peer sees itself a committed voter.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 06:58:16 -07:00
Chris LuandDevin fd3b437e04 s3api: write the null version's object-lock metadata to the plain entry (#12033)
* s3api: write the null version's lock metadata to the plain entry

PutObjectRetention and PutObjectLegalHold with versionId=null loaded the
pre-versioning entry but wrote the update under <key>.versions/v_null, so
the acknowledged lock never bound the object it was read from. Resolve the
entry and its write path in one helper that applies the same null-version
rule as getSpecificObjectVersion.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: cover the null-version lock write path

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: persist object-lock metadata with an in-place entry update

mkFile rebuilds the target entry from chunks alone, so a lock write on a
plain (null-version) entry discarded attributes, inline content, and the
remote reference. UpdateEntry keeps the whole fetched entry, matching how
the tagging and ACL writers persist metadata.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 06:57:02 -07:00
Chris LuandDevin 689ea6ea3e release: digest-bound provenance and SBOM attestations for container images (#12024)
* release: emit digest-bound provenance and SBOM attestations

Per-platform builds now record SLSA provenance at mode=max and a
BuildKit SBOM, so the builder identity and resolved dependencies are
captured instead of the empty predicate the 4.48 build record carried.
After the multi-arch index is assembled and signed, the merge job
publishes GitHub artifact attestations bound to the merged digest and a
syft SPDX SBOM of the image, giving digest-pinning consumers a
verifiable record on the exact index they pull.

* release: keep unfixed findings in the trivy SARIF upload

ignore-unfixed dropped every finding without an available patch, so the
uploaded report understated the image's exposure. The scan stays
report-only; only the completeness of what is uploaded changes.

* release: generate and attest SBOMs per platform digest

An index-wide syft scan only describes its default platform, so the SBOM
attested to the merged multi-arch digest misrepresented every other
manifest. Scan and attest each platform digest instead, which also drops
the sbom-action upload defaults that raced release-asset permissions and
artifact names across the merge matrix.

* release: resolve per-platform index to its image manifest before syft/cosign

push-by-digest records point at OCI indexes (image + attestation
manifests), so scanning or attesting the index digest picks the
runner's platform and fails outright for the other arches under
set -e. Resolve the non-attestation manifest inside each index and
bind the SBOM attestation to the image digest a same-platform pull
of the merged index actually returns.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 06:47:24 -07:00
Chris LuandDevin be081b49b7 notification: named webhook targets for per-path event routing (#11925)
* notification: honor the config prefix in the multi-queue guard

validateOneEnabledQueue checked "<name>.enabled" without the
notification prefix, so the fatal-on-two-queues guard never saw the
real keys and could never fire.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* notification: support named webhook targets under [notification.webhook.<name>]

The webhook queue only accepted one endpoint, so events could not be
routed to different services by path prefix. Each enabled
[notification.webhook.<name>] subsection now spins up its own target —
own filter, http client, retry middleware and worker budget — published
on a per-target topic so every matching endpoint gets each event exactly
once. The flat [notification.webhook] section keeps working as the
unnamed target, and named subsections alone now activate the queue.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* notification: require enabled for the flat webhook target

buildTargets included the flat section whenever an endpoint was set, so a
disabled [notification.webhook] block — which the scaffold ships with a
placeholder endpoint — would start receiving events as soon as any named
target activated the queue.

* notification: resolve named target enabled flags through config

namedTargetPrefixes read the raw map value, which never sees environment
variable overrides; GetBool resolves WEED_* env bindings, so a named
target disabled in the file can now be switched on by
WEED_NOTIFICATION_WEBHOOK_<NAME>_ENABLED.

* notification: cancel the webhook queue only after all workers stop

Each target's router goroutine deferred the shared cancel, so the first
worker to exit tore down the queue context for every other target and
the dead letter logger.

* notification: isolate webhook targets behind per-target queues

SendMessage published to each target's topic inline, so a subscriber
channel that filled up blocked every later target and the filer's
notification path with it. Each target now drains its own inbox through
a forwarder goroutine; SendMessage fails fast on a full inbox, matching
the "throw message" contract the filer already applies to errors.

* notification: name the webhook target in failure and dead letter logs

Every target dead-letters to the same topic, so without the name there
is no way to tell which endpoint failed or needs a replay.

* notification: assert delivery in the named targets test

The test only exercised SendMessage; it now records requests on both
endpoints and checks matching, non-matching, and unrouted keys reach
the right servers exactly once.

* notification: wait for the webhook router before accepting events

* notification: apply backpressure instead of dropping a full webhook inbox

* notification: bound queued and in-flight webhook deliveries per target

* notification: release the pending token when shutdown races the enqueue

A SendMessage that wins the pending select as the queue is canceling
could block forever on a full, undrained inbox.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* notification: stop waiting for a router that failed to start

A router that exits before reaching Running() would leave initialize
blocked on a context nothing had canceled yet.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* notification: route webhook events past saturated targets

A full pending reservation made SendMessage wait on the slow target
before it ever reached the healthy ones after it, coupling every
endpoint's latency to the most congested one. Targets that can take a
reservation now get the event immediately; saturated targets defer to a
second pass that still blocks for a slot, so backpressure is kept and no
event is dropped.

* notification: retry webhook deliveries before dead-lettering

PoisonQueue must wrap the retry middleware: added after it, the poison
middleware intercepts the first handler error, republishes to the dead
letter topic, and returns nil — the retry chain never sees a failure, so
max_retries/backoff settings were dead code and every delivery failed
straight to dead letter on the first attempt.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 02:12:17 -07:00
fb3fb9dcb8 s3api: support s3:RequestObjectTagKeys in bucket policies (#11993)
* s3api: support s3:RequestObjectTagKeys in bucket policies

Condition operators accept the ForAllValues: and ForAnyValue: qualifiers,
and s3:RequestObjectTagKeys holds the keys of the tags taken from
X-Amz-Tagging wherever s3:RequestObjectTag/<key> is set.

CopyObject no longer stores an empty-key tag for an empty segment in
X-Amz-Tagging, so it keeps the key set a policy has checked.

Fixes #7679

* s3: parse tagging header like the tag handlers, not url.ParseQuery

url.ParseQuery stops at 10,000 parameters, so an X-Amz-Tagging header
padded with empty & segments made requestObjectTags return nil: the copy
handler still stored the denied tag while every s3:RequestObjectTag*
condition saw nothing. Scan non-empty segments the way parseTagsHeader
does so the policy sees the same tags the object gets.

* test: gofmt s3tables test files after #11997

The format check fails on these files on master as merged, so every
PR's S3 Tables Format Check goes red until they are re-formatted.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 02:04:39 -07:00
Chris LuandDevin 7a9a9f10af ci: invoke pip via python -m in s3tests clone steps (#12010)
* ci: invoke pip via python -m in s3tests clone steps

The self-hosted runner reaches the Clone s3-tests step with the
setup-python toolcache interpreter on PATH but no pip shim, so the
bare 'pip install' exits 127. python -m pip resolves through the
interpreter that is already on PATH, matching the helm_ci workflow.

* ci: invoke pip via python3 -m in s3tests clone steps

The adjacent fix_s3_tests_bucket_conflicts step already runs python3, so
python3 is the interpreter guaranteed on the self-hosted PATH; a missing
python alias would fail the same way the missing pip shim did.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 23:05:56 -07:00
ecd778a4a0 iceberg maintenance: a merged file with null row ids names no first_row_id (#11973)
* iceberg maintenance: a merged file with null row ids names no first_row_id

Compaction stamps a merged v3 data file with the _row_id of its first row as
first_row_id (mergedFileFirstRowID). When the bin mixed rows that carry a
materialized _row_id with rows that carry none (a data file with no
first_row_id, as a v2 table's files are after an upgrade to v3, gives null
ids), the null rows then inherit first_row_id + their position in the file:
ids that other rows already hold, so two rows answer to the same row id.

A merged file now names a first_row_id only when its footer shows no null
_row_id in any row group. Otherwise the commit assigns the file a fresh range
from the table's next-row-id: the null rows get new, unused ids and the
materialized ones keep theirs (a materialized value overrides inheritance).

Test: TestMergedFileFirstRowIDOnlyWithoutNullIDs ("materialized first, a null
later" fails without the fix).

* iceberg maintenance: name no first_row_id when the merged file has a null _row_id

mergedFileFirstRowID stamped the merged file with its first row's id
without checking later rows. A null _row_id in a merged file inherits
first_row_id+position, an id another row already holds, so the commit
produced two rows answering to the same row id.

Check the footer's null counts for the _row_id column across row groups
first: any null means the file is committed without a first_row_id, so its
nulls take fresh ids from the table's next-row-id while materialized
values still override inheritance.

* shell: retry filer discovery until master knows a filer

A shell started against a freshly booted cluster (weed mini, the
s3tables/lance test harnesses) can list cluster nodes before the filer's
first heartbeat lands on the master. The empty list left FilerAddress
unset and the next command died with "dial tcp: missing address". Poll
ListClusterNodes until a filer appears or the connect timeout expires.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: discover the filer on first use instead of at startup

The startup retry loop made every weed shell invocation wait out the
connect timeout on clusters that have no filer, even for commands that
only talk to the master. Discovery now runs once, on the first command
that actually needs a filer, keeping the retry-until-registered fix for
the fresh-cluster race without stalling master-only commands.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: stop filer discovery retries on non-transient errors

A master that keeps answering ListClusterNodes with a permanent error
(permission denied, unimplemented) cannot gain a filer by retrying, so
discovery returns instead of polling until the deadline.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-10-23 22:53:43 -07:00
Chris LuandDevin 6a34d82f05 shell: retry filer discovery until master knows a filer (#11989)
* shell: discover the filer on first use instead of at startup

The startup retry loop made every weed shell invocation wait out the
connect timeout on clusters that have no filer, even for commands that
only talk to the master. Discovery now runs once, on the first command
that actually needs a filer, keeping the retry-until-registered fix for
the fresh-cluster race without stalling master-only commands.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: stop filer discovery retries on non-transient errors

A master that keeps answering ListClusterNodes with a permanent error
(permission denied, unimplemented) cannot gain a filer by retrying, so
discovery returns instead of polling until the deadline.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: explain an empty filer list when discovery times out

* shell: retry filer discovery while the address is still empty

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 22:52:51 -07:00
dependabot[bot]anduser.email b86b2f3cee build(deps): bump github.com/go-chi/chi/v5 from 5.3.1 to 5.4.0 (#12004)
Bumps [github.com/go-chi/chi/v5](https://github.com/go-chi/chi) from 5.3.1 to 5.4.0.
- [Release notes](https://github.com/go-chi/chi/releases)
- [Changelog](https://github.com/go-chi/chi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-chi/chi/compare/v5.3.1...v5.4.0)

---
updated-dependencies:
- dependency-name: github.com/go-chi/chi/v5
  dependency-version: 5.4.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
2026-10-23 22:47:23 -07:00
57a122fbcf filer.remote.sync: dispatch an update event by the side of the mount it touches instead of slicing the mount prefix off a path outside it (#11958)
* filer.remote.sync: dispatch an update event by the side of the mount it touches instead of slicing the mount prefix off a path outside it

The daemon subscribes to its mount prefix and to /etc/remote. An update event
whose path lies outside the mount still reaches processUpdateEvent: an update
of the /etc/remote directory entry itself (which fs.meta.load of a mapping dump
re-creates) through the /etc/remote prefix, or a rename across the mount
boundary through the path on the inside. processUpdateEvent then maps both
paths to remote keys by slicing the mount prefix off, which panics with
"slice bounds out of range [20:11]" on a path shorter than the mount and names
a key in the wrong tree on a longer one. The event sits in the replay window,
so every restart dies on it and the persisted offset never moves.

Before dispatch, an update event is now mapped onto the side the mount covers:
a rename into the mount is the creation of the new entry, a rename out of it
the deletion of the old entry's object (kept when the entry is remote-only and
the object holds its only copy), an event with neither path inside is not the
mount's and is dropped, and an update with both paths inside is unchanged.

* filer.remote.sync: extract processCreateEvent

Moves the create branch of eachEntryFunc into a function, matching
processDeleteEvent and processUpdateEvent, so the create path can be
exercised directly in tests. No behavior change.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.remote.sync: upload a create reshaped from a rename into the mount

A file synced under the mount keeps its RemoteEntry across renames. The
rename out deletes the remote object, and the rename back in reshapes to a
create whose inherited stamp still matches the entry, so shouldSendToRemote
skipped the upload and the new key stayed empty. The reshaped create now
sends unconditionally: the stamp describes the old key, not this one.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.remote.sync: a create reshaped from a rename into the mount writes an empty entry as a zero-byte object

* filer.remote.sync: a create reshaped from a rename sends a stamped entry truncated since its sync

A stamped entry still reports remote-only once truncated to zero (the
stamp's RemoteSize stays positive), so the remote-only exemption skipped
it and the new key got no zero-byte object. Gate the exemption on
isDroppedLocalCopy: only an entry whose size still matches its stamp has
its content already at the new key via the rename's remote copy.

* filer.remote.sync: recover a superseded create reshaped from a rename-in

A create reshaped from a rename into the mount whose write is superseded
was skipped like a plain create, but no later event covers this
destination: remote.uncache accepts the rename's inherited RemoteEntry
stamp and drops the local chunks, and the update it logs skips the
upload through isDroppedLocalCopy, while the source mount's sync still
deletes the source object. The destination object is then missing and
the content exists nowhere.

recoverSupersededCreate finishes the event from the entry the filer
holds now: a rewritten entry is left to its own event's upload or
uploaded here on the stamp edge, and a remote-only entry is completed by
copying the object its stamp describes -- resolved through a
pending-rename marker under this mount, or through the filer's mount
mappings to the source mount's remote location.

---------

Co-authored-by: james <james@medable.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: chrislusf <chris.lu@gmail.com>
2026-10-23 22:43:39 -07:00
Chris LuandDevin 5e01caa522 master: cap parallel vacuums per rack via master.vacuum.max_parallel_per_rack (#11935)
* master: cap parallel vacuums per rack via -maxParallelVacuumPerRack

Hosts that run one volume server per disk under a single rack label could
not bound host-level vacuum I/O: maxParallelVacuumPerServer limits each
process, so every disk on the machine compacted at once. A second quota
map keyed on rack id debits alongside the per-server one, so at most N
volume copies compact in parallel within a rack. Defaults to 0 (no
limit); the flag is wired on master, server, mini and master.follower.

* master: resolve a data node's rack key once per vacuum sweep

A volume server that unregisters mid-sweep loses its parent node, so the
quota credit path calling dn.GetRack() panicked and crashed the master.
Rack names are also local to their data center, so same-named racks in
different centers shared one quota bucket. Each node's rack key is now
captured when the limiters are built and carries the data center id, so
checks, debits and credits never touch the live parent pointer.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: reserve vacuum quota per needed copy count, not per copy

Each copy of a volume checked the shared server/rack quota independently,
so several same-rack replicas all passed against one remaining slot and
the debits drove the bucket negative. Needs are now counted per server
and per rack first; a volume whose own copies exceed a cap reserves the
whole bucket once it is free instead of pending forever.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master.follower: register -maxParallelVacuumPerRack as a flag

It was only a fixed default, so passing the flag on the command line
failed flag parsing.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: key the rack vacuum quota by rack node identity

Joining data center and rack names with "/" still let two different racks
share one bucket when the names themselves contained slashes. Key the
limiter by the rack node instead, which is unique per rack regardless of
naming.

* master: keep the tiered-replica vacuum test through the rebase

* master: document 0 as no limit for -maxParallelVacuumPerServer

The clamped per-server need makes a non-positive cap mean "no limit",
matching the new rack flag; say so in the flag help.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: key the rack-key cache by data node identity

Node ids are only unique within a rack, so two servers that share an id
in different racks would have reused the first server's cached rack and
escaped the per-rack vacuum cap.

* master: read the per-rack vacuum cap from master.toml

-maxParallelVacuumPerRack becomes the master.max_parallel_vacuum_per_rack
config key, so the new master limit is set the same way as other master
options instead of a flag wired into every command that embeds a master.

* master: let master.max_parallel_vacuum_per_server override the flag

The released -maxParallelVacuumPerServer flag keeps working, but a value
in master.toml now wins when set, matching how master.mdir already
overrides -mdir.

* master: group the vacuum caps under master.vacuum in master.toml

Rename master.max_parallel_vacuum_per_rack to master.vacuum.max_parallel_per_rack
and master.max_parallel_vacuum_per_server to master.vacuum.max_parallel_per_server,
so the keys sit in one subsystem table like master.volume_growth and
master.maintenance.

* master: deprecate -maxParallelVacuumPerServer for master.toml

The flag still works - and is overridden by
master.vacuum.max_parallel_per_server when set - but the help text now
points operators at the config key, matching the codebase style of
help-text-only deprecation.

* topology: read node parent atomically

UnlinkChildNode clears a data node's parent under the rack's lock while
readers like the vacuum sweep's rack capture reach it with no shared
lock, so a torn interface read could crash the master. Store the parent
in an atomic.Pointer and nil-guard the detached walk in
DataNode.GetTopology.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 22:41:36 -07:00
Chris Lu a1b3131bf6 test: handle unsupported catalog clients in S3 Tables CI (#12023)
test: distinguish unsupported catalog clients from S3 Tables failures
2026-10-23 22:40:54 -07:00
Chris Lu aba0cb7b2a ci: isolate S3 lifecycle mini from LAN clusters (#12022) 2026-10-23 22:31:15 -07:00
Chris Lu 81e0859c12 test(s3 remote cache): wait for the master to list the filer before shell commands (#12018)
A fresh single-filer mini registers its filer only after the ~10s
empty-membership confirm window, and weed shell locates the filer via
the master's listing, so remote.configure sent right after the S3
health check races it. Ports the wait from the enterprise test to the
OSS copy.
2026-10-23 20:54:52 -07:00
Chris LuandDevin 0d00e92178 iceberg maintenance: match percent-decoded keys when checking file references (#11978)
* iceberg maintenance: match percent-decoded keys when checking file references

Writers may percent-escape partition directory names in the file paths they
record in manifests (e.g. "part%3D1"), while the S3 layer decodes the key
before storing the object. remove_orphans compared the escaped reference to
the decoded filer path, counted the live file as an orphan, and deleted it
once it passed the age window — readers then got NoSuchKey on the next scan.

Match both spellings of every referenced path in remove_orphans, apply the
same coverage to the kept-file set in expire_snapshots, and delete expired
files at their decoded key so the object the S3 layer stored is the one
removed.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iceberg maintenance: retry filer lookups at the percent-decoded key

A manifest reference carrying an escaped segment resolves to a filer name
that does not exist, because the S3 layer decoded the key on write. When a
lookup misses and the reference contains escapes, retry once at the decoded
spelling so maintenance can still read the file.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iceberg maintenance: resolve referenced files verbatim-first, decoded fallback

filer_pb.LookupEntry reports misses as ErrNotFound rather than a gRPC
NotFound, so the escaped-reference retry never fired against a real filer.
Both maintenance reads and expired-snapshot deletes now resolve through one
verbatim-first/decoded-fallback lookup, and expiry refuses to delete a
resolved path outside the table's data directory.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iceberg maintenance: test expiry of a file stored at a literal percent key

* iceberg maintenance: resolve referenced files in one filer lookup

lookupFileByIcebergPath re-fetched the entry resolveFilerPath had just
looked up, doubling the RPC per referenced file during compaction and
expiration. The resolver now returns the entry alongside the path so a
hit costs one lookup and a decoded fallback two.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iceberg maintenance: keep every decoded spelling of a referenced path

The kept-reference sets recorded a path plus a single decode of it, but
lookupResolvedEntry decodes recursively until the spelling is stable. A
doubly-escaped reference could resolve to a spelling absent from the set
and be deleted while still referenced. addFilerPathSpellings now walks
the same decode chain the resolver does.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* iceberg maintenance: keep every decoded spelling of a referenced path

* iceberg maintenance: test expiry keeps a doubly-escaped kept reference

* iceberg maintenance: clean decoded spellings in the referenced-path set

* iceberg maintenance: test expiry keeps a kept reference resolved via decoded parent segment

* iceberg maintenance: decode only the reference in the spelling set

Decoding the composed filer path also decoded the table directory, so a
table at a literal percent-escaped path produced a kept set missing the
spelling the resolver returns, and expiry could delete a live file.

* test: gofmt the s3tables cluster helpers

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 20:49:49 -07:00
Chris LuandDevin 8541f959a9 filer.conf: merge fsync on presence like worm (#11953)
* filer.conf: merge fsync on presence like worm

fsync is a pure preference, so OR-merging it meant a nested rule could
never turn a parent's fsync off. Make PathConf.fsync optional and merge
it most-specific-wins, matching worm: an unset rule inherits, an explicit
value overrides in either direction. readOnly and disableChunkDeletion
stay OR'ed — they are fail-closed locks quota enforcement depends on.

Pre-version-2 configs were written with EmitUnpopulated, so every rule
carries an explicit fsync=false that meant nothing; it is dropped to
unset on load so it does not read as an opt-out.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.conf: test fsync override survives a save/reload round trip

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 20:45:46 -07:00
Chris LuandDevin 673d1a02e0 operation: euthanize pooled upload conns that stall far below a fast host (#12003)
* operation: euthanize pooled upload conns that stall far below a fast host

A pooled keep-alive conn can wedge and drain the request body at a trickle
while fresh conns to the same volume run at full rate; every later upload
routed onto it stalls for tens of seconds, indefinitely. Track per-host
healthy-rate uploads (>=1MiB/s on a >=256KiB body) and, on a host that was
fast within the last minute, cancel an upload still in flight past
max(10s, size/256KiB/s): the transport discards the wedged conn and the
existing retry dials a fresh one. A watchdog kill also clears the fast mark
so the retried attempt runs unbounded, and hosts that never prove fast keep
today's unbounded behavior - a chronically slow link is never penalized.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* operation: keep the stall watchdog armed through the response read

uploadDone was set as soon as Do() returned, so a conn that accepted the
request body then wedged on the reply escaped the watchdog and could
park the upload for hours — the same failure this watchdog exists to
kill. Deferring the done-store keeps the timer live until the function
returns, so a stalled io.ReadAll is cancelled and retried like a
stalled write. The redundant post_err recheck goes with it.

* operation: pin watchdog coverage of a stalled response read

* operation: disarm the stall watchdog for an upload's remaining attempts

Once the watchdog has cancelled an attempt, a concurrent healthy upload
to the same host can restore the fast mark during the retry backoff.
The re-armed watchdog would then cancel the retry too, and a large
upload that merely exceeds the floor could burn every attempt. Track
the watchdog kill on uploadStatusError and carry a
stallWatchdogDisarmed flag through retriedUploadData so the rest of
that upload's attempts run unbounded; other uploads to the host keep
their watchdog.

* operation: pin watchdog disarm on retry, fresh-dial, and slow-upload cases

* operation: drop the send-stall watchdog once the request body drains

A wedged pooled conn is only blameworthy while it is still trickling the
request body; once the body is on the wire the wait is server-side commit,
which the transport's response header timeout already bounds. Cancelling
then would fail a healthy write whose replication or fsync is simply slow.
Track body EOF through a wrapping reader, gate the send-side timer on it,
and re-arm the watchdog only around the response-body read, which the
header timeout does not cover.

* operation: prove a host fast only on a verified upload

A quick rejection moves bytes no faster than the wire did, so mark the
host from the body-send duration and only after the response checks out.

* operation: count an abandoned request body as sent

The transport closes a request body it never finished draining when the
server answers early, so EOF alone under-reports the send boundary and
the stall watchdog could still blame the conn during the reply wait.

* operation: scope the body-sent mark to the current attempt

A lagging Close or EOF from the first attempt's body could stamp the
shared mark after the inner retry reset it, disarming the watchdog for a
send that never happened; each attempt now points the watchdog at its
own sent timestamp.

* operation: drop only the fast mark the stall timer observed

A concurrent upload can re-prove the host while a stalled attempt is in
flight; deleting the key outright would erase that newer mark, so the
watchdog now removes only the value it armed against.

* operation: keep the stall disarm on a per-call upload option

retriedUploadData marked the caller's UploadOption, so a caller reusing it
across UploadData calls would start the next upload with the watchdog off
and lose the cancellation this adds.

* test: stall watchdog retry must fail if the conn gets cancelled

The fake retry slept through any cancellation, so a re-armed watchdog
firing mid-retry still let the upload succeed — the guard was untested.
Park on the request context like a real wedged conn so a re-armed
watchdog cancels the attempt and fails the test.

* operation: prove the fast-host mark with elapsed-to-response

Body-drain time only measures when the transport consumed the reader:
kernel socket buffers can swallow a small body instantly on a slow link,
inflating the rate and marking the host fast. The next upload then gets
its conn cancelled mid-send despite making progress. Elapsed-to-response
bounds remote speed from below — the server cannot answer before the
body lands — so the mark can only under-mark, never over-mark.

* operation: scale the stall budget off the host's measured rate

Fixed bps limits fit no link well: 256KiB/s never fires on a slow link
and waits far too long on a fast one. Record the wire rate of each
sizeable upload and let a later upload run for slowFactor times the
body's span at that rate — a wedged conn shows as an order-of-magnitude
drop on any link speed. The recorded rate uses the wire body size so a
pre-compressed payload no longer inflates the mark.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 20:34:31 -07:00
Chris LuandDevin 6e8a2cdb14 filer.backup: process metadata events through the concurrent processor (#11951)
* filer.backup: process metadata events through the concurrent processor

Each event was handled inline on the subscription loop, so one
fetch+write at a time capped the stream at a few entries per second and
a busy source kept building lag. Route events through the same
MetadataProcessor filer.sync uses: it fans jobs out behind -concurrency
while serializing same-path and subtree-barrier conflicts, and the
checkpoint now advances on the processor's contiguous watermark via
GetResumeTsNs, with Resubscribe replaying anything a failure pins.
-disableErrorRetry keeps its log-and-skip meaning by returning nil after
logging, since the processor would otherwise pin the checkpoint forever.

* filer.backup: drain in-flight jobs before the stream retry loop reconnects

FollowMetadata returns as soon as the stream ends while admitted jobs keep writing; the next doFilerBackup builds a fresh processor that cannot order its replayed writes against them, so an older upload could finish last and clobber a newer file.

* filer.backup: checkpoint the completed watermark on a ticker and at stream end

The offset callback only fires while events arrive, so work draining on a quiet stream stayed uncheckpointed until the next event and a restart recopied it. Persist the processed watermark every 3s when it advances, and once more after the post-stream drain.

* filer.backup: bound the checkpoint wait before reconnecting

setOffset has no context, so a stalled KvPut would hold persistDone and block the retry loop forever. The persist goroutine now performs the final write itself on stopPersist, and the caller waits at most 30s.

* filer.backup: carry the processed position across retry attempts

A follow attempt that ended with a pinned failure recomputed startFrom on retry, so a -timeAgo window could move past the failed event and skip it. doFilerBackup now returns the failure pin (or processed watermark) and the retry loop clamps startFrom to it.

* filer.backup: resume retries at the previous attempt's pinned position

The retry loop threaded resumeTsNs through but never consulted it, so a
-timeAgo run recomputed startFrom as now-minus-the-window on every attempt
— a failed event older than the window was skipped on reconnect. Use the
pinned position when one exists; -timeAgo and the persisted checkpoint
only choose the first starting point.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.backup: resume below the failure pin so the failed event replays

The metadata reader treats the resume position as exclusive — events at
or below it are skipped — so resuming at the failed event's own timestamp
would skip it. Back one tick off the pin.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.backup: label the progress line as the received-event rate

The counter increments when a job is admitted, not when its copy
finishes, so the old wording showed backup throughput during a burst of
slow copies while the processed watermark had not moved.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: bound the offset checkpoint RPC

setOffset called KvPut on the background context, so a stalled write
could pin a caller's drain or persistence loop indefinitely.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.backup: keep the resume pin when filer-source setup fails

A retried attempt carries resumeTsNs so the failed event replays from the
same position, but filer-source and SSE initialization failures returned 0,
so the next attempt recomputed -timeAgo and could move the window past the
pinned event.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: serialize checkpoint writes per key

A checkpoint write can outlive its caller's wait and land beside the next
subscription attempt's writes, moving saved progress backward. Serialize
writes per key inside setOffset and drop any value older than one already
written from this process.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer.backup: gofmt

* replication: serialize checkpoint writes per filer and key

A single process-wide lock made a stalled checkpoint RPC block every
stream's checkpoint, including the opposite direction of a bidirectional
filer.sync. Keeping the last-written watermark per persisted key also let
one stream's offset shadow another stream checkpointing the same key to
a different target filer. Lock and track progress per (filer, key).

* replication: keep the resume position behind a cleared failure pin

A redelivery that clears failedTs lets the processed watermark advance
past the earliest failure, but events dropped while the processor was
stopped — possibly sharing the failure's timestamp — have to replay.
Latched earliestFailedTsNs floors the resume position one tick below
that failure so reconnects still cover them.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: persist floored checkpoints through the write ordering

setOffset drops a value older than the last written, which also swallows
a deliberate regression: the resume floor, an operator-seeded offset, or
a snapshot high-water mark. allowRegress marks those callers so the
checkpoint can move backward when a failure forces replay.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: order checkpoint writes by issuance, not value

The value-based stale guard could never fire — every caller passed
allowRegress because floors and seeds are deliberate regressions — and
ordering by value cannot tell a stale high watermark apart from real
progress: a write that outlives its attempt could still land after a
floored resume and skip events owed a replay. Order by issuance instead:
a write issued before the last one to land under the key is dropped, so
late writes from a superseded attempt can no longer move the checkpoint
in either direction while deliberate regressions still apply.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: keep the resume position one tick below the frontier

* remote.sync: resume and persist through the floored watermark

* test: concurrent metadata jobs fan out up to the limit

Distinct-path events must all be in flight before any finishes — a
serial loop could only ever have one running — while the limit still
bounds the pool. Pins the property behind the backup-throughput fix.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 20:02:55 -07:00
github-actions[bot] cd562b8514 docs: regenerate star history chart 2026-10-24 02:56:42 +00:00
Chris LuandDevin 617599859b docker: build RocksDB v11.1.2 for grocksdb v1.11.1 (#12002)
The rocksdb release image fails to compile: grocksdb v1.11.1 calls
C.rocksdb_block_based_options_set_block_align,
C.rocksdb_block_based_options_set_index_block_search_type, and
C.rocksdb_block_based_options_set_separate_key_value_in_data_block, all
of which only exist in RocksDB v11.x headers. grocksdb v1.11.1's own
build script pins 11.1.2, so move every default from v10.10.1 to
v11.1.2 to match.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 18:50:27 -07:00
Chris Lu ce4c1fe31e ci: install fuse3 when runners lack fusermount3 (#12013)
Self-hosted runners do not ship fuse3, so the setuid repair step
failed with 'no fusermount3 in PATH' before the FUSE jobs could run.
Install the package instead of failing outright.
2026-10-23 18:49:04 -07:00
Chris Lu 462e4445e2 test: skip catalog client images that cannot execute on the host CPU (#12014)
New upstream releases build for x86-64-v3 (AVX2) - ClickHouse since
26.5, plus the DuckDB iceberg extension and pylance wheels - so on a
runner whose CPU lacks AVX2 the container dies on SIGILL before the
test reaches any SeaweedFS code. Probe each third-party image first
and skip on the illegal-instruction exit instead of failing; other
errors still surface normally.
2026-10-23 18:46:26 -07:00
Chris Lu 0d4e11a6e5 filer: bootstrap a timestamp-less peer entry onto a missing local path (#11975)
A zero local commit time conflated 'no local state' with 'state at the
epoch', so a missing entry was skipped whenever the sender reported no
commit time and the entry mtime was at or before the epoch. Only hold
the copy off when a local entry actually exists.
2026-10-23 18:45:09 -07:00
Chris Lu 85f2f9d2d4 ci: make docker registry and compose steps idempotent on self-hosted runners (#12012)
The registry container is started with --restart=always but never
removed during teardown, so once a machine runs the job, every later
run on that machine fails with a container name conflict. Reuse the
running container when it exists and recreate it only when missing or
broken.

Some self-hosted runners (e.g. seaweedfs-server03) also lack the
Docker Compose plugin, so install it via docker/setup-compose-action,
which is a no-op where compose is already available.
2026-10-23 18:37:36 -07:00
Chris LuandDevin 679835e4fc volume.fsck: run all filer-entry purges through the shared worker pool (#11960)
* volume.fsck: queue per-volume orphan deletes through the shared purge pool

The worker pool added for -reallyDeleteFilerEntries only covered entries
whose volume was missing entirely; the per-volume check that finds filer
entries pointing at absent needles still issued one synchronous HTTP
DELETE per orphan, which is the bulk of the work the report measured.
One pool now serves both purge phases and is drained before the empty-
directory sweep.

* volume.fsck: keep hard link ids in the filer fid file

Orphans queued by the per-volume check must hash into the pool by their
hard link id just like the missing-volume path, so two paths sharing an
id still serialize on one worker. The fid file now stores the id beside
each path and readFilerFileIdFile hands it to the caller.

* volume.fsck: cover orphan purge queuing and hard link id round-trip

* volume.fsck: fail on a truncated hard link id in the fid file

A .fid record cut short inside the hard link id length was emitted with a
nil id, so its purge could hash to a different worker than the entries it
shares the link with. Read the length strictly like the header and path.

* volume.fsck: drain the save consumer before returning a root genFn error

The early return left the save goroutine blocked on outputChan while the
caller ran finishPurge, so a late queuePurge could send on a closed
channel and panic.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* storage: skip EC shard-size encoding tests when disk space is low

TestCalculateExpectedShardSizeWithRealEncoding writes a .dat file plus
14 EC shards per subtest (~1.3GB peak for the 512MB case) into the temp
dir. Shared CI runners can run low on space while other test packages
write to /tmp in parallel, and the encode then dies with ENOSPC
mid-write, surfacing as a test failure. Check free space up front and
skip the subtest instead.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 18:33:51 -07:00
3810fb1123 s3: url-encode ListMultipartUploads results for encoding-type=url (#11998)
* s3: url-encode ListMultipartUploads results for encoding-type=url

The response echoed EncodingType=url but returned raw upload keys, and
XML 1.0 cannot carry control characters, so the encoder replaced them
with U+FFFD. The listed key then matched no upload: AbortMultipartUpload
with it answered 404 NoSuchUpload, leaving such uploads uncleanable.

Encode the key-named elements AWS encodes for this API (Key, Prefix,
Delimiter, KeyMarker, NextKeyMarker) the way ListObjects encodes its
keys when encoding-type=url is requested. Upload ids stay opaque, and
responses without encoding-type=url are unchanged.

Fixes #11991

* s3: echo EncodingType only when encoding-type=url is honored

A request with an unrecognized encoding-type echoed the value back in
<EncodingType> while the keys stayed raw, matching neither AWS (which
omits the element unless url encoding applies) nor the ListObjects
handlers, which gate the echo on the parsed url flag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 18:21:26 -07:00
Chris LuandDevin 12d0537215 test: gofmt s3tables test files (#12011)
Unformatted test files leave the S3 Tables Format Check red on every
branch until they are re-formatted.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 18:18:32 -07:00
Chris LuandDevin b2817e853d test: raise mini-server startup wait from 30s to 90s in Makefiles (#12005)
* test: raise mini-server startup wait from 30s to 90s in Makefiles

Self-hosted/persistent runners under load regularly need >30s for
weed mini (master+volume+filer+s3+iceberg+lance+webdav+worker) to
become healthy; hosted runners start in <10s. Siblings already wait
90s (s3/checksum, s3/cors). Fixes the S3 Filer Group first-run flake
on busy self-hosted runners.

* test: align Makefile startup guards and progress totals with 90s bound

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 18:15:01 -07:00
Chris Lu 1e0f571e75 ci: set up Go for CodeQL autobuild (#12007)
The workflow relied on a Go toolchain preinstalled on the runner
image; self-hosted runners without setup-go fail at autobuild with
'go: executable file not found'.
2026-10-23 17:52:50 -07:00
Chris Lu baf0dd680e ec.encode: require shard placement to meet -shardReplicaPlacement (#11947)
* ec: measure shard placement against replica-placement caps

ecShardPlacementViolations reports where a volume's newest-generation
shard placement exceeds the per-node and per-rack caps a
-shardReplicaPlacement requests, using the same digit mapping the
balance planner enforces (SameRackCount = shards/node, DiffRackCount =
shards/rack, dc-qualified rack keys).

* ec.encode: -strictPlacement keeps originals when placement breaches caps

-shardReplicaPlacement tells the balancer the requested per-node and
per-rack shard limits, but nothing verifies the outcome. With
-strictPlacement the pre-delete shard verification also measures the
final placement against those caps: a volume still over the node or
rack limit aborts before its source is deleted, so the encode only
completes when the requested failure-domain spread is actually met.

* ec.encode: cover the strict-placement delete gate with a test

The gate in verifyEcShardsBeforeDelete now runs against a stubbed
topology: a compliant spread passes while an over-cap placement is
refused before source deletion. The retry knobs become package vars so
the test does not wait out the heartbeat-propagation backoff.

* ec.encode: enforce -shardReplicaPlacement without a flag

Drop the -strictPlacement option: the requested per-node and per-rack
caps are always requirements at the pre-delete verification, so the
encode only completes when the placement actually honors them.
2026-10-23 17:29:36 -07:00
f5e319ae8e iam: GetUserPolicy returns NoSuchEntity for a policy name the user does not have (#11944)
* iam: GetUserPolicy returns NoSuchEntity for a policy name the user does not have

When the user has stored inline policies, an unknown name fell through to the
policy rebuilt from the user's actions, so a deleted policy still read back.
The standalone IAM server already returns NoSuchEntity here.

* iam: return service failure when listing inline policies fails

A ListUserInlinePolicies error in GetUserPolicy fell through to
action-based reconstruction, masking a store failure as a policy
document. Propagate it as ServiceFailureException instead, matching
the standalone iam server and the other callers in this file.

* iam: return service failure when the inline policy store read fails

---------

Co-authored-by: chrislusf <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 17:08:49 -07:00
Chris Lu f8579bf728 s3: gate POST Object acl fields behind PutObjectAcl (#11939)
* s3: gate POST Object acl fields behind PutObjectAcl

The POST Object handler promotes acl and x-amz-grant-* form fields onto
the upload's ACL headers, and resolveFileMode honors the promoted canned
ACL. The PUT path only applies such headers after an s3:PutObjectAcl
grant; mirror that on POST by running the shared per-key authorization
with the WriteAcp action whenever the request carries ACL headers.

* s3: cover the POST Object acl gate

Signed POST requests carrying an acl form field are denied for a writer
without the WriteAcp action and for a writer a bucket policy denies
s3:PutObjectAcl on the target key; the authorization boundary is
exercised directly as well.

* s3: cover x-amz-grant-* POST fields and the no-ACL case

* s3: encode POST policy conditions as JSON in the acl-gate test
2026-10-23 17:05:44 -07:00
Chris LuandDevin 168789d5f3 master: cap writable volumes per collection via -maxWritableVolumes (#11923)
* master: cap writable volumes per collection via -maxWritableVolumes

Some deployments serialize disk writes (e.g. an Unraid array where
concurrent writes to multiple disks collapse throughput). With more
than one writable volume in a collection, the master's weighted pick
spreads assignments over all of them, so several disks get written at
once.

-maxWritableVolumes=N narrows the pick candidates to the N fullest
writable volumes, so a collection's writes concentrate on one (or few)
volumes at a time. 0 keeps the current spread-everywhere behavior.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* master: give master.follower a maxWritableVolumes default

* master: cap writable volumes inside the requested location

* master: pick the capped volumes without sorting the writable list

* master: assert the capped candidate set instead of random pick outcomes

* master: describe -maxWritableVolumes as a per-layout cap

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 17:04:36 -07:00
Chris Lu 8b26da8c53 ci: gate go/java/python tool caches to github-hosted runners (#12001)
Follow-up to #11985/#11986: setup-go/setup-java/setup-python steps
added or restructured since (e.g. by #11981's shared-artifact layout)
were missed. On self-hosted runners setup-go restores/saves a ~5GB
module cache archive per job — pure waste when the module cache is
already on local disk — and the restore blocks the job for minutes.
2026-10-23 16:25:23 -07:00
Chris Lu cd97c93deb command: keep filer, s3, and sftp up when the local unix socket cannot bind (#11999)
* pb: share the stale-socket tolerant unix listener

ServeGrpcOnLocalSocket already treats the local socket as optional; pull the remove-and-listen block out so the filer, s3, and sftp http sockets can use it too.

* command: keep filer, s3, and sftp up when the local socket cannot bind

A stale socket file owned by another uid (e.g. left in /tmp by a containerized job) made os.Remove fail with EPERM and the Fatalf took the whole server down. The gRPC local socket already treats the unix listener as optional; do the same for the http/sftp sockets.

* filer: clear shared localSocket flag when the unix socket cannot bind

s3.localFilerSocket and (in weed server) the sftp local socket share the
filer's -localSocket flag pointer. If the filer cannot bind the socket it
now keeps running without it, so consumers still holding the configured
path would dial a dead or foreign-owned socket.

* command: keep sftp on its own localSocket flag under weed server

sftpOptions.localSocket was overwritten with the filer's -localSocket
flag, so -sftp.localSocket was ignored and an explicit filer socket path
made the sftp server try to bind the same socket the filer already held.

* s3: fall back to TCP when the filer local socket cannot be dialed

The embedded S3 server copies the filer's -localSocket flag before the filer tries to bind it, so a socket that never comes up left every filer HTTP request dialing a dead path. Try the unix socket first and fall back to the request's TCP address on dial failure.

* filer: keep the shared localSocket flag read-only after startup

Clearing *fo.localSocket raced with the embedded s3 goroutine reading the same flag pointer and could not help a reader that had already copied it. The s3 filer client's TCP fallback covers the unbound-socket case without mutating shared flag state.
2026-10-23 16:07:32 -07:00
Chris Lu 6c87fb70f9 server: skip local unix sockets that cannot be bound (#12000)
server: skip default local unix sockets that cannot be bound

weed server fatally exited during S3, filer, or SFTP startup when the default /tmp/seaweedfs-<svc>-<port>.sock path could not be removed or bound. On shared hosts (e.g. CI runners where another user's weed left a socket in the sticky /tmp directory) a stale socket file turns into 'operation not permitted' and kills the whole server, even though the TCP listeners are unaffected. Warn and skip the unix socket listener instead, matching how the gRPC local socket already degrades. An explicitly configured -localSocket path still fails fast, since a co-located S3 server may be configured to dial the filer through it.
2026-10-23 16:02:33 -07:00
Chris Lu 29b81a827d volume: also listen on localhost like other components (#11995)
* volume: also listen on localhost like other components

Master, filer and s3 bind both the detected host ip and localhost via
NewIpAndLocalListeners, but the volume server used a plain NewListener on
bindIp, so a volume bound to the host ip never answers on loopback. Any
tooling that polls localhost (e.g. the java test workflow's readiness
curl on localhost:8080/status) then waits out its full timeout even
though the volume is up and registered with the master.

Use NewIpAndLocalListeners for the volume grpc, public and cluster http
listeners, and add a small multiHttpdownServer composite so graceful
shutdown still covers both listeners.

* volume: stop and wait dual listeners concurrently

A serial multiHttpdownServer.Stop let the localhost listener keep
accepting requests while the bound-ip listener drained, and a serial
Wait hid a localhost serving failure until the bound-ip listener ended.
Run both waits and both stops in parallel: Wait reports the first error
promptly, Stop closes every listener when shutdown begins.
2026-10-23 15:44:18 -07:00
Chris Lu aeb5db1b13 tests: pin weed mini admin ports and disable unused aux services (#11997)
* tests: pin weed mini admin ports and disable unused aux services

Every weed mini that does not configure them still binds the fixed
default ports for the admin server (12535 + derived 22535 gRPC), its
legacy compat listener (23646 + 33646), WebDAV (7333), Lance (9101) and
the Iceberg REST catalog (8181). Sequential or concurrent minis on one
host then collide on them: the port check warns and shifts, and the
admin's late bind can land on a stolen ephemeral port, adding noise and
flakiness to test logs (e.g. run 40017181958, job 120659267479).

Add testutil.MiniAdminArgs to pin the admin server to an allocated
mini-style port (HTTP + port+10000 gRPC, both below the Linux ephemeral
port floor since admin binds late) and disable the legacy compat
listener. Every test that spawns weed mini now also disables the aux
services it does not use: WebDAV, Lance and the Iceberg REST endpoint
get port=0 unless the test actually exercises them, in which case their
ports stay pinned as before.

* tests: take mini admin ports from the below-ephemeral range

* tests: reuse MiniAdminArgs for the remote-s3 fetch mini

* tests: fully disable webdav and co-allocate mini admin ports

- -webdav.port=0 only makes the service bind a random port; add
  -webdav=false so minis that do not test WebDAV skip it entirely.
- Spark, RisingWave and s3 Spark launchers allocated the admin port in
  a second MustFreeMiniPort call, which could pick a port already
  handed out by the first batch. Fold 'Admin' into the same
  MustFreeMiniPorts call.
- test/fuse_integration is a separate Go module and cannot import
  testutil; allocate the admin ports with its local freePort helper
  instead.

* test/fuse: derive the mini admin gRPC port from its HTTP port

Two independent freePort calls could return the same port, making admin.port equal admin.port.grpc (or collide with the filer port). Allocate once and derive the gRPC port with the +10000 convention.

* test/fuse: pin the mini admin port in the fuse_test framework copy

The fuse_test package framework spawns weed mini too; without a pinned admin port it still binds the fixed defaults 12535/23646 and collides with other minis.

* tests: drop the redundant -webdav.port=0 flag

-webdav=false alone disables the service; the port flag is dead config.
2026-10-23 15:38:09 -07:00
Chris Lu c925869dcc shell: retry filer discovery until a filer registers (#11994)
* shell: retry filer discovery until a filer registers

When -filer is not given, the shell discovers a filer with a single
ListClusterNodes call right after connecting to the master. A filer can
take several seconds to register after its port reports ready, so the
list routinely comes back empty and the shell proceeds with an empty
FilerAddress. Commands then fail with "dial tcp: missing address", which
is how the s3tables Dremio integration test flaked in CI.

Treat an empty list as retryable and keep polling ListClusterNodes until
a filer appears or -connectTimeout expires. On timeout, print a clear
"no filer registered" notice instead of failing later inside a command;
master-only shell commands keep working either way.

* shell: report the last filer discovery attempt failure

A successful empty ListClusterNodes followed by a master outage left the
diagnostic claiming no filer registered, hiding the real RPC failure.
Track the outcome of the latest attempt instead: if the final list call
failed, report that error; only when the last response was genuinely
empty report that no filer registered within the timeout.

* shell: test filer discovery retries against a fake master

Extract the discovery loop into listFilers and cover the retry behavior
with a fake master_pb.SeaweedServer: an empty cluster turning populated,
an empty cluster exhausting the window without error, the last attempt's
RPC failure being reported, and master-only commands still running after
the discovery timeout.
2026-10-23 15:21:41 -07:00
34f06ee92f test/fuse: fix cluster startup in the fuse test framework (#11977)
* test/fuse: fix cluster startup in the fuse test framework

weed volume Fatalfs when -dir does not exist, so every cluster setup
failed its volume wait after 30s. The mount also exited on the unknown
-cacheSizeMB flag (it is -cacheCapacityMB now).

* test/fuse: manage cluster processes and allocate free service ports

Each service now runs through a managed process that captures its log and
reports an early exit from the port wait, instead of a bare 30s timeout.
Ports are allocated free pairs (http + grpc sibling) rather than fixed
19333/18080/18888 so the framework no longer collides with weed mini's
fixed ports or another framework instance.

* test/fuse: bail out of the mount wait when the mount process exits early

The port waits already report an early exit with the captured log tail; the
mount wait still polled the device id for the full timeout, so a mount that
died at startup (bad flag, missing filer) surfaced as a bare 30s timeout
with no diagnosis. Check the managed process the same way and include its
log tail.

* test/fuse: deduplicate service ports handed out by freeServicePort

A probed port is released before the service binds it, so the three
sequential allocations for master, volume and filer could all return the
same still-unbound port and leave later services dying at startup. Track
handed-out ports so each allocation is unique.

* test/fuse: allocate service ports in Setup after the external-mount branch

* test/fuse: verify service readiness through its own /healthz endpoint

* test/fuse: bound the readiness probe so a silent listener cannot hang setup

* test/fuse: only accept a ready response while the service process still runs

* test/fuse: include the service log tail on readiness timeout

A startup that stalls instead of exiting still failed with a bare
"not ready within timeout" — the same opaque message that made the
original flake hard to diagnose. The process-exit path already tails
the captured service log; the timeout path now does too.

* test/fuse: report pjdfstest category results without failing the suite

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: allow allow_other mounts in posix-compliance

The job mounts with -allowOthers but never sets user_allow_other in
/etc/fuse.conf, so fusermount rejects the option and the mount exits
before the suite runs. The other FUSE workflows set it; do the same.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: keep the standalone-framework service logs on failure

Cleanup removed the temp dir holding master/volume/filer/mount logs, so
a CI failure left nothing to diagnose. Dump the tails on failure and
copy them under /tmp/seaweedfs-fuse-logs/standalone/ for the artifact
upload, next to the mini framework's logs.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: gate setup on a real write through the mount

Mount readiness only proves the daemon is serving; a fresh cluster still
cannot assign a file id until a volume heartbeats in and a grow round
finishes, so the first ~10-30s of writes surface as EIO on close. Probe
with a real write/read/remove before handing the mount to the suite.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: gate the mini framework setup on a real write too

Same cold-start window as the standalone framework: mini's volume has to
register and grow before a write can be assigned. Keep the two
frameworks consistent so later suites do not flake on the same stall.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: save each framework's CI logs under its test name

Fixed-name mini.log/mount.log in /tmp/seaweedfs-fuse-logs let a later
framework instance overwrite an earlier test's logs, so an uploaded
artifact could describe a passing run instead of the failure under
investigation. Apply to both framework copies since they share the
directory.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: bound each waitForWritable probe attempt

A wedged mount can hold a file op open past the 90s deadline, so the
loop could not check the deadline or a mount exit and Setup hung until
the overall test timeout. Each attempt now runs off the loop and is
abandoned when the remaining time runs out. Same fix in both framework
copies.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test/fuse: keep one write probe in flight until it finishes or the deadline

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: chrislusf <chris.lu@seaweedfs.com>
2026-10-23 08:10:31 -07:00
Chris Lu 2c8079625c test: stop EC integration tests from pinning default master port 9333 (#11988)
The EC integration tests failed in CI with "raft.Server: Not current
leader" on every assign for tens of seconds — longer than any goraft
election takes. A stale weed process on the self-hosted runner was
still holding 9333, so each test's fresh master died at bind time and
the tests kept talking to a raft non-leader. Only the 9333 cluster
(startSeaweedFSCluster) failed; tests on 9334-9338 all passed.

Allocate mini-safe ports (http + grpc+10000 both verified free) for the
master and volume servers instead of hardcoding 9333/8080-8085, wait
for the master to actually report raft leadership via /cluster/status
before issuing raft-gated RPCs, and make waitForServer a real TCP dial —
grpc.NewClient never connects, so the old check always passed
instantly.
2026-10-23 08:08:49 -07:00
bati-valensasandChris Lu c4002fafc7 helm: fix bucket hook treating an existing bucket as missing (#11982)
* helm: fix bucket hook treating an existing bucket as missing

The skip check pipes awk into `grep -q`, which exits on the first match and
can leave awk writing into a closed pipe. awk then dies of SIGPIPE and
`set -o pipefail` makes the pipeline report that, so a bucket that exists
reads as missing: the hook runs `s3.bucket.create`, it errors with "already
exists", and `sh -ec` aborts the hook - failing the release.

Match in a single awk instead, which reads to EOF and cannot be killed by a
closing reader.

* helm: force string compare in bucket hook existence check

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-10-23 06:43:35 -07:00
Chris LuandDevin 384523761b ci: skip actions/cache on self-hosted runners (#11986)
cargo registry/target and docker-image caches are already local on a
persistent self-hosted runner; the save/restore pass is pure overhead
(the setup-go cache-save stall on awscli-tests was the same pattern).
Gate every actions/cache step on runner.environment == 'github-hosted'.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 06:19:56 -07:00
Chris LuandDevin 277c9809c8 ci: skip setup-go cache on self-hosted runners (#11985)
* ci: skip setup-go cache on self-hosted runners

The module cache is already local on a persistent self-hosted runner;
archiving ~/go/pkg/mod into cache.tzst at every job's end is pure cost
and can stall (a ~4-minute zstdmt hang was observed on awscli-tests).
runner.environment gates the cache to GitHub-hosted runners only.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: gate the remaining setup-go caches too

The first pass only covered go-version-file steps; go-version steps in
nine workflows (including awscli-tests' own file) kept uploading the
cache archive on self-hosted runners.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 06:05:33 -07:00
Chris LuandDevin 728fc69976 ci: cache docker build layers for container_dev (#11980)
* ci: cache docker build layers for container_dev

The Go-build image rebuilds its layers on every run; type=gha backend
caching reuses them across runs and platforms.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: pin container build to the pushed sha under layer caching

Dockerfile.go_build clones master inside the image; a cache-from layer
would otherwise keep serving the previous commit's clone and publish
stale code under the dev tag. Passing BRANCH=github.sha makes the clone
layer's key commit-specific while toolchain layers still hit.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: pass BRANCH via build-args so the per-commit cache key reaches buildx

docker/build-push-action has no `args` input — the BRANCH override was
silently dropped, leaving the build arg at its `master` default and the
clone/checkout layers serving a stale commit under the gha cache.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 06:02:08 -07:00
Chris LuandDevin 9b59bc6b0d ci: build test binaries once, share them across shard jobs (#11981)
* ci: build test binaries once, share them across shard jobs

Each of the 6 rust-volume-go-tests shards, the rust-integration-tests
job, and the 6 volume-server-integration-tests shards rebuilt the same
weed and weed-volume binaries — 13 duplicate builds per run. A
build-test-binaries / build-weed job now compiles them once under the
existing cargo cache and publishes them as artifacts; the consumers
download and chmod them instead of building.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: preserve binary paths through the artifact round-trip

upload-artifact stores single files flat, so weed/weed would come back
at the workspace root and the consumers' expected paths break. Pack the
binaries in a tar before upload and extract on download.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: extract the shared test binaries in the rust go-test shards

The download step leaves rust-test-binaries.tar at the workspace root;
without untarring it the chmod fails on missing files and every shard
fails before running a test.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 04:50:29 -07:00
Chris LuandDevin f7af816ab4 ci: cancel superseded workflow runs on PR pushes (#11979)
* ci: cancel superseded workflow runs on PR pushes

PR-triggered workflows without a concurrency group keep running (and
holding runner slots) after a force-push or quick follow-up commit
supersedes them. Add the same group/cancel-in-progress block the go
workflow already uses, keyed on head_ref, to the remaining PR workflows.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: key concurrency groups on PR number or ref, not head_ref

github.head_ref is empty on tag pushes and manual runs (every run then
shares one group and cancels the last) and collides across forks when
two PRs share a branch name. pull_request.number is unique per PR;
github.ref covers push/tag/dispatch. Applied to all workflows using the
pattern, including pre-existing ones.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: restrict cancel-in-progress to PR and branch-push runs

* ci: serialize container_latest manual runs by destination tag

Two workflow_dispatch runs sharing an image_tag could otherwise build
concurrently: their per-arch images publish to the same platform tags,
so one run's create-manifest job can combine platforms from different
source refs. Group dispatch runs by the destination tag (with
cancel-in-progress already disabled for non-PR/non-branch events) so
same-tag publishes queue instead of interleaving.

* ci: serialize manual publish runs by destination tag in remaining container workflows

* ci: group foundationdb release dispatch runs by release tag

A manual run now shares the tag-push concurrency group for the same
release, so a re-publish of a tag queues behind (or ahead of) the push
run instead of racing it on the same image tag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: serialize foundationdb_version dispatch runs by destination tag

Two manual runs resolving to the same docker tag now share a concurrency
group (explicit image_tag, else the computed
foundationdb_<fdb>_seaweedfs_<ref> default), so they queue instead of
racing pushes to the same tag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: drop unsupported toLower() from foundationdb_version concurrency group

* ci: key rocksdb_version dispatch group on its published tag

* ci: queue publish runs via turnstyle instead of cancel-free shared groups

* Revert "ci: queue publish runs via turnstyle instead of cancel-free shared groups"

This reverts commit 039148f8d5.

* ci: queue same-tag publish runs instead of replacing the pending one

Dispatch/tag-only workflows can never cancel in progress, so opt their
concurrency groups into queue: max — up to 100 same-tag publishes wait
in line rather than each new dispatch cancelling the pending one.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 04:49:52 -07:00
Chris Lu 7c478fd89a fuse integration: run posix suites on weed mini and allow allow_other mounts (#11984)
The separate master/volume/filer processes in framework.go bound fixed
ports, discarded child output, and never noticed an early process exit,
so a slow or dead volume server burned the whole 30s timeout with no
logs. Switch to the same weed-mini approach the fuse_test package
framework already uses: dynamic filer ports, child output captured to
log files, and readiness waits that bail out the moment the process
dies. The external-mount path (TEST_MOUNT_POINT) is unchanged.

The posix-compliance workflow mounts with -allowOthers, but fusermount
refuses allow_other unless user_allow_other is set in /etc/fuse.conf,
mirroring the fuse-integration workflow FUSE setup.
2026-10-23 04:33:14 -07:00
ad89f003a5 s3: end a catalog-vended session with the access key it was vended for (#11945)
* s3: end a catalog-vended session with the access key it was vended for

A table credential the Iceberg catalog vends is a self-contained STS session:
it stays valid until it expires (an hour by default) even after the access key
that loaded the table is deleted or deactivated, or its identity is removed
or disabled. An operator revoking a leaked key cannot cut off the table
credentials that key already obtained.

Sessions are stateless JWTs, so there is no revocation list to consult. This
change follows the same pattern as role sessions, which are already refused
once their role's ID no longer matches (checkSessionRoleBinding): a vended
session carries the access key it was vended for (src_ak, recorded by the
previous change), and validateSTSSessionToken checks it against the identity
table held in memory. A session of provider iceberg-catalog that carries a
source key is refused (InvalidAccessKeyId, as for an unknown key) once its
identity is gone or disabled, or the key is no longer one of that identity's
active, unexpired credentials.

Scope:
- Only catalog-vended sessions are checked; web-identity, AssumeRole and
  every other STS session are validated as before.
- The vendor binds a session to the caller's key only when the caller is a
  stored identity holding that key. A caller that signed with another STS
  session's key, an OIDC user, or a request that recorded no key gets an
  unbound session, as today; binding it would refuse every request made with
  it. A session minted before this change carries no key and is unbound.
- The check is one map lookup under the read lock and a scan of one
  identity's keys (about 11 ns, 0 allocations, with 1,000 identities).

Tests (s3api_iceberg_vended_session_test.go) mint real sessions through the
catalog vendor and validate them the way a data-path request is validated:
deactivating, expiring or deleting key A ends A's session while the session
vended through key B on the same identity keeps working; disabling or
deleting the identity ends it; callers that are not a stored identity holding
the key get unbound sessions that keep working; other STS sessions are not
checked. The first two tests and the third fail without this change.

* s3: keep the vended-session key binding when the key dies before vend

vendedSessionKeyLive gated the binding, so a key revoked between the
request's authentication and the vend skipped it and minted an unbound
session that stayed usable until it expired. Every catalog auth path
pairs the identity name in context with the key that authenticated, so
a stored-identity principal always means the request signed with that
identity's own key. Bind whenever the principal is a stored identity: a
session bound to a now-dead key is refused on its first request, which
is what the binding is for.

* s3: gate the vended-session key binding on the authenticated caller type

lookupByIdentityName(principal) named only that some identity exists, not
that the request signed with a stored key: an STS session or OIDC principal
that collides with an identity name got a session bound to its temporary
key, which vendedSessionKeyLive then refused on every request. Verifiers
now record the verified key as a stored credential, and the vendor binds
only when that marker is present — keeping the binding even when the key
was revoked between authentication and the vend.

* s3: enforce the vended key binding under a dedicated claim

Sessions minted before the binding existed carry the caller's key in the
audit-only SourceAccessKeyId — including temporary STS keys that can never
resolve in the identity table. Enforcing on that field would end live table
credentials at upgrade; the binding now travels under BoundAccessKeyId,
leaving legacy sessions unbound as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: bind vended sessions to a credential fingerprint

The key id alone cannot identify a credential: an operator can delete a
key and re-create it under the same id with a different secret, reviving
sessions minted for the original. Verifiers now record a fingerprint of
the secret at authentication; the session dies unless it still matches.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: keep recording the caller's key on every catalog session

SourceAccessKeyId was unconditional audit metadata before the binding
existed; gating it on the stored-credential marker dropped the record
for callers that authenticated another way. With the binding under its
own claim there is no ambiguity left, so the field records the caller's
key again regardless of how it authenticated.

* s3: key the vended-session credential fingerprint with the signing key

The bound-credential fingerprint travels inside the vended session
token. As a plain SHA-256 of the secret it gave any token holder an
offline guessing oracle for weak secrets, and CodeQL flags it twice.

Store the verified secret in the request marker instead and HMAC it
with the STS signing key at vend and at validation. The claim can no
longer be mined, and the per-request auth path stops hashing secrets.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 03:58:06 -07:00
Chris LuandDevin bf3081e23a s3 lifecycle: route version-file writes to current-version expiration (#11956)
* s3lifecycle: schedule ExpirationDays from version-file writes

On a versioned bucket a PUT lands in the meta-log as the version file
plus the .versions/ directory update — there is no bare-key event, so
the version-folder skip dropped every version-file event for all kinds
but ExpiredDeleteMarker. ExpirationDays therefore only fired via the
recovery walker's cold-start / rule-change pass: once a shard had a
saved cursor, no daily run ever scheduled another delete marker.

A version-file write is the latest for its logical key at write time,
so route it through the same event-driven evaluation the regular path
runs. The match carries the file's identity with an empty VersionID —
the dispatcher resolves the current latest at due time, and a fresher
PUT meanwhile resolves to STALE_IDENTITY there.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* lifecycle test: versioned Expiration fires on the second daily run

Establishes the shard cursor with a first pass, then PUTs and backdates
a new versioned object and runs again. The delete marker must appear
without a rule change forcing a recovery walk.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run TestLifecycleVersionedBucketExpiresOnSecondRun in the lifecycle matrix

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 03:19:41 -07:00
Chris LuandDevin 2e7e691bed volume: drop stale index entries that resolve to a different needle (#11948)
* volume: drop index entries that resolve to a different needle

An index row whose offset lands on a needle with a different id and a
different size cannot have been produced by any write path: every appender
indexes a key at the offset and size of the record it wrote, and
WriteNeedleBlob keeps its indexed size consistent with the blob header.
Such rows are corruption (for example an offset replayed with stale high
bytes) and until now made the volume reject every write for the key with
"mismatching cookie" while reads resolved to the wrong needle.

When the duplicate check finds the indexed record belongs to a different
key and size, drop the stale entry and let the append re-index the key.
Same-key conflicts still fail the cookie check, and entries staying
consistent with a re-keyed blob keep their existing semantics.

* volume: verify the indexed needle header before streaming reads

The buffered read path re-parses the record and rejects a needle whose
stored size disagrees with the index, but the streamed path serves data
bytes without ever looking at the header, so a stale index row would
stream another needle's payload. Probe the header once up front and
report a miss when the record is neither the requested key nor the
indexed size, the same staleness test the write path applies.

* rust volume: drop index entries that resolve to a different needle

Mirror of the Go volume change: when the duplicate check finds the
indexed record belongs to a different key and size, delete the stale
entry and let the append re-index the key.

* volume: overwrite a stale index entry on commit, not up front

Deleting the stale row during the duplicate check published a tombstone
before the replacement append and, for durable writes, before its fsync.
A failed append or sync then rolled back the data but left the key
deleted. The publish step already overwrites any older offset, so leave
the entry in place and let it be replaced once the record commits.

* volume: report streaming header-probe failures to the io error tracker

A failed header read returned before the tracked body-read loop, so
repeated EIO on the probe never reached checkReadWriteError. Skip io.EOF
the way the loop does: it is a truncated record, not a disk fault.

* rust volume: overwrite a stale index entry on commit, not up front

Mirror of the Go volume change: the staged delete published a tombstone
before the replacement append and its .dat flush, so a failed append or
sync left the key deleted. publish_write already overwrites any older
offset once the record commits.

* rust volume: report needle meta read failures to the io error tracker

read_meta is the header probe ahead of a streamed body: an EIO there
escaped check_read_write_error entirely, so a failing disk could not
trip the tolerance. Report the outcome the way read_full already does.

* volume: retry the wrapped offset before judging an index entry stale

A 4-byte index value can index past the 32 GiB wrap: the low offset may
hold another needle while the real record sits at +MaxPossibleVolumeSize,
which is where reads already retry on a size mismatch. Skipping the
cookie check on the low-offset record alone would accept a foreign-cookie
write over a live key. Retry the wrapped header read first, the same
condition the read path uses; a wrapped read that fails for reasons other
than EOF stops the write like the low-offset read does.

* volume: probe the streamed needle header under the access lock

With HasSlowRead the access lock is dropped after the index lookup, so a
compaction commit could swap the backend between the lookup and the
header probe and fail a valid download on the old offset. Take the read
lock for the probe and resolve the compaction revision first, the way
the streaming loop re-resolves it on every page.

* rust volume: retry the wrapped offset before judging an index entry stale

Mirror of the Go volume change: probe offset + MAX_POSSIBLE_VOLUME_SIZE
under the 4-byte-offset build before treating a mismatched record as
stale, so a live wrapped entry still gets its cookie checked.

* rust volume: do not clear the io error streak on a successful meta read

read_meta runs before the payload read in read_needle_for_get, so recording success there reset the streak before every failing download and the volume could never reach quarantine. Report failures only; read_full still records success after a complete read.

* volume: record a meta-only read's success against the io error streak

A meta read no longer clears the streak because a payload read still
follows on GETs, but requests the meta read ends outright — HEAD answers
and cookie mismatches — never reached the body path, so their success
left earlier failures counted. Note the success on those terminal paths;
mirrored in the Rust volume server.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* rust volume: report streamed reads to the io error streak

Streamed GETs never touched the tracker: a failing chunk read went
uncounted and a completed download never cleared the streak, unlike
readNeedleDataInto which does both.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: reset the io error streak only at terminal meta reads

A streamed GET's metadata read runs before its payload read, so clearing
the streak there erased the count before every failing download and the
volume could never reach quarantine. Record the meta-read success only
where the request actually ends: must-meta-only reads, HEAD responses,
and cookie mismatches — matching the Rust volume server's placement.

* rust volume: record the streamed read's success at the final frame

Hyper can complete a Content-Length response without polling the body
again, so a success recorded only when the stream is next polled could
be skipped, leaving an earlier io error streak in place after a
successful download.

* volume: record a meta-only read's success on conditional 304 replies

* rust volume: record a meta-only read's success on conditional 304 replies

* volume: reset the io error streak only after the checksum passes

A streamed read that ends in a CRC mismatch still cleared the EIO
streak, so corruption could mask progress toward quarantine. The Rust
side records success only after the final frame's checksum verifies;
move the reset below the CRC check to match.

* volume: satisfy codespell in the meta-read comment

* rust volume: report range reads to the io error streak

* rust volume: cover the range read's io error reporting

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 02:30:53 -07:00
Chris LuandDevin 27fa285d48 filer: do not stall startup waiting out the leader-settle window (#11972)
* filer: do not stall startup waiting out the leader-settle window

ListExistingPeerUpdates re-polled the leader for up to 10s when it
reported zero filers. A fresh single-filer cluster can never gain peers,
so every filer start paid the full settle timeout before listening —
past the 6s health window in weed mini and short probe budgets.

Take the leader's first answer instead; the fallback fullest-answer
scan below still covers a leader that is mid-convergence.

* filer: bound leader-settle re-poll below the startup health window

* filer: update ListExistingPeerUpdates doc for empty-leader fallback

* filer: re-check empty membership at the announce gate before registering

A fresh filer that scans empty membership during a master restart or
re-election skipped bootstrap and announced immediately, serving missing
files until a peer happened to reconnect — and a late peer only got an
incremental subscription, so pre-existing metadata was never backfilled.

The announce gate now re-polls ListExistingPeerUpdates while the scan can
still be disproved (10s), backfills from any late peers before their
streams start, and then waits for their catch-up like startup peers. A
genuinely empty cluster pays a delayed master listing rather than a
stalled startup, since this runs after the ports already serve.

* filer: overlap the late-peer copy cutoff and stream start

A peer event stamped just behind this filer's clock could land after
the traversal read but carry a timestamp below the stream start —
the startup scan margins both cutoffs by LogFlushInterval for exactly
this skew; mirror it for peers discovered at the announce gate.

* filer: bound the late-peer bootstrap copy by the shared announce deadline

MaybeBootstrapFromOnePeer now takes the caller's context so a stalled
peer metadata copy cannot out-wait the announce cap, and the bootstrap
re-check shares one announceCatchUpCap deadline with the catch-up wait
instead of each spending a full minute.

* filer: merge bootstrap entries by commit time instead of blind insert

The late-peer copy runs while gRPC already serves, so a blind InsertEntry
could replace a write the local filer just accepted. Routing each entry
through applyTraversedEntry orders it against the local commit time under
the replay locks — a fresher local write survives the backfill.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 02:04:27 -07:00
Chris LuandDevin b65832bdc9 iam/oidc: reject reused jti claims with a bounded replay cache (#11936)
* iam/oidc: reject reused jti claims with a bounded replay cache

* iam/sts: pass jti replay settings through convertToOIDCConfig

* iam/oidc: keep consumed jtis and stop the cleanup worker across re-init

Re-initializing a provider reset the JTI store, so an already-used token
could be presented again after a config refresh; the store is now
allocated once and survives Initialize. The cleanup worker is stopped on
every re-init (previously only when protection stayed enabled), is
started only after the rest of Initialize has succeeded, and Shutdown no
longer consumes a permanent sync.Once, so a re-initialized worker can be
stopped again.

* iam/oidc: enforce the jti replay cache bound for every token

Tokens without a usable jti claim skipped the cache entirely and stayed
reusable; they are now keyed by the token's own fingerprint so a repeat
presentation is still a replay. Tokens without exp previously failed as
expired on first use; their entry now gets a bounded retention. The
replay test-and-set now runs before any capacity work so replayed tokens
cannot each trigger a full-cache scan, the entry count is incremented as
part of the insert so concurrent validations cannot overshoot the bound,
and a guaranteed serialized sweep runs before a capacity rejection while
opportunistic sweeps are rate limited and confined to one at a time. Raw
token identifiers are no longer written to logs or errors, a stable
fingerprint is logged instead.

* iam/sts: shut down providers dropped by the managed-provider swap

Each managed OIDC provider runs a background JTI cleanup worker, but
SetIAMManagedOIDCProviders replaced the provider map without stopping
the old providers, so every refresh leaked goroutines. Providers absent
from the new map are now shut down; providers that stay keep running.

* iam/integration: reuse providers across refreshes for unchanged records

Every refresh rebuilt providers from their records, so a refreshed
provider forgot the JTIs it had already consumed and an already-used
token could be presented again. Records are now fingerprinted and an
unchanged record reuses its provider, keeping the consumed-token cache;
changed records are rebuilt, removed records are dropped, and the swap
in STS releases the workers of whatever is no longer referenced.

* iam/oidc: consume replay entries only on the single-use exchange path

Bearer authentication validates the same token on every request, so
consuming it in shared ValidateToken/Authenticate made every token
single-use and rejected all but the first request. The replay check now
runs only for the STS exchange via AuthenticateSingleUse; bearer paths
validate without consuming. A token without exp also no longer gets a
bounded replay entry that ages out while the token stays valid: its
entry is kept until the capacity bound evicts it, so it can never become
usable again.

* iam/oidc: keep the mock provider on the mock path for single-use calls

* iam/oidc: require exp on single-use tokens

A cache slot frees only by expiry, so an expiry-less entry would hold it
forever; a burst of distinct tokens could exhaust the bound permanently
and block every later exchange.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: give each web-identity test token a unique jti

Subtests minted tokens with identical claims inside one second, so the
exchange saw the same token fingerprint twice and the second call read
as a replay. Each exchange now carries its own jti.

* iam/oidc: carry the consumed-token replay store across provider rebuilds

A record change rebuilds the managed provider with a fresh JTI store, so a token consumed before the rebuild could be presented again until it expired. The new provider adopts the replaced provider's replay store before it is published, and the replaced provider is shut down so only one cleanup worker sweeps the shared store.

* iam/oidc: stop the cleanup worker of providers dropped from the store

A refresh removed the provider object from the managed cache but left its JTI cleanup goroutine running for the life of the process.

* iam/oidc: merge a provider's consumed replay entries into another's store

MergeReplayStateFrom folds one provider's consumed-token entries into
another's, skipping already-expired ones. Unlike AdoptReplayStateFrom it
swaps no pointers, so it is safe on a provider still serving validators.

* iam/integration: share one provider and replay store across identical issuer records

Records for the same issuer URL with identical validation settings now
reuse a single provider, so a token consumed through one account's
record is rejected through the other's — previously each record owned a
separate jti store and a spent token could be replayed cross-account.

A superseded provider's consumed entries merge into the shared store so
no rejection is lost, and a removed or rebuilt provider is only shut
down once no remaining record references it.

* iam/oidc: hand the superseded provider's live replay store to the shared one

* iam/oidc: carry merged replay entries without charging the capacity bound

* iam/oidc: key the jti-less replay fingerprint on the signed input

Hashing the whole token lets ECDSA signature malleability (s -> N-s)
mint a second cache key for identical claims, so a consumed token could
be replayed for another exchange. Key the fallback on header.payload
instead.

* iam/oidc: publish the jti cache bound before the enabled flag

Initialize stored jtiEnabled ahead of jtiMaxSize, so a provider flipped
from disabled to enabled exposed enabled=true with a zero bound for a
window; a single-use validation landing there would compute
newCount > maxSize and reject with "JTI cache capacity exceeded". Store
the bound first so the flag is the last thing published.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 00:46:14 -07:00
Chris LuandDevin 186bc5cf7f s3: bound ListMultipartUploads memory to the page size (#11971)
* s3: bound ListMultipartUploads memory to the page size

listMultipartUploads loaded every open upload in the bucket into a slice
and selectMultipartUploads filtered and sorted a second full copy, so one
request cost O(total open uploads) heap regardless of prefix, key-marker,
or max-uploads; concurrent calls multiplied it and could OOM the gateway.

The response is ordered by the object key kept in extended attributes, not
the upload id the filer sorts by, so every entry still has to be seen.
Streaming the listing into a bounded top-K collector keeps at most
maxUploads (capped at maxUploadsList) candidates, bounding heap by the
page instead of the folder. A caller-supplied max-uploads above the
server cap now also paginates rather than accumulating unboundedly.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: cover bounded ListMultipartUploads accumulation

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: report the effective max-uploads cap in ListMultipartUploads

* s3: cover ListMultipartUploads retry through the collector

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 00:46:10 -07:00
Chris LuandDevin 7154732b84 worker/ec: retry the pinned volume copy after a mid-copy compaction (#11929)
* worker/ec: retry the pinned volume copy after a mid-copy compaction

The EC task pins .idx/.dat copies to the CompactionRevision read from
ReadVolumeFileStatus so the pair comes from one volume generation. When
vacuum compaction lands between that read and either CopyFile, the
server rejects the copy with "volume N is compacted" and the whole task
fails — on clusters with queued maintenance this makes most EC tasks
die to a race they cannot avoid. On a compaction mismatch the copy now
restarts under a freshly read revision, up to a bounded number of
attempts; other errors still fail immediately.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* worker/ec: read the retried copy's expiry from its own file status

A mid-copy compaction makes the source reload the volume, and the fresh
ReadVolumeFileStatus can carry a different expireAtSec than the
pre-compaction read. Keeping the attempt-0 expiry could let the EC
volume outlive its source TTL.

* worker/ec: cover the compaction retry with a stub volume server

* worker/ec: also observe revised .idx bytes in the compaction retry test

* worker/ec: read the fake source under its mutex in the retry test

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 21:32:57 -07:00
Chris LuandDevin f2d5c27f65 operation: renew the write JWT when a fan-out rollback delete is rejected (#11970)
* operation: renew the cleanup JWT when a fan-out rollback delete is rejected

A stalled holder can out-wait the assignment JWT's 10s expiry, so the
type=replicate delete that rolls back landed copies comes back 401 and the
needle leaks. Retry the delete once with a fresh token minted through the new
LookupJwtFunc hook when a holder answers unauthorized.

* s3api: wire a cleanup-JWT renewer into chunked uploads

Keep the master client on the server and use it to mint fresh volume JWTs for
rollback deletes that run past the assignment token's expiry.

* admin: wire a cleanup-JWT renewer into file-browser uploads

* operation: bound the cleanup-JWT renewal with a per-attempt deadline

A rollback delete that hits 401 renewed the token through a background
context, so a master outage could park the wait for a connection and the
lookup RPC inside the failed upload's cleanup path. Thread a context
through LookupJwtFunc and LookupJwt, and bound each renewal with a 10s
deadline covering both the master wait and the lookup call.

* operation: let every holder's 401 retry with a renewed cleanup JWT

A single renewed flag meant only the first rejecting holder refreshed the
token; a holder that stalled past the renewed JWT's lifetime left later
copies stranded with a 401 they could not resolve. The retry limit is now
per holder — each delete is attempted once with the current token and once
more after a bounded renewal — so a slow middle holder cannot strand the
rest.

* operation: log the volume server's error body on a failed cleanup delete

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* operation: count test upload attempts atomically

The chunk upload loop fans AssignFunc out to parallel goroutines, so the
plain int counter in the partial-results test raced under -race.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 21:32:53 -07:00
github-actions[bot] b97ed0f101 4.49 2026-10-23 03:49:00 +00:00
github-actions[bot] 13cbe78537 docs: regenerate star history chart 2026-10-23 02:28:25 +00:00
Mohd Quamar Tyagi 94dd96a322 weed filer.remote.sync: ignore .system directory under buckets (#11966) (#11969) 2026-10-22 18:40:30 -07:00
Chris Lu 00e64441c6 fix: clone protobuf snapshots without copying internal locks (#11968)
* fix: clone protobuf snapshots without copying internal locks

* fix: reuse cloned snapshot fields when filtering
2026-10-22 18:38:33 -07:00
Chris Lu f08f075d23 deps: patch two open dependabot alerts (klauspost/compress, aws-smithy-json) (#11959)
* test/kafka: bump klauspost/compress to v1.20.3 in simple-publisher

Patch an s2 integer-overflow / out-of-bounds write affecting
klauspost/compress <= 1.18.6; matches the version the main module
already uses. The go directive rises to 1.25, still under the ^1.26
toolchain the kafka test workflows run with.

* seaweed-volume: bump aws-smithy-json to 0.62.7

Patch an uncontrolled-recursion denial of service in the unknown-key
skip path of aws-smithy-json <= 0.62.6 (reachable over the S3 SDK's
response parsing). The smithy runtime family moves together:
aws-smithy-runtime-api 1.12.3 and aws-smithy-types 1.5.0, all selected
within the workspace's Rust 1.91.1 compatibility bound.
2026-10-22 14:38:16 -07:00
Chris LuandDevin 7c980774e7 FUSE mount: add posix compliance reports (#7183)
* fuse integration: add POSIX compliance test suite

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: add posix-compliance workflow

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: skip non-executable weed path candidates

* fuse integration: size readv buffer to the written segment

* fuse integration: reference posix_Makefile explicitly in recursive make calls

GNU make does not propagate -f through MAKEFLAGS, so bare $(MAKE) re-read the default Makefile in this directory.

* fuse integration: fail posix test targets when go test fails

Every suite pipes go test through tee under /bin/sh, so the recipe status was tee's and test failures stayed green. Run recipes under bash -o pipefail and emit the color escapes with printf since bash echo does not expand them.

* fuse integration: run only benchmarks in benchmark-posix

go test -bench=. without -run also executes the whole suite, mixing compliance output into the benchmark log.

* fuse integration: emit single-number test counts in reports

grep -c prints 0 and exits nonzero when nothing matches, so '|| echo 0' produced two lines that broke the expr arithmetic and JSON fields. awk prints exactly one count either way.

* fuse integration: skip the sendfile test on macOS

macOS sendfile(2) only accepts a socket destination; the test copies into a regular file and always gets ENOTSOCK.

* fuse integration: honor TestConfig and own only self-started resources

NewFuseTestFramework dropped its config argument and Cleanup read a fresh
DefaultTestConfig, so SkipCleanup never did anything. Cleanup also skipped
teardown whenever TEST_SKIP_CLUSTER_SETUP was set, even when TEST_MOUNT_POINT
was empty and Setup had started the cluster itself. The framework now keeps
the config it was built with, runs external-mount tests inside a per-run work
dir under the mount, and only skips teardown when an external mount was
actually used.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: pass the debug flag before the subcommand

-v is a weed global flag; appended after master/volume/filer/mount it lands
on the subcommand flagset, which rejects it and the process dies at flag
parsing whenever EnableDebug is on.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: require a real mount before running tests

Setup creates the mount point before weed mount starts, so statting and
listing it succeeded on the plain directory and tests ran against local
disk whenever the mount was slow or died. waitForMount now waits until the
mount point's device id differs from its parent's.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: resolve the weed binary to an absolute path

The started processes run with cmd.Dir at the framework's temp dir, so a
relative candidate like ../weed no longer resolves. Prefer PATH, then probe
the usual build outputs and return absolute paths only.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: build the external test work dir with MkdirTemp

The suite staged cloned tools and generated scripts under a predictable
/tmp path another user could pre-create and swap mid-run.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: use the pjdfstest tree setup-pjdfstest installs

The test gated on a pjdfstest binary in PATH but then proved a suite it
cloned itself, so a tree installed by make setup-pjdfstest was never found.
Look for the built suite under external_tools first, then clone into the
work dir as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: require a real mount before posix tests

The wait loop fell back to listing the mount point, which passes on the
directory created before weed mount runs — a dead or slow mount meant the
suite exercised local disk. Only mountpoint counts now, and a dead mount
process fails fast instead of timing out.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: collect failure logs before cleanup deletes them

The always-on cleanup step removed /tmp/seaweedfs before the failure-only
debug step could copy the service logs, so failed runs uploaded nothing.
Run cleanup after log collection.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: point MOUNT_POINT at TEST_MOUNT_POINT

Targets that work against MOUNT_POINT (pjdfstest, nfstest, fio configs)
ignored TEST_MOUNT_POINT and ran against the local-disk default while the
go tests used the external mount.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: skip the permission-denied check as root

Root bypasses mode-bit checks, so writing the read-only file succeeded and
the assertion failed in root-run environments like containers.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: align the posix docs with the suite

The doc listed a docker-test-posix target and make variables that do not
exist, and ran go test against single files that cannot compile alone. The
roadmap also still claimed the platform-specific tests this suite now
implements were all skipped.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: honor SkipCleanup in the mini-based framework too

Same gap as the fuse-package framework: Cleanup read a fresh
DefaultTestConfig, so a test's SkipCleanup setting was ignored.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: normalize config in Setup and clean the work dir too

Setup(nil) panicked on config.EnableDebug and external-mount Cleanup left
the temp dir behind; use the stored config for the start* calls and remove
tempDir alongside the external work dir.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: point external posix suites at the workflow FUSE mount

* fuse integration: create the fio target directory before running

* fuse integration: use unix.O_DIRECT instead of a hard-coded x86 value

0x4000 is O_DIRECT only on x86 and amd64; on arm64 it is O_DIRECTORY.
The test module already depends on golang.org/x/sys.

* fuse integration: run autoreconf and configure before building pjdfstest

A fresh pjdfstest clone is autotools-based and has no Makefile, so the
Go-side setup fallback always failed and the suite silently skipped.

* ci: require external posix tools only when setup installed them

setup-external-tools always exits 0 because each sub-target tolerates
failure, so POSIX_REQUIRE_EXTERNAL_TOOLS was set even with no tools
installed and the suite then hard-failed on the missing suites.

* ci: fail the posix workflow when the mount point is not a FUSE mount

The file write/read check passes on the plain directory created before
weed mount, so a dead mount let the whole suite grade the local disk.

* fuse integration: skip when the host has no FUSE device node

Setup failed the test outright on machines without /dev/fuse instead of
skipping; cover every suite entry point by skipping inside Setup.

* fuse integration: drop the chmod 666 /dev/fuse troubleshooting tip

World-writable device access is a security risk; the fuse group
membership on the line below is the safe fix.

* fuse integration: require go 1.21+ in check-go, not 1.20+

* fuse integration: refuse external suites on an unmounted path

* fuse integration: verify TEST_MOUNT_POINT is a real mount

* fuse integration: clean TEST_MOUNT_POINT before the parent-device check

filepath.Dir of a trailing-slash path returns the mount itself, so a mounted TEST_MOUNT_POINT ending in / was refused.

* fuse integration: detect WinFsp mounts on Windows in mountedAt

A directory mount is a reparse point and a drive mount is a volume root; a plain directory is now refused instead of accepted.

* fuse integration: strip a trailing slash off MOUNT_POINT in require-mounted

The mount-table grep fallback on hosts without mountpoint(8) would not match a slash-suffixed entry.

* fuse integration: require an external mount to be a FUSE filesystem

A device-ID boundary alone does not prove FUSE: a tmpfs mount would pass mountedAt and report its results as FUSE compliance. Check the filesystem type (statfs magic on linux, fstypename on the BSDs); platforms without a check keep the mount and write verification.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 14:37:03 -07:00
Chris Lu 8f6121694a test: fix HEAD bucket assertions and FUSE failover startup timing (#11967)
test: fix HEAD bucket assertions and warm up failover clusters
2026-10-22 14:34:52 -07:00
Chris Lu 137d29633b s3: fix prefix filtering and delimiter pagination (#11962) 2026-10-22 09:31:20 -07:00
github-actions[bot] ca0314ea4f docs: regenerate star history chart 2026-10-22 12:49:43 +00:00
Chris LuandJavier de la Torre 82f0a82564 filer+volume: make acknowledged writes durable under the fsync rule (#11943)
* volume: never compact a volume tiered to remote storage

A tiered volume's data backend is the remote object, but compaction reads
the local .dat path. When tiering removed the local .dat, every vacuum of a
tiered volume with garbage fails with "no such file or directory", and the
master's sweep and the worker's vacuum task keep selecting it. When the
local .dat was kept, the compaction succeeds against the stale copy and the
commit reloads the volume to read the remote object (old layout) through the
compacted index, so live needles stop reading.

Mirror the Rust volume server (#11545):
- CompactByIndex and CompactByVolumeData refuse a volume with a remote file
  before creating .cpd/.cpx ("volume N is tiered to remote storage, cannot
  compact").
- CommitCompact refuses under the data-file locks when the volume became
  tiered during the copy, and removes .cpd/.cpx.
- Store.CheckCompactVolume reports no garbage for a tiered volume, so the
  master's vacuum and volume.vacuum do not select it.
- The worker's vacuum detection skips a volume with a remote copy, as balance
  and EC detection already do.

Tests (each fails on master): TestCompactRefusesTieredVolume,
TestCommitCompactRefusesVolumeTieredDuringCopy (on master the commit
succeeds and a surviving needle no longer reads),
TestPickPrimaryReplicaSkipsTieredVolumes.

* volume: skip tiered replicas in vacuum selection, keep their real ratio

Reporting a zero garbage ratio for a tiered volume hid it from
volume.tier.compact, whose preflight reads that ratio while the volume is
still remote, and did not exclude it from a vacuum with a zero threshold.
Report the real ratio again, as the Rust volume server does, and have the
master's vacuum check skip a replica whose volume info names remote
storage, whatever the threshold or request.

When CommitCompact refuses a volume tiered during the copy, also close the
temporary needle map and remove .cpldb: nothing else would, since no later
vacuum selects the volume.

The tiered-compaction test no longer depends on random needle sizes
reaching a 0.3 garbage ratio.

* volume tier: do not attach a copy of a .dat a compaction replaced

The upload copies the .dat, then records the remote file in the .vif and
switches the backend. A compaction commit in between swaps in a compacted
.dat and index; the upload then served the old-layout remote object through
the new index, so live needles stopped reading. CommitCompact's tiered
check cannot see this: the volume is not tiered until the switch.

AttachRemoteDatFile records the remote file and switches the backend under
the data-file locks, and refuses when the backend is no longer the file
that was copied. The upload then removes the unused remote copy.

* volume tier: keep a committed .vif entry when only the dir fsync fails

AttachRemoteDatFile rolled back the in-memory remote entry on any
SaveVolumeInfo error. A NotCrashDurableError comes after the rename, so
the .vif on disk already names the copy; with the entry gone, the upload
handler deleted that object, and a reload would serve from a missing
remote file. Keep the entry and switch to the copy as the .vif says,
and still return the error so the caller keeps the local .dat.

* vacuum worker: filter tiered replicas instead of skipping the volume

A partially tiered volume is the leftover of an interrupted
volume.tier.upload; skipping the whole volume left garbage on its local
replicas unreclaimed. Per-replica filtering matches the master's
batchVacuumVolumeCheck and is safe: a tiered replica refuses both
compaction and commit.

* filer: sync leveldb entry writes when the path rule asks for fsync

The fsync storage rule only reached the volume upload; the metadata
writes behind an acknowledged create, update or delete still went to
the leveldb journal with Sync:false, so a power loss could roll back
an acknowledged bucket or object entry.

A write-side flag on the entry context carries the request: the filer
sets it from the path's storage rule in CreateEntry, UpdateEntry,
EnsureDirectoryEntry and DeleteEntryMetaAndData, and the HTTP write
path sets it for an explicit ?fsync=true. The leveldb stores write
with Sync:true when the flag is present.

* volume: flush the .idx before acknowledging a durable write

A write with fsync synced only the .dat: the index row written on
publish stayed in the page cache, so a power loss could leave the
needle's bytes durable with nothing indexing them.

The durable paths now run one .idx sync after publish, before the
requests complete — one per syncWrite and one per batch run, paired
with the .dat sync, the same split the Rust volume server makes
between flush_dat and flush_idx. A failed .idx sync cannot be rolled
back, so it stops the volume taking writes and fails the requests
instead of acknowledging an index entry that may not survive.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
2026-10-21 23:57:09 -07:00
7603528aff s3api: support s3:RequestObjectTag/<key> in bucket policies (#11910)
* s3api: support s3:RequestObjectTag/<key> in bucket policies

The tags come from X-Amz-Tagging only where the handler stores them from
there: not on PutObjectTagging, which reads the body, and not on a copy
that keeps the source's tags. For a repeated key the last value counts,
since that is what CopyObject stores.

* s3api: extract RequestObjectTag only where the handler stores it

The header was honored on any request carrying it, so a tag attached to
a GET, DELETE, part upload, multipart completion, directory marker or a
PUT subresource satisfied a tag-conditioned policy the object never
receives. Extraction now follows the handlers: PutObject, a copy under
X-Amz-Tagging-Directive: REPLACE, and CreateMultipartUpload store the
header; a plain object PUT is identified by carrying only X-Amz-*
signing parameters.

* s3api: exclude multipart part PUTs before the copy-source check

A PUT carrying X-Amz-Copy-Source plus a numeric partNumber and uploadId
routes to CopyObjectPartHandler, which stores a part and never reads
X-Amz-Tagging. Checking the copy source first let such a request satisfy
an s3:RequestObjectTag condition under REPLACE with tags that are never
stored. Mirror the router: the part handlers match only when both
partNumber (digits only) and uploadId are present, so a lone or
non-numeric partNumber still falls through to the storing handlers.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: gate tag extraction on the routed subresource, not query key shape

Rejecting every PUT whose query held a key outside X-Amz- dropped
X-Amz-Tagging from ordinary object PUTs that carry arbitrary parameters —
Signature V2 presigns (AWSAccessKeyId/Signature/Expires) among them — so a
tag-conditioned Deny could be bypassed by appending an unrelated query
parameter while putToFiler still stored the tags. Exclude the object PUT
subresources the router actually dispatches (tagging, renameObject, acl,
retention, legal-hold) ahead of the copy-source check instead.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: cover the routed-subresource and part-copy tag extraction cases

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 23:55:01 -07:00
Chris LuandDevin cec96c02ea s3: reject requests carrying more than one routed subresource (#11942)
* s3: reject requests carrying more than one routed subresource

Routing picks the first matching subresource route while authorization
resolves a single action from the query, and the two can disagree. A
bucket-level PUT ?tagging&policy is authorized as s3:PutBucketTagging but
served by PutBucketPolicyHandler, letting a tagger rewrite the bucket
policy; the same shape lets a list grant read the policy via
?list-type=2&policy or ?versions&policy, and lets an object ACL writer
retag via PUT ?acl&tagging. Declining ambiguous requests keeps the
authorized action and the served handler identical.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: reuse hasAmbiguousSubresource in the routed-subresource guard

The standalone key list dropped versioning, so ?versioning&policy still
reached route matching and auth with two operations. The bucket-level
check already keys off operationSubresources plus bucketQueryActions;
running the same predicate keeps the lists from drifting apart.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 22:45:53 -07:00
Junker der Provinzandzouyixiong ee2718b14c filer: reduce forwarded lock log noise (#11933)
Forwarding a lock to its owner filer is routine in a multi-filer setup,
so the forward and acquire messages move from V(0) to V(3). The acquire
message is only logged when the forwarded response holds a renew token
and no application error. Transport failures still log at V(0).

Co-authored-by: zouyixiong <3390062+zouyixiong@users.noreply.github.com>
2026-10-21 20:46:23 -07:00
Chris LuandDevin 7ff77a4f3b s3api: skip error response body when http forbids it (#11931)
WriteResponse wrote the payload even for HEAD requests and statuses
that disallow a body (1xx, 204, 304), which made net/http return
ErrBodyNotAllowed and logged 'write err: http: request method or
response status code does not allow body' for ordinary requests.

Generated with [Devin](https://devin.ai)

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:43:32 -07:00
Chris LuandDevin 0b197867a1 filer leveldb2: tunable cache, buffer, block and table sizes (#11927)
* filer leveldb2: read cache and buffer sizes from filer.toml

The leveldb2 store hardcoded its block cache (32MiB) and write buffer
(16MiB) and offered no way to tune the table block size or compaction
table size. Optional block_cache_size_mb, write_buffer_size_mb,
block_size_kb and compaction_table_size_mb keys under [leveldb2] now
override them; zero or negative values keep the previous defaults.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2: document tuning options in scaffold filer.toml

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2: saturate size options instead of overflowing int

The option fields take int, so a MiB config value past math.MaxInt/MiB
(e.g. 2048 on 32-bit) wrapped negative and goleveldb would panic sizing
the memtable. Clamp the product instead of the input.

* filer leveldb2: reject size options too large for int bytes

Clamping a MiB setting at math.MaxInt silently hands goleveldb a
different size than configured; fail Initialize on values that cannot
fit instead.

* filer leveldb3: same tunable cache, buffer, block and table sizes

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* filer leveldb2/3: exercise oversized-option rejection on 64-bit too

The tests skipped whenever int was 64-bit, but write_buffer_size_mb
values above MaxInt/1MiB still overflow there, so the rejection path
had no coverage on the primary platform. Use MaxInt/1MiB + 1 so the
rejection triggers on every platform.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:39:06 -07:00
Chris LuandDevin 1451096c17 volume.fsck: delete missing filer entries through a worker pool (#11924)
* volume.fsck: delete missing filer entries through a worker pool

-reallyDeleteFilerEntries issued one synchronous HTTP DELETE per orphan
path inside the BFS consumer, so a million-entry purge ran one request
at a time. Queue the confirmed paths onto a bounded pool (default 16,
-tunable via -purgeConcurrency) so deletes run in parallel while the
traversal continues.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume.fsck: serialize command output across purge workers

* volume.fsck: serialize purges that share a hard link id

* volume.fsck: cover the purge worker pool

* volume.fsck: take the purge shard remainder in unsigned arithmetic

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:37:57 -07:00
Junker der Provinz 4854c6a0f4 s3: skip the local filer socket on Windows (#11921)
The filer only listens on its unix socket when GOOS is not windows, so
with a filer socket path configured on Windows the S3 server dialed a
socket that never exists.

Fixes #7278
2026-10-21 20:35:19 -07:00
74da88fc29 s3, iceberg: record the access key a request was signed with (#11901)
* s3, iceberg: record the access key a request was signed with

Several access keys can belong to one identity, but nothing downstream of
authentication says which one signed a request: the audit log carries the
identity name only, and every table credential the Iceberg catalog vends for
that identity gets the same session name. When an identity is shared (one key
per person or per service), the log cannot tell the holders apart.

The verifier that matched a credential now records that credential's access
key id in the request's identity holder (verifyV4Signature for header,
presigned, streaming and STS requests; both SigV2 verifiers). Nothing re-reads
the request for a key, so a second, unverified key elsewhere in the request
(a junk X-Amz-Credential on a SigV2 request, a junk SigV4 header on a SigV2
presigned URL) is never recorded.

- s3err.AccessLog gains access_key_id (omitempty): a static key's id, or an
  STS session's temporary key id. Every other field is unchanged.
- The Iceberg and Lance catalogs install the holder before authenticating a
  SigV4 request; a Bearer token records the key it was minted from (the token
  is verified with a key derived from it); a Lance x-api-key records its key.
- VendTableCredentials ends the session name with "-k" and the first 12 hex
  digits of SHA-256 of the key id (the table part gives way within the
  64-character limit; the tag is never cut), and carries the key id in the
  session token (src_ak, SessionInfo.SourceAccessKeyId).

Tests: auth_access_key_attribution_test.go and auth_access_key_shapes_test.go
(two keys on one identity, the deny path, SigV2 and presigned SigV2 with a
junk key elsewhere, streaming seed, STS session, failed signatures record
nothing), s3api_iceberg_session_name_test.go, iceberg and lance Auth tests,
sts source_access_key_test.go.

* s3: record the access key of a verified POST-policy form and of a retargeted signature

Two verified signatures left access_key_id empty in the audit log:

- A browser POST upload is verified in PostPolicyBucketHandler from its form
  fields by doesPolicySignatureV2Match / doesPolicySignatureV4Match, which did
  not record the key. Both now take the request context and record the
  credential's key once its signature verifies; a form that fails
  verification records nothing.
- verifyV4Signature refuses a signature that verified for one virtual-hosted
  bucket but was routed to another. The key was recorded only after that
  check, so the refusal carried none. It is now recorded as soon as a host
  candidate verifies.

Test: TestPostPolicyUploadRecordsTheSigningKey (SigV4 and SigV2 forms, and a
form with a wrong signature) and TestSignedVirtualHostRetargetRecordsTheSigningKey;
both fail before this change.

* s3: test VendTableCredentials end to end for the caller's access key

TestVendTableCredentialsRecordsTheCallersKey mints sessions through
VendTableCredentials with a configured STS service, the key recorded the way
the catalog's authentication records it, then validates each session token:
SourceAccessKeyId and the session name's key tag match the key, and a caller
without a key gets an untagged session. It fails if the vendor stops passing
either field.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:33:08 -07:00
Jaehoon KimandChris Lu 744ab586c5 fix: volume.fsck purges live chunks of files renamed during its filer walk (SQL stores) (#11883)
* shell: volume.fsck reads the filer from one store snapshot when it can

volume.fsck collects the filer's chunk references by walking directory
by directory, so each directory is seen at a different time. A file
renamed from a directory not yet listed into one already listed is never
seen, its chunks look like orphans, and -reallyDeleteFromVolume deletes
live data. Checkpoint rotation and write-to-tmp-then-rename hit this.

Add an optional filer store capability, EntryScanner, that visits every
entry below a directory from one consistent snapshot. The SQL stores
implement it with one REPEATABLE READ read-only transaction read in
primary-key pages, so a concurrent rename is seen under exactly one of
its names. Key-value rows are skipped by their dirhash, a bucket
directory whose table is missing fails the scan (from the snapshot it
cannot be told from one dropped a moment ago), and hard links are
resolved in the same transaction. SQLite offers it only in WAL mode, on
its own connection.

TraverseBfsMetadata serves the snapshot when asked via request metadata
sw-traverse-mode=snapshot and reports the mode in a response header;
without the metadata it walks as before, so peer bootstrap and admin
export are unchanged. Snapshot responses carry no entry_ts_ns, whose
per-entry lookup would cost more than the scan.

volume.fsck requests the snapshot and falls back to the walk on an older
filer or a store that cannot scan.

* filer, shell: fail a scan that cannot start, not fall back to the walk

A transient BeginTx failure and a preliminary GetEntry error were treated
like a missing capability or missing root, so fsck would take the
directory walk - the very path that misses renamed-away entries. Only a
genuinely unsupported scan or a confirmed-absent root may fall back.

* filer: only snapshot-listed bucket names keep a missing table fatal

A cached-only name, whose bucket may have been deleted between copying
the cache and opening the snapshot, is skipped when its table is gone.

* filer: roll back to a savepoint before skipping a cached-only table

On PostgreSQL a failed statement aborts the snapshot transaction, so
ignoring a missing-table error without the savepoint still left every
later cached-only table unreadable.

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 20:31:42 -07:00
364e19a493 remote_storage/s3: use the AWS default credential chain when a source is configured (#11881)
* remote_storage/s3: use the AWS default credential chain when a source is configured

Type s3 forced anonymous credentials whenever no access/secret key was
set, so IRSA, container credentials, profiles and env keys were never
consulted. Keep anonymous (public bucket) access only when the environment
names no AWS credential source.

* remote_storage/s3: also recognize shared-config and default-profile env sources

* remote_storage/s3: resolve env credentials on their own transport

A guarded HTTP client is for the S3 endpoint; the remote credential
providers it would also serve reach link-local addresses that guard must
not dial. Profiles load shared config, and a test resolves env keys.

* remote_storage/s3: clear every detected credential source in the test

* remote_storage/s3: resolve the full default chain on the guarded client

The eager credential session used NewSession, whose chain covers only
env, shared-credentials and ECS/EC2 providers. Web identity and
assume-role profiles resolve through shared config, so IRSA never
worked when a custom HTTP client was pinned. Enable shared config and
carry the configured region so STS can sign assume-role requests.

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:30:28 -07:00
df33e81915 volume: report the needle read error with a failed read (#11934)
* volume: report the needle read error with a failed read

A read that fails on the volume server, for example on a CRC mismatch,
returned a bare 500, so mount and filer logs only showed the status.
The volume server now writes the error into the response body, and the
read helpers add it to the error they return.

Co-authored-by: Bruce <82158894+half-life666@users.noreply.github.com>

* volume: keep the 500 bare on HEAD requests

writeJsonError's body write fails with ErrBodyNotAllowed under net/http
when the request method is HEAD, adding an error-writing log line on top
of the failed read. Send only the status, matching the no-error path.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util/http: decode gzip-encoded error bodies in statusError

Full-chunk reads set Accept-Encoding: gzip themselves, so the
transport does not transparently decompress; a gzip-encoded error
body reached statusError as compressed bytes and produced an
unreadable message. statusError now takes the response and decodes
Content-Encoding before extracting the bounded message.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: log the read error when the 500 stays bare

On HEAD requests the body write is skipped, which also skipped the
V(0) response log inside writeJson — a failed HEAD read left no
cause in normal volume-server logs. Log the error in the same
format before the bare 500.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util/http: check the status before decoding gzip in GetAuthenticated

The gzip reader eagerly consumes the header when it is created, so
statusError saw a drained or corrupt body on a gzipped error response
and the server message was lost. Match the other read helpers and check
the status first.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util/http: export StatusError

Other volume read paths outside this file need the same error body.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume: redact jwt query param when logging failed-read URLs

The new read-failure logs print the request URL, and volume reads accept
?jwt= as a credential, so a failed authenticated read would write the
bearer token to the volume server log.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* util/http: surface the error body on the remaining chunk-read paths

* volume: redact duplicate jwt params in failed-read logs

* operation: pass the lookup jwt on the chunk needle read

---------

Co-authored-by: Bruce <82158894+half-life666@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 20:24:48 -07:00
Chris LuandDevin 867a537802 volume.configure: keep the superblock in sync with the .vif (#11930)
* volume.configure: keep the superblock in sync with the .vif

ConfigureVolume only rewrote the .vif's replication field, leaving the
.dat superblock stale. A mounted volume also kept reporting the old
placement in heartbeats, and its next compaction would rewrite the stale
placement into a fresh .dat.

Parse the placement once, then update the live volume when mounted: the
in-memory .vif copy and superblock, and the .dat header through the
volume's own backend under the data-file locks. For unmounted volumes
rewrite the header directly; remote-only volumes skip it since the .vif
carries the placement.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* volume.configure: update the .dat superblock in the Rust volume server

Mirror the Go ConfigureVolume change: after rewriting the .vif, rewrite
the replica placement in the .dat header too, so a remounted volume's
superblock, heartbeats, and later compactions agree with the .vif. A
missing .dat (remote-only volume) is skipped since the .vif carries the
placement.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:18:56 -07:00
Chris LuandDevin 85c1b990b1 shell: bound the initial master connection wait behind -connectTimeout (#11922)
* shell: bound the initial master connection wait behind -connectTimeout

weed shell waited on WaitUntilConnected with a background context, so
an unreachable master hung the command forever. The wait now runs on
a 30s (configurable) timeout and reports the failure instead.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: stop the master reconnect loop when RunShell returns

KeepConnectedToMaster ran on context.Background(), so once RunShell
returned — including the new connect-timeout early return — the
reconnect goroutine kept dialing forever, and repeated RunShell calls
accumulated one each. Give it its own cancellable context, separate
from connectCtx whose deadline must not cut reconnects under a live
shell.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: bound filer discovery by -connectTimeout too

The discovery WithClient and ListClusterNodes ran on the unbounded
background context, so a master disconnect after the initial connect —
or a persistently failing ListClusterNodes — could block startup past
the deadline. Reusing connectCtx bounds the whole startup phase.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: report filer discovery failures instead of silently losing the filer target

* shell: give filer discovery its own budget after master connects

Discovery shared connectCtx, so a connection that landed near the
-connectTimeout deadline left ListClusterNodes almost no time and no room
for WithClient to retry. A fresh timeout keeps discovery bounded without
starving it.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: cover the -connectTimeout master wait with a test

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* shell: note filer discovery in the -connectTimeout help text

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:17:51 -07:00
ac319fe318 volume: never compact a volume tiered to remote storage (#11895)
* volume: never compact a volume tiered to remote storage

A tiered volume's data backend is the remote object, but compaction reads
the local .dat path. When tiering removed the local .dat, every vacuum of a
tiered volume with garbage fails with "no such file or directory", and the
master's sweep and the worker's vacuum task keep selecting it. When the
local .dat was kept, the compaction succeeds against the stale copy and the
commit reloads the volume to read the remote object (old layout) through the
compacted index, so live needles stop reading.

Mirror the Rust volume server (#11545):
- CompactByIndex and CompactByVolumeData refuse a volume with a remote file
  before creating .cpd/.cpx ("volume N is tiered to remote storage, cannot
  compact").
- CommitCompact refuses under the data-file locks when the volume became
  tiered during the copy, and removes .cpd/.cpx.
- Store.CheckCompactVolume reports no garbage for a tiered volume, so the
  master's vacuum and volume.vacuum do not select it.
- The worker's vacuum detection skips a volume with a remote copy, as balance
  and EC detection already do.

Tests (each fails on master): TestCompactRefusesTieredVolume,
TestCommitCompactRefusesVolumeTieredDuringCopy (on master the commit
succeeds and a surviving needle no longer reads),
TestPickPrimaryReplicaSkipsTieredVolumes.

* volume: skip tiered replicas in vacuum selection, keep their real ratio

Reporting a zero garbage ratio for a tiered volume hid it from
volume.tier.compact, whose preflight reads that ratio while the volume is
still remote, and did not exclude it from a vacuum with a zero threshold.
Report the real ratio again, as the Rust volume server does, and have the
master's vacuum check skip a replica whose volume info names remote
storage, whatever the threshold or request.

When CommitCompact refuses a volume tiered during the copy, also close the
temporary needle map and remove .cpldb: nothing else would, since no later
vacuum selects the volume.

The tiered-compaction test no longer depends on random needle sizes
reaching a 0.3 garbage ratio.

* volume tier: do not attach a copy of a .dat a compaction replaced

The upload copies the .dat, then records the remote file in the .vif and
switches the backend. A compaction commit in between swaps in a compacted
.dat and index; the upload then served the old-layout remote object through
the new index, so live needles stopped reading. CommitCompact's tiered
check cannot see this: the volume is not tiered until the switch.

AttachRemoteDatFile records the remote file and switches the backend under
the data-file locks, and refuses when the backend is no longer the file
that was copied. The upload then removes the unused remote copy.

* volume tier: keep a committed .vif entry when only the dir fsync fails

AttachRemoteDatFile rolled back the in-memory remote entry on any
SaveVolumeInfo error. A NotCrashDurableError comes after the rename, so
the .vif on disk already names the copy; with the entry gone, the upload
handler deleted that object, and a reload would serve from a missing
remote file. Keep the entry and switch to the copy as the .vif says,
and still return the error so the caller keeps the local .dat.

* vacuum worker: filter tiered replicas instead of skipping the volume

A partially tiered volume is the leftover of an interrupted
volume.tier.upload; skipping the whole volume left garbage on its local
replicas unreclaimed. Per-replica filtering matches the master's
batchVacuumVolumeCheck and is safe: a tiered replica refuses both
compaction and commit.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 20:16:02 -07:00
Chris LuandDevin 164a7d8b56 upload: upload -dir files concurrently behind a new -c flag (#11918)
* upload: upload -dir files concurrently behind a new -c flag

Each file under -dir was submitted serially inside filepath.Walk, so
every small file paid a sequential master-assign plus upload round
trip. A bounded pool now runs those submissions concurrently while
keeping results in walk order; failures are reported per file and the
command exits non-zero when any upload failed.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* submit: close opened readers when a batch assign fails

SubmitFiles returns early when the assign RPC fails, before
FilePart.Upload gets a chance to close each opened file. Close the
readers on that path so repeated assign failures do not leak handles.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* upload: stream -dir results as each file completes

Batching the output until every upload finished meant an interrupted
run lost the file ids of completed uploads. A per-index ready channel
lets a printer emit each result in walk order as soon as it lands.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* submit: close the opened file when Stat fails

newFilePart returned the still-open handle with the error, and the
concurrent upload path records the error without ever reaching
SubmitFiles, leaking one descriptor per failing file.

* upload: still upload files collected before a walk error

A walk that aborts partway used to leave the already-uploaded files
behind; collecting paths first must not drop them, so upload what was
gathered and still exit non-zero.

* upload: submit batched files concurrently

SubmitFiles fetched all file ids in one assign RPC but then uploaded
the files one after another, so a multi-file list still spent its time
in serial per-file round trips even with -dir concurrency. Uploads now
run through a small worker pool while results keep their input order.

Verified end to end: a 5-file list uploads with consecutive file keys
and each stored needle matches its source bytes.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* upload: print per-file results before exiting on a partial batch failure

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:08:57 -07:00
chuanying chenandChris Lu d35f4c1c6b Missing data read in super block extra parsing (#8392)
* storage: fail on short super block extra read even without an error

Some backends (e.g. s3_backend) clear io.EOF after a short read, returning
n < len(p) with a nil error. Checking n only when the error is non-nil
lets truncated extra data reach proto.Unmarshal with a zero-filled tail.

* storage: apply the same short-read check to the super block header

A backend that returns n < SuperBlockSize with a nil error previously
slipped through and parsed a partially zero-filled header.

* storage: test super block extra reads

Covers a round trip through ReadSuperBlock, a truncated extra region, a
backend returning io.EOF alongside a full read, and a backend returning
a short read with a nil error.

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 20:07:52 -07:00
1aad4cfe70 iam: optionally require an iam: grant for self-service access-key actions (#11937)
* iam: optionally require an iam: grant for self-service access-key actions

AuthorizeIamAction lets every authenticated identity run CreateAccessKey,
DeleteAccessKey, UpdateAccessKey, ListAccessKeys and GetUser on itself with
no grant. On AWS these actions need an IAM policy (the usual one grants them
on arn:aws:iam::*:user/${aws:username}); nothing is allowed implicitly.

Where identities hold data-plane rights only, the implicit carve-out lets a
leaked key mint further keys for its own identity, with no cap, that keep
working after the operator deletes the key that leaked; it can also
deactivate or delete the keys the operator issued.

A new option, off by default, makes these actions need the iam:<Action> grant
like every other IAM action: -s3.iam.selfServiceRequiresGrant on weed
server, filer and mini, -iam.selfServiceRequiresGrant on weed s3. With it
set, the grant is authorized against the target user's ARN, the caller's own
when UserName is empty, so a policy can confine it to the caller's own keys
(Resource arn:aws:iam:::user/${aws:username}), the same resource scoping
CreateServiceAccount already uses. The admin and iam:* grants are unchanged;
with the option off nothing changes.

Tests (s3api_embedded_iam_authz_test.go): with the option set, an identity
holding only s3:* and a bare identity are refused all five actions on an
empty, own and other target (fails without the change); a grant on the
caller's own user ARN, literal or ${aws:username}, allows them on itself and
not on another user; the admin and an iam:* grant keep every action; with the
option off self-service is allowed as before.

* iam: wire selfServiceRequiresGrant on the standalone iam server

Every other command that runs the embedded IAM API accepts the flag; the
deprecated weed iam could not enable the enforcement at all.

* iam: wire selfServiceRequiresGrant on weed server and filer -iam

The IamOptions used by 'weed server -iam' and 'weed filer -iam' never
received the flag, so startIamServer dereferenced a nil pointer at
startup. Share the -s3.iam.selfServiceRequiresGrant pointer with them.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 20:06:30 -07:00
Chris LuandDevin ea859a0844 sftp: reject empty passwords in PasswordAuthenticator (#11938)
* sftp: reject empty passwords in PasswordAuthenticator

FileStore.ValidatePassword already refuses a zero-length stored or
supplied password, but the guard belongs at the authenticator too: an
empty password is never valid, and checking before the store call keeps
any user.Store implementation without its own empty-secret check from
matching on zero-length input.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* sftp: keep the brute-force delay on empty-password failures

The early rejection returned before the shared failure path, so an empty
password failed instantly while other failures slept 100-199 ms. Folding
the length check into the store call keeps the store untouched and every
failed attempt equally slow.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:03:14 -07:00
Chris LuandDevin c0152e6be1 s3: keep '\' a literal object-key character behind -s3.allowBackslashInKey (#11911)
* s3: keep '\' a literal object-key character when AllowBackslashInKey is set

NormalizeObjectKey rewrites every S3 object key: '\' becomes '/' and
duplicate slashes collapse. On AWS S3 both are ordinary key characters,
so keys that are distinct there alias one object here — writing
probe/back/slash.txt silently overwrites probe/back\slash.txt.

The filer only treats '/' as a path separator, so a verbatim '\' stays
inside one entry name. Gate the backslash rewrite and the matching '\'
handling in IsValidObjectKey behind a process-wide mode the S3 server
sets from the new S3ApiServerOption. Duplicate-slash collapsing stays:
an entry name cannot contain an empty segment.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* command: add -s3.allowBackslashInKey flag for AWS-compatible key handling

Registers the flag on all four S3 entry points (s3, server -s3,
filer -s3, mini) and passes it through S3Options into
S3ApiServerOption. Default off, so existing normalization is
unchanged.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: correct IsValidObjectKey/NormalizeObjectKey docs under allowBackslashInKey

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: normalize DeleteObjects body keys the same as path keys

DeleteObjects keys arrive in the XML body, so they bypassed the
normalization applied to route-var keys. The lock, the authorization
checks, the table-bucket layout check and the filer delete itself all
addressed the raw key while the conditional check resolved the
normalized one, so a key spelled with '\' or '//' could be reported
deleted while the stored entry survived. Echo the client-sent key in
the result while operating on the normalized form.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: report a normalized-empty DeleteObjects key as a delete no-op

A multi-delete key of only slashes normalizes to the empty string. Left
to reach the filer delete, the empty name resolves to the bucket entry
itself, so an unauthorized caller could remove a bucket through the
body-key path. Such a key can never name a stored entry, so like a
missing key the delete is an idempotent no-op reported before any
permission check or filer call.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: drop unused AllowBackslashInObjectKey getter

* s3: answer conditional delete of a normalized-empty key with NoSuchKey

* s3: normalize the probed key in authorizeAnonymousBucketList

The masked-miss probe read the raw mux var, so a spelling like folder//
or folder\ was authorized against a key that can never exist while the
request itself targets the normalized folder/. Probe the normalized key
so the answer matches what the read path authorizes.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: cover anonymous authz of a literal-backslash key under allowBackslashInKey

A public grant on the slash key must not authorize an anonymous read of
the distinct backslash object when the flag keeps '\' literal; grant the
backslash key's own ACL and it reads back. Default mode still folds the
spelling onto the covered slash key.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: publish the key mode only after the S3 server is constructed

A failed NewS3ApiServerWithStore could previously flip the process-wide
key mode out from under an already-running gateway.

* s3: normalize the copy-source key before validation and authorization

pathToBucketObjectAndVersion returned the raw header spelling, so an
equivalent spelling (a//b, a\b) was authorized and compared against the
normalized destination while resolveCopySourceEntry looked up the
normalized key — a deny on the canonical key could be bypassed and
self-copy detection missed. Normalize once at parse so the CopyObject and
UploadPartCopy paths agree.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 20:02:04 -07:00
e057244b0a storage: read the super block extra before unmarshaling it (#11932)
* storage: read the super block extra before unmarshaling it

ReadSuperBlock allocated a buffer of ExtraSize bytes but never filled it,
so any volume with a super block extra failed to load with an invalid
wire-format error.

Co-authored-by: chuanying chen <18412672+ccy00808@users.noreply.github.com>

* storage: test super block extra read error paths

---------

Co-authored-by: chuanying chen <18412672+ccy00808@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 19:55:27 -07:00
Javier de la TorreandJavier de la Torre 763e2a2090 iceberg: record geometry bounds on the data files compaction writes (#11898)
* iceberg: record geometry bounds on the data files compaction writes

Compaction collected column statistics for every primitive column except
geometry: the bound converter has no geometry case, so a compacted file's
manifest entry carried value and null counts for a geometry column but no
lower or upper bound. Readers that skip files by these bounds for a spatial
filter (for example DuckDB for `geom && <box>`) had to open every compacted
file, where the files before compaction could be skipped.

The bound follows the spec's single-value serialization for geometry: the
lower bound is the point (xmin, ymin), the upper bound (xmax, ymax), each an
8-byte little-endian double; z and m are not written. It is the union of the
written file's row-group geospatial statistics, which parquet-go computes
from every non-null value.

A bound that is too small makes a reader skip rows, so any doubt gives none:
a row group with non-null values but no geospatial statistics (parquet-go
omits them when a value does not parse as WKB), a box that is not finite, an
inverted box, or a chunk whose statistics cannot be read. A row group of
nulls only constrains nothing. Geography gets no bound, as iceberg-go writes
none for it (geodesic edges, boxes across the antimeridian).

Tests: points and polygons far apart through a bin-packed compaction (the
bound equals the union of every geometry), a sorted compaction (any bound
covers every geometry), a column of nulls only, an unparseable value in a
second row group (no bound; without the rule the file got the bound of the
first row group only), geography, and the aggregation rules.

* iceberg: do not order compaction bins by a geometry column

When a table declares no usable sort order, compaction orders a bin's files
by the first column every file has bounds for. A v3 table whose first column
is a geometry qualifies as soon as its files carry geometry bounds (DuckDB
writes them, and compacted files now do too). Those bounds decode into geo
literals that compareLiterals cannot compare, so the order sorted nothing,
yet an oversized bin was still split into "ordered" runs by splitOrderedBin.

Geometry and geography columns are now skipped in both the declared sort
order and the inferred one, so such a table is ordered by its next column
with bounds.

Test: TestResolveCompactionOrderSkipsGeometry, a geometry column first and
an id column second; on master the order is the geometry column.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
2026-10-21 19:28:26 -07:00
917b6f4afd iceberg: authorize a commit before staging its metadata file (#11891)
* iceberg: authorize a commit before staging its metadata file

Both commit paths (CommitTable and CommitTransaction) write the new
vN.metadata.json into the table's metadata/ directory and only then reach
UpdateTable, whose authorization refuses a caller that may read the table but
not commit to it. The refused commit then deletes the staged file on the way
out, so such a caller could make the server write and delete a file under the
table's prefix on every attempt. The same shape was closed for creates in
#10991.

Both paths now ask Manager.AuthorizeUpdateTable right after loading the table
and answer 403 ForbiddenException before anything is staged. UpdateTable still
decides under the policies in force when the pointer flips, so a policy changed
mid-commit is caught as before.

Test: TestCommitDeniedBeforeStagingMetadata records every file the handlers
create; on master both the single-table and the transaction commit by a
GetTable-only identity write v2.metadata.json before answering 403, with the
change nothing is written. TestCommitAllowedStagesMetadata is the positive
control (the owner, and an identity whose own actions include UpdateTable).

* iceberg: authorize a view commit before staging its metadata

handleUpdateView has the same write-before-authorize gap the table commit
paths had: it stages the new metadata file and only reaches the UpdateView
authorization at the pointer flip, so a caller who may only load the view
could still make the server write and delete files under it. Gate it on
AuthorizeUpdateView, mirroring authorizeCommit.

* iceberg: seed committed view metadata in the denied-commit test

Without stored view metadata the request fails on the missing-metadata
check before it could stage a file, so the zero-write assertion could
not catch a write-before-denial regression. Seed the same metadata the
allowed-path test uses so the denied request would otherwise stage.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-21 19:13:02 -07:00
Chris LuandDevin 106fd31af1 s3: resolve versionId ACL and attributes requests to the version-scoped actions (#11916)
* s3: resolve versionId ACL requests to the version ACL actions

A request for an object's ACL carrying versionId acted on the historical version but authorized against the current-object action (s3:GetObjectAcl/s3:PutObjectAcl), because the acl block in resolveFromQueryParameters returned before versionId was considered - the same gap closed for tagging. An identity granted only current-object ACL access - or explicitly denied s3:*ObjectVersionAcl - could still read and modify ACLs on noncurrent versions.

?acl&versionId=<id> on an object now resolves to s3:GetObjectVersionAcl/s3:PutObjectVersionAcl; ?acl alone keeps the current-object actions and bucket ACL is unaffected. s3:PutObjectVersionAcl is mapped onto the WriteAcp identity action so policies can name it.

* s3: resolve versionId attributes requests to the version attributes action

GET /bucket/key?attributes&versionId=<id> fetched the historical version's attributes but authorized against s3:GetObjectAttributes, so a grant limited to the current-object attributes action could reach noncurrent versions. The request now resolves to s3:GetObjectVersionAttributes, and both attributes actions map onto the Read identity action so policies can name them (s3:GetObjectAttributes was previously rejected as an unmapped action).

* s3api: recheck the version action on versioned ACL/attributes requests

The phase-2 policy recheck inside GetObjectAclHandler,
PutObjectAclHandler, and GetObjectAttributesHandler passed the
current-object action even when the request carried versionId and phase 1
had resolved the version action. A Deny scoped to s3:*VersionAcl or
s3:GetObjectVersionAttributes could never fire in the recheck, and an
Allow limited to the version action went unevaluated. Select the action
the same way the resolver does — versioned whenever versionId is present.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: recheck the version action on versioned tagging requests

The same phase-2 gap exists in the tagging handlers left by the earlier
version-tagging resolution change: versioned GetObjectTagging /
PutObjectTagging / DeleteObjectTagging rechecks evaluated the
current-object tagging action, so a Deny on s3:*VersionTagging could not
fire.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: resolve HEAD ?acl as the head-object action, not an ACL action

No HEAD acl route exists, so HEAD /bucket/key?acl lands on
HeadObjectHandler — which serves object headers, not the ACL. Resolving
it to an ACL action denied readers who hold only GetObject(Version) and
let ACL grants reach a head-object read. Dropping HEAD from the acl
branch lets the request fall through to GetObject/GetObjectVersion,
matching how the tagging branch already treats HEAD.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: require the version read grant for versioned attributes

AWS asks for s3:GetObjectVersion alongside s3:GetObjectVersionAttributes
when a GetObjectAttributes request names a versionId; the resolved
version-scoped action on its own left a lone s3:GetObjectVersion Deny
unable to cover the attributes read. Check the version read grant too:
the identity side through VerifyActionPermission, the bucket-policy side
through the same recheck that evaluates the scoped action.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: check the version read grant on a synthesized versioned GET

The versioned attributes check passed the literal s3:GetObjectVersion
action, but both policy paths re-resolve the action from the request, so
the attributes query parameter mapped it back to
s3:GetObjectVersionAttributes and a lone version-read Deny never fired.
The literal action also reached the legacy CanDo path, where a plain
Read grant no longer matched. Evaluate the grant on a request shaped
like a plain versioned GET instead.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: reject a named versionId on buckets without versioning

Tagging, ACL, and attributes handlers answered versionId requests on
never-versioned buckets as the current object while evaluating the
version-scoped action, so a grant limited to the version action could
authorize changes to the live entry. Reject the request up front through
the existing checkVersionIdConfigured gate.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: carry presign params onto synthesized policy requests

objectKeyPolicyRequest rebuilt the query with only versionId and the STS
token, so a presigned request's s3:authType condition value silently
changed from REST-QUERY-STRING to unset during phase-2 rechecks (version
read grant, copy source, delete keys, post-policy writes). A policy
conditioned on s3:authType would then deny a request it already allowed.
Carry the SigV2/SigV4 presign keys through; none of them steer
ResolveS3Action.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 16:50:34 -07:00
Chris LuandDevin 137f23a229 ec distribution: implement the documented shard distribution API (#11915)
* ec distribution: parse EC and replication configs into a shard plan

NewECConfig/DefaultECConfig and NewReplicationConfig(FromString) turn an
EC ratio and a replica placement into ECConfig/ReplicationConfig values,
and CalculateDistribution derives the target and maximum shard counts per
data center, rack, and node. ECDistribution.Summary and
FaultToleranceAnalysis describe, per topology level, whether losing one
failure domain leaves enough shards to reconstruct the volume, so the EC
durability guarantees of a given setup can be answered directly instead of
only documented informally.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: analyze topology and plan shard moves

TopologyAnalysis records nodes and shard locations per DC/rack/node, and
Rebalancer.PlanRebalance emits the moves that level the layout: data
centers first, then racks within each DC, then nodes within each rack,
moving parity shards before data shards and honoring per-node free slots.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: cover the plan math and rebalancing with tests

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: keep AddShardLocation and node ShardIDs in sync

AddShardLocation updated the count maps but never the node's ShardIDs —
yet every rebalancer count reads len(node.ShardIDs), so the documented
AddNode-then-AddShardLocation flow produced an empty rebalance plan. It
now appends to the node's shard list and skips shards the node already
has, making the two registration styles equivalent and idempotent.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: key rack-scoped maps by dc/rack

Rack names are only unique within a data center — two DCs can both hold a
"rack1" — so ShardsByRack, RackToShards, and RackToNodes keyed by the bare
rack name merged distinct domains and corrupted both the DC-level shard
counts and rack-level balancing. Rack lookups now use the dc/rack pair;
DCToRacks still lists bare rack names.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: never move a shard onto a node already holding it

The move picker selected the destination node before the shard, so a node
already holding the chosen shard could be targeted. Pick the shard first
(parity-first) and choose among destination nodes that lack it.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: tolerate zero-value replication configs

A ReplicationConfig{} with zeroed minimums made maxShare divide by zero;
clamp each level's unit count to at least one. Covers the new behaviors
with tests.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: keep rack keys distinct when names contain "/"

rackKey joined data center and rack with "/", so dc "a"/rack "b/c" and
dc "a/b"/rack "c" shared one key and were counted as the same failure
domain. Separate the parts with a NUL byte, which cannot appear in a
name.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: plan rebalances on a copy of the topology

recordMove mutated the caller's TopologyAnalysis while planning, so an
interrupted or partial execution left the input describing moves that
never happened, and replanning on it returned nothing. PlanRebalance
now simulates on a cloned analysis; plan.Analysis reports the layout
the moves would produce and the caller's input is left untouched.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: balance nodes against the rack's current shard count

The node-level target still read the original analysis, so a rack that
had just sent or received shards during DC/rack balancing aimed at a
stale total and left its nodes unbalanced.

* ec distribution: register each node id once

Adding the same node twice appended it to RackToNodes twice, so every
count that walks rack nodes double-counted its shards.

* ec distribution: normalize replication counts in the stored plan

A zero-value ReplicationConfig computed targets against one domain per
level while Summary still reported zero DCs, racks, and nodes.

* ec distribution: cap a survivable level's max at the loss budget

* ec distribution: skip shards already inside the destination domain

* ec distribution: register nodes implied by shard locations

* ec distribution: merge repeat node registrations

A shard location now auto-registers an unknown node with no free slots;
a later AddNode carrying the real capacity was then dropped entirely.
Repeat registration now updates the node's capacity and adopts any new
shards, while the first registration keeps the topology placement.

* ec distribution: try other overloaded sources before stopping

When the most overloaded domain could not shed any shard, balance
returned early and skipped every other source that still had movable
shards. A source that cannot move now gets set aside for the rest of
the pass.

* ec distribution: collapse chained shard moves to the final destination

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ec distribution: schedule collapsed moves behind the departures that free slots

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 16:50:01 -07:00
Junker der ProvinzandDan Foster e645449872 fix(storage): count preallocated space when auto-sizing max volumes (#11919)
The max volume count subtracted each volume's remaining room from the
free disk space, except when the master had preallocation turned on,
in which case it subtracted nothing. That is wrong for a disk that
mixes preallocated and plain volumes, and on platforms where
preallocation is not supported.

Use the larger of the .dat file's size and its allocated blocks, in the
Go and the Rust volume server, so a preallocated volume counts as full
and a plain one keeps its remaining room reserved.

Fixes #5778

Co-authored-by: Dan Foster <4570306+danfoster@users.noreply.github.com>
2026-10-21 15:11:06 -07:00
527 changed files with 31080 additions and 4152 deletions

No files matched your search

@@ -0,0 +1,11 @@
name: Clean up stale SeaweedFS test mounts
description: Detach disconnected FUSE test mounts left by earlier jobs on a shared runner.
runs:
using: composite
steps:
- name: Unmount disconnected SeaweedFS test mounts
if: runner.os == 'Linux' && runner.environment == 'self-hosted'
shell: bash
env:
CLEANUP_SCRIPT: ${{ github.action_path }}/cleanup.py
run: python3 "$CLEANUP_SCRIPT"
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""Detach disconnected SeaweedFS integration-test mounts without touching live ones."""
import errno
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tempfile
def test_mounts(mountinfo):
"""Read the kernel table directly: traversing a stale mount may block."""
roots = {tempfile.gettempdir(), "/tmp", "/var/tmp"}
for line in mountinfo.splitlines():
fields, filesystem = line.split(" - ", 1)
if filesystem.split()[0].split(".")[0] != "fuse":
continue
fields = fields.split()
# mountinfo escapes whitespace and backslashes with octal sequences.
target = re.sub(r"\\([0-7]{3})", lambda m: chr(int(m[1], 8)), fields[4])
path = Path(target)
# Test mounts are a mount* leaf under a seaweedfs_fuse_* scratch dir
# (seaweedfs_fuse_test_*/mount, seaweedfs_fuse_failover_*/mount0, ...)
# or the posix-compliance workflow's fixed /tmp/seaweedfs/mount.
if (not path.name.startswith("mount")
or str(path.parent.parent) not in roots
or not (path.parent.name.startswith("seaweedfs_fuse")
or path.parent.name == "seaweedfs")):
continue
yield fields[0], target
def mounted(mount_id, target):
return (mount_id, target) in test_mounts(Path("/proc/self/mountinfo").read_text())
def cleanup():
failed = False
for mount_id, target in test_mounts(Path("/proc/self/mountinfo").read_text()):
try:
# A separate, bounded probe prevents a wedged FUSE daemon from
# hanging cleanup. Only ENOTCONN proves the daemon disconnected;
# a timeout or another error is not enough to detach a live mount.
probe = subprocess.run(
[sys.executable, __file__, "--probe", target], timeout=5,
capture_output=True, check=False,
)
except subprocess.TimeoutExpired:
print(f"::warning::FUSE probe timed out; leaving {target} mounted")
continue
if probe.returncode != errno.ENOTCONN or not mounted(mount_id, target):
continue
print(f"Unmounting disconnected FUSE test mount: {target}", flush=True)
# Lazy unmount detaches a dead mount even when an old process still
# holds it open. Prefer the unprivileged FUSE helper; sudo is fallback.
for command in (
["fusermount3", "-uz", "--", target],
["fusermount", "-uz", "--", target],
["sudo", "-n", "umount", "-l", "--", target],
):
try:
subprocess.run(command, timeout=10, check=False)
except (OSError, subprocess.TimeoutExpired) as error:
print(f"{command[0]}: {error}", file=sys.stderr)
if not mounted(mount_id, target):
break
else:
print(f"::error::Could not unmount disconnected FUSE test mount: {target}")
failed = True
continue
# The detached mount's parent still holds the crashed run's data and
# logs; test_mounts only yields targets under a SeaweedFS test dir in
# a temp root, so the parent is safe to reclaim here.
shutil.rmtree(Path(target).parent, ignore_errors=True)
return int(failed)
if __name__ == "__main__":
if len(sys.argv) == 3 and sys.argv[1] == "--probe":
try:
os.stat(sys.argv[2])
except OSError as error:
sys.exit(error.errno)
sys.exit(0)
sys.exit(cleanup())
@@ -0,0 +1,137 @@
import errno
import importlib.util
from pathlib import Path
import subprocess
import unittest
from unittest.mock import patch
spec = importlib.util.spec_from_file_location("cleanup", Path(__file__).with_name("cleanup.py"))
cleanup = importlib.util.module_from_spec(spec)
spec.loader.exec_module(cleanup)
TARGET = "/tmp/seaweedfs_fuse_test_123/mount"
MOUNTINFO = f"42 1 0:9 / {TARGET} rw - fuse.seaweedfs seaweedfs rw\n"
class CleanupTests(unittest.TestCase):
def run_cleanup(self, results, mounted=True):
with patch.object(cleanup.Path, "read_text", return_value=MOUNTINFO), \
patch.object(cleanup, "mounted", side_effect=mounted if isinstance(mounted, list) else None,
return_value=mounted) as check, \
patch.object(cleanup.shutil, "rmtree") as rmtree, \
patch.object(cleanup.subprocess, "run", side_effect=results) as run:
status = cleanup.cleanup()
return status, run, check, rmtree
def result(self, status):
return subprocess.CompletedProcess([], status)
def test_only_fuse_integration_test_mounts_are_candidates(self):
rows = MOUNTINFO
rows += MOUNTINFO.replace("fuse.seaweedfs", "ext4")
rows += MOUNTINFO.replace(TARGET, "/mnt/production")
rows += MOUNTINFO.replace(TARGET, "/tmp/seaweedfs_fuse_test_123/data")
rows += MOUNTINFO.replace(TARGET, "/home/user/seaweedfs_fuse_test_123/mount")
rows += MOUNTINFO.replace(TARGET, "/tmp/seaweedfs/data")
self.assertEqual(list(cleanup.test_mounts(rows)), [("42", TARGET)])
def test_all_test_mount_layouts_are_candidates(self):
targets = (
"/tmp/seaweedfs_fuse_test_123/mount",
"/tmp/seaweedfs_fuse_failover_456/mount0",
"/tmp/seaweedfs_fuse_failover_456/mount1",
"/var/tmp/seaweedfs_fuse_p2p_test_789/mount2",
"/tmp/seaweedfs/mount",
)
rows = "".join(
MOUNTINFO.replace("42", str(40 + i)).replace(TARGET, target)
for i, target in enumerate(targets)
)
self.assertEqual(
list(cleanup.test_mounts(rows)),
[(str(40 + i), target) for i, target in enumerate(targets)],
)
def test_mountinfo_paths_are_unescaped_once(self):
target = "/tmp/seaweedfs_fuse_test_space and\\slash/mount"
encoded = target.replace("\\", r"\134").replace(" ", r"\040")
rows = MOUNTINFO.replace(TARGET, encoded)
self.assertEqual(list(cleanup.test_mounts(rows)), [("42", target)])
def test_no_mounts_is_a_noop(self):
with patch.object(cleanup.Path, "read_text", return_value=""), \
patch.object(cleanup.subprocess, "run") as run:
self.assertEqual(cleanup.cleanup(), 0)
run.assert_not_called()
def test_healthy_mount_is_untouched(self):
status, run, _, _ = self.run_cleanup([self.result(0)])
self.assertEqual(status, 0)
self.assertEqual(run.call_count, 1)
def test_other_probe_errors_are_untouched(self):
for error in (errno.EACCES, errno.ENOENT, errno.EIO):
with self.subTest(error=error):
status, run, _, _ = self.run_cleanup([self.result(error)])
self.assertEqual(status, 0)
self.assertEqual(run.call_count, 1)
def test_probe_timeout_does_not_detach_a_possibly_live_mount(self):
status, run, _, _ = self.run_cleanup([subprocess.TimeoutExpired("probe", 5)])
self.assertEqual(status, 0)
self.assertEqual(run.call_count, 1)
def test_disconnected_mount_is_lazily_unmounted(self):
status, run, _, _ = self.run_cleanup([self.result(errno.ENOTCONN), self.result(0)], [True, False])
self.assertEqual(status, 0)
self.assertEqual(run.call_args_list[1].args[0], ["fusermount3", "-uz", "--", TARGET])
self.assertEqual(run.call_args_list[0].kwargs["timeout"], 5)
self.assertEqual(run.call_args_list[1].kwargs["timeout"], 10)
def test_replaced_or_removed_mount_is_untouched(self):
status, run, _, _ = self.run_cleanup([self.result(errno.ENOTCONN)], False)
self.assertEqual(status, 0)
self.assertEqual(run.call_count, 1)
def test_mount_identity_is_checked_without_stat(self):
with patch.object(cleanup.Path, "read_text", return_value=MOUNTINFO):
self.assertTrue(cleanup.mounted("42", TARGET))
self.assertFalse(cleanup.mounted("43", TARGET))
def test_missing_or_timed_out_helper_uses_fallback(self):
for error in (FileNotFoundError(), subprocess.TimeoutExpired("fusermount3", 10)):
with self.subTest(error=error):
status, run, _, _ = self.run_cleanup(
[self.result(errno.ENOTCONN), error, self.result(0)], [True, True, False])
self.assertEqual(status, 0)
self.assertEqual(run.call_args.args[0], ["fusermount", "-uz", "--", TARGET])
def test_failed_helpers_fall_back_to_sudo(self):
status, run, _, _ = self.run_cleanup(
[self.result(errno.ENOTCONN), self.result(1), self.result(1), self.result(0)],
[True, True, True, False])
self.assertEqual(status, 0)
self.assertEqual(run.call_args.args[0], ["sudo", "-n", "umount", "-l", "--", TARGET])
def test_cleanup_fails_when_stale_mount_remains(self):
status, run, _, rmtree = self.run_cleanup(
[self.result(errno.ENOTCONN), self.result(1), self.result(1), self.result(1)])
self.assertEqual(status, 1)
self.assertEqual(run.call_count, 4)
rmtree.assert_not_called()
def test_unmounted_test_dir_is_reclaimed(self):
status, _, _, rmtree = self.run_cleanup(
[self.result(errno.ENOTCONN), self.result(0)], [True, False])
self.assertEqual(status, 0)
rmtree.assert_called_once_with(Path(TARGET).parent, ignore_errors=True)
def test_probe_process_reports_errno(self):
for target, expected in ((str(Path(__file__)), 0), (str(Path(__file__) / "absent"), errno.ENOTDIR)):
with self.subTest(target=target):
result = subprocess.run([cleanup.sys.executable, cleanup.__file__, "--probe", target], check=False)
self.assertEqual(result.returncode, expected)
if __name__ == "__main__":
unittest.main()
@@ -23,8 +23,14 @@ runs:
bin=$(command -v fusermount3 || true)
if [ -z "$bin" ]; then
echo "no fusermount3 in PATH" >&2
exit 1
# Self-hosted runners are not guaranteed to carry fuse3.
sudo apt-get update
sudo apt-get install -y fuse3
bin=$(command -v fusermount3 || true)
if [ -z "$bin" ]; then
echo "no fusermount3 in PATH after installing fuse3" >&2
exit 1
fi
fi
if can_mount "$bin"; then
ls -l "$bin"
+30
View File
@@ -0,0 +1,30 @@
name: "actions: unit tests"
on:
push:
branches: [ master ]
paths:
- '.github/actions/**'
- '.github/workflows/action-tests.yml'
pull_request:
branches: [ master ]
paths:
- '.github/actions/**'
- '.github/workflows/action-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
cleanup-stale-fuse-mounts:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Run cleanup action unit tests
run: python3 .github/actions/cleanup-stale-fuse-mounts/test_cleanup.py
+8 -2
View File
@@ -9,8 +9,8 @@ on:
- '.github/workflows/codeql.yml'
concurrency:
group: ${{ github.head_ref }}/codeql
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
CodeQL-Build:
@@ -25,6 +25,12 @@ jobs:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4.39.0
+4
View File
@@ -8,6 +8,10 @@ on:
pull_request:
branches: [master]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
+12 -1
View File
@@ -13,6 +13,10 @@ on:
- 'go.sum'
- '.github/workflows/container_dev.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -20,7 +24,7 @@ jobs:
# ── Pre-build the Rust binaries natively ────────────────────────────
build-rust-binaries:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -51,6 +55,7 @@ jobs:
echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -193,6 +198,12 @@ jobs:
platforms: linux/amd64, linux/arm64
tags: ${{ steps.docker_meta.outputs.tags }}
labels: ${{ steps.docker_meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Key the checkout layer on the pushed sha so cached layers can't
# serve stale code; the Dockerfile fetch fallback picks it up.
build-args: |
BRANCH=${{ github.sha }}
- name: Sign
if: github.event_name != 'pull_request'
@@ -24,6 +24,10 @@ on:
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.event.inputs.image_tag || format('foundationdb_{0}_seaweedfs_{1}', github.event.inputs.fdb_version || '7.4.5', github.event.inputs.seaweedfs_ref || github.ref_name) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -64,6 +68,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: ${{ runner.environment == 'github-hosted' }}
- name: Run FoundationDB tagged tests
env:
+12 -3
View File
@@ -30,6 +30,11 @@ on:
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.event.inputs.image_tag || github.run_id }}
cancel-in-progress: false
queue: max
permissions:
contents: read
security-events: write
@@ -61,7 +66,7 @@ jobs:
# ── Pre-build the Rust binaries natively ────────────────────────────
build-rust-binaries:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -94,6 +99,7 @@ jobs:
echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -164,10 +170,13 @@ jobs:
uses: actions/checkout@v7
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.source_ref || github.ref }}
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Free Disk Space
run: |
echo "Available disk space before cleanup:"
df -h
timeout 15 df -h || true
# Remove pre-installed tools
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
# Clean package managers
@@ -179,7 +188,7 @@ jobs:
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
echo "Available disk space after cleanup:"
df -h
timeout 15 df -h || true
- name: Configure variant
id: config
@@ -11,6 +11,11 @@ on:
required: true
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || format('refs/tags/{0}', github.event.inputs.release_tag) || github.run_id }}
cancel-in-progress: false
queue: max
permissions:
contents: read
@@ -25,7 +25,7 @@ on:
rocksdb_version:
description: 'RocksDB git tag to use when variant=rocksdb'
required: false
default: 'v10.10.1'
default: 'v11.1.2'
permissions:
contents: read
@@ -47,7 +47,7 @@ jobs:
# amd64 and arm64 without QEMU, turning a 5-hour emulated cargo build into
# ~15 minutes of native compilation.
build-rust-binaries:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -78,6 +78,7 @@ jobs:
echo "CFLAGS_aarch64_unknown_linux_musl=-U_FORTIFY_SOURCE" >> "$GITHUB_ENV"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -179,6 +180,10 @@ jobs:
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Download pre-built Rust binaries
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/download-artifact@v8
@@ -218,7 +223,7 @@ jobs:
sudo docker system prune -af --volumes
[ -d ~/.cache/go-build ] && rm -rf ~/.cache/go-build || true
[ -d /go/pkg ] && rm -rf /go/pkg || true
df -h
timeout 15 df -h || true
- name: Docker meta
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -270,8 +275,11 @@ jobs:
file: ${{ matrix.dockerfile }}
platforms: ${{ matrix.platform }}
labels: ${{ steps.docker_meta.outputs.labels }}
# Flat single-platform manifest so imagetools create assembles cleanly.
provenance: false
# mode=max keeps resolvedDependencies complete and records the
# builder identity; attestations ride inside each pushed index and
# survive the imagetools merge into the multi-arch manifest.
provenance: mode=max
sbom: true
outputs: type=image,name=${{ env.IMAGE }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=${{ matrix.variant }}-${{ matrix.arch }}
# max only for rocksdb: its RocksDB compile is sha-independent and worth
@@ -281,7 +289,7 @@ jobs:
${{ matrix.build_args }}
BUILDKIT_INLINE_CACHE=1
BRANCH=${{ github.sha }}
${{ matrix.variant == 'rocksdb' && format('ROCKSDB_VERSION={0}', github.event.inputs.rocksdb_version || 'v10.10.1') || '' }}
${{ matrix.variant == 'rocksdb' && format('ROCKSDB_VERSION={0}', github.event.inputs.rocksdb_version || 'v11.1.2') || '' }}
- name: Export digest
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
@@ -307,6 +315,8 @@ jobs:
permissions:
contents: read
id-token: write
attestations: write
packages: write
strategy:
fail-fast: false
matrix:
@@ -414,6 +424,45 @@ jobs:
${{ env.IMAGE }}@${{ steps.manifest.outputs.digest }}
chrislusf/seaweedfs@${{ steps.manifest.outputs.digest }}
# Attest the merged multi-arch digest itself, so digest-pinning consumers
# can verify provenance against the exact index they pull. Both steps
# reuse the GHCR login above and push as OCI referrers.
- name: Attest build provenance for ${{ matrix.variant }}
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: actions/attest-build-provenance@v4
with:
subject-name: ${{ env.IMAGE }}
subject-digest: ${{ steps.manifest.outputs.digest }}
push-to-registry: true
# SBOMs are generated and attested per platform manifest: an index-wide
# scan only describes syft's default platform, while the manifests it
# merges differ (some ship the real binary, others placeholders), so the
# SBOM has to be bound to the platform digest a consumer actually pulls.
- name: Install syft
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: anchore/sbom-action/download-syft@v0
- name: Attest per-platform SBOMs for ${{ matrix.variant }}
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
working-directory: /tmp/digests
run: |
set -euo pipefail
for f in *; do
# Each saved digest is an OCI index (image + attestation manifests);
# scan and attest the image manifest a same-platform pull resolves.
ref="${{ env.IMAGE }}@sha256:${f}"
image=$(crane manifest "$ref" | jq -r '
[.manifests[]? | select((.annotations["vnd.docker.reference.type"] // "") != "attestation-manifest"
and (.platform.os // "unknown") != "unknown") | .digest][0] // empty')
ref="${{ env.IMAGE }}@${image:-sha256:${f}}"
syft "registry:${ref}" -o "spdx-json=/tmp/sbom-${f}.spdx.json"
cosign attest --yes \
--predicate "/tmp/sbom-${f}.spdx.json" \
--type spdxjson \
"$ref"
done
# Report-only trivy scan: uploads fixable HIGH/CRITICAL findings to GitHub
# Security for visibility, but never blocks the release. Releases (including
# `latest`) ship regardless — vulnerabilities are tracked, not gated, since
@@ -452,7 +501,9 @@ jobs:
scanners: vuln
vuln-type: os,library
severity: HIGH,CRITICAL
ignore-unfixed: true
# Keep unfixed findings in the report so digest-pinning consumers see
# the complete picture, not just what a patch could resolve.
ignore-unfixed: false
limit-severities-for-sarif: true
format: sarif
output: trivy-results.sarif
@@ -4,9 +4,9 @@ on:
workflow_dispatch:
inputs:
rocksdb_version:
description: 'RocksDB git tag or branch to build (e.g. v10.10.1)'
description: 'RocksDB git tag or branch to build (e.g. v11.1.2)'
required: true
default: 'v10.10.1'
default: 'v11.1.2'
seaweedfs_ref:
description: 'SeaweedFS git tag, branch, or commit to build'
required: true
@@ -16,6 +16,11 @@ on:
required: false
default: ''
concurrency:
group: ${{ github.workflow }}-${{ github.event.inputs.image_tag || format('rocksdb_{0}_seaweedfs_{1}', github.event.inputs.rocksdb_version, github.event.inputs.seaweedfs_ref) }}
cancel-in-progress: false
queue: max
permissions:
contents: read
+1
View File
@@ -27,6 +27,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'telemetry/server/go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build Telemetry Server
if: github.event_name == 'workflow_dispatch' && inputs.deploy
+4
View File
@@ -1,6 +1,10 @@
name: 'Dependency Review'
on: [pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
+5 -3
View File
@@ -19,8 +19,8 @@ on:
- '.github/workflows/e2e.yml'
concurrency:
group: ${{ github.head_ref }}/e2e
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -32,7 +32,7 @@ defaults:
jobs:
e2e:
name: FUSE Mount
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code into the Go module directory
@@ -42,6 +42,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Configure Docker Hub mirror
run: |
@@ -52,6 +53,7 @@ jobs:
uses: docker/setup-buildx-action@v4
- name: Cache Docker layers
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: /tmp/.buildx-cache
+6 -1
View File
@@ -18,18 +18,23 @@ on:
- 'go.sum'
- '.github/workflows/ec-integration-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
ec-integration-tests:
name: EC Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
+4 -3
View File
@@ -19,8 +19,8 @@ on:
- '.github/actions/fix-fusermount-setuid/**'
concurrency:
group: ${{ github.head_ref || github.ref }}/fuse-dlm-integration
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -28,7 +28,7 @@ permissions:
jobs:
fuse-dlm-integration:
name: FUSE DLM Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
@@ -39,6 +39,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Configure FUSE
run: |
+7 -3
View File
@@ -24,8 +24,8 @@ on:
- '.github/actions/fix-fusermount-setuid/**'
concurrency:
group: ${{ github.head_ref || github.ref }}/fuse-failover
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -33,7 +33,7 @@ permissions:
jobs:
fuse-failover:
name: FUSE Volume Server Failover
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 40
steps:
@@ -42,10 +42,14 @@ jobs:
with:
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Configure FUSE
run: |
+4 -3
View File
@@ -17,8 +17,8 @@ on:
- '.github/actions/fix-fusermount-setuid/**'
concurrency:
group: ${{ github.head_ref }}/fuse-integration
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -26,7 +26,7 @@ permissions:
jobs:
fuse-integration:
name: FUSE Integration Testing
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 50
steps:
@@ -37,6 +37,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Configure FUSE
run: |
+7 -3
View File
@@ -25,8 +25,8 @@ on:
- '.github/actions/fix-fusermount-setuid/**'
concurrency:
group: ${{ github.head_ref || github.ref }}/fuse-p2p-integration
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -34,17 +34,21 @@ permissions:
jobs:
fuse-p2p-integration:
name: FUSE P2P Peer Chunk Sharing
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Configure FUSE
run: |
+9 -2
View File
@@ -17,8 +17,8 @@ on:
- '.github/workflows/go.yml'
concurrency:
group: ${{ github.head_ref }}/go
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -35,6 +35,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Get dependencies
run: |
cd weed; go get -v -t -d ./...
@@ -57,6 +58,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Go Vet linux/386 (type-checks code and tests for 32-bit int overflows)
run: |
GOOS=linux GOARCH=386 go vet ./... 2>&1 | grep -v "MessageState contains sync.Mutex" | grep -v "IdentityAccessManagement contains sync.RWMutex" | tee vet-32bit-output.txt
@@ -72,6 +74,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build
run: cd weed; go build -tags "elastic gocdk sqlite ydb tarantool tikv rclone" -v .
@@ -94,6 +97,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# Nothing else on a PR compiles these, so per-OS syscall constants creep
# back into shared files unnoticed and only a release build catches it.
- name: Cross-compile
@@ -128,6 +132,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Test
env:
RUN_REDIS_TESTS: "1"
@@ -143,6 +148,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# 386 test binaries run natively on the amd64 runner. This catches what vet
# can't: unaligned 64-bit atomics and arithmetic that wraps at runtime.
# -short skips the e2e suites already covered on amd64.
@@ -159,6 +165,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# weed/kms/azure is excluded from every default build, so without this
# nothing compiles it and tag-gated code silently rots.
- name: Build and test with azurekms
+4
View File
@@ -8,6 +8,10 @@ on:
branches: [ master ]
paths: ['k8s/**', '.github/workflows/helm_ci.yml', '.github/scripts/helm_jwt_keys.py']
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
+7 -1
View File
@@ -14,6 +14,10 @@ on:
- 'weed/**'
- '.github/workflows/java_integration_tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
test:
name: Java Integration Tests
@@ -31,6 +35,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Java
@@ -38,7 +43,7 @@ jobs:
with:
java-version: ${{ matrix.java }}
distribution: 'temurin'
cache: 'maven'
cache: ${{ runner.environment == 'github-hosted' && 'maven' || '' }}
- name: Build SeaweedFS
run: |
@@ -55,6 +60,7 @@ jobs:
# Start SeaweedFS with optimized settings for CI
# Include S3 API for s3copier integration tests
weed server -dir="$WEED_DATA_DIR" \
-ip=127.0.0.1 -ip.bind=127.0.0.1 \
-master.raftHashicorp \
-master.electionTimeout=1s \
-master.volumeSizeLimitMB=100 \
+1 -1
View File
@@ -77,7 +77,7 @@ jobs:
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'
cache: ${{ runner.environment == 'github-hosted' && 'maven' || '' }}
server-id: central
server-username: MAVEN_CENTRAL_USERNAME
server-password: MAVEN_CENTRAL_PASSWORD
+5 -1
View File
@@ -12,6 +12,10 @@ on:
- 'other/java/**'
- '.github/workflows/java_unit_tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
test:
name: Java Unit Tests
@@ -30,7 +34,7 @@ jobs:
with:
java-version: ${{ matrix.java }}
distribution: 'temurin'
cache: 'maven'
cache: ${{ runner.environment == 'github-hosted' && 'maven' || '' }}
- name: Build and Install SeaweedFS Client
working-directory: other/java/client
+3 -3
View File
@@ -24,8 +24,8 @@ on:
workflow_dispatch: # Allow manual trigger
concurrency:
group: ${{ github.head_ref }}/kafka-quicktest
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -43,7 +43,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
**/go.sum
id: go
+9 -6
View File
@@ -23,8 +23,8 @@ on:
- '.github/workflows/kafka-tests.yml'
concurrency:
group: ${{ github.head_ref }}/kafka-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
# Force different runners for better isolation
env:
@@ -53,6 +53,7 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
@@ -98,6 +99,7 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
@@ -149,7 +151,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
**/go.sum
id: go
@@ -328,7 +330,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
**/go.sum
id: go
@@ -490,7 +492,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
**/go.sum
id: go
@@ -648,7 +650,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
**/go.sum
id: go
@@ -805,6 +807,7 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
+6 -4
View File
@@ -19,8 +19,8 @@ on:
- 'test/kms/**'
concurrency:
group: ${{ github.head_ref || github.ref }}-kms-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -32,7 +32,7 @@ defaults:
jobs:
kms-provider-tests:
name: KMS Provider Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -43,6 +43,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -77,7 +78,7 @@ jobs:
s3-kms-e2e-tests:
name: S3 KMS End-to-End Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
@@ -88,6 +89,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -21,8 +21,8 @@ on:
- '.github/workflows/master-cold-start-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/master-cold-start-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,7 +30,7 @@ permissions:
jobs:
master-cold-start-tests:
name: Master Cold Start Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Check out code
@@ -42,6 +42,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build weed binary
run: |
@@ -25,8 +25,8 @@ on:
- '.github/workflows/metadata-subscribe-tests.yml'
concurrency:
group: ${{ github.head_ref }}/metadata-subscribe-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -37,7 +37,7 @@ env:
jobs:
metadata-subscribe-integration:
name: Metadata Subscribe Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -48,6 +48,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS
run: |
+4 -2
View File
@@ -10,8 +10,8 @@ on:
branches: [ 'winfsp-bench**' ]
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -32,6 +32,7 @@ jobs:
- uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install WinFsp and rclone
run: choco install winfsp rclone -y --no-progress
@@ -150,6 +151,7 @@ jobs:
- uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Repair the fusermount3 setuid bit
uses: ./.github/actions/fix-fusermount-setuid
@@ -24,8 +24,8 @@ on:
- '.github/workflows/mount-windows-conformance.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -49,6 +49,7 @@ jobs:
- uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# cgofuse loads winfsp-x64.dll at run time, so WinFsp is needed here but
# not to build.
+3 -2
View File
@@ -24,8 +24,8 @@ on:
- '.github/workflows/mount-windows.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -49,6 +49,7 @@ jobs:
- uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
# cgofuse loads winfsp-x64.dll at run time, so WinFsp is needed here but
# not to build.
+4 -3
View File
@@ -21,8 +21,8 @@ on:
- '.github/workflows/multi-master-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/multi-master-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,7 +30,7 @@ permissions:
jobs:
multi-master-failover-tests:
name: Multi-Master Failover Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Check out code
@@ -40,6 +40,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install SeaweedFS
run: |
+7 -4
View File
@@ -45,8 +45,8 @@ on:
type: string
concurrency:
group: ${{ github.head_ref || github.ref }}/performance
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -67,6 +67,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build weed
run: go build -o weed_bin ./weed
@@ -175,13 +176,14 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install Rust toolchain
if: matrix.impl == 'rust'
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and target
if: matrix.impl == 'rust'
if: matrix.impl == 'rust' && runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -327,13 +329,14 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install Rust toolchain
if: matrix.impl == 'rust'
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and target
if: matrix.impl == 'rust'
if: matrix.impl == 'rust' && runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
+10 -2
View File
@@ -27,7 +27,7 @@ permissions:
jobs:
pjdfstest:
name: pjdfstest
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
@@ -38,15 +38,23 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Start local Docker registry
run: docker run -d --restart=always -p 5000:5000 --name registry registry:2
run: |
if ! docker start registry 2>/dev/null; then
docker rm -f registry >/dev/null 2>&1 || true
docker run -d --restart=always -p 5000:5000 --name registry registry:2
fi
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
driver-opts: network=host
- name: Set up Docker Compose
uses: docker/setup-compose-action@v2
- name: Build weed race binary
run: |
cd docker
+6 -1
View File
@@ -18,13 +18,17 @@ on:
- 'go.sum'
- '.github/workflows/plugin-workers.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
plugin-worker:
name: "Plugin Worker: ${{ matrix.worker }}"
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
fail-fast: false
@@ -41,6 +45,7 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
+791
View File
@@ -0,0 +1,791 @@
name: POSIX Compliance Tests
on:
push:
branches: [ main, master, develop ]
paths:
- 'weed/mount/**'
- 'weed/command/mount*'
- 'weed/command/fuse*'
- 'test/fuse_integration/**'
pull_request:
branches: [ main, master, develop ]
paths:
- 'weed/mount/**'
- 'weed/command/mount*'
- 'weed/command/fuse*'
- 'test/fuse_integration/**'
workflow_dispatch:
inputs:
test_type:
description: 'Type of POSIX tests to run (default: full for comprehensive testing)'
required: true
default: 'full'
type: choice
options:
- critical
- basic
- extended
- full
enable_external_tests:
description: 'Run external test suites (slower)'
required: false
default: false
type: boolean
permissions:
contents: read
pull-requests: write
issues: write
env:
GO_VERSION: '1.26'
TIMEOUT: '45m'
jobs:
posix-compliance-ubuntu:
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
fuse-version: ['2.9', '3.0']
fail-fast: false
steps:
- name: Checkout code
uses: actions/checkout@v7
with:
fetch-depth: 1
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go ${{ env.GO_VERSION }}
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
cache: ${{ runner.environment == 'github-hosted' }}
cache-dependency-path: |
go.sum
test/fuse_integration/go.sum
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
attr \
acl \
build-essential \
git \
python3-pip
# Install FUSE version specific packages
if [ "${{ matrix.fuse-version }}" = "2.9" ]; then
echo "Installing FUSE 2.9..."
sudo apt-get install -y fuse libfuse-dev
elif [ "${{ matrix.fuse-version }}" = "3.0" ]; then
echo "Installing FUSE 3.0..."
sudo apt-get install -y fuse3 libfuse3-dev
# Also install fuse2 for compatibility if needed
sudo apt-get install -y fuse libfuse-dev
fi
- name: Set up user permissions for FUSE
run: |
# Create fuse group if it doesn't exist
sudo groupadd -f fuse
# Add user to fuse group
sudo usermod -a -G fuse $USER
# Set permissions on FUSE device
sudo chmod 666 /dev/fuse
# Allow non-root mounts to pass -o allow_other (weed -allowOthers)
echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
sudo chmod 644 /etc/fuse.conf
# Ensure fuse module is loaded
sudo modprobe fuse || true
# Verify setup
echo "FUSE setup verification:"
ls -la /dev/fuse
groups $USER | grep fuse || echo "User not in fuse group yet (will be effective after login)"
lsmod | grep fuse || echo "FUSE module not loaded"
- name: Install external test tools
if: ${{ github.event.inputs.enable_external_tests == 'true' }}
run: |
# Install nfstest for POSIX API verification
pip3 install --user nfstest
# Install FIO for performance testing
sudo apt-get install -y fio
# Install additional test utilities
sudo apt-get install -y stress-ng
- name: Build SeaweedFS
run: |
make
# Verify binary exists and is executable (it's installed to GOPATH/bin)
weed version
# Make weed binary available in PATH (it's already there from go install)
sudo cp $(which weed) /usr/local/bin/weed
which weed
weed version
- name: Set up SeaweedFS cluster
run: |
start_and_wait() {
local name=$1
local url=$2
local pidfile=$3
shift 3
local cmd="$@"
echo "Starting $name..."
$cmd > "/tmp/seaweedfs/$name.log" 2>&1 &
local pid=$!
echo $pid > "$pidfile"
echo "Waiting for $name to start..."
for i in {1..30}; do
if curl -sf "$url" > /dev/null 2>&1; then
echo "$name is ready"
return 0
fi
if [ $i -eq 30 ]; then
echo "$name failed to start"
cat "/tmp/seaweedfs/$name.log"
exit 1
fi
sleep 2
done
}
mkdir -p /tmp/seaweedfs/{master,volume,filer,mount}
start_and_wait "master" "http://127.0.0.1:9333/cluster/status" "/tmp/seaweedfs/master.pid" \
"weed master -ip=127.0.0.1 -port=9333 -mdir=/tmp/seaweedfs/master -raftBootstrap=true"
start_and_wait "volume" "http://127.0.0.1:8080/status" "/tmp/seaweedfs/volume.pid" \
"weed volume -mserver=127.0.0.1:9333 -ip=127.0.0.1 -port=8080 -dir=/tmp/seaweedfs/volume -max=100"
start_and_wait "filer" "http://127.0.0.1:8888/" "/tmp/seaweedfs/filer.pid" \
"weed filer -master=127.0.0.1:9333 -ip=127.0.0.1 -port=8888"
echo "SeaweedFS cluster status:"
curl -s http://127.0.0.1:9333/cluster/status || true
- name: Set up FUSE mount
run: |
# Create mount point
MOUNT_POINT="/tmp/seaweedfs/mount"
mkdir -p $MOUNT_POINT
echo "Mounting SeaweedFS FUSE filesystem..."
# Mount SeaweedFS FUSE filesystem in background
weed mount \
-filer=127.0.0.1:8888 \
-dir=$MOUNT_POINT \
-filer.path=/ \
-dirAutoCreate=true \
-allowOthers=true \
-nonempty=true \
> /tmp/seaweedfs/mount.log 2>&1 &
MOUNT_PID=$!
echo $MOUNT_PID > /tmp/seaweedfs/mount.pid
# Wait for mount to be ready. The mount point directory already
# exists, so only a real mount counts — a directory listing would
# pass on the local disk even if weed mount died.
echo "Waiting for FUSE mount to be ready..."
for i in {1..30}; do
if mountpoint -q $MOUNT_POINT 2>/dev/null; then
echo "FUSE mount is ready"
break
fi
if ! kill -0 $MOUNT_PID 2>/dev/null; then
echo "FUSE mount process exited"
cat /tmp/seaweedfs/mount.log
exit 1
fi
if [ $i -eq 30 ]; then
echo "FUSE mount failed to be ready"
cat /tmp/seaweedfs/mount.log
echo "Mount status:"
mount | grep fuse || echo "No FUSE mounts found"
ps aux | grep weed || echo "No weed processes found"
exit 1
fi
sleep 2
done
# Verify mount is working
echo "Testing FUSE mount functionality..."
echo "test" > $MOUNT_POINT/test_file.txt
if [ "$(cat $MOUNT_POINT/test_file.txt)" = "test" ]; then
echo "FUSE mount is working correctly"
rm $MOUNT_POINT/test_file.txt
else
echo "FUSE mount is not working properly"
exit 1
fi
# Export mount point for tests
echo "SEAWEEDFS_MOUNT_POINT=$MOUNT_POINT" >> $GITHUB_ENV
- name: Set up test environment
run: |
cd test/fuse_integration
# Initialize Go module for tests
if [ ! -f go.mod ]; then
go mod init seaweedfs-posix-tests
go mod tidy
fi
# Create reports directory
mkdir -p reports
# Set up external tools if requested
# Note: External tools setup is now optional and won't fail the build
if [ "${{ github.event.inputs.enable_external_tests }}" = "true" ]; then
echo "Setting up external POSIX testing tools..."
make -f posix_Makefile setup-external-tools || true
# setup-external-tools exits 0 even when a sub-target failed, so
# require the tools only when they are actually installed
if [ -d external_tools/pjdfstest ] && command -v nfstest_posix >/dev/null 2>&1; then
export POSIX_REQUIRE_EXTERNAL_TOOLS="true"
echo "POSIX_REQUIRE_EXTERNAL_TOOLS=true" >> $GITHUB_ENV
echo "✅ External tools setup completed successfully"
else
echo "⚠️ External tools setup incomplete - continuing with basic tests only"
echo "External tools will be skipped but tests will continue"
echo "This is expected behavior when build tools are not available"
fi
fi
# Verify SeaweedFS cluster is accessible
echo "Verifying SeaweedFS cluster accessibility..."
curl -sf http://127.0.0.1:9333/cluster/status
curl -sf http://127.0.0.1:8080/status
curl -sf http://127.0.0.1:8888/
# Verify mount point
echo "Mount point: $SEAWEEDFS_MOUNT_POINT"
ls -la $SEAWEEDFS_MOUNT_POINT
# Verify mount is actually working by testing file operations
echo "Verifying SeaweedFS mount functionality..."
TEST_FILE="$SEAWEEDFS_MOUNT_POINT/.mount_verification_test"
echo "SeaweedFS mount test" > "$TEST_FILE"
if [ "$(cat "$TEST_FILE")" = "SeaweedFS mount test" ]; then
echo "✅ Mount verification successful"
rm -f "$TEST_FILE"
else
echo "❌ Mount verification failed - mount may not be working properly"
exit 1
fi
# Check if this is actually a FUSE mount — the plain directory created
# above passes the file test, so without this the suite can run
# against the local disk and report a pass it never earned
if mount | grep -q "$SEAWEEDFS_MOUNT_POINT"; then
echo "✅ FUSE mount detected:"
mount | grep "$SEAWEEDFS_MOUNT_POINT"
else
echo "❌ Mount point not found in mount table - refusing to test a local directory"
exit 1
fi
- name: Run POSIX compliance tests
id: posix-tests
env:
SEAWEEDFS_MOUNT_POINT: ${{ env.SEAWEEDFS_MOUNT_POINT }}
POSIX_REQUIRE_EXTERNAL_TOOLS: ${{ env.POSIX_REQUIRE_EXTERNAL_TOOLS }}
run: |
cd test/fuse_integration
# Determine which tests to run
TEST_TYPE="${{ github.event.inputs.test_type }}"
if [ -z "$TEST_TYPE" ]; then
TEST_TYPE="full"
fi
echo "Running POSIX tests: $TEST_TYPE"
if [ "$TEST_TYPE" = "full" ]; then
echo "🚀 Running COMPREHENSIVE POSIX test suite (including external test suites)"
else
echo "Using mount point: $SEAWEEDFS_MOUNT_POINT"
fi
# Set test configuration to use our mounted filesystem
export TEST_MOUNT_POINT="$SEAWEEDFS_MOUNT_POINT"
export TEST_SKIP_CLUSTER_SETUP="true"
# Debug: Show environment variables
echo "🔍 Test Environment:"
echo " TEST_MOUNT_POINT=$TEST_MOUNT_POINT"
echo " TEST_SKIP_CLUSTER_SETUP=$TEST_SKIP_CLUSTER_SETUP"
echo " SEAWEEDFS_MOUNT_POINT=$SEAWEEDFS_MOUNT_POINT"
case "$TEST_TYPE" in
"critical")
echo "Running critical POSIX tests (basic compliance only)..."
make -f posix_Makefile test-posix-critical
;;
"basic")
echo "Running basic POSIX tests (core file operations)..."
make -f posix_Makefile test-posix-basic
;;
"extended")
echo "Running extended POSIX tests (including advanced features)..."
make -f posix_Makefile test-posix-extended
;;
"full")
echo "🚀 Running COMPREHENSIVE POSIX test suite (all tests including external)..."
echo " This includes: basic, extended, and external test suites"
echo " External suites: pjdfstest, nfstest_posix, and custom stress tests"
make -f posix_Makefile test-posix-full
;;
*)
echo "Unknown test type: $TEST_TYPE, defaulting to full suite"
echo "🚀 Running full POSIX test suite (all tests including external)..."
make -f posix_Makefile test-posix-full
;;
esac
- name: Run external test suites
if: ${{ github.event.inputs.enable_external_tests == 'true' }}
continue-on-error: true
run: |
cd test/fuse_integration
# Point the external suites at the same mount as the Go tests;
# without these the Makefile falls back to a local-disk path.
export TEST_MOUNT_POINT="$SEAWEEDFS_MOUNT_POINT"
export TEST_SKIP_CLUSTER_SETUP="true"
# Run external tests (may fail on some systems)
make -f posix_Makefile test-nfstest-posix || echo "nfstest failed or not available"
make -f posix_Makefile test-fio-posix || echo "FIO tests failed or not available"
- name: Generate compliance report
if: always()
run: |
cd test/fuse_integration
make -f posix_Makefile generate-report
- name: Collect system information for debugging
if: failure()
run: |
cd test/fuse_integration
mkdir -p reports/debug
echo "System Information" > reports/debug/system_info.txt
uname -a >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "Mount Information" >> reports/debug/system_info.txt
mount >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "Disk Space" >> reports/debug/system_info.txt
timeout 15 df -h >> reports/debug/system_info.txt || true
echo "" >> reports/debug/system_info.txt
echo "Memory Information" >> reports/debug/system_info.txt
free -h >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "FUSE Information" >> reports/debug/system_info.txt
ls -la /dev/fuse >> reports/debug/system_info.txt
lsmod | grep fuse >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "SeaweedFS Version" >> reports/debug/system_info.txt
weed version >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "Go Version" >> reports/debug/system_info.txt
go version >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
echo "Running Processes" >> reports/debug/system_info.txt
ps aux | grep -E "(weed|fuse)" >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
# Collect SeaweedFS service logs
echo "=== SeaweedFS Service Logs ===" >> reports/debug/system_info.txt
if [ -f /tmp/seaweedfs/master.log ]; then
echo "Master Log:" >> reports/debug/system_info.txt
tail -50 /tmp/seaweedfs/master.log >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
fi
if [ -f /tmp/seaweedfs/volume.log ]; then
echo "Volume Log:" >> reports/debug/system_info.txt
tail -50 /tmp/seaweedfs/volume.log >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
fi
if [ -f /tmp/seaweedfs/filer.log ]; then
echo "Filer Log:" >> reports/debug/system_info.txt
tail -50 /tmp/seaweedfs/filer.log >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
fi
if [ -f /tmp/seaweedfs/mount.log ]; then
echo "Mount Log:" >> reports/debug/system_info.txt
tail -50 /tmp/seaweedfs/mount.log >> reports/debug/system_info.txt
echo "" >> reports/debug/system_info.txt
fi
# Copy full logs as separate files
if [ -f /tmp/seaweedfs/master.log ]; then
cp /tmp/seaweedfs/master.log reports/debug/master.log
fi
if [ -f /tmp/seaweedfs/volume.log ]; then
cp /tmp/seaweedfs/volume.log reports/debug/volume.log
fi
if [ -f /tmp/seaweedfs/filer.log ]; then
cp /tmp/seaweedfs/filer.log reports/debug/filer.log
fi
if [ -f /tmp/seaweedfs/mount.log ]; then
cp /tmp/seaweedfs/mount.log reports/debug/mount.log
fi
- name: Cleanup SeaweedFS cluster and FUSE mount
if: always()
run: |
stop_process() {
local name=$1
local pidfile=$2
if [ -f "$pidfile" ]; then
local pid=$(cat "$pidfile")
if kill -0 $pid 2>/dev/null; then
echo "Stopping $name process (PID: $pid)..."
kill -TERM $pid || true
sleep 2
kill -KILL $pid 2>/dev/null || true
fi
fi
}
echo "Cleaning up SeaweedFS cluster and FUSE mount..."
MOUNT_POINT="/tmp/seaweedfs/mount"
if mountpoint -q $MOUNT_POINT 2>/dev/null; then
echo "Unmounting FUSE filesystem..."
fusermount -u $MOUNT_POINT || umount $MOUNT_POINT || true
fi
stop_process "mount" "/tmp/seaweedfs/mount.pid"
stop_process "filer" "/tmp/seaweedfs/filer.pid"
stop_process "volume" "/tmp/seaweedfs/volume.pid"
stop_process "master" "/tmp/seaweedfs/master.pid"
pkill -f "weed " || true
fusermount -u $MOUNT_POINT 2>/dev/null || true
umount $MOUNT_POINT 2>/dev/null || true
rm -rf /tmp/seaweedfs || true
echo "Cleanup completed"
- name: Upload test results
uses: actions/upload-artifact@v7
if: always()
with:
name: posix-test-results-ubuntu-fuse${{ matrix.fuse-version }}
path: |
test/fuse_integration/reports/
test/fuse_integration/*.log
retention-days: 30
- name: Upload test coverage
uses: actions/upload-artifact@v7
if: always() && (github.event.inputs.test_type == 'full' || github.event.inputs.test_type == 'extended')
with:
name: posix-coverage-ubuntu
path: test/fuse_integration/reports/posix_coverage.html
retention-days: 14
- name: Comment PR with results
if: github.event_name == 'pull_request' && always()
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const path = require('path');
const reportPath = 'test/fuse_integration/reports/posix_compliance_summary.txt';
if (fs.existsSync(reportPath)) {
const report = fs.readFileSync(reportPath, 'utf8');
const comment = `## POSIX Compliance Test Results
**Go Version:** ${{ env.GO_VERSION }}
**FUSE Version:** ${{ matrix.fuse-version }}
**Test Type:** ${{ github.event.inputs.test_type || 'full' }}
<details>
<summary>Test Summary</summary>
\`\`\`
${report}
\`\`\`
</details>
📊 Full results available in [test artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
`;
try {
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
});
} catch (err) {
console.log('Could not post PR comment: ' + err.message);
}
}
posix-compliance-macos:
runs-on: macos-latest
timeout-minutes: 60
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install macFUSE
run: |
# Install macFUSE
brew install macfuse
# Note: macFUSE may require system extension approval on macOS
# This step may need manual intervention in some cases
- name: Build SeaweedFS
run: |
make
# Verify binary exists and is executable (it's installed to GOPATH/bin)
weed version
- name: Run critical POSIX tests (macOS)
continue-on-error: true # macOS FUSE can be more restrictive
run: |
cd test/fuse_integration
if [ ! -f go.mod ]; then
go mod init seaweedfs-posix-tests
go mod tidy
fi
mkdir -p reports
# Run basic tests only on macOS due to FUSE limitations
make -f posix_Makefile test-posix-critical
- name: Upload macOS test results
uses: actions/upload-artifact@v7
if: always()
with:
name: posix-test-results-macos
path: |
test/fuse_integration/reports/
test/fuse_integration/*.log
retention-days: 30
security-analysis:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install security analysis tools
run: |
echo "Installing staticcheck for security analysis..."
go install honnef.co/go/tools/cmd/staticcheck@latest
- name: Run security analysis on FUSE code
run: |
# Analyze mount and FUSE-related code for security and quality issues
echo "Running staticcheck security analysis..."
# Run staticcheck on FUSE-related code
echo "Analyzing FUSE mount code..."
staticcheck ./weed/mount/... > staticcheck-report.txt 2>&1 || true
staticcheck ./weed/command/mount* ./weed/command/fuse* >> staticcheck-report.txt 2>&1 || true
# Convert to JSON format for consistency
echo "{" > gosec-report.json
echo " \"tool\": \"staticcheck\"," >> gosec-report.json
echo " \"timestamp\": \"$(date -Iseconds)\"," >> gosec-report.json
echo " \"issues\": [" >> gosec-report.json
if [ -s staticcheck-report.txt ]; then
echo " \"$(cat staticcheck-report.txt | head -20 | sed 's/"/\\"/g' | tr '\n' ' ')\"" >> gosec-report.json
fi
echo " ]," >> gosec-report.json
echo " \"stats\": {" >> gosec-report.json
echo " \"files_analyzed\": $(find ./weed/mount ./weed/command -name '*.go' | wc -l)," >> gosec-report.json
echo " \"issues_found\": $(wc -l < staticcheck-report.txt 2>/dev/null || echo 0)" >> gosec-report.json
echo " }" >> gosec-report.json
echo "}" >> gosec-report.json
echo "Security analysis completed"
echo "Issues found: $(wc -l < staticcheck-report.txt 2>/dev/null || echo 0)"
- name: Upload security analysis results
uses: actions/upload-artifact@v7
if: always()
with:
name: security-analysis-results
path: gosec-report.json
retention-days: 30
performance-baseline:
runs-on: ubuntu-latest
if: github.event.inputs.test_type == 'full' || github.event_name == 'schedule'
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y fuse libfuse-dev fio
sudo usermod -a -G fuse $USER
sudo chmod 666 /dev/fuse
# The posix suites mount with -allowOthers; allow_other needs this
echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
- name: Build SeaweedFS
run: make
- name: Run performance baseline tests
run: |
cd test/fuse_integration
if [ ! -f go.mod ]; then
go mod init seaweedfs-posix-tests
go mod tidy
fi
mkdir -p reports
# Run performance benchmarks
make -f posix_Makefile benchmark-posix
# This job has no FUSE mount, so the fio suite can only skip; keep the
# run green but honest about producing no mounted-FS results
make -f posix_Makefile test-fio-posix || echo "FIO skipped - no mounted filesystem in this job"
- name: Store performance baseline
uses: actions/upload-artifact@v7
with:
name: performance-baseline-results
path: |
test/fuse_integration/reports/posix_benchmark_results.log
test/fuse_integration/reports/fio_*_results.log
retention-days: 90
# Schedule regular compliance checks
scheduled-compliance-check:
runs-on: ubuntu-latest
if: github.event_name == 'schedule'
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version: ${{ env.GO_VERSION }}
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y fuse libfuse-dev
pip3 install --user nfstest
# The posix suites mount with -allowOthers; allow_other needs this
echo 'user_allow_other' | sudo tee -a /etc/fuse.conf
- name: Build SeaweedFS
run: make
- name: Run comprehensive compliance check
run: |
cd test/fuse_integration
make -f posix_Makefile test-posix-full
make -f posix_Makefile generate-report
- name: Create compliance issue if tests fail
if: failure()
uses: actions/github-script@v7
with:
script: |
const issue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: 'POSIX Compliance Check Failed - ' + new Date().toISOString().split('T')[0],
body: `## POSIX Compliance Check Failure
The scheduled POSIX compliance check has failed. This may indicate:
- Regression in FUSE mount functionality
- Changes affecting POSIX compatibility
- Infrastructure issues with the test environment
**Action Required:** Review the test results and investigate any failures.
**Test Run:** [Workflow Run](https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId})
**Date:** ${new Date().toISOString()}
---
This issue was automatically created by the POSIX compliance workflow.
`,
labels: ['bug', 'posix-compliance', 'automated-issue']
});
console.log('Created issue:', issue.data.number);
# Schedule this workflow to run weekly
# Uncomment the following lines to enable scheduled runs:
# schedule:
# - cron: '0 2 * * 1' # Every Monday at 2 AM UTC
+4 -2
View File
@@ -23,8 +23,8 @@ on:
- '.github/workflows/postgres-tests.yml'
concurrency:
group: ${{ github.head_ref }}/postgres-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -41,6 +41,7 @@ jobs:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
@@ -56,6 +57,7 @@ jobs:
uses: docker/setup-buildx-action@v4
- name: Cache Docker layers
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: /tmp/.buildx-cache
+85 -57
View File
@@ -21,8 +21,8 @@ on:
- '.github/workflows/rust-volume-server-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -52,9 +52,68 @@ jobs:
- 'seaweed-volume/**'
- '.github/workflows/rust-volume-server-tests.yml'
build-test-binaries:
name: Build Test Binaries
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
# cargo tracks its own inputs but not the runner's C toolchain, so a cached
# target/ can carry C objects built against a different glibc than we link against.
- name: Fingerprint build toolchain
id: toolchain
run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
seaweed-volume/target
key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
restore-keys: |
rust-${{ steps.toolchain.outputs.fingerprint }}-
- name: Build Go weed binary
run: |
cd weed
go build -tags 5BytesOffset -o weed .
chmod +x weed
./weed version
- name: Build Rust volume binary
run: cd seaweed-volume && cargo build --release
- name: Package binaries
run: tar cf rust-test-binaries.tar weed/weed seaweed-volume/target/release/weed-volume
- name: Upload test binaries
uses: actions/upload-artifact@v7
with:
# archive layout mirrors the workspace: weed/weed and
# seaweed-volume/target/release/weed-volume
name: rust-test-binaries
if-no-files-found: error
retention-days: 1
path: rust-test-binaries.tar
rust-unit-tests:
name: Rust Unit Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
@@ -71,6 +130,7 @@ jobs:
run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -134,6 +194,7 @@ jobs:
run: echo "fingerprint=windows-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -152,8 +213,9 @@ jobs:
rust-integration-tests:
name: Rust Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
needs: [build-test-binaries]
steps:
- name: Checkout code
@@ -163,36 +225,18 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
# cargo tracks its own inputs but not the runner's C toolchain, so a cached
# target/ can carry C objects built against a different glibc than we link against.
- name: Fingerprint build toolchain
id: toolchain
run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
uses: actions/cache@v6
- name: Download test binaries
uses: actions/download-artifact@v7
with:
path: |
~/.cargo/registry
~/.cargo/git
seaweed-volume/target
key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
restore-keys: |
rust-${{ steps.toolchain.outputs.fingerprint }}-
name: rust-test-binaries
- name: Build Go weed binary
- name: Restore binaries
run: |
cd weed
go build -tags 5BytesOffset -o weed .
chmod +x weed
./weed version
- name: Build Rust volume binary
run: cd seaweed-volume && cargo build --release
tar xf rust-test-binaries.tar
chmod +x weed/weed seaweed-volume/target/release/weed-volume
./weed/weed version
- name: Run integration tests
env:
@@ -226,8 +270,9 @@ jobs:
rust-volume-go-tests:
name: Go Tests with Rust Volume (${{ matrix.test-type }} - Shard ${{ matrix.shard }})
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 45
needs: [build-test-binaries]
env:
# Keep in step with the length of matrix.shard below.
SHARD_COUNT: 3
@@ -245,36 +290,19 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
# cargo tracks its own inputs but not the runner's C toolchain, so a cached
# target/ can carry C objects built against a different glibc than we link against.
- name: Fingerprint build toolchain
id: toolchain
run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
uses: actions/cache@v6
- name: Download test binaries
uses: actions/download-artifact@v7
with:
path: |
~/.cargo/registry
~/.cargo/git
seaweed-volume/target
key: rust-${{ steps.toolchain.outputs.fingerprint }}-${{ hashFiles('seaweed-volume/Cargo.lock', 'seaweed-common/Cargo.lock') }}
restore-keys: |
rust-${{ steps.toolchain.outputs.fingerprint }}-
name: rust-test-binaries
- name: Build Go weed binary
- name: Restore binaries
run: |
cd weed
go build -tags 5BytesOffset -o weed .
chmod +x weed
./weed version
- name: Build Rust volume binary
run: cd seaweed-volume && cargo build --release
tar xf rust-test-binaries.tar
chmod +x weed/weed seaweed-volume/target/release/weed-volume
./weed/weed version
./seaweed-volume/target/release/weed-volume --help >/dev/null || true
# Dealing the listed tests out one by one keeps the shards even. Bucketing
# them by first letter did not: names cluster, so ^Test[I-S] drew 50 of
+4 -3
View File
@@ -17,8 +17,8 @@ on:
- '.github/workflows/rust-worker-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -26,7 +26,7 @@ permissions:
jobs:
rust-worker-build:
name: Rust Plugin Worker Build and Unit Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
@@ -45,6 +45,7 @@ jobs:
run: echo "fingerprint=$(getconf GNU_LIBC_VERSION | tr ' ' '-')-rustc-$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
+3 -1
View File
@@ -32,7 +32,7 @@ jobs:
permissions:
contents: write
needs: cleanup
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -46,6 +46,7 @@ jobs:
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -114,6 +115,7 @@ jobs:
targets: ${{ matrix.target }}
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
+20 -8
View File
@@ -15,7 +15,7 @@ jobs:
build-rust-volume-linux:
permissions:
contents: write
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -37,9 +37,13 @@ jobs:
if: matrix.cross
run: |
sudo dpkg --add-architecture arm64
sudo sed -i 's/^deb /deb [arch=amd64] /' /etc/apt/sources.list
echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy main restricted universe multiverse" | sudo tee /etc/apt/sources.list.d/arm64.list
echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy-updates main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm64.list
# noble keeps apt sources in deb822 ubuntu.sources, not sources.list —
# pin them to amd64 so apt does not probe arm64 indexes on
# archive/security.ubuntu.com (they 404).
sudo sed -i '/^Types: deb$/a Architectures: amd64' /etc/apt/sources.list.d/ubuntu.sources
for suite in noble noble-updates noble-security; do
echo "deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports $suite main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm64.list
done
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu libssl-dev:arm64
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
@@ -48,6 +52,7 @@ jobs:
echo "OPENSSL_LIB_DIR=/usr/lib/aarch64-linux-gnu" >> "$GITHUB_ENV"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -119,7 +124,7 @@ jobs:
build-rust-worker-linux:
permissions:
contents: write
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
matrix:
include:
@@ -145,14 +150,19 @@ jobs:
if: matrix.cross
run: |
sudo dpkg --add-architecture arm64
sudo sed -i 's/^deb /deb [arch=amd64] /' /etc/apt/sources.list
echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy main restricted universe multiverse" | sudo tee /etc/apt/sources.list.d/arm64.list
echo "deb [arch=arm64] http://ports.ubuntu.com/ jammy-updates main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm64.list
# noble keeps apt sources in deb822 ubuntu.sources, not sources.list —
# pin them to amd64 so apt does not probe arm64 indexes on
# archive/security.ubuntu.com (they 404).
sudo sed -i '/^Types: deb$/a Architectures: amd64' /etc/apt/sources.list.d/ubuntu.sources
for suite in noble noble-updates noble-security; do
echo "deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports $suite main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm64.list
done
sudo apt-get update
sudo apt-get install -y gcc-aarch64-linux-gnu
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -230,6 +240,7 @@ jobs:
targets: ${{ matrix.target }}
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
@@ -305,6 +316,7 @@ jobs:
uses: dtolnay/rust-toolchain@stable
- name: Cache cargo registry and target
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: |
+4 -3
View File
@@ -21,8 +21,8 @@ on:
- '.github/workflows/s3-etag-acl-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/s3-etag-acl-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,7 +30,7 @@ permissions:
jobs:
s3-etag-acl-tests:
name: S3 ETag + ACL Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out code
@@ -40,6 +40,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install SeaweedFS
run: |
@@ -13,8 +13,8 @@ on:
- '.github/workflows/s3-example-integration-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-integration-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -22,7 +22,7 @@ permissions:
jobs:
tests:
name: S3 Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
@@ -33,6 +33,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Build SeaweedFS
+8 -4
View File
@@ -12,8 +12,8 @@ on:
- '.github/workflows/s3-filer-group-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-filer-group-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -25,17 +25,21 @@ defaults:
jobs:
s3-filer-group-tests:
name: S3 Filer Group Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -50,7 +54,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Filer Group Tests ==="
# Run tests with automatic server management
+42 -17
View File
@@ -4,6 +4,7 @@ on:
pull_request:
paths:
- 'weed/s3api/**'
- 'weed/remote_storage/**'
- 'weed/filer/**'
- 'weed/server/**'
- 'test/s3/**'
@@ -12,8 +13,8 @@ on:
- '.github/workflows/s3-go-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-go-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -25,7 +26,7 @@ defaults:
jobs:
s3-versioning-tests:
name: S3 Versioning Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
matrix:
@@ -35,10 +36,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -53,7 +58,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
@@ -92,7 +97,7 @@ jobs:
s3-versioning-compatibility:
name: S3 Versioning Compatibility Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -103,6 +108,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -140,7 +146,7 @@ jobs:
s3-cors-compatibility:
name: S3 CORS Compatibility Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -151,6 +157,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -188,7 +195,7 @@ jobs:
s3-retention-tests:
name: S3 Retention Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
matrix:
@@ -198,10 +205,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -216,7 +227,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
@@ -255,7 +266,7 @@ jobs:
s3-lifecycle-tests:
name: S3 Lifecycle Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
fail-fast: false
@@ -283,6 +294,7 @@ jobs:
- TestLifecycleSuspendedVersioningExpiration
- TestLifecycleTagFilter
- TestLifecycleVersionedBucketCreatesDeleteMarker
- TestLifecycleVersionedBucketExpiresOnSecondRun
steps:
- name: Check out code
@@ -292,6 +304,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -326,17 +339,21 @@ jobs:
s3-checksum-tests:
name: S3 Checksum Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -351,7 +368,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Tests ==="
make test-with-server
@@ -381,7 +398,7 @@ jobs:
s3-cors-tests:
name: S3 CORS Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
matrix:
@@ -391,10 +408,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -409,7 +430,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Tests ==="
# Run tests with automatic server management
@@ -448,7 +469,7 @@ jobs:
s3-retention-worm:
name: S3 Retention WORM Integration Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -459,6 +480,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -497,7 +519,7 @@ jobs:
s3-versioning-stress:
name: S3 Versioning Stress Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 35
steps:
@@ -508,6 +530,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -565,7 +588,7 @@ jobs:
s3-tagging-tests:
# CI job for S3 object tagging tests
name: S3 Tagging Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -576,6 +599,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -620,7 +644,7 @@ jobs:
s3-remote-cache-tests:
name: S3 Remote Cache Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -631,6 +655,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
+14 -7
View File
@@ -20,8 +20,8 @@ on:
- '.github/workflows/s3-iam-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-iam-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -34,7 +34,7 @@ jobs:
# Unit tests for IAM components
iam-unit-tests:
name: IAM Unit Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
@@ -45,6 +45,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Get dependencies
@@ -80,7 +81,7 @@ jobs:
# S3 IAM integration tests with SeaweedFS services
s3-iam-integration-tests:
name: S3 IAM Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
strategy:
matrix:
@@ -90,10 +91,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -109,7 +114,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 IAM Integration Tests (${{ matrix.test-type }}) ==="
# Set WEED_BINARY to use the installed version
@@ -197,7 +202,7 @@ jobs:
# Distributed IAM tests
s3-iam-distributed-tests:
name: S3 IAM Distributed Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
@@ -208,6 +213,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -257,7 +263,7 @@ jobs:
# Performance and stress tests
s3-iam-performance-tests:
name: S3 IAM Performance Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
@@ -268,6 +274,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
+8 -4
View File
@@ -16,8 +16,8 @@ on:
- '.github/workflows/s3-keycloak-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-keycloak-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,17 +30,21 @@ jobs:
# Dedicated job for Keycloak integration tests
s3-keycloak-integration-tests:
name: S3 Keycloak Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -56,7 +60,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Keycloak Integration Tests ==="
# Set WEED_BINARY to use the installed version
@@ -12,8 +12,8 @@ on:
- '.github/workflows/s3-mutation-regression-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-mutation-regression-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -21,7 +21,7 @@ permissions:
jobs:
s3-versioning-regressions:
name: S3 Versioning Regression Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
@@ -32,6 +32,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Run S3 versioning regression tests
timeout-minutes: 20
@@ -74,7 +75,7 @@ jobs:
s3-delete-regressions:
name: S3 Delete Regression Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -85,6 +86,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Run S3 delete regression tests
timeout-minutes: 15
@@ -112,7 +114,7 @@ jobs:
s3-distributed-lock-regressions:
name: S3 Distributed Lock Regression Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
@@ -123,6 +125,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS
run: |
+7 -3
View File
@@ -23,6 +23,10 @@ env:
S3_ENDPOINT_URL: http://localhost:8333
BUCKET_NAME: test-parquet-bucket
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
parquet-integration-tests:
name: PyArrow Parquet Tests (Python ${{ matrix.python-version }})
@@ -42,13 +46,13 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
cache: ${{ runner.environment == 'github-hosted' && 'pip' || '' }}
cache-dependency-path: 'test/s3/parquet/requirements.txt'
- name: Install system dependencies
@@ -150,7 +154,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: true
cache: ${{ runner.environment == 'github-hosted' }}
- name: Run Go unit tests
run: |
+20 -9
View File
@@ -24,8 +24,8 @@ on:
- '.github/workflows/s3-policy-tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3-policy-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -38,7 +38,7 @@ jobs:
# Unit tests for policy components
policy-unit-tests:
name: S3 Policy Unit Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
@@ -49,6 +49,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Get dependencies
@@ -81,17 +82,21 @@ jobs:
# S3 Policy Variables Integration Tests
s3-policy-variables-tests:
name: S3 Policy Variables Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -106,7 +111,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Policy Variables Integration Tests ==="
@@ -187,7 +192,7 @@ jobs:
# S3 Policy Enforcement Integration Tests
s3-policy-enforcement-tests:
name: S3 Policy Enforcement Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
matrix:
@@ -197,10 +202,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -215,7 +224,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Policy Enforcement Tests (${{ matrix.test-case }}) ==="
@@ -308,7 +317,7 @@ jobs:
# Trusted Proxy Detection Tests
trusted-proxy-tests:
name: Trusted Proxy Detection Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -319,6 +328,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -395,7 +405,7 @@ jobs:
# S3 Policy Shell Integration Tests
s3-policy-shell-tests:
name: S3 Policy Shell Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
@@ -406,6 +416,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -21,8 +21,8 @@ on:
- '.github/workflows/s3-proxy-signature-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/s3-proxy-signature-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,7 +30,7 @@ permissions:
jobs:
proxy-signature-tests:
name: S3 Proxy Signature Verification Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out code into the Go module directory
@@ -40,6 +40,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -15,8 +15,8 @@ on:
- '.github/workflows/s3-sdk-v2-routing-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/s3-sdk-v2-routing-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -24,7 +24,7 @@ permissions:
jobs:
s3-sdk-v2-routing-tests:
name: S3 SDK V2 Routing Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Check out code
@@ -34,6 +34,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install SeaweedFS
run: |
+3 -2
View File
@@ -46,7 +46,7 @@ permissions:
jobs:
snowflake-s3compat-tests:
name: Snowflake S3Compat API tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
env:
WORK_DIR: /tmp/seaweedfs-snowflake-tests
@@ -60,6 +60,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Java
@@ -67,7 +68,7 @@ jobs:
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'
cache: ${{ runner.environment == 'github-hosted' && 'maven' || '' }}
- name: Install SeaweedFS
run: |
+8 -4
View File
@@ -11,8 +11,8 @@ on:
workflow_dispatch:
concurrency:
group: ${{ github.head_ref }}/s3-spark-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -20,17 +20,21 @@ permissions:
jobs:
s3-spark-issue-repro-tests:
name: S3 Spark Issue Reproduction Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -56,7 +60,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting S3 Spark Integration Tests ==="
go test -v -timeout 30m . 2>&1 | tee test-output.log || {
+20 -10
View File
@@ -23,8 +23,8 @@ on:
- 'test/s3/sse/**'
concurrency:
group: ${{ github.head_ref }}/s3-sse-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -36,7 +36,7 @@ defaults:
jobs:
s3-sse-integration-tests:
name: S3 SSE Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
matrix:
@@ -46,10 +46,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -64,7 +68,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting SSE Tests ==="
# Run tests with automatic server management
@@ -107,7 +111,7 @@ jobs:
s3-sse-compatibility:
name: S3 SSE Compatibility Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -118,6 +122,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -155,7 +160,7 @@ jobs:
s3-sse-metadata-persistence:
name: S3 SSE Metadata Persistence Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -166,6 +171,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -204,7 +210,7 @@ jobs:
s3-sse-copy-operations:
name: S3 SSE Copy Operations Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
@@ -215,6 +221,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -253,7 +260,7 @@ jobs:
s3-sse-multipart:
name: S3 SSE Multipart Upload Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 25
steps:
@@ -264,6 +271,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -302,7 +310,7 @@ jobs:
s3-sse-performance:
name: S3 SSE Performance Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 35
# Only run performance tests on master branch pushes to avoid overloading PR testing
if: github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main')
@@ -315,6 +323,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -352,7 +361,7 @@ jobs:
s3-volume-encryption:
name: S3 Volume Encryption Test
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -363,6 +372,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
+110 -36
View File
@@ -12,6 +12,10 @@ on:
- 'go.sum'
- '.github/workflows/s3-tables-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -19,17 +23,21 @@ permissions:
jobs:
s3-tables-tests:
name: S3 Tables Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Run go mod tidy
@@ -48,7 +56,8 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
# A stale FUSE mount can hang or fail df; keep diagnostics best effort.
timeout 15 df -h || true
echo "=== Starting S3 Tables Tests ==="
# Run S3 Tables integration tests
@@ -79,17 +88,21 @@ jobs:
iceberg-catalog-tests:
name: Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -109,7 +122,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Iceberg Catalog Tests ==="
# Run Iceberg catalog integration tests using Makefile (handles build)
@@ -140,17 +153,21 @@ jobs:
trino-iceberg-catalog-tests:
name: Trino Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -179,7 +196,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Trino Iceberg Catalog Tests ==="
# Run Trino + Iceberg catalog integration tests
@@ -210,17 +227,21 @@ jobs:
dremio-iceberg-catalog-tests:
name: Dremio Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -250,7 +271,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Dremio Iceberg Catalog Tests ==="
@@ -283,17 +304,21 @@ jobs:
doris-iceberg-catalog-tests:
name: Doris Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 35
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -323,7 +348,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Doris Iceberg Catalog Tests ==="
@@ -356,7 +381,7 @@ jobs:
clickhouse-iceberg-catalog-tests:
name: ClickHouse Iceberg Catalog Integration Tests (${{ matrix.tag }})
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
fail-fast: false
@@ -373,10 +398,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -408,7 +437,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting ClickHouse Iceberg Catalog Tests ==="
@@ -441,7 +470,7 @@ jobs:
olake-iceberg-catalog-tests:
name: OLake Iceberg Catalog Integration Tests (${{ matrix.tag }})
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
fail-fast: false
@@ -460,10 +489,14 @@ jobs:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -496,7 +529,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting OLake Iceberg Catalog Tests ==="
@@ -552,17 +585,21 @@ jobs:
polaris-integration-tests:
name: Polaris Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Run go mod tidy
@@ -590,7 +627,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Polaris Tests ==="
go test -v -timeout 20m ./test/s3tables/polaris 2>&1 | tee test/s3tables/polaris/test-output.log || {
@@ -620,17 +657,21 @@ jobs:
spark-iceberg-catalog-tests:
name: Spark Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -659,7 +700,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Spark Iceberg Catalog Tests ==="
# Run Spark + Iceberg catalog integration tests
@@ -690,17 +731,21 @@ jobs:
risingwave-catalog-tests:
name: RisingWave Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -730,7 +775,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting RisingWave Catalog Tests ==="
# Run RisingWave catalog integration tests
@@ -761,17 +806,21 @@ jobs:
sts-integration-tests:
name: STS Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -785,6 +834,7 @@ jobs:
- name: Restore python:3 image cache
id: python-image
if: runner.environment == 'github-hosted'
uses: actions/cache@v6
with:
path: /tmp/python3-image.tar
@@ -824,7 +874,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting STS Integration Tests ==="
# Run STS integration tests
@@ -855,17 +905,21 @@ jobs:
lakekeeper-integration-tests:
name: Lakekeeper Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Run go mod tidy
@@ -884,7 +938,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting Lakekeeper Integration Tests ==="
# Run Lakekeeper integration tests
@@ -915,17 +969,21 @@ jobs:
unity-catalog-integration-tests:
name: Unity Catalog Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -955,7 +1013,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Unity Catalog Tests ==="
@@ -988,17 +1046,21 @@ jobs:
lancedb-namespace-tests:
name: LanceDB Namespace Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -1027,7 +1089,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting LanceDB Namespace Tests ==="
@@ -1058,7 +1120,7 @@ jobs:
table-lifecycle-tests:
name: Table Lifecycle Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 40
steps:
@@ -1069,6 +1131,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -1126,7 +1189,7 @@ jobs:
duckdb-lance-tests:
name: DuckDB Lance Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
@@ -1137,10 +1200,14 @@ jobs:
# so do not leave a token in the checkout for it to pick up.
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -1170,7 +1237,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting DuckDB Lance Tests ==="
@@ -1201,7 +1268,7 @@ jobs:
spark-lance-namespace-tests:
name: Spark Lance Namespace Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 40
steps:
@@ -1212,10 +1279,14 @@ jobs:
# so do not leave a token in the checkout for it to pick up.
persist-credentials: false
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Configure Docker Hub mirror
@@ -1244,7 +1315,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
docker info
echo "=== Starting Spark Lance Namespace Tests ==="
@@ -1275,7 +1346,7 @@ jobs:
s3-tables-build-verification:
name: S3 Tables Build Verification
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
@@ -1286,6 +1357,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Verify S3 Tables Package Builds
@@ -1340,7 +1412,7 @@ jobs:
s3-tables-fmt-check:
name: S3 Tables Format Check
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
@@ -1351,6 +1423,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Check Go Format
@@ -1391,7 +1464,7 @@ jobs:
s3-tables-vet:
name: S3 Tables Go Vet Check
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
@@ -1402,6 +1475,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Run Go Vet
+24 -19
View File
@@ -25,8 +25,8 @@ on:
- '.github/workflows/s3tests.yml'
concurrency:
group: ${{ github.head_ref }}/s3tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -34,7 +34,7 @@ permissions:
jobs:
basic-s3-tests:
name: Basic S3 tests (KV store)
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out code into the Go module directory
@@ -44,6 +44,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Python
@@ -57,9 +58,9 @@ jobs:
cd s3-tests
sudo apt-get update -qq
sudo apt-get install -y -qq libxml2-dev libxslt1-dev zlib1g-dev
pip install -r requirements.txt
pip install tox
pip install -e .
python3 -m pip install -r requirements.txt
python3 -m pip install tox
python3 -m pip install -e .
- name: Fix S3 tests bucket creation conflicts
run: |
@@ -386,7 +387,7 @@ jobs:
versioning-tests:
name: S3 Versioning & Object Lock tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out code into the Go module directory
@@ -396,6 +397,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Python
@@ -409,9 +411,9 @@ jobs:
cd s3-tests
sudo apt-get update -qq
sudo apt-get install -y -qq libxml2-dev libxslt1-dev zlib1g-dev
pip install -r requirements.txt
pip install tox
pip install -e .
python3 -m pip install -r requirements.txt
python3 -m pip install tox
python3 -m pip install -e .
- name: Fix S3 tests bucket creation conflicts
run: |
@@ -556,7 +558,7 @@ jobs:
cors-tests:
name: S3 CORS tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Check out code into the Go module directory
@@ -566,6 +568,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Python
@@ -579,9 +582,9 @@ jobs:
cd s3-tests
sudo apt-get update -qq
sudo apt-get install -y -qq libxml2-dev libxslt1-dev zlib1g-dev
pip install -r requirements.txt
pip install tox
pip install -e .
python3 -m pip install -r requirements.txt
python3 -m pip install tox
python3 -m pip install -e .
- name: Run S3 CORS tests
timeout-minutes: 10
@@ -681,7 +684,7 @@ jobs:
copy-tests:
name: SeaweedFS Custom S3 Copy tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Check out code into the Go module directory
@@ -691,6 +694,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Run SeaweedFS Custom S3 Copy tests
@@ -783,7 +787,7 @@ jobs:
sql-store-tests:
name: Basic S3 tests (SQL store)
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out code into the Go module directory
@@ -793,6 +797,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Set up Python
@@ -806,9 +811,9 @@ jobs:
cd s3-tests
sudo apt-get update -qq
sudo apt-get install -y -qq libxml2-dev libxslt1-dev zlib1g-dev
pip install -r requirements.txt
pip install tox
pip install -e .
python3 -m pip install -r requirements.txt
python3 -m pip install tox
python3 -m pip install -e .
- name: Run Ceph S3 tests with SQL store
timeout-minutes: 15
+10 -2
View File
@@ -29,7 +29,7 @@ permissions:
jobs:
samba-integration:
name: samba-integration
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
@@ -40,15 +40,23 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Start local Docker registry
run: docker run -d --restart=always -p 5000:5000 --name registry registry:2
run: |
if ! docker start registry 2>/dev/null; then
docker rm -f registry >/dev/null 2>&1 || true
docker run -d --restart=always -p 5000:5000 --name registry registry:2
fi
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
driver-opts: network=host
- name: Set up Docker Compose
uses: docker/setup-compose-action@v2
- name: Build weed race binary
run: |
cd docker
+4 -3
View File
@@ -17,8 +17,8 @@ on:
- '.github/workflows/sftp-tests.yml'
concurrency:
group: ${{ github.head_ref }}/sftp-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -29,7 +29,7 @@ env:
jobs:
sftp-integration:
name: SFTP Integration Testing
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
@@ -40,6 +40,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Install dependencies
run: |
@@ -15,6 +15,10 @@ on:
- '.github/workflows/spark-integration-tests.yml'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
checks: write
@@ -38,12 +42,13 @@ jobs:
with:
java-version: '11'
distribution: 'temurin'
cache: maven
cache: ${{ runner.environment == 'github-hosted' && 'maven' || '' }}
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS binary
run: |
+5 -6
View File
@@ -10,12 +10,11 @@ permissions:
contents: write
concurrency:
# Only one chart regeneration per branch at a time; a newer run on the same
# branch cancels an in-flight one so overlapping runs never conflict on
# note/star_history.svg during rebase. Scoped by ref so a manual run on
# another branch can't cancel the daily master update.
# Only one chart regeneration per branch at a time so overlapping runs never
# conflict on note/star_history.svg during rebase. Scoped by ref so a manual
# run on another branch can't cancel the daily master update.
group: star-history-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
render:
@@ -33,7 +32,7 @@ jobs:
uses: actions/setup-python@v7
with:
python-version: "3.x"
cache: pip
cache: ${{ runner.environment == 'github-hosted' && 'pip' || '' }}
cache-dependency-path: .github/scripts/star_history.py
- name: Install matplotlib
@@ -14,6 +14,10 @@ on:
- 'weed/telemetry/**'
- '.github/workflows/telemetry-integration.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -30,6 +34,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build telemetry server
run: cd telemetry/server && go build -o telemetry-server .
+5
View File
@@ -8,6 +8,10 @@ on:
branches: [ master ]
paths: ['terraform/**', '.github/workflows/terraform_ci.yml']
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -62,6 +66,7 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build weed
run: go build -o "$RUNNER_TEMP/weed" ./weed
@@ -27,6 +27,10 @@ defaults:
run:
working-directory: weed
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
jobs:
awscli-tests:
runs-on: ubuntu-latest
@@ -37,6 +41,7 @@ jobs:
- uses: actions/setup-go@v7
with:
go-version: ^1.26
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS
run: |
+6 -1
View File
@@ -18,18 +18,23 @@ on:
- 'go.sum'
- '.github/workflows/tls-rotation-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
tls-rotation-tests:
name: TLS Rotation Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
+8 -4
View File
@@ -11,8 +11,8 @@ on:
- '.github/workflows/tus-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/tus-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -24,17 +24,21 @@ defaults:
jobs:
tus-integration-tests:
name: TUS Protocol Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Clean up stale FUSE mounts
uses: ./.github/actions/cleanup-stale-fuse-mounts
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
id: go
- name: Install SeaweedFS
@@ -49,7 +53,7 @@ jobs:
echo "=== System Information ==="
uname -a
free -h
df -h
timeout 15 df -h || true
echo "=== Starting TUS Tests ==="
# Run tests with automatic server management
@@ -18,18 +18,23 @@ on:
- 'go.sum'
- '.github/workflows/vacuum-integration-tests.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
jobs:
vacuum-integration-tests:
name: Vacuum Integration Tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v7
with:
cache: ${{ runner.environment == 'github-hosted' }}
go-version: ^1.26
id: go
@@ -21,8 +21,8 @@ on:
- '.github/workflows/volume-server-integration-tests.yml'
concurrency:
group: ${{ github.head_ref || github.ref }}/volume-server-integration-tests
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.event.pull_request.number || (github.event_name == 'push' && github.ref) || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' || (github.event_name == 'push' && !startsWith(github.ref, 'refs/tags/')) }}
permissions:
contents: read
@@ -33,10 +33,44 @@ env:
SHARD_COUNT: 3
jobs:
build-weed:
name: Build SeaweedFS Binary
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS binary
run: |
cd weed
go build -o weed .
chmod +x weed
./weed version
- name: Package binary
run: tar cf weed-binary.tar weed/weed
- name: Upload binary
uses: actions/upload-artifact@v7
with:
name: weed-binary
if-no-files-found: error
retention-days: 1
path: weed-binary.tar
volume-server-integration-tests:
name: Volume Server Integration Tests (${{ matrix.test-type }} - Shard ${{ matrix.shard }})
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 45
needs: [build-weed]
strategy:
fail-fast: false
matrix:
@@ -51,13 +85,18 @@ jobs:
uses: actions/setup-go@v7
with:
go-version-file: 'go.mod'
cache: ${{ runner.environment == 'github-hosted' }}
- name: Build SeaweedFS binary
- name: Download binary
uses: actions/download-artifact@v7
with:
name: weed-binary
- name: Restore binary
run: |
cd weed
go build -o weed .
chmod +x weed
./weed version
tar xf weed-binary.tar
chmod +x weed/weed
./weed/weed version
# Dealing the listed tests out one by one keeps the shards even. Bucketing
# them by first letter did not: names cluster, so ^Test[I-S] drew 50 of
+1
View File
@@ -144,3 +144,4 @@ weed_bin
telemetry/server/telemetry-server
.aider*
/seaweed-volume/docs
__pycache__/
+1 -1
View File
@@ -3,7 +3,7 @@ FROM golang:1.26 AS builder
RUN apt-get update
RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev
ARG ROCKSDB_VERSION=v10.10.1
ARG ROCKSDB_VERSION=v11.1.2
ENV ROCKSDB_VERSION=${ROCKSDB_VERSION}
# build RocksDB
+1 -1
View File
@@ -3,7 +3,7 @@ FROM golang:1.26 AS builder
RUN apt-get update
RUN apt-get install -y build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev libzstd-dev
ARG ROCKSDB_VERSION=v10.10.1
ARG ROCKSDB_VERSION=v11.1.2
ENV ROCKSDB_VERSION=${ROCKSDB_VERSION}
# build RocksDB
+1 -1
View File
@@ -362,7 +362,7 @@ require (
github.com/flynn/noise v1.1.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
github.com/geoffgarside/ber v1.2.0 // indirect
github.com/go-chi/chi/v5 v5.3.1 // indirect
github.com/go-chi/chi/v5 v5.4.0 // indirect
github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 // indirect
github.com/go-jose/go-jose/v4 v4.1.5 // indirect
github.com/go-logr/logr v1.4.4 // indirect
+2 -2
View File
@@ -483,8 +483,8 @@ github.com/gin-gonic/gin v1.11.0 h1:OW/6PLjyusp2PPXtyxKHU0RbX6I/l28FTdDlae5ueWk=
github.com/gin-gonic/gin v1.11.0/go.mod h1:+iq/FyxlGzII0KHiBGjuNn4UNENUlKbGlNmc+W50Dls=
github.com/go-asn1-ber/asn1-ber v1.5.8 h1:H9AZkK22UOmfX8J84ubyaZxKJZ3FMHVwn8swoMML7iQ=
github.com/go-asn1-ber/asn1-ber v1.5.8/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
github.com/go-chi/chi/v5 v5.3.1 h1:3j4HZLGZQ3JpMCrPJF/Jl3mYJfWLKBfNJ6quurUGCf8=
github.com/go-chi/chi/v5 v5.3.1/go.mod h1:R+tYY2hNuVUUjxoPtqUdgBqevM9s9njzkTLutVsOCto=
github.com/go-chi/chi/v5 v5.4.0 h1:+ufzvFTGKyin26Or5DLrlIDq4ygIDBVcj7oVNeeaYFE=
github.com/go-chi/chi/v5 v5.4.0/go.mod h1:d1IG7GtKPSjPyBP365PH8sAGVHk3qfB/IhjFNRH1wWM=
github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 h1:JnrjqG5iR07/8k7NqrLNilRsl3s1EPRQEGvbPyOce68=
github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348/go.mod h1:Czxo/d1g948LtrALAZdL04TL/HnkopquAjxYUuI02bo=
github.com/go-errors/errors v1.5.1 h1:ZwEMSLRCapFLflTpT7NKaAc7ukJ8ZPEjzlxt8rPN8bk=
+2 -2
View File
@@ -1,6 +1,6 @@
apiVersion: v1
description: SeaweedFS
name: seaweedfs
appVersion: "4.48"
appVersion: "4.51"
# Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
version: 4.48.2
version: 4.51.0
@@ -170,9 +170,6 @@ spec:
{{- if .Values.filer.metricsIp }}
-metricsIp={{ .Values.filer.metricsIp }} \
{{- end }}
{{- if .Values.filer.redirectOnRead }}
-redirectOnRead \
{{- end }}
{{- if .Values.filer.disableHttp }}
-disableHttp \
{{- end }}
@@ -172,7 +172,8 @@ spec:
{{- $bucketName := .name }}
{{- $bucketLock := or .lock .objectLock .withLock }}
bucket_list=$(/bin/echo 's3.bucket.list' | /usr/bin/weed shell) || { echo "Error listing s3 buckets"; exit 1; }
if echo "$bucket_list" | awk '{print $1}' | grep -Fxq "{{ $bucketName }}"; then
{{- /* One awk: grep -q exits early, awk takes SIGPIPE, pipefail fails the test. */}}
if printf '%s\n' "$bucket_list" | awk -v bucket="{{ $bucketName }}" '$1 "" == bucket { found = 1 } END { exit !found }'; then
echo "Bucket '{{ $bucketName }}' already exists, skipping creation."
else
echo "Creating bucket '{{ $bucketName }}'..."
-3
View File
@@ -740,9 +740,6 @@ filer:
# encrypt data on volume servers
encryptVolumeData: false
# Whether proxy or redirect to volume server during file GET request
redirectOnRead: false
# Limit sub dir listing size (default 100000)
dirListLimit: 100000
+1034 -1099
View File
File diff suppressed because it is too large. Load diff

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 53 KiB

+21 -1
View File
@@ -166,6 +166,10 @@ message Entry {
repeated FileChunk chunks = 3;
FuseAttributes attributes = 4;
map<string, bytes> extended = 5;
// filer-local commit timestamp of the version stored here: the meta log
// event's ts_ns for the mutation that produced this entry, or 0 for
// entries written before the field existed
int64 commit_ts_ns = 6;
bytes hard_link_id = 7;
int32 hard_link_counter = 8; // only exists in hard link meta data
bytes content = 9; // if not empty, the file content
@@ -281,6 +285,7 @@ message WriteCondition {
IF_EXTENDED_NOT_EQUAL = 7; // fail if present and extended[ext_key] == ext_value
IF_EXTENDED_TIME_ELAPSED = 8; // fail if present and extended[ext_key] (unix seconds) is in the future
IF_CHUNKS_EQUAL = 9; // fail unless the stored chunk fid multiset equals fids (absent entry = no chunks)
IF_ENTRY_EQUAL = 10; // fail unless the stored entry equals expected_entry
}
// Clause is one primitive comparison. IF_ETAG_MATCH holds when the current
// entry's ETag equals any value in etags; IF_ETAG_NOT_MATCH holds when it
@@ -309,6 +314,7 @@ message WriteCondition {
string gate_key = 7; // IF_EXTENDED_TIME_ELAPSED: only enforce when extended[gate_key] == gate_value
string gate_value = 8; // gate value (e.g. retention mode COMPLIANCE for governance bypass)
repeated string fids = 9; // chunk fid strings for IF_CHUNKS_EQUAL
Entry expected_entry = 10; // whole-entry comparison for IF_ENTRY_EQUAL
}
repeated Clause clauses = 1; // all must hold (logical AND)
}
@@ -323,6 +329,7 @@ enum FilerError {
EXISTING_IS_FILE = 4; // cannot overwrite file with directory
ENTRY_ALREADY_EXISTS = 5; // O_EXCL and entry already exists
PRECONDITION_FAILED = 6; // WriteCondition not satisfied
READ_ONLY = 7; // path matches a read-only storage rule (e.g. bucket over quota)
}
// ObjectMutation is one entry-level change applied by ObjectTransaction. All
@@ -351,6 +358,7 @@ message ObjectMutation {
bytes content = 11; // PATCH_EXTENDED: new Entry.content when set_content
bool touch_mtime = 12; // PATCH_EXTENDED: set the entry's Mtime to now (e.g. a metadata-replace copy)
bool remove_empty_parent = 13; // DELETE: also remove the parent directory when the delete leaves it empty (best-effort)
bool keep_remote_object = 14; // DELETE: leave the object on the mounted remote storage
}
// Recompute re-derives a pointer entry (directory/name on the mutation) from the
@@ -388,6 +396,8 @@ message ObjectTransactionRequest {
string condition_key = 6; // if set, evaluate the condition against this entry instead of lock_key (still locking lock_key)
string route_key = 7; // ring key identifying the owner filer; a non-owner forwards the whole transaction to it
bool is_moved = 8; // set on a forwarded transaction so the receiver applies it locally instead of forwarding again
string request_id = 9; // idempotency token: the owner records the outcome and replays it on a retried request instead of re-applying
string condition_alt_key = 10; // if the condition's entry is absent, evaluate it against this entry instead (e.g. null slot, else latest version)
}
message ObjectTransactionResponse {
@@ -477,6 +487,7 @@ message UpdateEntryRequest {
// the write, under the filer's per-path lock. The caller must route the
// key's writes to this entry's owner filer for the check to be authoritative.
WriteCondition condition = 6;
bool is_moved = 7; // set on a forwarded request so the owner applies it locally
}
message UpdateEntryResponse {
SubscribeMetadataResponse metadata_event = 1;
@@ -514,6 +525,7 @@ message DeleteEntryRequest {
bool is_from_other_cluster = 7;
repeated int32 signatures = 8;
int64 if_not_modified_after = 9;
bool keep_remote_object = 10;
}
message DeleteEntryResponse {
@@ -573,6 +585,9 @@ message AssignVolumeResponse {
// fsync is the storage rule's fsync decision for the assigned path, so the
// client can carry it onto the volume server upload request.
bool fsync = 11;
// error_code classifies error for callers that must branch on it; error
// keeps the human-readable text for clients that predate the field.
FilerError error_code = 12;
}
message LookupVolumeRequest {
@@ -709,6 +724,10 @@ message TraverseBfsMetadataRequest {
message TraverseBfsMetadataResponse {
string directory = 1;
Entry entry = 2;
// The sender's per-path commit timestamp (see meta_path_ts.go): the newest
// event time this filer committed for the path, 0 when unrecorded so
// subscribers fall back to the entry's mtime.
int64 entry_ts_ns = 3;
}
message LogEntry {
@@ -771,7 +790,8 @@ message FilerConf {
string replication = 3;
string ttl = 4;
string disk_type = 5;
bool fsync = 6;
// unset inherits from the enclosing path rule, set overrides it
optional bool fsync = 6;
uint32 volume_growth_count = 7;
bool read_only = 8;
string data_center = 9;
+41 -15
View File
@@ -86,7 +86,7 @@ version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -97,7 +97,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -510,10 +510,12 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.62.5"
version = "0.62.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9648b0bb82a2eedd844052c6ad2a1a822d1f8e3adee5fbf668366717e428856a"
checksum = "701a947f4797e52a911e114a898667c746c39feea467bbd1abd7b3721f702ffa"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-schema",
"aws-smithy-types",
]
@@ -563,11 +565,12 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.11.6"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "876ab3c9c29791ba4ba02b780a3049e21ec63dabda09268b175272c3733a79e6"
checksum = "9db177daa6ba8afb9ee1aefcf548c907abcf52065e394ee11a92780057fe0e8c"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api-macros",
"aws-smithy-types",
"bytes",
"http 0.2.12",
@@ -579,10 +582,32 @@ dependencies = [
]
[[package]]
name = "aws-smithy-types"
version = "1.4.7"
name = "aws-smithy-runtime-api-macros"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d73dbfbaa8e4bc57b9045137680b958d274823509a360abfd8e1d514d40c95c"
checksum = "8d7396fd9500589e62e460e987ecb671bad374934e55ec3b5f498cc7a8a8a7b7"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "aws-smithy-schema"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7442cb268338f0eb8278140a107c046756aa01093d8ef5e99628d34ae09c94f5"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-types",
"http 1.4.0",
]
[[package]]
name = "aws-smithy-types"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32b42fcf341259d85ca10fac9a2f6448a8ec691c6955a18e45bc3b71a85fab85"
dependencies = [
"base64-simd",
"bytes",
@@ -1325,7 +1350,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -2379,7 +2404,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.59.0",
]
[[package]]
@@ -3391,7 +3416,7 @@ dependencies = [
"errno 0.3.14",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -3676,7 +3701,7 @@ version = "1.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
dependencies = [
"errno 0.3.14",
"errno 0.2.8",
"libc",
]
@@ -3743,7 +3768,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a766e1110788c36f4fa1c2b71b387a7815aa65f88ce0229841826633d93723e"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -3877,7 +3902,7 @@ dependencies = [
"getrandom 0.4.2",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.2",
"windows-sys 0.52.0",
]
[[package]]
@@ -4576,6 +4601,7 @@ dependencies = [
"futures",
"hex",
"http-body 1.0.1",
"httpdate",
"hyper",
"hyper-util",
"image",
+3
View File
@@ -51,6 +51,7 @@ prost-types = "0.14"
# HTTP server
axum = { version = "0.8", features = ["multipart"] }
http-body = "1"
httpdate = "1"
hyper = { version = "1", features = ["full"] }
hyper-util = { version = "0.1", features = ["tokio", "service", "server-auto", "http1", "http2"] }
tower = { version = "0.5", features = ["util"] }
@@ -157,6 +158,8 @@ windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem"] }
[dev-dependencies]
tempfile = "3"
# Paused-time tests drive the write-queue batch window deterministically.
tokio = { version = "1", features = ["full", "test-util"] }
# Already a transitive dependency of aws-sdk-s3 at a single locked version;
# needed directly only for the canned HttpClient in remote_storage::s3 tests.
aws-smithy-runtime-api = "1"
+30
View File
@@ -233,6 +233,25 @@ pub struct Cli {
#[arg(long = "volume.allowUntrustedRemoteEndpoints", default_value_t = false)]
pub allow_untrusted_remote_endpoints: bool,
/// Minimum bytes an fsync batch gathers before a drained queue flushes
/// without waiting.
#[arg(long = "fsyncMinBatchBytes", default_value_t = 1024 * 1024)]
pub fsync_min_batch_bytes: u64,
/// Minimum requests an fsync batch gathers before a drained queue flushes
/// without waiting.
#[arg(long = "fsyncMinBatchRequests", default_value_t = 128)]
pub fsync_min_batch_requests: usize,
/// How long a below-minimum fsync batch waits for more requests once the
/// queue drains; 0 flushes immediately.
#[arg(
long = "fsyncBatchWait",
default_value = "500us",
value_parser = parse_go_duration
)]
pub fsync_batch_wait: std::time::Duration,
/// A file of command line options, each line in optionName=optionValue format.
#[arg(long = "options", default_value = "")]
pub options: String,
@@ -308,6 +327,14 @@ pub struct VolumeServerConfig {
/// If true, FetchAndWriteNeedle skips remote S3 endpoint validation
/// (allows loopback / link-local / metadata hosts).
pub allow_untrusted_remote_endpoints: bool,
/// Minimum bytes an fsync batch gathers before a drained queue flushes
/// without waiting.
pub fsync_min_batch_bytes: u64,
/// Minimum requests an fsync batch gathers before a drained queue flushes
/// without waiting.
pub fsync_min_batch_requests: usize,
/// How long a below-minimum fsync batch waits for stragglers.
pub fsync_batch_wait: std::time::Duration,
}
pub use crate::storage::needle_map::NeedleMapKind;
@@ -938,6 +965,9 @@ fn resolve_config_with_env(cli: Cli, env: EnvLookup<'_>) -> VolumeServerConfig {
concurrent_download_limit: cli.concurrent_download_limit_mb as i64 * 1024 * 1024,
inflight_upload_data_timeout: cli.inflight_upload_data_timeout,
inflight_download_data_timeout: cli.inflight_download_data_timeout,
fsync_min_batch_bytes: cli.fsync_min_batch_bytes,
fsync_min_batch_requests: cli.fsync_min_batch_requests,
fsync_batch_wait: cli.fsync_batch_wait,
has_slow_read: cli.has_slow_read,
read_buffer_size_mb: cli.read_buffer_size_mb,
ldb_timeout: cli.ldb_timeout,
+12 -2
View File
@@ -23,7 +23,7 @@ use seaweed_volume::server::request_id::GrpcRequestIdLayer;
use seaweed_volume::server::volume_server::{
RuntimeMetricsConfig, VolumeServerState, build_metrics_router,
};
use seaweed_volume::server::write_queue::WriteQueue;
use seaweed_volume::server::write_queue::{self, WriteQueue};
use seaweed_volume::storage::store::Store;
use seaweed_volume::storage::types::DiskType;
@@ -436,7 +436,17 @@ async fn run(
// The write queue is always on: durable (fsync) uploads go through it so
// a batch shares one .dat/.idx sync, the way Go's per-volume async writer
// group-commits. Non-fsync writes stay on the synchronous path.
let wq = WriteQueue::new(state.clone(), 128);
let min_bytes = usize::try_from(config.fsync_min_batch_bytes)
.map_err(|_| "fsyncMinBatchBytes exceeds this platform's maximum usize value")?;
let wq = WriteQueue::new(
state.clone(),
128,
write_queue::SyncBatchOption {
min_bytes,
min_requests: config.fsync_min_batch_requests,
wait: config.fsync_batch_wait,
},
);
let _ = state.write_queue.set(wq);
// Set initial metric gauges for concurrent limits and max volumes
+3
View File
@@ -393,6 +393,9 @@ mod tests {
fix_jpg_orientation: false,
read_mode: ReadMode::Local,
allow_untrusted_remote_endpoints: false,
fsync_min_batch_bytes: 1024 * 1024,
fsync_min_batch_requests: 128,
fsync_batch_wait: std::time::Duration::from_micros(500),
cpu_profile: String::new(),
mem_profile: String::new(),
compaction_byte_per_second: 0,
+366 -75
View File
@@ -207,9 +207,15 @@ fn read_needle_for_get(
if request.try_meta_first(plan.size()) {
plan.read_meta(&mut n)?;
if n.cookie != cookie {
// The meta read answered the request.
plan.note_meta_only_success();
return Ok(None);
}
if request.served_from_source(&n) {
if request.is_head {
// HEAD ends here; no payload read will record the success.
plan.note_meta_only_success();
}
let info = plan.into_stream_info(&n);
return Ok(Some((n, Some(info))));
}
@@ -237,6 +243,9 @@ struct StreamingBody {
hold_read_lock_for_stream: bool,
_held_read_lease: Option<crate::storage::volume::DataFileReadLease>,
io_unavailable: Arc<crate::storage::volume::IoUnavailable>,
/// Chunk read failures feed the volume's io-error streak, and finishing
/// the stream clears it — Go readNeedleDataInto parity.
io_errors: Arc<crate::storage::io_error::IoErrorTracker>,
/// Pending chunk read; it hands `buf` back with the result.
pending: Option<tokio::task::JoinHandle<(bytes::BytesMut, std::io::Result<bytes::Bytes>)>>,
/// Chunk buffer, reclaimed once the previous frame has been written out.
@@ -298,12 +307,18 @@ impl http_body::Body for StreamingBody {
),
)));
}
// Hyper may not poll again once Content-Length is
// satisfied, so the read's success is recorded here.
self.io_errors.check_read_write_error(None);
}
return std::task::Poll::Ready(Some(Ok(http_body::Frame::data(
chunk,
))));
}
Ok(Err(e)) => return std::task::Poll::Ready(Some(Err(e))),
Ok(Err(e)) => {
self.io_errors.check_read_write_error(Some(&e));
return std::task::Poll::Ready(Some(Err(e)));
}
Err(e) => {
return std::task::Poll::Ready(Some(Err(std::io::Error::other(e))));
}
@@ -314,6 +329,7 @@ impl http_body::Body for StreamingBody {
let total = self.data_size as usize;
if self.pos >= total {
self.io_errors.check_read_write_error(None);
return std::task::Poll::Ready(None);
}
@@ -1104,7 +1120,15 @@ async fn get_or_head_handler_inner(
};
let (etag, last_modified_str) = etag_and_last_modified(&n);
// A Range whose If-Range no longer matches is ignored and the whole
// content is sent, so a resumed download restarts instead of mixing
// two versions.
let range = range.filter(|_| if_range_matches(&headers, &etag, &last_modified_str));
if let Some(resp) = not_modified_response(&n, &headers, &etag, &last_modified_str) {
// A request that ends here was answered by the meta read alone.
if let Some(io_errors) = strategy.io_errors() {
io_errors.check_read_write_error(None);
}
return resp;
}
@@ -1147,14 +1171,20 @@ async fn get_or_head_handler_inner(
return head_from_meta_response(&info, response_headers);
}
ReadStrategy::RangeFromSource(info) => {
return range_from_source_response(
&state,
range.as_deref().unwrap_or_default(),
info,
response_headers,
track_download,
)
.await;
return match range.as_deref() {
Some(range) => {
range_from_source_response(
&state,
range,
info,
response_headers,
track_download,
)
.await
}
// If-Range dropped the range; serve the whole content.
None => stream_response(&state, info, response_headers, track_download),
};
}
ReadStrategy::Buffered => {}
}
@@ -1198,6 +1228,19 @@ struct ReadPlan {
strategy: ReadStrategy,
}
impl ReadStrategy {
/// The tracker a terminal meta read reports success to, when the reply
/// is served from the data file.
fn io_errors(&self) -> Option<&Arc<crate::storage::io_error::IoErrorTracker>> {
match self {
ReadStrategy::Stream(info)
| ReadStrategy::HeadFromMeta(info)
| ReadStrategy::RangeFromSource(info) => Some(&info.io_errors),
ReadStrategy::Buffered => None,
}
}
}
/// The 401 reply for a bad read JWT; Go checks it before NewVolumeId, so a
/// bad path with JWT enabled is a 401, not a 400.
fn reject_read_jwt(
@@ -1486,41 +1529,103 @@ fn not_modified_response(
etag: &str,
last_modified_str: &Option<String>,
) -> Option<Response> {
// If-Modified-Since first (Go order), then If-None-Match.
let not_modified = || {
let mut resp = StatusCode::NOT_MODIFIED.into_response();
resp.headers_mut()
.insert(header::ETAG, etag.parse().unwrap());
if let Some(lm) = last_modified_str {
resp.headers_mut()
.insert(header::LAST_MODIFIED, lm.parse().unwrap());
}
resp
};
// If-Modified-Since applies only when there is no If-None-Match (RFC 9110 13.2.2).
if headers.contains_key(header::IF_NONE_MATCH) {
let inm = headers
.get_all(header::IF_NONE_MATCH)
.iter()
.map(|v| v.to_str().unwrap_or(""))
.collect::<Vec<_>>()
.join(",");
return etag_list_matches(&inm, etag).then(not_modified);
}
if n.last_modified > 0
&& let Some(ims_header) = headers.get(header::IF_MODIFIED_SINCE)
&& let Ok(ims_str) = ims_header.to_str()
&& let Some(ims_str) = headers
.get(header::IF_MODIFIED_SINCE)
.and_then(|v| v.to_str().ok())
{
// Parse HTTP date format: "Mon, 02 Jan 2006 15:04:05 GMT"
if let Ok(ims_time) =
chrono::NaiveDateTime::parse_from_str(ims_str, "%a, %d %b %Y %H:%M:%S GMT")
&& (n.last_modified as i64) <= ims_time.and_utc().timestamp()
{
let mut resp = StatusCode::NOT_MODIFIED.into_response();
if let Some(lm) = last_modified_str {
resp.headers_mut()
.insert(header::LAST_MODIFIED, lm.parse().unwrap());
}
// Go sets ETag after the 304 paths, so a 304 has no ETag.
return Some(resp);
return Some(not_modified());
}
}
if let Some(if_none_match) = headers.get(header::IF_NONE_MATCH)
&& let Ok(inm) = if_none_match.to_str()
&& inm == etag
{
let mut resp = StatusCode::NOT_MODIFIED.into_response();
if let Some(lm) = last_modified_str {
resp.headers_mut()
.insert(header::LAST_MODIFIED, lm.parse().unwrap());
}
// Go sets ETag after the 304 paths, so a 304 has no ETag.
return Some(resp);
}
None
}
/// If-Range (RFC 9110 13.1.5): an entity-tag by strong comparison, or a date
/// equal to Last-Modified. The response etag here is always a strong tag.
fn if_range_matches(headers: &HeaderMap, etag: &str, last_modified_str: &Option<String>) -> bool {
let Some(raw) = headers.get(header::IF_RANGE) else {
return true;
};
let Ok(if_range) = raw.to_str() else {
return false;
};
let if_range = if_range.trim();
if if_range.is_empty() {
return true;
}
if if_range.starts_with('"') || if_range.starts_with("W/") {
return if_range == etag;
}
let Ok(date) = httpdate::parse_http_date(if_range) else {
return false;
};
last_modified_str
.as_deref()
.and_then(|lm| httpdate::parse_http_date(lm).ok())
.is_some_and(|lm| date == lm)
}
/// Weak comparison against "*" or a comma-separated entity-tag list (RFC 9110 13.1.2).
fn etag_list_matches(header: &str, etag: &str) -> bool {
if header.trim() == "*" {
return true;
}
let etag = etag.trim_matches('"');
split_etag_list(header).into_iter().any(|tag| {
let tag = tag.trim();
let tag = tag.strip_prefix("W/").unwrap_or(tag);
// Empty list elements are ignored, so padding like a trailing
// comma cannot match an empty opaque tag.
!tag.is_empty() && tag.trim_matches('"') == etag
})
}
/// Splits an entity-tag list on the commas between tags; a comma inside a
/// quoted opaque tag does not split.
fn split_etag_list(header: &str) -> Vec<&str> {
let mut tags = Vec::new();
let (mut start, mut quoted) = (0, false);
for (i, b) in header.bytes().enumerate() {
match b {
b'"' => quoted = !quoted,
b',' if !quoted => {
tags.push(&header[start..i]);
start = i + 1;
}
_ => {}
}
}
tags.push(&header[start..]);
tags
}
/// The 200 reply headers, and the extension, which falls back to the stored name's.
fn read_response_headers(
n: &Needle,
@@ -1722,6 +1827,7 @@ fn stream_response(
data_file_access_control: info.data_file_access_control,
hold_read_lock_for_stream: !state.has_slow_read,
io_unavailable: info.io_unavailable,
io_errors: info.io_errors,
pending: None,
buf: bytes::BytesMut::new(),
state: tracking_state,
@@ -2121,9 +2227,21 @@ fn handle_range_request_from_source(
return Ok(Vec::new());
}
let mut buf = vec![0u8; length as usize];
info.source
.read_exact_at(&mut buf, info.data_file_offset + start as u64)?;
Ok(buf)
match info
.source
.read_exact_at(&mut buf, info.data_file_offset + start as u64)
{
// A finished range read clears the streak a failed meta read
// counted, like readNeedleDataInto's terminal report.
Ok(()) => {
info.io_errors.check_read_write_error(None);
Ok(buf)
}
Err(e) => {
info.io_errors.check_read_write_error(Some(&e));
Err(e)
}
}
};
if ranges.len() == 1 {
@@ -2956,50 +3074,62 @@ pub async fn post_handler(
let needle_etag = n.etag();
let needle_has_name = n.has_name();
let write_result = write_upload_needle(&state, vid, n, fsync).await;
// Replicate to remote volume servers if this volume has replicas.
// Matches Go's GetWritableRemoteReplications: skip if copy_count == 1.
let needs_replication = !is_replicate && {
let store = state.store.read().unwrap();
store
.find_volume(vid)
.is_some_and(|(_, v)| v.super_block().replica_placement.get_copy_count() > 1)
};
// The fan-out runs concurrently with the local needle write, like Go's
// ReplicatedWrite: it reads the already-buffered request body, so the two
// sides share no mutable state. The ack still waits for both.
let replication = if needs_replication {
let state_clone = state.clone();
let path_clone = path.clone();
let query_clone = query.clone();
let headers_clone = headers.clone();
let body_clone = body.clone();
Some(tokio::spawn(async move {
do_replicated_request(
&state_clone,
vid.0,
Method::POST,
&path_clone,
&query_clone,
&headers_clone,
Some(body_clone),
)
.await
}))
} else {
None
};
let write_result = write_upload_needle(&state, vid, n, fsync).await;
// No leader-known gate: when the lookup cannot run, the write must fail
// rather than be acknowledged on fewer copies than required.
if !is_replicate && write_result.is_ok() {
let needs_replication = {
let store = state.store.read().unwrap();
store
.find_volume(vid)
.is_some_and(|(_, v)| v.super_block().replica_placement.get_copy_count() > 1)
};
if needs_replication {
let state_clone = state.clone();
let path_clone = path.clone();
let query_clone = query.clone();
let headers_clone = headers.clone();
let body_clone = body.clone();
let replication = tokio::spawn(async move {
do_replicated_request(
&state_clone,
vid.0,
Method::POST,
&path_clone,
&query_clone,
&headers_clone,
Some(body_clone),
)
.await
});
let replication_result = replication
.await
// rather than be acknowledged on fewer copies than required. The local
// write result takes precedence, matching Go's error ordering.
let replication_result = match replication {
Some(task) => Some(
task.await
.map_err(|e| format!("replication task failed: {}", e))
.flatten();
if let Err(e) = replication_result {
tracing::error!("replicated write failed: {}", e);
return json_error_with_query(
StatusCode::INTERNAL_SERVER_ERROR,
format!("replication failed: {}", e),
Some(&query),
);
}
}
.flatten(),
),
None => None,
};
if write_result.is_ok()
&& let Some(Err(e)) = replication_result
{
tracing::error!("replicated write failed: {}", e);
return json_error_with_query(
StatusCode::INTERNAL_SERVER_ERROR,
format!("replication failed: {}", e),
Some(&query),
);
}
let resp = match write_result {
@@ -5225,6 +5355,7 @@ mod tests {
hold_read_lock_for_stream: true,
_held_read_lease: None,
io_unavailable: Arc::default(),
io_errors: Arc::default(),
pending: None,
buf: bytes::BytesMut::new(),
state: None,
@@ -5597,6 +5728,127 @@ mod tests {
delete_handler(State(state.clone()), request).await.status()
}
#[test]
fn test_not_modified_checks_if_none_match_first() {
let n = Needle {
last_modified: 1700000000,
..Needle::default()
};
let (etag, last_modified_str) = etag_and_last_modified(&n);
let bare = n.etag();
let before = "Tue, 14 Nov 2023 22:12:20 GMT".to_string();
let after = "Tue, 14 Nov 2023 22:14:20 GMT".to_string();
let cases: Vec<(Vec<(axum::http::HeaderName, String)>, bool)> = vec![
(vec![(header::IF_NONE_MATCH, format!("\"{bare}\""))], true),
(vec![(header::IF_NONE_MATCH, format!("W/\"{bare}\""))], true),
(
vec![(header::IF_NONE_MATCH, format!("\"x\", \"{bare}\""))],
true,
),
(vec![(header::IF_NONE_MATCH, "*".into())], true),
(vec![(header::IF_NONE_MATCH, "\"x\"".into())], false),
(
vec![
(header::IF_NONE_MATCH, "\"x\"".into()),
(header::IF_NONE_MATCH, format!("\"{bare}\"")),
],
true,
),
(
vec![(header::IF_NONE_MATCH, format!("\"{bare},old\""))],
false,
),
(
vec![(header::IF_NONE_MATCH, format!("\"x\",,\"{bare}\""))],
true,
),
(
vec![
(header::IF_NONE_MATCH, "\"x\"".into()),
(header::IF_MODIFIED_SINCE, after.clone()),
],
false,
),
(
vec![
(header::IF_NONE_MATCH, "".into()),
(header::IF_MODIFIED_SINCE, after.clone()),
],
false,
),
(vec![(header::IF_MODIFIED_SINCE, after)], true),
(vec![(header::IF_MODIFIED_SINCE, before)], false),
];
for (pairs, want) in cases {
let mut headers = HeaderMap::new();
for (name, value) in &pairs {
headers.append(name, value.parse().unwrap());
}
assert_eq!(
not_modified_response(&n, &headers, &etag, &last_modified_str).is_some(),
want,
"{pairs:?}"
);
}
let mut headers = HeaderMap::new();
headers.insert(
header::IF_NONE_MATCH,
format!("\"{bare}\"").parse().unwrap(),
);
let resp = not_modified_response(&n, &headers, &etag, &last_modified_str).unwrap();
assert!(resp.headers().contains_key(header::ETAG));
assert!(resp.headers().contains_key(header::LAST_MODIFIED));
}
#[test]
fn test_if_range_matches() {
let etag = "\"v2\"".to_string();
let last_modified_str = Some("Sat, 24 Oct 2026 22:01:04 GMT".to_string());
let cases: Vec<(Option<&str>, bool)> = vec![
(None, true),
(Some("\"v2\""), true),
(Some("Sat, 24 Oct 2026 22:01:04 GMT"), true),
(Some("Saturday, 24-Oct-26 22:01:04 GMT"), true),
(Some("Sat Oct 24 22:01:04 2026"), true),
(Some("\"v1\""), false),
(Some("W/\"v2\""), false),
(Some("Sat, 24 Oct 2026 22:01:03 GMT"), false),
(Some("not a date"), false),
];
for (if_range, want) in cases {
let mut headers = HeaderMap::new();
if let Some(v) = if_range {
headers.insert(header::IF_RANGE, v.parse().unwrap());
}
assert_eq!(
if_range_matches(&headers, &etag, &last_modified_str),
want,
"{if_range:?}"
);
}
// A present value that is not valid UTF-8 fails the condition.
let mut headers = HeaderMap::new();
headers.insert(
header::IF_RANGE,
header::HeaderValue::from_bytes(b"\"\xff\xfev2\"").unwrap(),
);
assert!(!if_range_matches(&headers, &etag, &last_modified_str));
// No Last-Modified on the representation: a date validator fails,
// an entity-tag still applies.
let mut headers = HeaderMap::new();
headers.insert(header::IF_RANGE, "\"v2\"".parse().unwrap());
assert!(if_range_matches(&headers, &etag, &None));
headers.insert(
header::IF_RANGE,
"Sat, 24 Oct 2026 22:01:04 GMT".parse().unwrap(),
);
assert!(!if_range_matches(&headers, &etag, &None));
}
/// A GET whose needle read is parked must not hold the store lock: a
/// writer, here a real append to the same volume, must get through.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -5771,6 +6023,44 @@ mod tests {
assert_eq!(status, StatusCode::NOT_FOUND);
}
/// A ranged read served from the data file reports its outcome to the
/// io-error streak, like readNeedleDataInto: a finished range clears the
/// count a failed meta read left.
#[test]
fn test_range_from_source_reports_to_io_error_streak() {
use std::io::Write;
let tmp = tempfile::TempDir::new().unwrap();
let file = std::fs::OpenOptions::new()
.read(true)
.write(true)
.create(true)
.open(tmp.path().join("1.dat"))
.unwrap();
let data: Vec<u8> = (0..256u32).map(|i| i as u8).collect();
(&file).write_all(&data).unwrap();
let io_errors = Arc::new(crate::storage::io_error::IoErrorTracker::default());
let eio = crate::storage::volume::media_error_for_test();
io_errors.check_read_write_error(Some(&eio));
io_errors.check_read_write_error(Some(&eio));
assert_eq!(io_errors.get_io_error_state().1, 2);
let info = crate::storage::volume::NeedleStreamInfo {
source: crate::storage::volume::NeedleStreamSource::Local(file),
data_file_offset: 0,
data_size: data.len() as u32,
data_file_access_control: Arc::default(),
io_unavailable: Arc::default(),
io_errors: io_errors.clone(),
needle_id: NeedleId(1),
checksum: 0,
};
let resp = handle_range_request_from_source("bytes=10-19", info, HeaderMap::new(), None);
assert_eq!(resp.status(), StatusCode::PARTIAL_CONTENT);
assert_eq!(io_errors.get_io_error_state().1, 0);
}
/// Range is read as Go reads it: a byte its parseRange cannot parse is a
/// 416, Unicode whitespace around a range is trimmed, an empty value is no
/// range, and HEAD ignores it.
@@ -7223,6 +7513,7 @@ mod tests {
.set(crate::server::write_queue::WriteQueue::new(
state.clone(),
128,
crate::server::write_queue::SyncBatchOption::default(),
))
.is_ok()
);
+3
View File
@@ -132,6 +132,9 @@ mod tests {
fix_jpg_orientation: false,
read_mode: ReadMode::Local,
allow_untrusted_remote_endpoints: false,
fsync_min_batch_bytes: 1024 * 1024,
fsync_min_batch_requests: 128,
fsync_batch_wait: std::time::Duration::from_micros(500),
cpu_profile: String::new(),
mem_profile: String::new(),
compaction_byte_per_second: 0,
+164 -12
View File
@@ -56,6 +56,27 @@ const MAX_CONCURRENT_VOLUME_WRITES: usize = 8;
/// retirement is the volume leaving the store.
const IDLE_WORKER_POLL: std::time::Duration = std::time::Duration::from_secs(60);
/// Tunes the per-volume durable-write worker. Once the queue drains with the
/// batch still below min_bytes and min_requests, the batch stays open up to
/// wait for stragglers instead of paying an fsync per request. Mirrors Go's
/// storage.SyncBatchOption.
#[derive(Clone, Copy)]
pub struct SyncBatchOption {
pub min_bytes: usize,
pub min_requests: usize,
pub wait: std::time::Duration,
}
impl Default for SyncBatchOption {
fn default() -> Self {
SyncBatchOption {
min_bytes: 1024 * 1024,
min_requests: MAX_BATCH_SIZE,
wait: std::time::Duration::from_micros(500),
}
}
}
/// Handle for submitting write requests to the per-volume workers.
#[derive(Clone)]
pub struct WriteQueue {
@@ -67,23 +88,35 @@ pub struct WriteQueue {
/// alone would multiply the old single queue's capacity by volume count.
queue_permits: Arc<Semaphore>,
workers: Arc<Mutex<HashMap<VolumeId, mpsc::Sender<WriteRequest>>>>,
sync_batch: SyncBatchOption,
/// Submissions accepted, for tests asserting which writes route here.
#[cfg(test)]
submits: Arc<std::sync::atomic::AtomicU64>,
/// Times a worker armed the straggler window, for tests that must land a
/// write inside an already-open batch.
#[cfg(test)]
windows_armed: Arc<std::sync::atomic::AtomicU64>,
}
impl WriteQueue {
/// Create a new write queue. Per-volume workers are spawned lazily on the
/// first submission for that volume.
pub fn new(state: Arc<VolumeServerState>, capacity: usize) -> Self {
pub fn new(
state: Arc<VolumeServerState>,
capacity: usize,
sync_batch: SyncBatchOption,
) -> Self {
WriteQueue {
state,
capacity,
io_permits: Arc::new(Semaphore::new(MAX_CONCURRENT_VOLUME_WRITES)),
queue_permits: Arc::new(Semaphore::new(capacity)),
workers: Arc::new(Mutex::new(HashMap::new())),
sync_batch,
#[cfg(test)]
submits: Arc::new(std::sync::atomic::AtomicU64::new(0)),
#[cfg(test)]
windows_armed: Arc::new(std::sync::atomic::AtomicU64::new(0)),
}
}
@@ -93,6 +126,12 @@ impl WriteQueue {
self.submits.load(std::sync::atomic::Ordering::Relaxed)
}
/// How many batch windows workers have armed so far.
#[cfg(test)]
fn windows_armed_for_test(&self) -> u64 {
self.windows_armed.load(std::sync::atomic::Ordering::Relaxed)
}
/// Sender for the volume's worker, spawning one on first use.
fn worker_for(&self, volume_id: VolumeId) -> mpsc::Sender<WriteRequest> {
let mut workers = self.workers.lock().unwrap();
@@ -107,6 +146,9 @@ impl WriteQueue {
state: self.state.clone(),
io_permits: self.io_permits.clone(),
workers: self.workers.clone(),
sync_batch: self.sync_batch,
#[cfg(test)]
windows_armed: self.windows_armed.clone(),
}
.run(),
);
@@ -191,6 +233,9 @@ struct VolumeWriteWorker {
state: Arc<VolumeServerState>,
io_permits: Arc<Semaphore>,
workers: Arc<Mutex<HashMap<VolumeId, mpsc::Sender<WriteRequest>>>>,
sync_batch: SyncBatchOption,
#[cfg(test)]
windows_armed: Arc<std::sync::atomic::AtomicU64>,
}
impl VolumeWriteWorker {
@@ -238,14 +283,40 @@ impl VolumeWriteWorker {
let mut batch = Vec::with_capacity(MAX_BATCH_SIZE);
let mut batch_bytes: usize = first.needle.data.len();
batch.push(first);
// Armed once the queue first drains below the minimum batch; holds the
// batch open for stragglers so the wait bound is per batch, not per
// lull (Go's batchWait).
let mut window_deadline = None;
while batch.len() < MAX_BATCH_SIZE && batch_bytes < MAX_BATCH_BYTES {
match self.rx.try_recv() {
Ok(req) => {
batch_bytes += req.needle.data.len();
batch.push(req);
}
Err(_) => break,
if window_deadline.is_some_and(|d| tokio::time::Instant::now() >= d) {
break;
}
let req = match self.rx.try_recv() {
Ok(req) => req,
Err(mpsc::error::TryRecvError::Empty) => {
let sync = self.sync_batch;
if sync.wait.is_zero()
|| batch_bytes >= sync.min_bytes
|| batch.len() >= sync.min_requests
{
// a drained queue flushes right away once it holds enough
break;
}
let deadline = *window_deadline.get_or_insert_with(|| {
#[cfg(test)]
self.windows_armed
.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
tokio::time::Instant::now() + sync.wait
});
match tokio::time::timeout_at(deadline, self.rx.recv()).await {
Ok(Some(req)) => req,
Ok(None) | Err(_) => break,
}
}
Err(mpsc::error::TryRecvError::Disconnected) => break,
};
batch_bytes += req.needle.data.len();
batch.push(req);
}
debug!("processing write batch of {} requests", batch.len());
@@ -380,7 +451,7 @@ mod tests {
async fn test_write_queue_submit_no_volume() {
// Submit a write to a non-existent volume -- should return VolumeError::NotFound.
let state = make_test_state();
let queue = WriteQueue::new(state, MAX_BATCH_SIZE);
let queue = WriteQueue::new(state, MAX_BATCH_SIZE, SyncBatchOption::default());
let needle = Needle {
id: 1.into(),
@@ -402,7 +473,7 @@ mod tests {
async fn test_write_queue_concurrent_submissions() {
// Submit multiple concurrent writes -- all should complete (with errors since no volume).
let state = make_test_state();
let queue = WriteQueue::new(state, MAX_BATCH_SIZE);
let queue = WriteQueue::new(state, MAX_BATCH_SIZE, SyncBatchOption::default());
let mut handles = Vec::new();
for i in 0..10u64 {
@@ -430,7 +501,7 @@ mod tests {
async fn test_write_queue_batching() {
// Verify that many concurrent writes get processed (testing the batching path).
let state = make_test_state();
let queue = WriteQueue::new(state, MAX_BATCH_SIZE);
let queue = WriteQueue::new(state, MAX_BATCH_SIZE, SyncBatchOption::default());
// Submit MAX_BATCH_SIZE requests concurrently
let mut handles = Vec::new();
@@ -522,7 +593,7 @@ mod tests {
async fn test_write_queue_dropped_sender() {
// When the queue is dropped, subsequent submits should fail gracefully.
let state = make_test_state();
let queue = WriteQueue::new(state, 1);
let queue = WriteQueue::new(state, 1, SyncBatchOption::default());
// Clone then drop the original -- the worker keeps running via its rx handle.
let queue2 = queue.clone();
@@ -665,7 +736,7 @@ mod tests {
.unwrap();
}
}
let queue = WriteQueue::new(state, MAX_BATCH_SIZE);
let queue = WriteQueue::new(state, MAX_BATCH_SIZE, SyncBatchOption::default());
let in_append = Arc::new(Barrier::new(2));
let release = Arc::new(Barrier::new(2));
@@ -734,4 +805,85 @@ mod tests {
assert!(a1.await.unwrap().is_ok());
assert!(a2.await.unwrap().is_ok());
}
/// A durable write below the minimum batch rides out its window so a
/// straggler joins the same sync instead of each paying an fsync alone —
/// Go's batchWait. Paused time makes the hold observable: a lone write can
/// only complete once the runtime auto-advances to the window's deadline.
#[tokio::test(start_paused = true)]
async fn test_batch_window_defers_flush_for_stragglers() {
use crate::config::MinFreeSpace;
use crate::storage::types::{DiskType, NeedleId};
use crate::storage::volume::VolumeSpec;
use std::time::Duration;
let tmp = tempfile::TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let state = make_test_state();
{
let mut store = state.store.write().unwrap();
store
.add_location(
dir,
dir,
10,
DiskType::HardDrive,
MinFreeSpace::Percent(1.0),
Vec::new(),
)
.unwrap();
store
.add_volume(VolumeId(1), DiskType::HardDrive, &VolumeSpec::default())
.unwrap();
}
let queue = WriteQueue::new(
state,
MAX_BATCH_SIZE,
SyncBatchOption {
min_bytes: usize::MAX,
min_requests: MAX_BATCH_SIZE,
wait: Duration::from_secs(30),
},
);
let needle = |id: u64| Needle {
id: NeedleId(id),
cookie: 0x1111.into(),
data: vec![id as u8; 8],
data_size: 8,
..Needle::default()
};
let start = tokio::time::Instant::now();
queue.submit(VolumeId(1), needle(1), true).await.unwrap();
assert!(start.elapsed() >= Duration::from_secs(30));
// Wait for the worker to take the second write and open its batch
// window, so the straggler provably lands inside an open batch
// instead of arriving during the initial drain.
let armed = queue.windows_armed_for_test();
let second = tokio::spawn({
let queue = queue.clone();
async move { queue.submit(VolumeId(1), needle(2), true).await }
});
for _ in 0..10_000 {
if queue.windows_armed_for_test() > armed {
break;
}
tokio::task::yield_now().await;
}
assert!(queue.windows_armed_for_test() > armed);
let third = tokio::spawn({
let queue = queue.clone();
async move { queue.submit(VolumeId(1), needle(3), true).await }
});
let (second, third) = tokio::join!(second, third);
second.unwrap().unwrap();
third.unwrap().unwrap();
// One shared window ends both writes ~30s after their batch armed;
// back-to-back windows would keep the third out to ~90s.
let elapsed = start.elapsed();
assert!(elapsed >= Duration::from_secs(60) && elapsed < Duration::from_secs(90));
}
}
+5 -1
View File
@@ -740,7 +740,11 @@ impl DiskLocation {
if vol.is_read_only() {
continue;
}
let dat_size = vol.dat_file_size().unwrap_or(0);
// A preallocated volume already holds its space on disk.
let dat_size = vol
.dat_file_size()
.unwrap_or(0)
.max(vol.dat_allocated_size());
let idx_size = vol.idx_file_size();
let used = dat_size + idx_size;
if volume_size_limit > used {
+142 -43
View File
@@ -18,7 +18,7 @@ use crate::storage::erasure_coding::ec_volume::{
};
use crate::storage::needle::needle::{Needle, get_actual_size};
use crate::storage::needle_map::NeedleMapKind;
use crate::storage::super_block::{ReplicaPlacement, SUPER_BLOCK_SIZE};
use crate::storage::super_block::{ReplicaPlacement, SUPER_BLOCK_SIZE, SuperBlock};
use crate::storage::types::*;
use crate::storage::volume::{CompactionJob, VifVolumeInfo, Volume, VolumeError, VolumeSpec};
@@ -762,8 +762,8 @@ impl Store {
}
/// Configure a volume's replica placement on disk.
/// The volume must already be unmounted. This opens the .dat file directly,
/// modifies the replica_placement byte (offset 1), and writes it back.
/// The volume must already be unmounted. Rewrites the .vif and the .dat
/// superblock so both carry the new placement.
pub fn configure_volume(&self, vid: VolumeId, rp: ReplicaPlacement) -> Result<(), VolumeError> {
let (_, base_path, _) = self.find_volume_file_base(vid).ok_or_else(|| {
VolumeError::Io(io::Error::new(
@@ -775,7 +775,7 @@ impl Store {
let mut vif = load_vif_volume_info(&vif_path)?;
vif.replication = rp.to_string();
save_vif_volume_info(&vif_path, &vif)?;
Ok(())
update_dat_replica_placement(&format!("{}.dat", base_path), vid, rp)
}
// ---- Read / Write / Delete ----
@@ -943,11 +943,7 @@ impl Store {
let current = loc.max_volume_count.load(Ordering::Relaxed);
let (_, free) = super::disk_location::get_disk_stats(&loc.directory);
let unused_space = if self.get_preallocate() {
0
} else {
loc.unused_space(volume_size_limit)
};
let unused_space = loc.unused_space(volume_size_limit);
let unclaimed = (free as i64) - (unused_space as i64);
let vol_count = loc.volumes_len() as i32;
@@ -1744,6 +1740,61 @@ fn strip_volume_suffix(filename: &str) -> Option<&str> {
.or_else(|| filename.strip_suffix(".idx"))
}
/// Rewrites the replica placement in a .dat header for a volume that is not
/// mounted; a missing .dat means a remote-only volume, where the .vif carries
/// the placement.
fn update_dat_replica_placement(
dat_path: &str,
vid: VolumeId,
rp: ReplicaPlacement,
) -> Result<(), VolumeError> {
use std::io::{Read, Seek, SeekFrom, Write};
let mut file = match std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(dat_path)
{
Ok(f) => f,
Err(e) if e.kind() == io::ErrorKind::NotFound => return Ok(()),
Err(e) => {
return Err(VolumeError::Io(io::Error::new(
e.kind(),
format!("volume {} failed to open {}: {}", vid, dat_path, e),
)))
}
};
let mut header = [0u8; SUPER_BLOCK_SIZE];
file.read_exact(&mut header).map_err(|e| {
VolumeError::Io(io::Error::new(
e.kind(),
format!("volume {} failed to parse superblock: {}", vid, e),
))
})?;
let extra_size = u16::from_be_bytes([header[6], header[7]]) as usize;
let mut block = header.to_vec();
block.resize(SUPER_BLOCK_SIZE + extra_size, 0);
if extra_size > 0 {
file.read_exact(&mut block[SUPER_BLOCK_SIZE..]).map_err(|e| {
VolumeError::Io(io::Error::new(
e.kind(),
format!("volume {} failed to parse superblock: {}", vid, e),
))
})?;
}
let mut super_block = SuperBlock::from_bytes(&block).map_err(|e| {
VolumeError::Io(io::Error::new(
io::ErrorKind::InvalidData,
format!("volume {} failed to parse superblock: {}", vid, e),
))
})?;
super_block.replica_placement = rp;
file.seek(SeekFrom::Start(0))?;
file.write_all(&super_block.to_bytes())?;
file.sync_all()?;
Ok(())
}
fn load_vif_volume_info(path: &str) -> Result<VifVolumeInfo, VolumeError> {
let content = match std::fs::read_to_string(path) {
Ok(c) => c,
@@ -1981,6 +2032,33 @@ mod tests {
assert_eq!(store.total_volume_count(), 1);
}
// Mirrors Go's TestConfigureVolumeUpdatesSuperblock: configure must leave
// the .dat header and the .vif both on the new placement.
#[test]
fn test_configure_volume_updates_superblock() {
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let mut store = make_test_store(&[dir]);
store
.add_volume(VolumeId(1), DiskType::HardDrive, &VolumeSpec::default())
.unwrap();
assert!(store.unmount_volume(VolumeId(1)).unwrap());
let rp = ReplicaPlacement::from_string("010").unwrap();
store.configure_volume(VolumeId(1), rp).unwrap();
let base = volume_file_name(dir, "", VolumeId(1));
let vif = load_vif_volume_info(&format!("{}.vif", base)).unwrap();
assert_eq!(vif.replication, "010");
let dat = std::fs::read(format!("{}.dat", base)).unwrap();
let super_block = SuperBlock::from_bytes(&dat).unwrap();
assert_eq!(super_block.replica_placement, rp);
assert!(store.configure_volume(VolumeId(2), rp).is_err());
}
#[test]
fn test_mount_volume_by_id_hint_skips_lone_sidecar() {
// A lone .vif/.idx sidecar (e.g. an EC sidecar whose .ecx is on a
@@ -2479,7 +2557,7 @@ mod tests {
}
#[test]
fn test_maybe_adjust_volume_max_honors_preallocate_flag() {
fn test_maybe_adjust_volume_max_reserves_plain_volumes_while_preallocating() {
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
@@ -2494,13 +2572,6 @@ mod tests {
Vec::new(),
)
.unwrap();
// Use a large volume_size_limit so the unused-space difference between
// preallocate=true (0) and preallocate=false (~2 × limit) is big enough
// that integer-division rounding and disk-free fluctuations between the
// two get_disk_stats calls cannot make the quotients equal.
store
.volume_size_limit
.store(100 * 1024 * 1024, Ordering::Relaxed);
store
.add_volume(
VolumeId(61),
@@ -2511,39 +2582,67 @@ mod tests {
},
)
.unwrap();
// Room for 2.5 volumes: the plain volume claims one, leaving one more slot.
let (_, free) = crate::storage::disk_location::get_disk_stats(dir);
let volume_size_limit = (free as f64 / 2.5) as u64;
if volume_size_limit == 0 {
return; // cannot determine free disk space
}
store
.add_volume(
VolumeId(62),
.volume_size_limit
.store(volume_size_limit, Ordering::Relaxed);
store.locations[0].original_max_volume_count = 0;
store.set_preallocate(true);
store.maybe_adjust_volume_max();
assert_eq!(
store.locations[0].max_volume_count.load(Ordering::Relaxed),
2
);
}
#[cfg(target_os = "linux")]
#[test]
fn test_unused_space_counts_preallocated_volumes_as_full() {
const LIMIT: u64 = 1 << 20;
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let mut store = Store::new(NeedleMapKind::InMemory);
store
.add_location(
dir,
dir,
2,
DiskType::HardDrive,
&VolumeSpec {
collection: "preallocate_case",
..Default::default()
},
MinFreeSpace::Percent(1.0),
Vec::new(),
)
.unwrap();
for vid in [VolumeId(61), VolumeId(62)] {
let dat_path = store.find_volume(vid).unwrap().1.dat_path();
std::fs::OpenOptions::new()
.write(true)
.open(dat_path)
.unwrap()
.set_len((crate::storage::super_block::SUPER_BLOCK_SIZE + 1) as u64)
for (vid, preallocate) in [(VolumeId(71), LIMIT), (VolumeId(72), 0)] {
store
.add_volume(
vid,
DiskType::HardDrive,
&VolumeSpec {
collection: "preallocate_case",
preallocate,
..Default::default()
},
)
.unwrap();
}
store.locations[0].original_max_volume_count = 0;
store.locations[0]
.max_volume_count
.store(0, Ordering::Relaxed);
let preallocated = store.find_volume(VolumeId(71)).unwrap().1;
if preallocated.dat_allocated_size() < LIMIT {
return; // the filesystem does not support fallocate
}
store.set_preallocate(false);
assert!(store.maybe_adjust_volume_max());
let without_preallocate = store.locations[0].max_volume_count.load(Ordering::Relaxed);
store.set_preallocate(true);
assert!(store.maybe_adjust_volume_max());
let with_preallocate = store.locations[0].max_volume_count.load(Ordering::Relaxed);
assert!(with_preallocate > without_preallocate);
let plain = store.find_volume(VolumeId(72)).unwrap().1;
let plain_dat = plain.dat_file_size().unwrap();
// The filesystem may round a small file up to a larger allocation unit.
let plain_used = plain_dat.max(plain.dat_allocated_size()) + plain.idx_file_size();
assert_eq!(store.locations[0].unused_space(LIMIT), LIMIT - plain_used);
}
#[test]
+180 -9
View File
@@ -836,7 +836,7 @@ impl NeedleReadPlan {
pub(crate) fn read_meta(&mut self, n: &mut Needle) -> Result<(), VolumeError> {
let (source, size, version) = (&self.source, self.size, self.version);
let read_at = |buf: &mut [u8], at: u64| Ok(source.read_exact_at(buf, at)?);
with_offset_retry(&mut self.offset, |off| {
let result = with_offset_retry(&mut self.offset, |off| {
if version == VERSION_1 {
// A V1 body is all data, so its "meta" is the whole record.
let mut buf = vec![0u8; get_actual_size(size, version) as usize];
@@ -847,13 +847,26 @@ impl NeedleReadPlan {
} else {
Volume::read_needle_meta_blob_and_parse(read_at, n, off, size, version)
}
})?;
});
// Report failures only: a successful meta read precedes the payload
// read, so clearing the streak here would reset the count before
// every failing download and the volume could never reach quarantine.
if let Err(VolumeError::Io(e)) = &result {
self.io_errors.check_read_write_error(Some(e));
}
result?;
if needle_expired(n) {
return Err(VolumeError::NotFound);
}
Ok(())
}
/// A successful meta read ended the request (e.g. HEAD), so it may reset
/// the streak — unlike a meta read that precedes a payload read.
pub(crate) fn note_meta_only_success(&self) {
self.io_errors.check_read_write_error(None);
}
/// For a reply served from the data file, after `read_meta` filled `n`.
pub(crate) fn into_stream_info(self, n: &Needle) -> NeedleStreamInfo {
// V1 data starts right after the header, V2/V3 after the DataSize field.
@@ -868,6 +881,7 @@ impl NeedleReadPlan {
data_size: n.data_size,
data_file_access_control: self.data_file_access_control,
io_unavailable: self.io_unavailable,
io_errors: self.io_errors,
needle_id: self.needle_id,
checksum: n.checksum.0,
}
@@ -1144,6 +1158,9 @@ pub struct NeedleStreamInfo {
pub data_file_access_control: Arc<DataFileAccessControl>,
/// Checked before each chunk: the volume can become unavailable mid-stream.
pub(crate) io_unavailable: Arc<IoUnavailable>,
/// Streamed chunks report read failures here and clear the streak when
/// the whole body has been emitted, mirroring readNeedleDataInto.
pub(crate) io_errors: Arc<IoErrorTracker>,
pub needle_id: NeedleId,
/// Checksum stored in the needle tail, verified once the last chunk has
/// been read — before that frame is emitted.
@@ -2918,18 +2935,52 @@ impl Volume {
}
// Cookie validation for existing needle (matches Go: check whenever nm.Get returns ok)
if let Some(nm) = &io.nm
&& let Some(nv) = nm.get(n.id)?
let indexed = match io.nm.as_ref() {
Some(nm) => nm.get(n.id)?,
None => None,
};
if let Some(nv) = indexed
&& !nv.offset.is_zero()
{
let mut existing = Needle::default();
// Read only the header to check cookie
self.read_needle_header_unlocked(io, &mut existing, nv.offset.to_actual_offset())?;
if n.cookie.0 == 0 && !check_cookie {
n.cookie = existing.cookie;
if existing.id != n.id && existing.size != nv.size {
// a 4-byte index entry can index past the 32 GiB wrap: the
// low offset may hold another needle while the real record
// sits at the wrapped offset, where reads already retry
#[cfg(not(feature = "5bytes"))]
if nv.offset.to_actual_offset() < MAX_POSSIBLE_VOLUME_SIZE as i64 {
let mut wrapped = Needle::default();
match self.read_needle_header_unlocked(
io,
&mut wrapped,
nv.offset.to_actual_offset() + MAX_POSSIBLE_VOLUME_SIZE as i64,
) {
Ok(()) => existing = wrapped,
Err(VolumeError::Io(e)) if e.kind() == io::ErrorKind::UnexpectedEof => {}
Err(e) => return Err(e),
}
}
}
if existing.cookie != n.cookie {
return Err(VolumeError::CookieMismatch(n.cookie.0));
if existing.id != n.id && existing.size != nv.size {
// the entry resolves to a record that is neither the indexed
// key nor the indexed size, so it is stale rather than a
// duplicate — skip the cookie check and let publish_write
// overwrite it once the append commits
warn!(
"volume {} index entry for needle {} resolves to needle {}; overwriting stale index entry",
self.id.0, n.id.0, existing.id.0
);
} else {
if n.cookie.0 == 0 && !check_cookie {
n.cookie = existing.cookie;
}
if existing.cookie != n.cookie {
return Err(VolumeError::CookieMismatch(n.cookie.0));
}
}
}
Ok(false)
@@ -4985,6 +5036,25 @@ impl Volume {
self.current_dat_file_size()
}
/// Disk space held by the local .dat file, or 0 when it is unknown. It
/// exceeds the file size when the file was preallocated with
/// `FALLOC_FL_KEEP_SIZE`.
pub fn dat_allocated_size(&self) -> u64 {
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
let io = self.io.read().unwrap();
io.dat_file
.as_ref()
.and_then(|f| f.metadata().ok())
.map_or(0, |m| m.blocks() * 512)
}
#[cfg(not(unix))]
{
0
}
}
/// Get the modification time of the .dat file as Unix seconds.
pub fn dat_file_mod_time(&self) -> u64 {
let io = self.io.read().unwrap();
@@ -6048,7 +6118,7 @@ fn preallocate_file(file: &File, size: u64) {
/// An OS error the platform reports for failing storage media, which is
/// what counts toward the I/O error streak.
#[cfg(test)]
fn media_error_for_test() -> io::Error {
pub(crate) fn media_error_for_test() -> io::Error {
#[cfg(unix)]
{
io::Error::from_raw_os_error(libc::EIO)
@@ -9025,6 +9095,107 @@ mod tests {
assert!(matches!(err, VolumeError::CookieMismatch(_)));
}
/// A damaged index entry resolving to a different needle must miss on
/// reads and be dropped by the next write for the key, not rejected as a
/// cookie conflict.
#[test]
fn test_stale_index_entry_heals_on_write() {
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let v = make_test_volume(dir);
let mut first = Needle {
id: NeedleId(2),
cookie: Cookie(0x2222),
data: b"second".to_vec(),
data_size: 6,
..Needle::default()
};
v.write_needle(&mut first, true, false).unwrap();
// point key 9 at needle 2's record with a size that does not match
// the record, as a corrupt index entry would look
{
let mut io = v.io.write().unwrap();
let nm = io.nm.as_mut().unwrap();
let nv = nm.get(NeedleId(2)).unwrap().unwrap();
nm.put(NeedleId(9), nv.offset, Size(nv.size.0 + 1)).unwrap();
}
let mut probe = Needle {
id: NeedleId(9),
..Needle::default()
};
assert!(v.read_needle(&mut probe).is_err());
let mut n = Needle {
id: NeedleId(9),
cookie: Cookie(0xabcd),
data: b"ninth".to_vec(),
data_size: 5,
..Needle::default()
};
v.write_needle(&mut n, true, false).unwrap();
let mut back = Needle {
id: NeedleId(9),
..Needle::default()
};
v.read_needle(&mut back).unwrap();
assert_eq!(back.data, b"ninth");
assert_eq!(back.cookie, Cookie(0xabcd));
}
/// A stale index entry is only replaced once the new record commits: a
/// durable write rolled back by a failed fsync leaves the index exactly
/// as it found it.
#[test]
fn test_stale_index_entry_survives_failed_write() {
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let v = make_test_volume(dir);
let mut existing = Needle {
id: NeedleId(2),
cookie: Cookie(0x2222),
data: b"second".to_vec(),
data_size: 6,
..Needle::default()
};
v.write_needle(&mut existing, true, false).unwrap();
let stale_nv = {
let mut io = v.io.write().unwrap();
let nm = io.nm.as_mut().unwrap();
let nv = nm.get(NeedleId(2)).unwrap().unwrap();
nm.put(NeedleId(9), nv.offset, Size(nv.size.0 + 1)).unwrap();
nm.get(NeedleId(9)).unwrap().unwrap()
};
v.fail_next_fsync_for_test(true);
let mut replacement = Needle {
id: NeedleId(9),
cookie: Cookie(0xabcd),
data: b"ninth".to_vec(),
data_size: 5,
..Needle::default()
};
assert!(v.write_needle(&mut replacement, true, true).is_err());
v.fail_next_fsync_for_test(false);
let now = {
let io = v.io.read().unwrap();
io.nm
.as_ref()
.unwrap()
.get(NeedleId(9))
.unwrap()
.expect("a failed write must not drop the index entry it meant to replace")
};
assert_eq!(now.offset, stale_nv.offset);
assert_eq!(now.size, stale_nv.size);
}
// A size disagreeing with the blob's own header indexes the needle at the wrong
// length and, on v3, stamps the append timestamp into the middle of the needle.
#[test]
+2 -2
View File
@@ -1340,11 +1340,11 @@ async fn non_ascii_fid_and_ttl_are_rejected_not_panicked() {
#[tokio::test]
async fn write_queue_upload_returns_same_etag_as_direct_write() {
use seaweed_volume::server::write_queue::WriteQueue;
use seaweed_volume::server::write_queue::{SyncBatchOption, WriteQueue};
let (direct_state, _direct_tmp) = test_state();
let (queued_state, _queued_tmp) = test_state();
let wq = WriteQueue::new(queued_state.clone(), 128);
let wq = WriteQueue::new(queued_state.clone(), 128, SyncBatchOption::default());
let _ = queued_state.write_queue.set(wq);
let compressible = "seaweedfs ".repeat(200).into_bytes();
@@ -34,6 +34,7 @@ struct ListTablesResponse {
pub struct NamespaceClient {
base_url: String,
http: reqwest::Client,
api_key: Option<String>,
}
/// A namespace call that has not answered by now is not going to. Without this
@@ -41,12 +42,23 @@ pub struct NamespaceClient {
/// slot open forever, and the sweep never finishes.
const REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
const MAX_REDIRECT_HOPS: usize = 10;
impl NamespaceClient {
pub fn new(base_url: impl Into<String>) -> Self {
let http = reqwest::Client::builder()
.timeout(REQUEST_TIMEOUT)
.connect_timeout(CONNECT_TIMEOUT)
// reqwest forwards custom headers on a follow, so only a same-origin
// hop is allowed: anywhere else would carry x-api-key to a listener
// the credential was never meant for.
.redirect(reqwest::redirect::Policy::custom(|attempt| {
if may_follow(attempt.previous(), attempt.url()) {
attempt.follow()
} else {
attempt.stop()
}
}))
.build()
// The builder only fails on a bad TLS backend, which would break
// every call anyway; a client with no timeouts is worse than a panic
@@ -58,6 +70,32 @@ impl NamespaceClient {
Self {
base_url: base_url.into().trim_end_matches('/').to_string(),
http,
api_key: None,
}
}
/// The catalog accepts an S3 credential as `x-api-key`; where the worker
/// already holds one for storage it authenticates the namespace calls too.
pub fn with_fallback(mut self, fallback: &crate::dataset::FallbackOptions) -> Self {
if let (Some(access), Some(secret)) = (
fallback.get("aws_access_key_id"),
fallback.get("aws_secret_access_key"),
) {
if !protected_transport(&self.base_url) {
tracing::warn!(
"catalog url {} exposes the x-api-key credential in cleartext",
self.base_url
);
}
self.api_key = Some(format!("{access}:{secret}"));
}
self
}
fn authed(&self, request: reqwest::RequestBuilder) -> reqwest::RequestBuilder {
match &self.api_key {
Some(key) => request.header("x-api-key", key),
None => request,
}
}
@@ -65,8 +103,7 @@ impl NamespaceClient {
pub async fn list_all_tables(&self) -> Result<Vec<String>> {
let url = format!("{}/v1/table", self.base_url);
let response: ListTablesResponse = self
.http
.get(&url)
.authed(self.http.get(&url))
.send()
.await
.context("list tables")?
@@ -80,8 +117,7 @@ impl NamespaceClient {
pub async fn list_namespaces(&self, id: &[String]) -> Result<Vec<String>> {
let url = format!("{}/v1/namespace/{}/list", self.base_url, encode_id(id));
let response: ListNamespacesResponse = self
.http
.get(&url)
.authed(self.http.get(&url))
.send()
.await
.context("list namespaces")?
@@ -98,8 +134,7 @@ impl NamespaceClient {
let url = format!("{}/v1/table/{}/describe", self.base_url, encode_id(id));
let body = serde_json::json!({ "id": id, "vend_credentials": true });
let description: TableDescription = self
.http
.post(&url)
.authed(self.http.post(&url))
.json(&body)
.send()
.await
@@ -111,6 +146,41 @@ impl NamespaceClient {
}
}
/// Without a hop bound a catalog that redirects to itself keeps the request
/// alive until the call timeout, so the custom policy keeps the limit the
/// default one had.
fn may_follow(previous: &[reqwest::Url], next: &reqwest::Url) -> bool {
previous.len() < MAX_REDIRECT_HOPS
&& previous.last().is_some_and(|prev| same_origin(prev, next))
}
/// A redirect stays within an origin when scheme, host, and port all match, so
/// a catalog cannot bounce the api key onto a downgraded or foreign listener.
fn same_origin(from: &reqwest::Url, to: &reqwest::Url) -> bool {
from.scheme() == to.scheme()
&& from.host_str() == to.host_str()
&& from.port_or_known_default() == to.port_or_known_default()
}
/// A catalog url protects the api key when the hop is TLS or never leaves the
/// host; anything else sends it where the network can read it.
fn protected_transport(base_url: &str) -> bool {
let Ok(url) = reqwest::Url::parse(base_url) else {
return false;
};
if url.scheme() == "https" {
return true;
}
match url.host_str() {
Some("localhost") => true,
Some(host) => host
.trim_matches(['[', ']'])
.parse::<std::net::IpAddr>()
.is_ok_and(|ip| ip.is_loopback()),
None => false,
}
}
fn encode_id(id: &[String]) -> String {
if id.is_empty() {
DELIMITER.to_string()
@@ -127,3 +197,56 @@ pub fn parse_id(encoded: &str) -> Vec<String> {
.map(|part| part.to_string())
.collect()
}
#[cfg(test)]
mod tests {
use super::{MAX_REDIRECT_HOPS, may_follow, protected_transport, same_origin};
#[test]
fn may_follow_bounds_hops_within_the_origin() {
let url = |u: &str| reqwest::Url::parse(u).unwrap();
let first = [url("https://c.local/ns")];
let next = url("https://c.local/other");
assert!(may_follow(&first, &next));
assert!(!may_follow(&[], &next));
assert!(!may_follow(&first, &url("http://c.local/other")));
let full: Vec<_> = (0..MAX_REDIRECT_HOPS)
.map(|i| url(&format!("https://c.local/{i}")))
.collect();
assert!(!may_follow(&full, &next));
}
#[test]
fn same_origin_rejects_scheme_and_port_hops() {
for (from, to, expected) in [
("https://c.local/ns", "https://c.local/other", true),
("http://c.local", "http://c.local:80/next", true),
("https://c.local", "http://c.local/next", false),
("https://c.local:8443", "https://c.local/next", false),
("https://c.local", "https://other.local/next", false),
] {
let from = reqwest::Url::parse(from).unwrap();
let to = reqwest::Url::parse(to).unwrap();
assert_eq!(same_origin(&from, &to), expected, "{from} -> {to}");
}
}
#[test]
fn protected_transport_covers_tls_and_loopback() {
for url in [
"https://catalog.internal:8443",
"http://127.0.0.1:8080",
"http://localhost",
"http://[::1]:8080",
] {
assert!(protected_transport(url), "{url} should be protected");
}
for url in [
"http://catalog.internal",
"http://127.0.0.1.evil.test",
"not a url",
] {
assert!(!protected_transport(url), "{url} should warn");
}
}
}
@@ -55,7 +55,7 @@ impl CleanupVersionsHandler {
}
fn client(&self) -> NamespaceClient {
NamespaceClient::new(self.namespace_url.clone())
NamespaceClient::new(self.namespace_url.clone()).with_fallback(&self.fallback)
}
}
@@ -59,7 +59,7 @@ impl CompactHandler {
}
fn client(&self) -> NamespaceClient {
NamespaceClient::new(self.namespace_url.clone())
NamespaceClient::new(self.namespace_url.clone()).with_fallback(&self.fallback)
}
}
@@ -58,7 +58,7 @@ impl OptimizeIndicesHandler {
}
fn client(&self) -> NamespaceClient {
NamespaceClient::new(self.namespace_url.clone())
NamespaceClient::new(self.namespace_url.clone()).with_fallback(&self.fallback)
}
}
+1 -1
View File
@@ -78,7 +78,7 @@ impl SortHandler {
}
fn client(&self) -> NamespaceClient {
NamespaceClient::new(self.namespace_url.clone())
NamespaceClient::new(self.namespace_url.clone()).with_fallback(&self.fallback)
}
}
+1 -1
View File
@@ -35,7 +35,7 @@ impl PreviewProvider for LancePreview {
}
async fn preview(&self, object_id: &[String], row_limit: usize) -> Result<Preview> {
let client = NamespaceClient::new(self.namespace_url.clone());
let client = NamespaceClient::new(self.namespace_url.clone()).with_fallback(&self.fallback);
let table = dataset::open(&client, object_id, &self.fallback).await?;
let total_rows = table.dataset.count_rows(None).await.unwrap_or(0) as i64;
Loaded 100 of 527 files, more files were not shown because too many files have changed in this diff. Show more