mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-20 03:13:53 +01:00
5e003bae7df3121c15f663432dfda1d7f994fe51
16334
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5e003bae7d | docs: regenerate star history chart | ||
|
|
4ac3633f1c |
volume: check the needle id in the streamed-read probe and on blob writes (#12794)
* storage: reject a streamed read whose record id or size differs The header probe before a streamed read returned not-found only when both the record's id and its size differed from the index entry, so a record of another needle with the same size passed and its data was streamed under the requested key until the final checksum check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: refuse a needle blob whose header names another needle WriteNeedleBlob compared only the blob header's size with the size it was asked to index. A blob read from an offset that has since moved to another record of the same size was appended and indexed under the wrong key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): refuse a needle blob whose header names another needle Mirror of the Go check: write_needle_blob_and_index compared only the blob header's size, so a blob of another needle with the same size was appended and indexed under the requested key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): report not-found when a streamed read's record is another needle The meta read ahead of a streamed reply took the id from the record header without comparing it with the requested key. Return not-found when the index entry resolves to another needle, as the Go header probe does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: check the requested id after a meta-only read ReadNeedleMeta overwrites n.Id with the record's header id, so a meta-only read through an index entry that resolves to another needle returned that needle's metadata, and the streamed-read probe that follows compared the record with itself. Return not-found when the header names another needle, as the Rust meta read does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: copy a needle blob to another volume, not under another key WriteNeedleBlob now refuses a blob whose header names another needle, so the round trip writes the blob under its own id into a second volume and checks that storing it under another key is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: compare the streamed header with a deleted entry's record size A deleted index entry stores the negated size, so a read-deleted stream compared the positive header size with a negative one and missed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4a5dbe4c6f |
storage: copy an empty needle written during compaction in makeupDiff (#12796)
* storage: copy an empty needle written during compaction in makeupDiff A size-0 index row is a live empty needle, not a delete. makeupDiff grouped it with tombstones and appended a made-up delete marker, so the compacted .dat lost the record's cookie and held a delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): copy an empty needle written during compaction in makeup_diff A size-0 index row is a live empty needle. makeup_diff wrote a made-up delete marker and a tombstone row for it, so the needle read as not found after the commit. Copy the record like any other live row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: fail makeupDiff when the replayed needle write fails Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): check the compacted empty needle's id and cookie in the test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ba1bfa816c |
admin: show a pruned job's detail from its run record (#12795)
* admin: show a pruned job's detail from its run record Tracking keeps the newest 1000 jobs across all job types and deletes the details of older finished ones, while each job type's run history keeps its last 10 successful and 10 failed runs. A job listed in the run history could therefore open as "job detail not found". Fall back to the run record when the job is no longer tracked. Claude-Session: https://claude.ai/code/session_011UpvafcfUuC2mLxNAXLQJi * admin: show 100% progress for a pruned successful run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: use the newest run record for a job across both outcomes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: keep the run record a pruned job's detail was built from Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: keep looking for a run record past an unreadable history Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: pick a pruned job's newest run record across job types Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
be8ebc2e04 |
storage: build the temporary needle map in CompactByVolumeData (#12788)
* storage: red test for CommitCompact after CompactByVolumeData with a concurrent write or delete Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: build the temporary needle map in CompactByVolumeData Fixes #12783 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: require a successful commit in the CompactByVolumeData commit test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bfbeed61a7 |
storage: read only the header and trailer in the append-time search (#12791)
BinarySearchByAppendAtNs, its left/right probes and ResumeByLastEntry read every probed record through readNeedleAt, which parsed the whole body and verified the data checksum, although they use only the id, cookie, size and append time. One record with damaged data on the search path, even a needle deleted long ago, failed the search and with it the tail sender, a timestamp-based incremental copy and the volume.move catch-up. readNeedleAt now reads the header and the trailer and skips the data. The data checksum is still verified where the data is read, including the tail receiver for every record it gets. The Rust volume server already parses only the metadata on this path and does not verify the data checksum. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
05427e8108 |
storage: hold the data-file read lock across the append-time search (#12789)
* storage: red test for the append-time search racing a vacuum commit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: hold the data-file read lock across the append-time search Fixes #12785 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: Version() writes the superblock version only when it changes Version() stored the .vif version into SuperBlock.Version on every call, so each call was a write that raced with readers of the superblock that do not take superBlockAccessLock, such as the append-time search and the compaction setup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: harden the append-time search race test; add a lock-hold budget test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: Linux-only lock budget; stop before reading search failures Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
f7323f2b63 |
filer: optional HTTP Basic Auth on the filer HTTP ports (#12778)
* security: add a constant-time HTTP Basic Auth check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: require HTTP Basic Auth on the HTTP ports when configured With filer.basic_auth.password set in security.toml (or WEED_FILER_BASIC_AUTH_PASSWORD), a request on the filer HTTP ports, TUS included, must carry those credentials or a valid filer JWT, and an access level with no JWT signing key is no longer open. Valid Basic credentials grant read and write. 401 responses then carry a Basic challenge so browsers prompt. The credentials reload on SIGHUP with the signing keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: document filer.basic_auth in security.toml Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: enable reloaded Basic Auth before swapping the JWT keys Reload cleared the old signing keys before storing the new Basic credentials, so a switch from JWT to Basic Auth briefly left both off and admitted anonymous requests. Store new credentials first and clear old ones last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: refuse Basic credentials on cross-origin browser requests Browsers resend cached Basic credentials on requests other websites trigger, such as a form POST with mv.from, and the default cors.allowed_origins "*" lets them through. Accept Basic credentials only on requests without an Origin, from the filer's own host, or from an origin explicitly listed in cors.allowed_origins. A JWT is still accepted from any origin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: recheck Basic Auth after reading an empty signing key A request that loaded Basic Auth as off, then read the signing key after a JWT-to-Basic Reload had cleared it, was let through anonymously. Open access now also needs Basic Auth to be off after the key read; with Reload enabling the new credential before disabling the old one, the two loads cannot both miss a credential that was on throughout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * security: compare Basic Auth credentials as MACs under a per-process key A plain SHA-256 of the password is what password-hash scanners flag; keying the digests with a random per-process HMAC key keeps the length-independent constant-time compare without holding one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util: read a configuration file without merging into the shared viper LoadConfiguration merges each read into the shared viper, so a key removed from the file keeps its old value after a reload. ReadConfiguration reads the file into a fresh viper with the same search path and WEED_ environment overrides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: turn Basic Auth off when security.toml no longer sets it The shared viper merges each SIGHUP reload into what it held, so removing [filer.basic_auth] kept the old password valid. Read the credentials from security.toml alone at startup and on reload; an unreadable file keeps the current setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep Basic Auth when security.toml cannot be read A malformed file at startup dropped a WEED_FILER_BASIC_AUTH_PASSWORD from the environment, and viper reports an unsearchable config directory as a missing file, which turned Basic Auth off on reload. ReadConfiguration now always returns the environment overrides and reports whether the file was read; a reload that could not read it keeps the current credentials. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep reloading Basic Auth from the security.toml first read A search directory that turned unsearchable made viper fall through to a lower-priority security.toml, and a reload from a file without [filer.basic_auth] turned Basic Auth off. ReadConfiguration now takes the file to read, and the filer reads the file it first found on every reload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f5e091cf91 |
wdclient, s3: return an embedded S3 gateway to its local filer after failover (#12772)
* wdclient: sort newly discovered filers and log every current-filer switch Discovery appended new filers in map iteration order, so after the current filer left, the fallback at index 0 was a random filer. Append them sorted. Route every change of the current filer through one helper that logs it at V(0) with its reason (failover, return_home, discovery), so an operator can see which filer a gateway uses without new metrics. * wdclient: prefer a home filer and return to it after failover with backoff A FilerClient with FilerClientOption.HomeFiler set goes back to that filer once its circuit breaker allows a re-check, instead of staying on whichever peer it failed over to. Leaving home again soon after a return doubles the wait before the next return, up to 5 minutes; a return that holds for 10 minutes resets it. Volume lookups and discovery follow the same preference. * s3: embedded gateway uses its local filer as home filer weed filer -s3, weed server -s3 and weed mini now build the gateway's filer address in the form the filer registers in the master (ip:port.grpcPort), with the gRPC port default applied, and pass it to the gateway as its home filer, so the gateway returns to its local filer after any failover. Standalone weed s3 has no home filer and behaves as before. Refs #12768, follow-up to #12771 * wdclient: switch the current filer only from the filer a request started on A volume lookup that started on a peer and finished after the client had returned home moved it back to the peer and counted that as leaving home, doubling the return backoff although home never failed. Concurrent failovers could also each count as a separate departure. Failovers now name the filer they started on and take effect only while it is still current, via compare-and-swap, so only the winning switch logs and feeds the backoff. S3's failover walk uses the same rule. * wdclient: switch the current filer and record the return backoff in one step The index moved by compare-and-swap before the switch was recorded, and both ran under the shared read lock. A request reading the current filer in that gap saw the peer with the old eligibility time and, with home's breaker still closed after one failure, switched straight back home. The eligibility check, the move and the bookkeeping now all run under homeMu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: discovery falls back past a home that is still backing off When discovery removed the current filer and home was not yet eligible, the client took index 0, which is home whenever it is listed first. With home's breaker still closed after a single failure, requests went back to the filer they had just left. Take the next filer instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: start return timing on every switch back to the home filer Only a return_home switch stamped lastReturnHome. Discovery or a peer's failover could also put the client back on home, and a failure soon after such a return kept the backoff at its base value instead of doubling it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Konstantin Lebedev <whitefox@mayflower.work> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c9f3a02b94 |
filer: recover a leveldb store whose table a power loss damaged (#12780)
* filer: recover leveldb without rebuilding broken tables leveldb.Recover rebuilds a table with damaged blocks using the user comparer and filter on internal keys. The shortened index keys it writes are not valid internal keys, so a Get on the rebuilt table panics, and the bloom filter it writes misses every lookup. Recover with a comparer that keeps full index keys (same name, so the manifest still matches) and no filter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: recover a leveldb store when a compaction finds a corrupted table When a compaction reads a damaged table block, as a power loss can leave one, goleveldb latches the corruption error and returns it for every later write until the database is reopened. A plain reopen loads the same table and the next compaction fails again, so the store kept rejecting inserts even across restarts. Treat a corruption error from a write like a storage error, and reopen through leveldb.Recover, which rebuilds the table without the damaged blocks, then retry the write. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: rebuild damaged leveldb tables before Recover leveldb.Recover's own table rebuild orders internal keys with the user comparer, so a damaged table holding several versions of one key fails with "keys are not in increasing order" and the recovery gives up. Overriding the comparer cannot fix that, because the same comparer also serves the recovered database's user keys. Rebuild every table that has corrupted blocks first, using an internal key comparer and full index keys, then let Recover rebuild the manifest from tables it no longer needs to rewrite. Seed the test tables from a journal flush so they keep several versions of a key, and check keys the write test does not rewrite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: leave unreadable leveldb tables for Recover to drop table.NewReader reports a truncated table or a damaged footer through the reader, not its error, so rebuildDamagedTables never stopped on one, but it rewrote it as an empty table first. Skip tables with nothing readable and let Recover drop them, and test an unreadable table next to a healthy one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: let StrictRecovery drop damaged leveldb tables With StrictRecovery set, leveldb.Recover drops a table that has corrupted blocks. Salvaging the readable blocks first would hide the damage from it, so skip the rebuild under that option. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: hold the leveldb storage lock while rebuilding tables The table rebuild ran after the failed open released the storage lock and before Recover took it, so it could rewrite the tables of a database another caller had open on the same storage. Take the lock for the rebuild; Recover takes it again afterwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6bdcd17ed4 |
master: reject an assign on a follower as Unavailable (#12779)
* master: reject an assign on a follower as Unavailable Assign on a master that is not the raft leader returned raft.NotLeaderError as a plain error, which reaches gRPC clients as codes.Unknown. operation.Assign retries only Unavailable, ResourceExhausted, Canceled and DeadlineExceeded, so every assign made during a leader election failed at once instead of waiting for the new leader. Return it as codes.Unavailable, like the warming-up status in the same file, so clients retry within their 30 s assign budget. The message is unchanged, so callers that match "Not current leader" still work. StreamAssign now answers such a request in-band with the error message instead of ending the stream, as it already does for warm-up. TestAssignOnFollowerIsUnavailable covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: keep the cached channel when the peer answers Unavailable with details gRPC's own Unavailable (dial failure, closing transport) never carries details, so one that does came from the peer over a working channel. Closing the shared ClientConn for it cancels every other in-flight RPC on it with "the client connection is closing". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * master: tag the follower's assign rejection with a NOT_LEADER ErrorInfo The detail marks it as the master's answer, so clients retry it without tearing down their cached channel to that master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
729b872f01 |
test: let the merge-read FUSE test wait out the master's vacuum drain (#12777)
* test: let the merge-read vacuum outlast the master's write drain Before compacting a volume the master waits up to 30 s for writes it assigned to that volume to drain. The test cut each /vol/vacuum call at 30 s and gave the vacuum 60 s, so one or two slow drains failed it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: retry the merge-read vacuum until the master runs one A 409 means another vacuum holds the master's lock; check the chunks only after a vacuum this test asked for has finished. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
972f85159e |
util/http: probe dead replicas in the background and key no-answer dials by machine (#12776)
* util/http: probe an unanswering server in the background, not on a read Once the retry interval passed, the first read put the dead server first and waited out the full dial timeout before trying a live replica. Keep it last and dial it from a background probe with a short timeout instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: try every server of a machine that does not answer last Unreachability was kept per host:port, so a powered-off node running one volume server per disk cost one stalled read per server before each was known dead. A dial that times out or finds no route now marks the whole machine; a refused connection still marks only that port and shows the machine is up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: probe an unanswering server with an HTTP request A bare TCP connect cleared servers that accept connections but never send response headers, and could not dial an address without an explicit port. Probe with a HEAD request through the global client instead, and have the tests wait for the probe to settle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: keep a timed-out port last after a sibling port answers A dial timeout stored only the machine verdict, so a success on another port of that machine cleared it and the port that timed out went back to the front. Record the port as well as the machine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: classify refused and unroutable dials with Winsock errnos on Windows Windows reports these as WSAECONNREFUSED, WSAEHOSTUNREACH and friends, which errors.Is does not match against the syscall constants. Test the refusal with a real dial to a closed port. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: wait for background probes through a WaitGroup in tests A refused probe can finish before the test reads the claimed record, so inferring completion from the record's timestamp could wait out the deadline on a probe that already settled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: key machine verdicts apart from hosts without a port A host without an explicit port is also its machine name, so a sibling port answering cleared the timed-out host along with the machine verdict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: group a bracketed port-less IPv6 host with its machine net.SplitHostPort rejects "[2001:db8::9]", so the machine key kept the brackets and missed the verdict stored for "[2001:db8::9]:9501". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ed7324611c |
filer: metrics for metadata subscription delivery (lag, holds, ring fall-offs, claim and watermark ages) (#12773)
* filer: metrics for metadata subscription delivery (lag, holds, ring fall-offs, claim and watermark ages) * filer: measure subscribe delivery lag after the real send using the mutation stamp, and record a watermark hold once per hold * filer: skip empty progress markers when sampling subscribe delivery lag * filer: assert subscribe delivery lag is sampled only after the stream send succeeds * grafana: panels for the metadata subscription delivery metrics Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: record a watermark hold when the read passes the held entry, not when the next hold starts * filer: use a monotonic clock for the subscribe hold duration --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
05a1463855 |
wdclient: keep the embedded S3 gateway on its local filer across discovery (#12771)
* filer: keep an explicit gRPC port in the embedded gateways' filer address runFiler handed the embedded S3/WebDAV/IAM/SFTP gateways NewServerAddress(...).String(), and String() drops the .grpcPort part. With a non-default -port.grpc the gateways then dialed port+10000 and did not recognize their own filer in the master's list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: match discovered filers by gRPC address A filer listed as host:port and the same filer discovered as host:port.grpcPort were compared as raw strings, so a refresh pruned the configured filer, added every discovered filer as new in map order, and reset the current filer to whichever came first; a pushed join added it a second time. Key filers by their gRPC address: a survivor keeps its health and current position and takes the discovered address form, and pushed joins and leaves match the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c1c95e127f |
filer: scoped read-only tokens for metadata subscriptions (#12767)
* security: scoped read-only token for filer metadata subscriptions A token signed with jwt.filer_signing.subscribe.key names the subscribe methods and the path prefixes its holder may see. mTLS peers and admin tokens stay unrestricted. Internal subscribers attach an admin token when both keys are configured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: enforce subscription token scope on metadata streams With jwt.filer_signing.subscribe.key set, SubscribeMetadata and SubscribeLocalMetadata authorize the caller first. A scoped caller gets a stream that sends only events whose paths are all under its prefixes, filters batches, and never carries log file refs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: send an admin token on internal metadata subscriptions Peer filers, metadata followers and the lifecycle reader attach an admin token when the subscribe key is configured, so they keep the full stream without mTLS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: add fs.meta.token to mint metadata subscription tokens Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: end scoped metadata subscriptions at token expiry The scope now carries the token's exp. The scoped stream's context ends at that time and the call returns Unauthenticated, so an open stream no longer outlives its token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer.sync: sign metadata subscriptions with the source cluster's key MetadataFollowOption takes a FilerJwtProvider; filer.sync passes the one loaded from -a.security or -b.security, so clusters with different admin keys both accept the subscription. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: report fs.meta.token flag errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: give scoped metadata subscribers their own client id A scoped caller could reuse another subscriber's ClientId with a higher epoch and replace that registration, disconnecting it. Scoped streams now register under a server-chosen id that no other subscriber holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * security: accept only HS256 metadata subscription tokens Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * security: test HS384 metadata subscription token rejection Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: never let a scoped subscriber replace another registration The server-chosen id can be taken by another subscriber between choosing and registering it. Registering scoped streams with the lowest epoch makes addClient refuse them rather than replace the other registration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: send position markers for events a scoped token withholds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test a scoped stream withholding a whole event batch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: filter SendMsg on scoped metadata streams The scoped wrappers only overrode Send, so a SendMsg on the embedded stream would reach the client unfiltered. Route it through the same filter and refuse any other message type. Ported from #12790. Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test a scoped subscription through the real subscribe loop Run SubscribeLocalMetadata with a scoped token over a flushed backlog, the in-memory tail and live events: only in-scope events arrive, sibling, "..", encoded and cross-boundary rename paths are dropped, and no log file references are sent although the client asks for them. Without the subscribe key every event still arrives. Ported from #12790. Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test that the HTTP API refuses metadata subscription tokens GET, HEAD, PUT, POST and DELETE all refuse a subscription token, also when the subscribe key equals the read or write key. Ported from #12790. Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com> |
||
|
|
668ca45029 |
s3: ListParts returns each part number once, its latest attempt (#12770)
* s3: list each part number once, its latest attempt Every UploadPart writes its own NNNN_<uuid>.part entry, so a part number uploaded twice listed twice, superseded ETag included, and a max-parts page could hold only the stale attempt. ListParts now groups the entries by part number, reports the attempt sortEntriesByLatestChunk ranks first (the same order completion uses), and pages over distinct part numbers, reading past a page boundary that splits one part number's attempts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: test ListParts over repeated part attempts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: rank an empty part attempt by commit stamp or mtime An empty UploadPart body stores no chunks, so sortEntriesByLatestChunk gave it time zero and an empty re-upload always lost to the attempt it replaced. Attempts now rank by the filer's commit stamp when both carry one, else by the later of the first chunk's write time and the entry mtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: keep only the newest attempt per part while listing parts listPartAttempts held every attempt of a part number until the page was built, so a heavily retried part grew one ListParts request's memory past what max-parts bounds. It now compares each entry against the newest seen for its part number with compareLatestPartAttempt, the comparator sortEntriesByLatestChunk sorts with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: order part attempts by commit stamp, then attempt time Comparing commit stamps only when both attempts carried one made the order non-transitive across stamped and unstamped attempts, so listing and completion could pick different attempts depending on input order. The commit stamp now leads (unstamped entries predate the field) and the attempt time breaks ties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: list part 10000 after the four-digit part numbers Part names sort as text, so 10000_<uuid>.part lands between 1000.part and 1000_<uuid>.part. A page could end on part 10000 and the next marker then skipped parts 1000-9999, and a legacy 1000.part could be reported before the newer attempt of part 1000 was read. List the four-digit part numbers first and part 10000 in its own prefix listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: test ListParts paging across part 10000 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
dcbea0626f |
test/s3: patch s3-fast-list's data map race in the listing study build (#12766)
test/s3: build s3-fast-list with its data map draining the last page The data map task reads the channel and then the list-task state. When the list task sends its last page and completes between the two reads, the map is dumped without that page and the tool exits 0, so a one-page prefix comes back empty. Read the state before the channel. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
de4075660f |
telemetry: refresh the server module's go.mod so the integration test builds (#12765)
* telemetry: refresh the server module to the root module's dependency versions The server module replaces seaweedfs with ../.., so root dependency bumps raised prometheus/common, prometheus/procfs and x/sys past the versions its go.mod listed, and go build refused to run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the telemetry integration test on root go.mod changes The telemetry server module builds against the root module through a replace directive, so a root dependency bump can leave its go.mod stale without touching telemetry/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
652133a28a |
telemetry: keep the newest release in the versions chart (#12764)
The versions chart keeps the 8 versions with the most clusters on the last day and folds the rest into "other". A new release starts small, so 4.54 (282 clusters) was folded in while 4.53 shrank as clusters upgraded, which showed as a dip across the named versions. The newest plain release now always keeps its own band. Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e012fca9a2 |
mount: keep lookups from stalling behind directory cache rebuilds (#12754)
* mount: release the meta cache lock between batches of a rebuild wipe deleteFolderChildrenForRebuild held mc's write lock while it listed every cached child of the directory and deleted them. Every metadata read in the mount (FindEntry, IsNameFresh, isOversized) takes the read lock, and a waiting writer blocks new readers, so while a large directory was wiped, lookups and getattrs in unrelated directories queued behind the scan. The scan gets slower as leveldb accumulates the deletion markers of earlier wipes of the same directory, which happens quickly once lookups rebuild expired directories. Wipe rebuildWipeBatchSize (1024) children per lock hold and drop the lock between batches. A child is judged pinned under the same lock that deletes it, so a local-only create is kept whether it was pinned before the wipe started or between batches. The listing that refills the directory already ran outside the lock, so batching adds no interleaving the build did not allow. Add LevelDBStore.BatchDeleteEntries, the counterpart of BatchInsertEntries. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: bound how long a lookup waits on the directory rebuild it starts Since #11268 a lookup whose parent's cache expired rebuilds the parent and every expired ancestor up to the mount root before it answers, and every lookup beneath them joins the same builds. With a slow listing (a large directory, a busy filer, or the meta cache lock held by another rebuild) a plain stat, an ls -l, or the parent touch after mkdir waited for all of it. Wait at most lookupRebuildTimeout (2s), then fall back to one LookupEntry RPC. Since #12206 the shared build does not run on the requester's context, so it keeps going after the waiter gives up and later lookups find the directory cached. A timed-out wait records a rebuild attempt, so lookups during the cooldown do not queue on a build that is already known to be slow. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: wait for the rebuild's listing in TestLookupDoesNotWaitOutSlowRebuild The build runs on its own goroutine, so the lookup that stopped waiting for it can return before the build's ListEntries reaches the filer. The test read `started` without waiting right after the lookup returned, which relied on an ordering nothing guarantees, and the ListEntries count taken afterwards assumed the same. Wait for the listing to arrive, bounded by the test's deadline. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: keep children inserted while a rebuild wipe runs A deferred create inserts its placeholder before Create registers the handle that pins it. The single-lock wipe made such an insert wait until it finished, so the placeholder survived; a batch that runs in that gap judged it unpinned and dropped it, and the rebuilt listing then omitted the file until its flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drop a cached path that an event vacates during a rebuild wipe A build replays only the events later than its listing's snapshot and leaves the earlier ones to the listing, which starts after the wipe. With the wipe in batches a file's whole life fits inside it: created, flushed, then unlinked or renamed away. Both events are earlier than the snapshot and are skipped, the listing has no such file, and the placeholder the wipe kept stays in the published directory until the next rebuild. While a directory's wipe runs, an event buffered for it now takes the path it vacates out of the store at once, fenced by versions like any other event. After the wipe the store holds what the listing put there, and a removal earlier than the snapshot may still arrive, so from then on the event is left to the replay as before. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: check which file a removal took away during a rebuild wipe Two things the removal applied during a rebuild wipe got wrong. A placeholder is unversioned, so the version gate lets every removal through. One that arrived late, for an earlier file of the same name, took out the placeholder of a file created since. The entry the event took away is now compared with the cached one, by inode or, with no inode to compare, by creation time. An event that only names the file cannot tell and is taken at its word. The entry was deleted without OP=MV, so taking one name of a hard-linked file out of the cache also took one off the count its other names share. A replayed event that vacates a path already deletes with OP=MV. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: keep a removal the rebuild wipe declined out of the replay The wipe declines a removal when what is cached there is newer than the event or is another file than the one it took away. The event stayed buffered anyway. An unversioned listing clears the children's version records and replays every buffered event, so the declined removal then deleted the entry the wipe kept and the directory was published without it. A declined removal now buffers only the name it puts in place, so a rename still lands on replay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check a declined removal again when the build replays it The previous commit dropped the removal half of an event the wipe declined, judged by the file cached when the event arrived. Earlier buffered events can replace that file during the replay: a listing attempt that failed leaves its events buffered, so the retry's wipe may see the first attempt's file while the replay puts a later one in place first. The dropped delete then left that later file published. The wipe now records which removals it declined, and the replay checks each against the entry it finds at that point, replaying only the name the event puts in place when that entry is still another file. A removal declined by version is replayed as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a declined removal's invalidation off the placeholder's handle After the replay the build invalidates every buffered event again, raised to the snapshot, and it still sent the original of a removal the replay rejected. A late delete of an earlier file then reached the handle of the placeholder at that name: the placeholder is unversioned, so the raised timestamp got past it and the handle was marked deleted. The replay's decision on a declined removal now replaces the event in the buffer, and the invalidations use what was replayed. It is made before the snapshot check, so a removal the listing covered is judged the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep an event's signatures on the halves a rename is split into A declined removal replays as its rename's create half, and a rename across a building directory is split in two. Both dropped the event's filer signatures, so a handle stamped by that filer no longer fenced the replayed invalidation by timestamp and could take older listed contents over newer ones it had read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drain invalidations around the declined-removal build check Invalidations run on a background worker, so the baseline could miss the removal's own and the final read could run before the build's arrived. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: MapleWhiteZZZ <18653045+MapleWhiteZZZ@users.noreply.github.com> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
3159b8408c |
filer, volume: fsync chunk deletes under an fsync path rule (#12759)
* volume: fsync a batch delete that asks for it BatchDeleteRequest gets an fsync field, and the Go and Rust volume servers pass it to the needle delete, which already orders the tombstone, .dat sync, publish and .idx sync for a durable delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: pass an fsync flag to batch deletes DeleteFileIdsWithLookupVolumeId and DeleteFileIdsAtOneVolumeServer take an fsync argument and send it in the BatchDeleteRequest. Every caller passes false for now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: fsync chunk deletes under an fsync path rule DeleteChunks passes the path rule's fsync flag to the deletion queue. The flag is kept with the pending record until the delete is confirmed, so retries and re-queues keep it, and the batch and retry loops send marked ids in a durable BatchDelete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep fsync marks in the deletion ledger Marked ids go into the trailing parts of a manifest ledger, counted by fsyncParts. Older filers read those parts as plain ids and still delete them; a reload or a claimed foreign ledger restores the marks. A ledger without marks keeps the single array form. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: record a queued deletion before the delete loop can see it queueDeletions published the ids to the hot queue before taking the ledger lock, so a concurrent flush could delete an id before its fsync mark landed and then forget the mark with the record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: fsync a folder delete's chunks by each child's path rule A recursive delete collected its file children's chunks and resolved the rule once for the folder, which misses a trailing-slash rule on the folder itself and file-prefix rules. Each file child's own rule now decides fsync; the folder rule still decides chunk deletion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep a newer fsync request when a plain delete finishes An id re-queued with fsync while a plain delete of it was in flight lost its mark when that delete succeeded, so the queued copy also ran without fsync. A plain attempt now leaves a marked record for its queued copy; a durable delete of an already-deleted needle still syncs the tombstone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: delete a folder's plain child chunks without fsync Children whose own rule turns fsync off were still deleted durably when the folder's rule asked for fsync. Only the children marked by their own rule now go out with fsync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: retry a durable chunk delete the volume server failed A volume server rolls back a durable delete it cannot sync and answers with a 5xx carrying the I/O error text, which the classifier took as permanent, so the record and its fsync mark were dropped while the needle stayed live. A 5xx answer to a durable delete is now retryable, within the usual attempt limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f88751c3ff |
s3api: evaluate request condition keys for STS sessions (#12762)
* s3api: build request condition keys in one helper Split the keys ExtractConditionValuesFromRequest reads from the request itself into RequestConditionValues, keeping aws:SourceIp and the bucket-listing keys in the method. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: evaluate request condition keys for STS sessions The IAM integration context for STS and OIDC sessions carried only the source IP, legacy user agent/referer keys and claims, so role and session policy conditions on s3:x-amz-*, aws:SecureTransport, aws:CurrentTime, s3:RequestObjectTag/* and the other request keys never matched: a Deny on them never fired, and a negated Deny always did. Add the same request keys the static identity path evaluates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: report SigV4 presigned requests as REST-QUERY-STRING s3:authType recognized only SigV2 query authentication, so a policy condition on REST-QUERY-STRING never matched a SigV4 presigned URL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2089abecf3 |
ci: cache Snowflake suite Maven deps and fall back to a Central mirror (#12763)
* ci: cache the Snowflake suite's Maven dependencies under their own key setup-java's maven cache key hashes the repository pom.xml files, which other Java workflows share. The Snowflake job restored that cache, which lacks the suite's plugins and dependencies, and never saved its own because the key already existed. Every run downloaded the whole suite dependency tree from Maven Central and failed when Central answered 429. Key the cache on run.sh, which pins the suite revision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: fall back to Google's Maven Central mirror in the Snowflake job A cold cache still resolves about 36 MB of plugins and dependencies, and Maven gives up after three 429 retries within 30 s. Add the mirror as a settings-profile repository; Maven tries it first and moves on to Central for any artifact the mirror fails to serve. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6d0a6e0935 |
filer: mint read_auth for the data chunks behind manifest chunks (#12756)
* filer: mint read_auth for the data chunks behind manifest chunks * filer: cover every chunk behind manifests in read_auth (data + nested), minted after resolution, and skip when read signing is off * filer: pass the master client as manifest cache invalidator, and move the read_auth helpers below populateBatchReadAuth for the enterprise merge |
||
|
|
9b94f00325 |
test: mount pinned Iceberg jars into the Spark catalog tests (#12757)
* testutil: fetch pinned Maven Central jars with a mirror fallback Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: mount pinned Iceberg jars into the Spark catalog containers Each test started a new Spark container that resolved the Iceberg runtime and AWS bundle through Ivy from Maven Central, so a run downloaded them six times and a single refused request left Spark retrying until the package timed out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: cache the Iceberg jars for the Spark catalog tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * testutil: test the verified jar download Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: bound Maven jar downloads by one deadline shared across retries Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: give each Maven mirror attempt its own timeout within the download budget Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d2a0002869 |
s3api: fix listing regressions from argument validation (empty continuation-token, V1 Prefix encoding) (#12760)
* s3api: accept an empty ListObjectsV2 continuation-token An empty continuation-token lists from the start and is echoed back empty, as S3 does; rejecting it broke s3-tests test_bucket_listv2_continuationtoken_empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: leave the ListObjects V1 Prefix unencoded under encoding-type=url S3 encodes the V1 Delimiter, markers and keys but echoes Prefix as sent, and botocore, which always asks for url encoding, decodes the V1 Prefix only for V2. Encoding it broke s3-tests test_bucket_list_prefix_unreadable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: use a codespell-clean key in the versions encoding test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: URL-encode a V1 Prefix that XML cannot carry Leaving the V1 Prefix raw under encoding-type=url turned a control character such as \x01 into U+FFFD in the response. Such a prefix is URL-encoded again; prefixes that are valid XML text, including '\n', stay as sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c01e87689e | docs: regenerate star history chart | ||
|
|
7b4bdb77bb |
master: export writable volumes per disk and reset leader-only layout gauges on followers (#12751)
* master: export writable volumes per disk for each volume layout The layout gauges only report how many volumes of a collection take writes, not where they sit, so two writable volumes sharing one disk cannot be seen or alerted on. The leader now exports writable_volumes_per_disk with the layout labels plus data center, rack, node and disk id, counting the replicas of the layout's writable volumes on each disk (state=writable) and those of them that are crowded (state=crowded). Series for disks that no longer hold a writable volume of the layout, and for layouts that are gone, are deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * master: reset the leader-only layout gauges on followers The layout writable, crowded, under-replicated and per-disk gauges are only set by the leader, so a master that lost leadership kept exporting the values of its last term indefinitely, and dashboards that aggregate over masters could show numbers days old. A master that is not the leader now clears them on each periodic pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * grafana: chart disks holding several writable volumes of a collection Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * grafana: keep disks of different clusters apart in the per-disk panel Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1fd5008d63 |
s3api: listing argument validation and encoding-type=url for versions, prefix and delimiter (#12753)
* s3api: reject an empty ListObjectsV2 continuation-token
AWS answers continuation-token= with 400 InvalidArgument ("The continuation
token provided is incorrect"); SeaweedFS listed from the start of the bucket.
No token this server issues is empty, and raw-key and base64: tokens keep
resuming as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: reject an encoding-type other than url on listings
AWS answers any encoding-type other than url with 400 InvalidArgument
("Invalid Encoding Method specified in Request") on ListObjects,
ListObjectsV2, ListObjectVersions, ListMultipartUploads and ListParts;
SeaweedFS ignored the value and listed unencoded keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: apply encoding-type=url to ListObjectVersions
The handler accepted encoding-type=url but returned raw keys. It now
URL-encodes Key in Version and DeleteMarker entries, CommonPrefixes, Prefix,
Delimiter, KeyMarker and NextKeyMarker with the urlPathEscape form ListObjects
uses, and echoes EncodingType.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: URL-encode Prefix and Delimiter in ListObjects responses
Under encoding-type=url, ListObjects and ListObjectsV2 encoded keys,
CommonPrefixes, markers and StartAfter but echoed Prefix and Delimiter raw,
so a client decoding them once corrupted a prefix holding '%' or '+'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
646549eee1 |
volume growth: spread writable volumes of a layout over servers and disks (#12749)
* volume growth: spread writable volumes of a layout over servers and disks Volume growth picked every server, and the volume server picked every disk, independently: the server weighted by free slots, the disk by the lowest total volume count. A grow request therefore stacks several writable volumes of one collection on some disks while most disks get none (a Poisson spread: with 160 replicas on 479 disks about 20 disks get two or three), and their writes share those disks. Data centers and racks are still picked weighted by free slots. Inside a rack the server is now one of those holding the fewest writable volumes of the layout per disk, weighted by free slots among them; this covers the main server, same-rack replicas and replicas walked into other racks and data centers. On the volume server the disk is the one with the fewest writable volumes of the collection, then the lowest volume count as before. A grow request registers each volume before placing the next, so a request smaller than the number of disks lands on distinct disks. Fixes #12746 * volume growth: scan the layout load once per grow request Placing each volume rescanned all writable volumes of the layout under its read lock. RecordAssign takes that lock exclusively on every assign, so with thousands of writables (0.8 ms for 10000) every volume of a grow request stalled assigns once more. GrowByCountAndType now scans once and findAndGrow adds every volume it creates to the counts. Per volume, placement costs about 130-160 us on 480 servers whatever the number of writables, against 120 us before the spread change; the single scan is 0.17 ms for 2000 writables and 0.8 ms for 10000. * volume growth: address review of the spread placement - Count only disks that report volume slots in a server's load. An unavailable disk, or one low on space with no volumes, is still listed in DiskTags with no slots; counting it understated the load of a server with failed disks and piled volumes onto its remaining ones. - On the volume server, judge a volume full by its .dat size, which the master compares with the limit, not by ContentSize. - A server that refuses an allocation ranks last for the rest of the grow request, and the request retries the volume up to 3 times. The least loaded server is picked first deterministically, so without this one server that cannot fit a preallocated volume stopped the layout from growing. - Scan the layout load after warmup, so a grow request issued right after a leader change sees the writable volumes the returning servers report. * volume growth: keep the request-local placement load ahead of the request fields The JSON request fields stay together at the end of VolumeGrowOption, and the comment now says the load lives for the whole grow request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): place a new volume on the disk with the fewest writable volumes of its collection Store::add_volume picked the disk with the most free slots, so the Rust volume server ignored the collection spread rule the Go server now applies. find_free_location ranks disks by the writable volumes of the collection first (not read-only, .dat below the volume size limit), then by free slots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume growth: leave a refusing server out of every placement tier A server that refused an allocation only ranked last among the servers of its rack, so the weighted data center and rack picks could still choose its rack on every retry while other racks had room. It now offers no free slots for the rest of the grow request, which takes its slots out of the rack and data center weights too. If only refusing servers are left, they are tried again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume growth: require the full same-rack count only in the main rack The data center check asked every rack for SameRackCount+1 free servers, but only the main rack holds the same-rack copies; each other rack takes one. With a refused server left out of its rack, the check rejected a feasible placement, the retry put the refused server back, and it was picked again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Konstantin Lebedev <whitefox@mayflower.work> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9cd183b04f |
volume: fsync a needle delete that asks for it (#12736)
* volume: fsync a needle delete that asks for it A delete now takes fsync like a write: the tombstone is synced before the needle map drops the needle, and the .idx row before the delete is acknowledged. A failed .dat sync takes the tombstone back off the .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: honor fsync=true on DELETE and forward it to replicas Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): fsync a needle delete that asks for it Mirrors the Go volume server: DELETE with fsync=true flushes the tombstone before the index drops the needle and the .idx before the ack, and a failed .dat flush takes the tombstone back off the .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: let a durable delete through on a full volume A queued delete hit the batch worker's volume size check, which a tombstone append is exempt from inline, so fsync=true deletes failed once the volume reached its hard limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: sync a durable delete of an already-deleted needle An earlier delete without fsync may have dropped the needle with its tombstone still unsynced, so the inline path syncs before acking anyway, as the batch worker already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: refuse a batch request that races the worker stopping stopWorker can close the batch channel between startWorker handing it out and the send, which panicked. The request now falls back to the inline path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): sync a durable delete of an already-deleted needle Mirrors the Go volume server: an earlier delete without fsync may have dropped the needle with its tombstone still unsynced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): run a durable delete on a blocking thread Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: keep syncDelete's signature for existing callers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: refuse a write a delete cancelled before it appends A write cancelled while still queued behind the volume locks used to append, publish and then tombstone itself without a sync, after its delete may already have been acknowledged as durable. A crash could keep the row and lose the tombstone. Refuse it before the append. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): refuse a write a delete cancelled before it appends Mirrors the Go change: a write cancelled while still queued behind the volume locks no longer appends, publishes and tombstones itself without a sync after its delete may already be durable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): drop the write guards for a durable delete's flushes The durable delete held the write lease and the io write guard across both fsyncs, so reads of the volume waited on them. Append and publish the tombstone under the guards and flush with only the io read side, as write_synced_run and the Go path already do. do_delete_request keeps its signature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stop taking writes when a durable delete fails to index The needle map drops the needle before it appends the .idx row, so after a failed append a durable retry found nothing to delete and acked, and a later write could bury the unindexed tombstone so a reload brought the needle back. Fence the volume the way a failed durable write does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): stop taking writes when a durable delete fails to index Rust counterpart of the Go change: the needle map drops the needle before it appends the .idx row, so a failed append left a durable retry nothing to delete, and a later write could bury the unindexed tombstone. publish_delete_request now fences the volume for a durable delete, as publish_write does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stop taking writes when any delete fails to index A non-durable delete whose .idx append failed left the volume writable with the needle already dropped from the map, so a later durable retry found nothing to delete and acknowledged without the missing row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): stop taking writes when any delete fails to index Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: take the fsync flag on the existing delete functions DeleteVolumeNeedle, deleteNeedle2 and syncDelete take fsync instead of gaining *WithFsync variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): take the fsync flag on the existing delete functions delete_volume_needle and delete_needle take fsync instead of gaining *_with_fsync variants; the durable path and the deletable checks fold back into delete_needle, and the cancelled-write check mirrors Go's isCancelledWrite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
23eef59417 |
test/s3: run the s3-listing-study listing tools and strategies in CI (#12750)
* test(s3): s3-listing-study integration suite (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): run the s3-listing-study tools against SeaweedFS (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): s3-listing-study deleted-prefix probe, fresh data dir per run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): listing study ports keep clear of every gRPC companion free_ports.py compared only base ports, so a base port could land on another server's base+10000 gRPC listener. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study ports stay below the ephemeral range Base ports of 20000-55000 put both them and their gRPC companions inside 32768-60999, so an outgoing connection made while weed server starts could take a port before the server that was given it binds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study checks the order inside each listing page parse_page sorted Contents and CommonPrefixes together before the probes compared them, so keys returned out of order within one page passed. Check each list's byte order first; the versions pager does the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study applies --only before running a probe The filter was applied to the probe results, so a run limited to one tool still sent every reference listing request and the deleted-prefix check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study requires CommonPrefixes from shallow modes The CommonPrefixes comparison ran only when a tool returned at least one prefix, so a delimiter listing of the canary root that came back empty passed. Compare them for every shallow mode except the two whose normalizer keeps keys only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
04c7b12238 |
s3api: reject unknown notification elements; fail test-with-server on a stuck shutdown (#12752)
* s3api: reject unknown elements in PutBucketNotificationConfiguration xml.Unmarshal skips children that match no field, so a configuration whose only child was an unmapped element parsed as empty and returned success while nothing was stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: fail test-with-server when the server does not stop A passing suite returned success even when stop-server gave up with the server still running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f5418845ce |
test/s3: run tigrisdata-community s3-api-compat-tests in CI (#12741)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: prepare the suite again when the patch changes The checkout was reused whenever it sat at the pinned commit, so an edited seaweedfs-provider.patch, or a checkout whose patch failed to apply, kept being built as it was. Record the commit and the patch's hash once the patch applies, and prepare the checkout again when either differs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the pid file of a server already running start-server removed weed-server.pid before it found the port taken, so a second start lost the pid of the server holding it and stop-server could no longer stop that one. Remove the file only once the port is free, and have test-with-server stop only a server it started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: bound the S3 port probes A listener that accepts but never answers held curl, and with it the readiness loop and the cleanup after it, for good. Give each probe two seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: expect a listed test to end with its listed status A listed test passed the check with any non-pass status, so a setup error that skipped the call under test (RestoreObject's PutObject, say) went unnoticed. Entries now expect fail unless they name another status; only GetBucketAccelerateConfiguration, whose setup PUT is unsupported, expects error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: refuse to start while the recorded server still runs A second start-server before the first server answered on the S3 port passed the port probe and dropped the pid file of the live server, so stop-server could no longer stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its command line start-server and stop-server took any process named weed at the recorded pid for the test server, so a weed process that reused the pid would be reported as running and killed by stop-server. Match the command line start-server launches instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its start time A weed mini started from another checkout has the same command line, so a pid it reused still matched. Record the process start time next to the pid at launch and require both before treating the process as the test server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: PutObjectTagging_VersionId passes now Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the server records until the server exits stop-server waited a fixed two seconds after SIGTERM and then removed weed-server.pid and weed-server.started even if weed was still shutting down, so a later stop-server could not find it and clean could delete data it still used. Wait up to 30 seconds for the recorded server to exit and keep the records, failing, while it still runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: start the server on an empty data directory A data directory left by an earlier run on other ports made the master freeze volume growth (topology id mismatch), and every PutObject failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
53bc62c773 |
s3api: accept the accelerate and notification configurations the bucket already reports (#12747)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: prepare the suite again when the patch changes The checkout was reused whenever it sat at the pinned commit, so an edited seaweedfs-provider.patch, or a checkout whose patch failed to apply, kept being built as it was. Record the commit and the patch's hash once the patch applies, and prepare the checkout again when either differs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the pid file of a server already running start-server removed weed-server.pid before it found the port taken, so a second start lost the pid of the server holding it and stop-server could no longer stop that one. Remove the file only once the port is free, and have test-with-server stop only a server it started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: bound the S3 port probes A listener that accepts but never answers held curl, and with it the readiness loop and the cleanup after it, for good. Give each probe two seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: accept PutBucketAccelerateConfiguration with Status=Suspended GetBucketAccelerateConfiguration always reports Suspended, but the PUT answered 501 for every request. Suspended is now accepted, Enabled stays NotImplemented, and the accelerate subresource resolves to s3:GetAccelerateConfiguration / s3:PutAccelerateConfiguration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: accept an empty PutBucketNotificationConfiguration GetBucketNotificationConfiguration always reports an empty configuration, but the PUT answered 501, including for the empty configuration clients send to clear notifications. An empty one is now accepted; a topic, queue, Lambda or EventBridge destination stays NotImplemented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: reject accelerate and notification bodies with the wrong root Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: expect a listed test to end with its listed status A listed test passed the check with any non-pass status, so a setup error that skipped the call under test (RestoreObject's PutObject, say) went unnoticed. Entries now expect fail unless they name another status; only GetBucketAccelerateConfiguration, whose setup PUT is unsupported, expects error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: refuse to start while the recorded server still runs A second start-server before the first server answered on the S3 port passed the port probe and dropped the pid file of the live server, so stop-server could no longer stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its command line start-server and stop-server took any process named weed at the recorded pid for the test server, so a weed process that reused the pid would be reported as running and killed by stop-server. Match the command line start-server launches instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its start time A weed mini started from another checkout has the same command line, so a pid it reused still matched. Record the process start time next to the pid at launch and require both before treating the process as the test server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: PutObjectTagging_VersionId passes now Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the server records until the server exits stop-server waited a fixed two seconds after SIGTERM and then removed weed-server.pid and weed-server.started even if weed was still shutting down, so a later stop-server could not find it and clean could delete data it still used. Wait up to 30 seconds for the recorded server to exit and keep the records, failing, while it still runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: start the server on an empty data directory A data directory left by an earlier run on other ports made the master freeze volume growth (topology id mismatch), and every PutObject failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1f5cfc7552 |
s3api: return x-amz-version-id from object tagging on versioned buckets (#12745)
* s3api: return x-amz-version-id from object tagging on versioned buckets PutObjectTagging, GetObjectTagging and DeleteObjectTagging now report the version they acted on, "null" for an object written before versioning was enabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: set the tagging x-amz-version-id only on success The header was set before the tag-conditioned policy recheck, so a denied request carried the version id on its AccessDenied response. Also test the "null" version and require distinct version ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b1c315b837 |
s3api: report no ownership controls after DeleteBucketOwnershipControls (#12748)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: report no ownership controls after DeleteBucketOwnershipControls GetBucketOwnershipControls reports BucketOwnerEnforced for a bucket that never stored a setting, so it kept reporting it after the setting was deleted. The delete now records that it happened, and GET answers OwnershipControlsNotFoundError until ownership is put again. How objects are written after the delete is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: answer DeleteBucketOwnershipControls with 204 No Content AWS answers a successful DeleteBucketOwnershipControls with 204 and no body; this answered 200 with an empty OwnershipControls document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
aceb089bc8 |
lance: per-table cleanup retention and on-demand purge of deleted rows (#12739)
* lance: per-table cleanup retention and on-demand purge of deleted rows A vector deleted from an index stayed in storage until compaction rewrote its fragment (only above 10% deletions) and the cleanup job's 168 h retention, set once per worker, aged out every version that referenced it. Let a table override retain_hours and min_versions in its Lance config (seaweedfs.cleanup.retain_hours, seaweedfs.cleanup.min_versions), and let it request a purge (seaweedfs.purge.requested). The cleanup job proposes a table with a pending purge regardless of the version floor and then rewrites every fragment with deletions, retrains the indices (a remapped IVF_FLAT index still holds the deleted vector), keeps only the latest version, and records the request as served. A version pinned by a tag fails the purge and leaves the request pending. The test scans every file of a local table for a deleted vector's key, metadata and values, and finds none after the purge. * lance: refuse to purge a sharded vector index A retrain builds every index name over the whole table, so the purge turned shards of one vector index (one name per fragment set) into names that each covered every fragment, indexing every row once per shard. Check before compacting and fail with the request left pending, and retrain only the vector indices. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: rebuild scalar indices in a purge Compaction only remaps scalar indices, and a retrain leaves them alone, so a purge reported success while a bitmap index still listed the deleted row's value, and under stable row ids a BTree index still held the deleted key. Build each scalar index again from the remaining rows with the parameters it was built with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: purge through the cleanup job's retention policy The purge built its own cleanup policy, so a deployment that has the cleanup job keep more than the floor and the window (versions an unleased reader may still be on) would have lost them to a purge. Clean up through retention_policy, the one the job itself uses, with the latest version kept. Mark the request served only once the oldest version left is one the rebuild wrote, and record the rebuild so a purge that has to wait for its cleanup (a tag, or that policy) retries only the cleanup instead of compacting and retraining the indices on every pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vector: lease the version every list page reads A list page read its version without a lease unless it was a segment that read data and metadata twice. A purge removes every version but the latest, and only a lease tag stops it, so a listing that opened the version before the purge's compaction could lose its files mid-read. Pin it as get_vectors and query_vectors do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: give the cleanup job a build's execution time A purge compacts the table and retrains its indices inside a cleanup job, but the job declared no runtime defaults, so the admin held it to its 90 s default and could expire a purge mid-rebuild. Declare the same budget the indices job uses for a retrain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c7d8b106d5 |
mount: follow a peer unlink to the surviving hard link for closed inodes (#12737)
* mount: drop a peer-unlinked name only while its inode keeps another Move the survivor check from followSurvivingLink into InodeToPath.RemoveAlias so the check and the removal happen under one table lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: follow a peer unlink to the surviving hard link for closed inodes A peer deleting one name of a hard link only moved an open handle to the surviving name. A closed inode kept the deleted name as its primary path, so the next open by the kernel's nodeid failed with ENOENT. Co-authored-by: skymazer <54621405+skymazer@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: share the cached-entry freshness check of peer invalidations Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a name a local rename reclaimed before the peer unlink arrived Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a name a local rename or link gave the inode after an older peer delete The cached-entry check only covers a cached parent. InodeToPath now records, per hard-linked inode, the newest name a local rename or link gave it with the filer stamp, and the closed-inode peer-delete path keeps that name when the delete is older. The record lives in a side map dropped on Forget, so InodeEntry keeps its size class. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: sign rename stream responses with the filer signature A rename response carries the stamp of the filer that applied it, so the mount can tell which clock the stamp belongs to, as it does for create acks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: order a peer delete against a local rename or link only on one filer clock The stamp of a local rename or link comes from the filer that applied it, and a delete logged by another filer runs on a different clock. Keep the name only when both carry the same nonzero filer signature; otherwise drop it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drop the local-name stamp fence for uncached parents Ordering a delayed peer delete against a local rename by filer stamp needs a record per alias, a shared filer clock, and the open-handle path to follow; that is a larger change than this fix. Back to the cached-entry check, which covers a cached parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: skymazer <54621405+skymazer@users.noreply.github.com> |
||
|
|
a3b2fd722d |
mount: report opens for reading with TouchAccessTime (-accessReport) (#12721)
* mount: report opens for reading with TouchAccessTime (-accessReport) An open answered from the metadata cache never reaches the filer, so a filer that tracks access cannot see that a cached file is still being read. With -accessReport, the mount sends one TouchAccessTime per open for reading, with the open time, whether the open was answered from the cache or not. Write-only opens are not reported. Reports go out in the background and never hold up the open: a report that does not fit in the queue is dropped, and drops and failures are logged once a minute. A filer that answers Unimplemented pauses reporting for 10 minutes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: address review of the access report - Drop reports still queued when reporting pauses; the filer that answered Unimplemented would answer them the same way. - Bound each filer attempt to 2s, so a filer that accepts the connection but never answers leaves time for the next one, and keep using the filer that answered. - Fail over only when a filer cannot be reached. Any answer, Unimplemented included, ends the walk. - Report an O_CREAT open of an existing file with read access, which comes through Create instead of Open. A new file has nothing to read and is not reported. - Stop the reporter on unmount: stop ends the workers and cancels the report in flight. - Say in the flag help that it has no effect unless the filer implements TouchAccessTime. - Tests no longer wait a fixed time to see that nothing else was sent; they look at the queue of a reporter that is not started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail an access report over only when the filer did not answer A status the filer sent ends the walk even when its message contains a word the transient classifier matches, so an Unimplemented answer still pauses reporting and a rejected request is not resent elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a filer selection made while an access report failed over A report that reached a later filer moves the mount there only if the selection it started from is still current, so a slower report cannot undo another call's failover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: share an access report's time with the filers not yet tried Each attempt is capped by an equal share of the report time left, so a row of stalled filers cannot use up the report before a healthy one later in the list is tried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report the path an open for reading came through A create that opens an existing hard link reports the name it was given rather than the inode's first known name. Open reads the inode's path from the inode table rather than the handle's saved path, which other handle operations may rewrite concurrently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the access report stop and failover tests without races The post-stop report is checked on a reporter without workers, so no worker can take an accepted report before the check, and the failover test asserts that the stalled filer was tried first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: keep the shared channel when the peer answers with its own status gRPC never raises InvalidArgument, NotFound, AlreadyExists, FailedPrecondition, OutOfRange or DataLoss itself, so such a status is the peer's answer. It fell through to the string matching, and a message naming a path like /transport closed the cached channel and cancelled every other call on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: say which filer selection an access report keeps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: bound the in-flight call in the rejected-report channel test The call held in flight went through touchAccessTime, whose 2 s attempt cap could expire on a slow runner and fail the test on a healthy channel. It now uses callFiler with the test deadline. The wait for it to reach the filer also ends if it returns early or the deadline passes, and the blocked handler is released on every exit path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report each file at most once per -accessReport.interval Every read open used to send a TouchAccessTime, adding a filer metadata call per open. A path reported within the interval (default 1h) is now skipped; the last report times live in a bounded LRU. 0 keeps reporting every open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report an open again once the file was modified since its last report Within -accessReport.interval a read open was skipped even after a write, so the filer kept an atime older than the mtime until the interval ran out. relatime records that first read after a write; do the same by comparing the entry's mtime with the last report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep the report time apart from the mtime it saw Recording max(now, mtime) as the last report let a file dated a day ahead suppress reports for a day plus the interval. Record when the report was sent and the mtime the entry had, and treat any change of that mtime as a modification, so neither clock can stretch or shorten the interval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: MapleWhiteZZZ <18653045+MapleWhiteZZZ@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
9f413ff2c8 |
shell: volume.check.disk skips replicas whose live counts match (#12743)
* shell: volume.check.disk skips replicas whose live counts match FileCount counts every write and DeleteCount every delete or overwrite until a vacuum drops them, so a replica vacuumed while its peer was not keeps a different FileCount forever and was re-checked on every run although both hold the same live needles. Only treat counts as differing when the live counts (FileCount - DeleteCount) differ too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk skips equal live counts only across compaction revisions Equal live counts at the same compaction revision mean a missed write and a missed delete, not a vacuum-state difference, so keep checking those pairs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk real-time recheck ignores heartbeat revisions VolumeStatus counts are newer than the heartbeat's compaction revisions, so a vacuum in between could make a pair at the same revision look vacuum-skewed. Apply the old count rules there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk skips on live counts only when no repair could follow Equal live counts across compaction revisions can still hide a missed write and a missed delete. Keep checking such pairs when -syncDeleted could propagate the delete, or when -resurrectMissingNeedles could copy a needle into a never-vacuumed replica. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e037a9c1ce |
s3vectors: enforce AWS metadata, key and filter limits (#12738)
* s3vectors: enforce AWS metadata, key and filter limits PutVectors accepted vector metadata of any size and key count, bounded only by the request body limit, and keys of any length; QueryVectors accepted filters of any size. Apply the AWS S3 Vectors limits (https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-limitations.html), taking the ENHANCED index values where the modes differ, since this implementation prefilters and takes no indexMode: - 100 KB total metadata per vector, which also bounds a single non-filterable field; - 2 KB filterable metadata per vector; - 50 metadata keys per vector; - 1024-character vector keys; - at most 10 nonFilterableMetadataKeys of 1 to 63 characters at CreateIndex; - 100 filter constraints per QueryVectors request. A violation is a ValidationException before the engine is called. * s3vectors: accept metadata whose JSON is exactly at the byte limits metadataEntrySize counts a trailing comma for every member, so starting from both braces overcounted by one and rejected metadata that encodes to exactly 2 KB filterable or 100 KB total. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3vectors: rewrap the metadata limits header comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3db54ee7e5 |
kms/openbao: refresh the AppRole token before it expires (#12734)
* kms/openbao: refresh the AppRole token before it expires The provider logged in with AppRole once, when it was created, and never renewed the token. After the token's TTL every SSE-KMS upload, and every read of an SSE-KMS object whose data key was not cached, failed with "permission denied" (HTTP 500 to the client) until the S3 server was restarted. With AppRole authentication the provider now logs in again in the background once two thirds of the token's TTL have passed, and retries a failed login every 5 s. Each login is bounded by the client's timeout, or by 30 s when request_timeout is not positive. Close stops the refresh. A static token, or an AppRole token without a TTL, is left as it is. * kms: keep the OpenBao token refresh across configuration reloads - AddKMSProvider and InitializeGlobalKMS create a provider instance of their own with the new NewProvider instead of the registry's cached one: a reload closed the cached instance and then got it back, which now stopped its token refresh. The new instance is created before the old one is closed, so a configuration that fails leaves the current provider working - openbao: time each refresh from the start of the login that issued the token, since its TTL runs from then rather than from the response - tests: a reload gives a new instance and closes the old one, a failed reload keeps the old one; the refresh survives a reload; a slow login response * kms: create a replacement provider outside the manager lock AddKMSProvider ran the provider factory, which for OpenBao is a login that may wait for its timeout, while holding the manager lock, so every lookup, also of unrelated providers, waited for it. The new provider is now created before the lock is taken; the lock covers only the swap, and the replaced provider is closed after it. Test: a lookup while another provider is being created. |
||
|
|
94de65b3b5 |
s3api: answer a malformed Cache-Control on PutObject with InvalidRequest (#12733)
PutObject rejected a Cache-Control value it could not parse with InvalidDigest, whose message is about Content-MD5, even when the request sent no Content-MD5. CreateMultipartUpload already answers the same parse error with InvalidRequest; PutObject now does too. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ebc77400ca |
operation: reassign an upload after a proven-expired volume JWT (#12729)
* operation: reassign an upload after a proven-expired volume JWT * seaweed-volume: mark an expired write JWT refused before append Mirror the Go volume server: a multipart POST whose write JWT is signed with the current key, names this fid and fails only on exp gets X-Seaweed-Write-Outcome: expired-jwt-before-append on its 401, so the uploader reassigns instead of failing. Both checks use one guard snapshot, so a key rotation between them cannot drop the marker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: keep same-FID retry for filer-proxy expiries, snapshot the write signing key, and make the RetryForever budget test fail fast * operation: re-mint the filer credential before a proxy same-FID expiry retry * operation: make a proxy marked-expiry terminal in the outer retry too, not only the inner same-FID loop --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ecbf7af1ca |
mount: compare permissions against local ids without mapping them again (#12726)
* mount: compare permissions against local ids without mapping them again Access, open and create (without -defaultPermissions) ran FilerToLocal on entries that maybeReadEntry and maybeLoadEntry already return in local ids. Under a uid/gid map where a local id is also a mapped filer id, the owner was mapped twice and the check refused the file's own owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: cache a deferred create's placeholder in filer ids The meta cache stores filer uid/gid and maps them to local ids on every read, but createRegularFile cached the deferred or writeback placeholder with the caller's local ids. The second mapping in the permission checks used to cancel that out under a swapping id map; without it the owner of a placeholder was refused. Cache a mapped clone and keep the handle's entry in local ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test that a deferred create returns its entry in local ids Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b64895ed0a |
filer: stop following a peer that stays unreachable (#12725)
* filer: stop following a peer that stays unreachable * filer: re-track an answering peer only while it is still followed A signature read that succeeds just before the master removes the peer called initPeerWatermark after the removal, clearing its mark. The loop then exits, so the watermark it re-tracked never advances or drops and pins aggregated readers at the settled horizon. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: forget a peer's unreachable clock when the master removes it Only a successful connection cleared the clock, so every departed address that once failed stayed in the map for the filer's lifetime, and a re-added peer inherited the old outage. The removal now drops it, and the clock is guarded by peerChansLock so the removed peer's exiting loop cannot start a new one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: report the unreachable mark only when it is set Once peerResumeRetention forgets the mark of a peer that stays unreachable, markPeerWatermarkRemoved no longer marks it (its watermarks are gone), but notePeerUnreachable still saw no mark and logged "treating it as removed" on every retry. markPeerWatermarkRemoved now reports whether it set a new mark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: fix the unreachable-peer subscriber's start before the baseline write The subscriber goroutine picked its start time itself, so a baseline write that finished first fell before the subscription and the test failed after 20 seconds with correct delivery. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
72ac9e96b2 |
mount: answer an interrupted read with EINTR, not EIO (#12723)
* mount: answer an interrupted read with EINTR, not EIO The kernel's FUSE contract (Documentation/filesystems/fuse/fuse.rst, "Interrupting filesystem operations") lets a filesystem either ignore an INTERRUPT or honor it by replying EINTR to the original request. Read did neither: the FUSE cancel channel cancels the read's context, the chunk fetch fails with "context canceled", and every error became EIO. Interrupts come from any signal the reader handles. Go's runtime sends SIGURG for goroutine preemption, so Go programs reading under load hit this on healthy data: on a 128-thread 64 KiB random-read test about 250 of 5 million reads failed with EIO, and none with GODEBUG=asyncpreemptoff=1. Read now returns EINTR when its own context was cancelled, the way dirReadStatus already does for directory reads and lookups do since #11717. Go's os.File retries EINTR, so those readers see no error. A cancelled read also skips the entry refresh retry, and neither read log line reports an interrupt as a warning or error. Failures without an interrupt stay EIO. Tests: TestReadInterruptedReturnsEINTR (fails on the old code with EIO), TestReadFailureWithoutInterruptReturnsEIO; weed/mount/... passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: require the failed-read test to reach the filer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: skip the read retry once the request is interrupted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
11af0e0c12 |
mount: move an open file's read view to the chunks a merge wrote (#12722)
* mount: move an open file's read view to the chunks a merge wrote When a flush finds that a file's chunks take more than twice its size, maybeMergeChunks reads the file back and writes it again as new chunks. The commit that publishes them makes the filer delete the chunks they replaced. The flush put the new chunks into the handle's entry but left its read view on the old ones, so every descriptor that kept the file open went on reading the replaced chunks. Those read back until their volume is vacuumed. After that a read first retries the fetch of the missing needle for about 13 s; then refreshEntryForRead installs the filer's entry and the read is retried. The refresh skips a handle with writes it has not flushed, and such a read returns EIO instead. Files that are kept open and rewritten in place, such as PostgreSQL data files and SQLite databases, are the ones whose chunks bloat, so they are the ones that hit this. installRewrittenChunks puts the merge's chunks into the entry and the read view together, under the entry lock. Chunks attached while the merge ran were not part of it and stay. Flushes without a merge are unchanged. TestMergeMovesReadViewOffReplacedChunks covers the handle. TestReadOpenHandleAfterChunkMerge runs it on a real mount, with and without an unflushed write; for it, the fuse_integration framework starts mini on a free master port and exposes the address. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drain uploads before a merge replaces the chunks Uploads attach their chunks without the handle lock, first to the entry and then to the read view, and writes that landed after doFlush drained the pages may still be uploading when the flush takes its lock. An upload that attached during the merge could keep a chunk older than data the merge already holds, or put a chunk the merge replaced back into the read view. Either outranks the merge's zero-timestamp chunk. When the chunks are bloated enough to merge, the flush now drains the pages again while its lock holds writes off, and takes the chunks again. FlushAll returns only after every upload has attached to both the entry and the read view, so nothing attaches during the merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: leave a pre-merge upload failure to the next flush The drain before a merge returned an upload failure through the metadata flush. doFlush then retried it as a filer error and answered EIO, where the page flush answers a full disk with ENOSPC. The drain is there only to wait for the uploads: a failure stays with the pages, and the next flush reports it as a write error, as it would without a merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep chunks attached while a flush installs its compacted list A write that lands after doFlush drained the pages can still be uploading when flushMetadataToFiler takes its chunk snapshot. Its upload attaches the chunk without the handle lock, and the plain flush then replaced the entry's chunks with the list compacted from the snapshot, dropping it from the handle and from every later commit. Keep chunks attached after the snapshot, as the merge path already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail a merge's flush on a failed drain A failed upload found by the drain before a merge was only logged, and the flush went on to publish a size covering bytes that no chunk stores. It now fails the flush as a pageUploadError before anything is published; doFlush maps it with writeErrorToFuseStatus, as it does its own FlushData failure, and does not retry it as a filer error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: clone a flush's request entry under the entry lock The request snapshot cloned the live entry after GetEntry had released its read lock. Uploads attach chunks without the handle lock, so an AppendChunks that landed during the clone raced it on the Chunks field. Clone inside UpdateEntry, as content flush recovery does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: leave a merge to a later flush while uploads are pending Before a merge the flush drained the pages while it held the handle and publication locks. A writeback upload retries a transient storage error until it succeeds, so that drain could hold both locks for as long as storage stayed down, stalling reads, writes and stat on the handle. The flush now checks for pending uploads and, if any, publishes the compacted chunks and leaves the merge to a later flush. With none pending the lock still holds writes off, so nothing attaches during the merge, and a failed upload still fails the flush as a write error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep room for pending uploads when a flush merges onto a moved entry A flush that leaves the chunk merge to a later flush publishes while uploads are still pending, and writes that landed after the drain can be uploading on any flush. Merging a peer's inline content or a shrink into such a publication lets the upload attach its chunk next to the inline bytes, where reads never reach it. Tell createContentEntry when uploads are pending so the merge keeps room for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test chunk merges through the filer's chunk proxy, also under load The tests serve the filer's chunk proxy, and like the filer the fake deletes the chunks each commit replaces, so a read view still naming them fails: - TestFlushMergeThroughChunkProxy/Quiet: the flush merges, and reads through the open handle return the file from the merged chunk alone. - TestFlushMergeThroughChunkProxy/UploadInFlight: an older page is still uploading while a newer page of the same bytes has attached. The flush leaves the merge, reads return the newer page, and the next flush merges. - TestFlushMergeRacesWithWritesAndReads: four writers, three flushers and a reader on one handle, with uploads delayed at random. The writers pause now and then, so 14 to 18 flushes a run merge and the others leave the merge to a later one. Every read must succeed within 2 s, and at the end the handle and the filer must both hold exactly what was written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: let a held upload go when UploadInFlight fails The subtest's proxy holds the older page's upload until the test releases it, and only the passing path did. A failure in between left the handler blocked, so the proxy's cleanup waited on it and the failure surfaced as a suite timeout. Release it from a cleanup as well, once, and bound the waits for the upload and the drains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |