mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-19 02:15:02 +01:00
1fd5008d63081eb295fe4b6496edc96cf0bc266a
16305
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
1fd5008d63 |
s3api: listing argument validation and encoding-type=url for versions, prefix and delimiter (#12753)
* s3api: reject an empty ListObjectsV2 continuation-token
AWS answers continuation-token= with 400 InvalidArgument ("The continuation
token provided is incorrect"); SeaweedFS listed from the start of the bucket.
No token this server issues is empty, and raw-key and base64: tokens keep
resuming as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: reject an encoding-type other than url on listings
AWS answers any encoding-type other than url with 400 InvalidArgument
("Invalid Encoding Method specified in Request") on ListObjects,
ListObjectsV2, ListObjectVersions, ListMultipartUploads and ListParts;
SeaweedFS ignored the value and listed unencoded keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: apply encoding-type=url to ListObjectVersions
The handler accepted encoding-type=url but returned raw keys. It now
URL-encodes Key in Version and DeleteMarker entries, CommonPrefixes, Prefix,
Delimiter, KeyMarker and NextKeyMarker with the urlPathEscape form ListObjects
uses, and echoes EncodingType.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
* s3api: URL-encode Prefix and Delimiter in ListObjects responses
Under encoding-type=url, ListObjects and ListObjectsV2 encoded keys,
CommonPrefixes, markers and StartAfter but echoed Prefix and Delimiter raw,
so a client decoding them once corrupted a prefix holding '%' or '+'.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
646549eee1 |
volume growth: spread writable volumes of a layout over servers and disks (#12749)
* volume growth: spread writable volumes of a layout over servers and disks Volume growth picked every server, and the volume server picked every disk, independently: the server weighted by free slots, the disk by the lowest total volume count. A grow request therefore stacks several writable volumes of one collection on some disks while most disks get none (a Poisson spread: with 160 replicas on 479 disks about 20 disks get two or three), and their writes share those disks. Data centers and racks are still picked weighted by free slots. Inside a rack the server is now one of those holding the fewest writable volumes of the layout per disk, weighted by free slots among them; this covers the main server, same-rack replicas and replicas walked into other racks and data centers. On the volume server the disk is the one with the fewest writable volumes of the collection, then the lowest volume count as before. A grow request registers each volume before placing the next, so a request smaller than the number of disks lands on distinct disks. Fixes #12746 * volume growth: scan the layout load once per grow request Placing each volume rescanned all writable volumes of the layout under its read lock. RecordAssign takes that lock exclusively on every assign, so with thousands of writables (0.8 ms for 10000) every volume of a grow request stalled assigns once more. GrowByCountAndType now scans once and findAndGrow adds every volume it creates to the counts. Per volume, placement costs about 130-160 us on 480 servers whatever the number of writables, against 120 us before the spread change; the single scan is 0.17 ms for 2000 writables and 0.8 ms for 10000. * volume growth: address review of the spread placement - Count only disks that report volume slots in a server's load. An unavailable disk, or one low on space with no volumes, is still listed in DiskTags with no slots; counting it understated the load of a server with failed disks and piled volumes onto its remaining ones. - On the volume server, judge a volume full by its .dat size, which the master compares with the limit, not by ContentSize. - A server that refuses an allocation ranks last for the rest of the grow request, and the request retries the volume up to 3 times. The least loaded server is picked first deterministically, so without this one server that cannot fit a preallocated volume stopped the layout from growing. - Scan the layout load after warmup, so a grow request issued right after a leader change sees the writable volumes the returning servers report. * volume growth: keep the request-local placement load ahead of the request fields The JSON request fields stay together at the end of VolumeGrowOption, and the comment now says the load lives for the whole grow request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): place a new volume on the disk with the fewest writable volumes of its collection Store::add_volume picked the disk with the most free slots, so the Rust volume server ignored the collection spread rule the Go server now applies. find_free_location ranks disks by the writable volumes of the collection first (not read-only, .dat below the volume size limit), then by free slots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume growth: leave a refusing server out of every placement tier A server that refused an allocation only ranked last among the servers of its rack, so the weighted data center and rack picks could still choose its rack on every retry while other racks had room. It now offers no free slots for the rest of the grow request, which takes its slots out of the rack and data center weights too. If only refusing servers are left, they are tried again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume growth: require the full same-rack count only in the main rack The data center check asked every rack for SameRackCount+1 free servers, but only the main rack holds the same-rack copies; each other rack takes one. With a refused server left out of its rack, the check rejected a feasible placement, the retry put the refused server back, and it was picked again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Konstantin Lebedev <whitefox@mayflower.work> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9cd183b04f |
volume: fsync a needle delete that asks for it (#12736)
* volume: fsync a needle delete that asks for it A delete now takes fsync like a write: the tombstone is synced before the needle map drops the needle, and the .idx row before the delete is acknowledged. A failed .dat sync takes the tombstone back off the .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: honor fsync=true on DELETE and forward it to replicas Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): fsync a needle delete that asks for it Mirrors the Go volume server: DELETE with fsync=true flushes the tombstone before the index drops the needle and the .idx before the ack, and a failed .dat flush takes the tombstone back off the .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: let a durable delete through on a full volume A queued delete hit the batch worker's volume size check, which a tombstone append is exempt from inline, so fsync=true deletes failed once the volume reached its hard limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: sync a durable delete of an already-deleted needle An earlier delete without fsync may have dropped the needle with its tombstone still unsynced, so the inline path syncs before acking anyway, as the batch worker already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: refuse a batch request that races the worker stopping stopWorker can close the batch channel between startWorker handing it out and the send, which panicked. The request now falls back to the inline path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): sync a durable delete of an already-deleted needle Mirrors the Go volume server: an earlier delete without fsync may have dropped the needle with its tombstone still unsynced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): run a durable delete on a blocking thread Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: keep syncDelete's signature for existing callers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: refuse a write a delete cancelled before it appends A write cancelled while still queued behind the volume locks used to append, publish and then tombstone itself without a sync, after its delete may already have been acknowledged as durable. A crash could keep the row and lose the tombstone. Refuse it before the append. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): refuse a write a delete cancelled before it appends Mirrors the Go change: a write cancelled while still queued behind the volume locks no longer appends, publishes and tombstones itself without a sync after its delete may already be durable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): drop the write guards for a durable delete's flushes The durable delete held the write lease and the io write guard across both fsyncs, so reads of the volume waited on them. Append and publish the tombstone under the guards and flush with only the io read side, as write_synced_run and the Go path already do. do_delete_request keeps its signature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stop taking writes when a durable delete fails to index The needle map drops the needle before it appends the .idx row, so after a failed append a durable retry found nothing to delete and acked, and a later write could bury the unindexed tombstone so a reload brought the needle back. Fence the volume the way a failed durable write does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): stop taking writes when a durable delete fails to index Rust counterpart of the Go change: the needle map drops the needle before it appends the .idx row, so a failed append left a durable retry nothing to delete, and a later write could bury the unindexed tombstone. publish_delete_request now fences the volume for a durable delete, as publish_write does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stop taking writes when any delete fails to index A non-durable delete whose .idx append failed left the volume writable with the needle already dropped from the map, so a later durable retry found nothing to delete and acknowledged without the missing row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): stop taking writes when any delete fails to index Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: take the fsync flag on the existing delete functions DeleteVolumeNeedle, deleteNeedle2 and syncDelete take fsync instead of gaining *WithFsync variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): take the fsync flag on the existing delete functions delete_volume_needle and delete_needle take fsync instead of gaining *_with_fsync variants; the durable path and the deletable checks fold back into delete_needle, and the cancelled-write check mirrors Go's isCancelledWrite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
23eef59417 |
test/s3: run the s3-listing-study listing tools and strategies in CI (#12750)
* test(s3): s3-listing-study integration suite (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): run the s3-listing-study tools against SeaweedFS (WIP) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): s3-listing-study deleted-prefix probe, fresh data dir per run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test(s3): listing study ports keep clear of every gRPC companion free_ports.py compared only base ports, so a base port could land on another server's base+10000 gRPC listener. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study ports stay below the ephemeral range Base ports of 20000-55000 put both them and their gRPC companions inside 32768-60999, so an outgoing connection made while weed server starts could take a port before the server that was given it binds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study checks the order inside each listing page parse_page sorted Contents and CommonPrefixes together before the probes compared them, so keys returned out of order within one page passed. Check each list's byte order first; the versions pager does the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study applies --only before running a probe The filter was applied to the probe results, so a run limited to one tool still sent every reference listing request and the deleted-prefix check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(s3): listing study requires CommonPrefixes from shallow modes The CommonPrefixes comparison ran only when a tool returned at least one prefix, so a delimiter listing of the canary root that came back empty passed. Compare them for every shallow mode except the two whose normalizer keeps keys only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
04c7b12238 |
s3api: reject unknown notification elements; fail test-with-server on a stuck shutdown (#12752)
* s3api: reject unknown elements in PutBucketNotificationConfiguration xml.Unmarshal skips children that match no field, so a configuration whose only child was an unmapped element parsed as empty and returned success while nothing was stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: fail test-with-server when the server does not stop A passing suite returned success even when stop-server gave up with the server still running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f5418845ce |
test/s3: run tigrisdata-community s3-api-compat-tests in CI (#12741)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: prepare the suite again when the patch changes The checkout was reused whenever it sat at the pinned commit, so an edited seaweedfs-provider.patch, or a checkout whose patch failed to apply, kept being built as it was. Record the commit and the patch's hash once the patch applies, and prepare the checkout again when either differs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the pid file of a server already running start-server removed weed-server.pid before it found the port taken, so a second start lost the pid of the server holding it and stop-server could no longer stop that one. Remove the file only once the port is free, and have test-with-server stop only a server it started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: bound the S3 port probes A listener that accepts but never answers held curl, and with it the readiness loop and the cleanup after it, for good. Give each probe two seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: expect a listed test to end with its listed status A listed test passed the check with any non-pass status, so a setup error that skipped the call under test (RestoreObject's PutObject, say) went unnoticed. Entries now expect fail unless they name another status; only GetBucketAccelerateConfiguration, whose setup PUT is unsupported, expects error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: refuse to start while the recorded server still runs A second start-server before the first server answered on the S3 port passed the port probe and dropped the pid file of the live server, so stop-server could no longer stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its command line start-server and stop-server took any process named weed at the recorded pid for the test server, so a weed process that reused the pid would be reported as running and killed by stop-server. Match the command line start-server launches instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its start time A weed mini started from another checkout has the same command line, so a pid it reused still matched. Record the process start time next to the pid at launch and require both before treating the process as the test server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: PutObjectTagging_VersionId passes now Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the server records until the server exits stop-server waited a fixed two seconds after SIGTERM and then removed weed-server.pid and weed-server.started even if weed was still shutting down, so a later stop-server could not find it and clean could delete data it still used. Wait up to 30 seconds for the recorded server to exit and keep the records, failing, while it still runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: start the server on an empty data directory A data directory left by an earlier run on other ports made the master freeze volume growth (topology id mismatch), and every PutObject failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
53bc62c773 |
s3api: accept the accelerate and notification configurations the bucket already reports (#12747)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: prepare the suite again when the patch changes The checkout was reused whenever it sat at the pinned commit, so an edited seaweedfs-provider.patch, or a checkout whose patch failed to apply, kept being built as it was. Record the commit and the patch's hash once the patch applies, and prepare the checkout again when either differs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the pid file of a server already running start-server removed weed-server.pid before it found the port taken, so a second start lost the pid of the server holding it and stop-server could no longer stop that one. Remove the file only once the port is free, and have test-with-server stop only a server it started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: bound the S3 port probes A listener that accepts but never answers held curl, and with it the readiness loop and the cleanup after it, for good. Give each probe two seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: accept PutBucketAccelerateConfiguration with Status=Suspended GetBucketAccelerateConfiguration always reports Suspended, but the PUT answered 501 for every request. Suspended is now accepted, Enabled stays NotImplemented, and the accelerate subresource resolves to s3:GetAccelerateConfiguration / s3:PutAccelerateConfiguration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: accept an empty PutBucketNotificationConfiguration GetBucketNotificationConfiguration always reports an empty configuration, but the PUT answered 501, including for the empty configuration clients send to clear notifications. An empty one is now accepted; a topic, queue, Lambda or EventBridge destination stays NotImplemented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: reject accelerate and notification bodies with the wrong root Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3/tigris_compat: expect a listed test to end with its listed status A listed test passed the check with any non-pass status, so a setup error that skipped the call under test (RestoreObject's PutObject, say) went unnoticed. Entries now expect fail unless they name another status; only GetBucketAccelerateConfiguration, whose setup PUT is unsupported, expects error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: refuse to start while the recorded server still runs A second start-server before the first server answered on the S3 port passed the port probe and dropped the pid file of the live server, so stop-server could no longer stop it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its command line start-server and stop-server took any process named weed at the recorded pid for the test server, so a weed process that reused the pid would be reported as running and killed by stop-server. Match the command line start-server launches instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: recognize the test server by its start time A weed mini started from another checkout has the same command line, so a pid it reused still matched. Record the process start time next to the pid at launch and require both before treating the process as the test server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: PutObjectTagging_VersionId passes now Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: keep the server records until the server exits stop-server waited a fixed two seconds after SIGTERM and then removed weed-server.pid and weed-server.started even if weed was still shutting down, so a later stop-server could not find it and clean could delete data it still used. Wait up to 30 seconds for the recorded server to exit and keep the records, failing, while it still runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3/tigris_compat: start the server on an empty data directory A data directory left by an earlier run on other ports made the master freeze volume growth (topology id mismatch), and every PutObject failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1f5cfc7552 |
s3api: return x-amz-version-id from object tagging on versioned buckets (#12745)
* s3api: return x-amz-version-id from object tagging on versioned buckets PutObjectTagging, GetObjectTagging and DeleteObjectTagging now report the version they acted on, "null" for an object written before versioning was enabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: set the tagging x-amz-version-id only on success The header was set before the tag-conditioned policy recheck, so a denied request carried the version id on its AccessDenied response. Also test the "null" version and require distinct version ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b1c315b837 |
s3api: report no ownership controls after DeleteBucketOwnershipControls (#12748)
* test/s3: run tigrisdata-community s3-api-compat-tests in CI Runs the suite, pinned to a commit, against weed mini through a small provider patch, and fails on any test that is not in expected_failures.txt or on a listed test that starts passing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: report no ownership controls after DeleteBucketOwnershipControls GetBucketOwnershipControls reports BucketOwnerEnforced for a bucket that never stored a setting, so it kept reporting it after the setting was deleted. The delete now records that it happened, and GET answers OwnershipControlsNotFoundError until ownership is put again. How objects are written after the delete is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3api: answer DeleteBucketOwnershipControls with 204 No Content AWS answers a successful DeleteBucketOwnershipControls with 204 and no body; this answered 200 with an empty OwnershipControls document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
aceb089bc8 |
lance: per-table cleanup retention and on-demand purge of deleted rows (#12739)
* lance: per-table cleanup retention and on-demand purge of deleted rows A vector deleted from an index stayed in storage until compaction rewrote its fragment (only above 10% deletions) and the cleanup job's 168 h retention, set once per worker, aged out every version that referenced it. Let a table override retain_hours and min_versions in its Lance config (seaweedfs.cleanup.retain_hours, seaweedfs.cleanup.min_versions), and let it request a purge (seaweedfs.purge.requested). The cleanup job proposes a table with a pending purge regardless of the version floor and then rewrites every fragment with deletions, retrains the indices (a remapped IVF_FLAT index still holds the deleted vector), keeps only the latest version, and records the request as served. A version pinned by a tag fails the purge and leaves the request pending. The test scans every file of a local table for a deleted vector's key, metadata and values, and finds none after the purge. * lance: refuse to purge a sharded vector index A retrain builds every index name over the whole table, so the purge turned shards of one vector index (one name per fragment set) into names that each covered every fragment, indexing every row once per shard. Check before compacting and fail with the request left pending, and retrain only the vector indices. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: rebuild scalar indices in a purge Compaction only remaps scalar indices, and a retrain leaves them alone, so a purge reported success while a bitmap index still listed the deleted row's value, and under stable row ids a BTree index still held the deleted key. Build each scalar index again from the remaining rows with the parameters it was built with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: purge through the cleanup job's retention policy The purge built its own cleanup policy, so a deployment that has the cleanup job keep more than the floor and the window (versions an unleased reader may still be on) would have lost them to a purge. Clean up through retention_policy, the one the job itself uses, with the latest version kept. Mark the request served only once the oldest version left is one the rebuild wrote, and record the rebuild so a purge that has to wait for its cleanup (a tag, or that policy) retries only the cleanup instead of compacting and retraining the indices on every pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vector: lease the version every list page reads A list page read its version without a lease unless it was a segment that read data and metadata twice. A purge removes every version but the latest, and only a lease tag stops it, so a listing that opened the version before the purge's compaction could lose its files mid-read. Pin it as get_vectors and query_vectors do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * lance: give the cleanup job a build's execution time A purge compacts the table and retrains its indices inside a cleanup job, but the job declared no runtime defaults, so the admin held it to its 90 s default and could expire a purge mid-rebuild. Declare the same budget the indices job uses for a retrain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c7d8b106d5 |
mount: follow a peer unlink to the surviving hard link for closed inodes (#12737)
* mount: drop a peer-unlinked name only while its inode keeps another Move the survivor check from followSurvivingLink into InodeToPath.RemoveAlias so the check and the removal happen under one table lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: follow a peer unlink to the surviving hard link for closed inodes A peer deleting one name of a hard link only moved an open handle to the surviving name. A closed inode kept the deleted name as its primary path, so the next open by the kernel's nodeid failed with ENOENT. Co-authored-by: skymazer <54621405+skymazer@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: share the cached-entry freshness check of peer invalidations Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a name a local rename reclaimed before the peer unlink arrived Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a name a local rename or link gave the inode after an older peer delete The cached-entry check only covers a cached parent. InodeToPath now records, per hard-linked inode, the newest name a local rename or link gave it with the filer stamp, and the closed-inode peer-delete path keeps that name when the delete is older. The record lives in a side map dropped on Forget, so InodeEntry keeps its size class. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: sign rename stream responses with the filer signature A rename response carries the stamp of the filer that applied it, so the mount can tell which clock the stamp belongs to, as it does for create acks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: order a peer delete against a local rename or link only on one filer clock The stamp of a local rename or link comes from the filer that applied it, and a delete logged by another filer runs on a different clock. Keep the name only when both carry the same nonzero filer signature; otherwise drop it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drop the local-name stamp fence for uncached parents Ordering a delayed peer delete against a local rename by filer stamp needs a record per alias, a shared filer clock, and the open-handle path to follow; that is a larger change than this fix. Back to the cached-entry check, which covers a cached parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: skymazer <54621405+skymazer@users.noreply.github.com> |
||
|
|
a3b2fd722d |
mount: report opens for reading with TouchAccessTime (-accessReport) (#12721)
* mount: report opens for reading with TouchAccessTime (-accessReport) An open answered from the metadata cache never reaches the filer, so a filer that tracks access cannot see that a cached file is still being read. With -accessReport, the mount sends one TouchAccessTime per open for reading, with the open time, whether the open was answered from the cache or not. Write-only opens are not reported. Reports go out in the background and never hold up the open: a report that does not fit in the queue is dropped, and drops and failures are logged once a minute. A filer that answers Unimplemented pauses reporting for 10 minutes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: address review of the access report - Drop reports still queued when reporting pauses; the filer that answered Unimplemented would answer them the same way. - Bound each filer attempt to 2s, so a filer that accepts the connection but never answers leaves time for the next one, and keep using the filer that answered. - Fail over only when a filer cannot be reached. Any answer, Unimplemented included, ends the walk. - Report an O_CREAT open of an existing file with read access, which comes through Create instead of Open. A new file has nothing to read and is not reported. - Stop the reporter on unmount: stop ends the workers and cancels the report in flight. - Say in the flag help that it has no effect unless the filer implements TouchAccessTime. - Tests no longer wait a fixed time to see that nothing else was sent; they look at the queue of a reporter that is not started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail an access report over only when the filer did not answer A status the filer sent ends the walk even when its message contains a word the transient classifier matches, so an Unimplemented answer still pauses reporting and a rejected request is not resent elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a filer selection made while an access report failed over A report that reached a later filer moves the mount there only if the selection it started from is still current, so a slower report cannot undo another call's failover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: share an access report's time with the filers not yet tried Each attempt is capped by an equal share of the report time left, so a row of stalled filers cannot use up the report before a healthy one later in the list is tried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report the path an open for reading came through A create that opens an existing hard link reports the name it was given rather than the inode's first known name. Open reads the inode's path from the inode table rather than the handle's saved path, which other handle operations may rewrite concurrently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the access report stop and failover tests without races The post-stop report is checked on a reporter without workers, so no worker can take an accepted report before the check, and the failover test asserts that the stalled filer was tried first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: keep the shared channel when the peer answers with its own status gRPC never raises InvalidArgument, NotFound, AlreadyExists, FailedPrecondition, OutOfRange or DataLoss itself, so such a status is the peer's answer. It fell through to the string matching, and a message naming a path like /transport closed the cached channel and cancelled every other call on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: say which filer selection an access report keeps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: bound the in-flight call in the rejected-report channel test The call held in flight went through touchAccessTime, whose 2 s attempt cap could expire on a slow runner and fail the test on a healthy channel. It now uses callFiler with the test deadline. The wait for it to reach the filer also ends if it returns early or the deadline passes, and the blocked handler is released on every exit path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report each file at most once per -accessReport.interval Every read open used to send a TouchAccessTime, adding a filer metadata call per open. A path reported within the interval (default 1h) is now skipped; the last report times live in a bounded LRU. 0 keeps reporting every open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report an open again once the file was modified since its last report Within -accessReport.interval a read open was skipped even after a write, so the filer kept an atime older than the mtime until the interval ran out. relatime records that first read after a write; do the same by comparing the entry's mtime with the last report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep the report time apart from the mtime it saw Recording max(now, mtime) as the last report let a file dated a day ahead suppress reports for a day plus the interval. Record when the report was sent and the mtime the entry had, and treat any change of that mtime as a modification, so neither clock can stretch or shorten the interval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: MapleWhiteZZZ <18653045+MapleWhiteZZZ@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
9f413ff2c8 |
shell: volume.check.disk skips replicas whose live counts match (#12743)
* shell: volume.check.disk skips replicas whose live counts match FileCount counts every write and DeleteCount every delete or overwrite until a vacuum drops them, so a replica vacuumed while its peer was not keeps a different FileCount forever and was re-checked on every run although both hold the same live needles. Only treat counts as differing when the live counts (FileCount - DeleteCount) differ too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk skips equal live counts only across compaction revisions Equal live counts at the same compaction revision mean a missed write and a missed delete, not a vacuum-state difference, so keep checking those pairs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk real-time recheck ignores heartbeat revisions VolumeStatus counts are newer than the heartbeat's compaction revisions, so a vacuum in between could make a pair at the same revision look vacuum-skewed. Apply the old count rules there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: volume.check.disk skips on live counts only when no repair could follow Equal live counts across compaction revisions can still hide a missed write and a missed delete. Keep checking such pairs when -syncDeleted could propagate the delete, or when -resurrectMissingNeedles could copy a needle into a never-vacuumed replica. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e037a9c1ce |
s3vectors: enforce AWS metadata, key and filter limits (#12738)
* s3vectors: enforce AWS metadata, key and filter limits PutVectors accepted vector metadata of any size and key count, bounded only by the request body limit, and keys of any length; QueryVectors accepted filters of any size. Apply the AWS S3 Vectors limits (https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-limitations.html), taking the ENHANCED index values where the modes differ, since this implementation prefilters and takes no indexMode: - 100 KB total metadata per vector, which also bounds a single non-filterable field; - 2 KB filterable metadata per vector; - 50 metadata keys per vector; - 1024-character vector keys; - at most 10 nonFilterableMetadataKeys of 1 to 63 characters at CreateIndex; - 100 filter constraints per QueryVectors request. A violation is a ValidationException before the engine is called. * s3vectors: accept metadata whose JSON is exactly at the byte limits metadataEntrySize counts a trailing comma for every member, so starting from both braces overcounted by one and rejected metadata that encodes to exactly 2 KB filterable or 100 KB total. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3vectors: rewrap the metadata limits header comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3db54ee7e5 |
kms/openbao: refresh the AppRole token before it expires (#12734)
* kms/openbao: refresh the AppRole token before it expires The provider logged in with AppRole once, when it was created, and never renewed the token. After the token's TTL every SSE-KMS upload, and every read of an SSE-KMS object whose data key was not cached, failed with "permission denied" (HTTP 500 to the client) until the S3 server was restarted. With AppRole authentication the provider now logs in again in the background once two thirds of the token's TTL have passed, and retries a failed login every 5 s. Each login is bounded by the client's timeout, or by 30 s when request_timeout is not positive. Close stops the refresh. A static token, or an AppRole token without a TTL, is left as it is. * kms: keep the OpenBao token refresh across configuration reloads - AddKMSProvider and InitializeGlobalKMS create a provider instance of their own with the new NewProvider instead of the registry's cached one: a reload closed the cached instance and then got it back, which now stopped its token refresh. The new instance is created before the old one is closed, so a configuration that fails leaves the current provider working - openbao: time each refresh from the start of the login that issued the token, since its TTL runs from then rather than from the response - tests: a reload gives a new instance and closes the old one, a failed reload keeps the old one; the refresh survives a reload; a slow login response * kms: create a replacement provider outside the manager lock AddKMSProvider ran the provider factory, which for OpenBao is a login that may wait for its timeout, while holding the manager lock, so every lookup, also of unrelated providers, waited for it. The new provider is now created before the lock is taken; the lock covers only the swap, and the replaced provider is closed after it. Test: a lookup while another provider is being created. |
||
|
|
94de65b3b5 |
s3api: answer a malformed Cache-Control on PutObject with InvalidRequest (#12733)
PutObject rejected a Cache-Control value it could not parse with InvalidDigest, whose message is about Content-MD5, even when the request sent no Content-MD5. CreateMultipartUpload already answers the same parse error with InvalidRequest; PutObject now does too. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ebc77400ca |
operation: reassign an upload after a proven-expired volume JWT (#12729)
* operation: reassign an upload after a proven-expired volume JWT * seaweed-volume: mark an expired write JWT refused before append Mirror the Go volume server: a multipart POST whose write JWT is signed with the current key, names this fid and fails only on exp gets X-Seaweed-Write-Outcome: expired-jwt-before-append on its 401, so the uploader reassigns instead of failing. Both checks use one guard snapshot, so a key rotation between them cannot drop the marker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: keep same-FID retry for filer-proxy expiries, snapshot the write signing key, and make the RetryForever budget test fail fast * operation: re-mint the filer credential before a proxy same-FID expiry retry * operation: make a proxy marked-expiry terminal in the outer retry too, not only the inner same-FID loop --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ecbf7af1ca |
mount: compare permissions against local ids without mapping them again (#12726)
* mount: compare permissions against local ids without mapping them again Access, open and create (without -defaultPermissions) ran FilerToLocal on entries that maybeReadEntry and maybeLoadEntry already return in local ids. Under a uid/gid map where a local id is also a mapped filer id, the owner was mapped twice and the check refused the file's own owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: cache a deferred create's placeholder in filer ids The meta cache stores filer uid/gid and maps them to local ids on every read, but createRegularFile cached the deferred or writeback placeholder with the caller's local ids. The second mapping in the permission checks used to cancel that out under a swapping id map; without it the owner of a placeholder was refused. Cache a mapped clone and keep the handle's entry in local ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test that a deferred create returns its entry in local ids Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b64895ed0a |
filer: stop following a peer that stays unreachable (#12725)
* filer: stop following a peer that stays unreachable * filer: re-track an answering peer only while it is still followed A signature read that succeeds just before the master removes the peer called initPeerWatermark after the removal, clearing its mark. The loop then exits, so the watermark it re-tracked never advances or drops and pins aggregated readers at the settled horizon. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: forget a peer's unreachable clock when the master removes it Only a successful connection cleared the clock, so every departed address that once failed stayed in the map for the filer's lifetime, and a re-added peer inherited the old outage. The removal now drops it, and the clock is guarded by peerChansLock so the removed peer's exiting loop cannot start a new one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: report the unreachable mark only when it is set Once peerResumeRetention forgets the mark of a peer that stays unreachable, markPeerWatermarkRemoved no longer marks it (its watermarks are gone), but notePeerUnreachable still saw no mark and logged "treating it as removed" on every retry. markPeerWatermarkRemoved now reports whether it set a new mark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: fix the unreachable-peer subscriber's start before the baseline write The subscriber goroutine picked its start time itself, so a baseline write that finished first fell before the subscription and the test failed after 20 seconds with correct delivery. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
72ac9e96b2 |
mount: answer an interrupted read with EINTR, not EIO (#12723)
* mount: answer an interrupted read with EINTR, not EIO The kernel's FUSE contract (Documentation/filesystems/fuse/fuse.rst, "Interrupting filesystem operations") lets a filesystem either ignore an INTERRUPT or honor it by replying EINTR to the original request. Read did neither: the FUSE cancel channel cancels the read's context, the chunk fetch fails with "context canceled", and every error became EIO. Interrupts come from any signal the reader handles. Go's runtime sends SIGURG for goroutine preemption, so Go programs reading under load hit this on healthy data: on a 128-thread 64 KiB random-read test about 250 of 5 million reads failed with EIO, and none with GODEBUG=asyncpreemptoff=1. Read now returns EINTR when its own context was cancelled, the way dirReadStatus already does for directory reads and lookups do since #11717. Go's os.File retries EINTR, so those readers see no error. A cancelled read also skips the entry refresh retry, and neither read log line reports an interrupt as a warning or error. Failures without an interrupt stay EIO. Tests: TestReadInterruptedReturnsEINTR (fails on the old code with EIO), TestReadFailureWithoutInterruptReturnsEIO; weed/mount/... passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: require the failed-read test to reach the filer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: skip the read retry once the request is interrupted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
11af0e0c12 |
mount: move an open file's read view to the chunks a merge wrote (#12722)
* mount: move an open file's read view to the chunks a merge wrote When a flush finds that a file's chunks take more than twice its size, maybeMergeChunks reads the file back and writes it again as new chunks. The commit that publishes them makes the filer delete the chunks they replaced. The flush put the new chunks into the handle's entry but left its read view on the old ones, so every descriptor that kept the file open went on reading the replaced chunks. Those read back until their volume is vacuumed. After that a read first retries the fetch of the missing needle for about 13 s; then refreshEntryForRead installs the filer's entry and the read is retried. The refresh skips a handle with writes it has not flushed, and such a read returns EIO instead. Files that are kept open and rewritten in place, such as PostgreSQL data files and SQLite databases, are the ones whose chunks bloat, so they are the ones that hit this. installRewrittenChunks puts the merge's chunks into the entry and the read view together, under the entry lock. Chunks attached while the merge ran were not part of it and stay. Flushes without a merge are unchanged. TestMergeMovesReadViewOffReplacedChunks covers the handle. TestReadOpenHandleAfterChunkMerge runs it on a real mount, with and without an unflushed write; for it, the fuse_integration framework starts mini on a free master port and exposes the address. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drain uploads before a merge replaces the chunks Uploads attach their chunks without the handle lock, first to the entry and then to the read view, and writes that landed after doFlush drained the pages may still be uploading when the flush takes its lock. An upload that attached during the merge could keep a chunk older than data the merge already holds, or put a chunk the merge replaced back into the read view. Either outranks the merge's zero-timestamp chunk. When the chunks are bloated enough to merge, the flush now drains the pages again while its lock holds writes off, and takes the chunks again. FlushAll returns only after every upload has attached to both the entry and the read view, so nothing attaches during the merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: leave a pre-merge upload failure to the next flush The drain before a merge returned an upload failure through the metadata flush. doFlush then retried it as a filer error and answered EIO, where the page flush answers a full disk with ENOSPC. The drain is there only to wait for the uploads: a failure stays with the pages, and the next flush reports it as a write error, as it would without a merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep chunks attached while a flush installs its compacted list A write that lands after doFlush drained the pages can still be uploading when flushMetadataToFiler takes its chunk snapshot. Its upload attaches the chunk without the handle lock, and the plain flush then replaced the entry's chunks with the list compacted from the snapshot, dropping it from the handle and from every later commit. Keep chunks attached after the snapshot, as the merge path already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail a merge's flush on a failed drain A failed upload found by the drain before a merge was only logged, and the flush went on to publish a size covering bytes that no chunk stores. It now fails the flush as a pageUploadError before anything is published; doFlush maps it with writeErrorToFuseStatus, as it does its own FlushData failure, and does not retry it as a filer error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: clone a flush's request entry under the entry lock The request snapshot cloned the live entry after GetEntry had released its read lock. Uploads attach chunks without the handle lock, so an AppendChunks that landed during the clone raced it on the Chunks field. Clone inside UpdateEntry, as content flush recovery does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: leave a merge to a later flush while uploads are pending Before a merge the flush drained the pages while it held the handle and publication locks. A writeback upload retries a transient storage error until it succeeds, so that drain could hold both locks for as long as storage stayed down, stalling reads, writes and stat on the handle. The flush now checks for pending uploads and, if any, publishes the compacted chunks and leaves the merge to a later flush. With none pending the lock still holds writes off, so nothing attaches during the merge, and a failed upload still fails the flush as a write error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep room for pending uploads when a flush merges onto a moved entry A flush that leaves the chunk merge to a later flush publishes while uploads are still pending, and writes that landed after the drain can be uploading on any flush. Merging a peer's inline content or a shrink into such a publication lets the upload attach its chunk next to the inline bytes, where reads never reach it. Tell createContentEntry when uploads are pending so the merge keeps room for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test chunk merges through the filer's chunk proxy, also under load The tests serve the filer's chunk proxy, and like the filer the fake deletes the chunks each commit replaces, so a read view still naming them fails: - TestFlushMergeThroughChunkProxy/Quiet: the flush merges, and reads through the open handle return the file from the merged chunk alone. - TestFlushMergeThroughChunkProxy/UploadInFlight: an older page is still uploading while a newer page of the same bytes has attached. The flush leaves the merge, reads return the newer page, and the next flush merges. - TestFlushMergeRacesWithWritesAndReads: four writers, three flushers and a reader on one handle, with uploads delayed at random. The writers pause now and then, so 14 to 18 flushes a run merge and the others leave the merge to a later one. Every read must succeed within 2 s, and at the end the handle and the filer must both hold exactly what was written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: let a held upload go when UploadInFlight fails The subtest's proxy holds the older page's upload until the test releases it, and only the passing path did. A failure in between left the handler blocked, so the proxy's cleanup waited on it and the failure surfaced as a suite timeout. Release it from a cleanup as well, once, and bound the waits for the upload and the drains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
de5b3305a7 |
mount: ask the filer again when a deferred create is published mid-lookup (#12720)
* mount: ask the filer again when a deferred create is published mid-lookup A program that writes a file under a temporary name and renames it while the file is still open (safetensors save_file does, through Rust tempfile's persist) looks the temporary name up before the rename. When the parent directory's listing is not cached, that lookup goes to the filer. If a FLUSH publishes the deferred create while the lookup is in flight (a forked child closing its inherited descriptor sends one), the filer can answer not-found from a read taken before the create committed. By the time the answer arrives the flush has cleared the handle's dirty flag, so lookupEntry finds nothing local to explain the answer, logs "possible coherence bug" and returns ENOENT, and the rename fails although the file exists. Note before sending the request whether the inode had unpublished metadata (a dirty handle or an async flush in flight). On not-found, if it had and has since been published, ask the filer once more: the filer acknowledged the publish before the second request is sent. A second not-found stands, so a file another mount removed is not brought back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: judge a placeholder's not-found by one look at its base Review of the previous commit found two problems. The retry check and the not-found fallback read the handle separately, so a flush that finished between the two reads still turned the answer into ENOENT. And the pre-request read of dirtyMetadata, a plain bool that Write and the flush set under the handle lock, raced with them on every lookup that reached the filer. Note instead whether the handle is a local placeholder: a deferred create has no filer base (baseEntry) until a flush publishes it. A not-found for a placeholder is judged by one atomic look at that base. With none, the filer cannot have the file yet and the handle answers. With a base, the filer acknowledged the publish, so ask once more. dirtyMetadata is not read for this, and the not-found fallback keeps the existing check, with the placeholder decision added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: answer a placeholder from its own handle; a second miss is final Follow-ups from review of the previous commit: - Pick the placeholder after the async-flush wait, from the path's current mapping. - A placeholder that still has no base answers from the observed handle itself, instead of through the not-found fallback, which re-resolved the path and could miss once the handle was released and the cache entry evicted. - A not-found from the second request is final. The flush installs the base before it clears dirtyMetadata, and keeps it set when a write landed meanwhile, so the fallback could return a file another mount had removed. - Test gates release on cleanup, so a failing test leaves no handler blocked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: read a placeholder's entry the way GetAttr does The previous commit answered a not-found for a placeholder from the handle's live entry under entryLock. Neither of the entry's writers takes that lock: Write and the metadata flush change size, times and chunks under the handle lock, and async upload workers append chunks under the LockedEntry lock, so the lookup could read an entry while it changed. Answer with the copy the create cached when it is still this file's (same inode), as the not-found fallback already prefers. Otherwise copy the handle's entry under both writers' locks, in GetAttr's order (handle lock shared, then the LockedEntry lock), and convert the copy. No caller of lookupEntry holds a handle lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: cache a deferred create's placeholder in filer ids The meta cache stores filer uid/gid and maps them to local ids on every read, but createRegularFile cached the placeholder with the caller's local ids. Under a uid/gid map whose local id is also a mapped filer id, a lookup or stat answered from the placeholder reported the wrong owner. Cache a mapped clone and keep the handle's entry in local ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: assert the placeholder is cached before checking its owner Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the snapshot owner after evicting the cached placeholder Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
04e135c21e |
fix(helm): separate volume DNS discovery from ingress traffic (#12719)
* helm: set publishNotReadyAddresses on the volume headless service * fix(helm): separate volume DNS discovery from ingress traffic * test(helm): cover volume DNS discovery and ingress routing * fix(helm): rename the volume routing Service so volumes.client cannot collide * docs(helm): name the ready-only Service for explicit volume backendRefs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(helm): fail when the volume routing Service name collides after truncation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(helm): point explicit volume backendRefs at the rendered client-volume name Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
862913d199 |
volume, ec_balance: clean up failed EC shard copies; stop re-proposing empty-index volumes (#12718)
* volume: remove files a failed VolumeEcShardsCopy wrote The shard files were copied before the .ecx, so a source holding a 0-byte index failed the request only after its shards had landed, and nothing removed them: every retried move stranded another unmounted shard on a new disk. Copy the .ecx first, staged beside its target, so a bad index fails before any shard lands; rename it into place only after the shards and sidecars have copied, so a failed copy never truncates or replaces an .ecx the destination already has. On any error, remove exactly the files this call created; files already on the disk (an existing holder's shards, index or journal) are left alone. Go and Rust volume servers behave the same. Claude-Session: https://claude.ai/code/session_011UpvafcfUuC2mLxNAXLQJi * ec_balance: skip EC volumes whose every holder reports an empty .ecx A shard copy refuses a 0-byte .ecx, so detection proposed the same failing move of such a volume every cycle. Leave volumes with no holder reporting a needle out of planning; their shards still count toward disk occupancy. Claude-Session: https://claude.ai/code/session_011UpvafcfUuC2mLxNAXLQJi * volume: serialize VolumeEcShardsCopy per volume Overlapping copies of one vid shared the staged .ecx, and a failed copy's cleanup removed shards that a concurrent copy had already committed and mounted. Hold a per-vid EC lifecycle lock across the whole copy and its cleanup, named as in the enterprise build (Store.EcLifecycleLock / ec_lifecycle_lock). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: overwrite an existing .ecx in place when committing a copied index A mounted EcVolume reads its index through an open handle, so renaming the staged copy over it left the reader on the replaced inode: tombstones the source had folded into its .ecx never reached the mounted holder. An index already on the disk is now overwritten in place, as before the staging change; a fresh destination still gets the rename. Go and Rust. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stage replacements of shards a VolumeEcShardsCopy finds on disk The shard transfer opened an existing destination shard with O_TRUNC and removed it on a stream or write error, so a failed retry deleted a shard this disk already held. A shard already present is now copied to <shard>.copying and renamed into place only after every other file has copied; a mounted copy is reopened so it serves the new file. Fresh shards still write in place and are removed on failure. Go and Rust. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stage the .vif and .ecsum a VolumeEcShardsCopy brings until commit The sidecars landed as soon as they transferred, ahead of the deferred .ecx commit, so a failure after them left a new .vif beside the index the failed copy kept. They are now staged as .copying and renamed into place with the shards and index once every transfer has succeeded; a source without one still stages nothing. Go and Rust. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: run a Rust VolumeEcShardsCopy to completion when its caller goes away tonic drops a cancelled handler future at its current await, which released the EC lifecycle lock while file operations the copy had started (a cleanup remove_file on the blocking pool, a create or rename) could still land; a retry taking the lock could then lose its files to them. The copy now runs in a spawned task that owns the lock, as Go's handler runs to completion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: fail the sidecar copy tests with an unreadable source .ecsum A directory at the source path reads as "not found" on volume servers that skip non-regular EC files, which turned the failure into a no-op. A mode-000 file fails the transfer on every implementation; root and Windows, which can read it, skip the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: copy each shard a VolumeEcShardsCopy lists once A shard listed twice and already held here was staged twice; the first commit rename consumed the staged file, so the second failed the request after the shard was replaced, and every retry failed the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: wait out an in-flight VolumeEcShardsCopy before mounting its shards A shard the copy had already landed could be mounted while the copy was still running; the copy's failure cleanup then removed the file but left the shard registered and announced. MountEcShards takes the per-volume lifecycle lock, as the enterprise build already does, and the Rust mount handler takes the same lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: fail a VolumeEcShardsCopy whose replaced mounted shard cannot be reopened The copy reported success while the mounted shard kept serving the replaced file. The error lets the caller retry the copy instead of treating the shard as delivered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: check which transfer fails in the EC copy cleanup tests An earlier failure, before any shard or sidecar landed, would leave the destination unchanged and pass every assertion without exercising the cleanup. The sidecar test also skips on Windows, where a mode-000 file stays readable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: fail the sidecar transfer in the EC copy test by injection A mode-000 file is still readable to root and on Windows, so the test skipped there. The source now returns an error for the .ecsum request only, and the test checks that it served .vif before that failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: take the EC lifecycle lock for a blanket full teardown A teardown acknowledged while a detached copy of the same volume was still waiting on its source let that copy recreate the index and shards afterwards. Go and Rust now wait for the copy, as the enterprise build does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: keep the request id on a Rust EC shard copy The copy now runs in a spawned task, outside the gRPC layer's request id scope, so its CopyFile calls to the source went out with a new id. Carry the caller's id into the task. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: unmount a replaced EC shard whose reopen fails The failed copy left the mount reading the replaced file's inode, so the destination kept serving the old shard. Unmount it, which also reports the removal to the master, so a retry copies and mounts it again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: make the Rust EC copy tests fail for the right reason The sidecar test now serves a directory as the source .ecsum, which root cannot read either, so it no longer returns early there. The cancel test holds the store until the handler's join reports the cancellation, so a copy that finished before the abort can no longer pass it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: stage every shard a VolumeEcShardsCopy transfers A shard missing here was written in place with a cleanup guard armed before its transfer started, so a failed copy removed a shard that ReceiveFile wrote meanwhile. Every shard is now staged, and only one that did not exist when it is moved into place is removed on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: take the EC lifecycle lock for VolumeEcShardsRebuild A rebuild folds the deletion journal into .ecx. Run during a copy of the same volume, the copy's index then replaced the folded tombstones. Go and Rust take the lock the enterprise build already takes here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: check the staged .vif bytes in the EC sidecar failure test A source without the optional .vif also returns success, so the served extension list could not prove the transfer. The test now reads the staged .vif when the .ecsum request fails and compares it with the source's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: unmount a failed reopen's shard on the copy's disk only UnmountEcShards walks every disk, so a reopen failure on the destination also unmounted a healthy copy of the same shard on a sibling disk. The copy now unloads the shard from its own disk and reports that disk's removal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: take the EC lifecycle lock for an ordinary EC shard delete Shards a copy has staged are invisible to the delete's last-shard check, so deleting the only canonical shard removed the index the copy was about to commit its shards against. Go and Rust deletes now wait for an in-flight copy of the same volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: leave a mounted shard's copied file in place on copy failure A shard mounted on the copy's disk whose file was missing got the copied file reopened into its mount, then that file was unlinked by a later failure's cleanup while the mount kept serving it. Only an unmounted shard's new file is removed now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: skip repeated shard ids with a seen set Scanning the request prefix for each id made a request with many repeats quadratic while the copy held the volume's lifecycle lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: make the Go EC copy and lock tests observe their effects The fake source counts requests per extension, so the duplicate-id test proves .ec05 is fetched once, and returns fixture errors instead of failing the test from its handler goroutine. The staged .vif reaches the test through a channel. The mount and teardown lock tests seed real files and check that nothing happened while the lock was held and that the operation took effect after it was released. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: poll the handler directly in the Rust EC lifecycle lock tests A spawned handler that had not run when the 200 ms timer fired passed the blocking check without ever reaching the lock. The tests now poll the pinned handler future themselves, which guarantees it is waiting on the lock, and cover the ordinary shard delete as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: observe the lifecycle-lock waiter in EC copy lock tests The mount, teardown, shard-delete and rebuild tests slept 200 ms while holding the lock, so a handler scheduled only after the unlock passed without ever taking it. Wait until the handler's goroutine is blocked on the mutex instead; each test now fails within its timeout if the handler skips the lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: check the shard is staged before the failure-path write in a copy test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: recover a missing EC index under the volume's lifecycle lock Missing-index recovery fetched the index and then ran a store-wide mount of every orphan shard, without the per-volume lifecycle lock. It could mount shards an in-flight VolumeEcShardsCopy of another volume had just renamed in, and that copy's failure cleanup then unlinked a mounted shard. Each recovered volume now holds its lifecycle lock across a re-check, the fetch and a mount restricted to that volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): recover a missing EC index under the volume's lifecycle lock Mirror of the Go change: recover_missing_ec_indexes takes the volume's ec_lifecycle_lock, re-checks that the index is still missing, fetches it, and mounts only that volume's shards through mount_recovered_ec_shards_only, so an in-flight VolumeEcShardsCopy of another volume never has its renamed-in shards mounted from under it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: mount a volume whose index a copy installed during recovery Recovery re-checks under the lifecycle lock that the index is still missing. When a copy installed it in the meantime, the re-check skipped the mount too, and the volume's other shards stayed unmounted because later scans no longer list it. Skip only the fetch and still mount the volume's orphan shards under the lock, in Go and Rust. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: fetch a recovered EC index into the disk the locked re-check found The re-check under the lifecycle lock only kept a yes/no answer, so the fetch still wrote to the destination from the first scan, which may be a disk that has since gone away. Fetch into the record the re-check returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: mount a copy-installed EC index even when no peer is known Recovery skipped the locked step when the master was unreachable, the lookup failed or no peer was listed. A copy that installed the index after discovery then left the volume's other shards unmounted, and later scans no longer list it. Always run the locked re-check, and give up for lack of peers only when the index is still missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: grow the stack buffer before scanning for a lock waiter A full buffer can cut off the handler goroutine's stack, so the test would report a handler that is waiting as one that never waited. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): fetch a recovered EC index into the disk the locked re-check found Mirror of the Go change: fetch into the record the re-check under the lifecycle lock returned instead of the destination from the first scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): mount a copy-installed EC index even when no peer is known Mirror of the Go change: a failed lookup or an empty peer list no longer skips the locked re-check, so a volume whose index a copy installed after discovery still gets its orphan shards mounted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): run missing-index recovery to completion when the mount RPC is cancelled Dropping the mount handler mid-recovery released the volume's lifecycle lock while the .ecj merge it had started on a blocking thread was still landing, so a teardown could remove the volume before that late write. Run recovery on its own task, as the copy handler does, so the lock is held until it finishes; Go's handler already runs it to completion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: skip other volumes before stat in a single-volume EC scan Recovery re-checks and mounts each volume with its own scans, so a request with many orphans repeated whole-store discovery once per volume. The index-owner and orphan-shard scans now take the same volume filter and drop other volumes' files by name, before the per-file stat, and the locked re-check scans only its own volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): skip other volumes before stat in a single-volume EC scan Mirror of the Go change: the index-owner and orphan-shard scans take the volume filter and drop other volumes' files by name before the per-file stat, so recovery's per-volume re-check and mount no longer repeat whole-store discovery under the store lock for every orphan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): announce shards recovered after the mount RPC was cancelled Recovery now outlives a cancelled mount RPC, but the heartbeat nudge stayed in the handler, so shards it mounted after the caller left waited for the next full EC heartbeat. Nudge it from the recovery task, as Go's loads announce each recovered shard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: match the collection as well as the volume id in the recovery re-check Discovery keys orphans by (collection, vid), but the locked re-check matched the vid only. Two collections with orphans of the same vid could both resolve to the first record, so the other's index was never fetched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): match the collection as well as the volume id in the recovery re-check Mirror of the Go change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8c6d09f498 |
s3api: answer NoSuchBucket from GetBucketRequestPayment for a missing bucket (#12716)
* s3api: answer NoSuchBucket from GetBucketRequestPayment for a missing bucket The handler wrote the static BucketOwner answer without resolving the bucket, so a bucket that does not exist looked like one that does. Check the bucket first, as PutBucketRequestPayment and the other static configuration getters already do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: test GetBucketRequestPayment on existing and missing buckets Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
eb1dca8088 |
filer: release the stamps of parent directories a rename creates (#12711)
* filer: release the stamps of parent directories a rename creates * filer: test that a suppressed create keeps the moved entry's stamp in flight Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
deacd575ea |
operation: prepare upload content before assigning the volume (#12714)
* operation: prepare upload content before assigning the volume * operation: keep the existing upload function names for prepared payloads UploadData and doUpload prepare once and hand the prepared payload to retriedUploadData, so the retry loop, doUploadData, uploadChunk and uploadChunkToHolders take a preparedUpload directly instead of growing a prepared twin of each and branching on nil. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: send ciphertext whenever the prepared payload carries a cipher key doUploadData chose the cipher branch from the option, so a payload prepared under a different option could be sent as plaintext metadata or lose its key. Decide from the prepared payload itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: build a chunk's preparation and upload options in one place The chunk loop prepared with a hand-written option while uploadChunk and uploadChunkToHolders built their own, so a setting that shapes the bytes could reach one and not the other. chunkUploadOption now feeds all three. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * operation: test that a reassigned chunk reuses its prepared payload Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
194e6ede9f |
seaweed-volume: reject FetchAndWriteNeedle reads that miss the requested size (#12717)
The Go volume server fails the RPC when the remote read returns a different length than req.Size; the Rust server cached whatever came back as a full-size chunk. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
97f49571a1 |
wdclient: let master-pushed adds replace the gateway's lookup-fed locations (#12705)
* wdclient: let master-pushed adds replace the gateway's lookup-fed locations After a master leader change the S3 gateway clears its FilerClient cache and refills it through filer lookups. The filer's own master client may still answer from the previous leader's generation until the new leader has heard from the volume servers, so a volume server lost together with the old leader is cached again. The new leader never knew that server and never pushes its removal, and the gateway keeps trying it until reads fail on it volume by volume, one 30 s dial timeout at a time. The gateway's master client relearns every volume from the new leader's adds. On each add, cache the volume where the master client now knows it to be, in place of what a lookup cached, and keep lookups already in flight from caching their answer over it. A provider answer for a volume is now cached whole or not at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: take a lookup's fill epoch before its second cache check A master add that landed between the second cache check and the read of the fill epoch was invisible to both: the check had already missed, and the epoch the lookup then read already included the confirmation, so a filer answer that still named the lost server was merged back into the confirmed entry. Taking the epoch first means such an add either shows up in the check or stops the answer from being cached. The same order covers a reset or clear in that gap. TestConfirmDuringLookupIsNotUndone fires a confirmation from a test hook at that point and fails with the old order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: gate the master lookup provider's cache writes on the map epoch The provider wrote each answer straight into the vid map, so a lookup sent before a master reset and answered after it landed in the new generation, which a gateway's FilerClient now reads as the current master's view. The answer now goes through fillLocations, which drops it once a reset passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * wdclient: keep the vidMapClient doc comment on its type Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
91eaabf371 |
s3: answer ClientDisconnected for a GET the client aborted before the first byte (#12707)
* s3: answer ClientDisconnected for a GET the client aborted before the first byte Fixes #12706 * s3: tell a gone client on TLS and HTTP/2 too, and not on a connection the server closed |
||
|
|
c91da3d1f8 |
filer: fail health check when absent from lock ring (#12709)
* filer: fail health check when absent from lock ring If the master broadcasts a lock ring that does not include this filer (e.g. the late-cleanup race in seaweedfs#11442), every lock acquisition fails with "no lock server found" and locked writes, including S3 PUTs, hang indefinitely while /healthz and /readyz still report OK. Make the filer health handler return 503 once this filer has been missing from its own lock ring for longer than a 30s grace period, so Kubernetes liveness probes restart it and it re-registers with the master. The grace period covers startup before the first ring update and the master's ring-change batching delay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: hold the lock-ring health clock until the filer announces AnnounceOnceCaughtUp can hold the master registration for up to a minute while HTTP already serves, so the filer cannot be in the ring yet. Report healthy until the announce gate opens and start the grace period after it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep a probe racing a ring rejoin from reading a reset clock A probe that loses the CompareAndSwap to a concurrent healthy probe could load the reset zero and report the filer absent since the epoch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: trim the lock-ring health comments Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: restart the lock-ring health grace when a ring update brings the filer back The clock only reset when a probe saw the filer in the ring, so a rejoin and a second removal between two probes reused the first absence and failed the next probe early. Reset it from the ring snapshot callback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test the announce gate leaves the lock-ring health clock unset Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: re-read lock-ring membership after starting the health clock A probe whose first read raced a rejoin could start the clock after the snapshot callback reset it, leaving a stale start for the next removal. The ring is installed before the callback runs, so a second read sees the rejoin whenever the reset came first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: treat a health clock changed during the probe as a rejoin A rejoin and a second removal can both land between the probe's two membership reads. The second read then shows the filer absent again, and the probe judged the new absence by the expired start it had read before the rejoin's reset. The snapshot callback resets the clock before the next ring update is installed, so a clock that no longer matches the probe's start means a rejoin happened; report healthy and let the next probe use the fresh clock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test ring updates that land during a lock-ring health probe Move the absence clock into lockRingAbsence, which takes the membership read as a function, so the test can install ring updates between the probe's reads: a rejoin after the first read, and a rejoin plus removal between the two reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: give every lock-ring health reset a new clock value A probe that saw the filer absent started the clock with a CAS from 0, so a rejoin that reset the clock to 0 again before that CAS was invisible, and the probe could install a start time from before the rejoin. Each reset now stores a new non-positive value and the probe swaps from the value it loaded before the membership read, so any rejoin during the probe makes the CAS fail. This drops the second membership read and never waits on the LockRing mutex. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test lock-ring rejoins before and after the probe's membership read Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep the lock-ring version lookup out of the unhealthy health path LockRing.Version takes the ring mutex, which SetSnapshot holds while the snapshot callback transfers locks, so an expired-absence probe could wait on those transfers and time out instead of answering 503. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test an expired-absence health probe while the ring mutex is held Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
c76fb36416 |
test(fuse): two mounts staging the same files without -dlm (#12704)
* test(fuse): two mounts staging the same files without -dlm Two jobs that extract the same archive into one shared directory, each through its own mount and without -dlm, both create many of the same files: a file that exists is skipped, otherwise it is created, written and closed, then chmod'ed and given its mtime. Before #12593, #12594 and #12596 the attribute update and the content flush sent the mount's whole cached entry. Sent after the other mount had rewritten the file, the update put back chunks that the rewrite had dropped and the filer had deleted, so the entry pointed at deleted needles and the file failed to read once its volume was vacuumed. Nothing told the writers. TestStageSameFilesWithoutDLM runs that on the fuse_dlm cluster with the mounts started without -dlm (startTestCluster takes a dlm flag; startDLMTestCluster is unchanged) and checks every staged file: each chunk is still on the volume server (HEAD on the fid, without readDeleted) and the filer serves the bytes written. What the losing writer is told depends on the version and is only logged. With a 4.53 weed binary the test fails: 66, 71 and 70 of the 400 files were broken in three runs. With master it passes in about 48 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(fuse): bound the staging test's filer and volume checks The checks used the default HTTP client and a background context, so a server that accepts a request and stops answering held the test until the package timeout instead of failing the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(fuse): fail the staging test unless both mounts wrote the same files A mount whose MkdirAll failed returned early, and a mount that started late could skip every file, so the test passed with one writer. Start both loops together, fail on setup errors, and require files both mounts wrote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(fuse): bound the staging test's checks with one deadline A per-call timeout still let 400 hung lookups and reads run for hours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
0d3f5a220c |
s3: keep serving for -preStopSeconds after SIGTERM (#12703)
* s3: keep serving for -preStopSeconds after SIGTERM On SIGTERM the gateway closed its listeners at once, so requests a load balancer or Kubernetes Service still routed to it failed until the endpoint was removed. With -preStopSeconds set, the gateway makes /status, /healthz and /readyz return 503, keeps serving for that many seconds, then runs the existing graceful shutdown. 0 (the default) keeps the old behavior. The flag is registered for weed s3 and as -s3.preStopSeconds for weed filer, weed server and weed mini. A cancelled weed mini cluster context ends the wait early. The Helm chart passes s3.preStopSeconds and filer.s3.preStopSeconds through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: assert the preStopSeconds chart render under set -e, drop timing from the probe test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: wait before weed mini cancels its clients; size grace periods in the chart weed mini's interrupt hook runs before the S3 one and cancels the S3 gateway's shutdownCtx, so the pre-stop wait never happened there. The wait is now also registered with beforeMiniClientsShutdown, which runs before that cancel, and a sync.OnceFunc keeps the interrupt hook from waiting a second time. Outside weed mini the registration is a no-op. Helm: the s3 deployment fails to render when terminationGracePeriodSeconds is below preStopSeconds + 15. The filer statefulset and the all-in-one deployment, whose grace period is fixed at 60s, add preStopSeconds to it. allInOne.s3.preStopSeconds is passed through, inheriting s3.preStopSeconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: keep 64-bit counters aligned on 386; let allInOne.s3.preStopSeconds=0 disable the wait draining sat ahead of inFlightDataSize and inFlightUploads in S3ApiServer and shifted them off 8-byte alignment on 32-bit builds, so the atomic upload-limit counters panicked there. It now follows them. Helm: allInOne.s3.preStopSeconds inherits s3.preStopSeconds only when null; default treated an explicit 0 as unset. The s3 values and the grace period error now say a lifecycle preStop hook's time comes on top. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * grace: add OnInterruptFirst for hooks that must run before any shutdown Interrupt hooks run in reverse registration order, so a hook cannot run ahead of a service that registers later, such as a volume server that is still loading its volumes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: run the -preStopSeconds wait before every other interrupt hook Under weed server the volume server registers its hook once its volumes are loaded, often after the S3 gateway. Its hook then ran first and shut the volume server down, so the gateway spent the whole wait serving requests that could not read or write objects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
c4e1802e3e |
s3: route ';' queries as authorized, and report an empty s3:prefix to policies (#12702)
* s3: escape raw semicolons before the router matches a route registerRouter applied EscapeSemicolonsInQuery with router.Use, but mux runs router middleware only after it has picked the route. Queries() matchers split pairs on ';', so GET /bucket?uploads;x=1 reached the multipart listing handler while the handler and the IAM resolver read uploads%3Bx=1 and saw no uploads subresource. Wrap the S3 router with the escape so routing and authorization read the same query. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: report a listing without prefix as s3:prefix "" in policy conditions The advanced IAM path already sets s3:prefix to "" when a bucket listing names no prefix, as AWS does, but bucket policies and attached policies read conditions from ExtractConditionValuesFromRequest, which leaves the key out. The AWS home-directory policy that grants s3:prefix ["", "home/"] then denied the root listing, and a Deny keyed on the empty prefix never applied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: test the served handler composition for the semicolon escape Move the escape and the Vectors split into s3Handler so a command test can send ';' queries through the handler the server actually serves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: keep the isVectorsRequest doc comment on its function Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1d30b0de21 |
fix(s3api): complete bucket-listing IAM condition handling (#12699)
* fix(s3api): complete bucket-listing IAM condition handling Populate s3:delimiter and s3:max-keys for bucket. Add regression tests for listing conditions and variants. Assisted-by: gpt-6.1-sol, deepseek-v4.1-flash * fix(s3api): ignore max-keys for multipart listing policies Assisted-by: GPT-6.1 Terra * s3api: drop max-keys from multipart-upload listing policy conditions Bucket policies and embedded IAM policies read condition values through ExtractConditionValuesFromRequest, which still exposed max-keys on ?uploads listings. Those page by max-uploads, so an unused max-keys parameter could satisfy an s3:max-keys condition. Match the advanced IAM path and leave the key absent there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5681be241e |
topology: never report negative free volume slots (#12698)
* topology: never report negative free volume slots Free volume slots are computed as max + remote - volume - ecSlots. When phantom or over-registered volumes push the count past the configured max, the result is negative and rolls up into the cluster total, causing volume grow and assign to fail with a negative 'volumes left' count even while healthy servers have capacity. Clamp every free-volume count at zero so an over-committed server contributes no slots instead of subtracting from the cluster. * topology: drop the issue reference from the free-slot test comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: test free slots next to an over-committed server A server with more volumes than slots must not hide the free slots of the other servers in its rack, data center or topology. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: sum free slots per data node in racks, data centers and topology Clamping the rolled-up counters only clamped their sum, so one server with more volumes than slots still hid the free slots of every other server in its rack, data center and topology. Interior nodes now add up their children's free slots, so each data node counts as zero at worst. A data node in maintenance mode no longer adds slots to its parents either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: test that disk info keeps an over-committed deficit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: keep the signed free volume count in disk info EC encode retries and the balance planners add refunds and planned moves to the published FreeVolumeCount. Clamping it at zero dropped an over-committed disk's deficit, so a refund could make a full disk look free. Volume growth already reads the clamped AvailableSpaceFor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: test placement next to an over-committed server Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * topology: accept either data center order in the placement spread check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8ba33fae71 |
rust volume: bulk-load needle maps on redb 4.6 and compact the old floor (#12695)
* seaweed-volume: expect a fresh redb file to start at 72 KiB The assertion fails on redb 4.2.0, whose empty database is about 1 MiB. * seaweed-volume: bulk-load redb rebuilds with the stable cursor redb 4.6.0 has no experimental_cursor feature. Full rebuilds insert sorted needle ids through insert_before, and a new .rdb starts at 72 KiB. * seaweed-volume: test which redb files are still at the old floor The gate is missing, so the test does not compile yet. * seaweed-volume: decide when a reused redb is still at the old floor Compact only a file above 72 KiB and at most 1 MiB plus 8 KiB, and only when floor_compact has not been recorded. * seaweed-volume: test one-shot compact of an old-floor redb Reuse, shrink, and storage-error recovery are specified. The helpers they call are not implemented, so the tests do not compile yet. * seaweed-volume: compact a reused redb that is still at the old floor One compact runs when the file is above 72 KiB and at most 1 MiB plus 8 KiB. A storage error drops the handle and opens the same path again so the volume does not keep a PreviousIo latch. * seaweed-volume: publish a redb replay before compacting the old floor compact() is what would make a Durability::None tail durable, and a storage error then drops that tail on reopen. An in-band file is committed first. The floor_compact flag is read only inside the band, so a meta error cannot rebuild a larger index. * seaweed-volume: drop the redb experimental cursor CI step redb 4.6 has no redb-experimental-cursor feature. The needle-map tests that used it now run in the normal cargo test step. * seaweed-volume: drop the experimental-api-5 notes on redb reads redb's experimental-api-5 can no longer be enabled, so these reads always use the inherent ReadOnlyTable::get. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * seaweed-volume: drop the always-false flag from the old-floor band check Both callers pass false and read floor_compact on their own, so the band check only needs the file length. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d8a47042c5 |
master: unregister a volume server whose heartbeats stop (#12697)
* master: unregister a volume server whose heartbeats stop A volume server sends a heartbeat every 5 s. When its host hangs or its network drops packets, its SendHeartbeat stream stays open, and the master unregisters it only once gRPC keepalive closes the connection, 60 s plus 20 s after the last frame. Until then the master keeps assigning writes to volumes with a copy on it, and those writes hang. SendHeartbeat now ends the stream when no heartbeat arrives for three pulses (15 s), which unregisters the server through the existing deferred path and tells clients its locations are gone. The server reconnects and registers again once it can reach the master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * master: test that clients learn a silent server's volumes are gone Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * master: fail the silent-stream receive test instead of hanging Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
dbb2fdb93b |
pb: recover when a local gRPC socket file is deleted (#12694)
* pb: fall back to TCP when the local gRPC socket cannot be dialed A temp cleaner can delete a local gRPC socket file while its server keeps running. Every later dial to that server then failed with "no such file or directory", so the master could no longer reach its own volume server and volume growth stopped. Dial the server's TCP address when the socket dial fails, as the S3 gateway already does for the filer's HTTP socket. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: recreate a local gRPC socket whose file was removed Once its socket file was deleted, a server kept accepting on an unreachable listener until restart, so same-host RPCs stayed on the TCP fallback. Check the path every minute and, when the file is gone, close the old listener and bind the socket again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: retry a failed local socket rebind on the next check A failed rebind ended the watcher, so the socket stayed missing after the cause cleared. Bind the replacement before closing the old listener: a failed bind keeps the old listener serving and is retried on the next check, and the old Serve still reports server shutdown. The old listener no longer unlinks the path on close, since the replacement now owns it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * pb: set the local socket unlink policy before serving the listener Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b74971b6d4 |
server: add -grpc.localSocketDir for the local gRPC sockets (#12693)
* server: add -grpc.localSocketDir for the local gRPC sockets `weed server` and `weed mini` hard-coded each local gRPC Unix socket under /tmp. systemd-tmpfiles ages /tmp after 10 days and macOS after 3, so a long-running server loses its socket, and RegisterLocalGrpcSocket keeps routing local dials to the missing path: the master can no longer reach its volume server. Add -grpc.localSocketDir to both commands, defaulting to /tmp so an existing setup is unchanged, and build every socket path from it with filepath.Join. A folder that does not exist or is not a directory is refused at startup rather than at the first local RPC. A folder whose socket paths exceed the platform's sockaddr_un limit is refused too, because every bind then fails with EINVAL while the server keeps running and each local RPC dials a socket that does not exist. Refs #12692 * server: refuse a -grpc.localSocketDir the server cannot bind in An existing folder the server's user cannot write to passed the startup check, and then every socket bind failed: ListenOnLocalSocket only logs the error while GrpcDial keeps routing local dials to a socket that was never created. After the path-length check, create and remove a probe Unix socket in the folder, and refuse startup with the bind error if that fails. * server: give the -grpc.localSocketDir probe a random name A probe named only by the process id could be precreated by another user in a shared /tmp, or left by a crashed process whose pid was reused, and the server cannot remove such a file, so startup failed although the real socket paths were free. Name the probe with 8 random bytes, retry a name that is taken, and rely on the listener's close to unlink it. * server: keep the -grpc.localSocketDir probe name within the service socket lengths The probe was 5 bytes longer than the master and volume socket names, so a folder whose service sockets fit the sockaddr_un limit could still be refused. A 23-byte probe name is shorter than any service socket name, and a process with no local sockets skips the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * server: trim the -grpc.localSocketDir comments and drop issue references Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brent Pease <179639673+brentDSL@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
95f46c8fff |
admin: manage filer webhook targets from the UI with live reload (#12690)
* webhook: share the default target settings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * webhook: managed webhook targets with a drainable queue Targets kept as JSON (name, endpoint, token, filters, retry and worker settings) build their own queue; Close waits for queued and in-flight deliveries before stopping it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: run webhook targets stored in /etc/seaweedfs when notification.webhook.managed is set The targets in /etc/seaweedfs/notification_webhooks.json start with the filer and are rebuilt from the store whenever the file changes on any filer, alongside notification.toml. The previous queue drains before it stops. Events for the file itself, which holds bearer tokens, are not sent to notification queues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: manage filer webhook targets A Webhooks page under Management lists, adds, edits and deletes the targets kept in /etc/seaweedfs/notification_webhooks.json. Bearer tokens are write-only: the API reports whether one is set, and an empty token on edit keeps the stored one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep the webhook queue when the stored targets did not change Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep a replaced webhook queue open until its senders hand over Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: read chunk-backed webhook targets before editing them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: keep the stored webhook targets out of fs.meta.notify Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * webhook: reject negative managed target settings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * webhook: no signal handler on queues that reloads replace Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: swap the webhook queue without waiting behind blocked senders The read lock held across SendMessage made a reload wait for any sender blocked on a saturated target, and the waiting writer stalled every other filer write with it, which is when an operator disables that target. Close now refuses new sends and waits for the senders already inside SendMessage, and a send that hits the closing queue retries on its replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * webhook: release blocked test handlers before the server shuts down Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: always run the webhook targets managed in the admin UI Drop the notification.webhook.managed switch: every filer loads and live reloads the stored targets. Without stored targets no queue is created. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: place the admin-managed webhook note above enabled Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: separate the admin-managed webhook note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: note that admin-managed webhooks follow the notification queue Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: retry loading webhook targets after a failed read Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: retire the admin-managed webhook queue on shutdown Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
386385101d | docs: regenerate star history chart | ||
|
|
6342876392 |
ci: run the Python 3.9 jobs on 3.10 (#12691)
* ci: run the Python 3.9 jobs on 3.10 ubuntu-latest now resolves to Ubuntu 26.04, which has no Python 3.9 build for actions/setup-python, so the Ceph S3 tests and the PyArrow Parquet 3.9 job fail during setup. 3.10 is the oldest version built for 26.04. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: list the Python versions the parquet CI runs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
40182a68bc |
admin: keep the dark theme from flashing light on reload (#12689)
* admin: apply the saved theme inline at the top of the page head The theme script was an external file, so a reload had to fetch or revalidate it before the root got its theme, and the page could paint with the light default in the meantime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: set color-scheme on the root together with the theme Until the stylesheets load, the browser paints the page canvas from the root's color-scheme, which only the stylesheets set, so a dark theme showed a white canvas for the duration of every reload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
eed00c3737 | docs: regenerate star history chart | ||
|
|
b449301a4e |
ec: mount shards through the encode's .vif when a source volume's .vif shares the disk (#12676)
* ec: mount shards against the index .vif when the local one has no EC config A worker encode can send shards to a disk that still holds the source volume's .vif while the encode's .vif lands with the index on a sibling disk. The shard disk mounted through the source's config-free .vif, so it reported encode time 0 and fell back to the sidecar layout, and reads carrying the encode time were rejected as a different encode run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): mount shards against the index .vif when the local one has no EC config Mirrors the Go mount: a data-dir .vif that records no EC config yields to the idx dir's when that one records it, and load_vif_info_across_dirs reports the directory the .vif actually came from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: do not mirror EC sidecars onto a disk whose .vif has no EC config Startup mirroring copied .ecx/.ecj next to a source volume's leftover config-free .vif and kept that .vif, so after a restart the shard disk mounted self-contained with encode time 0. Leave such a disk on the cross-disk mount, which reads the owner's .vif. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): do not mirror EC sidecars onto a disk whose .vif has no EC config Mirrors the Go startup mirror: a shard disk holding a source volume's config-free .vif stays on the cross-disk mount, which reads the owner's .vif, instead of mounting self-contained with encode time 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: take the encode .vif from a sibling data dir when the index dir is shared With a separate -dir.idx the .ecx is shared by every disk, but the encode's .vif lands in the data dir of the disk that received it. A disk still holding a source volume's config-free .vif mounted through it at generation 0. NewEcVolume now accepts the server's data dirs and takes a sibling's EC .vif when the index is not local and every sibling EC .vif names the same run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: give EC mounts the data dir of every disk on the server NewStore records every data dir on each DiskLocation before loading, so the startup scan, cross-disk reconcile and MountEcShards all pass them to NewEcVolume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): take the encode .vif from a sibling data dir when the index dir is shared Mirrors the Go mount: EcVolume::new_with_vif_dirs takes a sibling's EC .vif when this disk's own records no EC config, the index is not local, and every sibling EC .vif names the same run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): give EC mounts the data dir of every disk on the server main sets Store.store_dirs before the first add_location, so a disk loaded before its siblings still sees them; each DiskLocation passes them to EcVolume::new_with_vif_dirs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: size startup EC validation by the sibling encode .vif With a shared -dir.idx, a disk that still holds the source .dat and its config-free .vif read the default data-shard count, judged custom-ratio shards an interrupted encode, and removed them along with the shared .ecx before the mount could take the sibling's .vif. The startup lookup now falls back to the same sibling .vif the mount uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): size startup EC validation by the sibling encode .vif Mirrors the Go change: validate_ec_volume falls back to the sibling data dir's encode .vif before judging shards beside a source .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: keep EC shards when sibling .vif files disagree on the encode SiblingEcConfigVif reports when a sibling .vif cannot be read or the siblings name different encode runs. The startup data-shard lookup then returns 0 instead of the default ratio, and validateEcVolume keeps the files rather than sizing custom-ratio shards by a ratio they do not use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): keep EC shards when sibling .vif files disagree on the encode Mirrors the Go change: sibling_ec_config_vif returns Err for an unreadable sibling .vif or disagreeing runs, ec_data_shards_from_vif returns 0 then, and validate_ec_volume keeps the files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: release the opened EC shard when its volume cannot be built Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9e5357782e |
volume: keep reads over the download limit local when readMode is local (#12679)
* volume: keep reads over the download limit local when readMode is local With -readMode=local the server never sends a read for a volume it does not hold elsewhere, but a read over -concurrentDownloadLimitMB on a replicated volume was still redirected to another replica. Skip the replica fallback in local mode, so the read waits for a download slot and answers 429 when the wait times out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): keep reads over the download limit local when readMode is local Mirror the Go volume server: with readMode local, a read over the download limit waits for a slot instead of being redirected to a replica. proxy_or_redirect_to_target no longer needs the local-mode redirect switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume: check download capacity under the wait lock waitForDownloadSlot read the in-flight size outside the cond lock and the read handler broadcast without it, so a download finishing between the check and the wait was missed and the read waited out the timeout for a 429. Check capacity with util.WaitUntil under the lock and broadcast under it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): register for the download notification before checking capacity The wait loop created its Notified future only after the capacity check, so a body dropped in between had its notify_waiters() lost and the read waited out the deadline for a 429. Create the future first, then check, as Go now checks under the cond lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
f4e638fa1d |
Fix/admin cluster submenu (#12677)
* admin: fix cluster submenu toggle * admin: update embedded JavaScript * admin: drop the object store submenu handlers The sidebar no longer renders an object store submenu, so its expand, link and toggle handlers match nothing; the toggle handler repeated the manual collapse that fought Bootstrap on the cluster submenu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: ivan <i.stepanov@navigine.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1e31c6dbf5 |
test(pjdfstest): check the sudo'd mount with ps, not kill -0 (#12673)
sudo switches its real uid to root while it starts, so kill -0 from the invoking user fails with EPERM even though the mount process is alive, and the wait loop reported "weed mount exited early" a few milliseconds after starting it. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0f0f3811bf |
filer: serve caught-up aggregated subscribers from memory, not a held disk pass (#12674)
* filer: serve caught-up aggregated subscribers from memory, not a held disk pass An aggregated SubscribeMetadata stream that does not take metadata chunks (filer.sync, filer.backup, filer.meta.backup, filer.meta.tail, the filer.remote.* syncs, the S3 gateway's subscriptions, and mounts that predate chunk support) re-ran the persisted-log pass whenever its cursor moved: after every memory hold at the delivery watermark and on every 2 s re-probe. The pass holds at the peers' flush low-watermark. Peers flush on their own one-minute schedules, so on a cluster that keeps writing some peer has always flushed past a caught-up cursor while the slowest has not: the pass held at its first entry, the loop came back to it, and memory was never read. The stream got each change only once the slowest peer's flush covered it, in bursts up to a flush interval late, and every pass listed and re-read the newest log files. When the aggregated ring was already filling at the cursor and evicted nothing after it, memory holds everything the persisted logs could add, since every peer's log reaches the ring through the aggregator. Skip the pass and read memory, held at the delivery watermark as before. The skipped pass proves nothing past the cursor, and the next-day scan for an empty pass does not apply. An empty ring, a ring that starts past the cursor, and a cursor memory just refused still take the pass. Chunk subscribers already skip it for a caught-up cursor: their ref listing stops below the flush hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: keep a new peer's pre-start history on the disk pass The ring only receives a peer's events from the start of its subscription. A peer followed for the first time after this filer started (a filer added to the cluster, a restarted pod with a new address, or the late-arriving peers bootstrap) starts at its notice time, so what it logged before then is only in the persisted logs. A cursor below that start passed the ring check and skipped the disk pass, losing those events for subscribers that do not take metadata chunks. Record the newest start of a peer subscription that had no resume point and send cursors below it through the disk pass as before. Peers that rejoin with a resume point (a restart under the same address) do not move it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |