mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-16 08:45:22 +01:00
d5b4e1352eb818e0d56060fc8fdcb8272a7c2c14
10637
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5ef99c1d8d | 4.54 | ||
|
|
8e15110b4f |
filer: wake metadata subscribers only for directories they can match (#12669)
* log_buffer: let a subscriber skip wake-ups for unwanted partition keys Every append woke every registered subscriber. SetSubscriberWakeFilter lets a reader sleep through appends whose partition key it would filter out; it still reads them on its next poll. Flushes and keyless appends wake everyone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer_pb: decide subscription matches from an event's directories MetadataDirectoryMayMatchSubscription rules out events whose directory no child path of can match the subscription, and ScanMetadataEventDirectories reads an event's directory and new parent path without allocating. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: wake subscribers of a move's destination directory Appends are keyed by the source directory, so a filtered subscriber of the destination would otherwise only see the move on its next poll. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: wake metadata subscribers only for directories they can match Every metadata event woke and was scanned by every subscriber, so filer CPU grew with the number of subscribers watching other paths. Readers now register a directory wake filter, and the per-entry prefilter rules entries out from their directories before the skeleton scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test that moves into a subscribed prefix pass the prefilter Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
df30c883fe |
s3api: keep a multipart upload's state on the object's owner filer (#12668)
* s3api: create a multipart upload's directory on the object's owner filer CreateMultipartUpload wrote .uploads/<id> to the gateway's own filer while the upload's completion commits on the object's write owner. With independent filer stores, a part or completion reaching another gateway before peer metadata replication caught up could not see the upload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: look up a multipart upload on the object's owner filer UploadPart, UploadPartCopy, ListParts, CompleteMultipartUpload, AbortMultipartUpload and the bucket policy SSE lookup read .uploads/<id> from the gateway's own filer, so a request on another gateway shortly after CreateMultipartUpload answered NoSuchUpload until replication caught up. Read it preferring the object's write owner, like object reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: write multipart parts to the upload's owner filer UploadPart and UploadPartCopy stored part entries on the gateway's own filer, so ListParts and CompleteMultipartUpload on another gateway could miss a part that replication had not delivered yet. Store them next to the upload directory on the object's write owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: list and clean up a multipart upload on its owner filer ListParts, CompleteMultipartUpload and AbortMultipartUpload listed and removed part entries on the gateway's own filer. Do it where the upload and its parts now live, so completion sees every part and cleanup does not miss entries replication has not delivered yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: test a multipart upload against a lagging gateway filer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: check the upload's owner before freeing a failed part's chunks A part write that the owner stored but answered with an error went to recovery with no object, so the owner was never among the filers checked. When the owner is outside the gateway's filer list, the others' not-found answers freed chunks the stored part still references. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: list an upload's parts on the prior owner after a ring change The upload record lookup already falls back to the key's prior owner while a ring change is in its cooling-off window, but the part listing read only the new owner. ListParts then reported no parts for an open upload. Fall back the same way when the new owner has nothing yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: find the upload record on the prior owner past a bare directory A part routed to a new owner that has not received the upload yet makes that filer create .uploads/<id> without the upload's key. The post-write check then read it as an aborted upload and removed the part. While the ring change is in its cooling-off window, a record on the prior owner now wins over such a directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: merge the prior owner's parts into an upload listing During a ring change's cooling-off window the new owner may have received only some of an upload's parts. Listing returned the new owner's parts alone once it had any, so ListParts dropped the rest and a completion naming them failed with InvalidPart. Both owners' pages are now merged, the owner's entry winning a shared name, and cut to the page limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: fail an upload listing the prior owner could not answer When the new owner has none of an upload's parts yet and the prior owner's listing fails, the parts may all be on the prior owner. That listing read as empty, so CompleteMultipartUpload answered NoSuchUpload for an upload that still exists; it now returns the error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: fail an upload lookup the prior owner could not answer During a ring change's cooling-off window an upload the new owner has not received yet, or holds only as a bare directory, read as NoSuchUpload when the prior owner's lookup failed. That failure is now returned, so the client sees a retryable error instead of an upload that is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b2dd93eac8 |
admin: support [maintenance.s3_lifecycle] in admin.toml (#12661)
* admin: support [maintenance.s3_lifecycle] in admin.toml
The admin.toml overlay covers vacuum, volume_balance and erasure_coding but
not s3_lifecycle, so the bucket lifecycle job's settings can only be changed
from the web UI and are lost whenever the admin data directory does not
persist. That is the worst combination for this job type: its
meta_log_retention_days knob decides whether bucket rules ever apply to
objects written before the job was enabled, and silently reverting it to the
default makes the rules no-op without any error.
Wire the four admin-side knobs into the existing overlay:
meta_log_retention_days retention window the meta-log replay can reach
walker_interval_minutes minimum gap between full bucket walks
cluster_deletes_per_second cluster-wide delete budget
cluster_deletes_burst token-bucket burst for that budget
All four are admin config values, not worker values: the engine partition is
computed on the admin side and the delete budget is divided across
execute-capable workers at dispatch time. enabled/retry_limit/
retry_backoff_seconds come for free from the shared section handling.
Tested with TestApplyPluginConfigFromTomlS3Lifecycle, which also pins the
retention=0 case: 0 means unbounded and must survive as an explicit 0 rather
than being treated as unset.
* admin: fix meta_log_retention_days guidance in admin.toml scaffold
Both reviewers caught a real error. `0` is not "unbounded": runShard does
retentionWindow := cfg.RetentionWindow
if retentionWindow <= 0 {
retentionWindow = maxTTL
}
so zero falls back to the longest rule TTL, which keeps every replay rule in
replay and promotes nothing to the walk set. Recommending 0 to force full
scans was wrong, and so was the claim that pre-existing objects are only
reachable through the walk set -- the cold-start recovery walk covers them
regardless of the retention value.
Describe zero as "no retention info supplied" and tell operators to set the
window the filer meta-log actually retains. The code change is unaffected:
it only wires the four keys into the existing overlay.
* admin: tie s3_lifecycle walk-set scans to walker_interval_minutes in admin.toml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* admin: show the s3_lifecycle retry defaults in admin.toml
The s3_lifecycle descriptor sets no retry limit or backoff, so a failed
run is not retried and the scheduler's 5 s backoff applies; the scaffold
listed the vacuum values instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* admin: shorten the s3_lifecycle toml section comment
The retention and walker knobs are read by the worker from the admin
values, not computed on the admin side.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* admin: cover cluster_deletes_burst in the s3_lifecycle toml test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* admin: check the s3_lifecycle retention zero is a stored int64
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
5f7fefadae |
s3 lifecycle: write the suspended expiration marker before deleting the null version (#12664)
* s3 test: evaluate mutation conditions and serve DeleteEntry in the fake filer The fake ObjectTransaction now checks each mutation's own condition before applying it, as the filer does, and reports the failed mutation. ETag clauses read the stored ETag attribute first, like the filer. DeleteEntry removes the entry so unrouted delete paths can run against the fake. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: write the suspended expiration marker before deleting the null version A suspended-versioning expiration deleted the null version first and then wrote the delete marker. When the pointer read or the marker write failed in between, the retry found no null version, resolved as NOT_FOUND or VERSION_NOT_LATEST, and never wrote the marker, so an older version became the latest. The marker now goes first. The latest pointer outranks the null object, so the key reads as deleted once the marker lands, and a null delete that fails after it leaves a noncurrent null version rather than exposing an older one. The null delete is pinned to the entry read before the marker: on the owner it carries IF_ENTRY_EQUAL on that read, and off the ring the entry is re-read and compared by identity first. A null object written after the read is a newer write and is no longer deleted. When the null version is the one that passed the expiration checks, a missing or changed entry at that read resolves the request as STALE_IDENTITY before anything is written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: retry a failed routed null delete instead of deleting on the default filer A routed null delete that failed fell through to the unrouted path, which reads and deletes on the default filer. That filer may not have the owner's newer null object yet, so the fallback could delete it without the owner-side IF_ENTRY_EQUAL guard. The failure now returns RETRY_LATER; the marker is already written, so the key keeps reading as deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: recheck and delete an unrouted null version under the object write lock Without a ring owner the identity recheck and the null delete were separate calls, so a suspended PUT landing between them was deleted. Off the ring writers take the object write lock, so the recheck and the delete now run under it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 test: strip quotes from the stored ETag in the fake filer The filer normalizes both sides of an ETag comparison; the fake now strips quotes from the stored ETag attribute as well as from the candidate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: retry the null delete when the object lock lookup fails on a routed key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: check the suspended marker against the validated version A suspended expiration that validated a named version wrote its marker without comparing the pointer, so a null PUT landing after the checks was hidden, and deleted too when it landed before the null read. The marker now compares the latest the way the enabled path does and resolves the request as VERSION_NOT_LATEST. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 test: count the marker's object write lock in the unrouted null delete Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: recheck the attempt deadline after reading the null version Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c09009a74f |
s3: accept the governance bypass header case-insensitively (#12667)
* s3: accept the governance bypass header case-insensitively botocore and the AWS CLI send x-amz-bypass-governance-retention: True, so the exact "true" match ignored the bypass and governance-locked deletes and retention changes failed with AccessDenied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: test governance bypass with SDK-style header values Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c0509cf499 |
s3api: evaluate each policy condition against its own values (#12665)
* s3api: normalize a nil policy value to no values Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: evaluate each policy condition against its own values The process-wide normalized value cache keyed entries by fmt.Sprint of the value, so ["a", "b"] and ["a b"] (or [] and [""]) shared one entry and a condition in one policy could be compared against another policy's values. Normalizing is cheaper than building that key, so drop the cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: test that look-alike condition values stay distinct Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
63806b7073 |
s3 lifecycle: bound each LifecycleDelete attempt with its own deadline (#12646)
* s3 lifecycle: bound each LifecycleDelete attempt with its own deadline Replay and walker deletes used the job context with no request deadline, so one stalled unary RPC could hold a shard for the whole job. Each attempt now runs under DeleteTimeout (default 1m), set after the limiter wait. A timeout while the job is still live is retried like a transport error and then halts the shard or walk without advancing; parent cancellation still stops both the attempt and the backoff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: admin setting for the delete RPC timeout Adds delete_timeout_seconds to the S3 Lifecycle admin form and passes it to both the replay drain and the walker. 0 keeps the 60s default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: describe replay and walker timeouts separately in the help text Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: fail the pass when a replay delete exhausts its attempts A transport error or attempt timeout that outlived the retries halted the shard with a nil error, so the job reported success. processMatches now returns the error (the cursor still holds at the event), and runShard treats a drain error as a clean stop only when its own context ended, since an attempt timeout also wraps DeadlineExceeded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: test that a replayed LifecycleDelete resolves after a lost reply Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: skip the expiration write once the attempt deadline passed A worker retries a LifecycleDelete whose attempt timed out, while the first attempt can still be running on the S3 server. An attempt that stalled before its marker write would then stack a second delete marker after the retry wrote one. The gRPC deadline reaches the handler context, so check it right before the expiration writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: recheck the attempt deadline after the suspended null delete The null-version delete can block on the owner filer past the attempt deadline; the worker may have retried and written a marker by then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: finish the suspended marker once the null version is deleted Skipping the marker after a successful null delete left an older version as the latest, and the retry then resolved as a no-op without hiding it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: let a routed delete marker commit only while its latest guard holds createDeleteMarkerIf passes an optional condition, evaluated on the .versions directory entry under the object's lock on the owner filer, and reports a refused commit as errLatestVersionChanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: write the expiration marker as a compare-and-set on the latest pointer A timed-out attempt can still be running on the S3 server while the worker's retry writes the marker. The marker commit is now conditioned on the .versions pointer as read just before it and skipped when the latest is already a delete marker, so whichever attempt commits second writes nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: resolve the marker as stale when another version became latest The marker's compare-and-set only covered changes after its own pointer read, so a version promoted after the expiration checks passed could be hidden by the marker. The enabled path now passes the validated version and a conflict retry that finds a different named latest resolves the request as VERSION_NOT_LATEST. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: resolve a null expiration as stale once a version takes the pointer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: read the marker's latest version the way getLatestObjectVersion does A pointer names a version only with its file name; an id alone falls back to the null object, and the null-is-latest signal makes the null version current. An expiration that validated a named version no longer hides a null written by a suspended PUT, and a current null under an id-only pointer now expires instead of resolving as stale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: hold the object write lock for an unrouted expiration marker Off the ring createDeleteMarkerIf cannot carry the pointer guard, so a PUT committing between the pointer read and the marker write was hidden. The read and the write now run under the object write lock that unrouted writers take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: share transient-error retries between walker and replay deletes (#12649) * s3 lifecycle: retry only transient LifecycleDelete failures dispatchWithRetry retried every RPC error except parent cancellation, so an authentication, permission or argument error spent the whole retry budget before surfacing. Retry only attempt timeouts and the failures util.IsTransientError recognizes (Unavailable, ResourceExhausted, network resets); everything else returns on the first attempt. A nil response is now an error instead of a nil dereference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: take a delete-rate token for every LifecycleDelete attempt processMatches waited on the shared limiter once per match, so the retries behind it ran outside the cluster-wide deletes/sec budget. The retry loop now waits on the limiter before each attempt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: give walker deletes the replay retry policy WalkerDispatcher sent each LifecycleDelete once, so one dropped connection aborted the bucket's daily walk while the same failure on the replay path was retried. The walker now goes through deleteWithRetry and shares its transient-error retries, per-attempt limiter wait and nil response handling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: retry a DeadlineExceeded the server sends back A server or proxy can end the RPC with DeadlineExceeded while the attempt's own deadline is still running. util.IsTransientError rejects that status, so the delete halted with retries left. It is a timed-out attempt like the client-side one, so retry it the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: say walker deletes get 3 attempts in the timeout help Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
504de9b562 |
security: refuse to start when per-target gRPC server allowlists are not applied (#12662)
* security: refuse to start when per-target server allowlists are not applied A component that dials every server with one option ignored <target>_allowed_server_* keys, so a config that only set those accepted any CA-signed server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: say which components scope server allowlists per target Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * security: also catch per-target server allowlists set through WEED_ env vars Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d80e905fc7 |
security: optional dialed-host check for gRPC server certificates (#12663)
* security: optionally check the dialed host against gRPC server certificates verify_server_host under [grpc] or a [grpc.<component>] section switches the client to certificate-and-host verification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * scaffold: document verify_server_host Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
799fd10342 |
s3 lifecycle: fail over between S3 fronts during a pass (#12647)
* s3 lifecycle: add a LifecycleDelete client that fails over between S3 fronts When the current front returns Unavailable, the client closes it, marks it failed for the rest of the pass, picks the next front that passes checkLifecycleReady and resends the same request there. Callers cut off by that switch closing their connection resend on the new front. When no front is left, every later call returns one error naming the fronts tried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: serve the pass through the failover client Execute hands the replay drain and the walker one failover client built on the selected front, reports each switch as job progress, and closes the current connection when the pass ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: keep failing over until a front serves the resent delete The resend after a switch went straight to the new front, so a second lost front ended the call even with a healthy third one, and the walker has no retry of its own. Resend through the failover loop instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: do not recheck fronts skipped when the pass started The failover client started with no failed fronts, so losing the selected front checked the fronts selectS3Front had already skipped again, holding every delete for another readiness timeout per front. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a91b333658 |
s3 lifecycle: persist daily walk checkpoints and resume them safely (#12650)
* s3 lifecycle: walk checkpoints take each entry's resume position Versions of one object share a Path, so a checkpoint set from Path after the first version made a resume skip the rest. The ListFunc now gives each entry the start value that continues after it, and leaves it empty on all but an object's last version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: resume a bucket listing where the walk stopped The filer listing skipped entries with key <= start, but its order is not key order: each directory emits its .versions groups before its files, and a directory's children come before a sibling like dir.txt. Positions are now one (pass, name) frame per directory level, so a resume seeds each listing and skips exactly what was finished. A version group is one unit, and a skipped group still keeps its bare null version out of the second pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: content hash of a walk view Hashes every non-disabled action of a view and the versioning of their buckets, so a saved walk position can be tied to the walk it came from: another rule set, another walk mode, or a bucket that turned versioned gives another hash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: persist daily walk progress and resume it A walk that failed or was cut short started over on the next pass, listing again every bucket it had already finished. WalkBuckets now keeps the finished buckets and each unfinished bucket's position in a checkpoint file next to the shard cursors, one per shard set, and the next walk of the same view and shards resumes from it. The record is saved every 30s and when the walk stops, and cleared when a walk completes, so the next walk is full again and sees keys added behind the position; a clear that fails fails the walk rather than leave a record that would skip buckets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a stuck resumed walk starts over A bucket that fails at the same place on every pass kept the walk from completing, so the buckets it had finished were never walked again and their due objects waited forever. A resumed walk that fails again without finishing anything clears its checkpoint, and the next walk is full. A canceled walk keeps it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: do not resume a walk checkpoint left for a week A checkpoint is cleared only by a later walk of the same shard set, so a set that stopped walking, such as a one-off per-shard walk, left its record behind, and a walk of that set much later would skip buckets finished long before. Records now carry their save time and are not resumed after seven days; a walk that keeps retrying saves on every attempt and stays fresh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a failed walk without progress leaves its checkpoint alone A pass canceled while it read the checkpoint failed the read, then saved its own empty progress over the record it never loaded, so the next pass started every bucket again. A failed walk that finished nothing has nothing new to record and no longer writes; the stuck resume reset still clears. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: space out walk checkpoint saves after a failed one The save time advanced only on success, so once the filer refused a periodic save every following entry tried again and logged another warning. The interval now counts from the last attempt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a walk checkpoint save that errors may still be on disk A save whose reply was lost left onDisk false, so a walk that then completed skipped the clear and the next walk resumed from the stale record instead of walking everything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: keep a stalled walk checkpoint for a day A resumed walk that failed again without progress dropped its record right away, so an outage spanning two attempts restarted the whole walk. Treat the walk as stuck only once its record has not advanced for a day. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a resumed walk counts its earlier buckets as walked Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: replay a resumed recovery walk from the walk that began it A recovery or cold-start pass rewinds replay to runNow - maxTTL. When the walk resumed a checkpoint, its finished buckets were listed in an earlier attempt, so an object there that was not yet due then and whose write is now older than maxTTL was neither expired by the walk nor replayed. The checkpoint now keeps the start of the walk that began it, WalkBuckets returns that time (its own start for a fresh walk), and runShard floors the recovery rewind and the cold-start replay at it. A record without a start is not resumed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a cold start that resumed a walk replays on the next pass The pass's shared subscription starts at runNow - maxTTL for a shard without a cursor, before its walk reports an earlier start, so flooring the cold-start drain at the walk start could not deliver the older events. Such a shard now saves its cursor at the walk start minus maxTTL and returns, like the recovery branch, and the next pass replays the window from that cursor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: test that a cursor below the floor opens the next subscription Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: test the cold-start replay across two Run passes A cold start whose walk resumed an earlier attempt sees an older event only on the next pass. The test runs Run twice against a filer that serves events past the requested floor, with a later in-window event that would otherwise move the cursor past the older one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4f70ad015e |
mount: fail the write when the swap file chunk cannot be created (#12655)
* mount: fail the write when the swap file chunk cannot be created #7957 added a nil check after NewSwapFileChunk so a failed swap file creation returns an error instead of panicking. The result was assigned to pageChunk, a PageChunk interface, before the check, and an interface holding a nil *SwapFileChunk is not nil, so the check never fired: SaveDataAt stored the nil chunk and panicked in WriteDataAt. Check the *SwapFileChunk before converting it. The new test points the swap directory below a regular file; on master it panics in SwapFileChunk.WriteDataAt. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * mount: grow the file only after its write is buffered Write set the new size, times, pending append offset and uncommitted bytes before AddPage. When AddPage failed (now that a swap file chunk that cannot be created returns an error), the write was reported as failed but the larger size stayed on the handle's entry, so the next flush published it with a hole where the data should be. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report a partly buffered write as a short write PageWriter.AddPage fills one chunk at a time, so when a later chunk cannot be created the earlier chunks already hold their bytes. Return how much was buffered, and let Write grow the size and times over that prefix and report it, instead of failing the whole write while the prefix still reaches the next flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: report the bytes a failed copy_file_range already buffered A streaming copy whose page fails partway returned an error even though earlier pages, or the start of the failing one, were already buffered and would reach the next flush with the destination size left short. Count the buffered bytes, grow the destination over them, and return them as a short copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: free the copy test's destination pages The buffered memory chunk counts toward the process-wide memory chunk ceiling, so repeated runs of the test pushed later ones onto the unusable swap directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: count only the bytes a swap chunk stored A swap chunk whose file write fails stores a prefix or nothing, but SaveDataAt returned no error and AddPage counted the whole span as buffered. SaveDataAt now reports the short write, and AddPage passes the stored count up so a write is never acknowledged past it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fail a partly buffered writeback write Kernel writeback ignores the count in a WRITE reply, so a short write there drops the unbuffered rest without an error. A write flagged as page-cache writeback now keeps its buffered prefix but returns the error, which the kernel reports on fsync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b067266e6b |
shell: fix s3.circuitBreaker help examples and error messages (#12657)
* shell: fix s3.circuitBreaker help examples and error messages The help examples used -type count, but -type accepts Count and MB (as its flag description says), so both examples failed with "type not valid, only 'count' and 'bytes' are allowed", an error that names neither accepted value. Two other errors pointed at flags that do not exist: -max and -counts, where the flag is -values. Use -type Count in the examples, name Count and MB in the type error, and -values in the value errors. TestCircuitBreakerHelpExamplesRun runs every help example and fails on master for the two -type count lines. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * shell: name the bad action and the -actions flag in s3.circuitBreaker The unknown-action error pointed at a nonexistent -action flag and echoed the whole -actions list instead of the action that was rejected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b3a58506aa |
filer: report a SQL listing that fails partway through (#12654)
* filer: report a SQL listing that fails partway through ListDirectoryPrefixedEntries never checked rows.Err() after its rows.Next() loop. When the result set failed partway, for example on a lost connection, rows.Next() returned false and the listing returned the entries read so far with a nil error, so callers saw a shorter directory instead of an error. Check rows.Err() once the loop has run to the end, as the scan in abstract_sql_store_scan.go already does. A listing the callback stopped on purpose is left alone: rows.Err() would return a canceled request context's error there. This covers every store built on AbstractSqlStore (mysql, mysql2, postgres, postgres2, sqlite). Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * filer: wait for the canceled SQL listing to release its rows in the test A fixed sleep let the callback return before database/sql closed the rows, so a listing that always consulted rows.Err could still pass. Wait, bounded, until the connection is back in the pool instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b42993d24a |
shell: make TestCircuitBreakerShell compare against the expected configs (#12656)
The test unmarshalled the command's output into both actual and expect, so it compared the output with itself and the expected JSON in each case was never checked; it also skipped the first case. Compare every case's output with its expected config as S3CircuitBreakerConfig messages (the output includes default fields such as "enabled": false that the expected JSON leaves out), and restore LoadConfig when the test ends. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> |
||
|
|
fd4603d657 |
s3 lifecycle: count filer entries a walk reads but skips as progress (#12658)
* dailyrun: count entries read from the filer, skipped ones included A versioned bucket's pre-pass and a resumed walk read entries without emitting them, so a walk can read for hours with no visited entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: count listed entries as progress work Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
53377bcb5d |
mount: keep a handle dirty when a write lands during its flush (#12652)
A flush uploads the dirty pages without the handle lock, then commits the entry under it. A write in between grows the entry's size, but its pages are not in the committed chunks. Clearing dirtyMetadata then made the next flush skip the commit, leaving a file with the new size and no chunk for the write, which reads back as zeros. Count page writes on the handle and clear dirtyMetadata only when none landed since the flush started uploading. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
1167ca8cf5 |
vacuum: report a request dropped because another vacuum is running (#12648)
* vacuum: report a request dropped because another vacuum is running The master runs one vacuum at a time. A request that arrives while another one is running, the automatic vacuum or an earlier request, is dropped: Topology.Vacuum logs "Vacuum is already running" and returns. The VacuumVolume RPC and /vol/vacuum then answered as if the vacuum had run, so volume.vacuum printed nothing and exited 0 with no volume compacted. Where the automatic vacuum is busy for hours, an explicit volume.vacuum -volumeId could do nothing every time, and nothing said so. Topology.Vacuum now returns ErrVacuumRunning for a dropped request. The RPC returns it and /vol/vacuum answers 409 with it. volume.vacuum names the request that failed, because a list of volumes or collections is sent as several requests and a vacuum that starts in between fails only the later ones. Requests are still dropped, not queued, and the automatic vacuum ignores the error as before. The integration tests that need their /vol/vacuum request to run ask again while the master answers 409. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * docs: note the 409 answer from /vol/vacuum while another vacuum runs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
512f898600 |
s3 lifecycle: report work-based job progress during a pass (#12644)
* s3 lifecycle: report work-based job progress during a pass The daily pass sent progress only while starting, so a healthy scan that ran for many hours could reach the admin's 24-hour no-progress guard. Worker heartbeats prove the worker is alive, not that this job moves. While dailyrun.Run is in progress the handler now sends a progress update every 5 minutes, and only when the pass's counters (entries visited, replay events, RPC outcomes including errors) moved since the last delivered update. An idle tick sends nothing, so a stalled pass still expires. A failed send is retried on the next tick. The reporter is stopped and joined before the completion is sent, so no progress can follow the terminal result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: test that progress, not heartbeats, keeps an old job active Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: label progress phase and bucket as the last started Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
858f8cbad8 |
s3 lifecycle: check internal S3 authentication before starting a pass (#12645)
* s3 lifecycle: add a LifecycleReady RPC that only authorizes the caller LifecycleReady runs the same checkAdminAuth as LifecycleDelete and reads no bucket or object metadata, so a lifecycle worker can confirm that an S3 front accepts its internal credentials before it starts a pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: check S3 fronts with LifecycleReady before the pass Execute used to dial the first registered S3 front and start loading policies and walking buckets, so a missing or mismatched jwt.filer_signing.key surfaced only as repeated per-bucket delete failures. It now checks each front in order with LifecycleReady, using the credentials LifecycleDelete sends, under a 10s bound that also follows parent cancellation. The first front that passes serves the pass; fronts that fail are reported in the job's progress and the log. When none passes, the job fails before any listing with an error that names the front and says whether this worker has the key set, without key or token material. Fronts that predate LifecycleReady are checked with an empty LifecycleDelete, which they authorize and then reject before any lookup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c819e3fbea |
s3 lifecycle: per-pass run results and last full-pass status (#12643)
* s3 lifecycle: collect per-pass bucket results and work counters WalkBuckets kept only the first bucket error and logged the rest. A RunStats shared by the walker, the walker dispatcher and the replay drain now records every bucket's walk result (walked, failed, skipped on cancellation), entry and object visits, replay events, and LifecycleDelete outcomes under the dispatch_total outcome labels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: persist the last fully successful pass Shard cursors and walk timestamps advance on partial passes as well, so neither says when lifecycle last covered every policy. Keep that in its own record next to the cursors, plus a worker gauge for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: report each pass's result in the job record The job ended with a fixed success summary, or with only the first error, and malformed lifecycle policies were V(1) logs. The completion now carries the pass status (complete, partial, failed, canceled), bucket totals and failures, invalid policies, entry, object and event counts, LifecycleDelete outcomes, the last phase and bucket, the elapsed time and the last full pass, on failed passes as well. Only a complete pass advances the full-pass record; a malformed policy makes the pass partial. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: keep a failed job's result in its record Completions with Success=false dropped Result.Summary and Result.OutputValues, so a handler could not report what a failed run had done, which buckets failed, or how far it got. Keep them; the error message still drives the job's message and run record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * grafana: chart the time since the last full lifecycle pass Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a replay halted on an unresolved delete is a partial pass BLOCKED, RETRY_LATER and exhausted transport retries stop a shard's replay without an error, leaving its due events for a later pass. Count those shards and report the pass as partial, so it no longer advances the full-pass record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: a pass with throttled walks is not a full pass WalkerInterval can skip the walk for walk-only rules (ExpirationDate, ExpiredDeleteMarker, NewerNoncurrent) with no error. Count those shards and keep the full-pass record for passes that evaluated every rule; the job reports full_pass and walks_deferred. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: keep the saved full-pass time when the save fails Publishing a timestamp that never reached the filer let the gauge move back on the next job's reload. Keep the previous value and report the save error in the job result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: drop a failed attempt's summary when the retry reports none Retries reuse the job record, and failed completions now keep their result summary, so a retry that completes without a Result showed the failed attempt's summary, and a successful one used it as its message. Clear the summary when a completion carries no Result, as the output values already are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6ad4b07598 |
mount: scope and order the final-release writeback fence (#12651)
* mount: fence a dropped handle only when it holds unpublished content dirtyMetadata is also set by chmod, xattr and fallocate, so a failed metadata-only close refused every later write of the inode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fence the inode before its last handle leaves the handle map A reopen between the map removal and the fence could publish at the stale kernel EOF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fence a failed metadata flush only when the handle holds content A refused synchronous flush or an abandoned writeback metadata flush of a handle with only chmod, xattr or fallocate changes fenced the inode, so later writes were refused until FORGET. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: count a size change on an open handle as unpublished content A truncate or extend through an open handle changes the entry's size and chunks, but it set only dirtyMetadata. A failed close then skipped the final-release fence, so a reopen before FORGET could publish against the untruncated entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: mark copied content dirty before each page of a streaming copy A copy that failed or was cancelled after earlier pages were accepted left dirtyContent false, so the final-release fence skipped that content. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: mark copied content dirty only after a page is accepted A copy whose first page is rejected leaves the destination unchanged and the kernel EOF where it was, so fencing the inode on release only blocked later writes until FORGET. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: skip the server-side copy while the destination holds unpublished content A whole-file server copy clears dirtyContent when it commits, so a page a cancelled streaming copy left behind would lose its release fence. Fall back to the streaming copy instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: skip the server-side copy while the source holds unpublished content The filer copies the stored source, so pages still pending on the source handle would be missing from the destination. The streaming copy reads them through the dirty pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the publication fence before a server-side whole-file copy The filer copy publishes the destination without going through the flush-time guard, so a fenced writeback inode could still take new content before FORGET. Fall back to the streaming copy, whose flush is guarded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
65ecb6aa12 |
fix(mount): prevent negative activity-score shifts (#12631)
fix(mount): bound activity decay shifts across time regressions |
||
|
|
0a38738076 |
filer: keep the listing snapshot below later and unfinished event stamps (#12642)
filer: take the listing snapshot as a claim below later and unfinished stamps A ListEntries snapshot tells the client that every event at or below it is already in the listing, so the mount drops those events from a directory build and fences them with the directory floor. The snapshot was a raw time.Now(), which an event stamp can tie on a coarse clock: on Windows a mkdir right after a listing of its parent was stamped with the snapshot's own timestamp, the mount took it as listed, and the new directory stayed missing from a parent marked cached. A create stamped before the snapshot but not yet in the store was dropped the same way. Take the snapshot through the same in-flight claim the idle heartbeat uses: it stays below every unfinished stamp, and every later stamp lands above it. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
10f8b2d5b1 |
mount: guard peer-truncate gaps, stale append EOF, and terminal writeback refusal (#12630)
* mount: guard append gaps over concurrent content merges * mount: fence a writeback inode when its async publication is abandoned A drain that gives up on a non-precondition error left the token clear, so a reopened writeback handle could publish past the stored EOF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fence a writeback inode when its async data flush fails Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: fence the inode when its last handle drops uncommitted content A synchronous release whose final flush fails destroys the handle's dirty pages, while under writeback the kernel EOF already counts them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b50d4a9cab |
mount: don't spend the close()/fsync commit deadline on uploads (#12641)
* mount: start the close()/fsync commit deadline after the data upload Flush, Fsync and the truncate path of SetAttr create the 30 s metadataFlushTimeout context on entry and pass it to doFlush, so the clock also runs through FlushData, the upload of the dirty pages. That upload has its own timeouts and retries. When an assign through a stalled filer or a slow volume server takes most of the 30 s, CreateEntry then fails at once with "context deadline exceeded" and close() returns EIO for data that was already stored. The deadline is meant for the commit alone (#10102: "a healthy CreateEntry completes in well under a second"). doFlush now takes the commit timeout and starts the deadline once the data is uploaded. close() still cannot hang on a stuck filer: the commit keeps its 30 s bound. Release and the flush before a rename pass 0, as before (no overall bound, only the per-call filer timeouts). Seen in production on a training cluster: a 30 s stall of one replica's disk, and a filer that stalled for about 30 s, each failed close() with EIO although the upload had succeeded. Tests: close(), fsync() and a truncate below dirty data, each after an assign slower than the deadline (all three fail before this change), and a filer that never answers CreateEntry still fails close() at the deadline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: charge the flush commit deadline only for its filer calls Starting the 30 s deadline after the data upload is not enough. flushMetadataToFiler does more than call the filer before it commits: it waits for the handle's locks, reads the file back and uploads it again when its chunks are bloated (maybeMergeChunks), and uploads a manifest once enough chunks have built up (MaybeManifestize). None of that takes the context. When it outlasts the deadline, CreateEntry again fails at once with "context deadline exceeded", and close() or fsync() returns an error for data that was stored. The deadline becomes a budget, commitBudget, charged only while a filer call of the commit runs: CreateEntry, and the read of a changed hard link before it. The attempts of one flush share it, and a flush is not retried once it is spent, so a stuck filer still fails close() after 30 s. Release, the flush before a rename and the async flush pass no budget, as before. Seen in production: the merge of a 1.8 MiB sqlite file with 183 chunks took 32 s because one chunk's read first tried a volume server that was down, and the commit failed 32 microseconds after the merge finished. Tests: a flush whose manifest upload is slower than the deadline commits (it fails without this change), and the filer calls of one flush share the budget. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> * mount: give the stalled-lookup refusal test the new doFlush argument TestContentFlushRefusalWithStalledLookup came in from master after this branch changed doFlush's signature, so the merged package did not build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
e0890391ac |
mount: keep a refused content create refused when the stored-entry read stalls (#12629)
The read of the stored entry ran in the create's filer callback, so a read that hit the call deadline turned the refusal into a filer timeout, and both retry layers resent the refused create. Read it in its own call, bounded to one filer call. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6eb2fb4842 |
filer: stop full-chunk reads at the chunk size after a mount truncate (#12632)
* util/http: stop a full-chunk read at the chunk's recorded size weed mount shrinks a chunk's Size in place on ftruncate but leaves its needle untouched. A view covering the whole shortened chunk is fetched without a Range header, so the reader returned every stored byte. The filer GET then wrote past the declared Content-Length and sent an empty body, and the stream reader served the stale tail. ReadUrlAsStream and the cipher path now deliver at most size bytes; size 0 still reads the whole needle for manifest and log chunks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: limit a plain chunk read to the requested size A full-chunk read sends no Range header, so a chunk shortened by a mount truncate streamed its whole needle into the S3 response. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * util/http: still report a failure past the chunk size The volume server aborts a full-needle response when the needle fails its checksum, and only at the end. Stopping at the chunk size missed that, so a damaged copy read as healthy. Read the rest of the body and return its error so the caller tries another copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
63c99183d1 |
s3vectors: name weed-vector processes servers (#12627)
* s3vectors: call the registered weed-vector processes servers The gateway's discovery named them workers, the name of the admin plugin workers. Also drops two comments left without a function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: build the Enterprise engine as weed-vector In an Enterprise checkout, make ./weed-vector builds the distributed engine, which also serves VectorService, instead of a separate weed-vector-worker. An open-source checkout builds weed-vector-server as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: distributed mode starts two more weed-vector engines It started weed-vector-worker, which no target builds now, and waited for a gateway log line that no longer exists. The Enterprise weed-vector's engines all join the placement controller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a8436a621a |
admin: vector servers on the dashboard (#12606)
* admin: vector servers on the dashboard A Vector servers count in the Cluster card and a Vector Servers table after S3 Servers, from the engines the dashboard already discovers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * admin: drop the dashboard's vector server kind Every vector server is a weed-vector engine, so the column said the same thing on every row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
58d25998d0 |
admin: Cluster → Vector Servers page (#12603)
Lists the weed-vector engines registered with the master, with version and health, next to the other server lists. Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4fc41872e3 |
mount: keep an open file publishing when a peer unlinks one of its hard link names (#12613)
* mount: record a hard link's looked-up names on its inode Lookup mapped a second name of a hard-linked file to the existing inode without adding it to the inode's names, so GetAllPaths missed it and the final forget left it mapped to a released inode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep an open file publishing when a peer unlinks one of its names A delete event from another client marked the inode's open handle deleted even when other names still held the hard-linked file, so every later fsync reported success without saving metadata. The delete event's entry carries the link count the filer decided on, so an alias delete now drops only that name, keeps the handle on a name this mount knows, and has the next flush read the survivor's stored entry before publishing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test peer unlinks of a hard link name held open Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: leave an unlinked file out of the periodic metadata flush flushFileMetadata did not check the handle's delete mark. A file unlinked while open before its deferred create reached the filer was created again by the next periodic flush, which expects no entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: read a pending link change before a periodic metadata flush Only flushMetadataToFiler applied linkRefresh. After a peer unlinked one name, the periodic flush kept publishing over the old link count, so the filer refused every one until a regular flush ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test the periodic flush after a peer unlinks a hard link name Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
08f9dcb6de |
admin: vector buckets link maintenance jobs to the Lance lane (#12633)
The Vector Engines and Maintenance card linked to the general plugin monitoring page; the vector index jobs run in the Lance lane. Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
68b6a2ed08 |
mount: model the refused async flush as a replaced file in recovery tests (#12628)
A peer's chunk append now merges into the flush, so these tests no longer saw the refusal they exercise. A replaced file is still refused. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fdd3a692d3 |
shell: show -collection, not -c, in ec.balance and ec.rebuild help (#12617)
Both usage lines showed [-c EACH_COLLECTION|<collection_name>], but the flag is -collection; there is no -c. ec.rebuild's own Options section already lists -collection. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> |
||
|
|
45cd6033a0 |
s3: make conditional DeleteObject atomic on versioned buckets (#12624)
* s3: re-check a versioned DeleteObject If-Match under the object lock A versioned or suspended DeleteObject without a versionId routed to the owner filer after checking If-Match outside any lock, so concurrent conditional deletes could all pass the check and each add a delete marker. Keep conditional deletes on the lock path, which re-checks the precondition under the object write lock, as conditional versioned PUTs already do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: guard a conditional version delete on the owner filer DeleteObject with a versionId checked If-Match before routing the delete to the owner filer, so concurrent conditional deletes of one version could all report success. Carry the If-Match clause into the owner's transaction as a guard on the version (the null entry for versionId=null), evaluated under the object's path lock before anything applies, and map that guard failing to 412 instead of the object-lock 403. An If-Match the filer cannot evaluate stays on the lock path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: carry an If-Match ETag list into the routed delete A DeleteObject whose If-Match listed several ETags fell back to the gateway's object write lock, while a single-ETag delete of the same key ran under the owner filer's path lock. The two locks do not exclude each other, so a list delete and a single-ETag delete could both report success. Send every candidate to the filer, whose strong comparison matches checkDeleteIfMatch, so all conditional deletes with an owner are decided under the owner's lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: fail a conditional null delete on an owner transport error The owner may have committed the delete before the error, and the gateway lock path it fell back to does not exclude routed deletes, so a second If-Match delete could pass there too. Answer 500 as the specific-version path does; unconditional deletes still fall back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: fail a conditional null delete on any owner error but the nested refusal Only the owner's non-empty-folder refusal is known to apply nothing. After any other owner error the delete may have been partly applied, and a lock-path re-check does not exclude a concurrent routed delete of the same version, so a conditional null-version delete answers 500 instead of falling back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: strip a nested conditional null delete on the owner A conditional versionId=null delete that the owner refuses because other keys are nested under the entry used to fall back to the gateway lock and re-check If-Match there. That lock does not exclude routed deletes: once the last nested key is removed, a concurrent routed delete can pass the same check on the owner, and the fallback then treats the missing entry as deleted, so both requests answer 204. Demote the entry on the owner instead, in a transaction guarded by the If-Match clauses and by the entry it was read as, with the object-lock guards on the request. A missing or already demoted entry answers 412. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: add an IF_OBJECT_EXISTS write condition IF_EXISTS also holds for a plain directory, which is what is left of a key once the object on it is stripped while other keys stay nested under it. IF_OBJECT_EXISTS holds only for an entry that carries an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: require an object for a conditional null delete with If-Match: * Stripping the null version off an entry other keys are nested under leaves a plain directory. Once the last nested key is gone, a routed delete with If-Match: * still found an entry there and removed the empty directory, so a second conditional delete could also answer 204. The owner now checks that the entry carries an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3: settle a wildcard null delete on the entry when the filer predates IF_OBJECT_EXISTS A filer older than IF_OBJECT_EXISTS fails it closed, so with the gateway upgraded first every If-Match: * delete of a null version answered 412. When the narrowed guard fails, the gateway now reads the entry on the owner: absent or a plain directory is a 412, otherwise the file is deleted, or the directory stripped, under IF_EXISTS and IF_ENTRY_EQUAL on the entry it read. The demote keeps the caller's If-Match clauses, since the entry guard already requires an object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d695d101b5 |
Fix/s3 abandoned chunk download (#12616)
* filer: cancel a chunk download that no reader waits for * filer: stop waiting for reader cache budget once cancelled * filer: keep a cancelled download's re-lookups off its context The refresh and retry lookups took the download's context. FilerClient coalesces concurrent lookups of a volume, so a cancelled download leading one failed the other readers waiting on it. Fetches still stop on cancel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test that cancelling a download leaves its re-lookup running Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: cancel unheld downloads when the reader cache is destroyed ReaderCache.destroy waited for every downloader, so closing a reader waited for a stalled prefetch nobody read. Downloads with no reader or stream on them are now cancelled first; ones still in use keep running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test that destroying the reader cache cancels an unread prefetch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: cancel a destroyed download once its last holder leaves destroy skipped downloads a reader or stream still held, and once they were out of the map nothing cancelled them, so destroy waited for a fetch nobody used. removeConsumedLocked now cancels any consumable download, whether or not it is still in the map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test a destroyed download cancelled after its last stream leaves Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: release the stream only after destroy dropped the download Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: test a cancelled stream leaving a waiting reader's download running Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
813ac74d79 |
s3tables: keep legacy-subject owners and policy principals for federated STS sessions (#12615)
* s3tables: re-anchor legacy owners for federated STS sessions A web-identity role session was keyed on its bare OIDC subject until owners became issuer-scoped. Its sessions now carry the scoped aws:userid but no asserted issuers, so the legacy subject check never applied and the session lost the table buckets and namespaces it had created. Read the configured issuer set for such sessions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: keep token claims from asserting trusted issuers on STS sessions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: give the catalog's table handler the local accounts and issuer set Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: test a federated session on a bucket its legacy subject owns Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * sts: drop a token-supplied aws:userid when a session has no parent user Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: take a session's aws:userid from its signed parent user Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: test that session claims keep only the server-asserted aws:userid Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: match legacy subject patterns in table bucket policy principals A policy written before issuer scoping can name role sessions by a pattern over their bare OIDC subjects. Match such a pattern against the caller's subject claim once that subject resolves to the session's scoped principal under the single configured issuer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3tables: test a federated session against a legacy subject pattern principal Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4bc7598a43 |
admin: escape dynamic text in S3 bucket modals (#12625)
admin: escape dynamic text in bucket modals |
||
|
|
d685709832 |
s3api: refuse writes to a stored version's physical path (#12622)
* s3api: refuse a PUT to a stored version's physical path In a Suspended bucket a PUT to <key>.versions/v_<id> replaced the stored version's body in place. Look the entry up when the key has the version file shape and deny the write when it is a version file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: refuse copy, multipart, POST and rename writes to a version file Apply the same version file check to the CopyObject, CreateMultipartUpload, CompleteMultipartUpload, POST Object and RenameObject destinations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: read a version file write target on its logical owner The guard resolved the key as an object, which on a ring reads the physical path's owner and, in a versioned bucket, may return a nested version under <key>.versions/v_<id>.versions instead of the stored version. Look up the physical entry directly, routed by the logical key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: confirm a version file miss on a skipped owner When the owner was recently unreachable the routed lookup answers from a replica, whose miss does not show the version is absent. Ask the owner directly in that case and refuse the write when it cannot answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: confirm every routed version file miss on the owner Whether the routed lookup skipped the owner can change between the lookup and a second unreachable check, letting a replica's miss through. Ask the owner after any miss instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d1a97f4c3d |
fix(mount): retain refused async metadata before releasing handle (#12598)
* fix(mount): retain refused async metadata before releasing handle * mount: sync the failed-metadata directories with util.FsyncDir Directory fsync fails on Windows, where the mount also builds, and returns EINVAL on some filesystems. Either kept the retention loop retrying a record that could never be saved, so the drain never finished. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep failed-metadata records in a per-user directory The write cache root defaults to the shared temp directory. The first mount to save a record created failed-metadata there as 0700, so another user's mount could never save one and its drain retried forever. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: drop the async conflict probe test TestContentAsyncConflictKeepsRecoveryRecord drives the same refused drain with the retry backoff stubbed out; the probe slept through the real backoff (7 s) and its kept-handle branch could not run. Its peer-entry check moves into the recovery record test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
5a54009439 |
filer/etcd: client keepalive and per-request deadlines (#12621)
* filer/etcd: client keepalive and per-request deadlines An etcd member that goes silent with its connections open (a dead or hung node, a network partition, a frozen process) keeps the store's requests balanced onto it waiting forever, because the client has no keepalive and the calls have no deadline. Writes through the filer stop although the other members keep quorum. keepalive_timeout (default 3s, "0s" turns keepalive off) enables gRPC keepalive with the ping interval fixed at gRPC's 10s minimum. gRPC also sets TCP_USER_TIMEOUT to it on Linux, so a connection whose sent data goes unacknowledged that long is closed by the kernel. request_timeout (default 3s) bounds every Get, Put and Delete unless the caller set a deadline; timeout keeps its meaning as the dial timeout. DeleteFolderChildren gets no request deadline: its DeleteRange grows with the directory, and keepalive still ends it if the member goes silent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update weed/filer/etcd/etcd_store.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * Update weed/filer/etcd/etcd_store_test.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * Update weed/command/scaffold/filer.toml Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * filer/etcd: no request deadline on entry writes A Put whose client deadline expires may still have been committed. The filer's upload handler treats the error as a failed write and deletes the uploaded chunks, which the saved entry may then point at. InsertEntry and UpdateEntry therefore keep the caller's context: etcd's own request timeout ends a Put a live member cannot commit, and keepalive ends one on a member that goes silent. Reads and deletes keep request_timeout; a timed-out store delete returns before the filer deletes any chunks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer/etcd: no request deadline on KvPut The filer wrapper stores a hard-linked file's attributes and chunk list through KvPut. An HTTP append to a hard-linked file whose KvPut times out after etcd saved it returns an error, and the upload handler deletes the appended chunks the saved hard link now points at. KvPut keeps the caller's context like InsertEntry and UpdateEntry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
911bfe4014 |
mount: stop overwriting the mount root's filer entry (#12623)
When a mount is rooted at an existing directory, two code paths replace that directory's filer entry with one the mount builds itself: - every mount start re-creates the root with a non-exclusive mkdir, which the filer handles as an update of the whole entry; - creating or removing an entry directly in the root saves the root's entry to touch its mtime, and maybeLoadEntry synthesizes that entry from the mount options instead of reading the filer. The filer keeps only crtime and inode on such an update, so the root loses its mode, owner and extended attributes. When the root is a bucket it also loses the bucket quota, which silently drops to 0 (unlimited), and the settings S3 keeps on the bucket entry. The explicit allowEmptyFolders=false that #11370 meant to keep is replaced by true at every mount start, because the mkdir has wiped it before ensureBucketAllowEmptyFolders looks. Create the root only when a lookup says it is missing, and then exclusively. Leave the root out of the filer-side mtime touch: its attributes come from the mount options (setRootAttr), so the filer copy of its mtime was never visible through the mount anyway. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
737ef68a11 |
mount: keep publishing an open file through its surviving hard link (#12609)
* mount: keep flushing an open file whose other hard link survives an unlink Unlink marks the inode's open handle deleted so no flush recreates the removed name. When the entry was one of several hard links the inode lives on under another name, yet the mark stayed and every later fsync and close returned OK without publishing metadata. Lift the unlink's own mark once the name is gone and the inode still has a path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: carry a link's stored entry into the file's open handle Link rewrites the source entry with the new hard link id, count and times, but an open handle kept its old base and entry. Its next conditional content flush was refused (EIO), and an unconditional one would have dropped the hard link id. Move the handle onto the stored entry when only the link fields and times changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: carry an alias unlink's stored entry into the open handle Removing one of several hard links lowers the stored link count and stamps a new ctime, and the handle may still hold the removed name. Look up the entry at the name the handle flushes through and adopt it, so the next content flush matches the stored entry, writes the right count, and attribute saves address the surviving name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test an open handle across link and alias unlink Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: read an alias unlink's lost entry again at the next flush When the read of the surviving entry right after an alias unlink fails, the handle kept a base with the old link count and ctime, and every later flush was refused even after the filer recovered. Leave the hard link id on the handle and have the next flush read the stored entry and adopt it before publishing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: publish a queued close before unlinking a name another link keeps With writeback caching, close only queues the flush. Unlink marked the handle deleted before waiting for that flush, so the flush skipped its metadata and the surviving link kept the content from before the close. When another link keeps the file, wait for the queued flush first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: let a queued close reach the filer before linking the file With writeback caching a new file is created on the filer by its queued close, and a listed directory serves the source from the cache, so a link right after close updated a name the filer did not have yet and failed with EIO. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: skip a link change no newer than the handle's entry A link change carried into the handle could replace a base already moved past it by a later acknowledgment. Use the same clock-domain check as installAckedEntry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep a link change's ctime in the dirty entry The flush does not restamp times, so a dirty entry carrying an earlier write's ctime moved the filer's ctime back from the link or unlink. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: flush through the file's current name when the resolved one is gone doFlush resolves the name before uploading the data. An unlink of that alias, or a rename, in the meantime sent the publish to a name the file no longer has. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: keep the pending link read when its name is already gone A flush could read the removed alias name, find nothing, and drop the pending read of the surviving link, leaving the stale base in place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test link and alias unlink across queued closes, flush races, versions and ctime Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: pick a flush's name again once its locks are held An unlink or rename that lands between the name pick and the locks would otherwise send the flush to the removed name, where the create guard fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test a flush waiting on its lock through an alias unlink Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: check the flush target after every lock acquisition A flush whose name kept changing published to the name chosen before its last lock wait. Check after each acquisition and return a retryable error once the attempts run out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: test a flush whose name changes during every lock wait Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bac2196d65 |
shell: return flag parse errors from every command (#12618)
* shell: return flag parse errors from every command 48 commands returned nil when their flags failed to parse, so a mistyped flag printed the flag error but the command still counted as a success: a piped weed shell run exited 0, the master maintenance loop logged no error, and admin script jobs reported the line as done. Return the parse error as s3.bucket.create (#12544) and the other shell commands already do. TestCommandsReturnFlagParseErrors runs every registered command with an unknown flag and fails for the 48 on master. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * shell: don't report -h as a failure With the parse errors returned, -h on those commands now returns flag.ErrHelp, so the shell printed "error: flag: help requested" and a piped run exited 1. The flag package has already printed the usage, so processEachCmd now treats flag.ErrHelp as success for every command. Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> * shell: keep the -h test beside the flag parse test TestProcessEachCmdTreatsHelpAsSuccess covers the same change as TestCommandsReturnFlagParseErrors, so it lives in the same new file and shell_liner_test.go stays as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c248ab7865 |
credential: keep IAM changes made during the legacy-file migration (#12614)
* credential: migrate the legacy IAM file without overwriting newer entries The migration wrote every identity, service account and group from the snapshot the load read, over whatever the per-entry files held by then, so a revoke or delete landing in between was undone for good. It now migrates only the legacy file's own entries, creates each per-entry file only when none exists, and waits for the next load when the legacy file changed after the snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * credential: test changes landing between the IAM load and its migration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * credential: keep a per-entry file deleted after the IAM load deleted An entry the load read from its own file is already migrated. When that file is removed before the migration runs, the legacy copy must not recreate it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * credential: test a deleted per-entry file the legacy file also holds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
aa901b7ac7 |
s3tables, iceberg: create missing directories exclusively so a concurrent create keeps its record (#12600)
* s3tables: create a missing directory exclusively in ensureDirectory ensureDirectory looked the path up and, when it was missing, created it with a plain CreateEntry. A table create that lost the lookup race then replaced the directory another create had made, claimed and confirmed since, erasing that table's record, so a third create could claim the name and win as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: create missing metadata parent directories exclusively saveMetadataBlob created each missing directory on the way to the metadata directory with a plain CreateEntry after a lookup. A table directory another create had made and written its record onto since the lookup was replaced by a bare directory, erasing that record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * iceberg: create missing stage-create marker directories exclusively The marker path starts at the table bucket directory, which carries the bucket record; a plain create after a missed lookup replaces a directory another writer made in between, with whatever it carries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e0beb59ef0 |
topology: treat a nil ttl as the empty ttl when looking for duplicate volume ids (#12601)
Volume layouts are keyed by ttl.String(), so a nil ttl and the empty ttl share one layout. ttlEqual told them apart, so a replica reported with a nil ttl into a layout created with the empty ttl matched its own layout as a duplicate, and every copy was quarantined. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fe14880f4d |
shell: tier download replicates the downloaded volume before deleting the remote object (#12599)
* shell: replicate the downloaded volume before deleting its remote object volume.tier.download downloaded every replica from the remote tier and let the last one in the master's location list delete the shared object, so a replica missing from that list kept reading a deleted object. Download onto one replica, copy it over every other remote-backed replica, re-read the topology so replicas that show up meanwhile are replaced too, and only then delete the last remote-backed replica together with the object before copying the local volume onto it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: test tier download replicates before deleting the remote object Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: copy the downloaded volume onto each replica's own disk type Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: test tier download keeps each replica's disk type Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: name the hdd disk type when copying a downloaded volume VolumeCopy treats an empty disk type as the source's, so an HDD replica copied from an SSD source was placed on SSD or failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: block deletes on the tier download copy source A readonly-can-delete source took deletes after VolumeCopy read its index, so the replaced replica lost them and served the files again. Hard-mark the source readonly while replicas are replaced, then restore the writable mark or reload the persisted state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: keep the remote object while replicas are unlisted Tier download deleted the shared object once the listed replicas were local, even when the replica placement expected more copies than the topology listed. Keep it in that case so a rerun can finish later. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: fail tier download when no replica is listed Also point the final-replica delete error at volume.fix.replication. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: keep the remote object when a server lists the volume twice A stale twin on another disk of a replaced server still references the object but is filtered out with its server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: restore the tier download source when its readonly mark fails VolumeMarkReadonly blocks local writes and deletes before its second master notification, so a failed notification returns an error with the source already frozen. Restore it whenever the status was read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: report a failed tier download source restore A restore that leaves the source unmounted or frozen was only printed, so the command still returned success. Join the restore error into the result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * shell: wrap the tier download restore error Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |