* s3 tier backend: bound connect and response-header waits
The tier client used the SDK default transport: a 30s dial timeout and no
limit on waiting for response headers. A blackholed endpoint cost about
two minutes per request with the SDK's retries, and an endpoint that
accepts connections but never answers held the request forever.
Dial with a 10s timeout and give up on a response whose headers do not
arrive within a minute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* volume: stop the disk scan from waiting on a stalled tier backend
A volume server registers with the master only after every volume on its
disks has loaded, and loading a tiered volume reads its superblock from the
remote object. With the S3 endpoint unreachable, each read waited out the
client's timeouts and retries on one of the loader workers, so start-up grew
with the number of tiered volumes and the local volumes were not served
until it finished.
A tiered volume already loads without its superblock when the read fails.
Give that read 10s, and once it stalls, let the backend's other volumes skip
it for a minute instead of each waiting out the timeout. The volume reads
from the backend as before once the endpoint answers again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-volume: bound connect and read waits of the S3 tier client
Mirror the Go tier client: a 10s connect timeout instead of the SDK's
3.1s default, and a minute to receive a response, so an endpoint that
accepts connections but never answers cannot hold a read forever.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-volume: stop volume loading from waiting on a stalled tier backend
Mirror the Go loader: the superblock read of a tiered volume gets 10s, and
once it stalls the backend's other volumes skip it for a minute. The
volume loads with its .vif version as before when the read fails, and
reads from the backend once the endpoint answers again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-volume: keep the .vif replication when a remote superblock read fails
Go applies the .vif replication to a tiered volume whether or not its
superblock read succeeded, so a volume loaded while its backend is down
still reports its replica placement. The Rust loader applied it only on
success and reported the default placement instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* volume: skip the .idx head repair while the tier backend is unreachable
The repair reads needle headers from the remote .dat, so after the
superblock read failed it would wait on the same backend again and hold
up the load. It runs on the next load that reaches the backend.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-volume: skip the .idx head repair while the tier backend is unreachable
Mirrors the Go load: after the remote superblock read failed, the
repair's remote header reads would wait on the same backend again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* s3 tier backend: give up a ranged read whose body stops arriving
The response-header timeout ends once headers arrive, so an endpoint
that answers and then sends no body held ReadAt, and a superblock
read's worker, forever. Cancel the request once the body makes no
progress for s3ReadTimeout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* seaweed-volume: test that a tier read whose body stalls gives up
The SDK's stalled-stream protection already fails it, matching the Go
read's no-progress cancel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* volume(rust): keep the S3 read timeout off upload_part
The SDK's read timeout runs from the start of the request, so a 64 MiB
part on a slow link would time out while its body is still being sent.
Go's ResponseHeaderTimeout only starts after the body is written.
upload_part now keeps the connect timeout and drops the read timeout;
stalled-stream protection still fails an upload that stops moving.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* volume(rust): bound the upload_part response wait from when the part is sent
Without a read timeout, an upload_part whose body was accepted but never
answered waited forever. The part body now goes to the connection in 64 KiB
chunks, and once the last chunk is taken the response must arrive within
S3_READ_TIMEOUT, like Go's ResponseHeaderTimeout. A retry rebuilds the body
and restarts that clock. The body keeps its in-memory contents, so signing
and the checksum header are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* volume(rust): keep a retried part's response clock from an earlier attempt
The send state was a single flag, and a retry body could be rebuilt and
fully sent before the waiter saw the flag drop, so the waiter kept the
earlier attempt's deadline. Each body now carries its attempt number, only
the latest attempt marks the part sent, and the deadline holds only while
no later attempt has started.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* volume server: open volume files with O_NOATIME
Nothing reads the atime of .dat, .idx, .sdx, or EC files, but every
needle read still dirtied the inode: even relatime writes atime on the
first read after each write, so an actively written volume paid a
metadata write per read/write cycle, and strictatime mounts paid one
per read. Open the serving handles with O_NOATIME, falling back to a
plain open when the file belongs to another owner (EPERM).
Claude-Session: https://claude.ai/code/session_015uVY4diBgEn3VYQoc2eMuD
* seaweed-volume: mirror the O_NOATIME volume file opens
Same change as the Go volume server: serving handles for .dat, .idx,
.sdx, .ecx, .ecj, and shard files open with O_NOATIME on Linux, with a
plain-open fallback on EPERM.
Claude-Session: https://claude.ai/code/session_015uVY4diBgEn3VYQoc2eMuD
* route the tier-down and recreate .dat opens through the no-atime helper
Review caught the Rust tier-down swap opening the local .dat directly.
The Go swapToLocalDatBackend and the zero-length read-only .dat
recreate in maybeWriteSuperBlock had the same gap: all three install
long-lived serving handles.
Claude-Session: https://claude.ai/code/session_015uVY4diBgEn3VYQoc2eMuD
* Fix: Fail fast when initializing volume with Version 0
* Fix: Fail fast when loading unsupported volume version (e.g. 0 or 4)
* Refactor: Use IsSupportedVersion helper function for version validation
