4 Commits

Include renames

Author	SHA1	Message	Date
3 people	a235473105	[Volume] Preserve request semantics in replica proxy reads (#12075) ... * fix 12074 * volume: shuffle a copy of cached replica locations for proxy reads LookupVolumeId serves cached location slices shared between lookups; shuffling in place races with concurrent proxied reads of the same volume. * seaweed-volume: forward inbound method and surface lookup failure in proxy reads Mirror the Go handler: proxy_request used to always send GET, turning a client HEAD into a replica download, and a failed master lookup returned 404 like missing data. Forward the inbound method and answer 500 for lookup failures in proxy mode. * volume: keep master lookup error detail out of the proxy error response * volume: bound the proxied-request wait in the method-preservation test * seaweed-volume: answer 404 for missing volumes in proxy-mode lookups Mirror Go's LookupVolumeId: a per-volume error in the master's lookup reply (e.g. a missing volume) means "no locations" and answers 404 downstream, not a lookup failure that proxy mode answers 500. * volume: guard nil lookup result in proxy path A master response that omits the requested vid leaves LookupVolumeId returning a nil *LookupResult with a nil error; dereferencing it panics the read handler. * seaweed-volume: answered lookup errors are missing data, not failures A master reply carrying a per-volume error with HTTP 404 (volume not found) now yields empty locations like Go's LookupResult, so proxy mode answers 404. Other failure statuses such as 503 during master warmup remain lookup failures that proxy mode answers 500. * seaweed-volume: drop hop-by-hop headers from replica proxy reads Go's inbound request drops Host, Content-Length and Transfer-Encoding into dedicated fields, so its forwarded header set never carries them. hyper keeps them; forwarding a client's Content-Length with no body stalls the replica's read, and forwarding its Host misnames the target. * volume: bound the proxy master lookup to the request context * volume: drop hop-by-hop and Connection-nominated headers from proxy reads * seaweed-volume: drop Connection-nominated headers from proxy reads * volume: skip the proxy lookup error when the inbound request is gone * volume: drop proxy-specific headers from replica proxy reads * seaweed-volume: drop proxy-specific headers from replica proxy reads Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: drop hop-by-hop headers from replica proxy responses * seaweed-volume: drop hop-by-hop headers from replica proxy responses * seaweed-volume: a failed lookup status without an error body is still a failure * volume: do not re-proxy an already-proxied read for a missing volume A proxied request that reaches a server which still lacks the volume was looked up and forwarded again, so a stale location answer could bounce the read between two servers until the client gave up. The Rust volume server's proxy_missing_volume already returns 404 for proxied=true; mirror it here. * seaweed-volume: replace stale proxied params in the forwarded query Go builds the proxied URL with query.Set(reqIsProxied), which replaces every proxied param the client sent. Appending after a client-supplied proxied=false left the replica's first-value lookup reading the stale marker, defeating the loop guard. --------- Co-authored-by: chrislusf <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-26 08:13:34 -07:00
4 people	8841eb6c70	volume: check If-None-Match before If-Modified-Since and match ETag lists (#12027) ... * filer: match If-Match and If-None-Match against "*" and ETag lists checkPreconditions compared the whole header as one ETag, so "*", a list of ETags and a weak W/ tag in If-None-Match never matched (RFC 9110 13.1.1, 13.1.2). * volume: check If-None-Match before If-Modified-Since and match ETag lists A volume GET answered 304 from If-Modified-Since even when If-None-Match named a different ETag, and compared If-None-Match as one exact string, so "*", lists and weak tags never matched (RFC 9110 13.1.2, 13.2.2). * filer: split etag lists only outside quoted tags A comma inside a quoted opaque tag is a legal etagc, not a list separator; strings.Split cut such tags into fragments that CanonicalizeETag could turn into a false match. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: match repeated If-Match and If-None-Match header lines Header.Get reads only the first field line, so a match on a later repeated header was missed; join all field values like any other comma-separated list. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: match repeated If-None-Match header lines Header.Get reads only the first field line; join all field values so a match on a later repeated header is seen. Also covers a comma inside a quoted opaque tag. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: send the etag on a 304 like the filer does RFC 9110 15.4.5 lists ETag among the fields a 304 must carry when a 200 would send it; SetEtag now runs before the notModified check. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: mirror the conditional-GET fixes in the rust server not_modified_response now checks If-None-Match before If-Modified-Since, matches "*", weak tags, quoted commas and repeated field lines through the same list rules, and sends the etag on a 304. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: treat an empty If-None-Match like the Go server does * filer: ignore empty elements in etag precondition lists * volume: ignore empty elements in etag lists like the filer * filer: let a present-but-empty precondition line suppress the date check * volume: gate If-Modified-Since on a present If-None-Match line * volume: gate If-Modified-Since on header presence like the Go server --------- Co-authored-by: user.email <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: chrislusf <chris.lu@gmail.com>	2026-10-24 07:05:29 -07:00
4 people	df33e81915	volume: report the needle read error with a failed read (#11934) ... * volume: report the needle read error with a failed read A read that fails on the volume server, for example on a CRC mismatch, returned a bare 500, so mount and filer logs only showed the status. The volume server now writes the error into the response body, and the read helpers add it to the error they return. Co-authored-by: Bruce <82158894+half-life666@users.noreply.github.com> * volume: keep the 500 bare on HEAD requests writeJsonError's body write fails with ErrBodyNotAllowed under net/http when the request method is HEAD, adding an error-writing log line on top of the failed read. Send only the status, matching the no-error path. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * util/http: decode gzip-encoded error bodies in statusError Full-chunk reads set Accept-Encoding: gzip themselves, so the transport does not transparently decompress; a gzip-encoded error body reached statusError as compressed bytes and produced an unreadable message. statusError now takes the response and decodes Content-Encoding before extracting the bounded message. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: log the read error when the 500 stays bare On HEAD requests the body write is skipped, which also skipped the V(0) response log inside writeJson — a failed HEAD read left no cause in normal volume-server logs. Log the error in the same format before the bare 500. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * util/http: check the status before decoding gzip in GetAuthenticated The gzip reader eagerly consumes the header when it is created, so statusError saw a drained or corrupt body on a gzipped error response and the server message was lost. Match the other read helpers and check the status first. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * util/http: export StatusError Other volume read paths outside this file need the same error body. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * volume: redact jwt query param when logging failed-read URLs The new read-failure logs print the request URL, and volume reads accept ?jwt= as a credential, so a failed authenticated read would write the bearer token to the volume server log. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * util/http: surface the error body on the remaining chunk-read paths * volume: redact duplicate jwt params in failed-read logs * operation: pass the lookup jwt on the chunk needle read --------- Co-authored-by: Bruce <82158894+half-life666@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>	2026-10-21 20:24:48 -07:00
3 people	acb44a68a9	fix(volume): skip unnecessary needle data reads on delete (#11711) ... * fix(volume): do not read a needle's data to delete it A delete read the needle's whole payload from disk only to discard it. The delete handler and BatchDelete need the needle's cookie and its chunk-manifest flag, which live in the header and trailer, but fetched them with a full needle read, the same call a GET uses. readNeedle now reads the header and trailer only when MustMetaOnly is set, whatever the needle's size, and the delete paths ask for that. The Rust volume server honours must_meta_only the same way for its delete paths. A compressed needle or a chunk manifest still gets the full read, as before, because their callers need the bytes. HEAD is unchanged. It used to set MustMetaOnly, which nothing read; it now asks for AttemptMetaOnly alone, so it keeps reading a needle up to PagedReadLimit whole. On a spinning disk that one read is cheaper than a header read followed by a trailer read. Version 1 needles are always read in full. They have no data size field and no metadata after the data, which ReadNeedleMeta relies on, so a metadata-only read of one failed with "invalid needle meta size". That already happened for version 1 needles above PagedReadLimit read with AttemptMetaOnly. A delete therefore no longer verifies the data CRC of the needle it removes. The cookie check is unchanged. * fix(volume): read transformed images fully on HEAD too A HEAD carrying resize or crop params reports the transformed length, which a metadata read cannot produce. Go and Rust both answered such HEADs from the source header, returning the stored size instead of the cropped or resized one. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-12 18:08:19 +08:00