mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-22 22:11:39 +02:00
Javier de la TorreandJavier de la Torre
21add33b6e
* s3 backend: optional HTTP client settings for the cloud tier The S3 tier backend builds its client from a bare aws.Config, so it always uses Go's default transport: HTTP/2 wherever the endpoint offers it over TLS, no cap on connections per host, and a 90s idle timeout. Against some S3-compatible services that is a poor fit. Over HTTP/2 every upload and range read of a tiered volume shares one connection and its flow-control window; with a small window, tier uploads and concurrent reads of tiered volumes become throughput-bound on that one connection, while HTTP/1.1 opens a connection per concurrent request. Some services also limit parallel connections per client address, or drop idle connections before 90s. Add three optional settings to [storage.backend.s3.<id>], all off by default so existing deployments keep the SDK's default client: - disable_http2: keep the client on HTTP/1.1 (ForceAttemptHTTP2=false and an empty TLSNextProto, the documented opt-out); - max_connections: MaxConnsPerHost, 0 = no cap; - idle_connection_timeout_seconds: IdleConnTimeout, 0 = Go's default. The session cache key includes the options when any is set, so two backends on the same endpoint with different settings do not share a client. ToProperties reports the settings; the master.toml scaffold documents them. Tests: TestS3BackendHTTPOptionsReachTheClient fails without the change (the settings never reach the client); TestS3BackendDefaultHTTPClientUnchanged, TestS3BackendSessionsDifferByHTTPOptions, TestParseNonNegative. * s3 backend: reject an idle timeout too large for time.Duration An idle_connection_timeout_seconds above math.MaxInt64/1e9 overflowed when multiplied by time.Second. The negative result made the transport keep its default timeout and ToProperties export a negative value. Warn and use the default instead. * s3 backend: parse the idle timeout as 64-bit seconds parseIdleConnTimeout went through strconv.Atoi, so on 32-bit platforms any value above 2147483647 seconds was rejected while the same value was accepted on 64-bit, and the overflow test failed under GOARCH=386. Parse with ParseInt(…, 64) and export the effective value with FormatInt so the accepted range is the same on every platform. --------- Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>