* remote_storage: build S3-compatible clients through one constructor
The eight non-s3 S3-SDK providers each duplicated the AWS session setup
and only the s3 maker could take a custom *http.Client. Route every
S3-compatible type (s3, wasabi, b2, aliyun, tencent, baidu, filebase,
storj, contabo) through MakeWithHTTPClient with a single options table,
and add S3CompatibleEndpoint so callers can resolve the endpoint a given
type dials. No behavior change.
* volume: apply the remote-endpoint check to all S3-compatible providers
FetchAndWriteNeedle validated the endpoint and used the pinned dialer only
for type "s3". Every S3-SDK backend (wasabi, b2, aliyun, tencent, baidu,
filebase, storj, contabo) dials a caller-supplied endpoint through the same
client, so gate on S3CompatibleEndpoint to apply the same check uniformly.
-volume.allowUntrustedRemoteEndpoints still opts out.
* volume: don't route the guarded remote-endpoint client through a proxy
The guarded client exists to dial the validated endpoint directly and
re-check the resolved IP at connect time. With http.ProxyFromEnvironment
set, the dialer only validates the proxy's address while the proxy
re-resolves the endpoint host, which reopens the rebinding window. Drop
the proxy on this path; operators that need one can opt out with
-volume.allowUntrustedRemoteEndpoints.
* volume: decode IPv6 transition addresses in the remote-endpoint guard
checkBlockedIP normalized only ::ffff: mapped IPv4, so NAT64 (64:ff9b::/96),
6to4 (2002::/16), Teredo (2001:0000::/32), and IPv4-compatible (::/96) addresses
that embed an internal IPv4 (loopback, 169.254.169.254, RFC 1918) passed the
endpoint guard even though the plain IPv4 forms are refused. Extract the
embedded IPv4 from those forms and re-check it against the deny list, which
covers both the up-front validation and the dial-time guard. Mirrored in the
Rust volume server.
* volume: require the full NAT64 well-known prefix before decoding
Only 64:ff9b::/96 carries the embedded IPv4 in the low 32 bits, so also require
bytes 4-11 to be zero before treating an address as NAT64; other 64:ff9b:
prefixes place the IPv4 elsewhere and are left untouched. Add public-target
coverage for 6to4, Teredo, and IPv4-compatible so every decoder is exercised on
both a blocked and an allowed destination. Mirrored in the Rust volume server.
volume: require admin auth and refuse loopback endpoints in FetchAndWriteNeedle
Gate the RPC behind checkGrpcAdminAuth for parity with the rest of the
destructive volume-server RPCs, and reject cluster-internal remote S3
endpoints (loopback / link-local / IMDS / RFC 1918 / CGNAT) before
dialing. Pin the validated address against DNS rebinding by routing the
AWS SDK through an HTTP transport whose DialContext re-resolves the host
and re-applies the deny list on every dial, so an endpoint that resolves
to a public IP at validate-time and then flips to 127.0.0.1 at connect
time is refused. Operators that legitimately fetch from private hosts
can opt out with -volume.allowUntrustedRemoteEndpoints.
Chris Lu