volume: validate remote S3 endpoints in FetchAndWriteNeedle (Rust) (#10001)

* volume: validate remote S3 endpoints in FetchAndWriteNeedle (Rust)

Port the Go volume server's SSRF guard to the Rust volume server. The
gRPC FetchAndWriteNeedle reads from a caller-supplied S3 endpoint, so an
unguarded server can be pointed at loopback / link-local / RFC1918 /
CGNAT / cloud-metadata hosts to read internal services. Resolve and
reject those endpoints unless -volume.allowUntrustedRemoteEndpoints is
set (default off), mirroring weed/server/volume_grpc_remote.go.

Connect-time re-validation against DNS rebinding (Go's guardedDialer) is
not yet ported: the aws-sdk-s3 client builds its own connector, so the
up-front resolve-and-check leaves a narrow TOCTOU window. Left as a
follow-up.

* volume: harden remote endpoint guard (IPv4-mapped IPv6, all S3 types)

Address SSRF review feedback:
- Normalize IPv4-mapped IPv6 (::ffff:a.b.c.d) to IPv4 before the deny
  checks, so ::ffff:127.0.0.1 / ::ffff:169.254.169.254 no longer slip
  past the IPv4 rules.
- Validate the endpoint for every S3-compatible backend, not just type
  "s3"; wasabi/backblaze/aliyun/... all dial a caller-supplied endpoint
  through the same client. Skip validation when the endpoint is empty
  (the provider default, e.g. real AWS S3, which cannot reach internal
  hosts).

* volume: set allow_untrusted_remote_endpoints in integration-test state

The tests/http_integration.rs VolumeServerState literal was missed, which
broke cargo test compilation (it builds the integration tests, unlike the
cargo test --lib used locally).
This commit is contained in:
Chris Lu authored and GitHub committed 2026-06-17 00:56:13 -07:00
1 parent e411ff491d
commit b5ecfcd28c
11 files changed
+457

No files matched your search

@@ -77,6 +77,9 @@ pub struct VolumeServerState {
pub s3_tier_registry: std::sync::RwLock<crate::remote_storage::s3_tier::S3TierRegistry>,
/// Read mode: local, proxy, or redirect for non-local volumes.
pub read_mode: ReadMode,
/// If true, FetchAndWriteNeedle skips remote S3 endpoint validation,
/// allowing arbitrary (incl. loopback / link-local / metadata) hosts.
pub allow_untrusted_remote_endpoints: bool,
/// First master address for volume lookups (e.g., "localhost:9333").
pub master_url: String,
/// Seed master addresses for UI rendering.