volume: validate remote S3 endpoints in FetchAndWriteNeedle (Rust) (#10001)

* volume: validate remote S3 endpoints in FetchAndWriteNeedle (Rust)

Port the Go volume server's SSRF guard to the Rust volume server. The
gRPC FetchAndWriteNeedle reads from a caller-supplied S3 endpoint, so an
unguarded server can be pointed at loopback / link-local / RFC1918 /
CGNAT / cloud-metadata hosts to read internal services. Resolve and
reject those endpoints unless -volume.allowUntrustedRemoteEndpoints is
set (default off), mirroring weed/server/volume_grpc_remote.go.

Connect-time re-validation against DNS rebinding (Go's guardedDialer) is
not yet ported: the aws-sdk-s3 client builds its own connector, so the
up-front resolve-and-check leaves a narrow TOCTOU window. Left as a
follow-up.

* volume: harden remote endpoint guard (IPv4-mapped IPv6, all S3 types)

Address SSRF review feedback:
- Normalize IPv4-mapped IPv6 (::ffff:a.b.c.d) to IPv4 before the deny
  checks, so ::ffff:127.0.0.1 / ::ffff:169.254.169.254 no longer slip
  past the IPv4 rules.
- Validate the endpoint for every S3-compatible backend, not just type
  "s3"; wasabi/backblaze/aliyun/... all dial a caller-supplied endpoint
  through the same client. Skip validation when the endpoint is empty
  (the provider default, e.g. real AWS S3, which cannot reach internal
  hosts).

* volume: set allow_untrusted_remote_endpoints in integration-test state

The tests/http_integration.rs VolumeServerState literal was missed, which
broke cargo test compilation (it builds the integration tests, unlike the
cargo test --lib used locally).

seaweed-volume/src/server/grpc_client.rs

@@ -189,6 +189,7 @@ mod tests {
             white_list: vec![],
             fix_jpg_orientation: false,
             read_mode: ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             cpu_profile: String::new(),
             mem_profile: String::new(),
             compaction_byte_per_second: 0,

seaweed-volume/src/server/grpc_server.rs

@@ -3490,6 +3490,25 @@ impl VolumeServer for VolumeGrpcService {
             .as_ref()
             .ok_or_else(|| Status::invalid_argument("remote storage configuration is required"))?;
 
+        // Reject SSRF-prone endpoints unless the operator opted out. Every
+        // S3-compatible backend (s3, wasabi, backblaze, ...) dials a
+        // caller-supplied endpoint URL directly via make_remote_storage_client,
+        // so validate whichever endpoint applies. An empty endpoint means the
+        // provider default (e.g. real AWS S3) and cannot target an internal
+        // host, so skip it. Extends the Go volume server's validateRemoteEndpoint
+        // gate, which only covered type "s3".
+        if !self.state.allow_untrusted_remote_endpoints {
+            if let Some(endpoint) = crate::remote_storage::s3_compatible_endpoint(remote_conf) {
+                if !endpoint.trim().is_empty() {
+                    crate::remote_storage::validate_remote_endpoint(endpoint)
+                        .await
+                        .map_err(|e| {
+                            Status::invalid_argument(format!("reject remote endpoint: {}", e))
+                        })?;
+                }
+            }
+        }
+
         // Create remote storage client
         let client =
             crate::remote_storage::make_remote_storage_client(remote_conf).map_err(|e| {
@@ -4728,6 +4747,7 @@ mod tests {
                 crate::remote_storage::s3_tier::S3TierRegistry::new(),
             ),
             read_mode: crate::config::ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             master_url: String::new(),
             master_urls: Vec::new(),
             seed_master_set: std::collections::HashSet::new(),
@@ -4832,6 +4852,7 @@ mod tests {
                 crate::remote_storage::s3_tier::S3TierRegistry::new(),
             ),
             read_mode: crate::config::ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             master_url: String::new(),
             master_urls: Vec::new(),
             seed_master_set: std::collections::HashSet::new(),
@@ -4940,6 +4961,7 @@ mod tests {
                 crate::remote_storage::s3_tier::S3TierRegistry::new(),
             ),
             read_mode: crate::config::ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             master_url: master_urls.first().cloned().unwrap_or_default(),
             master_urls,
             seed_master_set,

seaweed-volume/src/server/heartbeat.rs

@@ -1047,6 +1047,7 @@ mod tests {
             write_queue: std::sync::OnceLock::new(),
             s3_tier_registry: std::sync::RwLock::new(S3TierRegistry::new()),
             read_mode: ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             master_url: String::new(),
             master_urls: Vec::new(),
             seed_master_set: std::collections::HashSet::new(),

seaweed-volume/src/server/profiling.rs

@@ -131,6 +131,7 @@ mod tests {
             white_list: vec![],
             fix_jpg_orientation: false,
             read_mode: ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             cpu_profile: String::new(),
             mem_profile: String::new(),
             compaction_byte_per_second: 0,

seaweed-volume/src/server/volume_server.rs

@@ -77,6 +77,9 @@ pub struct VolumeServerState {
     pub s3_tier_registry: std::sync::RwLock<crate::remote_storage::s3_tier::S3TierRegistry>,
     /// Read mode: local, proxy, or redirect for non-local volumes.
     pub read_mode: ReadMode,
+    /// If true, FetchAndWriteNeedle skips remote S3 endpoint validation,
+    /// allowing arbitrary (incl. loopback / link-local / metadata) hosts.
+    pub allow_untrusted_remote_endpoints: bool,
     /// First master address for volume lookups (e.g., "localhost:9333").
     pub master_url: String,
     /// Seed master addresses for UI rendering.

seaweed-volume/src/server/write_queue.rs

@@ -205,6 +205,7 @@ mod tests {
                 crate::remote_storage::s3_tier::S3TierRegistry::new(),
             ),
             read_mode: crate::config::ReadMode::Local,
+            allow_untrusted_remote_endpoints: false,
             master_url: String::new(),
             master_urls: Vec::new(),
             seed_master_set: std::collections::HashSet::new(),