s3: register the advertised ip with the master (#10482)

* s3: register the advertised ip with the master

The cluster address came from the bind ip, falling back to the
auto-detected interface, so -ip never reached the S3 registration.
weed mini -ip=localhost binds the wildcard and ended up registering
whatever interface happened to sort first -- on a host with VPN
interfaces, an address that stops routing once the tunnel drops.

IAM changes are pushed to registered S3 servers over gRPC, so every
mutation then blocked the full 10s propagation deadline before logging
a failure, and cluster.ps and the admin UI listed a node nothing could
reach. Identities still arrived through the /etc/iam metadata
subscription, so this cost latency and visibility, not credentials.

Add an advertise ip to the gateway option, preferring it over the bind
address, and wire the parent -ip through server, filer and mini.

* s3: treat any unspecified bind address as a wildcard

net.ParseIP + IsUnspecified covers ::, [::] and the expanded IPv6 forms
instead of only the 0.0.0.0 literal, so an IPv6 wildcard bind no longer
registers an address peers cannot dial. Host names parse as nil and stay
addresses in their own right. Apply the same guard to the advertised ip.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-07-29 10:30:46 -07:00
1 parent 0002e5cc7f
commit 4149346bb7
6 files changed
+72 -7

No files matched your search

+1
View File
@@ -257,6 +257,7 @@ func runFiler(cmd *Command, args []string) bool {
startDelay := time.Duration(2)
if *filerStartS3 {
filerS3Options.filer = &filerAddress
filerS3Options.ip = f.ip
if *filerS3Options.bindIp == "" {
filerS3Options.bindIp = f.bindIp
}
+1
View File
@@ -1223,6 +1223,7 @@ func runMini(cmd *Command, args []string) bool {
miniFilerOptions.masters = pb.ServerAddresses(actualPeersForComponents).ToServiceDiscovery()
miniFilerOptions.ip = miniIp
miniFilerOptions.bindIp = miniBindIp
miniS3Options.ip = miniIp
miniS3Options.bindIp = miniBindIp
miniWebDavOptions.ipBind = miniBindIp
miniOptions.v.ip = miniIp
+3
View File
@@ -42,6 +42,7 @@ var (
// When adding a new field, update all four flag registration sites.
type S3Options struct {
filer *string
ip *string
bindIp *string
port *int
portHttps *int
@@ -83,6 +84,7 @@ type S3Options struct {
func init() {
cmdS3.Run = runS3 // break init cycle
s3StandaloneOptions.filer = cmdS3.Flag.String("filer", "localhost:8888", "comma-separated filer server addresses for high availability")
s3StandaloneOptions.ip = cmdS3.Flag.String("ip", "", "ip address advertised to the cluster. If empty, default to -ip.bind, or the auto-detected address.")
s3StandaloneOptions.bindIp = cmdS3.Flag.String("ip.bind", "", "ip address to bind to. If empty, default to 0.0.0.0.")
s3StandaloneOptions.port = cmdS3.Flag.Int("port", 8333, "s3 server http listen port")
s3StandaloneOptions.portHttps = cmdS3.Flag.Int("port.https", 0, "s3 server https listen port")
@@ -358,6 +360,7 @@ func (s3opt *S3Options) startS3Server() bool {
EnableIam: *s3opt.enableIam, // Embedded IAM API (enabled by default)
IamReadOnly: *s3opt.iamReadOnly,
Cipher: *s3opt.cipher, // encrypt data on volume servers
Ip: *s3opt.ip,
BindIp: *s3opt.bindIp,
GrpcPort: *s3opt.portGrpc,
ExternalUrl: s3opt.resolveExternalUrl(),
+1
View File
@@ -289,6 +289,7 @@ func runServer(cmd *Command, args []string) bool {
filerOptions.masters = pb.ServerAddresses(actualPeersForComponents).ToServiceDiscovery()
filerOptions.ip = serverIp
filerOptions.bindIp = serverBindIp
s3Options.ip = serverIp
if *s3Options.bindIp == "" {
s3Options.bindIp = serverBindIp
}