* util: add listenOnLocalhost parameter to NewIpAndLocalListeners
Nil keeps the current behavior of also binding localhost when the host
is a specific address; all callers pass nil for now.
* command: add -ip.bind.localhost flag to opt out of the extra localhost listener
Binding a command to a specific -ip.bind address also opened a second
listener on localhost for every port, with no way to turn it off. Each
command that calls NewIpAndLocalListeners (master, master.follower,
filer, s3, iam, volume, sftp, mq.broker, mq.agent) now accepts
-ip.bind.localhost=false to bind only the requested address. Embedded
services get prefixed flags (s3.ip.bind.localhost, sftp.ip.bind.localhost,
iam.ip.bind.localhost), and the command-level flag also disables them so
a single switch turns off every extra listener the command opens.
* mini: only require all-interface port availability while the loopback listener is on
With -ip.bind.localhost=false each service binds just -ip.bind, so a port
held on another address no longer collides; the all-interface precheck
would reject or relocate usable ports. WebDAV, the Admin UI, and the
Admin worker gRPC (which binds -admin.worker.ip, not -ip.bind) never open
a loopback companion listener, so their checks now use the address they
actually bind.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* mq.broker: bind the gRPC listener to -ip.bind instead of all interfaces
The listener used an empty host so -ip was ignored for binding and the
localhost companion was never created, leaving -ip.bind.localhost with
nothing to control. Give the broker the same -ip/-ip.bind pair the other
servers use; embedded mode binds to -ip.bind like its peers.
* server: add -sftp.ip.bind.localhost and -iam.ip.bind.localhost switches
weed filer already exposes the prefixed opt-outs for its embedded sftp
and iam; give weed server the same pair so each extra listener can be
disabled individually while -ip.bind.localhost=false still turns them
all off.
* mq.broker: keep the default all-interface bind unless localhost is off
An unset -ip.bind used to mean the standalone broker listened on every
interface; defaulting it to -ip silently cut off multi-homed clients. Only
fall back to -ip when -ip.bind.localhost=false, where a specific bind is
needed for the opt-out to take effect.
* mini: validate the Admin HTTP port on the address it will bind
The port check always used -ip.bind while startMiniAdminWithWorker falls
back to loopback when Admin has no credential, so a loopback-held port
passed validation and killed the admin at startup.
* mq.broker: keep the all-interface default bind when -ip.bind is unset
-ip only becomes the fallback once -ip.bind.localhost=false needs a
specific address for the listener.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* mini: validate the Admin HTTP port on its effective bind IP
When the Admin listener falls back to loopback (no credential or mTLS
configured), the pre-flight port check now probes that same address
instead of -ip.bind.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* util: keep NewIpAndLocalListeners source-compatible for 3-arg callers
* command: drop the -ip.bind.localhost flag; an explicit -ip.bind drops the extra listener
Instead of an opt-out flag, the loopback companion listener now simply
follows whether the bind address was chosen: it is created only while
-ip.bind is unset and the bind address falls back to -ip. Passing
-ip.bind=<addr> binds each port to exactly that address.
Each service options struct carries a listenOnLocalhost *bool that
embedders (weed server, weed mini, embedded s3/sftp/iam in weed filer)
set from the parent's -ip.bind state, captured before the bind flag is
repointed or filled in by the -ip fallback; standalone commands derive
it from their own flag before the fallback runs.
iam and mq.agent gain a real -ip.bind flag so they have a way to pin a
single address. mq.broker keeps its new -ip.bind flag but drops the
-ip.bind.localhost fallback hack: empty still listens on all interfaces.
* server: default embedded mq.agent brokers to the broker's bind address
With an explicit -ip.bind the embedded broker no longer listens on
localhost, so an unset -mq.agent.brokers defaulting to localhost:17777
could not reach it. Default to <serverBindIp>:<mq.broker.port> (the
address the embedded broker binds), falling back to localhost when the
bind address is a wildcard. An explicit -mq.agent.brokers still wins.
Also fix the NewIpAndLocalListeners skip message to say localhost.
* server: bind embedded mq.agent to serverBindIp
The embedded agent's ipBind was never set, so startQueueAgent fell back
to the advertised -ip address and an explicit -ip.bind did not pin the
agent listener.
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* util: don't let the activity timeout clobber externally-set conn deadlines
util.Conn extended the connection deadline at the start of every
Read/Write. net/http's server also sets deadlines directly on the same
conn - abortPendingRead sets one in the past to interrupt the pending
background read after each response. The activity extension raced with
and silently overwrote that interrupt, leaving the read blocked (and the
server's conn.serve goroutine stuck in abortPendingRead) until the full
-idleTimeout (default 30s) expired.
Wedged connections count as active, so the volume server's graceful HTTP
drain waited out its whole 30s StopTimeout on shutdown - observed as
weed mini taking ~30s to exit in the FUSE integration tests after any
filer->volume traffic.
Track externally-set deadlines, suspend the activity extension while one
is in force, and serialize deadline updates with a mutex. Activity still
extends both directions at once: a long write-only response must keep
the read deadline alive too, or the server's background read would time
out and cancel the in-flight request.
* util: extend read/write deadlines independently when one side is external
A server-configured WriteTimeout keeps an external write deadline in
force for the whole request, which previously suspended the activity
extension entirely - leaving the read deadline stale from before the
request and letting net/http's background read time out mid-response.
Extend each direction independently instead.
* fix: S3 downloads failing after idle timeout (#7618)
The idle timeout was incorrectly terminating active downloads because
read and write deadlines were managed independently. During a download,
the server writes data but rarely reads, so the read deadline would
expire even though the connection was actively being used.
Changes:
1. Simplify to single Timeout field - since this is a 'no activity timeout'
where any activity extends the deadline, separate read/write timeouts
are unnecessary. Now uses SetDeadline() which sets both at once.
2. Implement proper 'no activity timeout' - any activity (read or write)
now extends the deadline. The connection only times out when there's
genuinely no activity in either direction.
3. Increase default S3 idleTimeout from 10s to 120s for additional safety
margin when fetching chunks from slow storage backends.
Fixes#7618
* Update weed/util/net_timeout.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* server can start when no network for local dev
* fixed superfluous response.WriteHeader call" warning
* adaptive based on last write time
* more doc
* refactoring
* refactor(net_timeout): `listner` -> `listener`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(s3): `s3ApiLocalListner` -> `s3ApiLocalListener`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(filer): `localPublicListner` -> `localPublicListener`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(command): `masterLocalListner` -> `masterLocalListener`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
* refactor(net_timeout): `ipListner` -> `ipListener`
Signed-off-by: Ryan Russell <git@ryanrussell.org>
Signed-off-by: Ryan Russell <git@ryanrussell.org>
