mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-04 11:07:35 +01:00
Chris LuandClaude Opus 5.5
eeb8aefe70
* weed-vector: read the cluster's security.toml The engine finds security.toml where weed processes do (., ~/.seaweedfs, /usr/local/etc/seaweedfs, /etc/seaweedfs) with WEED_* environment variables over it, and builds gRPC client and server credentials by the rules of security.LoadClientTLS and LoadServerTLS: a dedicated client certificate is preferred, both halves of a pair must be set, and grpc.ca is required once either is. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: gRPC TLS to the masters and from the gateway The masters are dialed with [grpc.client]'s credentials when the cluster has them, the master's certificate checked against the host dialed or --tls-server-name. The engine's own listener takes its identity from [grpc.vector] and then requires a client certificate chaining to grpc.ca, which is how the gateway, dialing with [grpc.client], reaches it. The rustls provider is pinned before the first TLS use, as the other binaries linking both aws-lc-rs and ring do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * scaffold: document [grpc.vector] Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: put every gRPC hop under mutual TLS in the hardened leg VECTOR_TLS=1 now also writes [grpc.master], [grpc.volume], [grpc.filer], [grpc.s3], [grpc.client] and [grpc.vector], and probes the master's and the engine's gRPC ports: plaintext HTTP/2 and a client without a certificate are refused, the client certificate gets a SETTINGS frame. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: give the test volume server 64 volume slots Every collection grows 7 volumes and test_vector_bucket_encryption opens two buckets while the previous test's bucket is still being torn down, so 16 slots were marginal: the Go legs failed CreateIndex now and then with "failed to find writable volumes" and master.log showing "create 7 volume, created 4: Not enough data nodes found!". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: reload the listener's certificates and enforce allowed_commonNames The listener now runs the way weed servers run theirs. The certificate, key and CA files are re-read every WEED_TLS_CERT_REFRESH_INTERVAL (weed's default of 5h) and on SIGHUP, and new handshakes use the changed files without a restart; a file that no longer parses is logged and the working configuration stays. [grpc.vector] allowed_commonNames and grpc.allowed_wildcard_domain restrict which client certificates are served, checked against the subject common name as security.Authenticator and the Rust volume server's CommonNameVerifier do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * scaffold: allowed_commonNames and the refresh cadence for [grpc.vector] Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * test/s3vectors: the engine's allowlist keeps an outsider certificate out The hardened leg sets [grpc.vector] allowed_commonNames = "localhost" and probes the engine with a second certificate the same CA signed for CN=outsider: the handshake must be refused, while the gateway's certificate still gets an HTTP/2 SETTINGS frame. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * weed-vector: take the master's TLS name from the parsed endpoint A master given without a port (https://master.example) had the whole URL checked against its certificate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: bound the master TLS handshake connect_timeout covers only the TCP connect, so a master that accepted the connection and never answered the handshake held up the other masters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: look up security.toml keys the way viper does An empty WEED_* variable no longer hides the file's value, and keys match regardless of case, as for weed processes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: refuse master allowlists it cannot enforce weed checks a dialed server against allowed_server_commonNames and allowed_server_wildcard_domain on top of grpc.ca. The engine's master dial accepted every certificate the CA signed even when one was set; it now fails at startup and names the key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3vectors: read the whole frame header in the gRPC TLS probe Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test/s3vectors: probe the volume, filer and S3 gRPC listeners too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial credentials only when given masters A standalone engine never dials with grpc.client, so its settings no longer stop it from starting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: run each listener TLS handshake on its own task The accept loop awaited every handshake before accepting the next connection, so a peer that connected and sent nothing stopped every gateway connection after it. Handshakes now run concurrently, each under a 10s timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial's TLS files at every dial The client certificate, key and CA were read once at startup, so after the files rotated a reconnect still presented the old pair. Each dial now reads them, as the listener re-reads its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read every unit Go durations accept in the refresh interval WEED_TLS_CERT_REFRESH_INTERVAL values in ns, us or a leading + fell back to five hours in the engine while weed servers honored them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: fall back on a refresh interval too long to hold An oversized WEED_TLS_CERT_REFRESH_INTERVAL panicked the engine at startup; weed servers reject it and keep the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: refuse allowed_commonNames without the listener's certificate With [grpc.vector] allowed_commonNames set but no cert and key, the listener served plaintext and checked no client at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: read the master dial's credentials inside its connect timeout The CA, certificate and key were read synchronously before the timeout started, so a stalled credential path held up every other master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * weed-vector: let a set key win over an empty spelling of it With allowed_server_commonNames = "" beside allowed_server_commonnames = "master", the lookup stopped at the empty exact match and the allowlist went unnoticed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
275 lines
11 KiB
TOML
275 lines
11 KiB
TOML
# Put this file to one of the location, with descending priority
|
|
# ./security.toml
|
|
# $HOME/.seaweedfs/security.toml
|
|
# /etc/seaweedfs/security.toml
|
|
# this file is read by master, volume server, filer, and worker
|
|
|
|
# Any value below can also be supplied as an environment variable instead of
|
|
# living in this file: prefix the key with WEED_, upper-case it, and replace
|
|
# "." with "_". This is the recommended way to inject the secrets in here from
|
|
# a Kubernetes Secret (env valueFrom.secretKeyRef) rather than a ConfigMap.
|
|
# The JWT signing keys map to:
|
|
# WEED_JWT_SIGNING_KEY, WEED_JWT_SIGNING_READ_KEY (master <-> volume)
|
|
# WEED_JWT_FILER_SIGNING_KEY, WEED_JWT_FILER_SIGNING_READ_KEY (s3/clients <-> filer)
|
|
# A set env var wins over the file, so the file can stay empty (or absent).
|
|
|
|
# comma separated origins allowed to make requests to the filer and s3 gateway.
|
|
# enter in this format: https://domain.com, or http://localhost:port
|
|
[cors.allowed_origins]
|
|
values = "*"
|
|
|
|
# this jwt signing key is read by master and volume server, and it is used for write operations:
|
|
# - the Master server generates the JWT, which can be used to write a certain file on a volume server
|
|
# - the Volume server validates the JWT on writing
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.signing]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# by default, if the signing key above is set, the Volume UI over HTTP is disabled.
|
|
# by setting ui.access to true, you can re-enable the Volume UI. Despite
|
|
# some information leakage (as the UI is not authenticated), this should not
|
|
# pose a security risk.
|
|
[access]
|
|
ui = false
|
|
|
|
# by default the filer UI is enabled. This can be a security risk if the filer is exposed to the public
|
|
# and the JWT for reads is not set. If you don't want the public to have access to the objects in your
|
|
# storage, and you haven't set the JWT for reads it is wise to disable access to directory metadata.
|
|
# This disables access to the Filer UI, and will no longer return directory metadata in GET requests.
|
|
[filer.expose_directory_metadata]
|
|
enabled = true
|
|
|
|
# this jwt signing key is read by master and volume server, and it is used for read operations:
|
|
# - the Master server generates the JWT, which can be used to read a certain file on a volume server
|
|
# - the Volume server validates the JWT on reading
|
|
# NOTE: jwt for read is only supported with master+volume setup. Filer does not support this mode.
|
|
[jwt.signing.read]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
|
|
# If this JWT key is configured, Filer only accepts writes over HTTP if they are signed with this JWT:
|
|
# - f.e. the S3 API Shim generates the JWT
|
|
# - the Filer server validates the JWT on writing
|
|
# NOTE: This key is ALSO used as a fallback signing key for S3 STS if s3.iam.config does not specify a signingKey.
|
|
# NOTE: This key also gates the filer IAM gRPC service (CreateUser, PutPolicy,
|
|
# CreateAccessKey, ...). When set, every IAM RPC must carry a Bearer
|
|
# token signed with this key in its "authorization" gRPC metadata; mint
|
|
# such a token with security.GenJwtForFilerAdmin. When empty, the IAM
|
|
# gRPC service runs unauthenticated and refuses its OIDC provider and
|
|
# role RPCs. When set, set it on both filer and admin.
|
|
# The key does not protect the other filer gRPC RPCs, which can read and
|
|
# write the IAM store under /etc/iam; if the filer gRPC port is reachable
|
|
# beyond a trusted network, enable [grpc.filer] mTLS below.
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.filer_signing]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# If this JWT key is configured, Filer only accepts reads over HTTP if they are signed with this JWT:
|
|
# - f.e. the S3 API Shim generates the JWT
|
|
# - the Filer server validates the JWT on reading
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.filer_signing.read]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# gRPC mTLS configuration
|
|
# All gRPC TLS authentications are mutual (mTLS)
|
|
# The values for ca, cert, and key are paths to the certificate/key files
|
|
# The host name is not checked, so the certificate files can be shared
|
|
# Each [grpc.<component>] section also accepts optional client_cert/client_key,
|
|
# presented when that component dials other servers. Set them when your CA
|
|
# issues separate serverAuth-only and clientAuth-only certificates; when unset,
|
|
# the component reuses cert/key for both directions.
|
|
# If client and server certificates come from different issuing CAs, put both
|
|
# CA certificates in the ca PEM file.
|
|
# Once a component's cert/key are set, any error loading them or grpc.ca is
|
|
# fatal: the component never falls back to plaintext.
|
|
[grpc]
|
|
ca = ""
|
|
# Set wildcard domain for enable TLS authentication by common names
|
|
allowed_wildcard_domain = "" # .mycompany.com
|
|
# Optional allowlist a client checks the dialed server's certificate against:
|
|
# the subject common name, a DNS SAN, or a URI SAN (e.g. a SPIFFE id) must be
|
|
# listed, or match allowed_server_wildcard_domain, else the handshake fails.
|
|
# With a per-workload CA, list the identities each server may present so one
|
|
# workload cannot impersonate another component. Unset accepts any CA-signed
|
|
# server certificate, as before. Any [grpc.<component>] section may set its own
|
|
# allowed_server_commonNames to override this list, and scope it further per
|
|
# dialed target, e.g. the filer reads [grpc.filer]
|
|
# volume_allowed_server_commonNames for volume server calls.
|
|
allowed_server_commonNames = "" # comma-separated server identities
|
|
allowed_server_wildcard_domain = "" # .mycompany.com
|
|
|
|
# Volume server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to volume server
|
|
[grpc.volume]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# Master server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to master server
|
|
[grpc.master]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# Filer server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to filer server
|
|
[grpc.filer]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# S3 server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to S3 server
|
|
[grpc.s3]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.msg_broker]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.msg_agent]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.admin]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.worker]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.mq]
|
|
cert = ""
|
|
key = ""
|
|
client_cert = ""
|
|
client_key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# weed-vector gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to the vector engine; the S3
|
|
# gateway dials it with [grpc.client], and the engine dials the masters with
|
|
# [grpc.client] too. The files are re-read every WEED_TLS_CERT_REFRESH_INTERVAL
|
|
# (default 5h) and on SIGHUP.
|
|
[grpc.vector]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# gRPC client configuration for outgoing gRPC connections
|
|
# Used by clients (S3, mount, backup, benchmark, filer.copy, filer.replicate, upload, etc.)
|
|
# when connecting to any gRPC server (master, volume, filer)
|
|
[grpc.client]
|
|
cert = ""
|
|
key = ""
|
|
|
|
# HTTPS client configuration for outgoing HTTP connections
|
|
# Used by S3, mount, filer.copy, backup, and other clients when communicating with master/volume/filer
|
|
# Set enabled=true to use HTTPS instead of HTTP for data operations (separate from gRPC)
|
|
# If [https.filer] or [https.volume] are enabled on servers, clients must have [https.client] enabled=true
|
|
[https.client]
|
|
enabled = false # Set to true to enable HTTPS for all outgoing HTTP client connections
|
|
cert = "" # Client certificate for mTLS (optional if server doesn't require client cert)
|
|
key = "" # Client key for mTLS (optional if server doesn't require client cert)
|
|
ca = "" # CA certificate to verify server certificates (required when enabled=true)
|
|
insecure_skip_verify = false # Skip TLS certificate verification (NOT recommended for production)
|
|
|
|
# Volume server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to volume server
|
|
[https.volume]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Master server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to master server (web UI, HTTP API)
|
|
[https.master]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Filer server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to filer server (web UI, HTTP API)
|
|
[https.filer]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
# disable_tls_verify_client_cert = true|false (default: false)
|
|
|
|
# Admin server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to admin server
|
|
[https.admin]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Admin server authentication
|
|
# If password is set, users must login to access the admin interface
|
|
# These can be overridden by environment variables with WEED_ prefix:
|
|
# WEED_ADMIN_USER, WEED_ADMIN_PASSWORD
|
|
# WEED_ADMIN_READONLY_USER, WEED_ADMIN_READONLY_PASSWORD
|
|
[admin]
|
|
user = ""
|
|
password = ""
|
|
|
|
[admin.readonly]
|
|
user = ""
|
|
password = ""
|
|
|
|
# SSE-S3 server-side encryption key management
|
|
# These settings configure the Key Encryption Key (KEK) for S3 SSE-S3 encryption.
|
|
# Set exactly one of kek or key. If neither is set, SSE-S3 is disabled.
|
|
# Can also be set via env vars: WEED_S3_SSE_KEK, WEED_S3_SSE_KEY
|
|
[s3.sse]
|
|
# hex-encoded 256-bit key, same format as the legacy /etc/s3/sse_kek filer file.
|
|
# Use this to migrate from a filer-stored KEK: copy the value from /etc/s3/sse_kek.
|
|
# Generate a new one with: openssl rand -hex 32
|
|
kek = ""
|
|
# any secret string; a 256-bit key is derived automatically via HKDF-SHA256.
|
|
# Cannot be used while /etc/s3/sse_kek exists on the filer — delete it first.
|
|
key = ""
|
|
|
|
# Trusted reverse proxies allowed to set forwarding headers on behalf of the
|
|
# real client: X-Forwarded-For / X-Real-Ip for aws:SourceIp condition
|
|
# evaluation in S3 bucket and IAM policies, and X-Forwarded-Prefix for SigV4
|
|
# signature verification of prefix-stripped paths. When the direct TCP peer is
|
|
# in this list, these headers are honored; otherwise they are ignored. A
|
|
# trusted proxy must overwrite or strip client-supplied values of these
|
|
# headers, since honoring them means trusting them. Leave empty to always use
|
|
# the direct peer, which is the safe default.
|
|
# Can also be set via env var: WEED_S3_TRUSTED_PROXIES_WHITE_LIST
|
|
[s3.trusted_proxies]
|
|
white_list = "" # comma separated; bare IPs or CIDRs, e.g. "10.10.10.0/24"
|
|
|
|
# white list. It's checking request ip address.
|
|
[guard]
|
|
white_list = ""
|