Files
seaweedfs/seaweed-volume/src/config.rs
T
0d721fccee fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)
* fix(volume): keep preallocated growth above -volume.minFreeSpace

When free space fell below -volume.minFreeSpace, every volume went
read-only and the master kept growing new volumes. With preallocation on,
each new volume reserved its full size, driving the disk further below the
floor (15 Oct incident: 66 empty volumes reserved about 1980 GiB).

Two causes, both in weed/storage/store.go:

- MaybeAdjustVolumeMax sized a disk's volume slots from raw free space, so
  space inside the floor was counted as available. It now counts only free
  space above the floor.
- addVolume trusted the once-a-minute low-disk flag. A preallocated volume
  is now refused when the disk is low now, or when its reservation would
  not fit above the floor. Volumes without preallocation, including merges
  that copy existing data, are not affected.

addVolume runs under a store-level lock, so two concurrent creates cannot
both pass the room check against the same free space.

* volume: mirror headroom guard in the rust store

find_free_location only trusted the once-a-probe is_disk_space_low flag,
so a burst of preallocated creates could still run the disk under
-volume.minFreeSpace between probes, and maybe_adjust_volume_max sized
slots from raw free space. Adds MinFreeSpace::is_low/available_space to
match Go and applies both checks the same way. store.write() already
serializes add_volume, so no extra lock is needed.

* volume: read rust headroom tests from a fake disk

The floor arithmetic used a live free-space reading, so unrelated disk
activity could flip a test between pass and fail. Store.disk_stats_of is
now the same seam Go uses (diskStatusOf): the growth-loop and floor tests
run against a controlled disk, mirroring the Go suite one for one.

* volume: hold back create metadata from the headroom check

An exact-fit preallocation left free space exactly at the floor, so the
.vif and .idx writes that follow a create could dip under it and get the
location flagged read-only at the next probe. Reserve room for .vif on
the data filesystem, and for .idx growth only when dir_idx shares it.

* volume: mirror the metadata allowance in the rust store

Same rule as the Go side: charge .vif against the checked filesystem and
.idx only when dir_idx shares it, and grow the exact-fit test's fake
disk by the per-create cost.

* volume: skip the reservation check where create cannot preallocate

* volume: mirror the platform preallocation gate in the rust store

* volume.copy: count the preallocation reservation in the destination space check

The copy path picked a location with room for the copied .dat, then reserved
the master's full volume size limit on it. A volume smaller than the limit
could land on a disk whose free space above -volume.minFreeSpace fit the
copy but not the reservation, driving the disk under the floor the same way
the growth loop did.

Fetch the preallocation setting before choosing the location and require
room for the reservation itself, only where CreateVolumeFile actually
reserves blocks.

* seaweed-volume: store min-free-space-adjusted available space like Go

Go's DiskLocation.AvailableSpace holds free space above the
-volume.minFreeSpace floor; the Rust probe cached raw free instead, so the
VolumeCopy location check could accept a disk already inside the floor.
Store the same adjusted figure and cover it with a test.

* seaweed-volume: reserve the copied .dat like Go's CreateVolumeFile

VolumeCopy called set_len for the reservation, which only grows a sparse
file — no blocks are held, so a disk with room for the copied bytes but
not the full reservation only fails mid-copy. preallocate_file fallocates
KEEP_SIZE on Linux like Go, reserving the blocks up front; off Linux it
is a no-op, matching PREALLOCATE_SUPPORTED.

* seaweed-volume: charge a copy's destination pick for the .dat reservation

Go's VolumeCopy now sizes its free-location check against the master's
volume size limit when the platform preallocates, not just the copied
bytes. The Rust pick only measured the copied bytes, so a destination
could accept a copy whose reservation then dips under the floor. The
config fetch moves ahead of the pick (same order as Go) and feeds both
the space check and the reservation itself.

* volume.copy: charge the picked location's free space under volumeCreateLock

The copy path picked a destination off the once-a-minute AvailableSpace
probe and released the decision long before the .dat reservation landed.
Two copies could pass the same cached headroom and together drive the disk
under -volume.minFreeSpace.

ReserveFreeLocation serializes the pick under volumeCreateLock, verifies
the disk still has live room for the reservation, and charges it against
the location's cached free space until the next probe refreshes it.

* seaweed-volume: mirror the copy pick's live-check and headroom charge

Go's VolumeCopy now picks through Store.ReserveFreeLocation: live disk
stats under the store write lock, then the reservation charged against the
location's cached free space. The Rust plan_copy_destination gains the
same store.reserve_free_location so two copies cannot spend the same
headroom before the first .dat reservation lands.

* volume.copy: saturate the free-space charge against a live-space drop

The two statfs reads bracketing a pick can race a large external write; a
plain subtraction would underflow and leave the location looking roomy
until the next probe.

* volume.copy: release the headroom reservation when a copy does not land

ReserveFreeLocation charges the picked location's cached free space, but
a failed copy never returned it, so the headroom stayed booked until the
next disk-space probe and blocked retries. ReleaseFreeLocation refunds
the charge; the copy's cleanup defer now registers with the reservation
so every failure path (existing-replica delete, .note write, stream send)
releases it, including the post-transfer mount/validate failures covered
by the outer defer.

* seaweed-volume: mirror the copy headroom release on failed copies

release_free_location refunds reserve_free_location's charge. The copy
returns it when the plan's existing-replica delete fails, when the .note
write fails, and in cleanup_failed_copy, so a failed copy stops holding
the headroom until the next disk-space probe.

* volume.copy: keep pending copy headroom out of probe readings

ReleaseFreeLocation refunded the reservation into AvailableSpace even when
a CheckDiskSpace probe had already rebased the cache on live stats — the
probe's reading still shows the in-flight copy's bytes as free, so the
refund double-counted headroom and two copies could spend the same space,
driving the disk under -volume.minFreeSpace.

Track the outstanding charge in DiskLocation.ReservedSpace instead: probes
subtract it so they cannot re-spend it, a failed copy removes only its own
charge, and a landed copy commits it — the charge and the new usage cancel
out. hasRoomForNewVolume respects pending headroom too, so a preallocating
create cannot overrun an in-flight copy.

* seaweed-volume: mirror pending copy headroom across disk probes

reserved_space tracks the in-flight copy's charge separately from the
cached free space: check_disk_space subtracts it, release removes only the
failed copy's charge, a landed copy commits it, and has_room_for_new_volume
respects it — matching the Go side.

* volume.copy: settle a copy's headroom as its bytes land on disk

The pending charge held the copy's full reservation until commit, but
written and fallocated bytes already show up in statfs, so the live-space
check counted them twice. A slow copy could then block other copies and
volume growth despite free space above -volume.minFreeSpace.

Settle the charge instead: commit the bytes CreateVolumeFile actually
reserved (a failed fallocate leaves nothing committed), then settle each
file's write progress as it lands. The failure paths now release only the
unsettled remainder; allocatedSize is exported for the .dat's st_blocks.

* seaweed-volume: mirror copy headroom settling as bytes land

reserved_space keeps only the promised bytes statfs cannot see yet:
preallocate_dat settles whatever blocks the filesystem actually reserved
(0 when fallocate cannot), each copied file's progress settles the rest as
it lands, and a failed copy releases only the unsettled remainder —
matching the Go side.

* volume.copy: release a failed copy's headroom charge only once

Both defers fired on the same in-closure error: the inner released the
unsettled charge and the outer released it again, driving ReservedSpace
negative (an atomic.Uint64 underflow) and inflating AvailableSpace —
the location would then fail every later headroom check, or worse,
pass them with fabricated space. Zero the charge once it is released;
the Rust mirror already releases via swap(0) for the same reason.

* volume.copy: write into the preallocated .dat instead of staging over it

The .dat stream staged into .copying and renamed over the file
CreateVolumeFile had just fallocated, so the copy's transient footprint
was preallocate+dat — more than the reservation the pick just checked —
and the reservation itself was discarded at rename. Writing in place
reuses the fallocated extent, keeps the peak at the reserved maximum,
and leaves the copied volume preallocated like a created one.

* seaweed-volume: mirror the in-place write into the preallocated .dat

The .dat copy opened the just-fallocated file with truncate, freeing its
blocks at open and spending the headroom again as the stream re-grew it.
Append mode preserves them, matching the Go side.

---------

Co-authored-by: chrislusf <chrislusf@gmail.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
2026-10-26 08:03:25 -07:00

2201 lines
77 KiB
Rust

use clap::Parser;
use std::ffi::OsString;
use std::net::UdpSocket;
use std::path::{Path, PathBuf};
use crate::security::tls::TlsPolicy;
/// How this module reads the environment.
///
/// Production passes [`process_env`]; tests pass a map. Injecting the lookup
/// is what lets the tests cover `HOME` and the `WEED_*` overrides without
/// `std::env::set_var`, which is `unsafe` in Rust 2024 because it races with
/// every other thread in the process — and the lib test binary runs tokio
/// runtimes, TLS stacks and AWS clients that read the environment lazily on
/// threads no test mutex can reach.
type EnvLookup<'a> = &'a dyn Fn(&str) -> Option<OsString>;
/// The real process environment, as an [`EnvLookup`].
fn process_env(key: &str) -> Option<OsString> {
std::env::var_os(key)
}
/// `std::env::var`'s view of an [`EnvLookup`]: unset and non-UTF-8 both read
/// as absent, exactly as `std::env::var(key).ok()` does.
fn env_string(env: EnvLookup<'_>, key: &str) -> Option<String> {
env(key).and_then(|value| value.into_string().ok())
}
/// SeaweedFS Volume Server (Rust implementation)
///
/// Start a volume server to provide storage spaces.
#[derive(Parser, Debug)]
#[command(name = "weed-volume", version, about)]
pub struct Cli {
/// HTTP listen port
#[arg(long = "port", default_value_t = 8080)]
pub port: u16,
/// gRPC listen port. If 0, defaults to port + 10000.
#[arg(long = "port.grpc", default_value_t = 0)]
pub port_grpc: u16,
/// Port opened to public. If 0, defaults to same as --port.
#[arg(long = "port.public", default_value_t = 0)]
pub port_public: u16,
/// IP or server name, also used as identifier.
/// If empty, auto-detected.
#[arg(long = "ip", default_value = "")]
pub ip: String,
/// Volume server ID. If empty, defaults to ip:port.
#[arg(long = "id", default_value = "")]
pub id: String,
/// Publicly accessible address.
#[arg(long = "publicUrl", default_value = "")]
pub public_url: String,
/// IP address to bind to. If empty, defaults to same as --ip.
#[arg(long = "ip.bind", default_value = "")]
pub bind_ip: String,
/// Comma-separated master server addresses.
#[arg(long = "master", default_value = "localhost:9333")]
pub master: String,
/// Comma-separated master servers (deprecated, use --master instead).
#[arg(long = "mserver", default_value = "")]
pub mserver: String,
/// Number of seconds between stop sending heartbeats and stopping the volume server.
#[arg(long = "preStopSeconds", default_value_t = 10)]
pub pre_stop_seconds: u32,
/// Connection idle seconds.
#[arg(long = "idleTimeout", default_value_t = 30)]
pub idle_timeout: u32,
/// Current volume server's data center name.
#[arg(long = "dataCenter", default_value = "")]
pub data_center: String,
/// Current volume server's rack name.
#[arg(long = "rack", default_value = "")]
pub rack: String,
/// Choose [memory|redb|redbMedium|redbLarge] mode for memory~performance balance.
/// The redb tiers give each volume's on-disk index a 4, 8, or 16 MiB page
/// cache respectively; total index memory is roughly (volumes x cache).
/// `leveldb`/`leveldbMedium`/`leveldbLarge` are accepted as aliases for the
/// corresponding redb backends (Rust volume server uses redb under the hood).
#[arg(long = "index", default_value = "memory")]
pub index: String,
/// [hdd|ssd|<tag>] hard drive or solid state drive or any tag.
#[arg(long = "disk", default_value = "")]
pub disk: String,
/// Comma-separated tag groups per data dir; each group uses ':' (e.g. fast:ssd,archive).
#[arg(long = "tags", default_value = "")]
pub tags: String,
/// Adjust jpg orientation when uploading.
#[arg(long = "images.fix.orientation", default_value_t = false)]
pub fix_jpg_orientation: bool,
/// [local|proxy|redirect] how to deal with non-local volume.
#[arg(long = "readMode", default_value = "proxy")]
pub read_mode: String,
/// CPU profile output file.
#[arg(long = "cpuprofile", default_value = "")]
pub cpu_profile: String,
/// Memory profile output file.
#[arg(long = "memprofile", default_value = "")]
pub mem_profile: String,
/// Limit background compaction or copying speed in mega bytes per second.
#[arg(long = "compactionMBps", default_value_t = 0)]
pub compaction_mb_per_second: u32,
/// Limit maintenance (replication/balance) IO rate in MB/s. 0 means no limit.
#[arg(long = "maintenanceMBps", default_value_t = 0)]
pub maintenance_mb_per_second: u32,
/// Limit how many vacuum commits run at once on this server; queued
/// commits hold no volume lock. 0 means unlimited.
#[arg(long = "compactionMaxParallelCommits", default_value_t = 0)]
pub max_parallel_vacuum_commits: usize,
/// Limit file size to avoid out of memory.
#[arg(long = "fileSizeLimitMB", default_value_t = 256)]
pub file_size_limit_mb: u32,
/// Limit total concurrent upload size in MB, 0 means unlimited.
#[arg(long = "concurrentUploadLimitMB", default_value_t = 0)]
pub concurrent_upload_limit_mb: u32,
/// Limit total concurrent download size in MB, 0 means unlimited.
#[arg(long = "concurrentDownloadLimitMB", default_value_t = 0)]
pub concurrent_download_limit_mb: u32,
/// Enable pprof-equivalent HTTP handlers. Precludes --memprofile and --cpuprofile.
#[arg(long = "pprof", default_value_t = false)]
pub pprof: bool,
/// Prometheus metrics listen port.
#[arg(long = "metricsPort", default_value_t = 0)]
pub metrics_port: u16,
/// Metrics listen IP. If empty, defaults to same as --ip.bind.
#[arg(long = "metricsIp", default_value = "")]
pub metrics_ip: String,
/// Directories to store data files. dir[,dir]...
/// If empty, defaults to the platform temp directory (Go's os.TempDir()).
#[arg(long = "dir", default_value = "")]
pub dir: String,
/// Directory to store .idx files.
#[arg(long = "dir.idx", default_value = "")]
pub dir_idx: String,
/// Maximum numbers of volumes, count[,count]...
/// If set to zero, the limit will be auto configured as free disk space divided by volume size.
#[arg(long = "max", default_value = "8")]
pub max: String,
/// Comma separated IP addresses having write permission. No limit if empty.
#[arg(long = "whiteList", default_value = "")]
pub white_list: String,
/// Minimum free disk space (default to 1%). Low disk space will mark all volumes as ReadOnly.
/// Deprecated: use --minFreeSpace instead.
#[arg(long = "minFreeSpacePercent", default_value = "1")]
pub min_free_space_percent: String,
/// Min free disk space (value<=100 as percentage like 1, other as human readable bytes, like 10GiB).
/// Low disk space will mark all volumes as ReadOnly.
#[arg(long = "minFreeSpace", default_value = "")]
pub min_free_space: String,
/// Inflight upload data wait timeout of volume servers.
/// Go `time.Duration` syntax: `300ms`, `1.5h`, `2h45m`; a bare number other than `0` is rejected.
#[arg(
long = "inflightUploadDataTimeout",
default_value = "60s",
value_parser = parse_go_duration
)]
pub inflight_upload_data_timeout: std::time::Duration,
/// Inflight download data wait timeout of volume servers.
/// Go `time.Duration` syntax: `300ms`, `1.5h`, `2h45m`; a bare number other than `0` is rejected.
#[arg(
long = "inflightDownloadDataTimeout",
default_value = "60s",
value_parser = parse_go_duration
)]
pub inflight_download_data_timeout: std::time::Duration,
/// <experimental> if true, prevents slow reads from blocking other requests,
/// but large file read P99 latency will increase.
#[arg(long = "hasSlowRead", default_value_t = true)]
pub has_slow_read: bool,
/// <experimental> larger values can optimize query performance but will increase memory usage.
/// Use with hasSlowRead normally.
#[arg(long = "readBufferSizeMB", default_value_t = 4)]
pub read_buffer_size_mb: u32,
/// Alive time for leveldb (default to 0). If leveldb of volume is not accessed in
/// ldbTimeout hours, it will be offloaded to reduce opened files and memory consumption.
#[arg(long = "index.leveldbTimeout", default_value_t = 0)]
pub ldb_timeout: i64,
/// Serves runtime profiling data on the port specified by --debug.port.
#[arg(long = "debug", default_value_t = false)]
pub debug: bool,
/// HTTP port for debugging.
#[arg(long = "debug.port", default_value_t = 6060)]
pub debug_port: u16,
/// Path to security.toml configuration file for JWT signing keys.
#[arg(long = "securityFile", default_value = "")]
pub security_file: String,
/// If true, FetchAndWriteNeedle accepts arbitrary remote S3 endpoints
/// including loopback / link-local hosts. Default rejects internal /
/// metadata endpoints.
#[arg(long = "volume.allowUntrustedRemoteEndpoints", default_value_t = false)]
pub allow_untrusted_remote_endpoints: bool,
/// Minimum bytes an fsync batch gathers before a drained queue flushes
/// without waiting.
#[arg(long = "fsyncMinBatchBytes", default_value_t = 1024 * 1024)]
pub fsync_min_batch_bytes: u64,
/// Minimum requests an fsync batch gathers before a drained queue flushes
/// without waiting.
#[arg(long = "fsyncMinBatchRequests", default_value_t = 128)]
pub fsync_min_batch_requests: usize,
/// How long a below-minimum fsync batch waits for more requests once the
/// queue drains; 0 flushes immediately.
#[arg(
long = "fsyncBatchWait",
default_value = "500us",
value_parser = parse_go_duration
)]
pub fsync_batch_wait: std::time::Duration,
/// A file of command line options, each line in optionName=optionValue format.
#[arg(long = "options", default_value = "")]
pub options: String,
}
/// Resolved configuration after applying defaults and validation.
#[derive(Debug)]
pub struct VolumeServerConfig {
pub port: u16,
pub grpc_port: u16,
pub public_port: u16,
pub ip: String,
pub bind_ip: String,
pub public_url: String,
pub id: String,
pub masters: Vec<String>,
pub pre_stop_seconds: u32,
pub idle_timeout: u32,
pub data_center: String,
pub rack: String,
pub index_type: NeedleMapKind,
pub disk_type: String,
pub folders: Vec<String>,
pub folder_max_limits: Vec<i32>,
pub folder_tags: Vec<Vec<String>>,
pub min_free_spaces: Vec<MinFreeSpace>,
pub disk_types: Vec<String>,
pub idx_folder: String,
pub white_list: Vec<String>,
pub fix_jpg_orientation: bool,
pub read_mode: ReadMode,
pub cpu_profile: String,
pub mem_profile: String,
pub compaction_byte_per_second: i64,
pub maintenance_byte_per_second: i64,
/// Max concurrent vacuum commits; 0 means unlimited.
pub max_parallel_vacuum_commits: usize,
pub file_size_limit_bytes: i64,
pub concurrent_upload_limit: i64,
pub concurrent_download_limit: i64,
pub inflight_upload_data_timeout: std::time::Duration,
pub inflight_download_data_timeout: std::time::Duration,
pub has_slow_read: bool,
pub read_buffer_size_mb: u32,
pub ldb_timeout: i64,
pub pprof: bool,
pub metrics_port: u16,
pub metrics_ip: String,
pub debug: bool,
pub debug_port: u16,
pub ui_enabled: bool,
pub jwt_signing_key: Vec<u8>,
pub jwt_signing_expires_seconds: i64,
pub jwt_read_signing_key: Vec<u8>,
pub jwt_read_signing_expires_seconds: i64,
pub https_cert_file: String,
pub https_key_file: String,
pub https_ca_file: String,
pub https_client_enabled: bool,
pub https_client_cert_file: String,
pub https_client_key_file: String,
pub https_client_ca_file: String,
pub grpc_cert_file: String,
pub grpc_key_file: String,
pub grpc_client_cert_file: String,
pub grpc_client_key_file: String,
pub grpc_ca_file: String,
pub grpc_allowed_wildcard_domain: String,
pub grpc_volume_allowed_common_names: Vec<String>,
pub tls_policy: TlsPolicy,
/// Path to security.toml — stored for SIGHUP reload.
pub security_file: String,
/// If true, FetchAndWriteNeedle skips remote S3 endpoint validation
/// (allows loopback / link-local / metadata hosts).
pub allow_untrusted_remote_endpoints: bool,
/// Minimum bytes an fsync batch gathers before a drained queue flushes
/// without waiting.
pub fsync_min_batch_bytes: u64,
/// Minimum requests an fsync batch gathers before a drained queue flushes
/// without waiting.
pub fsync_min_batch_requests: usize,
/// How long a below-minimum fsync batch waits for stragglers.
pub fsync_batch_wait: std::time::Duration,
}
pub use crate::storage::needle_map::NeedleMapKind;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ReadMode {
Local,
Proxy,
Redirect,
}
#[derive(Debug, Clone)]
pub enum MinFreeSpace {
Percent(f64),
Bytes(u64),
}
impl MinFreeSpace {
/// Mirrors Go's MinFreeSpace.IsLow.
pub fn is_low(&self, free_bytes: u64, total_bytes: u64) -> bool {
match self {
MinFreeSpace::Percent(pct) => free_bytes as f64 / total_bytes as f64 * 100.0 < *pct,
MinFreeSpace::Bytes(min) => free_bytes < *min,
}
}
/// Free bytes above the floor; mirrors Go's MinFreeSpace.AvailableSpace.
pub fn available_space(&self, free_bytes: u64, total_bytes: u64) -> u64 {
let floor = match self {
MinFreeSpace::Percent(pct) => (total_bytes as f64 * pct / 100.0) as u64,
MinFreeSpace::Bytes(min) => *min,
};
free_bytes.saturating_sub(floor)
}
}
/// Convert single-dash long options to double-dash for clap compatibility.
/// Go's `flag` package uses `-port`, clap expects `--port`.
/// This allows both `-port 8080` and `--port 8080` to work.
fn normalize_args_vec(args: Vec<String>) -> Vec<String> {
let mut args = args;
// Skip args[0] (binary name).
let mut i = 1;
while i < args.len() {
let arg = &args[i];
// Stop processing after "--"
if arg == "--" {
break;
}
// Already double-dash or not a flag: leave as-is
if arg.starts_with("--") || !arg.starts_with('-') {
i += 1;
continue;
}
// Single char flags like -h, -V: leave as-is
let without_dash = &arg[1..];
// Check if it's a single-dash long option: more than 1 char and not a negative number
if without_dash.len() > 1 && !without_dash.starts_with(|c: char| c.is_ascii_digit()) {
// Handle -key=value format
if let Some(eq_pos) = without_dash.find('=') {
let key = &without_dash[..eq_pos];
if key.len() > 1 {
args[i] = format!("--{}", without_dash);
}
} else {
args[i] = format!("-{}", arg);
}
}
i += 1;
}
args
}
/// Parse CLI arguments and resolve all defaults — mirroring Go's `runVolume()` + `startVolumeServer()`.
///
/// Supports `-options <file>` to load defaults from a file (same format as Go's fla9).
/// CLI arguments take precedence over file values.
pub fn parse_cli() -> VolumeServerConfig {
let args: Vec<String> = std::env::args().collect();
let normalized = normalize_args_vec(args);
let merged = merge_options_file(normalized);
let cli = Cli::parse_from(merged);
resolve_config_with_env(cli, &process_env)
}
/// Find `-options`/`--options` in args, parse the referenced file, and inject
/// file-based defaults for any flags not already set on the command line.
///
/// File format (matching Go's fla9.ParseFile):
/// - One option per line: `key=value`, `key value`, or `key:value`
/// - Lines starting with `#` are comments; blank lines are ignored
/// - Leading `-` on key names is stripped
/// - CLI arguments take precedence over file values
fn merge_options_file(args: Vec<String>) -> Vec<String> {
// Find the options file path from the args
let options_path = find_options_arg(&args);
if options_path.is_empty() {
return args;
}
let content = match std::fs::read_to_string(&options_path) {
Ok(c) => c,
Err(e) => {
eprintln!(
"WARNING: could not read options file {}: {}",
options_path, e
);
return args;
}
};
// Collect which flags are already explicitly set on the command line.
let mut cli_flags: std::collections::HashSet<String> = std::collections::HashSet::new();
let mut i = 1; // skip binary name
while i < args.len() {
let arg = &args[i];
if arg == "--" {
break;
}
if let Some(long) = arg.strip_prefix("--") {
let key = if let Some(eq) = long.find('=') {
long[..eq].to_string()
} else {
long.to_string()
};
cli_flags.insert(key);
} else if arg.len() > 2
&& let Some(without_dash) = arg.strip_prefix('-')
{
// Single-dash long option (already normalized to -- at this point,
// but handle both for safety)
let key = if let Some(eq) = without_dash.find('=') {
without_dash[..eq].to_string()
} else {
without_dash.to_string()
};
cli_flags.insert(key);
}
i += 1;
}
// Parse file and append missing options
let mut extra_args: Vec<String> = Vec::new();
for line in content.lines() {
let trimmed = line.trim();
if trimmed.is_empty() || trimmed.starts_with('#') {
continue;
}
// Split on first `=`, ` `, or `:`
let (name, value) = if let Some(pos) = trimmed.find(['=', ' ', ':']) {
(
trimmed[..pos].trim().to_string(),
trimmed[pos + 1..].trim().to_string(),
)
} else {
(trimmed.to_string(), String::new())
};
// Strip leading dashes from name
let name = name.trim_start_matches('-').to_string();
if name.is_empty() || name == "options" {
continue;
}
// Skip if already set on CLI
if cli_flags.contains(&name) {
continue;
}
extra_args.push(format!("--{}", name));
if !value.is_empty() {
extra_args.push(value);
}
}
let mut merged = args;
merged.extend(extra_args);
merged
}
/// Extract the options file path from args (looks for --options or -options).
fn find_options_arg(args: &[String]) -> String {
for i in 1..args.len() {
if (args[i] == "--options" || args[i] == "-options") && i + 1 < args.len() {
return args[i + 1].clone();
}
if let Some(rest) = args[i].strip_prefix("--options=") {
return rest.to_string();
}
if let Some(rest) = args[i].strip_prefix("-options=") {
return rest.to_string();
}
}
String::new()
}
/// Parse a duration with Go's `time.ParseDuration` grammar, as Go's
/// `flag.Duration` flags do: `[0-9]*(\.[0-9]*)?<unit>` repeated, units
/// `ns`/`us`/`µs`/`μs`/`ms`/`s`/`m`/`h`. A bare `0` is the only unit-less
/// value; a leading sign parses in Go but a negative has no
/// `std::time::Duration`, so it is an error, not a clamp.
fn parse_go_duration(s: &str) -> Result<std::time::Duration, String> {
let orig = s;
let invalid = || format!("time: invalid duration {:?}", orig);
let mut s = s;
if let Some(rest) = s.strip_prefix('-') {
if rest.is_empty() {
return Err(invalid());
}
return Err(format!("negative duration {:?} is not supported", orig));
}
if let Some(rest) = s.strip_prefix('+') {
s = rest;
}
if s == "0" {
return Ok(std::time::Duration::ZERO);
}
if s.is_empty() {
return Err(invalid());
}
// Go bounds the total by 1<<63 (int64 nanoseconds): overflow is an error.
const MAX: u64 = 1 << 63;
let mut total: u64 = 0;
let mut bytes = s.as_bytes();
while !bytes.is_empty() {
if !(bytes[0] == b'.' || bytes[0].is_ascii_digit()) {
return Err(invalid());
}
// Consume [0-9]*
let mut i = 0;
let mut v: u64 = 0;
while i < bytes.len() && bytes[i].is_ascii_digit() {
v = v
.checked_mul(10)
.and_then(|v| v.checked_add(u64::from(bytes[i] - b'0')))
.filter(|v| *v <= MAX)
.ok_or_else(invalid)?;
i += 1;
}
let pre = i != 0;
bytes = &bytes[i..];
// Consume (\.[0-9]*)?
let mut f: u64 = 0;
let mut scale: f64 = 1.0;
let mut post = false;
if let Some((b'.', rest)) = bytes.split_first() {
bytes = rest;
let mut i = 0;
let mut overflow = false;
while i < bytes.len() && bytes[i].is_ascii_digit() {
if !overflow {
// Like Go's leadingFraction, digits past ~19 are dropped.
match f
.checked_mul(10)
.and_then(|f| f.checked_add(u64::from(bytes[i] - b'0')))
.filter(|f| *f <= MAX)
{
Some(next) => {
f = next;
scale *= 10.0;
}
None => overflow = true,
}
}
i += 1;
}
post = i != 0;
bytes = &bytes[i..];
}
if !pre && !post {
return Err(invalid());
}
// Consume the unit: everything up to the next digit or '.'.
let unit_len = bytes
.iter()
.position(|c| *c == b'.' || c.is_ascii_digit())
.unwrap_or(bytes.len());
if unit_len == 0 {
return Err(format!("time: missing unit in duration {:?}", orig));
}
// The boundary is an ASCII byte or the end, so it is a char boundary.
let unit = std::str::from_utf8(&bytes[..unit_len]).map_err(|_| invalid())?;
bytes = &bytes[unit_len..];
let unit_nanos: u64 = match unit {
"ns" => 1,
"us" | "\u{00b5}s" | "\u{03bc}s" => 1_000,
"ms" => 1_000_000,
"s" => 1_000_000_000,
"m" => 60 * 1_000_000_000,
"h" => 3600 * 1_000_000_000,
_ => {
return Err(format!(
"time: unknown unit {:?} in duration {:?}",
unit, orig
));
}
};
if v > MAX / unit_nanos {
return Err(invalid());
}
v *= unit_nanos;
if f > 0 {
// Go's float math, so `1.5h` rounds the same way on both sides.
v += (f as f64 * (unit_nanos as f64 / scale)) as u64;
if v > MAX {
return Err(invalid());
}
}
total = total
.checked_add(v)
.filter(|t| *t <= MAX)
.ok_or_else(invalid)?;
}
if total > MAX - 1 {
return Err(invalid());
}
Ok(std::time::Duration::from_nanos(total))
}
/// Byte-size suffixes of Go's `util.ParseBytes`, lower-cased: SI (`kb`, `k`)
/// and IEC (`kib`, `ki`) forms, `b`, and no suffix.
fn byte_unit_multiplier(unit: &str) -> Option<u64> {
const K: u64 = 1_000;
const KI: u64 = 1 << 10;
Some(match unit {
"" | "b" => 1,
"kb" | "k" => K,
"kib" | "ki" => KI,
"mb" | "m" => K * K,
"mib" | "mi" => KI * KI,
"gb" | "g" => K * K * K,
"gib" | "gi" => KI * KI * KI,
"tb" | "t" => K * K * K * K,
"tib" | "ti" => KI * KI * KI * KI,
"pb" | "p" => K * K * K * K * K,
"pib" | "pi" => KI * KI * KI * KI * KI,
"eb" | "e" => K * K * K * K * K * K,
"eib" | "ei" => KI * KI * KI * KI * KI * KI,
_ => return None,
})
}
/// Parse a human-readable byte count with Go's `util.ParseBytes` grammar:
/// decimal number, optional whitespace, then a [`byte_unit_multiplier`]
/// suffix, case-insensitive. `10GiB` → 10737418240, `1.5TB` → 1500000000000.
///
/// Thousands commas parse as in Go but never arrive from `-minFreeSpace`,
/// which is split on commas into per-directory entries first.
fn parse_bytes(s: &str) -> Result<u64, String> {
let digits_end = s
.find(|c: char| !(c.is_ascii_digit() || c == '.' || c == ','))
.unwrap_or(s.len());
let num = s[..digits_end].replace(',', "");
let value: f64 = num
.parse()
.map_err(|e| format!("invalid byte size {:?}: {}", s, e))?;
let unit = s[digits_end..].trim().to_lowercase();
let multiplier =
byte_unit_multiplier(&unit).ok_or_else(|| format!("unhandled size name: {}", unit))?;
let bytes = value * multiplier as f64;
if bytes >= u64::MAX as f64 {
return Err(format!("too large: {}", s));
}
Ok(bytes as u64)
}
/// Parse one `-minFreeSpace` entry like Go's `util.ParseMinFreeSpace()`: a
/// plain number is a percentage in `0..=100`, anything else a byte size
/// above 100 bytes. `150` errors rather than reading as bytes.
fn parse_min_free_space(s: &str) -> Result<MinFreeSpace, String> {
let s = s.trim();
if let Ok(percent) = s.parse::<f64>() {
// Go's range check lets NaN through; rejecting it is the one
// intentional difference.
if !(0.0..=100.0).contains(&percent) {
return Err(format!("minFreeSpace is invalid: {:?}", s));
}
return Ok(MinFreeSpace::Percent(percent));
}
match parse_bytes(s) {
Ok(bytes) if bytes > 100 => Ok(MinFreeSpace::Bytes(bytes)),
_ => Err(format!("minFreeSpace is invalid: {:?}", s)),
}
}
/// Parse minFreeSpace / minFreeSpacePercent like Go's
/// `util.MustParseMinFreeSpace()`: `--minFreeSpace` wins when set, and any
/// invalid comma-separated entry fails the whole list.
fn parse_min_free_spaces(
min_free_space: &str,
min_free_space_percent: &str,
) -> Result<Vec<MinFreeSpace>, String> {
let source = if !min_free_space.is_empty() {
min_free_space
} else {
min_free_space_percent
};
source.split(',').map(parse_min_free_space).collect()
}
/// Parse comma-separated tag groups like "fast:ssd,archive" into per-folder tag vectors.
/// Mirrors Go's `parseVolumeTags()`.
fn parse_volume_tags(tags_arg: &str, folder_count: usize) -> Vec<Vec<String>> {
if folder_count == 0 {
return vec![];
}
let tags_arg = tags_arg.trim();
let tag_entries: Vec<&str> = if tags_arg.is_empty() {
vec![]
} else {
tags_arg.split(',').collect()
};
let mut folder_tags: Vec<Vec<String>> = vec![vec![]; folder_count];
if tag_entries.len() == 1 && !tag_entries[0].is_empty() {
// Single entry: replicate to all folders
let normalized: Vec<String> = tag_entries[0]
.split(':')
.map(|t| t.trim().to_lowercase())
.filter(|t| !t.is_empty())
.collect();
for tags in folder_tags.iter_mut() {
*tags = normalized.clone();
}
} else {
for (i, tags) in folder_tags.iter_mut().enumerate() {
if i < tag_entries.len() {
*tags = tag_entries[i]
.split(':')
.map(|t| t.trim().to_lowercase())
.filter(|t| !t.is_empty())
.collect();
}
}
}
folder_tags
}
fn resolve_config_with_env(cli: Cli, env: EnvLookup<'_>) -> VolumeServerConfig {
// Backward compatibility: --mserver overrides --master
let master_string = if !cli.mserver.is_empty() {
&cli.mserver
} else {
&cli.master
};
let masters: Vec<String> = master_string
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
// Parse folders
let dir_value = if cli.dir.trim().is_empty() {
default_volume_dir()
} else {
cli.dir
};
let folders: Vec<String> = dir_value
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
let folder_count = folders.len();
// Parse max volume counts
let mut folder_max_limits: Vec<i32> = cli
.max
.split(',')
.map(|s| {
s.trim().parse::<i32>().unwrap_or_else(|_| {
panic!("The max specified in --max is not a valid number: {}", s)
})
})
.collect();
// Replicate single value to all folders
if folder_max_limits.len() == 1 && folder_count > 1 {
let v = folder_max_limits[0];
folder_max_limits.resize(folder_count, v);
}
if folders.len() != folder_max_limits.len() {
panic!(
"{} directories by --dir, but only {} max is set by --max",
folders.len(),
folder_max_limits.len()
);
}
// Parse min free spaces
let mut min_free_spaces =
parse_min_free_spaces(&cli.min_free_space, &cli.min_free_space_percent).unwrap_or_else(
|e| panic!("The value specified in -minFreeSpace not a valid value: {e}"),
);
if min_free_spaces.len() == 1 && folder_count > 1 {
let v = min_free_spaces[0].clone();
min_free_spaces.resize(folder_count, v);
}
if folders.len() != min_free_spaces.len() {
panic!(
"{} directories by --dir, but only {} minFreeSpace values",
folders.len(),
min_free_spaces.len()
);
}
// Parse disk types
let mut disk_types: Vec<String> = cli.disk.split(',').map(|s| s.trim().to_string()).collect();
if disk_types.len() == 1 && folder_count > 1 {
let v = disk_types[0].clone();
disk_types.resize(folder_count, v);
}
if folders.len() != disk_types.len() {
panic!(
"{} directories by --dir, but only {} disk types by --disk",
folders.len(),
disk_types.len()
);
}
// Parse tags
let folder_tags = parse_volume_tags(&cli.tags, folder_count);
// Resolve IP
let ip = if cli.ip.is_empty() {
detect_host_address()
} else {
cli.ip
};
// Resolve bind IP
let bind_ip = if cli.bind_ip.is_empty() {
ip.clone()
} else {
cli.bind_ip
};
// Resolve public port
let public_port = if cli.port_public == 0 {
cli.port
} else {
cli.port_public
};
// Resolve gRPC port
let grpc_port = if cli.port_grpc == 0 {
10000 + cli.port
} else {
cli.port_grpc
};
// Resolve public URL
let public_url = if cli.public_url.is_empty() {
format!("{}:{}", ip, public_port)
} else {
cli.public_url
};
// Resolve volume server ID
let id = if cli.id.is_empty() {
format!("{}:{}", ip, cli.port)
} else {
cli.id
};
// Resolve metrics IP
let metrics_ip = if !cli.metrics_ip.is_empty() {
cli.metrics_ip
} else if !bind_ip.is_empty() {
bind_ip.clone()
} else {
ip.clone()
};
// Parse index type. Accept both `redb*` (preferred — what the volume server
// actually uses) and the legacy `leveldb*` names as aliases.
let index_type = match cli.index.as_str() {
"memory" => NeedleMapKind::InMemory,
"redb" | "leveldb" => NeedleMapKind::Redb,
"redbMedium" | "leveldbMedium" => NeedleMapKind::RedbMedium,
"redbLarge" | "leveldbLarge" => NeedleMapKind::RedbLarge,
other => panic!(
"Unknown index type: {}. Use memory|redb|redbMedium|redbLarge \
(leveldb/leveldbMedium/leveldbLarge accepted as aliases)",
other
),
};
// Parse read mode
let read_mode = match cli.read_mode.as_str() {
"local" => ReadMode::Local,
"proxy" => ReadMode::Proxy,
"redirect" => ReadMode::Redirect,
other => panic!("Unknown readMode: {}. Use local|proxy|redirect", other),
};
// Parse security config from TOML file
let sec = parse_security_config_with_env(&cli.security_file, env);
// Parse whitelist: merge CLI --whiteList with guard.white_list from security.toml
let mut white_list: Vec<String> = cli
.white_list
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
white_list.extend(sec.guard_white_list.iter().cloned());
VolumeServerConfig {
port: cli.port,
grpc_port,
public_port,
ip,
bind_ip,
public_url,
id,
masters,
pre_stop_seconds: cli.pre_stop_seconds,
idle_timeout: cli.idle_timeout,
data_center: cli.data_center,
rack: cli.rack,
index_type,
disk_type: cli.disk,
folders,
folder_max_limits,
folder_tags,
min_free_spaces,
disk_types,
idx_folder: cli.dir_idx,
white_list,
fix_jpg_orientation: cli.fix_jpg_orientation,
read_mode,
cpu_profile: cli.cpu_profile,
mem_profile: cli.mem_profile,
compaction_byte_per_second: cli.compaction_mb_per_second as i64 * 1024 * 1024,
maintenance_byte_per_second: cli.maintenance_mb_per_second as i64 * 1024 * 1024,
max_parallel_vacuum_commits: cli.max_parallel_vacuum_commits,
file_size_limit_bytes: cli.file_size_limit_mb as i64 * 1024 * 1024,
concurrent_upload_limit: cli.concurrent_upload_limit_mb as i64 * 1024 * 1024,
concurrent_download_limit: cli.concurrent_download_limit_mb as i64 * 1024 * 1024,
inflight_upload_data_timeout: cli.inflight_upload_data_timeout,
inflight_download_data_timeout: cli.inflight_download_data_timeout,
fsync_min_batch_bytes: cli.fsync_min_batch_bytes,
fsync_min_batch_requests: cli.fsync_min_batch_requests,
fsync_batch_wait: cli.fsync_batch_wait,
has_slow_read: cli.has_slow_read,
read_buffer_size_mb: cli.read_buffer_size_mb,
ldb_timeout: cli.ldb_timeout,
pprof: cli.pprof,
metrics_port: cli.metrics_port,
metrics_ip,
debug: cli.debug,
debug_port: cli.debug_port,
ui_enabled: sec.jwt_signing_key.is_empty() || sec.access_ui,
jwt_signing_key: sec.jwt_signing_key,
jwt_signing_expires_seconds: sec.jwt_signing_expires,
jwt_read_signing_key: sec.jwt_read_signing_key,
jwt_read_signing_expires_seconds: sec.jwt_read_signing_expires,
https_cert_file: sec.https_cert_file,
https_key_file: sec.https_key_file,
https_ca_file: sec.https_ca_file,
https_client_enabled: sec.https_client_enabled,
https_client_cert_file: sec.https_client_cert_file,
https_client_key_file: sec.https_client_key_file,
https_client_ca_file: sec.https_client_ca_file,
grpc_cert_file: sec.grpc_cert_file,
grpc_key_file: sec.grpc_key_file,
grpc_client_cert_file: sec.grpc_client_cert_file,
grpc_client_key_file: sec.grpc_client_key_file,
grpc_ca_file: sec.grpc_ca_file,
grpc_allowed_wildcard_domain: sec.grpc_allowed_wildcard_domain,
grpc_volume_allowed_common_names: sec.grpc_volume_allowed_common_names,
tls_policy: sec.tls_policy,
security_file: cli.security_file,
allow_untrusted_remote_endpoints: cli.allow_untrusted_remote_endpoints,
}
}
fn default_volume_dir() -> String {
std::env::temp_dir().to_string_lossy().into_owned()
}
/// Parsed security configuration from security.toml.
#[derive(Debug, Default)]
pub struct SecurityConfig {
pub jwt_signing_key: Vec<u8>,
pub jwt_signing_expires: i64,
pub jwt_read_signing_key: Vec<u8>,
pub jwt_read_signing_expires: i64,
pub https_cert_file: String,
pub https_key_file: String,
pub https_ca_file: String,
pub https_client_enabled: bool,
pub https_client_cert_file: String,
pub https_client_key_file: String,
pub https_client_ca_file: String,
pub grpc_cert_file: String,
pub grpc_key_file: String,
pub grpc_client_cert_file: String,
pub grpc_client_key_file: String,
pub grpc_ca_file: String,
pub grpc_allowed_wildcard_domain: String,
pub grpc_volume_allowed_common_names: Vec<String>,
pub tls_policy: TlsPolicy,
pub access_ui: bool,
/// IPs from [guard] white_list in security.toml
pub guard_white_list: Vec<String>,
}
const SECURITY_CONFIG_FILE_NAME: &str = "security.toml";
/// Parse a security.toml file to extract JWT signing keys and TLS configuration.
/// Format:
/// ```toml
/// [jwt.signing]
/// key = "secret"
/// expires_after_seconds = 60
///
/// [jwt.signing.read]
/// key = "read-secret"
/// expires_after_seconds = 60
///
/// [https.volume]
/// cert = "/path/to/cert.pem"
/// key = "/path/to/key.pem"
/// ca = "/path/to/ca.pem"
///
/// [tls]
/// min_version = "TLS 1.2"
/// max_version = "TLS 1.3"
/// cipher_suites = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
///
/// [https.client]
/// enabled = true
/// cert = "/path/to/cert.pem"
/// key = "/path/to/key.pem"
/// ca = "/path/to/ca.pem"
///
/// [grpc]
/// ca = "/path/to/ca.pem"
/// allowed_wildcard_domain = ".example.com"
///
/// [grpc.volume]
/// cert = "/path/to/cert.pem"
/// key = "/path/to/key.pem"
/// client_cert = "/path/to/client-cert.pem"
/// client_key = "/path/to/client-key.pem"
/// allowed_commonNames = "volume-a.internal,volume-b.internal"
/// ```
pub fn parse_security_config(path: &str) -> SecurityConfig {
parse_security_config_with_env(path, &process_env)
}
fn parse_security_config_with_env(path: &str, env: EnvLookup<'_>) -> SecurityConfig {
let Some(config_path) = resolve_security_config_path(path, env) else {
let mut cfg = SecurityConfig::default();
apply_env_overrides(&mut cfg, env);
return cfg;
};
let content = match std::fs::read_to_string(&config_path) {
Ok(c) => c,
Err(_) => {
let mut cfg = SecurityConfig::default();
apply_env_overrides(&mut cfg, env);
return cfg;
}
};
let mut cfg = SecurityConfig::default();
#[derive(PartialEq)]
enum Section {
None,
JwtSigning,
JwtSigningRead,
HttpsClient,
Grpc,
HttpsVolume,
GrpcVolume,
Tls,
Guard,
Access,
}
let mut section = Section::None;
for line in content.lines() {
let trimmed = line.trim();
if trimmed.starts_with('#') || trimmed.is_empty() {
continue;
}
if trimmed == "[jwt.signing.read]" {
section = Section::JwtSigningRead;
continue;
}
if trimmed == "[jwt.signing]" {
section = Section::JwtSigning;
continue;
}
if trimmed == "[https.client]" {
section = Section::HttpsClient;
continue;
}
if trimmed == "[grpc]" {
section = Section::Grpc;
continue;
}
if trimmed == "[https.volume]" {
section = Section::HttpsVolume;
continue;
}
if trimmed == "[grpc.volume]" {
section = Section::GrpcVolume;
continue;
}
if trimmed == "[tls]" {
section = Section::Tls;
continue;
}
if trimmed == "[guard]" {
section = Section::Guard;
continue;
}
if trimmed == "[access]" {
section = Section::Access;
continue;
}
if trimmed.starts_with('[') {
section = Section::None;
continue;
}
if let Some((key, value)) = trimmed.split_once('=') {
let key = key.trim();
let value = value.trim().trim_matches('"');
match section {
Section::JwtSigningRead => match key {
"key" => cfg.jwt_read_signing_key = value.as_bytes().to_vec(),
"expires_after_seconds" => {
cfg.jwt_read_signing_expires = value.parse().unwrap_or(60)
}
_ => {}
},
Section::JwtSigning => match key {
"key" => cfg.jwt_signing_key = value.as_bytes().to_vec(),
"expires_after_seconds" => {
cfg.jwt_signing_expires = value.parse().unwrap_or(10)
}
_ => {}
},
Section::HttpsClient => match key {
"enabled" => cfg.https_client_enabled = value.parse().unwrap_or(false),
"cert" => cfg.https_client_cert_file = value.to_string(),
"key" => cfg.https_client_key_file = value.to_string(),
"ca" => cfg.https_client_ca_file = value.to_string(),
_ => {}
},
Section::Grpc => match key {
"ca" => cfg.grpc_ca_file = value.to_string(),
"allowed_wildcard_domain" => {
cfg.grpc_allowed_wildcard_domain = value.to_string()
}
_ => {}
},
Section::HttpsVolume => match key {
"cert" => cfg.https_cert_file = value.to_string(),
"key" => cfg.https_key_file = value.to_string(),
"ca" => cfg.https_ca_file = value.to_string(),
_ => {}
},
Section::GrpcVolume => match key {
"cert" => cfg.grpc_cert_file = value.to_string(),
"key" => cfg.grpc_key_file = value.to_string(),
"client_cert" => cfg.grpc_client_cert_file = value.to_string(),
"client_key" => cfg.grpc_client_key_file = value.to_string(),
// Go only reads CA from [grpc], not [grpc.volume]
"allowed_commonNames" => {
cfg.grpc_volume_allowed_common_names =
value.split(',').map(|name| name.to_string()).collect();
}
_ => {}
},
Section::Tls => match key {
"min_version" => cfg.tls_policy.min_version = value.to_string(),
"max_version" => cfg.tls_policy.max_version = value.to_string(),
"cipher_suites" => cfg.tls_policy.cipher_suites = value.to_string(),
_ => {}
},
Section::Guard => {
if key == "white_list" {
cfg.guard_white_list = value
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
}
}
Section::Access => {
if key == "ui" {
cfg.access_ui = value.parse().unwrap_or(false)
}
}
Section::None => {}
}
}
}
// Match Go's v.SetDefault: when a signing key is present but
// expires_after_seconds was never specified, apply Go's defaults.
if !cfg.jwt_signing_key.is_empty() && cfg.jwt_signing_expires == 0 {
cfg.jwt_signing_expires = 10;
}
if !cfg.jwt_read_signing_key.is_empty() && cfg.jwt_read_signing_expires == 0 {
cfg.jwt_read_signing_expires = 60;
}
// Override with WEED_ environment variables (matches Go's Viper convention:
// prefix WEED_, uppercase, replace . with _).
// e.g. WEED_JWT_SIGNING_KEY overrides [jwt.signing] key
apply_env_overrides(&mut cfg, env);
cfg
}
fn resolve_security_config_path(path: &str, env: EnvLookup<'_>) -> Option<PathBuf> {
if !path.is_empty() {
return Some(PathBuf::from(path));
}
default_security_config_candidates(
std::env::current_dir().ok().as_deref(),
home_dir_from_env(env).as_deref(),
)
.into_iter()
.find(|candidate| candidate.is_file())
}
fn default_security_config_candidates(
current_dir: Option<&Path>,
home_dir: Option<&Path>,
) -> Vec<PathBuf> {
let mut candidates = Vec::new();
if let Some(dir) = current_dir {
candidates.push(dir.join(SECURITY_CONFIG_FILE_NAME));
}
if let Some(home) = home_dir {
candidates.push(home.join(".seaweedfs").join(SECURITY_CONFIG_FILE_NAME));
}
candidates.push(PathBuf::from("/usr/local/etc/seaweedfs").join(SECURITY_CONFIG_FILE_NAME));
candidates.push(PathBuf::from("/etc/seaweedfs").join(SECURITY_CONFIG_FILE_NAME));
candidates
}
fn home_dir_from_env(env: EnvLookup<'_>) -> Option<PathBuf> {
env("HOME")
.filter(|v| !v.is_empty())
.map(PathBuf::from)
.or_else(|| {
env("USERPROFILE")
.filter(|v| !v.is_empty())
.map(PathBuf::from)
})
}
/// Apply WEED_ environment variable overrides to a SecurityConfig.
/// Matches Go's Viper convention: WEED_ prefix, uppercase, dots replaced with underscores.
fn apply_env_overrides(cfg: &mut SecurityConfig, env: EnvLookup<'_>) {
if let Some(v) = env_string(env, "WEED_JWT_SIGNING_KEY") {
cfg.jwt_signing_key = v.into_bytes();
}
if let Some(v) = env_string(env, "WEED_JWT_SIGNING_EXPIRES_AFTER_SECONDS") {
cfg.jwt_signing_expires = v.parse().unwrap_or(cfg.jwt_signing_expires);
}
if let Some(v) = env_string(env, "WEED_JWT_SIGNING_READ_KEY") {
cfg.jwt_read_signing_key = v.into_bytes();
}
if let Some(v) = env_string(env, "WEED_JWT_SIGNING_READ_EXPIRES_AFTER_SECONDS") {
cfg.jwt_read_signing_expires = v.parse().unwrap_or(cfg.jwt_read_signing_expires);
}
if let Some(v) = env_string(env, "WEED_HTTPS_VOLUME_CERT") {
cfg.https_cert_file = v;
}
if let Some(v) = env_string(env, "WEED_HTTPS_VOLUME_KEY") {
cfg.https_key_file = v;
}
if let Some(v) = env_string(env, "WEED_HTTPS_VOLUME_CA") {
cfg.https_ca_file = v;
}
if let Some(v) = env_string(env, "WEED_HTTPS_CLIENT_ENABLED") {
cfg.https_client_enabled = v == "true" || v == "1";
}
if let Some(v) = env_string(env, "WEED_HTTPS_CLIENT_CERT") {
cfg.https_client_cert_file = v;
}
if let Some(v) = env_string(env, "WEED_HTTPS_CLIENT_KEY") {
cfg.https_client_key_file = v;
}
if let Some(v) = env_string(env, "WEED_HTTPS_CLIENT_CA") {
cfg.https_client_ca_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_CERT") {
cfg.grpc_cert_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_KEY") {
cfg.grpc_key_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_CLIENT_CERT") {
cfg.grpc_client_cert_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_CLIENT_KEY") {
cfg.grpc_client_key_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_CA") {
cfg.grpc_ca_file = v;
} else if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_CA") {
cfg.grpc_ca_file = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_ALLOWED_WILDCARD_DOMAIN") {
cfg.grpc_allowed_wildcard_domain = v;
}
if let Some(v) = env_string(env, "WEED_GRPC_VOLUME_ALLOWED_COMMONNAMES") {
cfg.grpc_volume_allowed_common_names = v.split(',').map(|name| name.to_string()).collect();
}
if let Some(v) = env_string(env, "WEED_TLS_MIN_VERSION") {
cfg.tls_policy.min_version = v;
}
if let Some(v) = env_string(env, "WEED_TLS_MAX_VERSION") {
cfg.tls_policy.max_version = v;
}
if let Some(v) = env_string(env, "WEED_TLS_CIPHER_SUITES") {
cfg.tls_policy.cipher_suites = v;
}
if let Some(v) = env_string(env, "WEED_GUARD_WHITE_LIST") {
cfg.guard_white_list = v
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
}
if let Some(v) = env_string(env, "WEED_ACCESS_UI") {
cfg.access_ui = v == "true" || v == "1";
}
}
/// Detect the host's IP address.
/// Mirrors Go's `util.DetectedHostAddress()`.
fn detect_host_address() -> String {
// Connect to a remote address to determine the local outbound IP
if let Ok(socket) = UdpSocket::bind("0.0.0.0:0")
&& socket.connect("8.8.8.8:80").is_ok()
&& let Ok(addr) = socket.local_addr()
{
return addr.ip().to_string();
}
"localhost".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use std::sync::{Mutex, MutexGuard, OnceLock};
/// Serialises the tests that move the process working directory against
/// the ones that read it. `set_current_dir` is safe, but it is still
/// process-global: without this, a `parse_security_config("")` in one test
/// would search another test's temporary directory.
///
/// Nothing here guards the environment any more — tests inject an
/// [`EnvLookup`] instead of mutating the real one.
fn process_state_lock() -> MutexGuard<'static, ()> {
static LOCK: OnceLock<Mutex<()>> = OnceLock::new();
LOCK.get_or_init(|| Mutex::new(())).lock().unwrap()
}
fn with_temp_current_dir<F: FnOnce()>(dir: &Path, f: F) {
let previous = std::env::current_dir().unwrap();
std::env::set_current_dir(dir).unwrap();
f();
std::env::set_current_dir(previous).unwrap();
}
/// The whole environment a test wants the config layer to see, ready to
/// pass as `&env`. `&|_| None` is an environment with nothing set at all.
fn fake_env(pairs: &[(&str, &str)]) -> impl Fn(&str) -> Option<OsString> + use<> {
let map: HashMap<String, OsString> = pairs
.iter()
.map(|(key, value)| ((*key).to_string(), OsString::from(*value)))
.collect();
move |key| map.get(key).cloned()
}
#[test]
fn test_parse_go_duration_accepts_go_grammar() {
use std::time::Duration;
let cases = [
("60s", Duration::from_secs(60)),
("5m", Duration::from_secs(300)),
("1h", Duration::from_secs(3600)),
("1.5h", Duration::from_secs(5400)),
("2h45m", Duration::from_secs(2 * 3600 + 45 * 60)),
("300ms", Duration::from_millis(300)),
("1.5s", Duration::from_millis(1500)),
("250us", Duration::from_micros(250)),
("250\u{00b5}s", Duration::from_micros(250)),
("250\u{03bc}s", Duration::from_micros(250)),
("7ns", Duration::from_nanos(7)),
(".5s", Duration::from_millis(500)),
("5.s", Duration::from_secs(5)),
("+10s", Duration::from_secs(10)),
("0", Duration::ZERO),
("0s", Duration::ZERO),
// Go: fraction digits past the 19th are ignored, not an error.
("1.00000000000000000001s", Duration::from_secs(1)),
];
for (input, expected) in cases {
assert_eq!(parse_go_duration(input), Ok(expected), "input={:?}", input);
}
}
#[test]
fn test_parse_go_duration_rejects_what_go_rejects() {
// Each of these makes Go's flag.Duration fail flag parsing; the old
// Rust parser turned them all into the 60 s default.
let rejected = [
"30", // bare number other than 0
"30sec", // unknown unit
"abc", // no number at all
"", // empty
"1h ", // trailing space is not a unit
"s", // unit without number
".s", // dot without digits
"1.5", // fraction without unit
"1h-30m", // sign is only allowed at the front
"-", // sign alone
"5124095576030432h", // overflows int64 nanoseconds
"307445734561825861m", // overflows int64 nanoseconds
"9223372036854775808ns", // 1<<63, one past int64
];
for input in rejected {
let result = parse_go_duration(input);
assert!(result.is_err(), "input={:?} parsed as {:?}", input, result);
let message = result.unwrap_err();
assert!(
message.contains(&format!("{:?}", input)) || input.is_empty(),
"error for {:?} does not name the value: {}",
input,
message
);
}
// Largest value Go accepts: 1<<63 - 1 nanoseconds.
assert_eq!(
parse_go_duration("9223372036854775807ns"),
Ok(std::time::Duration::from_nanos(i64::MAX as u64))
);
// Go accepts negatives; std::time::Duration cannot, so it errors.
let negative = parse_go_duration("-1s").unwrap_err();
assert!(negative.contains("negative"), "{}", negative);
}
/// Clap refuses the flag so the process exits with a usage error, exactly
/// where Go's flag package does.
#[test]
fn test_cli_rejects_invalid_duration_flags() {
for value in ["30sec", "abc", "30"] {
let result = Cli::try_parse_from(["bin", "--inflightUploadDataTimeout", value]);
assert!(
result.is_err(),
"--inflightUploadDataTimeout={} was accepted as {:?}",
value,
result.map(|cli| cli.inflight_upload_data_timeout)
);
let err = Cli::try_parse_from(["bin", "--inflightDownloadDataTimeout", value])
.err()
.unwrap_or_else(|| panic!("--inflightDownloadDataTimeout={} was accepted", value));
assert_eq!(err.kind(), clap::error::ErrorKind::ValueValidation);
assert!(err.to_string().contains(value), "{}", err);
}
let cli = Cli::parse_from(["bin", "--inflightUploadDataTimeout", "1.5h"]);
assert_eq!(
cli.inflight_upload_data_timeout,
std::time::Duration::from_secs(5400)
);
// The default still parses through the same grammar.
let cli = Cli::parse_from(["bin"]);
assert_eq!(
cli.inflight_upload_data_timeout,
std::time::Duration::from_secs(60)
);
assert_eq!(
cli.inflight_download_data_timeout,
std::time::Duration::from_secs(60)
);
}
#[test]
fn test_parse_bytes_matches_go_parse_bytes() {
// Expected values are what Go's util.ParseBytes returns (weed/util/bytes_test.go).
let cases: [(&str, u64); 14] = [
("42", 42),
("42MB", 42_000_000),
("42 MB", 42_000_000),
("42 mib", 44_040_192),
("42M", 42_000_000),
("100Ki", 100 * 1024),
("10GiB", 10 * 1024 * 1024 * 1024),
("10gib", 10 * 1024 * 1024 * 1024),
("10GB", 10_000_000_000),
("1.5TB", 1_500_000_000_000),
// Go's ParseBytes accepts thousands commas; the -minFreeSpace flag
// never delivers one (see test_min_free_space_flag_commas_are_list_separators).
("1,024MB", 1_024_000_000),
("1,024", 1024),
("1PiB", 1 << 50),
("1eb", 1_000_000_000_000_000_000),
];
for (input, expected) in cases {
assert_eq!(parse_bytes(input), Ok(expected), "input={:?}", input);
}
for input in [
"10GiBx",
"GiB",
"",
"-5GB",
"1.2.3GB",
"10 Gi B",
"99999999999999999999EB",
] {
assert!(
parse_bytes(input).is_err(),
"input={:?} was accepted",
input
);
}
}
fn expect_percent(result: Result<MinFreeSpace, String>, expected: f64, input: &str) {
match result {
Ok(MinFreeSpace::Percent(v)) => {
assert!(
(v - expected).abs() < f64::EPSILON,
"input={:?}: {}",
input,
v
)
}
other => panic!(
"input={:?}: expected Percent({}), got {:?}",
input, expected, other
),
}
}
fn expect_bytes(result: Result<MinFreeSpace, String>, expected: u64, input: &str) {
match result {
Ok(MinFreeSpace::Bytes(v)) => assert_eq!(v, expected, "input={:?}", input),
other => panic!(
"input={:?}: expected Bytes({}), got {:?}",
input, expected, other
),
}
}
#[test]
fn test_parse_min_free_space_matches_go() {
// Fixtures from weed/util/minfreespace_test.go plus the byte-unit table.
expect_percent(parse_min_free_space("42"), 42.0, "42");
expect_percent(parse_min_free_space(" 42 "), 42.0, " 42 ");
expect_percent(parse_min_free_space("50"), 50.0, "50");
expect_percent(parse_min_free_space("100"), 100.0, "100");
expect_percent(parse_min_free_space("0"), 0.0, "0");
expect_percent(parse_min_free_space("2.5"), 2.5, "2.5");
expect_bytes(parse_min_free_space("100Ki"), 100 * 1024, "100Ki");
expect_bytes(parse_min_free_space("100GiB"), 100 << 30, "100GiB");
expect_bytes(parse_min_free_space("42M"), 42_000_000, "42M");
expect_bytes(parse_min_free_space(" 42M "), 42_000_000, " 42M ");
expect_bytes(parse_min_free_space("10GiB"), 10 << 30, "10GiB");
expect_bytes(parse_min_free_space("10gib"), 10 << 30, "10gib");
expect_bytes(parse_min_free_space("1.5TB"), 1_500_000_000_000, "1.5TB");
// Comma inputs never arrive whole from the flag; see
// test_min_free_space_flag_commas_are_list_separators.
expect_bytes(parse_min_free_space("1,024MB"), 1_024_000_000, "1,024MB");
expect_bytes(parse_min_free_space("1,000"), 1000, "1,000");
// Go: percent outside 0..=100, or a byte size of at most 100 bytes.
for input in [
"150", "101", "-1", "1e3", "inf", "nan", "100B", "50B", "0B", "10GiBx", "", "abc",
"10 Gi B",
] {
let result = parse_min_free_space(input);
assert!(
result.is_err(),
"input={:?} was accepted as {:?}",
input,
result
);
assert!(
result
.as_ref()
.unwrap_err()
.contains(&format!("{:?}", input.trim())),
"error for {:?} does not name the value: {:?}",
input,
result
);
}
}
#[test]
fn test_parse_min_free_spaces_percent() {
let result = parse_min_free_spaces("", "1").unwrap();
assert_eq!(result.len(), 1);
match &result[0] {
MinFreeSpace::Percent(v) => assert!((v - 1.0).abs() < f64::EPSILON),
_ => panic!("Expected Percent"),
}
}
#[test]
fn test_parse_min_free_spaces_bytes() {
let result = parse_min_free_spaces("10GiB", "").unwrap();
assert_eq!(result.len(), 1);
match &result[0] {
MinFreeSpace::Bytes(v) => assert_eq!(*v, 10 * 1024 * 1024 * 1024),
_ => panic!("Expected Bytes"),
}
}
#[test]
fn test_parse_min_free_spaces_list_and_precedence() {
// --minFreeSpace wins over --minFreeSpacePercent, as in Go's EmptyTo().
let result = parse_min_free_spaces("5, 10GiB", "1").unwrap();
assert_eq!(result.len(), 2);
assert!(matches!(result[0], MinFreeSpace::Percent(v) if v == 5.0));
assert!(matches!(result[1], MinFreeSpace::Bytes(v) if v == 10 << 30));
// One bad entry fails the whole list and names that entry.
let err = parse_min_free_spaces("5,10GiBx", "1").unwrap_err();
assert!(err.contains("\"10GiBx\""), "{}", err);
// Both flags empty: Go fatals (ParseMinFreeSpace("") fails) rather than
// assuming 1%.
assert!(parse_min_free_spaces("", "").is_err());
}
/// A comma in `-minFreeSpace` separates per-directory entries, as in Go's
/// `MustParseMinFreeSpace`: `1,024MB` is `1` (percent) and `024MB`, and
/// one `-dir` fails the count check.
#[test]
fn test_min_free_space_flag_commas_are_list_separators() {
let result = parse_min_free_spaces("1,024MB", "").unwrap();
assert_eq!(result.len(), 2, "{result:?}");
assert!(
matches!(result[0], MinFreeSpace::Percent(v) if v == 1.0),
"{result:?}"
);
assert!(
matches!(result[1], MinFreeSpace::Bytes(v) if v == 24_000_000),
"{result:?}"
);
let _guard = process_state_lock();
let cli = Cli::parse_from(["bin", "--dir", "/tmp/a", "--minFreeSpace", "1,024MB"]);
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
resolve_config_with_env(cli, &|_| None).min_free_spaces
}));
assert!(
outcome.is_err(),
"one -dir with two minFreeSpace entries must abort"
);
}
/// Negative control at the startup layer: resolving the config with an
/// invalid --minFreeSpace must abort, as Go's MustParseMinFreeSpace does.
#[test]
fn test_resolve_config_rejects_invalid_min_free_space() {
// The security-config search reads the working directory.
let _guard = process_state_lock();
for (flag, value) in [
("--minFreeSpace", "10GiBx"),
("--minFreeSpace", "150"),
("--minFreeSpace", "50B"),
("--minFreeSpacePercent", "150"),
("--minFreeSpacePercent", "abc"),
] {
let cli = Cli::parse_from(["bin", flag, value]);
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
resolve_config_with_env(cli, &|_| None).min_free_spaces
}));
let payload = match outcome {
Ok(spaces) => panic!("{}={} was accepted as {:?}", flag, value, spaces),
Err(payload) => payload,
};
let message = payload
.downcast_ref::<String>()
.cloned()
.or_else(|| payload.downcast_ref::<&str>().map(|s| s.to_string()))
.unwrap_or_default();
assert!(
message.contains("-minFreeSpace") && message.contains(value),
"{}={}: unexpected panic message {:?}",
flag,
value,
message
);
}
let cfg =
resolve_config_with_env(Cli::parse_from(["bin", "--minFreeSpace", "100"]), &|_| None);
assert!(matches!(cfg.min_free_spaces[0], MinFreeSpace::Percent(v) if v == 100.0));
}
#[test]
fn test_parse_volume_tags_single() {
let tags = parse_volume_tags("fast:ssd", 3);
assert_eq!(tags.len(), 3);
assert_eq!(tags[0], vec!["fast", "ssd"]);
assert_eq!(tags[1], vec!["fast", "ssd"]);
assert_eq!(tags[2], vec!["fast", "ssd"]);
}
#[test]
fn test_parse_volume_tags_multi() {
let tags = parse_volume_tags("fast:ssd,archive", 3);
assert_eq!(tags.len(), 3);
assert_eq!(tags[0], vec!["fast", "ssd"]);
assert_eq!(tags[1], vec!["archive"]);
assert_eq!(tags[2], Vec::<String>::new());
}
#[test]
fn test_parse_volume_tags_empty() {
let tags = parse_volume_tags("", 2);
assert_eq!(tags.len(), 2);
assert_eq!(tags[0], Vec::<String>::new());
assert_eq!(tags[1], Vec::<String>::new());
}
#[test]
fn test_normalize_args_single_dash_to_double() {
let args = vec![
"bin".into(),
"-port".into(),
"8080".into(),
"-ip.bind".into(),
"127.0.0.1".into(),
"-dir".into(),
"/data".into(),
];
let norm = normalize_args_vec(args);
assert_eq!(
norm,
vec![
"bin",
"--port",
"8080",
"--ip.bind",
"127.0.0.1",
"--dir",
"/data",
]
);
}
#[test]
fn test_normalize_args_double_dash_unchanged() {
let args = vec![
"bin".into(),
"--port".into(),
"8080".into(),
"--master".into(),
"localhost:9333".into(),
];
let norm = normalize_args_vec(args);
assert_eq!(
norm,
vec!["bin", "--port", "8080", "--master", "localhost:9333",]
);
}
#[test]
fn test_normalize_args_single_char_flags_unchanged() {
let args = vec!["bin".into(), "-h".into(), "-V".into()];
let norm = normalize_args_vec(args);
assert_eq!(norm, vec!["bin", "-h", "-V"]);
}
#[test]
fn test_normalize_args_equals_format() {
let args = vec!["bin".into(), "-port=8080".into(), "-ip.bind=0.0.0.0".into()];
let norm = normalize_args_vec(args);
assert_eq!(norm, vec!["bin", "--port=8080", "--ip.bind=0.0.0.0"]);
}
#[test]
fn test_normalize_args_stop_at_double_dash() {
let args = vec![
"bin".into(),
"-port".into(),
"8080".into(),
"--".into(),
"-notaflag".into(),
];
let norm = normalize_args_vec(args);
assert_eq!(norm, vec!["bin", "--port", "8080", "--", "-notaflag"]);
}
#[test]
fn test_resolve_config_defaults_dir_to_platform_temp_dir() {
// With no --securityFile, the security-config search reads the process
// working directory, so this has to hold the lock against
// `with_temp_current_dir`. The environment is injected, not shared.
let _guard = process_state_lock();
let cfg = resolve_config_with_env(Cli::parse_from(["bin"]), &|_| None);
assert_eq!(cfg.folders, vec![default_volume_dir()]);
}
#[test]
fn test_resolve_config_index_accepts_redb_and_leveldb_aliases() {
// As above: the security-config search reads the working directory.
let _guard = process_state_lock();
let pairs = [
("memory", NeedleMapKind::InMemory),
("redb", NeedleMapKind::Redb),
("leveldb", NeedleMapKind::Redb),
("redbMedium", NeedleMapKind::RedbMedium),
("leveldbMedium", NeedleMapKind::RedbMedium),
("redbLarge", NeedleMapKind::RedbLarge),
("leveldbLarge", NeedleMapKind::RedbLarge),
];
for (input, expected) in pairs {
let cfg =
resolve_config_with_env(Cli::parse_from(["bin", "--index", input]), &|_| None);
assert_eq!(cfg.index_type, expected, "input={}", input);
}
}
#[test]
fn test_parse_security_config_access_ui() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[jwt.signing]
key = "secret"
[access]
ui = true
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(cfg.jwt_signing_key, b"secret");
assert!(cfg.access_ui);
}
#[test]
fn test_parse_security_config_discovers_current_directory_default() {
let _guard = process_state_lock();
let tmp = tempfile::TempDir::new().unwrap();
std::fs::write(
tmp.path().join(SECURITY_CONFIG_FILE_NAME),
r#"
[jwt.signing]
key = "cwd-secret"
"#,
)
.unwrap();
with_temp_current_dir(tmp.path(), || {
let cfg = parse_security_config_with_env("", &|_| None);
assert_eq!(cfg.jwt_signing_key, b"cwd-secret");
});
}
#[test]
fn test_parse_security_config_discovers_home_default() {
let _guard = process_state_lock();
let current_dir = tempfile::TempDir::new().unwrap();
let home_dir = tempfile::TempDir::new().unwrap();
let seaweed_home = home_dir.path().join(".seaweedfs");
std::fs::create_dir_all(&seaweed_home).unwrap();
std::fs::write(
seaweed_home.join(SECURITY_CONFIG_FILE_NAME),
r#"
[jwt.signing]
key = "home-secret"
"#,
)
.unwrap();
let env = fake_env(&[("HOME", home_dir.path().to_str().unwrap())]);
with_temp_current_dir(current_dir.path(), || {
let cfg = parse_security_config_with_env("", &env);
assert_eq!(cfg.jwt_signing_key, b"home-secret");
});
}
#[test]
fn test_parse_security_config_uses_grpc_root_ca() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[grpc]
ca = "/etc/seaweedfs/grpc-ca.pem"
[grpc.volume]
cert = "/etc/seaweedfs/volume-cert.pem"
key = "/etc/seaweedfs/volume-key.pem"
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(cfg.grpc_ca_file, "/etc/seaweedfs/grpc-ca.pem");
assert_eq!(cfg.grpc_cert_file, "/etc/seaweedfs/volume-cert.pem");
assert_eq!(cfg.grpc_key_file, "/etc/seaweedfs/volume-key.pem");
}
#[test]
fn test_parse_security_config_uses_grpc_volume_client_cert() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[grpc.volume]
cert = "/etc/seaweedfs/volume-cert.pem"
key = "/etc/seaweedfs/volume-key.pem"
client_cert = "/etc/seaweedfs/volume-client-cert.pem"
client_key = "/etc/seaweedfs/volume-client-key.pem"
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(
cfg.grpc_client_cert_file,
"/etc/seaweedfs/volume-client-cert.pem"
);
assert_eq!(
cfg.grpc_client_key_file,
"/etc/seaweedfs/volume-client-key.pem"
);
}
#[test]
fn test_parse_security_config_uses_grpc_peer_name_policy() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[grpc]
allowed_wildcard_domain = ".example.com"
[grpc.volume]
allowed_commonNames = "volume-a.internal,volume-b.internal"
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(cfg.grpc_allowed_wildcard_domain, ".example.com");
assert_eq!(
cfg.grpc_volume_allowed_common_names,
vec![
String::from("volume-a.internal"),
String::from("volume-b.internal")
]
);
}
#[test]
fn test_parse_security_config_uses_https_client_settings() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[https.client]
enabled = true
cert = "/etc/seaweedfs/client-cert.pem"
key = "/etc/seaweedfs/client-key.pem"
ca = "/etc/seaweedfs/client-ca.pem"
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert!(cfg.https_client_enabled);
assert_eq!(cfg.https_client_cert_file, "/etc/seaweedfs/client-cert.pem");
assert_eq!(cfg.https_client_key_file, "/etc/seaweedfs/client-key.pem");
assert_eq!(cfg.https_client_ca_file, "/etc/seaweedfs/client-ca.pem");
}
#[test]
fn test_parse_security_config_uses_tls_policy_settings() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[tls]
min_version = "TLS 1.2"
max_version = "TLS 1.3"
cipher_suites = "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
"#,
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(cfg.tls_policy.min_version, "TLS 1.2");
assert_eq!(cfg.tls_policy.max_version, "TLS 1.3");
assert_eq!(
cfg.tls_policy.cipher_suites,
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
);
}
#[test]
fn test_merge_options_file_basic() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(tmp.path(), "port=9999\ndir=/data\nmaster=localhost:9333\n").unwrap();
let args = vec![
"bin".into(),
"--options".into(),
tmp.path().to_str().unwrap().into(),
];
let merged = merge_options_file(args);
// Should contain the original args plus the file-based ones
assert!(merged.contains(&"--port".to_string()));
assert!(merged.contains(&"9999".to_string()));
assert!(merged.contains(&"--dir".to_string()));
assert!(merged.contains(&"/data".to_string()));
}
#[test]
fn test_merge_options_file_cli_precedence() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(tmp.path(), "port=9999\ndir=/data\n").unwrap();
let args = vec![
"bin".into(),
"--port".into(),
"8080".into(),
"--options".into(),
tmp.path().to_str().unwrap().into(),
];
let merged = merge_options_file(args);
// port should NOT be duplicated from file since CLI already set it
let port_count = merged.iter().filter(|a| *a == "--port").count();
assert_eq!(
port_count, 1,
"CLI port should take precedence, file port skipped"
);
// dir should be added from file
assert!(merged.contains(&"--dir".to_string()));
}
#[test]
fn test_merge_options_file_comments_and_blanks() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
"# this is a comment\n\nport=9999\n# another comment\ndir=/data\n",
)
.unwrap();
let args = vec![
"bin".into(),
"--options".into(),
tmp.path().to_str().unwrap().into(),
];
let merged = merge_options_file(args);
assert!(merged.contains(&"--port".to_string()));
assert!(merged.contains(&"--dir".to_string()));
}
#[test]
fn test_merge_options_file_with_dashes_in_key() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(tmp.path(), "-port=9999\n--dir=/data\nip.bind=0.0.0.0\n").unwrap();
let args = vec![
"bin".into(),
"--options".into(),
tmp.path().to_str().unwrap().into(),
];
let merged = merge_options_file(args);
assert!(merged.contains(&"--port".to_string()));
assert!(merged.contains(&"--dir".to_string()));
assert!(merged.contains(&"--ip.bind".to_string()));
}
#[test]
fn test_find_options_arg() {
assert_eq!(
find_options_arg(&["bin".into(), "--options".into(), "/tmp/opts".into()]),
"/tmp/opts"
);
assert_eq!(
find_options_arg(&["bin".into(), "-options".into(), "/tmp/opts".into()]),
"/tmp/opts"
);
assert_eq!(
find_options_arg(&["bin".into(), "--options=/tmp/opts".into()]),
"/tmp/opts"
);
assert_eq!(
find_options_arg(&["bin".into(), "--port".into(), "8080".into()]),
""
);
}
#[test]
fn test_env_override_jwt_signing_key() {
// No file path, so the search reads the working directory.
let _guard = process_state_lock();
let env = fake_env(&[("WEED_JWT_SIGNING_KEY", "env-secret")]);
let cfg = parse_security_config_with_env("", &env);
assert_eq!(cfg.jwt_signing_key, b"env-secret");
}
#[test]
fn test_env_override_takes_precedence_over_file() {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
r#"
[jwt.signing]
key = "file-secret"
"#,
)
.unwrap();
let env = fake_env(&[("WEED_JWT_SIGNING_KEY", "env-secret")]);
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &env);
assert_eq!(cfg.jwt_signing_key, b"env-secret");
}
#[test]
fn test_env_override_guard_white_list() {
// No file path, so the search reads the working directory.
let _guard = process_state_lock();
let env = fake_env(&[("WEED_GUARD_WHITE_LIST", "10.0.0.0/8, 192.168.1.0/24")]);
let cfg = parse_security_config_with_env("", &env);
assert_eq!(cfg.guard_white_list, vec!["10.0.0.0/8", "192.168.1.0/24"]);
}
#[test]
fn test_env_override_access_ui() {
// No file path, so the search reads the working directory.
let _guard = process_state_lock();
let env = fake_env(&[("WEED_ACCESS_UI", "true")]);
let cfg = parse_security_config_with_env("", &env);
assert!(cfg.access_ui);
}
}