* ec: mount shards against the index .vif when the local one has no EC config A worker encode can send shards to a disk that still holds the source volume's .vif while the encode's .vif lands with the index on a sibling disk. The shard disk mounted through the source's config-free .vif, so it reported encode time 0 and fell back to the sidecar layout, and reads carrying the encode time were rejected as a different encode run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): mount shards against the index .vif when the local one has no EC config Mirrors the Go mount: a data-dir .vif that records no EC config yields to the idx dir's when that one records it, and load_vif_info_across_dirs reports the directory the .vif actually came from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: do not mirror EC sidecars onto a disk whose .vif has no EC config Startup mirroring copied .ecx/.ecj next to a source volume's leftover config-free .vif and kept that .vif, so after a restart the shard disk mounted self-contained with encode time 0. Leave such a disk on the cross-disk mount, which reads the owner's .vif. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): do not mirror EC sidecars onto a disk whose .vif has no EC config Mirrors the Go startup mirror: a shard disk holding a source volume's config-free .vif stays on the cross-disk mount, which reads the owner's .vif, instead of mounting self-contained with encode time 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ec: take the encode .vif from a sibling data dir when the index dir is shared With a separate -dir.idx the .ecx is shared by every disk, but the encode's .vif lands in the data dir of the disk that received it. A disk still holding a source volume's config-free .vif mounted through it at generation 0. NewEcVolume now accepts the server's data dirs and takes a sibling's EC .vif when the index is not local and every sibling EC .vif names the same run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: give EC mounts the data dir of every disk on the server NewStore records every data dir on each DiskLocation before loading, so the startup scan, cross-disk reconcile and MountEcShards all pass them to NewEcVolume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): take the encode .vif from a sibling data dir when the index dir is shared Mirrors the Go mount: EcVolume::new_with_vif_dirs takes a sibling's EC .vif when this disk's own records no EC config, the index is not local, and every sibling EC .vif names the same run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): give EC mounts the data dir of every disk on the server main sets Store.store_dirs before the first add_location, so a disk loaded before its siblings still sees them; each DiskLocation passes them to EcVolume::new_with_vif_dirs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: size startup EC validation by the sibling encode .vif With a shared -dir.idx, a disk that still holds the source .dat and its config-free .vif read the default data-shard count, judged custom-ratio shards an interrupted encode, and removed them along with the shared .ecx before the mount could take the sibling's .vif. The startup lookup now falls back to the same sibling .vif the mount uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): size startup EC validation by the sibling encode .vif Mirrors the Go change: validate_ec_volume falls back to the sibling data dir's encode .vif before judging shards beside a source .dat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: keep EC shards when sibling .vif files disagree on the encode SiblingEcConfigVif reports when a sibling .vif cannot be read or the siblings name different encode runs. The startup data-shard lookup then returns 0 instead of the default ratio, and validateEcVolume keeps the files rather than sizing custom-ratio shards by a ratio they do not use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume(rs): keep EC shards when sibling .vif files disagree on the encode Mirrors the Go change: sibling_ec_config_vif returns Err for an unreadable sibling .vif or disagreeing runs, ec_data_shards_from_vif returns 0 then, and validate_ec_volume keeps the files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * storage: release the opened EC shard when its volume cannot be built Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS Volume Server (Rust)
A drop-in replacement for the SeaweedFS Go volume server, rewritten in Rust. It uses binary-compatible storage formats (.dat, .idx, .vif) and speaks the same HTTP and gRPC protocols, so it works with an unmodified Go master server.
Building
Requires Rust 1.91.1+ (2024 edition), matching rust-version in Cargo.toml.
The patch release matters: 1.91.0 does not build. The edition itself only needs
1.85; the higher floor comes from the dependency tree — chiefly the AWS SDK — so
it moves with those crates. CI builds on the latest stable.
cd seaweed-volume
cargo build --release
The binary is produced at target/release/seaweed-volume.
Running
Start a Go master server first, then point the Rust volume server at it:
# Minimal
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7
# Multiple data directories
seaweed-volume --port 8080 --master localhost:9333 \
--dir /mnt/ssd1,/mnt/ssd2 --max 100,100 --disk ssd
# With datacenter/rack topology
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--dataCenter dc1 --rack rack1
# With JWT authentication
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--securityFile /etc/seaweedfs/security.toml
# With TLS (configured in security.toml via [https.volume] and [grpc.volume] sections)
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--securityFile /etc/seaweedfs/security.toml
Common flags
| Flag | Default | Description |
|---|---|---|
--port |
8080 |
HTTP listen port |
--port.grpc |
port+10000 |
gRPC listen port |
--master |
localhost:9333 |
Comma-separated master server addresses |
--dir |
/tmp |
Comma-separated data directories |
--max |
8 |
Max volumes per directory (comma-separated) |
--ip |
auto-detect | Server IP / identifier |
--ip.bind |
same as --ip |
Bind address |
--dataCenter |
Datacenter name | |
--rack |
Rack name | |
--disk |
Disk type tag: hdd, ssd, or custom |
|
--index |
memory |
Needle map type: memory, leveldb, leveldbMedium, leveldbLarge |
--readMode |
proxy |
Non-local read mode: local, proxy, redirect |
--fileSizeLimitMB |
256 |
Max upload file size |
--minFreeSpace |
1 (percent) |
Min free disk space before marking volumes read-only |
--securityFile |
Path to security.toml for JWT keys and TLS certs |
|
--metricsPort |
0 (disabled) |
Prometheus metrics endpoint port |
--whiteList |
Comma-separated IPs with write permission | |
--preStopSeconds |
10 |
Graceful drain period before shutdown |
--compactionMBps |
0 (unlimited) |
Compaction I/O rate limit |
--compactionMaxParallelCommits |
0 (unlimited) |
Max concurrent vacuum commits |
--pprof |
false |
Enable pprof HTTP handlers |
Set RUST_LOG=debug (or trace, info, warn) for log level control.
Durable (fsync=true) uploads always go through per-volume batching workers
that share one .dat/.idx sync per batch, matching the Go volume server;
non-fsync writes take the synchronous path.
Features
- Binary compatible -- reads and writes the same
.dat/.idx/.viffiles as the Go server; seamless migration with no data conversion. - HTTP + gRPC -- full implementation of the volume server HTTP API and all gRPC RPCs including streaming operations (copy, tail, incremental copy, vacuum).
- Master heartbeat -- bidirectional streaming heartbeat with the Go master server; volume and EC shard registration, leader failover, graceful shutdown deregistration.
- JWT authentication -- signing key configuration via
security.tomlwith token source precedence (query > header > cookie), file_id claims validation, and separate read/write keys. - TLS -- HTTPS for the HTTP API and mTLS for gRPC, configured through
security.toml. - Erasure coding -- Reed-Solomon EC shard management: mount/unmount, read, rebuild, copy, delete, and shard-to-volume reconstruction.
- S3 remote storage --
FetchAndWriteNeedlereads from any S3-compatible backend (AWS, MinIO, Wasabi, Backblaze, etc.) and writes locally. SupportsVolumeTierMoveDatToRemote/FromRemotefor tiered storage. - Needle map backends -- in-memory HashMap, LevelDB (via
rusty-leveldb), or redb (pure Rust disk-backed) needle maps. - Image processing -- on-the-fly resize/crop, JPEG EXIF orientation auto-fix, WebP support.
- Streaming reads -- large files (>1MB) are streamed via
spawn_blockingto avoid blocking the async runtime. - Auto-compression -- compressible file types (text, JSON, CSS, JS, SVG, etc.) are gzip-compressed on upload.
- Prometheus metrics -- counters, histograms, and gauges exported at a dedicated metrics port; optional push gateway support.
- Graceful shutdown -- SIGINT/SIGTERM handling with configurable
preStopSecondsdrain period.
Testing
Rust unit tests
cd seaweed-volume
cargo test
Go integration tests
The Go test suite can target either the Go or Rust volume server via the VOLUME_SERVER_IMPL environment variable:
# Run all HTTP + gRPC integration tests against the Rust server
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 1200s \
./test/volume_server/grpc/... ./test/volume_server/http/...
# Run a single test
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 60s \
-run "TestName" ./test/volume_server/http/...
# Run S3 remote storage tests
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 180s \
-run "TestFetchAndWriteNeedle" ./test/volume_server/grpc/...
Load testing
A load test harness is available at test/volume_server/loadtest/. See that directory for usage instructions and scenarios.
Architecture
The server runs three listeners concurrently:
- HTTP (Axum 0.7) -- admin and public routers for file upload/download, status, and stats endpoints.
- gRPC (Tonic 0.12) -- all
VolumeServerRPCs from the SeaweedFS protobuf definition. - Metrics (optional) -- Prometheus scrape endpoint on a separate port.
Key source modules:
| Path | Description |
|---|---|
src/main.rs |
Entry point, server startup, signal handling |
src/config.rs |
CLI parsing and configuration resolution |
src/server/volume_server.rs |
HTTP router setup and middleware |
src/server/handlers.rs |
HTTP request handlers (read, write, delete, status) |
src/server/grpc_server.rs |
gRPC service implementation |
src/server/heartbeat.rs |
Master heartbeat loop |
src/storage/volume.rs |
Volume read/write/delete logic |
src/storage/needle.rs |
Needle (file entry) serialization |
src/storage/store.rs |
Multi-volume store management |
src/security.rs |
JWT validation and IP whitelist guard |
src/remote_storage/ |
S3 remote storage backend |
See DEV_PLAN.md for the full development history and feature checklist.