Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-06 22:41:56 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
afce0a3dd3bcb5f41b52034ced64af0ff0eb43ee
seaweedfs/seaweed-volume/README.md
T
Eliah RusinClaude Opus 5Chris Lu
 516e251f9e rust volume: move the crate to edition 2024 (#11244) 
* rust volume: move the crate to edition 2024

Edition 2024 turns three things in this crate into hard errors, and changes
drop order in a further 34 places without changing compilation. The compiler
errors are fixed here; the silent changes were audited against
`RUSTFLAGS='-W rust-2024-compatibility' cargo check --all-targets` output
captured before the flip, since edition 2024 stops reporting them.

`std::env::set_var`/`remove_var` are unsafe as of 2024 because they race with
concurrent readers. All six call sites are safe by construction rather than by
assertion, and the SAFETY comments say why: the build script runs
single-threaded before anything else in the process, and every test reaching
the `config.rs` helpers holds `process_state_lock()` for the duration.

The two `ref` bindings in handlers.rs sit in patterns that already borrow
implicitly, so removing the modifier leaves both bindings at `&String`.

On the 34 drop-order sites: no lock guard's scope is extended anywhere, and
`volume.rs` has none. Most are moved-from `Option`/`Result` husks — `if let
Some(v) = map.remove(&k)`, `while let Some(m) = stream.next().await` — where
the value is moved into the binding and the temporary has nothing left to drop;
where closing order actually matters these paths already call `v.close()`,
`ec_vol.destroy()` or `drop(writer)` explicitly. Two sites get strictly better
ordering: the metrics read guard in `run_metrics_push_loop` shrinks to the end
of its initializer block (it never crossed an `.await` either way), and an EC
test now closes the volume's descriptors before the `TempDir` removes the
directory.

No `rust-version` is declared. Edition 2024 needs rustc 1.85, but that is not
the binding constraint — the dependency tree already requires 1.91.1 through
the `aws-sdk-s3`/`aws-smithy-*` family, so `cargo +1.85 check` fails on the
deps regardless. CI builds on `dtolnay/rust-toolchain@stable`.

`vendor/reed-solomon-erasure` is a separate package and keeps edition 2021.
Cargo.lock is unchanged despite edition 2024 implying resolver 3.

Verified: `cargo test` 551 passed / 0 failed, `cargo test
--no-default-features` 550 passed / 0 failed (the two feature sets produce an
identical migration site list), `cargo build --release` clean. No automated
test covers shutdown ordering, so the channel and runtime sites in `main.rs`,
`write_queue.rs` and `grpc_server.rs` were read individually.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC

* rust volume: address edition-2024 review feedback

Three fixes from review of the edition bump.

Serialize the two environment-reading tests. The SAFETY comments on the
`env::set_var`/`remove_var` helpers claim every test touching the environment
holds `process_state_lock()`, but `test_resolve_config_defaults_dir_to_platform_temp_dir`
and `test_resolve_config_index_accepts_redb_and_leveldb_aliases` called
`resolve_config` — which reads HOME/USERPROFILE, SEAWEED_WRITE_QUEUE and the
WEED_* set — without taking it. `set_var` is unsafe precisely because a
concurrent *reader* is UB, not only a concurrent writer, so the comment was
overclaiming. An audit of the module found exactly these two; every other
environment-touching test already held the lock. The race predates edition
2024, which only made the requirement explicit.

Declare `rust-version = "1.91.1"`. The edition needs 1.85, but that was never
the binding constraint: `cargo +1.90 check --all-targets` fails on the
`aws-sdk-s3`/`aws-smithy-*` family, and 1.91.1 checks clean. Declaring the
verified floor turns a wall of per-dependency errors into one clear message.
Cargo.lock is unchanged despite this making the resolver MSRV-aware.

Update the README, which advertised "Rust 1.75+ (2021 edition)". 1.75 was
already stale before this branch — the tree has needed 1.91 for a while.

Verified: `cargo test` 551 passed / 0 failed, `cargo test --no-default-features`
550 passed / 0 failed, `cargo build --release` clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC

* rust volume: state the exact MSRV patch release in the README

The README said "Rust 1.91+", which reads as 1.91.0 and is wrong by one patch
release: `cargo +1.91.0 check --all-targets` fails on the aws-sdk-s3 family,
`cargo +1.91.1` passes. Say 1.91.1+, matching `rust-version` in Cargo.toml, and
call out that the patch component is load-bearing so nobody installs 1.91.0 and
hits the same wall.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC

* rust worker: move the workspace to edition 2024

Moves the seaweed-worker workspace (core, lance, sort) from edition 2021 to
2024, the same migration seaweed-volume just got in this branch.

Edition 2024 turns exactly one thing in this workspace into a hard error. The
baseline came from RUSTFLAGS='-W rust-2024-compatibility' cargo check
--all-targets, run before the flip; unlike seaweed-volume's 34 silent +
8 hard sites, the worker reports only the one hard site and no
tail_expr_drop_order or if_let_rescope sites at all. The worker is a much
smaller crate and none of its expressions hold a guard or temporary whose drop
order the edition changes, so there is nothing to audit on the silent side.

Fixed (1 site):

std::env::set_var is unsafe as of 2024 because it races with concurrent
readers. The single call is in crates/core/build.rs, which sets PROTOC from
protoc_bin_vendored the way seaweed-volume's build script does. A build
script's main runs single-threaded before anything else in the process, so
no other thread can be reading the environment concurrently; the SAFETY comment
says so. There are no config.rs-style test helpers here -- the worker's tests
do not mutate the environment -- so unlike the volume crate there are no
process_state_lock() callers to audit.

No redundant ref bindings to clean up: a grep for ref across the three
crates finds none.

MSRV:

rust-version = "1.94.1", verified rather than inferred. Edition 2024 only
needs 1.85, but the dependency tree needs more: lance's aws feature pulls in
a newer cut of the same aws-sdk-*/aws-smithy-* family that sets
seaweed-volume's 1.91.1 floor, and that newer cut requires 1.94.1.
cargo +1.94.0 check --all-targets fails on that family; cargo +1.94.1
check --all-targets is clean. The worker's floor is therefore higher than
the volume's, and moves with lance and the AWS SDK rather than with the
edition. CI builds on dtolnay/rust-toolchain@stable, so nothing changes
there.

The edition is set once in [workspace.package] and inherited by each member
via edition.workspace = true; rust-version is added the same way. The
workspace keeps its explicit resolver = "2" -- edition 2024 would default to
resolver 3, but the pin is deliberate and Cargo.lock is unchanged by this
commit either way.

The README gains a "Requires Rust 1.94.1+ (2024 edition)" line in its Building
section, matching the one seaweed-volume's README now carries, and calling out
that the patch release is load-bearing (1.94.0 does not build) so nobody
installs 1.94.0 and hits the same wall.

Verification:

* cargo check --all-targets -- clean, zero warnings (default toolchain 1.97)
* cargo +1.94.1 check --all-targets -- clean
* cargo +1.94.0 check --all-targets -- fails on the AWS SDK, as claimed
* cargo test --all-targets -- 40 passed, 0 failed
  (core 13, sort 11, lance lib 3, lance bin 2, compaction 6, lifecycle 1,
  sort integration 4)
* Cargo.lock unchanged

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-09-09 10:54:06 -07:00

144 lines
6.7 KiB
Markdown
Raw Blame History

# SeaweedFS Volume Server (Rust)
A drop-in replacement for the [SeaweedFS](https://github.com/seaweedfs/seaweedfs) Go volume server, rewritten in Rust. It uses binary-compatible storage formats (`.dat`, `.idx`, `.vif`) and speaks the same HTTP and gRPC protocols, so it works with an unmodified Go master server.
## Building
Requires Rust 1.91.1+ (2024 edition), matching `rust-version` in `Cargo.toml`.
The patch release matters: 1.91.0 does not build. The edition itself only needs
1.85; the higher floor comes from the dependency tree — chiefly the AWS SDK — so
it moves with those crates. CI builds on the latest stable.
```bash
cd seaweed-volume
cargo build --release
```
The binary is produced at `target/release/seaweed-volume`.
## Running
Start a Go master server first, then point the Rust volume server at it:
```bash
# Minimal
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7
# Multiple data directories
seaweed-volume --port 8080 --master localhost:9333 \
--dir /mnt/ssd1,/mnt/ssd2 --max 100,100 --disk ssd
# With datacenter/rack topology
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--dataCenter dc1 --rack rack1
# With JWT authentication
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--securityFile /etc/seaweedfs/security.toml
# With TLS (configured in security.toml via [https.volume] and [grpc.volume] sections)
seaweed-volume --port 8080 --master localhost:9333 --dir /data/vol1 --max 7 \
--securityFile /etc/seaweedfs/security.toml
```
### Common flags
| Flag | Default | Description |
|------|---------|-------------|
| `--port` | `8080` | HTTP listen port |
| `--port.grpc` | `port+10000` | gRPC listen port |
| `--master` | `localhost:9333` | Comma-separated master server addresses |
| `--dir` | `/tmp` | Comma-separated data directories |
| `--max` | `8` | Max volumes per directory (comma-separated) |
| `--ip` | auto-detect | Server IP / identifier |
| `--ip.bind` | same as `--ip` | Bind address |
| `--dataCenter` | | Datacenter name |
| `--rack` | | Rack name |
| `--disk` | | Disk type tag: `hdd`, `ssd`, or custom |
| `--index` | `memory` | Needle map type: `memory`, `leveldb`, `leveldbMedium`, `leveldbLarge` |
| `--readMode` | `proxy` | Non-local read mode: `local`, `proxy`, `redirect` |
| `--fileSizeLimitMB` | `256` | Max upload file size |
| `--minFreeSpace` | `1` (percent) | Min free disk space before marking volumes read-only |
| `--securityFile` | | Path to `security.toml` for JWT keys and TLS certs |
| `--metricsPort` | `0` (disabled) | Prometheus metrics endpoint port |
| `--whiteList` | | Comma-separated IPs with write permission |
| `--preStopSeconds` | `10` | Graceful drain period before shutdown |
| `--compactionMBps` | `0` (unlimited) | Compaction I/O rate limit |
| `--pprof` | `false` | Enable pprof HTTP handlers |
Set `RUST_LOG=debug` (or `trace`, `info`, `warn`) for log level control.
Set `SEAWEED_WRITE_QUEUE=1` to enable batched async write processing.
## Features
- **Binary compatible** -- reads and writes the same `.dat`/`.idx`/`.vif` files as the Go server; seamless migration with no data conversion.
- **HTTP + gRPC** -- full implementation of the volume server HTTP API and all gRPC RPCs including streaming operations (copy, tail, incremental copy, vacuum).
- **Master heartbeat** -- bidirectional streaming heartbeat with the Go master server; volume and EC shard registration, leader failover, graceful shutdown deregistration.
- **JWT authentication** -- signing key configuration via `security.toml` with token source precedence (query > header > cookie), file_id claims validation, and separate read/write keys.
- **TLS** -- HTTPS for the HTTP API and mTLS for gRPC, configured through `security.toml`.
- **Erasure coding** -- Reed-Solomon EC shard management: mount/unmount, read, rebuild, copy, delete, and shard-to-volume reconstruction.
- **S3 remote storage** -- `FetchAndWriteNeedle` reads from any S3-compatible backend (AWS, MinIO, Wasabi, Backblaze, etc.) and writes locally. Supports `VolumeTierMoveDatToRemote`/`FromRemote` for tiered storage.
- **Needle map backends** -- in-memory HashMap, LevelDB (via `rusty-leveldb`), or redb (pure Rust disk-backed) needle maps.
- **Image processing** -- on-the-fly resize/crop, JPEG EXIF orientation auto-fix, WebP support.
- **Streaming reads** -- large files (>1MB) are streamed via `spawn_blocking` to avoid blocking the async runtime.
- **Auto-compression** -- compressible file types (text, JSON, CSS, JS, SVG, etc.) are gzip-compressed on upload.
- **Prometheus metrics** -- counters, histograms, and gauges exported at a dedicated metrics port; optional push gateway support.
- **Graceful shutdown** -- SIGINT/SIGTERM handling with configurable `preStopSeconds` drain period.
## Testing
### Rust unit tests
```bash
cd seaweed-volume
cargo test
```
### Go integration tests
The Go test suite can target either the Go or Rust volume server via the `VOLUME_SERVER_IMPL` environment variable:
```bash
# Run all HTTP + gRPC integration tests against the Rust server
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 1200s \
./test/volume_server/grpc/... ./test/volume_server/http/...
# Run a single test
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 60s \
-run "TestName" ./test/volume_server/http/...
# Run S3 remote storage tests
VOLUME_SERVER_IMPL=rust go test -v -count=1 -timeout 180s \
-run "TestFetchAndWriteNeedle" ./test/volume_server/grpc/...
```
## Load testing
A load test harness is available at `test/volume_server/loadtest/`. See that directory for usage instructions and scenarios.
## Architecture
The server runs three listeners concurrently:
- **HTTP** (Axum 0.7) -- admin and public routers for file upload/download, status, and stats endpoints.
- **gRPC** (Tonic 0.12) -- all `VolumeServer` RPCs from the SeaweedFS protobuf definition.
- **Metrics** (optional) -- Prometheus scrape endpoint on a separate port.
Key source modules:
| Path | Description |
|------|-------------|
| `src/main.rs` | Entry point, server startup, signal handling |
| `src/config.rs` | CLI parsing and configuration resolution |
| `src/server/volume_server.rs` | HTTP router setup and middleware |
| `src/server/handlers.rs` | HTTP request handlers (read, write, delete, status) |
| `src/server/grpc_server.rs` | gRPC service implementation |
| `src/server/heartbeat.rs` | Master heartbeat loop |
| `src/storage/volume.rs` | Volume read/write/delete logic |
| `src/storage/needle.rs` | Needle (file entry) serialization |
| `src/storage/store.rs` | Multi-volume store management |
| `src/security.rs` | JWT validation and IP whitelist guard |
| `src/remote_storage/` | S3 remote storage backend |
See [DEV_PLAN.md](DEV_PLAN.md) for the full development history and feature checklist.
Reference in new issue View Git Blame Copy Permalink