From 516e251f9e1b8376ca208543f2aa76e3b8be7b53 Mon Sep 17 00:00:00 2001 From: Eliah Rusin Date: Wed, 9 Sep 2026 20:54:06 +0300 Subject: [PATCH] rust volume: move the crate to edition 2024 (#11244) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * rust volume: move the crate to edition 2024 Edition 2024 turns three things in this crate into hard errors, and changes drop order in a further 34 places without changing compilation. The compiler errors are fixed here; the silent changes were audited against `RUSTFLAGS='-W rust-2024-compatibility' cargo check --all-targets` output captured before the flip, since edition 2024 stops reporting them. `std::env::set_var`/`remove_var` are unsafe as of 2024 because they race with concurrent readers. All six call sites are safe by construction rather than by assertion, and the SAFETY comments say why: the build script runs single-threaded before anything else in the process, and every test reaching the `config.rs` helpers holds `process_state_lock()` for the duration. The two `ref` bindings in handlers.rs sit in patterns that already borrow implicitly, so removing the modifier leaves both bindings at `&String`. On the 34 drop-order sites: no lock guard's scope is extended anywhere, and `volume.rs` has none. Most are moved-from `Option`/`Result` husks — `if let Some(v) = map.remove(&k)`, `while let Some(m) = stream.next().await` — where the value is moved into the binding and the temporary has nothing left to drop; where closing order actually matters these paths already call `v.close()`, `ec_vol.destroy()` or `drop(writer)` explicitly. Two sites get strictly better ordering: the metrics read guard in `run_metrics_push_loop` shrinks to the end of its initializer block (it never crossed an `.await` either way), and an EC test now closes the volume's descriptors before the `TempDir` removes the directory. No `rust-version` is declared. Edition 2024 needs rustc 1.85, but that is not the binding constraint — the dependency tree already requires 1.91.1 through the `aws-sdk-s3`/`aws-smithy-*` family, so `cargo +1.85 check` fails on the deps regardless. CI builds on `dtolnay/rust-toolchain@stable`. `vendor/reed-solomon-erasure` is a separate package and keeps edition 2021. Cargo.lock is unchanged despite edition 2024 implying resolver 3. Verified: `cargo test` 551 passed / 0 failed, `cargo test --no-default-features` 550 passed / 0 failed (the two feature sets produce an identical migration site list), `cargo build --release` clean. No automated test covers shutdown ordering, so the channel and runtime sites in `main.rs`, `write_queue.rs` and `grpc_server.rs` were read individually. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust volume: address edition-2024 review feedback Three fixes from review of the edition bump. Serialize the two environment-reading tests. The SAFETY comments on the `env::set_var`/`remove_var` helpers claim every test touching the environment holds `process_state_lock()`, but `test_resolve_config_defaults_dir_to_platform_temp_dir` and `test_resolve_config_index_accepts_redb_and_leveldb_aliases` called `resolve_config` — which reads HOME/USERPROFILE, SEAWEED_WRITE_QUEUE and the WEED_* set — without taking it. `set_var` is unsafe precisely because a concurrent *reader* is UB, not only a concurrent writer, so the comment was overclaiming. An audit of the module found exactly these two; every other environment-touching test already held the lock. The race predates edition 2024, which only made the requirement explicit. Declare `rust-version = "1.91.1"`. The edition needs 1.85, but that was never the binding constraint: `cargo +1.90 check --all-targets` fails on the `aws-sdk-s3`/`aws-smithy-*` family, and 1.91.1 checks clean. Declaring the verified floor turns a wall of per-dependency errors into one clear message. Cargo.lock is unchanged despite this making the resolver MSRV-aware. Update the README, which advertised "Rust 1.75+ (2021 edition)". 1.75 was already stale before this branch — the tree has needed 1.91 for a while. Verified: `cargo test` 551 passed / 0 failed, `cargo test --no-default-features` 550 passed / 0 failed, `cargo build --release` clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust volume: state the exact MSRV patch release in the README The README said "Rust 1.91+", which reads as 1.91.0 and is wrong by one patch release: `cargo +1.91.0 check --all-targets` fails on the aws-sdk-s3 family, `cargo +1.91.1` passes. Say 1.91.1+, matching `rust-version` in Cargo.toml, and call out that the patch component is load-bearing so nobody installs 1.91.0 and hits the same wall. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust worker: move the workspace to edition 2024 Moves the seaweed-worker workspace (core, lance, sort) from edition 2021 to 2024, the same migration seaweed-volume just got in this branch. Edition 2024 turns exactly one thing in this workspace into a hard error. The baseline came from RUSTFLAGS='-W rust-2024-compatibility' cargo check --all-targets, run before the flip; unlike seaweed-volume's 34 silent + 8 hard sites, the worker reports only the one hard site and no tail_expr_drop_order or if_let_rescope sites at all. The worker is a much smaller crate and none of its expressions hold a guard or temporary whose drop order the edition changes, so there is nothing to audit on the silent side. Fixed (1 site): std::env::set_var is unsafe as of 2024 because it races with concurrent readers. The single call is in crates/core/build.rs, which sets PROTOC from protoc_bin_vendored the way seaweed-volume's build script does. A build script's main runs single-threaded before anything else in the process, so no other thread can be reading the environment concurrently; the SAFETY comment says so. There are no config.rs-style test helpers here -- the worker's tests do not mutate the environment -- so unlike the volume crate there are no process_state_lock() callers to audit. No redundant ref bindings to clean up: a grep for ref across the three crates finds none. MSRV: rust-version = "1.94.1", verified rather than inferred. Edition 2024 only needs 1.85, but the dependency tree needs more: lance's aws feature pulls in a newer cut of the same aws-sdk-*/aws-smithy-* family that sets seaweed-volume's 1.91.1 floor, and that newer cut requires 1.94.1. cargo +1.94.0 check --all-targets fails on that family; cargo +1.94.1 check --all-targets is clean. The worker's floor is therefore higher than the volume's, and moves with lance and the AWS SDK rather than with the edition. CI builds on dtolnay/rust-toolchain@stable, so nothing changes there. The edition is set once in [workspace.package] and inherited by each member via edition.workspace = true; rust-version is added the same way. The workspace keeps its explicit resolver = "2" -- edition 2024 would default to resolver 3, but the pin is deliberate and Cargo.lock is unchanged by this commit either way. The README gains a "Requires Rust 1.94.1+ (2024 edition)" line in its Building section, matching the one seaweed-volume's README now carries, and calling out that the patch release is load-bearing (1.94.0 does not build) so nobody installs 1.94.0 and hits the same wall. Verification: * cargo check --all-targets -- clean, zero warnings (default toolchain 1.97) * cargo +1.94.1 check --all-targets -- clean * cargo +1.94.0 check --all-targets -- fails on the AWS SDK, as claimed * cargo test --all-targets -- 40 passed, 0 failed (core 13, sort 11, lance lib 3, lance bin 2, compaction 6, lifecycle 1, sort integration 4) * Cargo.lock unchanged Co-Authored-By: Claude Opus 5 (1M context) --------- Co-authored-by: Claude Opus 5 (1M context) Co-authored-by: Chris Lu --- seaweed-volume/Cargo.toml | 5 +++- seaweed-volume/README.md | 5 +++- seaweed-volume/build.rs | 7 +++++- seaweed-volume/src/config.rs | 34 ++++++++++++++++++++------ seaweed-volume/src/server/handlers.rs | 4 +-- seaweed-worker/Cargo.toml | 6 ++++- seaweed-worker/README.md | 6 +++++ seaweed-worker/crates/core/Cargo.toml | 1 + seaweed-worker/crates/core/build.rs | 7 +++++- seaweed-worker/crates/lance/Cargo.toml | 1 + seaweed-worker/crates/sort/Cargo.toml | 1 + 11 files changed, 62 insertions(+), 15 deletions(-) diff --git a/seaweed-volume/Cargo.toml b/seaweed-volume/Cargo.toml index 01f97e1d7..b3c1fd866 100644 --- a/seaweed-volume/Cargo.toml +++ b/seaweed-volume/Cargo.toml @@ -1,7 +1,10 @@ [package] name = "weed-volume" version = "0.1.0" -edition = "2021" +edition = "2024" +# The edition needs 1.85; the dependency tree needs more. Verified with +# `cargo +1.91.1 check --all-targets` (1.90 fails on the AWS SDK). +rust-version = "1.91.1" description = "SeaweedFS Volume Server — Rust implementation" [lib] diff --git a/seaweed-volume/README.md b/seaweed-volume/README.md index 4367a0722..039cb436b 100644 --- a/seaweed-volume/README.md +++ b/seaweed-volume/README.md @@ -4,7 +4,10 @@ A drop-in replacement for the [SeaweedFS](https://github.com/seaweedfs/seaweedfs ## Building -Requires Rust 1.75+ (2021 edition). +Requires Rust 1.91.1+ (2024 edition), matching `rust-version` in `Cargo.toml`. +The patch release matters: 1.91.0 does not build. The edition itself only needs +1.85; the higher floor comes from the dependency tree — chiefly the AWS SDK — so +it moves with those crates. CI builds on the latest stable. ```bash cd seaweed-volume diff --git a/seaweed-volume/build.rs b/seaweed-volume/build.rs index 99af370a3..90ca9ef26 100644 --- a/seaweed-volume/build.rs +++ b/seaweed-volume/build.rs @@ -3,7 +3,12 @@ fn main() -> Result<(), Box> { // one, so the build needs no package manager and always sees the same // version. An explicit PROTOC still wins, for packagers supplying their own. if std::env::var_os("PROTOC").is_none() { - std::env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?); + // SAFETY: a build script's main runs single-threaded before anything + // else in this process, so no other thread can be reading the + // environment concurrently. + unsafe { + std::env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?); + } } let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR")?); diff --git a/seaweed-volume/src/config.rs b/seaweed-volume/src/config.rs index 007ad4b0a..5eb5292cf 100644 --- a/seaweed-volume/src/config.rs +++ b/seaweed-volume/src/config.rs @@ -1209,21 +1209,30 @@ mod tests { LOCK.get_or_init(|| Mutex::new(())).lock().unwrap() } + // SAFETY (all env mutation in this module): `set_var`/`remove_var` are + // unsafe as of Rust 2024 because they race with concurrent readers in + // other threads. Every test that reaches these helpers holds + // `process_state_lock()` for the duration, so only one test at a time + // touches the environment and none observes another's edit. fn with_temp_env_var(key: &str, value: Option<&str>, f: F) { let previous = std::env::var_os(key); - match value { - Some(v) => std::env::set_var(key, v), - None => std::env::remove_var(key), + unsafe { + match value { + Some(v) => std::env::set_var(key, v), + None => std::env::remove_var(key), + } } f(); restore_env_var(key, previous); } fn restore_env_var(key: &str, value: Option) { - if let Some(value) = value { - std::env::set_var(key, value); - } else { - std::env::remove_var(key); + unsafe { + if let Some(value) = value { + std::env::set_var(key, value); + } else { + std::env::remove_var(key); + } } } @@ -1268,7 +1277,10 @@ mod tests { .collect(); for key in KEYS { - std::env::remove_var(key); + // SAFETY: as above — the caller holds `process_state_lock()`. + unsafe { + std::env::remove_var(key); + } } f(); @@ -1404,12 +1416,18 @@ mod tests { #[test] fn test_resolve_config_defaults_dir_to_platform_temp_dir() { + // resolve_config reads HOME/USERPROFILE and the WEED_* set, so it has to + // hold the same lock the mutation helpers take — a concurrent set_var + // during this read is exactly what makes those calls unsafe. + let _guard = process_state_lock(); let cfg = resolve_config(Cli::parse_from(["bin"])); assert_eq!(cfg.folders, vec![default_volume_dir()]); } #[test] fn test_resolve_config_index_accepts_redb_and_leveldb_aliases() { + // As above: resolve_config reads the environment. + let _guard = process_state_lock(); let pairs = [ ("memory", NeedleMapKind::InMemory), ("redb", NeedleMapKind::Redb), diff --git a/seaweed-volume/src/server/handlers.rs b/seaweed-volume/src/server/handlers.rs index da7513810..fb40070b1 100644 --- a/seaweed-volume/src/server/handlers.rs +++ b/seaweed-volume/src/server/handlers.rs @@ -2425,7 +2425,7 @@ pub async fn post_handler( } else { None }; - if let (Some(ref expected_md5), Some(ref actual_md5)) = (&content_md5, &original_content_md5) { + if let (Some(expected_md5), Some(actual_md5)) = (&content_md5, &original_content_md5) { if expected_md5 != actual_md5 { return json_error_with_query( StatusCode::BAD_REQUEST, @@ -3382,7 +3382,7 @@ async fn try_expand_chunk_manifest( response_headers.insert(header::ACCEPT_RANGES, "bytes".parse().unwrap()); // Last-Modified — Go sets this on the response writer before tryHandleChunkedFile - if let Some(ref lm) = last_modified_str { + if let Some(lm) = last_modified_str { if let Ok(hval) = lm.parse() { response_headers.insert(header::LAST_MODIFIED, hval); } diff --git a/seaweed-worker/Cargo.toml b/seaweed-worker/Cargo.toml index c3064d660..3255ddb46 100644 --- a/seaweed-worker/Cargo.toml +++ b/seaweed-worker/Cargo.toml @@ -10,7 +10,11 @@ members = ["crates/core", "crates/lance", "crates/sort"] [workspace.package] version = "0.1.0" -edition = "2021" +edition = "2024" +# The edition needs 1.85; the dependency tree needs more. Verified with +# `cargo +1.94.1 check --all-targets` (1.94.0 fails on the AWS SDK that +# lance's `aws` feature pulls in). +rust-version = "1.94.1" [workspace.dependencies] anyhow = "1" diff --git a/seaweed-worker/README.md b/seaweed-worker/README.md index 6dc6eb6cc..f4de6ece3 100644 --- a/seaweed-worker/README.md +++ b/seaweed-worker/README.md @@ -14,6 +14,12 @@ one. ## Building +Requires Rust 1.94.1+ (2024 edition), matching `rust-version` in `Cargo.toml`. +The patch release matters: 1.94.0 does not build. The edition itself only needs +1.85; the higher floor comes from the dependency tree — lance's `aws` feature +pulls in the AWS SDK — so it moves with those crates. CI builds on the latest +stable. + `core` compiles `plugin.proto` with the protoc that protoc-bin-vendored ships, the way seaweed-volume does, so it needs no system install. diff --git a/seaweed-worker/crates/core/Cargo.toml b/seaweed-worker/crates/core/Cargo.toml index e1ae1a10e..2a785124a 100644 --- a/seaweed-worker/crates/core/Cargo.toml +++ b/seaweed-worker/crates/core/Cargo.toml @@ -2,6 +2,7 @@ name = "seaweed-worker-core" version.workspace = true edition.workspace = true +rust-version.workspace = true description = "SeaweedFS plugin.proto worker contract" [lib] diff --git a/seaweed-worker/crates/core/build.rs b/seaweed-worker/crates/core/build.rs index 07a4ab57d..f76f0fdab 100644 --- a/seaweed-worker/crates/core/build.rs +++ b/seaweed-worker/crates/core/build.rs @@ -4,7 +4,12 @@ fn main() -> Result<(), Box> { // version. An explicit PROTOC still wins, for packagers supplying their own // and for the lance crates, whose own build scripts read the same variable. if std::env::var_os("PROTOC").is_none() { - std::env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?); + // SAFETY: a build script's main runs single-threaded before anything + // else in this process, so no other thread can be reading the + // environment concurrently. + unsafe { + std::env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?); + } } // Compiled straight out of the Go tree, the way seaweed-volume already reads diff --git a/seaweed-worker/crates/lance/Cargo.toml b/seaweed-worker/crates/lance/Cargo.toml index 962a2b7ac..c8cf3648a 100644 --- a/seaweed-worker/crates/lance/Cargo.toml +++ b/seaweed-worker/crates/lance/Cargo.toml @@ -2,6 +2,7 @@ name = "weed-lance-worker" version.workspace = true edition.workspace = true +rust-version.workspace = true description = "SeaweedFS maintenance worker for Lance tables" [lib] diff --git a/seaweed-worker/crates/sort/Cargo.toml b/seaweed-worker/crates/sort/Cargo.toml index c24a548c1..4918f42c7 100644 --- a/seaweed-worker/crates/sort/Cargo.toml +++ b/seaweed-worker/crates/sort/Cargo.toml @@ -2,6 +2,7 @@ name = "seaweed-worker-sort" version.workspace = true edition.workspace = true +rust-version.workspace = true description = "The sort specification shared by SeaweedFS sorting jobs" [lib]