mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 13:30:46 +02:00
516e251f9e
* rust volume: move the crate to edition 2024 Edition 2024 turns three things in this crate into hard errors, and changes drop order in a further 34 places without changing compilation. The compiler errors are fixed here; the silent changes were audited against `RUSTFLAGS='-W rust-2024-compatibility' cargo check --all-targets` output captured before the flip, since edition 2024 stops reporting them. `std::env::set_var`/`remove_var` are unsafe as of 2024 because they race with concurrent readers. All six call sites are safe by construction rather than by assertion, and the SAFETY comments say why: the build script runs single-threaded before anything else in the process, and every test reaching the `config.rs` helpers holds `process_state_lock()` for the duration. The two `ref` bindings in handlers.rs sit in patterns that already borrow implicitly, so removing the modifier leaves both bindings at `&String`. On the 34 drop-order sites: no lock guard's scope is extended anywhere, and `volume.rs` has none. Most are moved-from `Option`/`Result` husks — `if let Some(v) = map.remove(&k)`, `while let Some(m) = stream.next().await` — where the value is moved into the binding and the temporary has nothing left to drop; where closing order actually matters these paths already call `v.close()`, `ec_vol.destroy()` or `drop(writer)` explicitly. Two sites get strictly better ordering: the metrics read guard in `run_metrics_push_loop` shrinks to the end of its initializer block (it never crossed an `.await` either way), and an EC test now closes the volume's descriptors before the `TempDir` removes the directory. No `rust-version` is declared. Edition 2024 needs rustc 1.85, but that is not the binding constraint — the dependency tree already requires 1.91.1 through the `aws-sdk-s3`/`aws-smithy-*` family, so `cargo +1.85 check` fails on the deps regardless. CI builds on `dtolnay/rust-toolchain@stable`. `vendor/reed-solomon-erasure` is a separate package and keeps edition 2021. Cargo.lock is unchanged despite edition 2024 implying resolver 3. Verified: `cargo test` 551 passed / 0 failed, `cargo test --no-default-features` 550 passed / 0 failed (the two feature sets produce an identical migration site list), `cargo build --release` clean. No automated test covers shutdown ordering, so the channel and runtime sites in `main.rs`, `write_queue.rs` and `grpc_server.rs` were read individually. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust volume: address edition-2024 review feedback Three fixes from review of the edition bump. Serialize the two environment-reading tests. The SAFETY comments on the `env::set_var`/`remove_var` helpers claim every test touching the environment holds `process_state_lock()`, but `test_resolve_config_defaults_dir_to_platform_temp_dir` and `test_resolve_config_index_accepts_redb_and_leveldb_aliases` called `resolve_config` — which reads HOME/USERPROFILE, SEAWEED_WRITE_QUEUE and the WEED_* set — without taking it. `set_var` is unsafe precisely because a concurrent *reader* is UB, not only a concurrent writer, so the comment was overclaiming. An audit of the module found exactly these two; every other environment-touching test already held the lock. The race predates edition 2024, which only made the requirement explicit. Declare `rust-version = "1.91.1"`. The edition needs 1.85, but that was never the binding constraint: `cargo +1.90 check --all-targets` fails on the `aws-sdk-s3`/`aws-smithy-*` family, and 1.91.1 checks clean. Declaring the verified floor turns a wall of per-dependency errors into one clear message. Cargo.lock is unchanged despite this making the resolver MSRV-aware. Update the README, which advertised "Rust 1.75+ (2021 edition)". 1.75 was already stale before this branch — the tree has needed 1.91 for a while. Verified: `cargo test` 551 passed / 0 failed, `cargo test --no-default-features` 550 passed / 0 failed, `cargo build --release` clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust volume: state the exact MSRV patch release in the README The README said "Rust 1.91+", which reads as 1.91.0 and is wrong by one patch release: `cargo +1.91.0 check --all-targets` fails on the aws-sdk-s3 family, `cargo +1.91.1` passes. Say 1.91.1+, matching `rust-version` in Cargo.toml, and call out that the patch component is load-bearing so nobody installs 1.91.0 and hits the same wall. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018nty5Rj7ssMQdFxHHjZgDC * rust worker: move the workspace to edition 2024 Moves the seaweed-worker workspace (core, lance, sort) from edition 2021 to 2024, the same migration seaweed-volume just got in this branch. Edition 2024 turns exactly one thing in this workspace into a hard error. The baseline came from RUSTFLAGS='-W rust-2024-compatibility' cargo check --all-targets, run before the flip; unlike seaweed-volume's 34 silent + 8 hard sites, the worker reports only the one hard site and no tail_expr_drop_order or if_let_rescope sites at all. The worker is a much smaller crate and none of its expressions hold a guard or temporary whose drop order the edition changes, so there is nothing to audit on the silent side. Fixed (1 site): std::env::set_var is unsafe as of 2024 because it races with concurrent readers. The single call is in crates/core/build.rs, which sets PROTOC from protoc_bin_vendored the way seaweed-volume's build script does. A build script's main runs single-threaded before anything else in the process, so no other thread can be reading the environment concurrently; the SAFETY comment says so. There are no config.rs-style test helpers here -- the worker's tests do not mutate the environment -- so unlike the volume crate there are no process_state_lock() callers to audit. No redundant ref bindings to clean up: a grep for ref across the three crates finds none. MSRV: rust-version = "1.94.1", verified rather than inferred. Edition 2024 only needs 1.85, but the dependency tree needs more: lance's aws feature pulls in a newer cut of the same aws-sdk-*/aws-smithy-* family that sets seaweed-volume's 1.91.1 floor, and that newer cut requires 1.94.1. cargo +1.94.0 check --all-targets fails on that family; cargo +1.94.1 check --all-targets is clean. The worker's floor is therefore higher than the volume's, and moves with lance and the AWS SDK rather than with the edition. CI builds on dtolnay/rust-toolchain@stable, so nothing changes there. The edition is set once in [workspace.package] and inherited by each member via edition.workspace = true; rust-version is added the same way. The workspace keeps its explicit resolver = "2" -- edition 2024 would default to resolver 3, but the pin is deliberate and Cargo.lock is unchanged by this commit either way. The README gains a "Requires Rust 1.94.1+ (2024 edition)" line in its Building section, matching the one seaweed-volume's README now carries, and calling out that the patch release is load-bearing (1.94.0 does not build) so nobody installs 1.94.0 and hits the same wall. Verification: * cargo check --all-targets -- clean, zero warnings (default toolchain 1.97) * cargo +1.94.1 check --all-targets -- clean * cargo +1.94.0 check --all-targets -- fails on the AWS SDK, as claimed * cargo test --all-targets -- 40 passed, 0 failed (core 13, sort 11, lance lib 3, lance bin 2, compaction 6, lifecycle 1, sort integration 4) * Cargo.lock unchanged Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>
111 lines
5.1 KiB
Markdown
111 lines
5.1 KiB
Markdown
# SeaweedFS Rust workers
|
|
|
|
`weed/pb/plugin.proto` is a language-agnostic contract: a maintenance worker
|
|
connects out to admin, announces the job types it can detect and execute, and
|
|
answers requests on that one stream. `weed worker -admin=host:23646` is the Go
|
|
implementation of it from outside the admin process. This workspace is the Rust
|
|
one.
|
|
|
|
crates/core the contract: stream, handshake, heartbeat, registry, config forms
|
|
crates/lance maintenance jobs for Lance tables, and a binary
|
|
|
|
`core` knows nothing about any job. A second worker is a new crate beside
|
|
`lance` that depends on it, not a fork of the protocol.
|
|
|
|
## Building
|
|
|
|
Requires Rust 1.94.1+ (2024 edition), matching `rust-version` in `Cargo.toml`.
|
|
The patch release matters: 1.94.0 does not build. The edition itself only needs
|
|
1.85; the higher floor comes from the dependency tree — lance's `aws` feature
|
|
pulls in the AWS SDK — so it moves with those crates. CI builds on the latest
|
|
stable.
|
|
|
|
`core` compiles `plugin.proto` with the protoc that protoc-bin-vendored ships,
|
|
the way seaweed-volume does, so it needs no system install.
|
|
|
|
The lance crates compile protos of their own, in their own build-script
|
|
processes, which nothing our build script sets can reach. They need a protoc of
|
|
their own: either one on PATH — `brew install protobuf`, `apt install
|
|
protobuf-compiler` — or `PROTOC` naming one. CI points it at the vendored
|
|
binary for the runner's platform, resolved from the version in `Cargo.lock`.
|
|
|
|
## Running
|
|
|
|
cargo run -p weed-lance-worker -- --admin 127.0.0.1:23646
|
|
|
|
The admin's *HTTP* address is what an operator has; the gRPC port is derived
|
|
from it the way the Go side does. Dialling the HTTP port fails as "frame with
|
|
invalid size", which reads like a protocol bug rather than a wrong port.
|
|
|
|
The binary is `weed-worker`, not `weed-lance-worker`: it is the Rust side of
|
|
`weed worker`, and lance is the first family of jobs it carries rather than the
|
|
only one it ever will.
|
|
|
|
Released builds do not need a toolchain. The worker ships inside the SeaweedFS
|
|
image, beside the Rust volume server, under the verb that mirrors
|
|
`volume-rust`:
|
|
|
|
docker run chrislusf/seaweedfs worker-rust --admin admin:23646
|
|
|
|
and as `weed-worker_linux_{amd64,arm64}.tar.gz` on each GitHub release. Both are
|
|
linux amd64/arm64 only — lance, arrow and datafusion make every extra target an
|
|
expensive build, and the worker runs beside the cluster it maintains. On an
|
|
architecture without a build the image carries an empty placeholder and the
|
|
entrypoint says so rather than failing as "not found".
|
|
|
|
## Metrics
|
|
|
|
cargo run -p weed-lance-worker -- --admin 127.0.0.1:23646 --metrics-port 9328
|
|
|
|
Serves `/health`, `/ready` and `/metrics` on that port, the same three the Go
|
|
worker serves under `weed worker -metricsPort`, so one scrape config covers
|
|
workers in either language. Off by default, and bound to loopback unless
|
|
`--metrics-ip` says otherwise, because the endpoint is unauthenticated. 9328
|
|
continues the series the other components use (master 9324, volume 9325, filer
|
|
9326, s3 9327); an IPv6 address works with or without brackets.
|
|
|
|
Grafana: the "Plugin Workers" row of `other/metrics/grafana_seaweedfs.json`
|
|
graphs these. Its panels filter on `$cluster`, which comes from the scrape job's
|
|
labels, so scrape the worker the way the rest of the cluster is scraped or the
|
|
row stays empty.
|
|
|
|
Names are `SeaweedFS_worker_*`, matching the Go side's convention. The pair
|
|
worth alerting on is `objects_seen_total` and `objects_skipped_total`: a sweep
|
|
that proposes nothing and a sweep that could read nothing look identical from
|
|
`proposals_total` alone.
|
|
|
|
SeaweedFS_worker_connected 1
|
|
SeaweedFS_worker_objects_seen_total{job_type="lance_compact"} 7
|
|
SeaweedFS_worker_proposals_total{job_type="lance_compact"} 2
|
|
SeaweedFS_worker_jobs_total{job_type="lance_compact",result="ok"} 2
|
|
SeaweedFS_worker_lance_fragments_removed_total 25
|
|
|
|
`/ready` follows the control stream: a worker whose admin has gone away is
|
|
running but is not going to do anything.
|
|
|
|
## Credentials
|
|
|
|
The worker holds none. It asks the namespace to describe a table with
|
|
`vend_credentials` and hands the `storage_options` that come back to lance. A
|
|
gateway without STS configured vends no credentials at all, so `--access-key`
|
|
and `--secret-key` supply a fallback; anything the namespace does vend wins over
|
|
them.
|
|
|
|
## State
|
|
|
|
All three jobs are implemented and tested end to end against a live gateway:
|
|
|
|
compaction result: 12 fragments became 1
|
|
reindex result: 512 uncovered rows became 0
|
|
cleanup result: removed 14 versions and 24272 bytes
|
|
|
|
`cargo test -p weed-lance-worker` runs them when `WEED_LANCE_NAMESPACE` names a
|
|
live namespace and skips otherwise, the way the Go integration tests skip
|
|
without Docker. Each test seeds the table it needs, including building a vector
|
|
index and then appending rows outside it, so a run does not depend on what the
|
|
previous one left behind — the first version of these did, and quietly stopped
|
|
testing anything once it had done its job.
|
|
|
|
The handshake, descriptor exchange and heartbeat work against a live admin,
|
|
which logs the worker connecting and prefetches all three descriptors.
|