Files
seaweedfs/seaweed-volume/src/security/tls.rs
T
7643f4f541 rust: a seaweed-common crate for the address and TLS helpers both crates carry (#11358)
* rust: a seaweed-common crate for the address and TLS helpers both crates carry

seaweed-volume and seaweed-worker are separate cargo trees with separate
lockfiles and no root manifest, so anything both of them need has had to be
written twice. Two of those copies are a correctness risk rather than a typing
cost, and this crate is where they stop being copies.

address.rs is the HTTP<->gRPC port rule: `host:port` means gRPC on port+10000,
`host:port.grpcPort` names it outright. The two copies had already drifted —
the worker's bracketed IPv6 literals, the volume server's did not — so the rule
lives here once, returning a typed AddressError whose Display text is the volume
server's original wording, with join_host_port public beside it. A test asserts
two of those messages in full rather than by substring, because the wording is
the contract its callers hand to a Status or an io::Error; the other three end
in a std ParseIntError message, which is std's to reword. The enum is
#[non_exhaustive] so a future variant is not a breaking change for either
consumer. The tests are both crates' cases together, plus the IPv6,
already-bracketed and normalisation cases neither copy covered on its own.

tls.rs is install_default_crypto_provider. Both binaries link aws-lc-rs and ring
transitively, so rustls cannot auto-select and tonic's client TLS panics on
first use; each binary has to pin one and it has to be the same one, which is
exactly the kind of choice that should not exist twice. It is safe to share
because `cargo tree -i rustls` resolves a single rustls in each tree (0.23.37 in
seaweed-volume, 0.23.43 in seaweed-worker) and cargo unifies all
semver-compatible `rustls = "0.23"` requirements into one crate per binary, so
this crate writes the same process-wide static its consumer reads. rustls is
already in both graphs — directly in the volume server, through tonic's
tls-aws-lc in seaweed-worker-core — so the dependency adds no crate to either.

rust-version is 1.91.1, the lower of the two consumers' floors, so depending on
this crate cannot raise either tree's MSRV; verified with
`cargo +1.91.1 check --all-targets`. The lockfile is committed even though this
is a library: CI builds it directly, so a committed lock is what makes those
runs reproducible and their caches stable.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* rust: take the address and TLS helpers from seaweed-common

Both public signatures are kept, so no caller outside the two wrapper files
changes. parse_grpc_address stays `Result<String, String>` and maps the typed
error through Display; server_to_grpc_address stays `Option<String>` and drops
it with .ok(). Their doc comments and the volume server's 13 call sites are
otherwise untouched.

Three behaviours change, each in the direction of the copy that was already
right:

- The volume server now brackets IPv6 literals. `::1:19333` used to come back as
  `::1:29333`, which build_grpc_endpoint rejects with "invalid gRPC endpoint
  http://::1:19333: invalid authority" — an IPv6 master or EC peer could not be
  dialled at all. Two tests in grpc_client.rs pin it, one on the string and one
  on the endpoint the string builds.
- The volume server now emits the *parsed* gRPC port of the dotted form instead
  of the original text it had just validated, so `host:8080.018080` and
  `host:8080.+18080` come back as `host:18080` rather than as authorities the
  URI parser rejects. Same port either way; only malformed spellings change.
- The worker's dotted form now validates the HTTP port it discards.
  `server_to_grpc_address("host:abc.18080")` used to answer Some("host:18080");
  it now answers None, which is what the volume server's copy has always done.

install_default_crypto_provider becomes a re-export in both trees, so
`crate::security::tls::install_default_crypto_provider` and
`weed_lance_worker::tls::install_default_crypto_provider` still resolve. The
lance crate's `rustls = "0.23"` was its only direct use of rustls and goes away
with the body; seaweed-common states the same requirement, so neither the
resolved version nor the enabled features move in either lockfile.

The PEM test fixtures stay where they are. The two tests that use them are not
duplicates: the volume server's exercises build_grpc_endpoint, and the lance one
exists precisely because aws-lc-rs and ring are both linked in that crate's
graph. Only the literals are shared, and exporting test fixtures from a library
to dedupe two constants costs more than it saves.

A path dependency outside both trees means every build context that copies one
crate directory has to copy the other. The repo has one: the Rust source-build
stage of docker/Dockerfile.go_build, which now copies seaweed-common beside
seaweed-volume. Every workflow whose `paths:` filter keys on a crate directory
gains `seaweed-common/**` — the two Rust test workflows, rust_binaries_dev,
container_dev and performance. The tag- and dispatch-triggered ones
(rust_binaries_release, container_release_unified, container_latest) have no
`paths:` filter and need nothing.

The two Rust test workflows also run `cargo test` in seaweed-common, from their
unit-test job, because a path dependency is not a workspace member and neither
tree's own `cargo test` reaches it. Each step builds into its job's cached
target directory, and both cache keys now hash seaweed-common/Cargo.lock as well
so a change there invalidates the cache it would otherwise silently reuse.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docker: keep go_build working for BRANCH revisions without seaweed-common

The rust_builder stage copies seaweed-common unconditionally now that seaweed-volume path-depends on it, but BRANCH can name any revision — including ones that predate the crate. Create the directory in the builder stage so the COPY always has a source; an empty dir beside an old seaweed-volume is harmless.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-09-20 13:30:58 -07:00

446 lines
15 KiB
Rust

use std::collections::HashSet;
use std::fmt;
use std::sync::Arc;
use rustls::client::danger::HandshakeSignatureValid;
use rustls::crypto::CryptoProvider;
use rustls::crypto::aws_lc_rs;
use rustls::pki_types::UnixTime;
use rustls::pki_types::{CertificateDer, PrivateKeyDer};
use rustls::server::WebPkiClientVerifier;
use rustls::server::danger::{ClientCertVerified, ClientCertVerifier};
use rustls::{
CipherSuite, DigitallySignedStruct, DistinguishedName, RootCertStore, ServerConfig,
SignatureScheme, SupportedCipherSuite, SupportedProtocolVersion,
};
use x509_parser::prelude::{FromDer, X509Certificate};
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct TlsPolicy {
pub min_version: String,
pub max_version: String,
pub cipher_suites: String,
}
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct GrpcClientAuthPolicy {
pub allowed_common_names: Vec<String>,
pub allowed_wildcard_domain: String,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TlsPolicyError(String);
impl fmt::Display for TlsPolicyError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
impl std::error::Error for TlsPolicyError {}
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
enum GoTlsVersion {
Ssl3,
Tls10,
Tls11,
Tls12,
Tls13,
}
#[derive(Debug)]
struct CommonNameVerifier {
inner: Arc<dyn ClientCertVerifier>,
allowed_common_names: HashSet<String>,
allowed_wildcard_domain: String,
}
impl ClientCertVerifier for CommonNameVerifier {
fn offer_client_auth(&self) -> bool {
self.inner.offer_client_auth()
}
fn client_auth_mandatory(&self) -> bool {
self.inner.client_auth_mandatory()
}
fn root_hint_subjects(&self) -> &[DistinguishedName] {
self.inner.root_hint_subjects()
}
fn verify_client_cert(
&self,
end_entity: &CertificateDer<'_>,
intermediates: &[CertificateDer<'_>],
now: UnixTime,
) -> Result<ClientCertVerified, rustls::Error> {
self.inner
.verify_client_cert(end_entity, intermediates, now)?;
let common_name = parse_common_name(end_entity).map_err(|e| {
rustls::Error::General(format!(
"parse client certificate common name failed: {}",
e
))
})?;
if common_name_is_allowed(
&common_name,
&self.allowed_common_names,
&self.allowed_wildcard_domain,
) {
return Ok(ClientCertVerified::assertion());
}
Err(rustls::Error::General(format!(
"Authenticate: invalid subject client common name: {}",
common_name
)))
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &CertificateDer<'_>,
dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
self.inner.verify_tls12_signature(message, cert, dss)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &CertificateDer<'_>,
dss: &DigitallySignedStruct,
) -> Result<HandshakeSignatureValid, rustls::Error> {
self.inner.verify_tls13_signature(message, cert, dss)
}
fn supported_verify_schemes(&self) -> Vec<SignatureScheme> {
self.inner.supported_verify_schemes()
}
}
// aws-lc-rs and ring both get linked transitively, so rustls can't auto-select
// a provider and tonic's client TLS panics on first use. Pin the default to
// aws-lc-rs, matching the server config. Idempotent. The body lives in
// seaweed-common so this binary and the Rust plugin workers cannot end up
// installing different providers; re-exported here so callers keep their
// import path.
pub use seaweed_common::tls::install_default_crypto_provider;
pub fn build_rustls_server_config(
cert_path: &str,
key_path: &str,
ca_path: &str,
policy: &TlsPolicy,
) -> Result<ServerConfig, TlsPolicyError> {
build_rustls_server_config_with_client_auth(cert_path, key_path, ca_path, policy, None)
}
pub fn build_rustls_server_config_with_grpc_client_auth(
cert_path: &str,
key_path: &str,
ca_path: &str,
policy: &TlsPolicy,
client_auth_policy: &GrpcClientAuthPolicy,
) -> Result<ServerConfig, TlsPolicyError> {
build_rustls_server_config_with_client_auth(
cert_path,
key_path,
ca_path,
policy,
Some(client_auth_policy),
)
}
fn build_rustls_server_config_with_client_auth(
cert_path: &str,
key_path: &str,
ca_path: &str,
policy: &TlsPolicy,
client_auth_policy: Option<&GrpcClientAuthPolicy>,
) -> Result<ServerConfig, TlsPolicyError> {
let cert_chain = read_cert_chain(cert_path)?;
let private_key = read_private_key(key_path)?;
let provider = build_crypto_provider(policy)?;
let versions = build_supported_versions(policy)?;
let builder = ServerConfig::builder_with_provider(provider.clone())
.with_protocol_versions(&versions)
.map_err(|e| TlsPolicyError(format!("invalid TLS version policy: {}", e)))?;
let builder = if ca_path.is_empty() {
builder.with_no_client_auth()
} else {
let roots = read_root_store(ca_path)?;
let verifier =
WebPkiClientVerifier::builder_with_provider(Arc::new(roots), provider.clone())
.build()
.map_err(|e| TlsPolicyError(format!("build client verifier failed: {}", e)))?;
let verifier: Arc<dyn ClientCertVerifier> = if let Some(client_auth_policy) =
client_auth_policy.filter(|policy| {
!policy.allowed_common_names.is_empty()
|| !policy.allowed_wildcard_domain.is_empty()
}) {
Arc::new(CommonNameVerifier {
inner: verifier,
allowed_common_names: client_auth_policy
.allowed_common_names
.iter()
.cloned()
.collect(),
allowed_wildcard_domain: client_auth_policy.allowed_wildcard_domain.clone(),
})
} else {
verifier
};
builder.with_client_cert_verifier(verifier)
};
builder
.with_single_cert(cert_chain, private_key)
.map_err(|e| TlsPolicyError(format!("build rustls server config failed: {}", e)))
}
fn read_cert_chain(cert_path: &str) -> Result<Vec<CertificateDer<'static>>, TlsPolicyError> {
let cert_pem = std::fs::read(cert_path).map_err(|e| {
TlsPolicyError(format!(
"Failed to read TLS cert file '{}': {}",
cert_path, e
))
})?;
rustls_pemfile::certs(&mut &cert_pem[..])
.collect::<Result<Vec<_>, _>>()
.map_err(|e| {
TlsPolicyError(format!(
"Failed to parse TLS cert PEM '{}': {}",
cert_path, e
))
})
}
fn read_private_key(key_path: &str) -> Result<PrivateKeyDer<'static>, TlsPolicyError> {
let key_pem = std::fs::read(key_path).map_err(|e| {
TlsPolicyError(format!("Failed to read TLS key file '{}': {}", key_path, e))
})?;
rustls_pemfile::private_key(&mut &key_pem[..])
.map_err(|e| TlsPolicyError(format!("Failed to parse TLS key PEM '{}': {}", key_path, e)))?
.ok_or_else(|| TlsPolicyError(format!("No private key found in '{}'", key_path)))
}
fn read_root_store(ca_path: &str) -> Result<RootCertStore, TlsPolicyError> {
let ca_pem = std::fs::read(ca_path)
.map_err(|e| TlsPolicyError(format!("Failed to read TLS CA file '{}': {}", ca_path, e)))?;
let ca_certs = rustls_pemfile::certs(&mut &ca_pem[..])
.collect::<Result<Vec<_>, _>>()
.map_err(|e| TlsPolicyError(format!("Failed to parse TLS CA PEM '{}': {}", ca_path, e)))?;
let mut roots = RootCertStore::empty();
for cert in ca_certs {
roots
.add(cert)
.map_err(|e| TlsPolicyError(format!("Failed to add CA cert '{}': {}", ca_path, e)))?;
}
Ok(roots)
}
fn build_crypto_provider(policy: &TlsPolicy) -> Result<Arc<CryptoProvider>, TlsPolicyError> {
let mut provider = aws_lc_rs::default_provider();
let cipher_suites = parse_cipher_suites(&provider.cipher_suites, &policy.cipher_suites)?;
if !cipher_suites.is_empty() {
provider.cipher_suites = cipher_suites;
}
Ok(Arc::new(provider))
}
pub fn build_supported_versions(
policy: &TlsPolicy,
) -> Result<Vec<&'static SupportedProtocolVersion>, TlsPolicyError> {
let min_version = parse_go_tls_version(&policy.min_version)?;
let max_version = parse_go_tls_version(&policy.max_version)?;
let versions = [&rustls::version::TLS13, &rustls::version::TLS12]
.into_iter()
.filter(|version| {
let current = go_tls_version_for_supported(version);
min_version.map(|min| current >= min).unwrap_or(true)
&& max_version.map(|max| current <= max).unwrap_or(true)
})
.collect::<Vec<_>>();
if versions.is_empty() {
return Err(TlsPolicyError(format!(
"TLS version range min='{}' max='{}' is unsupported by rustls",
policy.min_version, policy.max_version
)));
}
Ok(versions)
}
fn parse_go_tls_version(value: &str) -> Result<Option<GoTlsVersion>, TlsPolicyError> {
match value.trim() {
"" => Ok(None),
"SSLv3" => Ok(Some(GoTlsVersion::Ssl3)),
"TLS 1.0" => Ok(Some(GoTlsVersion::Tls10)),
"TLS 1.1" => Ok(Some(GoTlsVersion::Tls11)),
"TLS 1.2" => Ok(Some(GoTlsVersion::Tls12)),
"TLS 1.3" => Ok(Some(GoTlsVersion::Tls13)),
other => Err(TlsPolicyError(format!("invalid TLS version {}", other))),
}
}
fn parse_cipher_suites(
available: &[SupportedCipherSuite],
value: &str,
) -> Result<Vec<SupportedCipherSuite>, TlsPolicyError> {
let trimmed = value.trim();
if trimmed.is_empty() {
return Ok(Vec::new());
}
trimmed
.split(',')
.map(|name| {
let suite = parse_cipher_suite_name(name.trim())?;
available
.iter()
.copied()
.find(|candidate| candidate.suite() == suite)
.ok_or_else(|| {
TlsPolicyError(format!(
"TLS cipher suite '{}' is unsupported by the Rust implementation",
name.trim()
))
})
})
.collect()
}
fn parse_cipher_suite_name(value: &str) -> Result<CipherSuite, TlsPolicyError> {
match value {
"TLS_AES_128_GCM_SHA256" => Ok(CipherSuite::TLS13_AES_128_GCM_SHA256),
"TLS_AES_256_GCM_SHA384" => Ok(CipherSuite::TLS13_AES_256_GCM_SHA384),
"TLS_CHACHA20_POLY1305_SHA256" => Ok(CipherSuite::TLS13_CHACHA20_POLY1305_SHA256),
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256" => {
Ok(CipherSuite::TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256)
}
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384" => {
Ok(CipherSuite::TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384)
}
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256" => {
Ok(CipherSuite::TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256)
}
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" => {
Ok(CipherSuite::TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256)
}
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384" => {
Ok(CipherSuite::TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384)
}
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" => {
Ok(CipherSuite::TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256)
}
other => Err(TlsPolicyError(format!(
"TLS cipher suite '{}' is unsupported by the Rust implementation",
other
))),
}
}
fn parse_common_name(cert: &CertificateDer<'_>) -> Result<String, TlsPolicyError> {
let (_, certificate) = X509Certificate::from_der(cert.as_ref())
.map_err(|e| TlsPolicyError(format!("parse X.509 certificate failed: {}", e)))?;
let common_name = certificate
.subject()
.iter_common_name()
.next()
.and_then(|common_name| common_name.as_str().ok())
.map(str::to_string);
match common_name {
Some(common_name) => Ok(common_name),
None => Ok(String::new()),
}
}
fn common_name_is_allowed(
common_name: &str,
allowed_common_names: &HashSet<String>,
allowed_wildcard_domain: &str,
) -> bool {
(!allowed_wildcard_domain.is_empty() && common_name.ends_with(allowed_wildcard_domain))
|| allowed_common_names.contains(common_name)
}
fn go_tls_version_for_supported(version: &SupportedProtocolVersion) -> GoTlsVersion {
match version.version {
rustls::ProtocolVersion::TLSv1_2 => GoTlsVersion::Tls12,
rustls::ProtocolVersion::TLSv1_3 => GoTlsVersion::Tls13,
_ => unreachable!("rustls only exposes TLS 1.2 and 1.3"),
}
}
#[cfg(test)]
mod tests {
use super::{TlsPolicy, build_supported_versions, common_name_is_allowed, parse_cipher_suites};
use rustls::crypto::aws_lc_rs;
use std::collections::HashSet;
#[test]
fn test_build_supported_versions_defaults_to_tls12_and_tls13() {
let versions = build_supported_versions(&TlsPolicy::default()).unwrap();
assert_eq!(
versions,
vec![&rustls::version::TLS13, &rustls::version::TLS12]
);
}
#[test]
fn test_build_supported_versions_filters_to_tls13() {
let versions = build_supported_versions(&TlsPolicy {
min_version: "TLS 1.3".to_string(),
max_version: "TLS 1.3".to_string(),
cipher_suites: String::new(),
})
.unwrap();
assert_eq!(versions, vec![&rustls::version::TLS13]);
}
#[test]
fn test_build_supported_versions_rejects_unsupported_legacy_range() {
let err = build_supported_versions(&TlsPolicy {
min_version: "TLS 1.0".to_string(),
max_version: "TLS 1.1".to_string(),
cipher_suites: String::new(),
})
.unwrap_err();
assert!(err.to_string().contains("unsupported by rustls"));
}
#[test]
fn test_parse_cipher_suites_accepts_go_names() {
let cipher_suites = parse_cipher_suites(
&aws_lc_rs::default_provider().cipher_suites,
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_128_GCM_SHA256",
)
.unwrap();
assert_eq!(cipher_suites.len(), 2);
}
#[test]
fn test_common_name_is_allowed_matches_exact_and_wildcard() {
let allowed_common_names =
HashSet::from([String::from("volume-a.internal"), String::from("worker-7")]);
assert!(common_name_is_allowed(
"volume-a.internal",
&allowed_common_names,
"",
));
assert!(common_name_is_allowed(
"node.prod.example.com",
&allowed_common_names,
".example.com",
));
assert!(!common_name_is_allowed(
"node.prod.other.net",
&allowed_common_names,
".example.com",
));
}
}