volume: an EC volume needs an .ecx to mount, and a 0-byte stub never outranks a real index (#11415)

* volume: an EC volume needs a non-empty .ecx to mount

Two gaps against Go in how the Rust volume server treats the .ecx.

EcVolume::new mounted with no index at all. The per-shard
VolumeEcShardsMount path picks the disk by shard file alone, so a shard
whose .ecx was on no local directory still registered and was
advertised to the master; every VolumeEcShardRead then failed with
"ecx file not open", and add_shard's 0-byte guard was neutralised
because ecx_file_size stayed 0. Go's NewEcVolume returns an error
wrapping os.ErrNotExist. EcVolume::new now fails with NotFound, and
Store::mount_ec_shard looks up the .ecx owner across all disks first
(findEcxIdxDirForVolume) so a shard on a sibling disk of its index
still mounts instead of turning into a hard failure.

A 0-byte .ecx stub, as left by a failed EC distribute copy, counted as
a valid index. Go requires Size() > 0 wherever the file steers a
decision: HasEcxFileOnDisk, findEcxIdxDirForVolume, indexEcxOwners
(shared by reconcile and mirror), and VolumeEcShardsCopy removes a
copied 0-byte .ecx and fails the copy. Mirror each through one
is_usable_ecx_file helper. NewEcVolume itself still accepts a lone
0-byte .ecx as a legitimate empty index, but prefers a non-empty copy,
local directory first, over a stub in the other directory; the
resolution in EcVolume::new now follows the same order.

Tests that mounted EC volumes without any .ecx get a real fixture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* volume: mount_ec_shard tries every disk; reconcile ignores a 0-byte local .ecx

mount_ec_shard returned the first disk's error, so an unusable shard copy
(a 0-byte .ecNN left by an interrupted move) hid a good copy on the next
disk. Like Go's MountEcShards, keep scanning: NotFound means "not this
disk", any other failure is collected, and an all-disks-fail error names
every disk tried. "No .ecx on any local disk" is now told apart from
"shard not on this server".

The orphan-shard reconcile took its locally-mirrored fast path whenever a
local .ecx existed at all. A 0-byte stub there registered the shards against
an empty index while the owner index skipped that same stub. Go gates the
fast path on HasEcxFileOnDisk; do the same. ec_local_ecx_path loses its last
production caller and becomes test-only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* volume: match Go's mount error text and skip the owner stat on the owning disk

MountEcShards in Go skips the HasEcxFileOnDisk stat when the disk's own
directories already hold the .ecx, dedups a shared -dir.idx across
locations in findEcxIdxDirForVolume, and reports "load failures" with
the same wording. Also drop two issue-number references from comments.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-09-24 07:08:56 +08:00
1 parent c1ccbcda13
commit f0afcf904d
7 files changed
+513 -43

No files matched your search

+12 -1
View File
@@ -3565,7 +3565,18 @@ impl VolumeServer for VolumeGrpcService {
let file = tokio::fs::File::create(&file_path)
.await
.map_err(|e| Status::internal(format!("create {}: {}", file_path, e)))?;
drain_copy_stream_to_file(&mut stream, file, &file_path, ".ecx").await?;
let written = drain_copy_stream_to_file(&mut stream, file, &file_path, ".ecx").await?;
// A source that genuinely holds a 0-byte .ecx would leave a stub
// here that no placement or mount decision accepts. Catch it at
// distribute time, as Go does, so the orchestrator can pick another
// source instead of learning about it at mount.
if written == 0 {
let _ = tokio::fs::remove_file(&file_path).await;
return Err(Status::internal(format!(
"VolumeEcShardsCopy volume {}: source .ecx is 0 bytes",
vid
)));
}
}
// Copy .ecj file if requested
+6
View File
@@ -1849,6 +1849,8 @@ mod tests {
let shard_path = format!("{}/ec_metrics_case_27.ec00", dir);
std::fs::write(&shard_path, b"ec-shard").unwrap();
// An EC volume needs its .ecx to mount.
std::fs::write(format!("{}/ec_metrics_case_27.ecx", dir), [0u8; 16]).unwrap();
store.locations[0]
.mount_ec_shards(VolumeId(27), "ec_metrics_case", &[0], "")
.unwrap();
@@ -1893,6 +1895,8 @@ mod tests {
.unwrap();
std::fs::write(format!("{}/expired_heartbeat_ec_31.ec00", dir), b"expired").unwrap();
// An EC volume needs its .ecx to mount.
std::fs::write(format!("{}/expired_heartbeat_ec_31.ecx", dir), [0u8; 16]).unwrap();
store.locations[0]
.mount_ec_shards(VolumeId(31), "expired_heartbeat_ec", &[0], "")
.unwrap();
@@ -2194,6 +2198,8 @@ mod tests {
let previous = collect_ec_shard_delta_messages(&store);
std::fs::write(format!("{}/ec_delta_case_81.ec00", dir), b"delta").unwrap();
// An EC volume needs its .ecx to mount.
std::fs::write(format!("{}/ec_delta_case_81.ecx", dir), [0u8; 16]).unwrap();
store.locations[0]
.mount_ec_shards(VolumeId(81), "ec_delta_case", &[0], "")
.unwrap();