mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-22 05:54:24 +02:00
fix(ec): blanket-clean every destination over the full shard range (#9512)
* fix(ec): blanket-clean every destination over the full shard range The previous cleanup pass walked t.sources only, with the shard ids the topology had reported at detection time. In the wild, a destination can end up with EC shards mounted that the topology snapshot didn't list — shards on a sibling disk that hadn't heartbeated, or shards left over from a concurrent attempt's mount step. FindEcVolume still returns true, so the next ReceiveFile trips the mounted-volume guard. Cleanup now unions t.sources (with ShardIds) and t.targets and issues unmount + delete over [0..totalShards-1] on each. Both RPCs are idempotent on missing shards, so the wider sweep is free. Two new tests cover the gap: shards mounted beyond what t.sources lists, and a target-only destination with no source row. * log(ec): include disk_id in EC unmount/delete/refusal log lines The current logs identify the volume and shard but leave disk_id off, which makes the cross-server cleanup story hard to follow when multiple disks of one server hold pieces of the same volume: UnmountEcShards 4121.1 -> add disk_id ec volume video-recordings_4121 shard delete [1 5] -> add per-loc disk_id volume server X:Y deletes ec shards from 4121 [...] -> add disk_id ReceiveFile: ec volume 4121 is mounted; refusing... -> add disk_ids ReceiveFile's refusal now names the disk_ids actually holding the mount so operators can see whether the next cleanup pass needs to target a sibling disk. Added Store.FindEcVolumeDiskIds / Store::find_ec_volume_disk_ids as the supporting primitive. Mirrored in seaweed-volume/src/ (unmount log in Store::unmount_ec_shard, heartbeat delete log in diff_ec_shard_delta_messages, refusal in the ReceiveFile handler). * test(ec): stub VolumeEcShardsUnmount/Delete on the fake volume server The plugin-worker EC tests boot a fake volume server that embeds UnimplementedVolumeServerServer. After the worker started calling VolumeEcShardsUnmount + VolumeEcShardsDelete pre-distribute, the default Unimplemented response surfaced as fourteen "method not implemented" errors and TestErasureCodingExecutionEncodesShards failed. Both RPCs are no-ops here — nothing on the fake server has mounted state or persisted shard files to remove.
This commit is contained in:
1 parent
bf9110ebd3
commit
2a41e76101
10 files changed
+255
-42
No files matched your search
@@ -1511,9 +1511,11 @@ impl VolumeServer for VolumeGrpcService {
|
||||
// EcVolume holds fds on the same inodes, so overwriting
|
||||
// corrupts live readers.
|
||||
if store.has_ec_volume(VolumeId(info.volume_id)) {
|
||||
let mounted_disks =
|
||||
store.find_ec_volume_disk_ids(VolumeId(info.volume_id));
|
||||
resp_error = Some(format!(
|
||||
"ec volume {} is mounted; unmount before ReceiveFile",
|
||||
info.volume_id
|
||||
"ec volume {} is mounted on disk_ids:{:?}; unmount before ReceiveFile",
|
||||
info.volume_id, mounted_disks
|
||||
));
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -344,6 +344,12 @@ fn diff_ec_shard_delta_messages(
|
||||
if !current.contains_key(key) {
|
||||
let mut deleted = message.clone();
|
||||
deleted.shard_sizes = vec![0];
|
||||
tracing::info!(
|
||||
volume_id = deleted.id,
|
||||
disk_id = deleted.disk_id,
|
||||
ec_index_bits = deleted.ec_index_bits,
|
||||
"deletes ec shards"
|
||||
);
|
||||
deleted_ec_shards.push(deleted);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user