fix(ec): persist EC source readonly mark and skip writable replicas on orphan cleanup (#9950)

* fix(ec): persist the EC source replica readonly mark

markReplicasReadonly marked each regular replica readonly without
persisting it, so a source-server restart during or after encoding
silently reopened the volume to writes. Those writes are not in the EC
shards, and the later orphan-source cleanup would then delete the
replica, losing them. Send Persist:true so the mark survives a restart;
rollbackReadonly still clears it via VolumeMarkWritable on a failed
encode.

* fix(ec): don't delete a writable source replica during orphan cleanup

cleanupOrphanSourceReplicas issued VolumeDelete to every regular replica
once the EC shard set looked complete, without checking the replica's
current state. A replica that came back writable may hold writes the EC
shards do not contain, so deleting it loses data. Re-probe each replica
via VolumeStatus and skip any that is no longer readonly, logging a
warning instead of deleting.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-06-13 21:26:16 -07:00
1 parent 1e858d8af0
commit 240f82d6d2
4 files changed
+157 -1

No files matched your search

+4 -1
View File
@@ -342,7 +342,10 @@ func (t *ErasureCodingTask) markReplicasReadonly(ctx context.Context) error {
addr := loc.ServerAddress()
err := operation.WithVolumeServerClient(false, addr, t.grpcDialOption,
func(client volume_server_pb.VolumeServerClient) error {
_, e := client.VolumeMarkReadonly(ctx, &volume_server_pb.VolumeMarkReadonlyRequest{VolumeId: t.volumeID})
// Persist the readonly mark so a source-server restart during or
// after encoding cannot silently reopen the volume to writes that
// the EC shards would not contain. rollbackReadonly clears it.
_, e := client.VolumeMarkReadonly(ctx, &volume_server_pb.VolumeMarkReadonlyRequest{VolumeId: t.volumeID, Persist: true})
return e
})
if err != nil {