fix(ec): preserve source disk type across EC encoding (#9423) (#9449)

* fix(ec): carry source disk type on VolumeEcShardsMount (#9423)

When EC shards land on a target whose disk type differs from the
source volume's, master heartbeats wrongly reported under the target
disk's type. Add source_disk_type to VolumeEcShardsMountRequest; the
target server applies it to the in-memory EcVolume via SetDiskType so
the mount notification and steady-state heartbeat both carry the
source's disk type. Empty value falls back to the location's disk
type (used by disk-scan reload paths).

The override is not persisted with the volume — disk type stays an
environmental property and .vif remains portable.

* fix(ec): plumb source disk type through plugin worker (#9423)

Add source_disk_type to ErasureCodingTaskParams (field 8; 7 reserved),
populate it from the metric the detector already collects, thread it
through ec_task into the MountEcShards helper, and forward it on the
VolumeEcShardsMount RPC.

* fix(ec): mirror source disk type plumbing in rust volume server (#9423)

The volume_ec_shards_mount handler now forwards source_disk_type into
mount_ec_shard → DiskLocation::mount_ec_shards. When non-empty it
overrides ec_vol.disk_type (and each mounted shard's disk_type) via
the new set_disk_type method; empty value keeps the location's disk
type, so disk-scan reload and reconcile paths are unchanged.

Also picks up two pre-existing proto drifts that 'make gen' synced
from weed/pb (LockRingUpdate in master.proto, listing_cache_ttl_seconds
in remote.proto).

* feat(ec): bias placement toward preferred disk type (#9423)

Add DiskCandidate.DiskType and PlacementRequest.PreferredDiskType.
When PreferredDiskType is non-empty, SelectDestinations partitions
suitable disks into matching/fallback tiers and runs the rack/server/
disk-diversity passes on the matching tier first; the fallback tier
is only consulted if the matching pool can't satisfy ShardsNeeded.
PlacementResult.SpilledToOtherDiskType lets callers warn on spillover.

Empty PreferredDiskType keeps the existing single-pool behavior.

* fix(ec): plumb source disk type into placement planner (#9423)

diskInfosToCandidates now copies DiskInfo.DiskType into the placement
candidate, and ecPlacementPlanner.selectDestinations forwards
metric.DiskType as PreferredDiskType so EC shards land on disks
matching the source volume's disk type when possible. A glog warning
fires when placement had to spill to other disk types.

* test(ec): integration coverage for source-disk-type plumbing (#9423)

store_ec_disk_type_test exercises Store.MountEcShards end-to-end: a
shard physically lives on an HDD location, MountEcShards is called
with sourceDiskType="ssd", and the test asserts that the in-memory
EcVolume, the mounted shard, the NewEcShardsChan notification, and
the steady-state heartbeat all report under the source's disk type.
A companion test pins the empty-source path so disk-scan reload
keeps the location's disk type.

detection_disk_type_test exercises the worker plumbing: with a
cluster of nodes carrying both HDD and SSD disks, planECDestinations
must place every shard on SSD when metric.DiskType="ssd"; with only
one SSD node and 13 HDD nodes it must still satisfy a 10+4 layout
via spillover (and log a warning).

* revert(ec): drop unrelated proto drift in seaweed-volume/proto (#9423)

make gen pulled two pre-existing OSS changes into the rust proto
tree (LockRingUpdate / by_plugin in master.proto,
listing_cache_ttl_seconds in remote.proto). Reviewers flagged it as
scope creep — none of the rust EC fix references those fields.
Restore both files to origin/master so this branch only touches
EC-related symbols.

* fix(ec placement): treat empty disk type as hdd and skip used racks on spill (#9423)

partitionByDiskType used raw string comparison, so a PreferredDiskType
of "hdd" never matched candidates whose DiskType is "" (the
HardDriveType sentinel that weed/storage/types uses). EC encoding of
an HDD source would spill onto any HDD reporting "" even when the
cluster has plenty of matching capacity. Normalize both sides
through normalizeDiskType, which lowercases and folds "" → "hdd",
mirroring types.ToDiskType without taking a dependency on it.

selectFromTier's rack-diversity pass also kept revisiting racks the
preferred tier had already used when running on the fallback tier,
which negated PreferDifferentRacks on spillover. Skip racks already
in usedRacks so fallback placements still spread onto new racks.

* fix(ec): empty-source remount must not clobber existing disk type (#9423)

mount_ec_shards_with_idx_dir runs more than once per vid (RPC mount,
disk-scan reload, orphan-shard reconcile). After an RPC sets the
source-derived disk type, any later call passing source_disk_type=""
was resetting ec_vol.disk_type back to the location's value, which
reintroduces the heartbeat drift this PR is meant to fix. Only
default to the location's disk type when the EC volume is fresh
(no shards mounted yet); otherwise leave the recorded type alone so
empty-source reloads preserve whatever the original mount RPC set.

weed/worker/tasks/erasure_coding/detection.go

@@ -286,7 +286,7 @@ func Detection(ctx context.Context, metrics []*types.VolumeHealthMetrics, cluste
 					Targets: createECTargets(multiPlan, dataShards, parityShards),
 
 					TaskParams: &worker_pb.TaskParams_ErasureCodingParams{
-						ErasureCodingParams: createECTaskParams(dataShards, parityShards),
+						ErasureCodingParams: createECTaskParams(dataShards, parityShards, metric.DiskType),
 					},
 				}
 
@@ -391,7 +391,7 @@ func newECPlacementPlanner(activeTopology *topology.ActiveTopology, preferredTag
 	}
 }
 
-func (p *ecPlacementPlanner) selectDestinations(sourceRack, sourceDC string, shardsNeeded int) ([]*placement.DiskCandidate, error) {
+func (p *ecPlacementPlanner) selectDestinations(sourceRack, sourceDC, sourceDiskType string, shardsNeeded int) ([]*placement.DiskCandidate, error) {
 	if p == nil || p.activeTopology == nil {
 		return nil, fmt.Errorf("ec placement planner is not initialized")
 	}
@@ -406,6 +406,10 @@ func (p *ecPlacementPlanner) selectDestinations(sourceRack, sourceDC string, sha
 		MaxTaskLoad:            topology.MaxTaskLoadForECPlacement,
 		PreferDifferentServers: true,
 		PreferDifferentRacks:   true,
+		// Bias placement toward disks matching the source volume's disk
+		// type; placement spills to other types only if the preferred
+		// pool can't satisfy ShardsNeeded (#9423).
+		PreferredDiskType: sourceDiskType,
 	}
 
 	var lastErr error
@@ -415,6 +419,10 @@ func (p *ecPlacementPlanner) selectDestinations(sourceRack, sourceDC string, sha
 		}
 		result, err := placement.SelectDestinations(candidates, config)
 		if err == nil {
+			if result.SpilledToOtherDiskType {
+				glog.Warningf("EC placement spilled to disks outside preferred disk type %q to reach %d shards (source rack=%s dc=%s)",
+					sourceDiskType, shardsNeeded, sourceRack, sourceDC)
+			}
 			return result.SelectedDisks, nil
 		}
 		lastErr = err
@@ -633,8 +641,9 @@ func planECDestinations(planner *ecPlacementPlanner, metric *types.VolumeHealthM
 		}
 	}
 
-	// Select best disks for EC placement with rack/DC diversity using the cached planner
-	selectedDisks, err := planner.selectDestinations(sourceRack, sourceDC, totalShards)
+	// Select best disks for EC placement with rack/DC diversity using the cached planner.
+	// Pass source disk type so placement prefers matching-type disks (#9423).
+	selectedDisks, err := planner.selectDestinations(sourceRack, sourceDC, metric.DiskType, totalShards)
 	if err != nil {
 		return nil, err
 	}
@@ -786,11 +795,14 @@ func convertTaskSourcesToProtobuf(sources []topology.TaskSourceSpec, volumeID ui
 	return protobufSources, nil
 }
 
-// createECTaskParams creates clean EC task parameters (destinations now in unified targets)
-func createECTaskParams(dataShards, parityShards int) *worker_pb.ErasureCodingTaskParams {
+// createECTaskParams creates clean EC task parameters (destinations now in unified targets).
+// sourceDiskType is forwarded to VolumeEcShardsMount so the resulting EC volume
+// reports under the source's disk type rather than the target location's (#9423).
+func createECTaskParams(dataShards, parityShards int, sourceDiskType string) *worker_pb.ErasureCodingTaskParams {
 	return &worker_pb.ErasureCodingTaskParams{
-		DataShards:   int32(dataShards),
-		ParityShards: int32(parityShards),
+		DataShards:     int32(dataShards),
+		ParityShards:   int32(parityShards),
+		SourceDiskType: sourceDiskType,
 	}
 }
 
@@ -824,6 +836,7 @@ func diskInfosToCandidates(disks []*topology.DiskInfo) []*placement.DiskCandidat
 			DiskID:         disk.DiskID,
 			DataCenter:     disk.DataCenter,
 			Rack:           disk.Rack,
+			DiskType:       disk.DiskType,
 			VolumeCount:    disk.DiskInfo.VolumeCount,
 			MaxVolumeCount: disk.DiskInfo.MaxVolumeCount,
 			ShardCount:     ecShardCount,

weed/worker/tasks/erasure_coding/detection_disk_type_test.go

@@ -0,0 +1,94 @@
+package erasure_coding
+
+import (
+	"testing"
+
+	"github.com/seaweedfs/seaweedfs/weed/storage/erasure_coding"
+	"github.com/seaweedfs/seaweedfs/weed/worker/types"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+// TestPlanECDestinationsPrefersSourceDiskType_FullCluster covers the
+// #9423 happy path through the worker plumbing: every node carries both
+// an HDD and an SSD disk, so the cluster has enough SSD slots to fully
+// satisfy a 10+4 placement. With metric.DiskType="ssd", every selected
+// destination must land on an SSD disk.
+func TestPlanECDestinationsPrefersSourceDiskType_FullCluster(t *testing.T) {
+	// 14 nodes × 2 disks per node (hdd + ssd) — enough SSD slots alone
+	// for a 10+4 layout with one-shard-per-(server,disk) diversity.
+	activeTopology := buildActiveTopology(t, erasure_coding.TotalShardsCount, []string{"hdd", "ssd"}, 100, 0)
+
+	planner := newECPlacementPlanner(activeTopology, nil)
+	require.NotNil(t, planner)
+
+	metric := &types.VolumeHealthMetrics{
+		VolumeID:   1,
+		Server:     "10.0.0.1:8080",
+		Size:       100 * 1024 * 1024,
+		Collection: "",
+		DiskType:   "ssd", // the property being plumbed end-to-end
+	}
+
+	plan, err := planECDestinations(planner, metric, NewDefaultConfig(), erasure_coding.DataShardsCount, erasure_coding.ParityShardsCount)
+	require.NoError(t, err)
+	require.Len(t, plan.Plans, erasure_coding.TotalShardsCount)
+
+	// buildActiveTopology assigns DiskID by index into the diskTypes
+	// slice, so SSD is DiskID 1. Every plan entry must point at an SSD
+	// disk — anything else means metric.DiskType was dropped on the way
+	// to the placement layer.
+	for _, p := range plan.Plans {
+		assert.Equalf(t, uint32(1), p.TargetDisk,
+			"target %s diskID = %d, want SSD disk (DiskID=1)", p.TargetNode, p.TargetDisk)
+	}
+}
+
+// TestPlanECDestinationsSpillsToOtherDiskType_WhenPreferredScarce pins
+// the prefer-with-spillover policy: only one node in the cluster has an
+// SSD disk; the rest are HDD-only. A 10+4 placement must still succeed
+// (no hard fail), filling the one SSD slot first and the remaining 13
+// from HDD nodes. Without the spillover behavior this would either
+// error out or starve the placement.
+func TestPlanECDestinationsSpillsToOtherDiskType_WhenPreferredScarce(t *testing.T) {
+	// Start with every node carrying both HDD and SSD, then strip SSD
+	// from all but the first node so the SSD pool is too small alone.
+	activeTopology := buildActiveTopology(t, erasure_coding.TotalShardsCount, []string{"hdd", "ssd"}, 100, 0)
+	topo := activeTopology.GetTopologyInfo()
+	for _, dc := range topo.DataCenterInfos {
+		for _, rack := range dc.RackInfos {
+			for i, node := range rack.DataNodeInfos {
+				if i == 0 {
+					continue // keep the first node's SSD
+				}
+				delete(node.DiskInfos, "ssd")
+			}
+		}
+	}
+	require.NoError(t, activeTopology.UpdateTopology(topo))
+
+	planner := newECPlacementPlanner(activeTopology, nil)
+	require.NotNil(t, planner)
+
+	metric := &types.VolumeHealthMetrics{
+		VolumeID:   2,
+		Server:     "10.0.0.1:8080",
+		Size:       100 * 1024 * 1024,
+		Collection: "",
+		DiskType:   "ssd",
+	}
+
+	plan, err := planECDestinations(planner, metric, NewDefaultConfig(), erasure_coding.DataShardsCount, erasure_coding.ParityShardsCount)
+	require.NoError(t, err)
+	require.Len(t, plan.Plans, erasure_coding.TotalShardsCount)
+
+	// Exactly one SSD target (the only SSD node), the rest on HDD.
+	ssdHits := 0
+	for _, p := range plan.Plans {
+		if p.TargetDisk == 1 {
+			ssdHits++
+		}
+	}
+	assert.Equalf(t, 1, ssdHits,
+		"expected exactly 1 SSD placement (the only SSD node in the cluster), got %d", ssdHits)
+}

weed/worker/tasks/erasure_coding/detection_test.go

@@ -84,7 +84,7 @@ func TestECPlacementPlannerPrefersTaggedDisks(t *testing.T) {
 	planner := newECPlacementPlanner(activeTopology, []string{"fast"})
 	require.NotNil(t, planner)
 
-	selected, err := planner.selectDestinations("", "", 2)
+	selected, err := planner.selectDestinations("", "", "", 2)
 	require.NoError(t, err)
 	require.Len(t, selected, 2)
 
@@ -113,7 +113,7 @@ func TestECPlacementPlannerFallsBackWhenTagsInsufficient(t *testing.T) {
 	planner := newECPlacementPlanner(activeTopology, []string{"fast"})
 	require.NotNil(t, planner)
 
-	selected, err := planner.selectDestinations("", "", 3)
+	selected, err := planner.selectDestinations("", "", "", 3)
 	require.NoError(t, err)
 	require.Len(t, selected, 3)
 

weed/worker/tasks/erasure_coding/ec_task.go

@@ -37,6 +37,7 @@ type ErasureCodingTask struct {
 	// EC parameters
 	dataShards      int32
 	parityShards    int32
+	sourceDiskType  string                  // source volume's disk type, forwarded to Mount RPC (#9423)
 	targets         []*worker_pb.TaskTarget // Unified targets for EC shards
 	sources         []*worker_pb.TaskSource // Unified sources for cleanup
 	shardAssignment map[string][]string     // destination -> assigned shard types
@@ -68,6 +69,7 @@ func (t *ErasureCodingTask) Execute(ctx context.Context, params *worker_pb.TaskP
 
 	t.dataShards = ecParams.DataShards
 	t.parityShards = ecParams.ParityShards
+	t.sourceDiskType = ecParams.SourceDiskType
 	t.workDir = ecParams.WorkingDir
 	t.targets = params.Targets // Get unified targets
 	t.sources = params.Sources // Get unified sources
@@ -540,7 +542,7 @@ func (t *ErasureCodingTask) distributeEcShards(shardFiles map[string]string) err
 
 // mountEcShards mounts EC shards on destination servers
 func (t *ErasureCodingTask) mountEcShards() error {
-	return erasure_coding.MountEcShards(t.volumeID, t.collection, t.shardAssignment, t.grpcDialOption, t.GetLogger())
+	return erasure_coding.MountEcShards(t.volumeID, t.collection, t.shardAssignment, t.sourceDiskType, t.grpcDialOption, t.GetLogger())
 }
 
 // deleteOriginalVolume deletes the original volume and all its replicas from all servers

weed/worker/tasks/erasure_coding/plugin_handler.go

@@ -713,6 +713,7 @@ func decodeErasureCodingTaskParams(job *plugin_pb.JobSpec) (*worker_pb.TaskParam
 	if parityShards <= 0 {
 		parityShards = int32(ecstorage.ParityShardsCount)
 	}
+	sourceDiskType := strings.TrimSpace(pluginworker.ReadStringConfig(job.Parameters, "source_disk_type", ""))
 	totalShards := int(dataShards + parityShards)
 
 	if volumeID == 0 {
@@ -758,8 +759,9 @@ func decodeErasureCodingTaskParams(job *plugin_pb.JobSpec) (*worker_pb.TaskParam
 		Targets: targets,
 		TaskParams: &worker_pb.TaskParams_ErasureCodingParams{
 			ErasureCodingParams: &worker_pb.ErasureCodingTaskParams{
-				DataShards:   dataShards,
-				ParityShards: parityShards,
+				DataShards:     dataShards,
+				ParityShards:   parityShards,
+				SourceDiskType: sourceDiskType,
 			},
 		},
 	}, nil