fix(volume_server): pin EC shard auto-select to the .ecx-owning disk (#9212) (#9245)

* fix(volume_server): pin EC shard auto-select to the .ecx-owning disk (#9212)

ec.rebuild only sets CopyEcxFile=true on the first shard sent to the
rebuilder; subsequent shards rely on VolumeEcShardsCopy / ReceiveFile
auto-select to land on the same disk. The old auto-select used
FindEcVolume (in-memory) to detect the "already has this volume" case.
Mid-rebuild, no EC volume has been mounted yet on the destination, so
FindEcVolume returns nothing and the fallback picks "any HDD with free
space" — which can split shards from their .ecx across disks of the
same node and feed the orphan-shard layout reported in #9212 / fixed
on the loader side in #9244.

Add Store.FindEcShardTargetLocation as the canonical placement
primitive: prefer a mounted EC volume, then a disk that has the .ecx
on disk, then any HDD, then any disk. DiskLocation.HasEcxFileOnDisk is
the new on-disk check, and it looks at IdxDirectory first with a
fallback to Directory to handle .ecx written before -dir.idx was
configured.

Both VolumeEcShardsCopy and ReceiveFile now route through the new
helper, dropping their duplicated 4-level fallback ladder. No protocol
changes; explicit DiskId callers are unaffected.

* fix(volume_server): treat directories named *.ecx as no-match in HasEcxFileOnDisk

os.Stat(".ecx") succeeds for both files and directories. If something
happens to leave a directory named X.ecx in the data or idx folder,
HasEcxFileOnDisk would currently report true and FindEcShardTargetLocation
would route shards to that disk — where NewEcVolume's eventual
OpenFile(O_RDWR) on the same path errors out.

Add a !info.IsDir() check on both stat sites. Cheap and conservative.

Suggested in PR #9245 review by @gemini-code-assist.

* refactor(volume_server): collapse EC placement helper to a single pass

FindEcShardTargetLocation called FindFreeLocation up to four times. Each
call iterates s.Locations and acquires VolumesLen / EcShardCount RLocks
per disk — for a typical 4-disk node that's 32 RLock cycles per
placement decision.

Walk s.Locations once, score each disk by tier (mounted > .ecx-on-disk
> HDD > any-disk), break ties by free count. The free-slot math is
factored into a small helper that mirrors FindFreeLocation's formula
without re-entering the location's locks. Behaviour is unchanged: each
existing tier still wins over later tiers, and within a tier the disk
with the most free count still wins, matching the original max-tracking
in FindFreeLocation.

Suggested in PR #9245 review by @gemini-code-assist.

* refactor(volume_server): thread dataShardCount as a parameter through EC placement

ecFreeShardCount and FindEcShardTargetLocation referenced
erasure_coding.DataShardsCount directly. Take it as a parameter so
custom-ratio builds (e.g. enterprise) can swap the default without
touching the helper itself, and so unit tests can pin a specific ratio
independent of the package constant. Default callsites in
VolumeEcShardsCopy and ReceiveFile now pass the package default
explicitly; tests pass a literal 10 for clarity.

* fix(volume_server): treat MaxVolumeCount=0 as unlimited in EC placement

ecFreeShardCount computed `MaxVolumeCount - VolumesLen()` and went
negative when MaxVolumeCount was 0 — the "unlimited disk" sentinel
already honoured by Store.hasFreeDiskLocation and friends. With a
negative free count, FindEcShardTargetLocation's `freeCount <= 0`
guard skipped the disk entirely, so unlimited disks could never receive
EC shards via the placement helper.

Special-case MaxVolumeCount<=0: report a synthetic large free count
that decrements with current usage, so unlimited disks are eligible
and tie-breaks still prefer the less-loaded one. Added
TestFindEcShardTargetLocation_HonoursUnlimitedDisk as the regression.

Reported in PR #9245 review by @gemini-code-assist.

* fix(volume_server): account in shard slots, not volume slots, in ecFreeShardCount

FindFreeLocation in store.go ends with `free /= DataShardsCount`,
converting "shard slots free" back to "volume-equivalent slots." The
truncation is harmless there, but my new ecFreeShardCount inherited
the same final divide and re-introduced exactly the orphan-shard
hazard #9245 was meant to prevent: with MaxVolumeCount=1,
VolumesLen=0, EcShardCount=1 the formula reports 0 even though the
disk has room for 9 more shards, so subsequent shards route off the
.ecx-owning disk into the HDD-fallback tier.

Drop the trailing divide and return the count directly in shard slots.
Same shape, finer granularity; tie-breaks still order by free count.
The unlimited branch's "used" calculation is updated to match (mix
volume-slots and shard-slots in shard units). Added
TestFindEcShardTargetLocation_TightProvisioningKeepsEcxDisk as the
regression.

Reported in PR #9245 review by @coderabbitai.

weed/server/volume_grpc_copy.go

@@ -572,8 +572,12 @@ func (vs *VolumeServer) ReceiveFile(stream volume_server_pb.VolumeServer_Receive
 				}
 
 				// disk_id=0 means "unset" (protobuf default), so auto-select
-				// mirrors VolumeEcShardsCopy: prefer a disk already holding
-				// this volume's shards, then any HDD, then any disk.
+				// using the same primitive as VolumeEcShardsCopy: prefer a
+				// disk that has the EC volume mounted, then a disk that owns
+				// the .ecx on disk (the volume hasn't been mounted yet —
+				// relevant when shards stream in mid-rebuild before any
+				// mount has happened; see #9212), then any HDD, then any
+				// disk.
 				var targetLocation *storage.DiskLocation
 				if fileInfo.DiskId > 0 {
 					if fileInfo.DiskId >= uint32(len(vs.store.Locations)) {
@@ -584,20 +588,10 @@ func (vs *VolumeServer) ReceiveFile(stream volume_server_pb.VolumeServer_Receive
 					}
 					targetLocation = vs.store.Locations[fileInfo.DiskId]
 				} else {
-					targetLocation = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-						_, found := loc.FindEcVolume(needle.VolumeId(fileInfo.VolumeId))
-						return found
-					})
-					if targetLocation == nil {
-						targetLocation = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-							return loc.DiskType == types.HardDriveType
-						})
-					}
-					if targetLocation == nil {
-						targetLocation = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-							return true
-						})
-					}
+					// Pass the build's default data-shard count for the helper's
+					// free-slot maths; it's a parameter so custom-ratio builds
+					// (e.g. enterprise) can swap it without touching this file.
+					targetLocation = vs.store.FindEcShardTargetLocation(fileInfo.Collection, needle.VolumeId(fileInfo.VolumeId), erasure_coding.DataShardsCount)
 				}
 				if targetLocation == nil {
 					glog.Errorf("ReceiveFile: no storage location available")

weed/server/volume_grpc_erasure_coding.go

@@ -257,24 +257,15 @@ func (vs *VolumeServer) VolumeEcShardsCopy(ctx context.Context, req *volume_serv
 		location = vs.store.Locations[req.DiskId]
 		glog.V(1).Infof("Using disk %d for EC shard copy: %s", req.DiskId, location.Directory)
 	} else {
-		// Prefer a location that already has shards for this volume,
-		// so all shards end up on the same disk for rebuild.
-		location = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-			_, found := loc.FindEcVolume(needle.VolumeId(req.VolumeId))
-			return found
-		})
-		if location == nil {
-			// Fall back to any HDD location with free space
-			location = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-				return loc.DiskType == types.HardDriveType
-			})
-		}
-		if location == nil {
-			// Fall back to any location with free space
-			location = vs.store.FindFreeLocation(func(loc *storage.DiskLocation) bool {
-				return true
-			})
-		}
+		// Auto-select the target disk: prefer a disk that already has the
+		// EC volume mounted, then a disk that owns the .ecx on disk (the
+		// volume hasn't been mounted yet — relevant for ec.rebuild, where
+		// only the first shard carries .ecx and subsequent shards must
+		// land on the same disk; see #9212), then any HDD, then any disk.
+		// Pass the build's default data-shard count for free-slot maths;
+		// the helper takes it as a parameter so custom-ratio builds (e.g.
+		// enterprise) can swap it without touching this file.
+		location = vs.store.FindEcShardTargetLocation(req.Collection, needle.VolumeId(req.VolumeId), erasure_coding.DataShardsCount)
 		if location == nil {
 			return nil, fmt.Errorf("no space left")
 		}