mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-18 17:55:26 +01:00
ecbalancer: honour total-shards-per-rack cap in Place / PlaceDurabilityFirst (#11553)
* ecbalancer: honour total-shards-per-rack cap in Place / PlaceDurabilityFirst
Worker auto-EC encode places via Topology.Place, which capped each shard
type independently (ceil(data/racks), ceil(parity/racks)). On an 8-rack
topology that permits 3 total shards on one rack, so losing two racks
strands 6/14 and a 10+4 volume becomes unreadable.
- tryPlace caps the total shards (data + parity) per rack in both modes,
whether or not ReplicaPlacement is set.
- rackTotalCap picks the smallest per-rack total the racks' real room
(free slots, bounded by the per-disk cap and node free slots, counting
shards already placed) can satisfy. On a uniform cluster it is
ceil(shards/racks); a nearly full rack raises it just enough that the
cap alone never fails an encode.
- PlaceDurabilityFirst gets a last rung that drops the rack cap
("rack-total-cap" in Relaxed), so it fails only when no disk has room.
PlaceStrict keeps the cap as a hard limit.
- chooseShardDest tries the next rack when the chosen one has no node
that fits, and room checks count the per-disk cap, so a rack whose
disks are all at the cap is no longer picked and then failed on
(pre-existing: 3-node rack + single-disk rack failed at shard 9).
- Docs no longer claim the cap guarantees surviving rack loss; the
placement error names the caps in effect; the encode warning no longer
says replica placement when other constraints were relaxed.
place_rack_cap_test.go covers 10+4 over 8 racks (max 2/rack, 3/rack on
master), a starved rack, nearly full racks, the preferred-tag tier, the
full-disk rack, and rackTotalCap directly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* ecbalancer: size the rack total cap from room left under SameRackCount
The rack total cap counted each rack's free disk room, but attempts that
enforce ReplicaPlacement also stop a node at SameRackCount shards. With
SameRackCount=1, four one-node racks and four three-node racks got cap 2,
which fits only 12 of 14 shards: strict placement failed and
durability-first relaxed replica placement although 1 per small rack and
up to 3 per large rack fits.
Attempts that enforce ReplicaPlacement now use a cap sized from each
node's remaining SameRackCount allowance; attempts that relax it keep the
disk-room cap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
164c3db606
commit
2a42d56437
3 files changed
+455
-65
No files matched your search
@@ -485,9 +485,10 @@ func buildNodeAddressMap(at *topology.ActiveTopology) map[string]string {
|
||||
// in the same detection cycle see the reduced capacity. Rebuilding it per volume
|
||||
// is O(volumes × topology) and times out on large clusters.
|
||||
//
|
||||
// Encode is lenient (PlaceDurabilityFirst): it relaxes caps/anti-affinity/RP as
|
||||
// needed rather than fail, and prefers the source disk type but spills if that
|
||||
// type can't hold every shard. rp is the resolved replica placement (may be nil).
|
||||
// Encode is lenient (PlaceDurabilityFirst): it relaxes caps/anti-affinity/RP and,
|
||||
// last, the total-shards-per-rack cap as needed, failing only when no eligible
|
||||
// disk has room. It prefers the source disk type but spills if that type can't
|
||||
// hold every shard. rp is the resolved replica placement (may be nil).
|
||||
func planECDestinations(snap *ecbalancer.Topology, nodeAddresses map[string]string, metric *types.VolumeHealthMetrics, ecConfig *Config, rp *super_block.ReplicaPlacement, dataShards, parityShards int) (*topology.MultiDestinationPlan, [][]uint32, error) {
|
||||
if snap == nil {
|
||||
return nil, nil, fmt.Errorf("EC placement snapshot not available")
|
||||
@@ -529,8 +530,8 @@ func planECDestinations(snap *ecbalancer.Topology, nodeAddresses map[string]stri
|
||||
if len(res.Relaxed) > 0 {
|
||||
// Encode is best-effort (PlaceDurabilityFirst): it relaxes these constraints
|
||||
// rather than defer when the cluster can't satisfy them. Surface it so a tight
|
||||
// replica placement isn't silently weakened; rebalancing tightens the spread.
|
||||
glog.Warningf("EC volume %d: placed with relaxed constraints %v; replica placement not fully satisfied (rebalancing will adjust)", metric.VolumeID, res.Relaxed)
|
||||
// replica placement or rack spread isn't silently weakened.
|
||||
glog.Warningf("EC volume %d: placed with relaxed placement constraints %v", metric.VolumeID, res.Relaxed)
|
||||
}
|
||||
|
||||
// Group the per-shard destinations into one plan per (node,disk), iterating
|
||||
|
||||
Reference in new issue
Block a user