mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-23 06:20:48 +02:00
Chris LuandDevin
137f23a229
* ec distribution: parse EC and replication configs into a shard plan NewECConfig/DefaultECConfig and NewReplicationConfig(FromString) turn an EC ratio and a replica placement into ECConfig/ReplicationConfig values, and CalculateDistribution derives the target and maximum shard counts per data center, rack, and node. ECDistribution.Summary and FaultToleranceAnalysis describe, per topology level, whether losing one failure domain leaves enough shards to reconstruct the volume, so the EC durability guarantees of a given setup can be answered directly instead of only documented informally. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: analyze topology and plan shard moves TopologyAnalysis records nodes and shard locations per DC/rack/node, and Rebalancer.PlanRebalance emits the moves that level the layout: data centers first, then racks within each DC, then nodes within each rack, moving parity shards before data shards and honoring per-node free slots. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: cover the plan math and rebalancing with tests Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: keep AddShardLocation and node ShardIDs in sync AddShardLocation updated the count maps but never the node's ShardIDs — yet every rebalancer count reads len(node.ShardIDs), so the documented AddNode-then-AddShardLocation flow produced an empty rebalance plan. It now appends to the node's shard list and skips shards the node already has, making the two registration styles equivalent and idempotent. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: key rack-scoped maps by dc/rack Rack names are only unique within a data center — two DCs can both hold a "rack1" — so ShardsByRack, RackToShards, and RackToNodes keyed by the bare rack name merged distinct domains and corrupted both the DC-level shard counts and rack-level balancing. Rack lookups now use the dc/rack pair; DCToRacks still lists bare rack names. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: never move a shard onto a node already holding it The move picker selected the destination node before the shard, so a node already holding the chosen shard could be targeted. Pick the shard first (parity-first) and choose among destination nodes that lack it. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: tolerate zero-value replication configs A ReplicationConfig{} with zeroed minimums made maxShare divide by zero; clamp each level's unit count to at least one. Covers the new behaviors with tests. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: keep rack keys distinct when names contain "/" rackKey joined data center and rack with "/", so dc "a"/rack "b/c" and dc "a/b"/rack "c" shared one key and were counted as the same failure domain. Separate the parts with a NUL byte, which cannot appear in a name. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: plan rebalances on a copy of the topology recordMove mutated the caller's TopologyAnalysis while planning, so an interrupted or partial execution left the input describing moves that never happened, and replanning on it returned nothing. PlanRebalance now simulates on a cloned analysis; plan.Analysis reports the layout the moves would produce and the caller's input is left untouched. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: balance nodes against the rack's current shard count The node-level target still read the original analysis, so a rack that had just sent or received shards during DC/rack balancing aimed at a stale total and left its nodes unbalanced. * ec distribution: register each node id once Adding the same node twice appended it to RackToNodes twice, so every count that walks rack nodes double-counted its shards. * ec distribution: normalize replication counts in the stored plan A zero-value ReplicationConfig computed targets against one domain per level while Summary still reported zero DCs, racks, and nodes. * ec distribution: cap a survivable level's max at the loss budget * ec distribution: skip shards already inside the destination domain * ec distribution: register nodes implied by shard locations * ec distribution: merge repeat node registrations A shard location now auto-registers an unknown node with no free slots; a later AddNode carrying the real capacity was then dropped entirely. Repeat registration now updates the node's capacity and adopts any new shards, while the first registration keeps the topology placement. * ec distribution: try other overloaded sources before stopping When the most overloaded domain could not shed any shard, balance returned early and skipped every other source that still had movable shards. A source that cannot move now gets set aside for the rest of the pass. * ec distribution: collapse chained shard moves to the final destination Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ec distribution: schedule collapsed moves behind the departures that free slots Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>