mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-14 10:08:35 +02:00
Чепурнов СемёнandChepurnov Simon
624a13ecc6
* feat: prefer same-host worker and local EC replica Admin executor pick ranked idle workers only, so a job whose volume lived on full-03 could still run on full-01 and pull the whole .dat across the LAN. Rank workers whose Hello host matches any replica first (fallback to others if the local worker is full). Detection named Sources[0] / the lowest server id; execute then SyncAndSelectBestReplica and could switch the copy source to another rack. After sync, if a replica on this machine has a complete file count, copy from that URL so affinity and pull stay on one host. * feat: pin scheduler jobs to replica-host workers until load K Reserve prefers the least-loaded replica-host worker and waits while that load is below 5 instead of spilling to a free remote worker. K=0 restores immediate spill. Waterfill uses reservation load so four idle preferred workers each get a job. * feat: park scheduled jobs on per-worker admin queues Assign a detection batch to the least-loaded replica-host worker in one pass, then let ExecutionConcurrency goroutines pop a home-runnable job or steal when that home queue reaches K. Dispatch no longer holds a job in reserve-backoff, so a free worker is not blocked by another home's pin. Tracked jobs stay pending/queued and record the home worker for the admin UI. * fix: rehome queued jobs when the home worker dies A live home still pins below K. A home missing from live executors is stolen immediately onto a free worker so a short orphaned queue does not sit until the dispatch window ends. * feat: wait before stealing a dead home worker queue Orphaned queues are held for 30s so a worker reconnect blink does not dump jobs onto other machines. After the grace, steal without K as before. Zero disables the wait. A home that comes back during grace keeps its queue. * fix: late-bind EC from to worker hostname After replicas are consistent (equal FileCount), rebind encode source to a volume server on the same hostname as worker -address. Any port on that host is local. Diverged FileCount still uses the max replica. * feat: assign homeless jobs to global FIFO Jobs whose replica hostname has no registered worker skip pin and orphan grace. Free workers take own home, then steal at K/orphan, then leftover FIFO. * fix: take homeless FIFO before steal for no-home workers Workers without a replica-host match take leftover first; steal (K/orphan) only when FIFO is empty. Home workers keep own-queue, steal, then leftover. --------- Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
25 lines
702 B
Go
25 lines
702 B
Go
package util
|
|
|
|
import "testing"
|
|
|
|
func TestHostKeyStripsPort(t *testing.T) {
|
|
t.Parallel()
|
|
if got := HostKey("10.87.0.3:8080"); got != "10.87.0.3" {
|
|
t.Fatalf("HostKey replica: got %q", got)
|
|
}
|
|
if got := HostKey("10.87.0.3:33646"); got != "10.87.0.3" {
|
|
t.Fatalf("HostKey worker: got %q", got)
|
|
}
|
|
if HostKey("10.87.0.3:8080") != HostKey("10.87.0.3:33646") {
|
|
t.Fatal("worker and volume server on the same IP must share a HostKey")
|
|
}
|
|
}
|
|
|
|
func TestHostKeysFromAddressesDedups(t *testing.T) {
|
|
t.Parallel()
|
|
got := HostKeysFromAddresses([]string{"10.0.0.1:8080", "10.0.0.1:8081", "10.0.0.2:8080", ""})
|
|
if len(got) != 2 || got[0] != "10.0.0.1" || got[1] != "10.0.0.2" {
|
|
t.Fatalf("got %v", got)
|
|
}
|
|
}
|