Files
seaweedfs/weed/plugin/worker/handler_registry.go
T
Чепурнов СемёнandChepurnov Simon 624a13ecc6 EC worker: pin jobs to replica-host workers, late-bind local source (#11718)
* feat: prefer same-host worker and local EC replica

Admin executor pick ranked idle workers only, so a job whose volume
lived on full-03 could still run on full-01 and pull the whole .dat
across the LAN. Rank workers whose Hello host matches any replica
first (fallback to others if the local worker is full).

Detection named Sources[0] / the lowest server id; execute then
SyncAndSelectBestReplica and could switch the copy source to another
rack. After sync, if a replica on this machine has a complete file
count, copy from that URL so affinity and pull stay on one host.

* feat: pin scheduler jobs to replica-host workers until load K

Reserve prefers the least-loaded replica-host worker and waits while
that load is below 5 instead of spilling to a free remote worker.
K=0 restores immediate spill. Waterfill uses reservation load so four
idle preferred workers each get a job.

* feat: park scheduled jobs on per-worker admin queues

Assign a detection batch to the least-loaded replica-host worker in
one pass, then let ExecutionConcurrency goroutines pop a home-runnable
job or steal when that home queue reaches K. Dispatch no longer holds
a job in reserve-backoff, so a free worker is not blocked by another
home's pin. Tracked jobs stay pending/queued and record the home
worker for the admin UI.

* fix: rehome queued jobs when the home worker dies

A live home still pins below K. A home missing from live executors
is stolen immediately onto a free worker so a short orphaned queue
does not sit until the dispatch window ends.

* feat: wait before stealing a dead home worker queue

Orphaned queues are held for 30s so a worker reconnect blink does
not dump jobs onto other machines. After the grace, steal without K
as before. Zero disables the wait. A home that comes back during
grace keeps its queue.

* fix: late-bind EC from to worker hostname

After replicas are consistent (equal FileCount), rebind encode source
to a volume server on the same hostname as worker -address. Any port
on that host is local. Diverged FileCount still uses the max replica.

* feat: assign homeless jobs to global FIFO

Jobs whose replica hostname has no registered worker skip pin and orphan grace.
Free workers take own home, then steal at K/orphan, then leftover FIFO.

* fix: take homeless FIFO before steal for no-home workers

Workers without a replica-host match take leftover first; steal (K/orphan)
only when FIFO is empty. Home workers keep own-queue, steal, then leftover.

---------

Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
2026-10-13 05:31:34 +08:00

134 lines
3.6 KiB
Go

package pluginworker
import (
"fmt"
"strings"
"sync"
"google.golang.org/grpc"
)
// JobCategory groups job types by resource profile so that workers can be
// configured with a category name instead of an explicit list of job types.
type JobCategory string
const (
CategoryAll JobCategory = "all" // pseudo-category matching every handler
CategoryDefault JobCategory = "default" // lightweight, safe for any worker
CategoryHeavy JobCategory = "heavy" // resource-intensive jobs
)
// HandlerFactory describes how to build a JobHandler for a single job type.
type HandlerFactory struct {
// JobType is the canonical job type string (e.g. "vacuum").
JobType string
// Category controls which category label selects this handler.
Category JobCategory
// Aliases are alternative CLI names that resolve to this job type
// (e.g. "ec" for "erasure_coding").
Aliases []string
// Build constructs the JobHandler.
Build func(opts HandlerBuildOptions) (JobHandler, error)
}
// HandlerBuildOptions carries parameters forwarded from the CLI to handler
// constructors.
type HandlerBuildOptions struct {
GrpcDialOption grpc.DialOption
MaxExecute int
WorkingDir string
// WorkerAddress is the advertised weed worker -address (host:port).
// It is not a volume-server URL; handlers match replicas by hostname.
WorkerAddress string
}
var (
registryMu sync.Mutex
registry []HandlerFactory
)
// RegisterHandler adds a handler factory to the global registry.
// It is intended to be called from handler init() functions.
func RegisterHandler(f HandlerFactory) {
registryMu.Lock()
defer registryMu.Unlock()
registry = append(registry, f)
}
// ResolveHandlerFactories takes a comma-separated token list that can contain
// category names ("all", "default", "heavy") and/or explicit job type names
// (including aliases). It returns a deduplicated, ordered slice of factories.
func ResolveHandlerFactories(tokens string) ([]HandlerFactory, error) {
registryMu.Lock()
snapshot := make([]HandlerFactory, len(registry))
copy(snapshot, registry)
registryMu.Unlock()
parts := strings.Split(tokens, ",")
result := make([]HandlerFactory, 0, len(snapshot))
seen := make(map[string]bool)
for _, raw := range parts {
tok := strings.ToLower(strings.TrimSpace(raw))
if tok == "" {
continue
}
if cat, ok := tokenAsCategory(tok); ok {
for _, f := range snapshot {
if cat == CategoryAll || f.Category == cat {
if !seen[f.JobType] {
seen[f.JobType] = true
result = append(result, f)
}
}
}
continue
}
f, err := findFactory(snapshot, tok)
if err != nil {
return nil, err
}
if !seen[f.JobType] {
seen[f.JobType] = true
result = append(result, f)
}
}
if len(result) == 0 {
return nil, fmt.Errorf("no job types resolved from %q", tokens)
}
return result, nil
}
// tokenAsCategory returns the category and true when tok is a known category
// keyword. "all" is treated as a special pseudo-category that matches every
// registered handler.
func tokenAsCategory(tok string) (JobCategory, bool) {
switch tok {
case string(CategoryAll):
return CategoryAll, true
case string(CategoryDefault):
return CategoryDefault, true
case string(CategoryHeavy):
return CategoryHeavy, true
default:
return "", false
}
}
func findFactory(factories []HandlerFactory, tok string) (HandlerFactory, error) {
for _, f := range factories {
if strings.EqualFold(f.JobType, tok) {
return f, nil
}
for _, alias := range f.Aliases {
if strings.EqualFold(alias, tok) {
return f, nil
}
}
}
return HandlerFactory{}, fmt.Errorf("unknown job type %q", tok)
}