Files
seaweedfs/weed/worker/tasks/base/generic_components.go
T
Junker der Provinz d86c692a39 admin: make the maintenance policy actually reach the task detectors
Loading the persisted task configs into the maintenance policy only
matters if something reads that policy, and nothing did.

MaintenanceIntegration pushes the policy into every registered detector
and scheduler through interface{ SetEnabled(bool) } and
interface{ SetMaxConcurrent(int) } type assertions. Every task registered
through base.RegisterTask is backed by base.GenericDetector and
base.GenericScheduler, and neither implemented either method, so all four
assertions failed silently for every task on every startup. The policy's
enabled flag reached nothing: ScanWithTaskDetectors gates on
detector.IsEnabled(), and the queue's policy lookups for max concurrent
and repeat interval are fallbacks that only fire when the scheduler
reports zero, which the generic scheduler never does.

Add the setters, delegating to the TaskConfig.SetEnabled the interface
already declares and to TaskDefinition.MaxConcurrent, which is what
GetMaxConcurrent returns.

Applying the policy required three more fixes, because with the
assertions working the policy could now do damage as well as good:

- IsTaskEnabled reports false for a task type the policy has no entry
  for, so applying it unconditionally would have disabled every task the
  policy does not list. Skip task types with no policy entry: no entry
  means no opinion, not disabled.

- ec_balance was exactly such a task. It is registered like the other
  three but had no entry in the policy builder and no accessor on
  ConfigPersistence at all, so its configuration could never be
  persisted. Add SaveEcBalanceTaskPolicy/LoadEcBalanceTaskPolicy, the
  task_ec_balance.pb file, the SaveTaskPolicy dispatcher case, and the
  policy entry.

- InitMaintenanceManager ran before loadTaskConfigurationsFromPersistence,
  which replaces each task's whole config object, so the policy was
  applied and then immediately thrown away. Swap the order. Both read the
  same files, so the policy is now the last writer and stays
  authoritative.

MaintenanceManager.UpdateConfig also updated the queue's and the
scanner's policy but not the integration's, so a policy changed at
runtime never reached the detectors. Add MaintenanceIntegration.SetPolicy
and call it.

While building the policy, stop hand-copying each task's fields and use
the task's own ToTaskPolicy(). The hand-written version was a second
definition of every task's policy and had already lost the erasure coding
preferred tags and replica placement and the balance IO rate limit. For
the same reason, the "nothing persisted yet" branches of
LoadVacuumTaskPolicy, LoadErasureCodingTaskPolicy and
LoadBalanceTaskPolicy now derive from each task's NewDefaultConfig()
instead of a third hand-written copy. Those copies had drifted, so with a
data directory but no config file on disk the effective defaults differed
from what the task and the admin UI schema both advertise:

  vacuum          scan interval  24h  -> 2h
  balance         scan interval   6h  -> 30m
  balance         imbalance      0.1  -> 0.2
  erasure coding  scan interval 168h  -> 1h
  erasure coding  fullness      0.90  -> 0.95
  erasure coding  min volume   1024MB -> 30MB

Finally, weed/admin/dash and weed/admin/maintenance each carried a copy
of the policy builder and they had already diverged. Export the
maintenance one as BuildPolicyFromTaskConfigs and have dash call it.

Refs #10874
2026-08-23 05:44:24 +02:00

161 lines
5.0 KiB
Go

package base
import (
"time"
"github.com/seaweedfs/seaweedfs/weed/worker/types"
)
// GenericDetector implements TaskDetector using function-based logic
type GenericDetector struct {
taskDef *TaskDefinition
}
// NewGenericDetector creates a detector from a task definition
func NewGenericDetector(taskDef *TaskDefinition) *GenericDetector {
return &GenericDetector{taskDef: taskDef}
}
// GetTaskType returns the task type
func (d *GenericDetector) GetTaskType() types.TaskType {
return d.taskDef.Type
}
// ScanForTasks scans using the task definition's detection function
func (d *GenericDetector) ScanForTasks(volumeMetrics []*types.VolumeHealthMetrics, clusterInfo *types.ClusterInfo) ([]*types.TaskDetectionResult, error) {
if d.taskDef.DetectionFunc == nil {
return nil, nil
}
return d.taskDef.DetectionFunc(volumeMetrics, clusterInfo, d.taskDef.Config)
}
// ScanInterval returns the scan interval from task definition
func (d *GenericDetector) ScanInterval() time.Duration {
if d.taskDef.ScanInterval > 0 {
return d.taskDef.ScanInterval
}
return 30 * time.Minute // Default
}
// IsEnabled returns whether this detector is enabled
func (d *GenericDetector) IsEnabled() bool {
return d.taskDef.Config.IsEnabled()
}
// SetEnabled turns detection for this task type on or off.
//
// The admin maintenance policy is applied to detectors through an
// interface{ SetEnabled(bool) } type assertion (see
// MaintenanceIntegration.configureDetectorFromPolicy). Every registered task is
// backed by this generic detector, so without this method that assertion failed
// for every task and the policy never reached the flag that
// ScanWithTaskDetectors actually gates on. See issue #10874.
func (d *GenericDetector) SetEnabled(enabled bool) {
d.taskDef.Config.SetEnabled(enabled)
}
// GenericScheduler implements TaskScheduler using function-based logic
type GenericScheduler struct {
taskDef *TaskDefinition
}
// NewGenericScheduler creates a scheduler from a task definition
func NewGenericScheduler(taskDef *TaskDefinition) *GenericScheduler {
return &GenericScheduler{taskDef: taskDef}
}
// GetTaskType returns the task type
func (s *GenericScheduler) GetTaskType() types.TaskType {
return s.taskDef.Type
}
// CanScheduleNow determines if a task can be scheduled using the task definition's function
func (s *GenericScheduler) CanScheduleNow(task *types.TaskInput, runningTasks []*types.TaskInput, availableWorkers []*types.WorkerData) bool {
if s.taskDef.SchedulingFunc == nil {
return s.defaultCanSchedule(task, runningTasks, availableWorkers)
}
return s.taskDef.SchedulingFunc(task, runningTasks, availableWorkers, s.taskDef.Config)
}
// defaultCanSchedule provides default scheduling logic
func (s *GenericScheduler) defaultCanSchedule(task *types.TaskInput, runningTasks []*types.TaskInput, availableWorkers []*types.WorkerData) bool {
if !s.taskDef.Config.IsEnabled() {
return false
}
// Count running tasks of this type
runningCount := 0
for _, runningTask := range runningTasks {
if runningTask.Type == s.taskDef.Type {
runningCount++
}
}
// Check concurrency limit
maxConcurrent := s.taskDef.MaxConcurrent
if maxConcurrent <= 0 {
maxConcurrent = 1 // Default
}
if runningCount >= maxConcurrent {
return false
}
// Check if we have available workers
for _, worker := range availableWorkers {
if worker.CurrentLoad < worker.MaxConcurrent {
for _, capability := range worker.Capabilities {
if capability == s.taskDef.Type {
return true
}
}
}
}
return false
}
// GetPriority returns the priority for this task
func (s *GenericScheduler) GetPriority(task *types.TaskInput) types.TaskPriority {
return task.Priority
}
// GetMaxConcurrent returns max concurrent tasks
func (s *GenericScheduler) GetMaxConcurrent() int {
if s.taskDef.MaxConcurrent > 0 {
return s.taskDef.MaxConcurrent
}
return 1 // Default
}
// GetDefaultRepeatInterval returns the default repeat interval
func (s *GenericScheduler) GetDefaultRepeatInterval() time.Duration {
if s.taskDef.RepeatInterval > 0 {
return s.taskDef.RepeatInterval
}
return 24 * time.Hour // Default
}
// IsEnabled returns whether this scheduler is enabled
func (s *GenericScheduler) IsEnabled() bool {
return s.taskDef.Config.IsEnabled()
}
// SetEnabled turns scheduling for this task type on or off. Detector and scheduler
// share one TaskDefinition, so this is the same flag GenericDetector.SetEnabled sets;
// both setters exist because the maintenance integration configures the two
// independently. See GenericDetector.SetEnabled and issue #10874.
func (s *GenericScheduler) SetEnabled(enabled bool) {
s.taskDef.Config.SetEnabled(enabled)
}
// SetMaxConcurrent applies the policy's concurrency limit for this task type. It is
// the value GetMaxConcurrent returns, which the maintenance queue uses to decide
// whether another task of this type may start. Non-positive limits are ignored
// rather than turning into the implicit default of 1.
func (s *GenericScheduler) SetMaxConcurrent(maxConcurrent int) {
if maxConcurrent <= 0 {
return
}
s.taskDef.MaxConcurrent = maxConcurrent
}