mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-13 17:48:29 +02:00
Loading the persisted task configs into the maintenance policy only
matters if something reads that policy, and nothing did.
MaintenanceIntegration pushes the policy into every registered detector
and scheduler through interface{ SetEnabled(bool) } and
interface{ SetMaxConcurrent(int) } type assertions. Every task registered
through base.RegisterTask is backed by base.GenericDetector and
base.GenericScheduler, and neither implemented either method, so all four
assertions failed silently for every task on every startup. The policy's
enabled flag reached nothing: ScanWithTaskDetectors gates on
detector.IsEnabled(), and the queue's policy lookups for max concurrent
and repeat interval are fallbacks that only fire when the scheduler
reports zero, which the generic scheduler never does.
Add the setters, delegating to the TaskConfig.SetEnabled the interface
already declares and to TaskDefinition.MaxConcurrent, which is what
GetMaxConcurrent returns.
Applying the policy required three more fixes, because with the
assertions working the policy could now do damage as well as good:
- IsTaskEnabled reports false for a task type the policy has no entry
for, so applying it unconditionally would have disabled every task the
policy does not list. Skip task types with no policy entry: no entry
means no opinion, not disabled.
- ec_balance was exactly such a task. It is registered like the other
three but had no entry in the policy builder and no accessor on
ConfigPersistence at all, so its configuration could never be
persisted. Add SaveEcBalanceTaskPolicy/LoadEcBalanceTaskPolicy, the
task_ec_balance.pb file, the SaveTaskPolicy dispatcher case, and the
policy entry.
- InitMaintenanceManager ran before loadTaskConfigurationsFromPersistence,
which replaces each task's whole config object, so the policy was
applied and then immediately thrown away. Swap the order. Both read the
same files, so the policy is now the last writer and stays
authoritative.
MaintenanceManager.UpdateConfig also updated the queue's and the
scanner's policy but not the integration's, so a policy changed at
runtime never reached the detectors. Add MaintenanceIntegration.SetPolicy
and call it.
While building the policy, stop hand-copying each task's fields and use
the task's own ToTaskPolicy(). The hand-written version was a second
definition of every task's policy and had already lost the erasure coding
preferred tags and replica placement and the balance IO rate limit. For
the same reason, the "nothing persisted yet" branches of
LoadVacuumTaskPolicy, LoadErasureCodingTaskPolicy and
LoadBalanceTaskPolicy now derive from each task's NewDefaultConfig()
instead of a third hand-written copy. Those copies had drifted, so with a
data directory but no config file on disk the effective defaults differed
from what the task and the admin UI schema both advertise:
vacuum scan interval 24h -> 2h
balance scan interval 6h -> 30m
balance imbalance 0.1 -> 0.2
erasure coding scan interval 168h -> 1h
erasure coding fullness 0.90 -> 0.95
erasure coding min volume 1024MB -> 30MB
Finally, weed/admin/dash and weed/admin/maintenance each carried a copy
of the policy builder and they had already diverged. Export the
maintenance one as BuildPolicyFromTaskConfigs and have dash call it.
Refs #10874
161 lines
5.0 KiB
Go
161 lines
5.0 KiB
Go
package base
|
|
|
|
import (
|
|
"time"
|
|
|
|
"github.com/seaweedfs/seaweedfs/weed/worker/types"
|
|
)
|
|
|
|
// GenericDetector implements TaskDetector using function-based logic
|
|
type GenericDetector struct {
|
|
taskDef *TaskDefinition
|
|
}
|
|
|
|
// NewGenericDetector creates a detector from a task definition
|
|
func NewGenericDetector(taskDef *TaskDefinition) *GenericDetector {
|
|
return &GenericDetector{taskDef: taskDef}
|
|
}
|
|
|
|
// GetTaskType returns the task type
|
|
func (d *GenericDetector) GetTaskType() types.TaskType {
|
|
return d.taskDef.Type
|
|
}
|
|
|
|
// ScanForTasks scans using the task definition's detection function
|
|
func (d *GenericDetector) ScanForTasks(volumeMetrics []*types.VolumeHealthMetrics, clusterInfo *types.ClusterInfo) ([]*types.TaskDetectionResult, error) {
|
|
if d.taskDef.DetectionFunc == nil {
|
|
return nil, nil
|
|
}
|
|
return d.taskDef.DetectionFunc(volumeMetrics, clusterInfo, d.taskDef.Config)
|
|
}
|
|
|
|
// ScanInterval returns the scan interval from task definition
|
|
func (d *GenericDetector) ScanInterval() time.Duration {
|
|
if d.taskDef.ScanInterval > 0 {
|
|
return d.taskDef.ScanInterval
|
|
}
|
|
return 30 * time.Minute // Default
|
|
}
|
|
|
|
// IsEnabled returns whether this detector is enabled
|
|
func (d *GenericDetector) IsEnabled() bool {
|
|
return d.taskDef.Config.IsEnabled()
|
|
}
|
|
|
|
// SetEnabled turns detection for this task type on or off.
|
|
//
|
|
// The admin maintenance policy is applied to detectors through an
|
|
// interface{ SetEnabled(bool) } type assertion (see
|
|
// MaintenanceIntegration.configureDetectorFromPolicy). Every registered task is
|
|
// backed by this generic detector, so without this method that assertion failed
|
|
// for every task and the policy never reached the flag that
|
|
// ScanWithTaskDetectors actually gates on. See issue #10874.
|
|
func (d *GenericDetector) SetEnabled(enabled bool) {
|
|
d.taskDef.Config.SetEnabled(enabled)
|
|
}
|
|
|
|
// GenericScheduler implements TaskScheduler using function-based logic
|
|
type GenericScheduler struct {
|
|
taskDef *TaskDefinition
|
|
}
|
|
|
|
// NewGenericScheduler creates a scheduler from a task definition
|
|
func NewGenericScheduler(taskDef *TaskDefinition) *GenericScheduler {
|
|
return &GenericScheduler{taskDef: taskDef}
|
|
}
|
|
|
|
// GetTaskType returns the task type
|
|
func (s *GenericScheduler) GetTaskType() types.TaskType {
|
|
return s.taskDef.Type
|
|
}
|
|
|
|
// CanScheduleNow determines if a task can be scheduled using the task definition's function
|
|
func (s *GenericScheduler) CanScheduleNow(task *types.TaskInput, runningTasks []*types.TaskInput, availableWorkers []*types.WorkerData) bool {
|
|
if s.taskDef.SchedulingFunc == nil {
|
|
return s.defaultCanSchedule(task, runningTasks, availableWorkers)
|
|
}
|
|
return s.taskDef.SchedulingFunc(task, runningTasks, availableWorkers, s.taskDef.Config)
|
|
}
|
|
|
|
// defaultCanSchedule provides default scheduling logic
|
|
func (s *GenericScheduler) defaultCanSchedule(task *types.TaskInput, runningTasks []*types.TaskInput, availableWorkers []*types.WorkerData) bool {
|
|
if !s.taskDef.Config.IsEnabled() {
|
|
return false
|
|
}
|
|
|
|
// Count running tasks of this type
|
|
runningCount := 0
|
|
for _, runningTask := range runningTasks {
|
|
if runningTask.Type == s.taskDef.Type {
|
|
runningCount++
|
|
}
|
|
}
|
|
|
|
// Check concurrency limit
|
|
maxConcurrent := s.taskDef.MaxConcurrent
|
|
if maxConcurrent <= 0 {
|
|
maxConcurrent = 1 // Default
|
|
}
|
|
if runningCount >= maxConcurrent {
|
|
return false
|
|
}
|
|
|
|
// Check if we have available workers
|
|
for _, worker := range availableWorkers {
|
|
if worker.CurrentLoad < worker.MaxConcurrent {
|
|
for _, capability := range worker.Capabilities {
|
|
if capability == s.taskDef.Type {
|
|
return true
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// GetPriority returns the priority for this task
|
|
func (s *GenericScheduler) GetPriority(task *types.TaskInput) types.TaskPriority {
|
|
return task.Priority
|
|
}
|
|
|
|
// GetMaxConcurrent returns max concurrent tasks
|
|
func (s *GenericScheduler) GetMaxConcurrent() int {
|
|
if s.taskDef.MaxConcurrent > 0 {
|
|
return s.taskDef.MaxConcurrent
|
|
}
|
|
return 1 // Default
|
|
}
|
|
|
|
// GetDefaultRepeatInterval returns the default repeat interval
|
|
func (s *GenericScheduler) GetDefaultRepeatInterval() time.Duration {
|
|
if s.taskDef.RepeatInterval > 0 {
|
|
return s.taskDef.RepeatInterval
|
|
}
|
|
return 24 * time.Hour // Default
|
|
}
|
|
|
|
// IsEnabled returns whether this scheduler is enabled
|
|
func (s *GenericScheduler) IsEnabled() bool {
|
|
return s.taskDef.Config.IsEnabled()
|
|
}
|
|
|
|
// SetEnabled turns scheduling for this task type on or off. Detector and scheduler
|
|
// share one TaskDefinition, so this is the same flag GenericDetector.SetEnabled sets;
|
|
// both setters exist because the maintenance integration configures the two
|
|
// independently. See GenericDetector.SetEnabled and issue #10874.
|
|
func (s *GenericScheduler) SetEnabled(enabled bool) {
|
|
s.taskDef.Config.SetEnabled(enabled)
|
|
}
|
|
|
|
// SetMaxConcurrent applies the policy's concurrency limit for this task type. It is
|
|
// the value GetMaxConcurrent returns, which the maintenance queue uses to decide
|
|
// whether another task of this type may start. Non-positive limits are ignored
|
|
// rather than turning into the implicit default of 1.
|
|
func (s *GenericScheduler) SetMaxConcurrent(maxConcurrent int) {
|
|
if maxConcurrent <= 0 {
|
|
return
|
|
}
|
|
s.taskDef.MaxConcurrent = maxConcurrent
|
|
}
|