telemetry: keep only clusters that store at least 10 GiB (#11138)

* telemetry: tidy the server module after the protobuf bump

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ

* telemetry: keep only clusters that store at least 10 GiB

Fresh weed server runs, CI jobs and throwaway containers each mint their
own cluster id. They came in at tens of thousands a day, were most of
the counted clusters and held almost none of the bytes, and the state
file and the metrics page grew with every one of them. Reports under
the floor are counted and dropped, and a state file written before the
floor sheds them on the first restart.

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ

* master: report telemetry only once the cluster stores 10 GiB

A throwaway cluster no longer registers itself with its first report a
minute after start; a real one begins reporting at the first daily tick
after it crosses the floor.

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ
This commit is contained in:
Chris Lu authored and GitHub committed 2026-09-03 12:36:00 -07:00
1 parent a8f763e717
commit 0973634fd4
14 files changed
+168 -23

No files matched your search

+8 -1
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"os"
"path/filepath"
"github.com/seaweedfs/seaweedfs/telemetry/proto"
)
// persistedState is the on-disk snapshot of the in-memory instance map.
@@ -16,7 +18,8 @@ type persistedState struct {
// LoadState restores the instance map and Prometheus gauges from a state file
// written by SaveStateIfDirty. A missing file is not an error. Original
// ReceivedAt timestamps are preserved so cleanup and the active-cluster
// windows stay correct across restarts.
// windows stay correct across restarts. Clusters under proto.MinDiskBytes are
// dropped, so state written before the floor sheds them on the first restart.
func (s *PrometheusStorage) LoadState(path string) (int, error) {
b, err := os.ReadFile(path)
if err != nil {
@@ -39,6 +42,10 @@ func (s *PrometheusStorage) LoadState(path string) (int, error) {
if instance == nil || instance.TelemetryData == nil || instance.TelemetryData.TopologyId == "" {
continue
}
if instance.TelemetryData.TotalDiskBytes < proto.MinDiskBytes {
s.dirty = true // so the next save sheds it
continue
}
s.instances[id] = instance
s.setClusterMetrics(instance.TelemetryData)
loaded++