Commit Graph
6 Commits
Author SHA1 Message Date
Chris Lu 0973634fd4 telemetry: keep only clusters that store at least 10 GiB (#11138)
* telemetry: tidy the server module after the protobuf bump

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ

* telemetry: keep only clusters that store at least 10 GiB

Fresh weed server runs, CI jobs and throwaway containers each mint their
own cluster id. They came in at tens of thousands a day, were most of
the counted clusters and held almost none of the bytes, and the state
file and the metrics page grew with every one of them. Reports under
the floor are counted and dropped, and a state file written before the
floor sheds them on the first restart.

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ

* master: report telemetry only once the cluster stores 10 GiB

A throwaway cluster no longer registers itself with its first report a
minute after start; a real one begins reporting at the first daily tick
after it crosses the floor.

Claude-Session: https://claude.ai/code/session_01VGiphDxpsKMwu9XpUFhybQ
2026-09-03 12:36:00 -07:00
Chris Lu c167af541e telemetry: confirm a cluster after a week of reports, not two days (#10899)
* telemetry: sync the server module to go 1.26

The root module moved to go 1.26 but the telemetry server module, which
replaces seaweedfs with the repo root, stayed on 1.25.8, so go refuses
to build or test it until the directive catches up.

* telemetry: confirm a cluster after a week of reports, not two days

Two days of history still lets recurring CI and demo clusters into the
confirmed fleet: anything torn down and rebuilt across a UTC midnight
counts. Requiring seven distinct UTC days keeps the fleet charts and the
version/OS distributions to clusters that actually stay up; real
clusters qualify after their first week, and the fallback to all active
clusters while none is confirmed is unchanged.
2026-08-23 11:14:17 -07:00
Chris Lu 88fd2d1be8 telemetry: stack volume servers per cluster, drop the total disk usage chart (#10550)
* telemetry: stack volume servers per cluster over time

The fleet-wide server count says how many volume servers reported, but not
who they belong to. Carry per-cluster counts in /api/cluster-sizes and draw
them the same way as cluster sizes, sharing one cluster ranking so a cluster
keeps its colour across both stacks.

* telemetry: drop the total disk usage chart from the dashboard

The stacked cluster sizes chart right below it has the same fleet total as
its stack height, plus the per-cluster breakdown. /api/metrics still serves
the aggregate for anyone graphing it elsewhere.
2026-08-03 13:13:48 -07:00
Chris Lu 01937cfad1 telemetry: build the over-time charts from confirmed clusters (#10489)
Short-lived clusters report once under a fresh raft topology id and never
again, so CI runs and demo stacks each become their own cluster. Held
forward for the whole active window they pile up, and the volume server
line climbs every day while capacity stays flat.

Sum the fleet series over confirmed clusters only, the same set the
version and OS charts already use.
2026-07-29 18:16:01 -07:00
Chris Lu c8cafd8a1a telemetry: fix total disk usage over time (#10476)
* telemetry: total disk usage over time counted each cluster on one day

GetMetrics aggregated s.instances, which holds only each cluster's most
recent report. Every cluster therefore landed in a single date bucket --
the day it last reported on -- so the chart plotted the disk usage of
clusters that went silent that day, and piled the whole live fleet onto
today. Aggregate the daily histories instead, reusing the day alignment
that the per-cluster size series already does.

* telemetry: don't pad the charts with days the server has no history for

The dashboard asks for 30 days, but daily history only starts when a
server first collects it, so the charts opened on a run of zeros and then
jumped -- reading as a fleet that appeared overnight. Start the window at
the oldest sample on hand when it is younger than the requested range.
2026-07-28 16:22:29 -07:00
Chris Lu c194924d13 telemetry: per-cluster size over time on the dashboard (#10417)
The dashboard charted one summed disk-usage line, so a step in the total
gave no hint which cluster moved. A new panel stacks each cluster's daily
size as its own band: the top of the stack is the fleet total, each band
is one cluster, and the clusters past the twentieth are summed into an
"other" band so the stack still adds up to the total.

The series is built from the per-cluster daily histories and served by
/api/cluster-sizes. Clusters report roughly once a day at no fixed hour,
so a day with no report carries the previous value forward — dropping it
to zero would sag the total every day as the clusters that have not
reported yet fall out from under it. A cluster that stops reporting past
the active window ends at its last sample instead of holding capacity
forever. Ranking is by the most recent day, tie-broken on cluster id so
the colors do not shuffle between refreshes.

Hover and click resolve to the band under the pointer: Chart.js's builtin
interaction modes match the nearest line, which on a stack of thin bands
is rarely the band being pointed at. Clicking one fills the per-cluster
history lookup below it.
2026-07-24 01:43:48 -07:00