admin: credit an unitemized slot to a pending job only after a later heartbeat (#12385)

* admin: credit an unitemized slot to a pending job only after a later heartbeat

A pending reservation the worker does not list took one of the slots
the worker counts without itemizing, even when that heartbeat came
before the reservation and could not have counted the job. The slot
belonged to unknown work, so the admin undercounted it and could admit
one job too many, past a job type's per-worker limit, until the next
heartbeat.

A reservation now records when it was made, and claims an unitemized
slot only from a heartbeat received at or after that time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: time unitemized-slot credit by the heartbeat, not the last hello

A reconnect's hello moves LastSeenAt but keeps the old heartbeat, so a
pending reservation could claim a slot counted before it existed, and a
completed job's echo could end without a new heartbeat. Record when the
heartbeat itself arrived and compare reservations against that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: a completed job claims no slot from a heartbeat before its reservation

A job that completes before any heartbeat counts it has no echo, so the
slot the last heartbeat counted is other work. Crediting it to the job's
type let another job type pass its per-worker limit until the next
heartbeat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

weed/admin/plugin/plugin_scheduler.go

@@ -1220,6 +1220,7 @@ type execReservation struct {
 	workerID   string
 	jobType    string
 	jobID      string
+	reservedAt time.Time
 	releasedAt time.Time
 }
 
@@ -1282,9 +1283,10 @@ func (l executorLoad) holders() string {
 // jobType "" counts work of every type, otherwise only that type. A
 // heartbeat entry whose job id matches a reservation is the same slot seen
 // twice and counts once. Slots the worker counts without itemizing them go
-// first to this admin's jobs it does not list, a completed one only until the
-// worker's next heartbeat; the rest are of unknown type and count against
-// every type. Caller holds r.execResMu.
+// first to this admin's jobs it does not list, and only from a heartbeat
+// that came after the job's reservation: a pending job's from then on, a
+// completed job's until the worker's next heartbeat. The rest are of unknown
+// type and count against every type. Caller holds r.execResMu.
 func (r *Plugin) executorLoadLocked(executor *WorkerSession, jobType string) executorLoad {
 	load := executorLoad{
 		reservedLanes: make(map[SchedulerLane]int),
@@ -1330,13 +1332,13 @@ func (r *Plugin) executorLoadLocked(executor *WorkerSession, jobType string) exe
 		}
 		switch {
 		case !released:
-			if !isListed {
+			if !isListed && !executor.heartbeatAt.Before(res.reservedAt) {
 				unitemized--
 			}
 			countReserved(res)
 		case isListed:
 			countReserved(res)
-		case executor.LastSeenAt.Before(res.releasedAt):
+		case !executor.heartbeatAt.Before(res.reservedAt) && executor.heartbeatAt.Before(res.releasedAt):
 			unlistedEchoes = append(unlistedEchoes, res)
 		}
 	}
@@ -1442,9 +1444,10 @@ func (r *Plugin) reserveExecutorSlot(
 	r.execResSeq++
 	token := r.execResSeq
 	r.execRes[token] = execReservation{
-		workerID: strings.TrimSpace(executor.WorkerID),
-		jobType:  jobType,
-		jobID:    jobID,
+		workerID:   strings.TrimSpace(executor.WorkerID),
+		jobType:    jobType,
+		jobID:      jobID,
+		reservedAt: time.Now(),
 	}
 	release := func() {
 		r.execResMu.Lock()