2 Commits

Include renames

Author	SHA1	Message	Date
3 people	74ed12d6b3	s3 lifecycle: replay only the rules that fit the retention window (#11789) ... * s3 lifecycle: replay only the rules that fit the retention window runShard handed drainShardEvents the full snapshot, so rules the retention window assigns to the walk were also matched during meta-log replay. Their matches are not due yet, and drainShardEvents freezes the cursor at the first event with a not-yet-due match. With a 3-day and a 30-day expiration rule on one bucket and RetentionWindow of 8 days, every shard's cursor stopped at the first 30-day object's PUT and stayed there until that object came due. Each pass then re-read the log from that point and re-dispatched every 3-day expiration already handled, which surfaces as NOOP_RESOLVED dispatches, a growing cursor_lag_max, and passes that still report status=ok. Snapshot.ReplayView drops the replay-kind actions promoted to the walk and keeps every other action as compiled, and the drain uses it. Promoted rules are served by the walk as before. RulesForShard, PromotedHash and ReplayView share one isPromoted predicate. With RetentionWindow unset no active rule is promoted, so behaviour without a configured window is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: scale the replay-view test with DaysToDuration The test took its retention window from DaysToDuration but dated its PUTs in hours, so under the s3tests build tag (one lifecycle day = 10s) every event fell before the seeded cursor and replay never ran. Express the event ages in DaysToDuration too, so the test exercises replay at both scales. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3 lifecycle: rehabilitate scan_only mode in the replay drain view ReplayView shared the base CompiledAction pointers, so a replay-kind rule locked into ModeScanOnly by a prior compile stayed invisible to router.Route's event_driven gate even when the runtime retention window kept it in the replay partition - and it is absent from the walk view too, so the rule would be served by neither. Clone non-promoted replay-kind actions the way RulesForShard does (Mode forced to event_driven), reactivating the scan_only case whose bootstrap rendezvous already completed, while preserving the base active bit for genuinely pre-bootstrap actions. Disabled actions are excluded for parity with the other views, and DESIGN.md now documents the view. * s3 lifecycle: keep drain activation observable, require walker with retention ReplayView now shares pointers for event-driven actions so a mid-pass MarkActive flip remains visible to the drain; only a scan_only-locked action is cloned to event_driven, and reactivated only when its bootstrap rendezvous completed (tracked via a new bootstrapComplete field on CompiledAction). validate() rejects RetentionWindow without Walker: promoted rules would lose their only dispatch path while the drain cursor still advanced past their events. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com>	2026-10-17 15:24:35 -07:00
Chris Lu	3f4cb6d2fb	feat(s3/lifecycle/engine): daily-replay view surface (Phase 4 engine) (#9447) ... * feat(s3/lifecycle/engine): daily-replay view surface (Phase 4 engine) Adds the engine-side API the new daily-replay worker reaches for: per-view snapshot construction (RulesForShard, RecoveryView), the two cursor hashes that gate recovery (ReplayContentHash, PromotedHash), and the cursor sliding-window helper (MaxEffectiveTTL). CurrentSnapshot is a stub keyed on a package-level atomic that the worker startup wiring populates. Views return new *Snapshot instances holding cloned *CompiledAction values so per-clone active/Mode never leak across partitions. Replay clones force Mode=ModeEventDriven to rehabilitate any persistent ModeScanOnly carried over from PriorState; walk and recovery clones preserve Mode as-is. Disabled actions are excluded from all views. No production caller is wired here — Phase 4's walker/dailyrun integration is the follow-up. dailyrun's local helpers (localReplayContentHash, localMaxEffectiveTTL) become one-line redirects to these exports. API surface: - CurrentSnapshot() *Snapshot — stub until Phase 4 wiring. - SetCurrentEngine(*Engine) — Phase 4 wiring entry point. - Snapshot.RulesForShard(shardID, retentionWindow) (replay, walk *Snapshot) - RecoveryView(s *Snapshot) *Snapshot — force-active over the full set. - ReplayContentHash(s *Snapshot) [32]byte — partition-independent. - PromotedHash(s *Snapshot, retentionWindow) [32]byte — partition-flip. - MaxEffectiveTTL(s *Snapshot) time.Duration — over active replay only. 30 unit tests covering clone isolation, Mode rewrite, partition membership including the multi-action-kind XML rule split, RecoveryView activating pre-BootstrapComplete actions, ReplayContentHash partition-independence, PromotedHash sensitivity to promotion in either direction, MaxEffectiveTTL aggregation. Build + race-tests green. * refactor(s3/lifecycle/engine): consolidate hash helpers; clarify shardID semantics Addresses PR #9447 review feedback. Three medium-priority items from gemini, all code-quality refinements (no behavior change): 1. Duplicated sort comparator between ReplayContentHash and PromotedHash. Extract sortHashItems shared helper so the two hashes use the same ordering by construction — if one drifted, the cursor could see a spurious "rule changed" on a no-op snapshot rebuild. 2. Duplicated writeField/writeInt closures. Extract hashWriter struct holding the sha256 running hash + lenbuf, with method helpers. Same allocation profile (one Hash, one tiny stack buffer per helper); just deduplicates ~20 lines. 3. shardID parameter on RulesForShard is unused. Per the design's open question, every shard sees every rule today (shard filter runs at the entry-iteration site, not view construction). Keep the parameter for API stability — removing it now would force a breaking change when bucket-shard ownership lands — and update the doc comment to explain why it's reserved. go build ./... clean; engine test suite green.	2026-05-11 18:07:54 -07:00