Commit Graph
6 Commits
Author SHA1 Message Date
pingqiuandClaude Opus 4.6 c7eb87c587 feat: Phase 09 — V2 execution primitives and production closure
Engine execution layer for V2 replication protocol:
- RebuildInstaller: full state handoff (dirty map, WAL, superblock, flusher)
- TruncateToLSN: exact safety predicate (checkpointLSN == truncateLSN),
  ErrTruncationUnsafe escalation to NeedsRebuild
- SyncReceiverProgress: unconditional Store for post-rebuild alignment
- V2StatusSnapshot: CommittedLSN = nextLSN-1 for sync_all

V2 bridge real I/O executors:
- TransferFullBase: TCP streaming + RebuildInstaller + second catch-up
- TransferSnapshot: SHA-256 verified streaming to disk
- TruncateWAL: ErrTruncationUnsafe detection + escalation
- StreamWALEntries: rebuild-mode TCP apply

Engine executor interfaces:
- CatchUpIO.TruncateWAL, RebuildIO.TransferFullBase returns achievedLSN
- CatchUpExecutor truncation-only skip, NeedsRebuild escalation
- RebuildExecutor uses achievedLSN for progress tracking

Design docs reorganized: superseded planning docs removed, protocol
truths and closure map added.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:25:23 -07:00
pingqiuandClaude Opus 4.6 8b1b6ec1c0 fix: update executor doc comment to reflect P2 implementation status
Executor comment now reflects reality:
- StreamWALEntries, TransferFullBase, TransferSnapshot: real
- TruncateWAL: stub
- Implements engine.CatchUpIO and engine.RebuildIO interfaces

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 15:14:34 -07:00
pingqiuandClaude Opus 4.6 1578adfba5 fix: wire real v2bridge I/O into engine executors (Phase 08 P2 closure)
Engine executors now have IO interfaces for real bridge I/O:
- CatchUpExecutor.IO (CatchUpIO): StreamWALEntries
- RebuildExecutor.IO (RebuildIO): TransferFullBase, TransferSnapshot,
  StreamWALEntries (for tail replay)

When IO is set, executor calls real bridge I/O during execution.
When IO is nil, executor uses caller-supplied progress (test mode).

RecoveryPlan.CatchUpStartLSN: bound at plan time for IO bridge.

v2bridge.Executor now implements both interfaces:
- StreamWALEntries: real ScanFrom
- TransferFullBase: validates extent accessible
- TransferSnapshot: validates checkpoint accessible

Chain tests wire IO:
- CatchUpClosure: exec.IO = executor → real WAL scan through engine
- RebuildClosure: exec.IO = executor → real transfer through engine

This closes the engine → executor → v2bridge → blockvol chain.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 15:10:50 -07:00
pingqiuandClaude Opus 4.6 c9671c4e47 feat: integrated execution chain — catch-up + rebuild + cleanup (Phase 08 P2)
Live catch-up chain:
- Assignment → engine plan → v2bridge WAL scan → blockvol ScanFrom
- StreamWALEntries transfers real entries (transferred=5)
- V1 interim: engine classifies ZeroGap (committed=0), but WAL scan
  chain proven mechanically (executor→v2bridge→blockvol→progress)

Live rebuild chain (full-base):
- ForceFlush advances checkpoint → NeedsRebuild detected
- TransferFullBase now real: validates extent accessible at committed LSN
- Engine rebuild session: connect → handshake → source select →
  transfer → complete → InSync

Execution cleanup:
- CancelPlan releases resources + invalidates session
- Log shows plan_cancelled with reason

Observability:
- sender_added + escalated events explain execution causality
- Escalation includes proof reason from RetainedHistory

4 new execution chain tests + TransferFullBase implementation.

Carry-forward:
- Post-checkpoint catch-up not proven as integrated engine chain
  (V1 CommittedLSN=0 collapses to ZeroGap)
- TransferSnapshot: stub
- TruncateWAL: stub

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 14:22:27 -07:00
pingqiuandClaude Opus 4.6 785a7d7efd feat: wire real pinner into flusher retention + real WAL scan executor (Phase 07 P1)
Pinner wired to real retention:
- NewPinner calls vol.SetV2RetentionFloor(p.MinWALRetentionFloor)
- Flusher.RetentionFloorFn() / SetRetentionFloorFn() exposed
- SetV2RetentionFloor chains with existing shipper retention floor
- Holds actually prevent WAL reclaim (not just tracked state)

Executor uses real WAL scan:
- BlockVol.ScanWALEntries(fromLSN, callback) wraps wal.ScanFrom
  with real fd, walOffset, checkpointLSN
- Executor.StreamWALEntries uses ScanWALEntries (not stub)
- Reads real WAL entries, tracks highest LSN scanned

CommittedLSN mapping:
- Explicitly documented as interim V1 model (committed = checkpointed)
- Will diverge when V2 distributed commit separates from local flush

Carry-forward:
- TransferSnapshot/TransferFullBase/TruncateWAL: stubs (need extent I/O)
- Control intent from confirmed failover: deferred

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 20:01:46 -07:00
pingqiuandClaude Opus 4.6 c00c9e3e3d feat: add real BlockVolPinner + BlockVolExecutor in v2bridge (Phase 07 P1)
Pinner (pinner.go):
- HoldWALRetention: validates startLSN >= current tail, tracks hold
- HoldSnapshot: validates checkpoint exists + trusted
- HoldFullBase: tracks hold by ID
- MinWALRetentionFloor: returns minimum held position across all
  WAL/snapshot holds — designed for flusher RetentionFloorFn hookup
- Release functions remove holds from tracking map

Executor (executor.go):
- StreamWALEntries: validates range against real WAL tail/head
  (actual ScanFrom integration deferred to network-layer wiring)
- TransferSnapshot/TransferFullBase/TruncateWAL: stubs for P1

Key integration points:
- Pinner reads real StatusSnapshot for validation
- Pinner.MinWALRetentionFloor can wire into flusher.RetentionFloorFn
- Executor validates WAL range availability from real state

Carry-forward:
- Real ScanFrom wiring needs WAL fd + offset (network layer)
- TransferSnapshot/TransferFullBase need extent I/O
- Control intent from confirmed failover (master-side)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:54:24 -07:00