Stable identity on wire: - ServerID fields in proto (replica_server_id, server_id on ReplicaAddrMessage) - volumeServerId wired through volume.go → BlockService.SetServerID - Identity derived from canonical server ID, not transport addresses Assignment convergence: - V2 idempotence via lastAppliedAssignment.equals (full replica set comparison) - setupPrimaryReplication/Multi idempotence guards - ProcessAssignments with V2 + V1 dual-path assignment handling Master-driven control loop: - RecoveryManager: serialized cancel-and-drain via done channels - Per-replica heartbeat state reporting (ReplicaShipperStatus) - masterServerBackend: VolumeBackend calling real MasterServer in-process - RestoreBlockSnapshot RPC (master + volume server proto) QA tests (P10 P1-P4): - Identity: ServerID on wire, fail-closed on missing - Convergence: assignment delivery, epoch monotonicity, registry coherence - Idempotence: repeated assignment, multi-replica set comparison - Control loop: integrationMaster + real allocator + proto round-trip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8.1 KiB
Phase 10
Date: 2026-04-02
Status: complete
Purpose: close the main end-to-end control-plane gaps on the chosen RF=2 sync_all path without reopening accepted backend execution semantics
Why This Phase Exists
Phase 09 closed the main backend execution gaps on the chosen path:
- real
TransferFullBase - real
TransferSnapshot - real
TruncateWALunder the accepted narrowed contract - stronger live runtime ownership on the volume-server path
What still does not exist is stronger end-to-end control-plane closure.
The main remaining gap is no longer:
- whether the backend execution path is real
It is now:
- whether the real control path drives and reflects the chosen path coherently enough for product use
Phase Goal
Strengthen from accepted assignment-entry closure to stronger end-to-end control-plane closure on the chosen path.
Scope
In scope
- heartbeat / gRPC-level control delivery proof on the chosen path
- reassignment / failover result convergence through the real control path
- cleaner local identity than transport-shaped
listenAddr - bounded idempotence / repeated-assignment cleanup when it directly affects live control/runtime ownership
Out of scope
- reopening accepted
P1/P2/P3/P4backend execution semantics RF>2best_effort/sync_quorum- product-surface rebinding (
CSI/NVMe/iSCSI) - broad performance optimization
Phase 10 Items
P0: Control-Plane Closure Plan
Goal:
- start
Phase 10with one substantial control-plane closure package, not a loose collection of follow-up fixes
Must prove:
- the phase is centered on real control-path closure rather than backend execution rework
- the required closure targets are explicit:
- heartbeat / gRPC delivery
- reassignment / result convergence
- identity cleanup
- bounded repeated-assignment/idempotence cleanup
- the chosen-path bound remains explicit
Verification mechanism:
- architect review:
- control-plane scope is explicit and bounded
- proposed slices do not reopen accepted backend execution semantics
- tester review:
- required end-to-end proofs are explicit
- manager review:
- the package is concrete enough to assign the first implementation slice
Output artifacts:
- explicit control-plane closure targets
- explicit reject shapes
- initial slice order inside
Phase 10
Execution note:
- use
phase-10-log.mdas the technical pack for:- semantic scope
- execution scope
- proof shapes
- assignment templates for
swandtester
Reject if:
Phase 10is framed as a vague "polish/control" phase without concrete closure targets- accepted
Phase 09execution semantics are quietly reopened - product surfaces or unrelated hardening work are absorbed into this phase
- no explicit end-to-end proof shape is defined
Status:
- accepted
P1: Identity And Control-Truth Closure
Goal:
- close stable identity on the real chosen-path control wire so assignment truth, local ingest truth, and
ReplicaIDconstruction no longer depend on transport-shaped fallback
Accepted scope:
- stable server identity preserved on the block assignment proto wire
- master assignment generation preserves stable identity on the chosen path
- volume-server local identity uses the same canonical server identity as the main volume server
- real ingress proof:
- proto/decode
ProcessAssignments()ControlBridge- engine sender identity
- fail-closed behavior for missing stable identity
Status:
- accepted
Carry-forward from P1:
- fuller reassignment / failover result convergence is still open
- broader control-plane reporting closure is still open
P2: Reassignment / Result Convergence
Goal:
- prove that reassignment and failover converge through the real control path without stale local ownership or stale reported truth lingering after control truth changes
Accepted scope:
- real failover / reassignment convergence through the chosen control path
- no stale local runtime owner after control truth changes
- no stale control/reporting truth after reassignment
- one-chain proof through the real control path, not only local helper logic
- no overclaim of broader hardening or product-surface closure
Status:
- accepted
Carry-forward from P2:
P2proves stale owner removal and no stale residue after control truth changes- bounded repeated-assignment/idempotence cleanup is still open where repeated primary assignment can still emit rebuild-server relisten warnings
P2does not claim broad master-driven failover infrastructure closure beyond the accepted volume-server-side chosen-path ingress
P3: Bounded Repeated-Assignment / Idempotence Cleanup
Goal:
- close the remaining low-severity repeated-assignment/runtime-idempotence gap on the chosen path so duplicate or replacement primary assignments do not leave avoidable relisten/restart noise or ambiguous live-control ownership
Accepted scope:
- repeated primary assignment on the same chosen-path volume should converge idempotently
- rebuild-server/runtime side effects should not relaunch noisily when the authoritative control truth is unchanged or already active
- bounded proof that repeated-assignment cleanup does not reopen accepted
P2convergence or acceptedPhase 09execution semantics - no expansion into broad runtime polish, product surfaces, or unrelated restart hardening
Status:
- accepted
Carry-forward from P3:
- chosen-path repeated unchanged assignment is now absorbed idempotently across the accepted V2 + V1 live path
P3remains bounded cleanup; it does not itself close the remaining master-driven heartbeat/gRPC control-loop gap- fuller master-originated control delivery proof is still open
P4: Master-Driven Control-Loop Closure
Goal:
- close the remaining chosen-path control-plane gap by proving that master-originated assignment truth delivered through the real heartbeat / gRPC control loop reaches the live volume-server path and converges without split truth
Required scope:
- one bounded end-to-end proof from real master-produced chosen-path assignment truth into the live volume-server control path
- proof that the real heartbeat / gRPC delivery path preserves the already accepted identity and convergence properties
- proof that externally visible post-delivery state reflects the same new truth after the real master-driven path runs
- no reopening of accepted
P1/P2/P3semantics except for narrow bugs directly exposed by the fuller control-loop proof - no expansion into product surfaces,
RF>2, or broad cluster-hardening work
Status:
- accepted
Carry-forward from P4:
- bounded chosen-path master-driven heartbeat / gRPC control-loop closure is now accepted
P4does not claim full live transport-stream deployment proof or broad product hardening- the next phase should move to
Phase 11product-surface rebinding
Planned slice direction after P0
P1:- identity and control-truth closure on the live control path
P2:- reassignment / failover result convergence through the real control path
P3:- bounded idempotence / repeated-assignment cleanup after accepted
P1/P2
- bounded idempotence / repeated-assignment cleanup after accepted
P4:- master-driven heartbeat / gRPC control-loop closure on the chosen path
Assignment For sw
Current next tasks:
- treat
Phase 10as closed and keep acceptedP1/P2/P3/P4semantics stable - start
Phase 11product-surface rebinding fromv2-phase-development-plan.md - keep the first
Phase 11slice bounded to selected product surfaces rather than broad hardening - do not reopen accepted backend execution or control-plane closure except for narrow bug fixes
Assignment For tester
Current next tasks:
- treat
P4as accepted bounded control-loop closure on the chosen path - validate the first
Phase 11slice as bounded product-surface rebinding rather than renewed control-plane work - keep no-overclaim active around:
- accepted
Phase 09execution closure - accepted
Phase 10control-plane closure - selected
Phase 11surface scope vs broader product readiness - chosen path vs future paths/modes
- accepted