seaweedfs/sw-block/design/v2-phase-development-plan.md

V2 Phase Development Plan

Date: 2026-04-02 Status: active Purpose: define the execution-oriented phase plan after the current candidate-path work, with explicit module status and target phase ownership

Why This Document Exists

The project now needs a development plan that is:

1. phase-oriented
2. execution-oriented
3. large enough to avoid overhead-heavy micro-slices
4. explicit about which module belongs to which future phase

This document is the planning bridge between:

1. v2-product-completion-overview.md
2. ../.private/phase/phase-08.md
3. future implementation phases

Planning Rules

Use these rules for all later phases:

1. one phase should close one meaningful product/engineering outcome
2. every phase must have a clear delivery object and a clear closed-loop validation mechanism
3. every slice inside a phase should also name:
   • what is delivered
   • what loop is proven closed
   • what reject shapes remain insufficient
4. phases should prefer real code/test/evidence over wording-only progress
5. later phases may reuse V1 engineering reality, but must not inherit V1 recovery semantics as truth
6. a phase is too small if it does not move the overall product-completion state clearly

Current Baseline

Current accepted path through Phase 09:

1. protocol/algo truth set is strong
2. engine recovery core is strong
3. real control delivery exists on the chosen path
4. real one-chain catch-up and rebuild closure exist on the chosen path
5. production-grade execution closure is accepted on the chosen path:

• real TransferFullBase
• real TransferSnapshot
• real TruncateWAL
• stronger live runtime ownership on the volume-server path

6. unified hardening validation exists on the chosen path
7. one bounded accepted path exists for:

• RF=2
• sync_all
• existing master / volume-server heartbeat path

Phase-accounting note:

1. Phase 08 is closed
2. Phase 09 is also closed
3. this roadmap should now be read from the post-Phase 09 state, not the post-Phase 08 state

This means the next phases should focus mainly on:

1. stronger control-plane closure
2. later product-surface rebinding
3. production hardening
4. bounded cleanup of low-severity residuals without reopening accepted execution semantics

Phase Roadmap

Phase 09: Production Execution Closure

Goal:

1. turn validation-grade backend execution into production-grade backend execution

Must prove:

1. full-base rebuild performs real data transfer
2. snapshot rebuild performs real image transfer
3. replica-ahead path is physically executable, not only detected
4. runtime execution ownership is stronger than the current bounded candidate path

Typical outputs:

1. real TransferFullBase
2. real TransferSnapshot
3. real TruncateWAL
4. stronger executor/runtime integration in the live volume-server path

Verification mechanism:

1. one-chain execution tests on real backend paths
2. cleanup assertions after success/failure/cancel
3. focused adversarial tests for truncation and rebuild execution

Workload:

1. large
2. this is likely the single biggest remaining engineering phase

Status:

1. complete
2. accepted closeout exists in ../.private/phase/phase-09.md

Phase 10: Real Control-Plane Closure

Goal:

1. strengthen from accepted assignment-entry closure to fuller end-to-end control-plane closure

Why this is next:

1. Phase 09 already closed the main backend execution gaps
2. the most important remaining product risk is no longer storage execution itself
3. it is now control-path completeness:
   • heartbeat / gRPC delivery
   • reassignment / result convergence
   • cleaner local identity than transport-shaped listenAddr
4. Phase 10 can also absorb bounded low-severity cleanup discovered during Phase 09 if it is directly relevant to live control/runtime ownership

Current accepted progress inside Phase 10:

1. P1 accepted:
   • stable identity and control-truth closure on the chosen block assignment wire
2. P2 accepted:
   • reassignment/result convergence through the accepted volume-server-side chosen-path ingress
3. P3 accepted:
   • bounded repeated-assignment / idempotence cleanup on the chosen live path
4. P4 accepted:
   • master-driven heartbeat / gRPC control-loop closure on the chosen path
5. Phase 10 is now closed:
   • bounded end-to-end control-plane closure for the chosen path is accepted

Must prove:

1. heartbeat/gRPC-level delivery is real for the chosen path
2. failover / reassignment state converges through the real control path
3. local and remote identity are consistent enough for product use

Typical outputs:

1. stronger heartbeat/gRPC delivery proof
2. stronger result/reporting convergence
3. cleaner local identity than transport-shaped listenAddr

Verification mechanism:

1. real failover/reassignment tests at the fuller control-plane level
2. identity/fencing assertions through the end-to-end path

Suggested first targets:

1. keep accepted Phase 10 control-plane closure closed
2. start Phase 11 with one bounded product-surface rebinding slice
3. prefer selected surface proofs over broad surface explosion
4. keep any residual control-path cleanup narrow; do not reopen accepted Phase 10 closure casually

Workload:

1. medium-large

Phase 11: Product Surface Rebinding

Goal:

1. bind product-facing surfaces onto the V2-backed block path after backend closure is strong enough

Must prove:

1. the V2-backed backend can support selected product surfaces without semantic drift
2. reuse of V1 surfaces does not reintroduce V1 recovery truth

Candidate areas:

1. snapshot product path
2. CSI
3. NVMe
4. iSCSI

Recommended first slice:

1. start with bounded snapshot product path rebinding
2. defer CSI and NVMe / iSCSI until one simpler product-visible surface is already accepted

Suggested slice order:

1. P1 snapshot product-path rebinding
2. P2 CSI rebinding
3. P3 NVMe / iSCSI front-end rebinding
4. P4 broader workflow closure such as snapshot restore/clone if still needed

Verification mechanism:

1. selected surface integration tests
2. product-surface contract checks
3. no-overclaim review that the surface does not imply unsupported backend capability

Workload:

1. medium-large
2. can be split by product surface if needed, but only after backend closure is strong

Phase 12: Production Hardening

Goal:

1. move from candidate-safe to production-safe

Must prove:

1. restart/recovery stability under repeated disturbance
2. long-run/soak viability
3. operational diagnosability
4. acceptable production blockers list or production-ready gate

Verification mechanism:

1. soak/adversarial runs
2. failover/restart under disturbance
3. runbook/debug validation

Workload:

1. large

Recommended initial planning cut:

1. treat P0 as hardening-plan freeze
2. first hardening slice should likely target restart / recovery disturbance before soak or perf

Current slice order:

1. P0 hardening-plan freeze
2. P1 restart / recovery disturbance hardening
3. P2 soak / long-run stability hardening
4. P3 diagnosability / blocker accounting / runbook hardening
5. P4 performance floor / rollout-gate hardening

Slice delivery / closed-loop bar:

1. every Phase 12 slice must end with:
   • one bounded delivery object
   • one bounded closed-loop validation object
   • one explicit no-overclaim boundary
2. “tests exist” is not enough:
   • the tests must close the loop from disturbance/input to visible accepted truth
3. “code changed” is also not required:
   • a hardening slice may legitimately close by proving existing production code is already correct under the targeted disturbance class

Current status:

1. P0 accepted:
   • hardening object frozen as the accepted chosen path from Phase 09 + Phase 10 + Phase 11
   • slice order frozen as P1 / P2 / P3 / P4
   • evidence ladder frozen as disturbance correctness, soak, diagnosability, then perf/rollout gates
2. P1 accepted:
   • acceptance object = correctness under restart/disturbance on the chosen path
   • not soak, not diagnosability, not performance, not rollout readiness
3. P2 accepted:
   • acceptance object = bounded soak / long-run stability on the chosen path
   • repeated-cycle coherence and bounded runtime-state hygiene are accepted inside a bounded test envelope
4. P3 accepted:
   • acceptance object = bounded diagnosability / blocker accounting / runbook hardening on the chosen path
   • bounded operator-visible diagnosis surfaces and finite blocker accounting are accepted
5. P4 active:
   • acceptance object = bounded performance floor / rollout-gate hardening on the chosen path
   • not broad rollout readiness beyond the named launch envelope

Current P4 first delivery shape:

1. proof-first hardening slice with explicit measured floor and launch-gate artifacts
2. one bounded performance package:
   • named workload envelope
   • repeatable measurement harness
   • explicit floor values
3. one explicit rollout-gate artifact:
   • finite supported launch envelope
   • cleared blockers/gates
   • remaining blockers/gates
4. current evidence shape:
   • measured floor values are tied to one named accepted workload envelope
   • cost/resource trade-offs are explicit
   • rollout discussion is bounded by an explicit finite gate package
5. current reuse boundary:
   • accepted chosen-path runtime/control/product surfaces remain stable unless perf-floor work exposes a real bug or measurement gap
   • focused benchmarks/tests and bounded launch-gate artifacts carry the main delivery burden

Closed-loop expectation for P4 review:

1. one bounded workload envelope runs on the accepted chosen path
2. measured floor values and cost characteristics are explicit
3. launch claims map back to accepted prior slices plus the measured envelope
4. remaining rollout blockers are explicit and finite
5. claims remain bounded to measured floor / named launch envelope only

After Phase 12:

1. move to a productionization program, not more protocol-discovery phases
2. freeze production blockers and the supported launch envelope
3. run a limited internal pilot with incident-driven hardening
4. perform controlled rollout only after explicit launch-gate review

Module Status Map

Module area	Current status	Current owner phase	Next target phase	Notes
sw-block/engine/replication core FSM/orchestrator/driver	Strong	Phase 09 accepted	Phase 10-12	Main later work is control-plane/runtime integration and hardening, not new core semantics.
Engine executor real I/O boundary (CatchUpIO / RebuildIO)	Strong on chosen path	Phase 09 accepted	Phase 10/12	Keep the boundary stable; later work is orchestration/control proof and hardening.
weed/storage/blockvol/v2bridge/control.go	Strong on chosen path	Phase 08/09/10 accepted	Phase 12	Chosen-path control mapping is accepted; later work is hardening, not new mapping semantics.
weed/storage/blockvol/v2bridge/reader.go	Strong	Phase 09 accepted	Phase 12	Mostly stable; later work is verification/hardening, not new semantics.
weed/storage/blockvol/v2bridge/pinner.go	Strong	Phase 09 accepted	Phase 12	Retention safety proven; later work is hardening under disturbance.
weed/storage/blockvol/v2bridge/executor.go WAL scan	Strong	Phase 09 accepted	Phase 12	Real execution path closed on chosen path; later work is hardening.
v2bridge TransferFullBase	Strong on chosen path	Phase 09 P1 accepted	Phase 12	Execution closure accepted; do not reopen casually.
v2bridge TransferSnapshot	Strong on chosen path	Phase 09 P2 accepted	Phase 12	Execution closure accepted; do not reopen casually.
v2bridge TruncateWAL	Strong on chosen path	Phase 09 P3 accepted	Phase 12	Narrow Option A contract accepted; later work is hardening/planning improvement.
weed/server/volume_server_block.go V2 assignment intake	Strong on chosen path	Phase 10 P4 accepted	Phase 11/12	VS-side ingress, convergence, idempotence, and bounded master-driven closure are accepted; later work is product-surface integration and hardening.
weed/server/block_recovery.go live runtime ownership	Strong on chosen path	Phase 09/10 accepted	Phase 11/12	Serialized ownership and bounded control-plane integration are accepted; later work is product-surface integration and hardening.
blockvol WAL/flusher/checkpoint runtime	Reuse reality	Existing production code	Phase 12	Reuse implementation; later work is disturbance/restart hardening.
blockvol rebuild transport/server reality	Reuse with redesign boundary	Existing production code	Phase 12	Bounded chosen-path integration is accepted; later work is hardening under disturbance.
local server identity (localServerID)	Strong on chosen path	Phase 10 P1 accepted	Phase 12	Canonical volumeServerId now backs chosen-path local identity; later work is hardening only.
Snapshot product path	Strong on chosen path	Phase 11 accepted	Phase 12	Product-visible snapshot create/list/delete closure and restore workflow closure are accepted; later work is hardening, not rebinding.
CSI integration	Strong on chosen path	Phase 11 accepted	Phase 12	Bounded controller/node lifecycle rebinding accepted; later work is hardening.
NVMe / iSCSI front-ends	Strong on chosen path	Phase 11 accepted	Phase 12	Publication/address truth rebinding accepted; later work is runtime/perf hardening.
Testrunner / infra / metrics	Strong support layer	existing	Phase 11-12	Reuse to validate later product-surface and hardening phases.

Completion-State Targets

Use these rough targets to judge whether a phase is moving the product meaningfully.

Phase	Expected completion move
Phase 09	from validation-grade backend execution to accepted execution closure on the chosen path
Phase 10	from bounded control-entry proof to stronger end-to-end control-plane closure
Phase 11	from backend-ready path to selected product-surface readiness
Phase 12	from candidate-safe to production-safe

Near-Term Execution Direction

If the goal is to maximize product completion efficiently, the recommended order is:

1. keep Phase 09 closed and do not reopen accepted execution semantics casually
2. keep Phase 10 closed and do not reopen accepted bounded control-plane closure casually
3. move next to Phase 11 product surface rebinding
4. then Phase 12 production hardening

The most important near-term engineering weight should now go to Phase 12.

Short Summary

The V2 line now has accepted execution closure on one bounded chosen path. The next development plan should treat later work as control/product completion phases, not more protocol discovery.

The main heavy engineering work still ahead is:

1. stronger end-to-end control-plane closure
2. later product-surface rebinding
3. production hardening
4. bounded cleanup of residual operational rough edges without reopening accepted semantics