pingqiu and Claude Opus 4.6
3a5fbbfded
fix: Batch 3 wiring — production path uses runtime helpers, legacy isolated
...
H wiring: block_recovery.go now uses runtime.PendingCoordinator
- Removed local pendingRecoveryExecution type + store/take/peek/has/cancel
- ExecutePendingCatchUp/Rebuild delegate to coord.TakeCatchUp/TakeRebuild
- Shutdown uses coord.CancelAll
- Added CancelAll to PendingCoordinator
I wiring: executeCatchUpPlan/executeRebuildPlan replaced
- ExecutePendingCatchUp now calls rt.ExecuteCatchUpPlan with RecoveryManager
as RecoveryCallbacks (OnCatchUpCompleted/OnRebuildCompleted)
- ExecutePendingRebuild follows same pattern
- Local executeCatchUpPlan/executeRebuildPlan methods removed
J structural: legacy no-core branches extracted
- executeLegacyCatchUp: wraps rt.ExecuteCatchUpPlan for v2Core==nil path
- executeLegacyRebuild: wraps rt.ExecuteRebuildPlan for v2Core==nil path
- Clear "LEGACY NO-CORE COMPATIBILITY" section with structural separation
- runCatchUp/runRebuild now branch cleanly: legacy helper vs core coordinator
Test updates: pendingRecoveryExecution → rt.PendingExecution, field casing,
Plan type assertions.
Validation: all P4, P16B, and ApplyAssignments tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-04 01:20:41 -07:00
pingqiu and Claude Opus 4.6
8c2485e0e9
feat: Phase 15 + Phase 16A/B — V2 core integration + checkpoint review
...
Phase 15: V2 core wired into BlockService
- volume_server_block.go: v2Core field, applyCoreAssignmentEvent,
core command executors (ApplyRole, StartReceiver, ConfigureShipper,
InvalidateSession, StartCatchUp, StartRebuild, PublishProjection)
- Assignment processing now goes through core engine → command emission
→ bounded execution, replacing direct V1 replication setup
- master_block_registry.go: ClusterHealthSummary, VolumeMode in entries
- master_server_handlers_block.go: blockStatusHandler, entryToVolumeInfo
refactored with entryReplicaSurface
Phase 16A: Core projection surfaces
Phase 16B: Bounded closure (checkpoint review ready)
Test fixes: add v2Core to manually-constructed BlockService in
idempotence, convergence, soak, and CP13-8A tests (required because
V1 replication setup paths now delegate to core engine).
All tests pass (21s regression).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-03 20:58:12 -07:00
Ping Qiu and Claude Opus 4.6
0e234f5c80
feat: Phase 10 CP10-1 -- NVMe/TCP target MVP, 109 tests
...
NVMe over Fabrics (TCP) target implementation sharing the same BlockVol
engine, fencing, replication, and failover as the existing iSCSI target.
New package: weed/storage/blockvol/nvme/ (11 files, 2,242 production LOC)
- protocol.go: PDU types, opcodes, status codes, marshal/unmarshal
- wire.go: TCP reader/writer with header bounds validation
- controller.go: IC handshake, per-queue state, command dispatch, KATO
- fabric.go: Connect (admin+IO), PropertyGet/Set, Disconnect
- identify.go: Controller/Namespace/NS list/NS descriptors (Linux 5.15)
- admin.go: SetFeatures, GetFeatures, GetLogPage (SMART/ANA), KeepAlive
- io.go: Read (C2HData), Write (inline), Flush, WriteZeros/Trim
- server.go: TCP listener, admin session registry, graceful shutdown
- adapter.go: BlockVol-to-NVMe bridge, error mapping, ANA state
Integration: NVMeConfig + CLI flags (-block.nvme.*), disabled by default.
Key design: inline-data writes only (no R2T), MaxH2CDataLength=32KB,
single ANA group coherent with BlockVol role, CNTLID session registry
for cross-connection IO queues, HostNQN continuity enforcement.
Tests: 65 dev + 44 QA adversarial = 109 total, all passing.
Bugs fixed during review: IO queue cross-connection (A), header bounds
validation (B), write payload size check (C), disconnect error (D),
stream desync prevention (E), HostNQN enforcement (F), capsule-before-IC
state guard (H), flowCtlOff SQHD timing (I).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-08 16:52:37 -07:00
Ping Qiu and Claude Opus 4.6
9acd187587
feat: Phase 8 complete -- CP8-5 stability gate, lease grant fix, Docker e2e, 13 chaos scenarios
...
Phase 8 closes with all 6 checkpoints done (CP8-1 through CP8-5 + CP8-3-1):
- CP8-5: 12/12 enterprise QA scenarios PASS on real hardware (m01/M02)
- Master-authoritative lease grants (BUG-CP85-11): master renews primary
write leases on every heartbeat response, replacing retain-until-confirmed
assignment queue semantics that caused 30s lease expiry
- Post-rebuild WAL shipping gap fix (BUG-CP85-1): syncLSNAfterRebuild
advances replica nextLSN so WAL entries are accepted after rebuild
- Block heartbeat startup race fix (BUG-CP85-10): dynamic blockService
check on each tick instead of one-shot at loop start
- 8 new tests: 4 engine lease grant + 4 registry lease grant
- 13 new YAML scenarios: chaos (kill-loop, partition, disk-full),
database integrity (sqlite crash, ext4 fsck), perf baseline,
metrics verify, snapshot stress, expand-failover, session storm,
role flap, 24h soak
- 12 new testrunner actions (database, fsck, grep_log, write_loop_bg,
stop_bg, assert_metric_gt/eq/lt) + phase repeat support
- Docker compose setup + getting-started guide for block storage users
- 960+ cumulative unit tests, 24 YAML scenarios
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-07 21:30:14 -08:00
Ping Qiu and Claude Opus 4.6
8b2b5f6f66
feat: Phase 6 CP6-3 -- failover + rebuild in Kubernetes, 126 tests
...
Wire low-level fencing primitives to master/VS control plane and CSI:
- Proto: replica/rebuild address fields on assignment/info/response messages
- Assignment queue: retain-until-confirmed (Peek+Confirm), stale epoch pruning
- VS assignment receiver: processes assignments from HeartbeatResponse
- BlockService replication: ProcessAssignments, deterministic ports (FNV hash)
- Registry replica tracking: SetReplica/ClearReplica/SwapPrimaryReplica
- CreateBlockVolume: primary + replica, enqueues assignments, single-copy mode
- Failover: lease-aware promotion, deferred timers with cancellation on reconnect
- ControllerPublish: returns fresh primary iSCSI address after failover
- Recovery: recoverBlockVolumes drains pendingRebuilds, enqueues Rebuilding
- Real integration tests on M02: failover address switch, rebuild data
consistency, full lifecycle failover+rebuild (3 tests, all PASS)
Review fixes (12 findings, 5 High, 5 Medium, 2 Low):
- R1-1: AllocateBlockVolume returns replication ports
- R1-2: setupPrimaryReplication starts rebuild server
- R1-3: VS sends periodic block heartbeat for assignment confirmation
- R2-F1: LastLeaseGrant set before Register (no stale-lease race)
- R2-F2: Deferred promotion timers cancelled on VS reconnect
- R2-F3: SwapPrimaryReplica uses RoleToWire instead of uint32(1)
- R2-F4: DeleteBlockVolume deletes replica (best-effort)
- R2-F5: SwapPrimaryReplica computes epoch atomically under lock
- QA: SetReplica removes old replica from byServer index (BUG-QA-CP63-1)
126 CP6-3 tests (67 dev + 48 QA + 8 integration + 3 real).
Cumulative Phase 6: 352 tests. All PASS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-05 00:52:05 -08:00
Ping Qiu and Claude Opus 4.6
80801b0fac
feat: Phase 3 — performance tuning, iSCSI session refactor, store integration
...
Phase 3 delivers five checkpoints:
CP1 Engine Tuning: BlockVolConfig tunables, 256-shard DirtyMap, adaptive
group commit (low-watermark immediate flush), WAL pressure handling with
backpressure and ErrWALFull timeout.
CP2 iSCSI Session Refactor: RX/TX goroutine split with respCh (cap 64),
txLoop for serialized response writes, StatSN assignment modes. Login
phase stays single-goroutine; full-duplex after login.
CP3 Store Integration: BlockVolAdapter (iscsi.BlockDevice interface),
BlockVolumeStore management, BlockService in volume_server_block.go,
CLI flags (--block.listen/dir/iqn.prefix), sw-block-attach.sh helper.
CP5 Concurrency Hardening: WAL reuse guard (LSN validation in ReadLBA),
opsOutstanding counter with beginOp/endOp + Close drain, appendWithRetry
shared by WriteLBA and TrimLBA, flusher LSN guard in FlushOnce.
Bug fixes (P3-BUG-2–11): unbounded pending queue cap, Data-Out timeout,
flusher error logging, GroupCommitter panic recovery, Close vs concurrent
ops guard, target shutdown race, WAL-full retry vs Close, WRITE SAME(16)
for XFS, MODE SENSE(10) + VPD 0xB0/0xB2 for Linux kernel compatibility.
797 tests passing (517 engine + 280 iSCSI), go vet clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-01 10:43:34 -08:00
