pingqiu and Claude Opus 4.6
55013e103b
feat: Phase 20 Stage 0+1 closure — bootstrap + sustained workload on hardware
...
Stage 0 (bootstrap closure): PASS on m01/M02
- create RF=2 sync_all → 10s shipper wait → 4k fsync → publish_healthy
- Proves: BarrierAccepted observation, ShipperConnected, DurableLSN > 0
Stage 1 (sustained workload): 32/33 actions PASS
- bootstrap → fio 10s randwrite → dd_write 1M×2 fsync → data checksum
- Remaining: auto-failover promotion (separate issue)
Key fixes:
- BarrierAccepted callback: SyncCache success → core DurableLSN update
- BarrierRejected callback: barrier failures surface to core with reason
- Shipper state callback for new volumes (not just startup volumes)
- CatchUpTo ctrl conn reset: prevents stale control channel after recovery
- CP13-6 max-bytes budget suspended: uses replicaFlushedLSN which can't
advance without barrier; kills healthy shippers during async writes.
Will be replaced by v2 negotiated sync/recovery protocol.
- Barrier diagnostic logging: start/fail/success with reason and LSN
- Scenario restructured: Stage 0 (bootstrap-closure) + Stage 1 (failover)
- dd_write: sync_mode param + real stderr capture
- sw-test-runner suite command: deploy once, run N scenarios
- WAL size plumbing: proto + API + handler (forward-compatible)
Known: 6 blockvol/server test failures from Barrier() path change
(bounded catch-up in Barrier). Need test updates to match new semantics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-06 19:55:12 -07:00
Ping Qiu and Claude Opus 4.6
1b3edd7856
feat: CP11A-2 coordinated expand protocol for replicated block volumes
...
Two-phase prepare/commit/cancel protocol ensures all replicas expand
atomically. Standalone volumes use direct-commit (unchanged behavior).
Engine: PrepareExpand/CommitExpand/CancelExpand with on-disk
PreparedSize+ExpandEpoch in superblock, crash recovery clears stale
prepare state on open, v.mu serializes concurrent expand operations.
Proto: 3 new RPCs (PrepareExpand/CommitExpand/CancelExpandBlockVolume).
Coordinator: expandClean flag pattern — ReleaseExpandInflight only on
clean success or full cancel. Partial replica commit failure calls
MarkExpandFailed (keeps ExpandInProgress=true, suppresses heartbeat
size updates). ClearExpandFailed for manual reconciliation.
Registry: AcquireExpandInflight records PendingExpandSize+ExpandEpoch.
ExpandFailed state blocks new expands until cleared.
Tests: 15 engine + 4 VS + 10 coordinator + heartbeat suppression
regression + updated QA CP82/durability tests with prepare/commit mocks.
Also includes CP11A-1 remaining: QA storage profile tests, QA
io_backend config tests, testrunner perf-baseline scenarios and
coordinated-expand actions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-12 15:06:48 -07:00
Ping Qiu and Claude Opus 4.6
0e234f5c80
feat: Phase 10 CP10-1 -- NVMe/TCP target MVP, 109 tests
...
NVMe over Fabrics (TCP) target implementation sharing the same BlockVol
engine, fencing, replication, and failover as the existing iSCSI target.
New package: weed/storage/blockvol/nvme/ (11 files, 2,242 production LOC)
- protocol.go: PDU types, opcodes, status codes, marshal/unmarshal
- wire.go: TCP reader/writer with header bounds validation
- controller.go: IC handshake, per-queue state, command dispatch, KATO
- fabric.go: Connect (admin+IO), PropertyGet/Set, Disconnect
- identify.go: Controller/Namespace/NS list/NS descriptors (Linux 5.15)
- admin.go: SetFeatures, GetFeatures, GetLogPage (SMART/ANA), KeepAlive
- io.go: Read (C2HData), Write (inline), Flush, WriteZeros/Trim
- server.go: TCP listener, admin session registry, graceful shutdown
- adapter.go: BlockVol-to-NVMe bridge, error mapping, ANA state
Integration: NVMeConfig + CLI flags (-block.nvme.*), disabled by default.
Key design: inline-data writes only (no R2T), MaxH2CDataLength=32KB,
single ANA group coherent with BlockVol role, CNTLID session registry
for cross-connection IO queues, HostNQN continuity enforcement.
Tests: 65 dev + 44 QA adversarial = 109 total, all passing.
Bugs fixed during review: IO queue cross-connection (A), header bounds
validation (B), write payload size check (C), disconnect error (D),
stream desync prevention (E), HostNQN enforcement (F), capsule-before-IC
state guard (H), flowCtlOff SQHD timing (I).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-08 16:52:37 -07:00
Ping Qiu and Claude Opus 4.6
9acd187587
feat: Phase 8 complete -- CP8-5 stability gate, lease grant fix, Docker e2e, 13 chaos scenarios
...
Phase 8 closes with all 6 checkpoints done (CP8-1 through CP8-5 + CP8-3-1):
- CP8-5: 12/12 enterprise QA scenarios PASS on real hardware (m01/M02)
- Master-authoritative lease grants (BUG-CP85-11): master renews primary
write leases on every heartbeat response, replacing retain-until-confirmed
assignment queue semantics that caused 30s lease expiry
- Post-rebuild WAL shipping gap fix (BUG-CP85-1): syncLSNAfterRebuild
advances replica nextLSN so WAL entries are accepted after rebuild
- Block heartbeat startup race fix (BUG-CP85-10): dynamic blockService
check on each tick instead of one-shot at loop start
- 8 new tests: 4 engine lease grant + 4 registry lease grant
- 13 new YAML scenarios: chaos (kill-loop, partition, disk-full),
database integrity (sqlite crash, ext4 fsck), perf baseline,
metrics verify, snapshot stress, expand-failover, session storm,
role flap, 24h soak
- 12 new testrunner actions (database, fsck, grep_log, write_loop_bg,
stop_bg, assert_metric_gt/eq/lt) + phase repeat support
- Docker compose setup + getting-started guide for block storage users
- 960+ cumulative unit tests, 24 YAML scenarios
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-07 21:30:14 -08:00
Ping Qiu and Claude Opus 4.6
da1b81d1c9
feat: CP8-3-1 durability modes + testrunner platform + 21 adversarial tests
...
Durability mode implementation (sync_all, sync_quorum, best_effort):
- DurabilityMode type with superblock persistence, parse/validate/string
- MakeDistributedSync mode-aware barrier enforcement in dist_group_commit
- blockerr sentinel package (ErrDurabilityBarrierFailed, ErrDurabilityQuorumLost)
- gRPC create path: mode validation, idempotent create consistency, partial cleanup
- F1: strict mode rejects partial replica provisioning with cleanup
- F3: empty heartbeat does not overwrite persisted strict mode
- F4: SCSI error mapping uses errors.Is sentinels (not string matching)
- Proto/wire/blockapi/CLI/UI plumbing for durability_mode field
- Observability dashboard: cluster health cards + per-volume columns
Testrunner platform (YAML-driven integration test framework):
- Engine, parser, registry, reporter (JUnit XML + HTML), metrics scraping
- 52 registered actions: block, iSCSI, I/O, fault injection, assertions
- Baseline regression framework with 7 hard-fail conditions
- 15 YAML scenarios (smoke, crash, HA, fault, consistency, snapshot)
- 49 unit tests for testrunner internals
QA adversarial suite (21 tests, all PASS):
- Idempotent create mode/RF mismatch detection
- Heartbeat mode downgrade prevention (F3)
- sync_all/sync_quorum partial replica enforcement (F1)
- Concurrent create race safety
- Failover/expand mode preservation
- Cleanup resilience when delete fails
- Master restart auto-register mode handling
- Superblock roundtrip all 3 modes
- Validate edge cases (mode×RF matrix)
- RequiredReplicas quorum math verification
- Sentinel error categorization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-06 01:06:51 -08:00
Ping Qiu and Claude Opus 4.6
979a9b496c
feat: Phase 8 CP8-1/2/3/4 -- ops control plane, multi-replica, CSI snapshots, observability
...
CP8-1: HTTP REST API (create/delete/lookup/list/assign/servers), blockapi Go
client with multi-master failover, 5 shell commands, HTML dashboard at /block/.
CP8-2: RF=2/RF=3 multi-replica support -- ShipperGroup fan-out, distributed
sync, health scoring, segment-based scrub, gated promotion (heartbeat
freshness + WAL LSN + role checks), failover/rebuild for N>2 replicas.
CP8-3: CSI snapshot + expansion -- CreateSnapshot/DeleteSnapshot/ListSnapshots
RPCs, NodeExpandVolume with iSCSI rescan, snapshot ID helpers, 20 adversarial
tests covering concurrent ops, edge cases, and error injection.
CP8-4: Observability -- EngineMetrics atomic counters for flusher/group-commit/
WAL-shipper/scrub, 10 new Prometheus metrics, barrier_lag_lsn SLO gauge,
failover/promotion/rebuild counters, request ID correlation in master gRPC
logs, baseline regression framework with 7 hard-fail conditions.
Total: 63 files, ~11.2K LOC, 160+ new tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-06 00:05:17 -08:00
Ping Qiu and Claude Opus 4.6
5a9a52f2d0
feat: Phase 6 CP6-2 -- CSI control-plane integration + csi-sanity/k3s validation
...
CP6-2 wires the CSI driver to SeaweedFS master/volume-server control plane:
- Proto: block volume messages in master.proto/volume_server.proto, codegen
- Master registry: in-memory BlockVolumeRegistry with Pending->Active status,
full/delta heartbeat, inflight lock, placement (fewest volumes)
- VS gRPC: AllocateBlockVolume/DeleteBlockVolume handlers, shared naming
- Master RPCs: CreateBlockVolume (retry up to 3 servers), Delete, Lookup
- Heartbeat: block volume fields wired into bidirectional stream
- CSI Controller: VolumeBackend interface (Local + Master), returns volume_context
- CSI Node: reads volume_context for remote targets, staged map + IQN derivation
- Mode flag: --mode=controller/node/all, --master for control-plane
- K8s manifests: csi-driver.yaml, csi-controller.yaml, csi-node.yaml
csi-sanity conformance (33 pass, 58 skip) found 6 bugs:
- BUG-SANITY-1/2/3: missing VolumeCapabilities/VolumeCapability validation
- BUG-SANITY-4: NodePublish used mount instead of bind mount
- BUG-SANITY-5: NodeUnpublish didn't remove target path
- BUG-SANITY-6: NodeUnpublish failed on unmounted path
k3s Level 4 (PVC->Pod data persistence) found 1 bug:
- BUG-K3S-1: IsLoggedIn didn't handle iscsiadm exit code 21
226 CSI tests + 54 server tests = 280 new tests, all passing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-04 11:01:08 -08:00
