Commit Graph
14 Commits
Author SHA1 Message Date
Ping QiuandClaude Opus 4.6 44da35faf6 test: add integration test infrastructure for blockvol iSCSI
Test harness for running blockvol iSCSI tests on WSL2 and remote nodes
(m01/M02). Includes Node (SSH/local exec), ISCSIClient (discover/login/
logout), WeedTarget (weed volume server lifecycle), and test suites for
smoke, stress, crash recovery, chaos, perf benchmarks, and apps (fio/dd).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:05:29 -08:00
Ping QiuandClaude Opus 4.6 c39080ceaa feat: Phase 4A CP4b-4 -- HA integration tests, admin HTTP, 5 bug fixes
Add HTTP admin server to iscsi-target binary (POST /assign, GET /status,
POST /replica, POST /rebuild) and 7 HA integration tests validating
failover, split-brain prevention, epoch fencing, and demote-under-IO.

New files:
- admin.go: HTTP admin endpoint with input validation
- ha_target.go: HATarget helper wrapping Target + admin HTTP calls
- ha_test.go: 7 HA tests (all PASS on WSL2, 67.7s total)

Bug fixes:
- BUG-CP4B4-1: CmdSN init (expCmdSN=0 not 1, first SCSI cmd was dropped)
- BUG-CP4B4-2: RoleNone->RoleReplica missing SetEpoch (WAL rejected)
- BUG-CP4B4-3: replica applyEntry didn't update vol.nextLSN (status=0)
- BUG-CP4B4-4: PID discovery killed primary instead of replica (shared
  binPath; fixed by grepping volFile)
- BUG-CP4B4-5: artifact collector overwrote primary log with replica log
  (added CollectLabeled method)

Also: 3s write deadline on WAL shipper data connection to avoid 120s TCP
retransmission timeout when replica is dead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:04:59 -08:00
Ping QiuandClaude Opus 4.6 c95500cc57 test: Phase 4A CP4b-1 QA adversarial tests (19 tests)
Boundary tests for RoleFromWire, LeaseTTLToWire overflow/clamp/negative,
ToBlockVolumeInfoMessage with primary/stale/closed/concurrent volumes,
BlockVolumeAssignment roundtrip, and heartbeat collection edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:34:35 -08:00
Ping QiuandClaude Opus 4.6 ffdde15bcd feat: Phase 4A CP4b-1 -- wire types, conversion helpers, heartbeat collection
Add BlockVolumeInfoMessage, BlockVolumeShortInfoMessage, BlockVolumeAssignment
wire-type structs (proto-shaped Go structs). Add conversion helpers with
DiskType plumbing, overflow-safe LeaseTTLToWire, validated RoleFromWire.
Add CollectBlockVolumeHeartbeat on BlockVolumeStore. 9 new tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:34:00 -08:00
Ping QiuandClaude Opus 4.6 09c7e40d29 feat: Phase 4A CP4a -- simulated master, assignment sequence tests, BlockVolumeStatus
Add SimulatedMaster test helper + 20 assignment sequence tests (8 sequence,
5 failover, 5 adversarial, 2 status). Add BlockVolumeStatus struct and
Status() method. Includes QA test files for CP1-CP4a. 940 total unit tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:33:45 -08:00
Ping QiuandClaude Opus 4.6 b31383e294 feat: Phase 4A CP3 -- promotion, rebuild, split-brain prevention
Add master-driven lifecycle operations: promotion, demotion, rebuild,
and split-brain prevention. All testable on Windows with mock TCP.

New files:
- promotion.go: HandleAssignment (single entry point for role changes),
  promote (Replica/None -> Primary with durable epoch), demote
  (Primary -> Draining -> Stale with drain timeout)
- rebuild.go: RebuildServer (WAL catch-up + full extent streaming),
  StartRebuild client (WAL catch-up with full extent fallback,
  two-phase rebuild with second catch-up for concurrent writes)

Modified:
- wal_writer.go: ScanFrom() method, ErrWALRecycled sentinel
- repl_proto.go: rebuild message types + RebuildRequest encode/decode
- blockvol.go: assignMu, drainTimeout, rebuildServer fields;
  HandleAssignment/StartRebuildServer/StopRebuildServer methods;
  rebuild server stop in Close()
- dirty_map.go: Clear() method for full extent rebuild

32 new tests covering WAL scan, promotion/demotion, rebuild server,
rebuild client, split-brain prevention, and full lifecycle scenarios.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 21:26:39 -08:00
Ping QiuandClaude Opus 4.6 16a796e56d feat: Phase 4A CP2 — WAL shipping, replica barrier, distributed group commit
Primary ships WAL entries to replica over TCP (data channel), confirms
durability via barrier RPC (control channel). SyncCache runs local fsync
and replica barrier in parallel via MakeDistributedSync. When replica is
unreachable, shipper enters permanent degraded mode and falls back to
local-only sync (Phase 3 behavior).

Key design: two separate TCP ports (data+control), contiguous LSN
enforcement, epoch equality check, WAL-full retry on replica,
cond.Wait-based barrier with configurable timeout, BarrierFsyncFailed
status code. Close lifecycle: shipper → receiver → drain → committer →
flusher → fd.

New files: repl_proto.go, wal_shipper.go, replica_apply.go,
replica_barrier.go, dist_group_commit.go
Modified: blockvol.go, blockvol_test.go

27 dev tests + 21 QA tests = 48 new tests; 889 total (609 engine + 280
iSCSI), all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 20:12:39 -08:00
Ping QiuandClaude Opus 4.6 a107685f00 feat: Phase 4A CP1 — epoch, lease, role state machine, write gate
Local fencing primitives for block volumes. Every write path validates
role + epoch + lease before accepting data. RoleNone (default) skips
all checks for Phase 3 backward compatibility.

New files: epoch.go, lease.go, role.go, write_gate.go
Modified: superblock.go (Epoch field), blockvol.go (fencing fields,
writeGate in WriteLBA/Trim), group_commit.go (PostSyncCheck/Gotcha A),
dirty_map.go (P3-BUG-9 power-of-2 panic)

Bug fixes: BUG-4A-1 (atomic epoch), BUG-4A-2 (CAS SetRole),
BUG-4A-3 (mutex SetEpoch), BUG-4A-4 (single role.Load),
BUG-4A-6 (safeCallback recover)

837 tests (557 engine + 280 iSCSI), all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 16:00:06 -08:00
Ping QiuandClaude Opus 4.6 80801b0fac feat: Phase 3 — performance tuning, iSCSI session refactor, store integration
Phase 3 delivers five checkpoints:

CP1 Engine Tuning: BlockVolConfig tunables, 256-shard DirtyMap, adaptive
group commit (low-watermark immediate flush), WAL pressure handling with
backpressure and ErrWALFull timeout.

CP2 iSCSI Session Refactor: RX/TX goroutine split with respCh (cap 64),
txLoop for serialized response writes, StatSN assignment modes. Login
phase stays single-goroutine; full-duplex after login.

CP3 Store Integration: BlockVolAdapter (iscsi.BlockDevice interface),
BlockVolumeStore management, BlockService in volume_server_block.go,
CLI flags (--block.listen/dir/iqn.prefix), sw-block-attach.sh helper.

CP5 Concurrency Hardening: WAL reuse guard (LSN validation in ReadLBA),
opsOutstanding counter with beginOp/endOp + Close drain, appendWithRetry
shared by WriteLBA and TrimLBA, flusher LSN guard in FlushOnce.

Bug fixes (P3-BUG-2–11): unbounded pending queue cap, Data-Out timeout,
flusher error logging, GroupCommitter panic recovery, Close vs concurrent
ops guard, target shutdown race, WAL-full retry vs Close, WRITE SAME(16)
for XFS, MODE SENSE(10) + VPD 0xB0/0xB2 for Linux kernel compatibility.

797 tests passing (517 engine + 280 iSCSI), go vet clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 10:43:34 -08:00
Ping QiuandClaude Opus 4.6 9b7be60b0c Add QA adversarial tests for iSCSI target (55 tests)
9 categories: PDU, Params, Login, Discovery, SCSI, DataIO, Session,
Target, Integration. 2,183 lines. All 229 tests pass (164 dev + 55 QA).
No new production bugs found.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 11:15:18 -08:00
Ping QiuandClaude Opus 4.6 feef0206ad fix: address code review findings (nil handler, CmdSN, Data-Out order)
1. Discovery session nil handler crash: reject SCSI commands with
   Reject PDU when s.scsi is nil (discovery sessions have no target).

2. CmdSN window enforcement: validate incoming CmdSN against
   [ExpCmdSN, MaxCmdSN] using serial arithmetic. Drop out-of-window
   commands per RFC 7143 section 4.2.2.1.

3. Data-Out buffer offset validation: enforce BufferOffset == received
   for ordered data (DataPDUInOrder=Yes). Prevents silent corruption
   from out-of-order or overlapping data.

4. ImmediateData enforcement: reject immediate data in SCSI command
   PDU when negotiated ImmediateData=No.

5. UNMAP descriptor length alignment: reject blockDescLen not a
   multiple of 16 bytes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 10:12:16 -08:00
Ping QiuandClaude Opus 4.6 bd73a81e00 fix: handle pipelined SCSI commands during Data-Out collection
The Linux kernel iSCSI initiator pipelines multiple SCSI commands on
the same TCP connection (command queuing). When a write needs R2T for
data beyond the immediate portion, collectDataOut may read a pipelined
SCSI command instead of the expected Data-Out PDU.

Fix: queue non-Data-Out PDUs received during collectDataOut into a
pending buffer. The main dispatch loop drains pending PDUs before
reading from the connection. This correctly handles interleaved
commands during multi-PDU write transfers.

Bug found during WSL2 smoke test: mkfs.ext4 hangs at "Writing
superblocks" because inode table zeroing sends large writes that
exceed FirstBurstLength, triggering R2T while the kernel has already
queued the next command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 10:08:32 -08:00
Ping QiuandClaude Opus 4.6 6546c549eb fix: iSCSI login and discovery bugs found in WSL2 smoke test
- Skip InitiatorAlias in negotiation (was returning NotUnderstood)
- Capture TargetName in StageLoginOp direct-jump path (iscsiadm skips
  security stage, sends CSG=LoginOp directly -- nil SCSIHandler crash)
- Add portalAddr to TargetServer for discovery responses (listener on
  [::] is not routable from WSL2 clients)
- Add -portal flag to iscsi-target binary

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 10:04:41 -08:00
Ping QiuandClaude Opus 4.6 6a400f6760 feat: add BlockVol engine and iSCSI target (Phase 1 + Phase 2)
Phase 1: Extent-mapped block storage engine with WAL, crash recovery,
dirty map, flusher, and group commit. 174 tests, zero SeaweedFS imports.

Phase 2: Pure Go iSCSI target (RFC 7143) with PDU codec, login
negotiation, SendTargets discovery, 12 SCSI opcodes, Data-In/Out/R2T
sequencing, session management, and standalone iscsi-target binary.
164 tests. IQN->BlockDevice binding via DeviceLookup interface.

Total: 338 tests, 14.6K lines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:32:01 -08:00