Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-11-09 05:18:23 +01:00
Code Issues Packages Projects Releases Wiki Activity
13,152 Commits 225 Branches 366 Tags
3da4c19046e06ec7e75cb449e0d9287ef1a3473b
Commit Graph
3 Commits
Author SHA1 Message Date
Ping QiuandClaude Opus 4.6 548e47e482 feat: reconnect handshake + WAL catch-up protocol (CP13-5) 
Adds the sync_all reconnect protocol: when a degraded shipper
reconnects, it performs a handshake (ResumeShipReq/Resp) to
determine the replica's durable progress, then streams missed
WAL entries to close the gap before resuming live shipping.

New wire messages:
- MsgResumeShipReq (0x03): primary sends epoch, headLSN, retainStart
- MsgResumeShipResp (0x04): replica returns status + flushedLSN
- MsgCatchupDone (0x05): marks end of catch-up stream

Decision matrix after handshake:
- R == H: already caught up → InSync
- S <= R+1 <= H: recoverable gap → CatchingUp → stream → InSync
- R+1 < S: gap exceeds retained WAL → NeedsRebuild
- R > H: impossible progress → NeedsRebuild

WALAccess interface: narrow abstraction (RetainedRange + StreamEntries)
avoids coupling shipper to raw WAL internals.

Bootstrap vs reconnect split: fresh shippers (HasFlushedProgress=false)
use CP13-4 bootstrap path. Previously-synced shippers use handshake.

Catch-up retry budget: maxCatchupRetries=3 before NeedsRebuild.

ReplicaReceiver now initializes receivedLSN/flushedLSN from volume's
nextLSN on construction (handles receiver restart on existing volume).

TestBug2_SyncAll_SyncCache_AfterDegradedShipperRecovers flips FAIL→PASS.
All previously-passing baseline tests remain green.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 15:38:06 -07:00
Ping QiuandClaude Opus 4.6 979a9b496c feat: Phase 8 CP8-1/2/3/4 -- ops control plane, multi-replica, CSI snapshots, observability 
CP8-1: HTTP REST API (create/delete/lookup/list/assign/servers), blockapi Go
client with multi-master failover, 5 shell commands, HTML dashboard at /block/.

CP8-2: RF=2/RF=3 multi-replica support -- ShipperGroup fan-out, distributed
sync, health scoring, segment-based scrub, gated promotion (heartbeat
freshness + WAL LSN + role checks), failover/rebuild for N>2 replicas.

CP8-3: CSI snapshot + expansion -- CreateSnapshot/DeleteSnapshot/ListSnapshots
RPCs, NodeExpandVolume with iSCSI rescan, snapshot ID helpers, 20 adversarial
tests covering concurrent ops, edge cases, and error injection.

CP8-4: Observability -- EngineMetrics atomic counters for flusher/group-commit/
WAL-shipper/scrub, 10 new Prometheus metrics, barrier_lag_lsn SLO gauge,
failover/promotion/rebuild counters, request ID correlation in master gRPC
logs, baseline regression framework with 7 hard-fail conditions.

Total: 63 files, ~11.2K LOC, 160+ new tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 00:05:17 -08:00
Ping QiuandClaude Opus 4.6 7940e6b7c9 feat: Phase 4A CP4b-4 Windows iSCSI + instrumentation + QA tests 
Windows iSCSI Initiator compatibility:
- Add TargetPortalGroupTag to login response (RFC 7143 S13.9)
- Add REQUEST_SENSE, START_STOP_UNIT, MODE_SELECT(6/10) handlers
- Add PERSISTENT_RESERVE_IN/OUT, MAINTENANCE_IN (REPORT SUPPORTED OPCODES)
- Implement MODE SENSE caching (page 0x08) and control (page 0x0A) pages
- Fix Data-In residual underflow/overflow flags (U/O bits on final PDU)
- Rename ScsiReadCapacity16 -> ScsiServiceActionIn16 for correctness

Instrumentation and tooling:
- Add instrumentedAdapter with periodic PERF stats logging
- Add pprof endpoints on admin HTTP server (/debug/pprof/*)
- Add blockbench CLI tool for standalone block device benchmarking
- Add SCSI CDB debug logging in session dispatch

HA integration fixes:
- Move HA test replica ports to 9011-9014 to avoid conflicts
- Add QA adversarial tests for Phase 4A CP4b-4 (755 lines)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-02 21:17:57 -08:00