Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-10 08:17:44 +02:00
Code Issues Packages Projects Releases Wiki Activity
13,186 Commits 202 Branches 361 Tags
f20ec2ef79dcbdb5f8766fbfbc15db2ff0246293
Commit Graph
11 Commits
Author SHA1 Message Date
pingqiu 43dbebfa04 refactor: close bounded recovery drain and invalidation seams 
Move removed-replica drain and replica-scoped invalidation onto explicit core-command paths so the widened multi-replica runtime no longer depends on coarse host-side recovery handling.

Made-with: Cursor
 2026-04-04 11:01:12 -07:00
pingqiuandClaude Opus 4.6 11c6aaf316 feat: Batch 7 + Phase 16C-E — command dispatch extraction + engine refinements 
Batch 7: Command dispatch binding extraction
- New weed/server/blockcmd package: CommandHandler interface + DispatchCommands
- volume_server_block.go applyCoreCommandsWithAssignment delegates to dispatcher
- weed/server still owns RecordCommand, EmitCoreEvent, PublishProjection
- v2bridge NOT given command-switch or event-emission semantics

Phase 16C: Rebuilding assignment enters core command path
Phase 16D: Rebuild recovery-task startup is command-driven
Phase 16E: Catch-up recovery-task startup is command-driven

Engine refinements:
- RecoveryTarget on AssignmentDelivered event
- shouldStartRecoveryTask / shouldStartReceiver guards
- bootstrapReason: awaiting_rebuild_start

Bridge/contract updates:
- control_adapter.go: refined translation helpers
- contract.go: executor port alignment

Migration design docs (Batch 1-3 delivered, design artifacts):
- v2-first/second/third-migration-batch.md + task-pack.md
- v2-assignment-translation-unification.md
- v2-execution-muscles-inventory.md
- v2-separation-port-layer-audit.md
- v2-legacy-runtime-exit-criteria.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-04 02:13:08 -07:00
pingqiuandClaude Opus 4.6 8c2485e0e9 feat: Phase 15 + Phase 16A/B — V2 core integration + checkpoint review 
Phase 15: V2 core wired into BlockService
- volume_server_block.go: v2Core field, applyCoreAssignmentEvent,
  core command executors (ApplyRole, StartReceiver, ConfigureShipper,
  InvalidateSession, StartCatchUp, StartRebuild, PublishProjection)
- Assignment processing now goes through core engine → command emission
  → bounded execution, replacing direct V1 replication setup
- master_block_registry.go: ClusterHealthSummary, VolumeMode in entries
- master_server_handlers_block.go: blockStatusHandler, entryToVolumeInfo
  refactored with entryReplicaSurface

Phase 16A: Core projection surfaces
Phase 16B: Bounded closure (checkpoint review ready)

Test fixes: add v2Core to manually-constructed BlockService in
idempotence, convergence, soak, and CP13-8A tests (required because
V1 replication setup paths now delegate to core engine).

All tests pass (21s regression).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-03 20:58:12 -07:00
pingqiuandClaude Opus 4.6 34f42078fb docs: Phase 13 CP13-9 accepted + Phase 14 preparation docs 
- phase-13.md: CP13-8/8A/9 accepted with carry-forward
- phase-13-log.md: CP13-9 technical/delivery packs
- phase-13-cp9-mode-normalization.md: minor updates
- v2-protocol-claim-and-evidence.md: CP13-8/8A claims updated,
  constrained-V1-runtime interpretation rule added

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-03 16:13:03 -07:00
pingqiu fb0da91196 feat: start Phase 14 V2 core shell 
Make the first V2 core owner explicit in sw-block by freezing Phase 14 docs, mode/readiness/publication semantics, and bounded command emission rules. This turns accepted Phase 13 constraints into executable core behavior without overclaiming live runtime cutover.

Made-with: Cursor
 2026-04-03 16:11:38 -07:00
pingqiuandClaude Opus 4.6 4c7fbefe25 feat: CP13-8 PASSES — real-workload validation on RF=2 sync_all 
CP13-8 scenario results on m01/M02 (25Gbps RoCE):
  fsck_ext4:       CLEAN
  file count:      200 (assert_equal PASS)
  checksum match:  MATCH (assert_contains PASS)
  pgbench TPS:     565.69 (assert_greater PASS)
  auto-failover:   10.0.0.1:18480 → 10.0.0.3:18480

Code changes (tester + scenario):
- volume_server_block.go: readiness state, assignment lifecycle cleanup
- block_heartbeat_loop.go: readiness-aware heartbeat reporting
- store_blockvol.go: readiness tracking
- master_server_handlers_block.go: block API handler updates
- cp13-8-real-workload-validation.yaml: redesigned scenario
  (removed block_promote, use natural auto-failover flow,
  bootstrap write before wait_volume_healthy)
- testrunner/actions/devops.go: scenario action improvements
- replica_read_test.go: component-level replica read test

Phase docs: CP13-7 accepted, CP13-8/8A technical packs updated,
design docs updated for protocol closure evidence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-03 14:24:13 -07:00
pingqiuandClaude Opus 4.6 21ccf06ef3 docs: Phase 13 CP13-1..CP13-7 technical packs, acceptance status, design updates 
- phase-13.md: CP13-1 through CP13-6 accepted, CP13-7 active
- phase-13-log.md: full technical + delivery packs for CP13-2..CP13-7
- phase-13-cp4-state-eligibility.md: refined barrier behavior table
  (Disconnected/Degraded as recovery entry points, not eligibility)
- phase-12.md: minor cross-reference updates
- Older phase docs: minor wording alignment
- Design docs: V2 development plan and completion overview updated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-03 08:48:05 -07:00
pingqiuandClaude Opus 4.6 c7eb87c587 feat: Phase 09 — V2 execution primitives and production closure 
Engine execution layer for V2 replication protocol:
- RebuildInstaller: full state handoff (dirty map, WAL, superblock, flusher)
- TruncateToLSN: exact safety predicate (checkpointLSN == truncateLSN),
  ErrTruncationUnsafe escalation to NeedsRebuild
- SyncReceiverProgress: unconditional Store for post-rebuild alignment
- V2StatusSnapshot: CommittedLSN = nextLSN-1 for sync_all

V2 bridge real I/O executors:
- TransferFullBase: TCP streaming + RebuildInstaller + second catch-up
- TransferSnapshot: SHA-256 verified streaming to disk
- TruncateWAL: ErrTruncationUnsafe detection + escalation
- StreamWALEntries: rebuild-mode TCP apply

Engine executor interfaces:
- CatchUpIO.TruncateWAL, RebuildIO.TransferFullBase returns achievedLSN
- CatchUpExecutor truncation-only skip, NeedsRebuild escalation
- RebuildExecutor uses achievedLSN for progress tracking

Design docs reorganized: superseded planning docs removed, protocol
truths and closure map added.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 16:25:23 -07:00
pingqiuandClaude Opus 4.6 46faf0f7e3 feat: Phase 09 P0 — production execution closure plan 
Execution-closure targets:
- P1: TransferFullBase — reuse rebuild.go TCP protocol
- P2: TransferSnapshot — checkpoint image + WAL tail
- P3: TruncateWAL — AdvanceTail + superblock update
- P4: Runtime ownership — V2 orchestrator drives execution

Key reuse sources identified:
- rebuild.go: rebuildFullExtent (client), RebuildServer (server)
- wal_writer.go: AdvanceTail
- flusher.go: updateSuperblockCheckpoint
- blockvol.go: ScanWALEntries (already wired)

Slice order: full-base first (highest value), then snapshot,
then truncation, then runtime ownership.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 17:25:09 -07:00
pingqiuandClaude Opus 4.6 46ef79ce35 fix: stable ServerID in assignments, fail-closed on missing identity, wire into ProcessAssignments 
Finding 1: Identity no longer address-derived
- ReplicaAddr.ServerID field added (stable server identity from registry)
- BlockVolumeAssignment.ReplicaServerID field added (scalar RF=2 path)
- ControlBridge uses ServerID, NOT address, for ReplicaID
- Missing ServerID → replica skipped (fail closed), logged

Finding 2: Wired into real ProcessAssignments
- BlockService.v2Bridge field initialized in StartBlockService
- ProcessAssignments converts each assignment via v2Bridge.ConvertAssignment
  BEFORE existing V1 processing (parallel, not replacing yet)
- Logged at glog V(1)

Finding 3: Fail-closed on missing identity
- Empty ServerID in ReplicaAddrs → replica skipped with log
- Empty ReplicaServerID in scalar path → no replica created
- Test: MissingServerID_FailsClosed verifies both paths

7 tests: StableServerID, AddressChange_IdentityPreserved,
MultiReplica_StableServerIDs, MissingServerID_FailsClosed,
EpochFencing_IntegratedPath, RebuildAssignment, ReplicaAssignment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 10:46:17 -07:00
pingqiuandClaude Opus 4.6 edec7098e8 feat: add V2 protocol simulator and enginev2 sender/session prototype 
Adds sw-block/ directory with:

- distsim: protocol correctness simulator (96 tests)
  - cluster model with epoch fencing, barrier semantics, commit modes
  - endpoint identity, control-plane flow, candidate eligibility
  - timeout events, timer races, same-tick ordering
  - session ownership tracking with ID-based stale fencing

- enginev2: standalone V2 sender/session implementation (63 tests)
  - per-replica Sender with identity-preserving reconciliation
  - RecoverySession with FSM phase transitions and session ID
  - execution APIs: BeginConnect, RecordHandshake, BeginCatchUp,
    RecordCatchUpProgress, CompleteSessionByID — all sender-authority-gated
  - recovery outcome branching: zero-gap, catch-up, needs-rebuild
  - assignment-intent orchestration with epoch fencing

- design docs: acceptance criteria, open questions, first-slice spec,
  protocol development process

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 10:38:27 -07:00