* ec: compare live idx NeedleId sets before encode
Refuse encode when replicas' live .idx NeedleId sets differ. Walk only
live rows (skip deleted and Size<=0) and compare NeedleId, ignoring Size
so MIME/flags drift does not abort a matching id set. Mark the volume
read-only for encode and reject writes after that.
* fix(ec): compare all replicas then pick capacity source
Log matching live idx sets with "replicas are consistent" for the
shell CI check. After readonly, compare every replica's live NeedleId
set, then require the encode source to have FreeVolumeCount>=2.
Empty live-idx vs non-empty refuses encode instead of skipping
superblock stubs. Drop the FileCount rebind after the idx gate.
* volume: mark readonly under the write lock so checked appends finish first
* ec: fix readonly deadlock and restore worker-host source preference
MarkVolumeReadonly held dataFileAccessLock while calling
reopenIdxForWrite, which re-acquires the same mutex and deadlocks
TestMarkVolumeReadonlyCanDelete. Split out reopenIdxForWriteLocked for
callers already holding the lock, matching the existing Locked pattern.
The worker task lost its worker-host source preference when the idx gate
replaced the replica union. Late-bind the source to a replica on the
worker host using only the checked location set, with no FileCount
rebinding. Drop the now-unused helpers, trim comments added by the
earlier commits, and rename tests whose names still claimed convergence
while asserting refusal.
---------
Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
Co-authored-by: Devin <devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* refactor(volume): extract replica sync/select into shared volume_replica package
Move the volume replica reconciliation helpers (status, union builder,
SyncAndSelectBestReplica, ReadNeedleMeta) out of the shell into a new
weed/storage/volume_replica package so both the shell (ec.encode, volume.tier.move,
volume.check.disk) and the EC encode worker can reuse them. No behavior change.
* fix(ec): bring ec.encode worker to parity with the shell
- Sync replicas and encode the most-complete one (via the shared
volume_replica.SyncAndSelectBestReplica) instead of a possibly-stale replica,
marking all replicas readonly first. Prevents silent data loss when a stale
replica is encoded and the originals deleted.
- Skip remote/tiered volumes in detection (shell ec.encode excludes them).
- Min-node safety gate: refuse to encode when cluster nodes < parity shards.
- Align default thresholds with the shell (fullness 0.95, quiet 1h).
* fix(vacuum): plugin path honors min_volume_age_seconds override
deriveVacuumConfig hard-coded MinVolumeAgeSeconds=0, dropping any configured
value. Read it from worker config (default 0, matching the shell/master vacuum
which has no age gate) so an explicit override is honored.
* address review feedback
- config.go: align GetConfigSpec schema defaults (quiet_for_seconds=3600,
fullness_ratio=0.95) with the runtime defaults so UI/bootstrap flows match the
shell (coderabbitai).
- ec_task.go: roll back readonly when markReplicasReadonly fails partway, so
already-marked replicas don't stay readonly (coderabbitai).
- volume_replica: pass the caller's replica statuses into buildUnionReplica instead
of re-fetching them, and skip the per-needle ReadNeedleMeta RPC when the source
replica is read-only (gemini-code-assist).
* test(plugin_workers/ec): make fixtures eligible under the new defaults
The default EC encode thresholds were raised to match the shell (fullness 0.95,
quiet 1h), but the plugin-worker integration fixtures still used 90%-full /
10-minute-old volumes, so detection found no eligible volumes and the tests failed
in CI. Bump the eligible fixtures to 96% full and 2h old.
