ec: compare live idx NeedleId sets before encode (#11812)

* ec: compare live idx NeedleId sets before encode

Refuse encode when replicas' live .idx NeedleId sets differ. Walk only
live rows (skip deleted and Size<=0) and compare NeedleId, ignoring Size
so MIME/flags drift does not abort a matching id set. Mark the volume
read-only for encode and reject writes after that.

* fix(ec): compare all replicas then pick capacity source

Log matching live idx sets with "replicas are consistent" for the
shell CI check. After readonly, compare every replica's live NeedleId
set, then require the encode source to have FreeVolumeCount>=2.
Empty live-idx vs non-empty refuses encode instead of skipping
superblock stubs. Drop the FileCount rebind after the idx gate.

* volume: mark readonly under the write lock so checked appends finish first

* ec: fix readonly deadlock and restore worker-host source preference

MarkVolumeReadonly held dataFileAccessLock while calling
reopenIdxForWrite, which re-acquires the same mutex and deadlocks
TestMarkVolumeReadonlyCanDelete. Split out reopenIdxForWriteLocked for
callers already holding the lock, matching the existing Locked pattern.

The worker task lost its worker-host source preference when the idx gate
replaced the replica union. Late-bind the source to a replica on the
worker host using only the checked location set, with no FileCount
rebinding. Drop the now-unused helpers, trim comments added by the
earlier commits, and rename tests whose names still claimed convergence
while asserting refusal.

---------

Co-authored-by: Chepurnov Simon <195125934+CHEPURNOV-S-S@users.noreply.github.com>
Co-authored-by: Devin <devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
This commit is contained in:
authored and GitHub committed 2026-10-19 00:56:17 -07:00
1 parent 10ce19cb17
commit d04d838ebd
13 files changed
+763 -200

No files matched your search

+5
View File
@@ -65,7 +65,12 @@ func loadVolumeWithoutWorker(dirname string, dirIdx string, collection string, i
func (v *Volume) reopenIdxForWrite() error {
v.dataFileAccessLock.Lock()
defer v.dataFileAccessLock.Unlock()
return v.reopenIdxForWriteLocked()
}
// reopenIdxForWriteLocked is reopenIdxForWrite for callers already holding
// dataFileAccessLock (MarkVolumeReadonly).
func (v *Volume) reopenIdxForWriteLocked() error {
oldNm, isSorted := v.nm.(*SortedFileNeedleMap)
if !isSorted {
return nil