mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-31 17:30:42 +01:00
80aad76986
* rust volume: remove the redb index when a failed VolumeCopy cleans up A copy that mounts and then fails validation is unmounted, not deleted. With a redb needle map the mount has already built <idx base>.rdb, and cleanup_failed_copy removed .dat/.idx/.vif/.note but not the .rdb, so the index database outlived the copy. A later copy of the same volume to that server reuses it (RedbNeedleMap::try_reuse_rdb keeps an .rdb whose recorded idx size is not larger than the new .idx and replays only the tail), which is wrong once the source has been compacted in between. Move the partial-file removal into CopyDestination::remove_partial_files and include the .rdb, next to the .idx where Volume::file_name puts it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Et288CnR1mxDyQNdYNMBNq * rust volume: clear a stale index before a copy writes, and report cleanup failures A crashed copy, or one whose partial-file removal failed, leaves a stale .rdb/.sdx next to the .idx; the next copy's mount would reuse it via try_reuse_rdb's tail replay. remove_partial_files now also covers .sdx and returns the first non-NotFound error: callers that can still abort (the pre-copy clear) do so, while the post-failure cleanup warns so a surviving index is visible instead of silently poisoning the next copy. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: restore the pre-copy leftover clear A leftover index check-in accidentally gated remove_partial_files behind a constant false, so a stale .rdb/.sdx at the destination survived the copy that was meant to clear it. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(volume): expect 409 when a read-only upload is refused x-seaweed-write-outcome: refused-before-append. The HTTP integration test still expected 500, so Rust unit tests failed on Linux and Windows. Deletes stay 500. * rust volume: keep a shared .vif, and clear a stale index beside the .dat remove_partial_files deleted <data>.vif even when a sealed .ecx still shares it, undoing the keep in Volume::destroy. It also cleared the index only under the index base, so a leftover .idx beside the .dat still made load trust that directory's .rdb. Skip .vif while a non-empty .ecx is present, and clear both bases when the index directory differs from the data directory. * rust volume: do not delete .dat files in the index directory The index directory can be another location's data directory. Clearing .dat and .note there removes that replica. Those files come off the data base only; .idx, .rdb, and .sdx are still cleared on both. * rust volume: serialize copies of a volume on one server A second VolumeCopy of the same volume could pass the existing-volume snapshot before the first copy mounts, then clear or overwrite the files the first copy still owns. Hold a per-volume claim for the whole copy, like ec_decodes_in_flight does for decodes, and fail a duplicate fast so the caller can retry. Mirrors the Go volumeCopiesInFlight guard. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: keep files when the mounted-copy delete is refused When a copy mounted and then failed, deleting through the store can be refused (e.g. the volume is compacting), leaving it registered. The cleanup then unlinked its files anyway, stranding a live replica. Keep the files while the volume is still in the store. Mirrors the Go copy cleanup. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: init volume_copies_in_flight in the http test state Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: refuse a mount while that volume is being copied volume_copy claims the vid, snapshots the store, then waits on the source. A VolumeMount in that window can load the replica, and the copy's pre-write cleanup then unlinks its files. Check the claim under the store write lock and return Unavailable. * rust volume: refuse configure while that volume is being copied volume_configure unmounts and remounts. A copy that already snapshotted the volume as absent can then unlink the files that remount loaded. Check the copy claim under the store write lock and return Unavailable. * rust volume: preserve files a copy displaces so a failed copy restores them * rust volume: keep an interrupted copy's .note with the files it marks displace_existing_files skipped the .note, so a crashed copy's leftovers were saved without the marker the load scan rejects them by, and remove_partial_files then deleted it. A retry that failed restored the partial set marker-free: after a restart the server could load an unfinished copy as a normal replica and serve wrong data. Displace the .note like every other file at the basename so the restore puts back the exact state the copy found. * rust volume: restore displaced files over a surviving partial restore_displaced_files dropped the backup whenever the basename was still occupied. When the partial cleanup could not remove every file the copy landed, an existing path just meant the removal failed — deleting the saved file destroyed the old replica and let the surviving partial pair with restored siblings of another generation. Clear the leftover first so the displaced file goes back, and keep the backup when it cannot: the operator can recover from copying/ rather than lose the replica outright. * rust volume: back up an EC-shared .vif the copy truncates in place A .vif a sealed .ecx shares was left out of the displaced set so the EC volume keeps it — but the incoming copy opens the same path with truncate, so a failed copy left the shared .vif overwritten by the rejected copy's settings with no way back. Go keeps the original via a hard link because its staged file swaps in by rename; truncating the shared inode makes a link useless here. Copy the content into copying/replaced_ like the other backups instead: the file stays in place for the EC volume during the copy, and the restore puts the original settings back on failure. * rust volume: serialize reload loads with the copy claim A SIGHUP reload scans every volume basename and sweeps .note-marked files as interrupted copies — including one a live VolumeCopy is still writing. Mirror Go's loadGuard: the copy-claim set is shared with each disk location, the scan skips a claimed vid outright and claims any other through its load, and reconcile_compact_states takes the same skip before rolling a leftover commit over the copy's files. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: hold the load claim through open_volumes The scan released each claim when its iteration ended, before the queued volume's index read ran in open_volumes. A VolumeCopy taking the vid in that gap could displace or remove files out from under Volume::new — the register the guard exists to prevent. Queued claims now live until the open pass returns; a second collection of an already-claimed vid merges into the entry instead of re-claiming. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: refuse allocation while that volume is being copied AllocateVolume registered a volume while a copy of it held the destination: create_volume loads the index before registration, and the copy then clears or rewrites files the live replica already opened, leaving it mounted on a stale index — the window VolumeMount and VolumeConfigure already refuse. Claim the volume id like the Go AllocateVolume does, so registration either wins the claim before the copy starts or sees it and refuses with Unavailable. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: refuse staged adoption while that volume is being copied The from_staged branch of volume_ec_shards_to_volume renames .copying files into place and mounts, all while a copy of the same vid could be rewriting the destination — the same registration window VolumeMount refuses. Hold the copy claim for the whole adoption, as Go's adoptStagedVolume does. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * rust volume: drop a partial .note when its write aborts the copy A write_note failure after displace_existing_files restores the saved replica but leaves the empty or partial marker it just made; the next load scan then deletes the restored replica as an unfinished copy. Remove the marker before restoring, as the Go side's error defer does. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: user.email <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: chrislusf <chris.lu@gmail.com>