mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-08 21:08:27 +01:00
f99cc6ca94
* volume.fsck: batch and deduplicate filer entry deletions In volume.fsck with -findMissingChunksInFiler and -reallyDeleteFilerEntries, deleting orphaned filer entries was slow and CPU intensive due to tiny worker channel buffers (128), lack of batch processing, redundant HTTP DELETE requests for multi-chunk files, and lock contention on parent directory tracking. Optimize by: - Adding startPurgeBatchWorkers with 4096-buffered channels to prevent producer stalls and batch opportunistic deletes up to 64 items. - Deduplicating missing chunks per file path during volume checks so that multi-chunk files are queued once. - Processing batches with in-batch path deduplication and aggregating parent directories before updating purgedDirs in a single lock acquisition. - Buffering temporary fid file writes (64KB) during the traversal to avoid millions of unbuffered syscalls. * volume.fsck: report a failed write of the filer fid files The fid files are written through a buffer now, so the last part of each one reaches the disk only in the deferred Flush. Its error was dropped: a full temp disk left a short fid file and the collection still succeeded, and the volume check would then take the chunks missing from it for orphans. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume.fsck: retry a batched path whose earlier delete failed A path was marked done before its delete ran, so a second copy of it in the same batch was skipped after a transient failure and the entry stayed. Only skip paths already deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume.fsck: keep reporting a failed DELETE response read The non-verbose path now discards the body without checking the copy, so a connection cut after the headers went unreported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * volume.fsck: keep one purge worker starter and one delete function startPurgeWorkers and the single-path httpDelete were left only as wrappers the tests called. Give their names to the batch versions and adapt the tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>