Nguyễn Đăng Minh LựcandChris Lu 71f8128d75 shell: fs.verify -pruneEntries deletes entries whose needles are lost (#11338) ...

* shell: fs.verify -pruneEntries deletes entries whose needles are lost

* shell: harden fs.verify -pruneEntries guards; VolumeNeedleStatus returns NotFound for absent needles

* shell: resolve chunk manifests in fs.verify metadata path; require confirmed deletion before counting prunes

* shell: anchor fs.verify legacy missing-needle error matching

* shell: keep fs.verify metadata scan alive on manifest resolution failures

* shell: classify EC missing needles and keep manifest failures unverified

VolumeNeedleStatus now canonicalizes erasure_coding.NotFoundError to
codes.NotFound, so absent needles in EC volumes reach the prune path
through the same stable contract as regular volumes. The client-side
isNeedleMissingError keeps recognizing the legacy wrapped EC shape
("locate in local ec volume: ... needle not found") for mixed-version
clusters.

A chunk manifest that fails to resolve is now an entry-level
verification failure even when the raw top-level chunks are healthy:
the file is not fully readable without the manifest. Raw chunks are
still verified on a resolution failure so a missing top-level manifest
needle is classified and can be pruned. The per-entry logic is
extracted into resolveAndVerify for testability.

* shell: trim fs.verify prune comments

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>

2026-09-15 20:42:18 -07:00

..

constants

Nit: use time.Durations instead of constants in seconds. (#7438)

2025-11-04 13:02:22 -08:00

filer_ui

Fix UI prefix url encoding (#9344)

2026-05-06 19:14:36 -07:00

master_ui

feat: improve aio support for admin/volume ingress and fix UI links (#8679)

2026-03-18 13:20:55 -07:00

postgres

fix(postgres): prevent uint32 underflow & OOM in message parsing (#10099)

2026-06-24 20:05:43 -07:00

volume_server_ui

fix: EC UI template error when viewing shard details (#7955)

2026-01-03 22:45:48 -08:00

common_test.go

master: honour -volume.fileSizeLimitMB on the master's /submit (#11176)

2026-09-05 12:58:17 -07:00

common.go

master: honour -volume.fileSizeLimitMB on the master's /submit (#11176)

2026-09-05 12:58:17 -07:00

filer_grpc_server_admin.go

wdclient: bound the wait for a master leader by the caller's context (#11002)

2026-08-27 22:27:45 -07:00

filer_grpc_server_batch_lookup_test.go

filer: batch exact lookup RPC, authoritative volume lookup, VolumeDelete status codes (#11122)

2026-09-03 14:52:05 -07:00

filer_grpc_server_batch_lookup.go

filer: batch exact lookup RPC, authoritative volume lookup, VolumeDelete status codes (#11122)

2026-09-03 14:52:05 -07:00

filer_grpc_server_chunk_condition_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_condition_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_condition.go

filer: conditional UpdateEntry with a chunk-set write condition (#10382)

2026-07-21 00:19:15 -07:00

filer_grpc_server_create_lock_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_create_route_test.go

filer: route exclusive and conditional creates to the entry's ring owner (#11109)

2026-09-02 19:56:11 -07:00

filer_grpc_server_dlm_test.go

dlm: resilient distributed locks via consistent hashing + backup replication (#8860)

2026-03-30 23:29:56 -07:00

filer_grpc_server_dlm.go

dlm: resilient distributed locks via consistent hashing + backup replication (#8860)

2026-03-30 23:29:56 -07:00

filer_grpc_server_kv.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

filer_grpc_server_mount_peer_test.go

chore(filer): remove -mount.p2p flag; registry is always on (#9183)

2026-04-21 23:00:11 -07:00

filer_grpc_server_mount_peer.go

chore(filer): remove -mount.p2p flag; registry is always on (#9183)

2026-04-21 23:00:11 -07:00

filer_grpc_server_object_txn_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_object_txn_ttl_test.go

[s3] force filer apply s3 expiry metadata (#10469)

2026-08-05 23:55:20 -07:00

filer_grpc_server_posix_lock_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_posix_lock.go

filer: warm-up + fail-closed cooling for POSIX locks on owner (re)start (#9673)

2026-05-25 13:14:05 -07:00

filer_grpc_server_recompute_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_remote.go

read cold remote objects straight from the origin while caching (#10731)

2026-08-12 23:00:10 -07:00

filer_grpc_server_rename_test.go

filer: do not 404 a TUS session on a transient chunk-load failure (#11153)

2026-09-04 16:39:24 -07:00

filer_grpc_server_rename.go

filer: self-heal chunk manifest reads when volume locations go stale (#11107)

2026-09-02 17:43:46 -07:00

filer_grpc_server_route_test.go

filer: keep a moved key on its prior owner while the ring settles (#11108)

2026-09-02 19:40:53 -07:00

filer_grpc_server_route.go

filer: route exclusive and conditional creates to the entry's ring owner (#11109)

2026-09-02 19:56:11 -07:00

filer_grpc_server_statistics_test.go

mount: report data sizes to df with -df.logical (#10459)

2026-07-27 14:28:29 -07:00

filer_grpc_server_stream_mutate_bench_test.go

perf(filer): parallelize StreamMutateEntry with path-keyed scheduler (#9171)

2026-04-21 11:25:09 -07:00

filer_grpc_server_stream_mutate_scheduler_test.go

perf(filer): parallelize StreamMutateEntry with path-keyed scheduler (#9171)

2026-04-21 11:25:09 -07:00

filer_grpc_server_stream_mutate_scheduler.go

perf(filer): parallelize StreamMutateEntry with path-keyed scheduler (#9171)

2026-04-21 11:25:09 -07:00

filer_grpc_server_stream_mutate.go

perf(filer): parallelize StreamMutateEntry with path-keyed scheduler (#9171)

2026-04-21 11:25:09 -07:00

filer_grpc_server_sub_meta_gap_test.go

filer: end local-only metadata subscriptions when remote peers appear (#11251)

2026-09-09 20:16:50 -07:00

filer_grpc_server_sub_meta_prefilter_test.go

Cut per-subscriber replay decode and widen replay concurrency (#9917)

2026-06-10 13:08:34 -07:00

filer_grpc_server_sub_meta_test.go

Cut per-subscriber replay decode and widen replay concurrency (#9917)

2026-06-10 13:08:34 -07:00

filer_grpc_server_sub_meta_watermark_test.go

fix(filer): bound aggregated metadata reads by peer watermarks (#10803)

2026-08-19 18:38:46 -07:00

filer_grpc_server_sub_meta.go

filer: end local-only metadata subscriptions when remote peers appear (#11251)

2026-09-09 20:16:50 -07:00

filer_grpc_server_subscribers.go

admin: add connected Mount Clients page and dashboard section (#9968)

2026-06-14 21:44:10 -07:00

filer_grpc_server_test.go

fix(filer): apply default disk type after location-prefix resolution in gRPC AssignVolume (#8836)

2026-03-29 14:18:24 -07:00

filer_grpc_server_traverse_meta_test.go

Stop the filer test helpers from pinning gigabytes of log buffers (#10560)

2026-08-04 11:38:38 -07:00

filer_grpc_server_traverse_meta.go

filer: bound TraverseBfsMetadata memory by queuing directory paths (#9814)

2026-06-03 10:28:42 -07:00

filer_grpc_server.go

filer: guard FoundationDB 100KB value limit and pack earlier (#11161)

2026-09-04 23:12:27 -07:00

filer_jwt_test.go

filer: authorize the chunk proxy and the root listing like the rest of the filer port (#11152)

2026-09-04 16:39:36 -07:00

filer_pipelined_sender_test.go

fix(filer): stop skipping recent unflushed events on metadata subscription gaps (#10501)

2026-08-02 00:06:22 -07:00

filer_server_handlers_copy_test.go

Use filer-side copy for mounted whole-file copy_file_range (#8747)

2026-03-23 18:35:15 -07:00

filer_server_handlers_copy.go

filer: self-heal chunk manifest reads when volume locations go stale (#11107)

2026-09-02 17:43:46 -07:00

filer_server_handlers_iam_grpc_test.go

filer/iam-grpc: make admin Bearer auth opt-in (fixes #9509) (#9514)

2026-05-15 13:15:20 -07:00

filer_server_handlers_iam_grpc.go

filer/iam-grpc: make admin Bearer auth opt-in (fixes #9509) (#9514)

2026-05-15 13:15:20 -07:00

filer_server_handlers_proxy_test.go

filer: authorize the chunk proxy and the root listing like the rest of the filer port (#11152)

2026-09-04 16:39:36 -07:00

filer_server_handlers_proxy.go

filer: authorize the chunk proxy and the root listing like the rest of the filer port (#11152)

2026-09-04 16:39:36 -07:00

filer_server_handlers_read_dir.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

filer_server_handlers_read.go

make the remote-mount cache wait configurable per mount (#11168)

2026-09-04 23:50:11 -07:00

filer_server_handlers_tagging.go

filer: reuse the caller's fetched entry in CreateEntry (#9645)

2026-05-23 21:40:41 -07:00

filer_server_handlers_write_autochunk.go

filer: guard FoundationDB 100KB value limit and pack earlier (#11161)

2026-09-04 23:12:27 -07:00

filer_server_handlers_write_merge.go

filer: self-heal chunk manifest reads when volume locations go stale (#11107)

2026-09-02 17:43:46 -07:00

filer_server_handlers_write_upload.go

filer: stop TUS uploads from turning into garbage (#10945)

2026-08-25 09:24:51 -07:00

filer_server_handlers_write.go

filer: serve "//" paths at the cleaned path instead of redirecting (#11150)

2026-09-04 00:02:33 -07:00

filer_server_handlers.go

filer: authorize the chunk proxy and the root listing like the rest of the filer port (#11152)

2026-09-04 16:39:36 -07:00

filer_server_rocksdb.go

go fix

2026-02-20 18:42:00 -08:00

filer_server_storage_rule_ttl_test.go

filer: serve "//" paths at the cleaned path instead of redirecting (#11150)

2026-09-04 00:02:33 -07:00

filer_server_tus_complete_test.go

filer: apply the path's storage rule TTL on every write path (#10963)

2026-08-26 08:49:25 -07:00

filer_server_tus_concat_test.go

filer: TUS concatenation extension (#10702)

2026-08-10 12:32:45 -07:00

filer_server_tus_handlers.go

filer: do not 404 a TUS session on a transient chunk-load failure (#11153)

2026-09-04 16:39:24 -07:00

filer_server_tus_idor_test.go

filer: configurable TUS max upload size and session expiry (#10638)

2026-08-07 21:56:15 -07:00

filer_server_tus_jwt_test.go

filer: TUS concatenation extension (#10702)

2026-08-10 12:32:45 -07:00

filer_server_tus_lock_test.go

filer: stop TUS uploads from turning into garbage (#10945)

2026-08-25 09:24:51 -07:00

filer_server_tus_session.go

filer: apply the path's storage rule TTL on every write path (#10963)

2026-08-26 08:49:25 -07:00

filer_server_tus_transient_test.go

filer: do not 404 a TUS session on a transient chunk-load failure (#11153)

2026-09-04 16:39:24 -07:00

filer_server.go

filer: apply SSRF guard to the lazy-remote fetch/list/delete paths (#11294)

2026-09-13 14:43:55 -07:00

filer_subscribe_agg_test.go

filer: stop aggregated metadata subscribers from spinning on a peer watermark hold (#10863)

2026-08-21 15:22:05 -07:00

filer_subscribe_loop_test.go

log_buffer: end bounded reads that find the buffer empty (#10750)

2026-08-13 13:25:52 -07:00

filer_subscribe_upgrade_test.go

filer: end local-only metadata subscriptions when remote peers appear (#11251)

2026-09-09 20:16:50 -07:00

master_grpc_server_admin_ping_test.go

master: accept volume-server Ping targets on follower masters (#9614)

2026-05-21 10:19:59 -07:00

master_grpc_server_admin.go

shell: show who holds the cluster lock (#10353)

2026-07-17 12:30:42 -07:00

master_grpc_server_assign_test.go

master: name the unlabeled disk layout plainly in assign errors (#11290)

2026-09-13 11:56:32 -07:00

master_grpc_server_assign.go

master: name the unlabeled disk layout plainly in assign errors (#11290)

2026-09-13 11:56:32 -07:00

master_grpc_server_changed_volumes_test.go

expose whether a volume replica is backed by remote storage, and prefer local replicas (#11105)

2026-09-02 16:12:46 -07:00

master_grpc_server_changed_volumes_tier_test.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

master_grpc_server_cluster.go

chore: execute goimports to format the code (#7983)

2026-01-07 13:06:08 -08:00

master_grpc_server_collection_test.go

master, filer, s3api: bound the collection deletes that strand a caller (#11026)

2026-08-28 16:32:30 -07:00

master_grpc_server_collection.go

master, filer, s3api: bound the collection deletes that strand a caller (#11026)

2026-08-28 16:32:30 -07:00

master_grpc_server_digest_test.go

heartbeat: send only the volumes that changed (#10640)

2026-08-07 23:36:28 -07:00

master_grpc_server_raft_auth_test.go

master: gate raft membership RPCs behind the admin whitelist (#10649)

2026-08-08 09:14:57 -07:00

master_grpc_server_raft_test.go

Add cluster.raft.leader.transfer command for graceful leader change (#7819)

2025-12-19 00:15:39 -08:00

master_grpc_server_raft.go

master: let the leader admit a master that starts with no raft state (#10865)

2026-08-21 15:22:22 -07:00

master_grpc_server_statistics_test.go

master: cap the reported capacity at what the disks hold (#10960)

2026-08-26 00:12:56 -07:00

master_grpc_server_test.go

master: don't let a dead KeepConnected handler close its successor's channel (#10900)

2026-08-23 11:36:00 -07:00

master_grpc_server_volume_move_test.go

expose whether a volume replica is backed by remote storage, and prefer local replicas (#11105)

2026-09-02 16:12:46 -07:00

master_grpc_server_volume_warmup_test.go

filer: batch exact lookup RPC, authoritative volume lookup, VolumeDelete status codes (#11122)

2026-09-03 14:52:05 -07:00

master_grpc_server_volume.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

master_grpc_server.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

master_server_handlers_admin.go

master: honour -volume.fileSizeLimitMB on the master's /submit (#11176)

2026-09-05 12:58:17 -07:00

master_server_handlers_lookup_test.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

master_server_handlers_ui.go

hide millseconds in up time (#7553)

2025-11-26 08:01:19 -08:00

master_server_handlers.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

master_server.go

fix(master): stop goraft server on shutdown and bump raft to v1.2.1 (#11284)

2026-09-11 22:25:05 -07:00

raft_common.go

fix: improve raft leader election reliability and failover speed (#8692)

2026-03-18 23:28:07 -07:00

raft_hashicorp_test.go

master: let the leader admit a master that starts with no raft state (#10865)

2026-08-21 15:22:22 -07:00

raft_hashicorp.go

master: never re-seed a raft cluster over committed state under -raftBootstrap (#10883)

2026-08-23 11:10:20 -07:00

raft_server_handlers.go

master: let the leader admit a master that starts with no raft state (#10865)

2026-08-21 15:22:22 -07:00

raft_server.go

master: never re-seed a raft cluster over committed state under -raftBootstrap (#10883)

2026-08-23 11:10:20 -07:00

volume_grpc_admin_auth_coverage_test.go

volume: widen the gRPC admin gate and stop it drifting (#10443)

2026-07-25 23:53:29 -07:00

volume_grpc_admin_delete_test.go

filer: batch exact lookup RPC, authoritative volume lookup, VolumeDelete status codes (#11122)

2026-09-03 14:52:05 -07:00

volume_grpc_admin.go

shell: fs.verify -pruneEntries deletes entries whose needles are lost (#11338)

2026-09-15 20:42:18 -07:00

volume_grpc_batch_delete_ec_test.go

volume: an already-deleted EC needle is not a delete failure (#11071)

2026-09-01 10:37:25 -07:00

volume_grpc_batch_delete_test.go

volume: require admin auth on BatchDelete (#9438)

2026-05-11 13:50:48 -07:00

volume_grpc_batch_delete.go

volume: an already-deleted EC needle is not a delete failure (#11071)

2026-09-01 10:37:25 -07:00

volume_grpc_client_to_master_test.go

volume server: route VolumeMarkReadonly to raft leader (#10120)

2026-06-26 14:22:57 -07:00

volume_grpc_client_to_master.go

heartbeat: send only the volumes that changed (#10640)

2026-08-07 23:36:28 -07:00

volume_grpc_copy_count_test.go

[Volume] Validate record counts after volume copy (#11238)

2026-09-09 02:18:18 -07:00

volume_grpc_copy_incremental.go

move to https://github.com/seaweedfs/seaweedfs

2022-07-29 00:17:28 -07:00

volume_grpc_copy_traversal_test.go

volume: validate the file extension in CopyFile and ReceiveFile (#10644)

2026-08-08 09:25:57 -07:00

volume_grpc_copy_verify_test.go

fix(volume): Harden Volume Copy Validation and Failure Handling (#11252)

2026-09-09 23:33:05 -07:00

volume_grpc_copy_writefile_test.go

EC bitrot follow-ups: protect destination sidecar on optional copy; cap sidecar block_size (#9763)

2026-05-31 23:42:33 -07:00

volume_grpc_copy.go

fix(volume): Harden Volume Copy Validation and Failure Handling (#11252)

2026-09-09 23:33:05 -07:00

volume_grpc_ec_generation_fence_test.go

fix(ec): fence stale-worker EC shard cleanup by encode generation (#9953)

2026-06-14 01:54:04 -07:00

volume_grpc_ec_shared_index_test.go

ec: remove .ecsum sidecars on destroy / shard delete; align Go and Rust cleanup (#10307)

2026-07-10 22:06:36 -07:00

volume_grpc_erasure_coding_dirs_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

volume_grpc_erasure_coding_recover_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

volume_grpc_erasure_coding_recover.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

volume_grpc_erasure_coding_test.go

ec: remove .ecsum sidecars on destroy / shard delete; align Go and Rust cleanup (#10307)

2026-07-10 22:06:36 -07:00

volume_grpc_erasure_coding.go

Mount req with collection (#11249)

2026-09-09 12:55:47 -07:00

volume_grpc_maintenance_test.go

volume: let evacuation proceed on a server in maintenance mode (#11145)

2026-09-03 18:39:45 -07:00

volume_grpc_query.go

fix(volume): reject overflowing needle ID deltas (#10342)

2026-07-15 23:24:04 -07:00

volume_grpc_read_all.go

volume: require admin auth on ReadAllNeedles and VolumeNeedleStatus (#9437)

2026-05-11 13:50:19 -07:00

volume_grpc_read_write.go

Export gRPC file_{read,write}_failures metrics on volume servers. (#9177)

2026-04-22 11:22:21 -07:00

volume_grpc_remote_test.go

filer: apply SSRF guard to the lazy-remote fetch/list/delete paths (#11294)

2026-09-13 14:43:55 -07:00

volume_grpc_remote.go

filer: apply SSRF guard to the lazy-remote fetch/list/delete paths (#11294)

2026-09-13 14:43:55 -07:00

volume_grpc_scrub_ec_merge_test.go

[Volume] Scrub every disk's EC shards for a volume id, not just the first (#11258)

2026-09-12 14:19:18 -07:00

volume_grpc_scrub_test.go

volume server: release the store lock before scrubbing EC volumes (#11235)

2026-09-08 19:12:30 -07:00

volume_grpc_scrub.go

[Volume] Scrub every disk's EC shards for a volume id, not just the first (#11258)

2026-09-12 14:19:18 -07:00

volume_grpc_state.go

volume: widen the gRPC admin gate and stop it drifting (#10443)

2026-07-25 23:53:29 -07:00

volume_grpc_tail.go

volume: widen the gRPC admin gate and stop it drifting (#10443)

2026-07-25 23:53:29 -07:00

volume_grpc_tier_download_test.go

Cloud/remote storage & tiering: configurable multipart upload/download concurrency (#11319)

2026-09-14 22:09:08 -07:00

volume_grpc_tier_download.go

Cloud/remote storage & tiering: configurable multipart upload/download concurrency (#11319)

2026-09-14 22:09:08 -07:00

volume_grpc_tier_test.go

Cloud/remote storage & tiering: configurable multipart upload/download concurrency (#11319)

2026-09-14 22:09:08 -07:00

volume_grpc_tier_upload.go

Cloud/remote storage & tiering: configurable multipart upload/download concurrency (#11319)

2026-09-14 22:09:08 -07:00

volume_grpc_vacuum.go

fix(volume): add authentication to destructive gRPC admin endpoints (#8876)

2026-05-04 21:14:55 -07:00

volume_server_handlers_admin.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

volume_server_handlers_helper_test.go

server: reject a Range start offset equal to the file size (#10898)

2026-08-23 08:20:25 -07:00

volume_server_handlers_helper.go

server: 416 only when no requested range overlaps, with Content-Range, and the Rust mirror (#10889)

2026-08-23 11:13:36 -07:00

volume_server_handlers_read.go

Export file_read_invalid_needles metric for REST read requests on invalid file IDs. (#9241)

2026-04-27 12:22:42 -07:00

volume_server_handlers_ui.go

storage: stop keeping the remote storage key on the master (#10672)

2026-08-09 12:43:31 -07:00

volume_server_handlers_write.go

volume: an already-deleted EC needle is not a delete failure (#11071)

2026-09-01 10:37:25 -07:00

volume_server_handlers.go

security: hot-reload JWT signing keys on SIGHUP (#9826)

2026-06-04 22:26:08 -07:00

volume_server_parse_path_test.go

fix(volume): reject overflowing needle ID deltas (#10342)

2026-07-15 23:24:04 -07:00

volume_server.go

security: hot-reload JWT signing keys on SIGHUP (#9826)

2026-06-04 22:26:08 -07:00

webdav_listed_entries_test.go

webdav: answer PROPFIND child stats from the listing (#10492)

2026-07-30 11:58:14 -07:00

webdav_listed_entries.go

webdav: answer PROPFIND child stats from the listing (#10492)

2026-07-30 11:58:14 -07:00

webdav_server_test.go

filer: clean up manifest resolve error propagation and add webdav tes… (#11297)

2026-09-13 18:34:09 -07:00

webdav_server.go

filer: clean up manifest resolve error propagation and add webdav tes… (#11297)

2026-09-13 18:34:09 -07:00

wrapped_webdav_fs_test.go

[s3] validate indirect filer path inputs (#9931)

2026-06-11 21:56:16 -07:00

wrapped_webdav_fs.go

webdav: describe a listed entry the way clients expect (#10993)

2026-08-27 16:21:14 -07:00