3 people 5df7faf0e3 fix(storage): update deletion metrics in SortedFileNeedleMap.Delete (#12363) (#12364) ...

* fix(storage): update deletion metrics in SortedFileNeedleMap.Delete (#12363)

* storage: test canDelete volume counters across restarts

A canDelete volume reloaded from .sdx must count its deletes so
volume.deleteEmpty and vacuum see it drain, and the counters it reports
must match what the next reload computes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-03 08:05:45 -08:00

..

backend

volume: don't let an unreachable tier backend hold up volume server startup (#12312)

2026-11-01 18:35:37 -08:00

erasure_coding

ec.rebuild: remount shards the rebuild replaced, keep regenerated copies (#12193)

2026-10-29 00:58:30 -07:00

idx

fix(scrub): don't flag offset-0 logical tombstones in volume scrub (#10148)

2026-06-30 02:01:14 -07:00

needle

volume: replicas keep the primary's compression and metadata (#12350)

2026-11-02 11:24:51 -08:00

needle_map

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

super_block

Missing data read in super block extra parsing (#8392)

2026-10-21 20:07:52 -07:00

test_files

Have volume scrubs account for zero-sized volumes. (#9609)

2026-05-21 09:42:07 -07:00

types

volume: resolve the disk IO slow-latency threshold per disk (#10976)

2026-08-27 10:01:05 -07:00

volume_info

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

volume_replica

ec: compare live idx NeedleId sets before encode (#11812)

2026-10-19 00:56:17 -07:00

dir_scan_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

dir_scan.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

disk_location_ec_realworld_test.go

volume.fsck: run all filer-entry purges through the shared worker pool (#11960)

2026-10-23 18:33:51 -07:00

disk_location_ec_shard_size_test.go

fix(volume_server): load orphan EC shards across disks on startup (#9212) (#9244)

2026-04-27 16:01:10 -07:00

disk_location_ec_test.go

volume.copy: drop the needle-map index a failed copy leaves behind (#12252)

2026-10-30 09:20:55 -07:00

disk_location_ec.go

ec.rebuild: remount shards the rebuild replaced, keep regenerated copies (#12193)

2026-10-29 00:58:30 -07:00

disk_location_linux_test.go

fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)

2026-10-26 08:03:25 -07:00

disk_location_scan_order_test.go

volume.copy: drop the needle-map index a failed copy leaves behind (#12252)

2026-10-30 09:20:55 -07:00

disk_location_test.go

volume: rebuild a missing .idx from the .dat (#11115)

2026-09-03 08:43:10 -07:00

disk_location.go

volume.copy: drop the needle-map index a failed copy leaves behind (#12252)

2026-10-30 09:20:55 -07:00

ec_startup_shard_safety_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

io_error_other.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

io_error_windows.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

io_error.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

needle_map_file_pool_test.go

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

needle_map_file_pool.go

volume: open volume files with O_NOATIME (#11055)

2026-08-31 21:41:50 -07:00

needle_map_leveldb_test.go

fix(volume): persist the leveldb needle map watermark at batch boundaries (#10557)

2026-08-04 13:33:31 -07:00

needle_map_leveldb.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

needle_map_memory.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

needle_map_metric_test.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

needle_map_metric.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

needle_map_sorted_file_test.go

fix(storage): update deletion metrics in SortedFileNeedleMap.Delete (#12363) (#12364)

2026-11-03 08:05:45 -08:00

needle_map_sorted_file.go

fix(storage): update deletion metrics in SortedFileNeedleMap.Delete (#12363) (#12364)

2026-11-03 08:05:45 -08:00

needle_map.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

remote_tier_integration_test.go

shell: keep volume.tier.compact replicas on the same remote object (#11783)

2026-10-16 19:45:17 -07:00

store_collection_metrics_test.go

volume: stop reporting read-only volumes that are no longer here (#10867)

2026-08-21 22:33:01 -07:00

store_configure_volume_test.go

volume.configure: keep the superblock in sync with the .vif (#11930)

2026-10-21 20:18:56 -07:00

store_consolidate_index_test.go

Mount req with collection (#11249)

2026-09-09 12:55:47 -07:00

store_delete_collection_test.go

heartbeat: keep the master current through collection churn (#10657)

2026-08-08 20:23:10 -07:00

store_delete_volume_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

store_disk_space_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_duplicate_vid_test.go

volume server: refuse the rest of a grouped run after a durable index failure (#11576)

2026-10-03 21:04:30 +08:00

store_ec_9plus3_reboot_repro_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_delete.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_disk_type_test.go

batch drain delta heartbeat messages (#9914)

2026-06-10 13:33:45 -07:00

store_ec_hybrid_repro_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

store_ec_interval_read_test.go

ec: rebuild a needle's suspect shard ranges when it fails to parse (#12170)

2026-10-28 08:45:57 -07:00

store_ec_journal_test.go

volume: merge .ecj as a set union on EC shard copy + index recovery (Rust+Go) (#11554)

2026-10-02 22:46:36 +08:00

store_ec_journal.go

volume: compact an oversized .ecj at mount, safely (Rust + Go) (#11555)

2026-10-03 09:08:19 +08:00

store_ec_location_cache_test.go

volume: route lookups to the leader master only (#11888)

2026-10-21 13:36:05 -07:00

store_ec_mirror_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_mirror.go

fix(ec): mirror EC sidecars onto every shard-bearing disk at startup (#9525)

2026-05-17 19:55:15 -07:00

store_ec_mount_cross_disk_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_multidisk_lifecycle_test.go

test: assert EC shard identity and empty-view in multi-disk lifecycle tests (#10723)

2026-08-12 20:18:39 -07:00

store_ec_orphan_shard_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_phantom_dat_test.go

fix(storage): never let an empty .dat delete healthy distributed EC shards (#9930)

2026-06-11 20:26:20 -07:00

store_ec_rebuild_remount_test.go

ec.rebuild: remount shards the rebuild replaced, keep regenerated copies (#12193)

2026-10-29 00:58:30 -07:00

store_ec_reconcile.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

store_ec_reconstruct_test.go

ec: split the shard-interval recovery into a gather and a rebuild (#11005)

2026-08-28 15:55:12 -07:00

store_ec_recover_fanout_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_recover_test.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recover.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recovery_test.go

Update store_ec_recovery_test.go

2026-01-23 16:38:36 -08:00

store_ec_scrub_reads_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_scrub.go

volume: route lookups to the leader master only (#11888)

2026-10-21 13:36:05 -07:00

store_ec_target_location_test.go

ec: pin auto-selected shard placement to the disk that already owns the shard (#11029)

2026-08-29 12:12:41 -07:00

store_ec.go

ec.rebuild: remount shards the rebuild replaced, keep regenerated copies (#12193)

2026-10-29 00:58:30 -07:00

store_headroom_test.go

fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)

2026-10-26 08:03:25 -07:00

store_heartbeat_bench_test.go

volume: stop reporting read-only volumes that are no longer here (#10867)

2026-08-21 22:33:01 -07:00

store_heartbeat_digest_test.go

master: keep new volumes and writes off servers in maintenance mode (#11147)

2026-09-03 23:50:12 -07:00

store_load_balancing_simple_test.go

create new volumes on less occupied disk locations (#7349)

2025-10-20 16:11:29 -07:00

store_load_balancing_test.go

volume: cut idle memory at high volume counts (#10861)

2026-08-21 13:04:56 -07:00

store_mark_readonly_can_delete_test.go

fix(storage): update deletion metrics in SortedFileNeedleMap.Delete (#12363) (#12364)

2026-11-03 08:05:45 -08:00

store_maxvolume_deadzone_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_maxvolume_preallocate_test.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

store_mount_validation_test.go

fix(volume): Harden Volume Copy Validation and Failure Handling (#11252)

2026-09-09 23:33:05 -07:00

store_state.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

store_status_test.go

volume: forward fsync=true to replicas in ReplicatedWrite (#10805)

2026-08-18 17:17:50 -07:00

store_vacuum_fs_unix.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum_fs_windows.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum_test.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum.go

volume server: ec.decode verifies, cleans up and compacts like Go, off the runtime (#11547)

2026-10-03 14:55:15 +08:00

store_volume_report_test.go

master: deleting a read-only volume no longer drifts the active count (#12285)

2026-10-31 13:28:32 -07:00

store_volume_report.go

master: deleting a read-only volume no longer drifts the active count (#12285)

2026-10-31 13:28:32 -07:00

store.go

volume: serve a ranged GET without reading the whole needle (#12301)

2026-11-01 12:00:22 -08:00

vid_stamp_test.go

Federated volume-id bound via heartbeat stamps (#11763) (#11819)

2026-10-19 17:01:19 -07:00

vid_stamp.go

master, volume, filer: start on a read-only mounted directory and serve existing data read-only (#12081)

2026-10-26 09:28:49 -07:00

volume_allocated_unix.go

fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)

2026-10-26 08:03:25 -07:00

volume_allocated_windows.go

fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)

2026-10-26 08:03:25 -07:00

volume_async_worker_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

volume_backup_test.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

volume_backup.go

backup: carry index-only tombstones through incremental copies (#12231)

2026-10-29 22:37:31 -07:00

volume_checking_tail_repair_test.go

volume: repair torn tails on canDelete volumes; reject damaged 1-3 byte records (#11876)

2026-10-20 23:40:51 -07:00

volume_checking_test.go

volume: repair an unindexed .dat tail at load instead of loading read-only (#11870)

2026-10-20 13:03:00 -07:00

volume_checking.go

volume: repair torn tails on canDelete volumes; reject damaged 1-3 byte records (#11876)

2026-10-20 23:40:51 -07:00

volume_destroy_ec_vif_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

volume_idx_rebuild_test.go

volume: rebuild a missing .idx from the .dat (#11115)

2026-09-03 08:43:10 -07:00

volume_idx_rebuild.go

master, volume, filer: start on a read-only mounted directory and serve existing data read-only (#12081)

2026-10-26 09:28:49 -07:00

volume_idx_repair_test.go

volume: drop stale index entries that resolve to a different needle (#11948)

2026-10-23 02:30:53 -07:00

volume_idx_repair.go

volume: recover .idx rows overwritten by tiered deletes (#10474)

2026-07-28 16:48:30 -07:00

volume_info_counts_test.go

storage: count a volume's needles in uint32 (#10718)

2026-08-11 16:36:36 -07:00

volume_info_intern_test.go

storage: stop keeping the remote storage key on the master (#10672)

2026-08-09 12:43:31 -07:00

volume_info_layout_test.go

storage: order VolumeInfo by alignment (#10669)

2026-08-09 09:41:50 -07:00

volume_info_test.go

master: keep disk_id when registering volumes from incremental heartbeats (#10686)

2026-08-10 00:40:29 -07:00

volume_info.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

volume_io_error_test.go

EC: handle zero-sized shard files uniformly (moves, rebuilds, startup cleanup) (#10753)

2026-08-13 21:38:22 -07:00

volume_loading_corrupt_idx_test.go

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

volume_loading_open_fail_test.go

volume: return error instead of panicking when .dat open fails (#11619)

2026-10-06 14:24:42 +08:00

volume_loading.go

volume: don't let an unreachable tier backend hold up volume server startup (#12312)

2026-11-01 18:35:37 -08:00

volume_mark_writable_test.go

volume tier: accept deletes on a volume as soon as it is tiered (#11894)

2026-10-21 14:01:10 -07:00

volume_read_all.go

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

volume_read_test.go

volume: serve a ranged GET without reading the whole needle (#12301)

2026-11-01 12:00:22 -08:00

volume_read.go

volume: serve a ranged GET without reading the whole needle (#12301)

2026-11-01 12:00:22 -08:00

volume_remote_superblock_log_test.go

s3tables, iceberg: log client refusals at V(1), not ERROR (#11896)

2026-10-21 14:00:33 -07:00

volume_remote_superblock_stall_test.go

volume: don't let an unreachable tier backend hold up volume server startup (#12312)

2026-11-01 18:35:37 -08:00

volume_report_hash.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

volume_super_block_replication_test.go

test: pin that a .vif replication outranks the superblock (#10499)

2026-07-30 17:03:38 -07:00

volume_super_block.go

volume: don't let an unreachable tier backend hold up volume server startup (#12312)

2026-11-01 18:35:37 -08:00

volume_tier_attach_test.go

volume: never compact a volume tiered to remote storage (#11895)

2026-10-21 20:16:02 -07:00

volume_tier_deletable_test.go

volume tier: accept deletes on a volume as soon as it is tiered (#11894)

2026-10-21 14:01:10 -07:00

volume_tier_intern_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

volume_tier.go

volume: never compact a volume tiered to remote storage (#11895)

2026-10-21 20:16:02 -07:00

volume_ttl_expiry_test.go

fix(volume): keep the TTL clock across a vacuum commit instead of rescanning (#11630)

2026-10-08 22:58:44 +08:00

volume_vacuum_5byte_offset_test.go

volume: fix 5-byte index offset corruption in makeupDiff (#11411)

2026-09-21 00:59:35 -07:00

volume_vacuum_crash_safe_test.go

volume.copy: drop the needle-map index a failed copy leaves behind (#12252)

2026-10-30 09:20:55 -07:00

volume_vacuum_test.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

volume_vacuum_tiered_test.go

volume: never compact a volume tiered to remote storage (#11895)

2026-10-21 20:16:02 -07:00

volume_vacuum.go

volume: never compact a volume tiered to remote storage (#11895)

2026-10-21 20:16:02 -07:00

volume_write_fsync_test.go

volume: run the durable sync outside the reader-visible lock (#11863)

2026-10-20 12:58:02 -07:00

volume_write_test.go

volume: keep a below-minimum fsync batch open for stragglers (#12016)

2026-10-24 07:04:05 -07:00

volume_write.go

fix(volume): return conflict and explicit proof for read-only upload refusals (#12236)

2026-10-29 23:33:56 -07:00

volume.go

fix(volume): keep preallocated growth above -volume.minFreeSpace (#12067)

2026-10-26 08:03:25 -07:00