Junker der ProvinzandDan Foster e645449872 fix(storage): count preallocated space when auto-sizing max volumes (#11919) ...

The max volume count subtracted each volume's remaining room from the
free disk space, except when the master had preallocation turned on,
in which case it subtracted nothing. That is wrong for a disk that
mixes preallocated and plain volumes, and on platforms where
preallocation is not supported.

Use the larger of the .dat file's size and its allocated blocks, in the
Go and the Rust volume server, so a preallocated volume counts as full
and a plain one keeps its remaining room reserved.

Fixes #5778

Co-authored-by: Dan Foster <4570306+danfoster@users.noreply.github.com>

2026-10-21 15:11:06 -07:00

..

backend

volume: use FALLOC_FL_KEEP_SIZE and log preallocation failures (#11920)

2026-10-21 15:07:55 -07:00

erasure_coding

ec: re-plan balance moves when a target reports no space left (#11759)

2026-10-15 18:47:07 -07:00

idx

fix(scrub): don't flag offset-0 logical tombstones in volume scrub (#10148)

2026-06-30 02:01:14 -07:00

needle

fix(volume): validate sizes in ReadNeedleBlob and WriteNeedleBlob (#11399)

2026-09-19 21:28:37 -07:00

needle_map

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

super_block

perf(weed/storage/super_block): intern the byte-encoded replica placements (#10610)

2026-08-07 00:53:02 -07:00

test_files

Have volume scrubs account for zero-sized volumes. (#9609)

2026-05-21 09:42:07 -07:00

types

volume: resolve the disk IO slow-latency threshold per disk (#10976)

2026-08-27 10:01:05 -07:00

volume_info

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

volume_replica

ec: compare live idx NeedleId sets before encode (#11812)

2026-10-19 00:56:17 -07:00

dir_scan_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

dir_scan.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

disk_location_ec_realworld_test.go

EC bitrot detection: per-shard checksum sidecars (#9761)

2026-05-31 18:52:44 -07:00

disk_location_ec_shard_size_test.go

fix(volume_server): load orphan EC shards across disks on startup (#9212) (#9244)

2026-04-27 16:01:10 -07:00

disk_location_ec_test.go

volume server: ec.decode verifies, cleans up and compacts like Go, off the runtime (#11547)

2026-10-03 14:55:15 +08:00

disk_location_ec.go

volume server: ec.decode verifies, cleans up and compacts like Go, off the runtime (#11547)

2026-10-03 14:55:15 +08:00

disk_location_linux_test.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

disk_location_scan_order_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

disk_location_test.go

volume: rebuild a missing .idx from the .dat (#11115)

2026-09-03 08:43:10 -07:00

disk_location.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

ec_startup_shard_safety_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

io_error_other.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

io_error_windows.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

io_error.go

fix(volume): handle faulty storage media (Go + Rust) (#11233)

2026-09-08 21:42:56 -07:00

needle_map_file_pool_test.go

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

needle_map_file_pool.go

volume: open volume files with O_NOATIME (#11055)

2026-08-31 21:41:50 -07:00

needle_map_leveldb_test.go

fix(volume): persist the leveldb needle map watermark at batch boundaries (#10557)

2026-08-04 13:33:31 -07:00

needle_map_leveldb.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

needle_map_memory.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

needle_map_metric_test.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

needle_map_metric.go

storage: hash index-metric bloom filter with xxhash

2026-10-19 01:47:25 -07:00

needle_map_sorted_file_test.go

fix(volume): seed indexFileOffset in SortedFileNeedleMap so Delete appends (#9483)

2026-05-13 10:22:01 -07:00

needle_map_sorted_file.go

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

needle_map.go

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

remote_tier_integration_test.go

shell: keep volume.tier.compact replicas on the same remote object (#11783)

2026-10-16 19:45:17 -07:00

store_collection_metrics_test.go

volume: stop reporting read-only volumes that are no longer here (#10867)

2026-08-21 22:33:01 -07:00

store_consolidate_index_test.go

Mount req with collection (#11249)

2026-09-09 12:55:47 -07:00

store_delete_collection_test.go

heartbeat: keep the master current through collection churn (#10657)

2026-08-08 20:23:10 -07:00

store_delete_volume_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

store_disk_space_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_duplicate_vid_test.go

volume server: refuse the rest of a grouped run after a durable index failure (#11576)

2026-10-03 21:04:30 +08:00

store_ec_9plus3_reboot_repro_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_delete.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_disk_type_test.go

batch drain delta heartbeat messages (#9914)

2026-06-10 13:33:45 -07:00

store_ec_hybrid_repro_test.go

fix(ec): never delete recoverable EC shards on startup/reconcile (the non-empty-.dat sibling of the stub bug) (#9941)

2026-06-12 23:51:29 -07:00

store_ec_interval_read_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_journal_test.go

volume: merge .ecj as a set union on EC shard copy + index recovery (Rust+Go) (#11554)

2026-10-02 22:46:36 +08:00

store_ec_journal.go

volume: compact an oversized .ecj at mount, safely (Rust + Go) (#11555)

2026-10-03 09:08:19 +08:00

store_ec_location_cache_test.go

volume: route lookups to the leader master only (#11888)

2026-10-21 13:36:05 -07:00

store_ec_mirror_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_mirror.go

fix(ec): mirror EC sidecars onto every shard-bearing disk at startup (#9525)

2026-05-17 19:55:15 -07:00

store_ec_mount_cross_disk_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_multidisk_lifecycle_test.go

test: assert EC shard identity and empty-view in multi-disk lifecycle tests (#10723)

2026-08-12 20:18:39 -07:00

store_ec_orphan_shard_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_ec_phantom_dat_test.go

fix(storage): never let an empty .dat delete healthy distributed EC shards (#9930)

2026-06-11 20:26:20 -07:00

store_ec_reconcile.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

store_ec_reconstruct_test.go

ec: split the shard-interval recovery into a gather and a rebuild (#11005)

2026-08-28 15:55:12 -07:00

store_ec_recover_fanout_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_recover_test.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recover.go

ec: recover EC shards whose .ecx index lives only on a peer server (#10108)

2026-06-25 10:38:14 -07:00

store_ec_recovery_test.go

Update store_ec_recovery_test.go

2026-01-23 16:38:36 -08:00

store_ec_scrub_reads_test.go

ec: uniform shard block layout (#10932)

2026-08-28 20:46:59 -07:00

store_ec_scrub.go

volume: route lookups to the leader master only (#11888)

2026-10-21 13:36:05 -07:00

store_ec_target_location_test.go

ec: pin auto-selected shard placement to the disk that already owns the shard (#11029)

2026-08-29 12:12:41 -07:00

store_ec.go

volume: route lookups to the leader master only (#11888)

2026-10-21 13:36:05 -07:00

store_heartbeat_bench_test.go

volume: stop reporting read-only volumes that are no longer here (#10867)

2026-08-21 22:33:01 -07:00

store_heartbeat_digest_test.go

master: keep new volumes and writes off servers in maintenance mode (#11147)

2026-09-03 23:50:12 -07:00

store_load_balancing_simple_test.go

create new volumes on less occupied disk locations (#7349)

2025-10-20 16:11:29 -07:00

store_load_balancing_test.go

volume: cut idle memory at high volume counts (#10861)

2026-08-21 13:04:56 -07:00

store_mark_readonly_can_delete_test.go

Mount req with collection (#11249)

2026-09-09 12:55:47 -07:00

store_maxvolume_deadzone_test.go

[CheckDisk]: implement disk health detection (#9560)

2026-06-02 09:02:05 -07:00

store_maxvolume_preallocate_test.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

store_mount_validation_test.go

fix(volume): Harden Volume Copy Validation and Failure Handling (#11252)

2026-09-09 23:33:05 -07:00

store_state.go

chore: remove ~50k lines of unreachable dead code (#8913)

2026-04-03 16:04:27 -07:00

store_status_test.go

volume: forward fsync=true to replicas in ReplicatedWrite (#10805)

2026-08-18 17:17:50 -07:00

store_vacuum_fs_unix.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum_fs_windows.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum_test.go

vacuum: check compaction space against live bytes, not volume size (#11524)

2026-10-03 08:31:56 +08:00

store_vacuum.go

volume server: ec.decode verifies, cleans up and compacts like Go, off the runtime (#11547)

2026-10-03 14:55:15 +08:00

store_volume_report_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

store_volume_report.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

store.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

vid_stamp_test.go

Federated volume-id bound via heartbeat stamps (#11763) (#11819)

2026-10-19 17:01:19 -07:00

vid_stamp.go

Federated volume-id bound via heartbeat stamps (#11763) (#11819)

2026-10-19 17:01:19 -07:00

volume_allocated_unix.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

volume_allocated_windows.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00

volume_async_worker_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

volume_backup_test.go

refactoring

2019-04-17 22:04:49 -07:00

volume_backup.go

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

volume_checking_tail_repair_test.go

volume: repair torn tails on canDelete volumes; reject damaged 1-3 byte records (#11876)

2026-10-20 23:40:51 -07:00

volume_checking_test.go

volume: repair an unindexed .dat tail at load instead of loading read-only (#11870)

2026-10-20 13:03:00 -07:00

volume_checking.go

volume: repair torn tails on canDelete volumes; reject damaged 1-3 byte records (#11876)

2026-10-20 23:40:51 -07:00

volume_destroy_ec_vif_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

volume_idx_rebuild_test.go

volume: rebuild a missing .idx from the .dat (#11115)

2026-09-03 08:43:10 -07:00

volume_idx_rebuild.go

volume: rebuild a missing .idx from the .dat (#11115)

2026-09-03 08:43:10 -07:00

volume_idx_repair_test.go

volume: forward fsync=true to replicas in ReplicatedWrite (#10805)

2026-08-18 17:17:50 -07:00

volume_idx_repair.go

volume: recover .idx rows overwritten by tiered deletes (#10474)

2026-07-28 16:48:30 -07:00

volume_info_counts_test.go

storage: count a volume's needles in uint32 (#10718)

2026-08-11 16:36:36 -07:00

volume_info_intern_test.go

storage: stop keeping the remote storage key on the master (#10672)

2026-08-09 12:43:31 -07:00

volume_info_layout_test.go

storage: order VolumeInfo by alignment (#10669)

2026-08-09 09:41:50 -07:00

volume_info_test.go

master: keep disk_id when registering volumes from incremental heartbeats (#10686)

2026-08-10 00:40:29 -07:00

volume_info.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

volume_io_error_test.go

EC: handle zero-sized shard files uniformly (moves, rebuilds, startup cleanup) (#10753)

2026-08-13 21:38:22 -07:00

volume_loading_corrupt_idx_test.go

admin: allow setting volume read-only and read/write modes (#11217)

2026-09-07 18:40:37 -07:00

volume_loading_open_fail_test.go

volume: return error instead of panicking when .dat open fails (#11619)

2026-10-06 14:24:42 +08:00

volume_loading.go

s3tables, iceberg: log client refusals at V(1), not ERROR (#11896)

2026-10-21 14:00:33 -07:00

volume_mark_writable_test.go

volume tier: accept deletes on a volume as soon as it is tiered (#11894)

2026-10-21 14:01:10 -07:00

volume_read_all.go

[Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)

2026-09-24 06:57:44 +08:00

volume_read_test.go

fix(volume): skip unnecessary needle data reads on delete (#11711)

2026-10-12 18:08:19 +08:00

volume_read.go

fix(volume): skip unnecessary needle data reads on delete (#11711)

2026-10-12 18:08:19 +08:00

volume_remote_superblock_log_test.go

s3tables, iceberg: log client refusals at V(1), not ERROR (#11896)

2026-10-21 14:00:33 -07:00

volume_report_hash.go

volume: avoid read-only replica write targets (#11195)

2026-09-07 09:23:56 -07:00

volume_super_block_replication_test.go

test: pin that a .vif replication outranks the superblock (#10499)

2026-07-30 17:03:38 -07:00

volume_super_block.go

volume: open volume files with O_NOATIME (#11055)

2026-08-31 21:41:50 -07:00

volume_tier_deletable_test.go

volume tier: accept deletes on a volume as soon as it is tiered (#11894)

2026-10-21 14:01:10 -07:00

volume_tier_intern_test.go

volume: cut the memory a server holding millions of volumes still uses (#10999)

2026-08-27 22:25:15 -07:00

volume_tier.go

volume tier: accept deletes on a volume as soon as it is tiered (#11894)

2026-10-21 14:01:10 -07:00

volume_ttl_expiry_test.go

fix(volume): keep the TTL clock across a vacuum commit instead of rescanning (#11630)

2026-10-08 22:58:44 +08:00

volume_vacuum_5byte_offset_test.go

volume: fix 5-byte index offset corruption in makeupDiff (#11411)

2026-09-21 00:59:35 -07:00

volume_vacuum_crash_safe_test.go

shell: let volume.deleteEmpty drop volumes with no live needles (#11437)

2026-09-26 15:53:39 +08:00

volume_vacuum_test.go

storage: unify needle-map counters across index loaders and warn on copy-count drift (#11701)

2026-10-10 22:31:46 +08:00

volume_vacuum.go

volume: run the durable sync outside the reader-visible lock (#11863)

2026-10-20 12:58:02 -07:00

volume_write_fsync_test.go

volume: run the durable sync outside the reader-visible lock (#11863)

2026-10-20 12:58:02 -07:00

volume_write_test.go

ec: compare live idx NeedleId sets before encode (#11812)

2026-10-19 00:56:17 -07:00

volume_write.go

volume: run the durable sync outside the reader-visible lock (#11863)

2026-10-20 12:58:02 -07:00

volume.go

fix(storage): count preallocated space when auto-sizing max volumes (#11919)

2026-10-21 15:11:06 -07:00