Commit Graph
31 Commits
Author SHA1 Message Date
ssshr-66andChris Lu b9ad62fc16 [Volume] Keep DAT and index state consistent after async batch Sync failure (#11425)
* fix 11400

* persist failed-recovery quarantine and harden rollback

- record the unavailable state in a .unavailable marker, fsync it, and
  re-arm it on load so a restart cannot serve an unverified pair
- quarantine the volume so heartbeats stop advertising it
- block MarkVolumeWritable while unavailable, rechecked under noWriteLock
- fail every request of a failed batch, not only the succeeded ones
- restore the needle map and truncate .dat on inline fsync rollback failure
- add truncateIndex for the sorted-file needle map
- mirror the fail-closed semantics in the Rust volume server

* volume: erase rolled-back mappings instead of leaving tombstones

A rolled-back batch or failed inline write used Delete() to undo a
needle that did not exist beforehand, leaving a tombstoned map entry
whose stale offset makes the next write to that needle fail reading a
header that no longer exists. Add removeMapping/restoreMapping to the
mappers so recovery erases entries that were absent before the batch
and reinstates the exact prior offset/size for ones that were,
including tombstones. The index row still goes through Delete so a
replay forgets the needle.

* volume: gate bulk readers on unavailable and fsync the marker's dir

- fsync_dir(&self.dir) synced the volume dir's parent, not the dir
  holding .unavailable; pass the marker path so the create survives
  a host crash
- export UnavailableError and check it in ReadAllNeedles,
  VolumeTailSender, VolumeIncrementalCopy, and IncrementalBackup so
  replica-sync paths cannot stream or append data from an unverified
  .dat/.idx pair; mirror on the Rust side via read_dat_slice,
  read_all_needles, dat_scan_plan, and the incremental-copy handler

* volume: drop issue references from comments near touched code

* volume: stop active scans when the volume becomes unavailable

The stream entry-point checks ran once per RPC, so a volume quarantined
by a failed recovery mid-scan kept serving data. Recheck availability
per needle/chunk on the detached read paths: tail scan and heartbeat,
read-all, incremental copy, incremental backup writes, and the Rust
StreamingBody chunk reads. Rust incremental copy also rejects a
quarantined volume before sync_to_disk touches the backend.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-09-24 06:57:44 +08:00
chrislu 96632a34b1 add version to volume proto 2025-06-16 22:05:06 -07:00
chrislu 1c143cfcea display actual offset value 2024-11-21 00:38:03 -08:00
wyang 361e7ebc74 fix volume.move io.EOF (#5960) 2024-09-02 06:17:41 -07:00
chrislu 14546f0ad8 minor 2023-07-06 09:12:28 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu 3afa451cdc volume: find a non-empty offset when binary searching by timestamp 2021-10-11 22:00:41 -07:00
Chris Lu b530f12327 volume: find a non-empty offset when binary searching by timestamp
fix https://github.com/chrislusf/seaweedfs/issues/2364
2021-10-11 16:11:50 -07:00
Chris Lu ce1efeb0eb adds more error message 2021-10-09 04:54:14 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
bingoohuang 7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
Chris Lu 9ac4935f22 read from volume index file directly instead of open a separate file
fix https://github.com/chrislusf/seaweedfs/issues/1640

read from volume index file directly instead of open a separate file,
to ensure reading latest index entries.
2020-11-27 16:18:48 -08:00
Chris Lu 6d30b21b10 volume: add "-dir.idx" option for separate index storage
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu f2723c1bc8 do not idx file format
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu c9ab8d05fa fixes for reading deleted fid 2020-09-10 14:42:52 -07:00
Chris Lu 6ccd7f0a4d refactoring 2020-08-18 18:01:37 -07:00
Chris Lu 892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu 6827cabfbb volume: dataFileAccessLock change to RW Lock 2019-12-06 06:59:57 -08:00
Chris Lu 19b6a16003 changed from os.file to backend.DataStorageBackend 2019-10-29 00:35:16 -07:00
Chris Lu fc412e428b refactor ScanVolumeFileFrom() 2019-10-22 00:50:30 -07:00
Chris Lu f9c1cd7370 avoid dead lock from dataFileAccessLock
fix https://github.com/chrislusf/seaweedfs/issues/1043
2019-08-15 09:24:54 -07:00
Chris Lu d829df4f59 volume: protect against nil needle map
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
2019-08-14 01:08:01 -07:00
Chris Lu 17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu 3b3651dea3 volume: atomic copying file, adds version and stopOffset 2019-04-19 12:29:49 -07:00
Chris Lu ac2727853f fix needle map entry size 2019-04-19 00:39:34 -07:00
Chris Lu e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu b09e8dbb37 add VolumeStreamFollow, but not used yet 2019-04-18 00:18:29 -07:00
Chris Lu 13ad5c1966 refactoring 2019-04-17 22:04:49 -07:00