Commit Graph
3 Commits
Author SHA1 Message Date
9575032b4c volume: forward fsync=true to replicas in ReplicatedWrite (#10805)
* volume: forward fsync=true to replicas in ReplicatedWrite

When a write request carries fsync=true, only the primary volume server
flushed to disk: the replica fan-out URL in ReplicatedWrite only carried
type/ttl/ts/cm, so replicas always wrote without fsync even when the
client explicitly requested a durable write.

Forward the fsync request parameter to the replica volume servers so a
durable write means every replica has flushed to disk, not just the
primary. Replicas without fsync are untouched (zero behavior change).

* storage: flush a durable write inline while stopping

The fsync flag on the write path really selects the async batch worker,
and it was switched off once the store is stopping. So a fsync=true write
landing during the pre-stop drain got acked without ever being flushed -
and now that ReplicatedWrite forwards fsync, that covers replicas too.

Flush it inline instead of queueing it. The drain keeps accepting writes,
which is the whole point of preStopSeconds, and the ack still means the
.dat is on disk. If the fsync fails, the append comes back off the .dat
and the needle map goes back to what it pointed at before, so nothing
resolves to an offset past the truncated end.

* storage: make the store's stopping flag atomic

SetStopping runs on the signal handler goroutine while the write and
vacuum paths read the flag, so every read of it was racy. Nothing about
the shutdown ordering changes; only the flag itself is now safe to read.

* topology: check the errors the replication test was dropping

The mock replica ignored its response write and the mock master ignored
whatever Serve returned, so a broken mock would have shown up as a
confusing timeout rather than a failure. Also drops the explicit listener
close: grpc.Server.Stop already closes the listener it was given.

---------

Co-authored-by: hzsunchao <hzsunchao@corp.netease.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-08-18 17:17:50 -07:00
Chris Lu c6e1387f59 shell: multi-target fs.mergeVolumes and volume.mark -readonlyCanDelete (#10706)
* shell: fs.mergeVolumes distributes one volume across multiple -toVolumeId targets

* volume: volume.mark -readonlyCanDelete rejects writes but keeps accepting deletes

* seaweed-volume: mirror readonlyCanDelete volume state
2026-08-10 16:31:26 -07:00
Chris Lu 7c252e1f16 fix(volume): reopen .idx writable after MarkVolumeWritable (fixes #9515) (#9526)
* fix(volume): reopen .idx writable after MarkVolumeWritable

When .vif has ReadOnly=true, load() opens .idx as O_RDONLY and builds a
SortedFileNeedleMap whose Put returns os.ErrInvalid. MarkVolumeWritable
only flipped noWriteOrDelete back to false and rewrote .vif, so writes
still failed at v.nm.Put. Reopen .idx in O_RDWR and rebuild v.nm in its
writable form (in-memory or leveldb small/medium/large) before flipping
the flag.

Mirror the same fix in seaweed-volume: the Rust load path leaves
CompactNeedleMap/RedbNeedleMap with no idx_file writer when the volume
boots read-only, so post-MarkVolumeWritable puts silently succeeded
in-memory only and were lost on the next restart. set_writable now
reattaches an append-mode writer when one is missing.

* fix(volume): keep old needle map until replacement is built; defer writable flag

Go: build the writable needle map into a local before swapping. A
construction failure now leaves v.nm pointing at the original
SortedFileNeedleMap so MarkVolumeWritable can roll back, instead of
stranding the volume with v.nm == nil.

Rust: attach the .idx writer before flipping no_write_or_delete to
false. A transient open/metadata failure used to leave the volume
marked writable with no writer attached, and subsequent puts would
silently skip the on-disk append.
2026-05-18 20:51:04 -07:00