* mount: merge a refused content flush onto the stored entry
A content flush conditioned on the handle's base failed with EIO whenever a
peer had changed the file since, so two mounts appending to one file could
not both close it cleanly. On a refused condition, reread the entry and
replay this handle's changes since its base onto it: the chunks it added or
dropped, and the attributes, extended keys and inline content it changed.
A local truncation keeps its size and clips the peer's chunks to it;
otherwise the larger size wins. The merged entry is published against the
stored one, up to three times, and installed in the handle so its next
flush keeps the peer's chunks. A replaced or deleted file is still refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: merge a peer's content change into a dirty handle
A subscription event carrying a peer's content change replaced a dirty
handle's entry and discarded its unflushed pages, so the handle's writes
were lost and its close still returned OK. Fold the peer's entry into the
handle with the same merge a refused flush uses, keep the dirty pages, and
take the peer's entry as the new base. A replaced file still invalidates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: stop a flush merge once a filer accepts the write
withFilerClient can call the callback on several filers. A refusal from
one left its stored entry behind, so a later filer's success went on to
another merge round.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: refuse a merge that puts inline content next to chunks
A read inside the inline bytes returns them without looking at chunks,
so one side's inline content merged with the other side's chunks hid
data after a successful flush.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: refuse to merge a whole-file rewrite over a peer's chunk change
maybeMergeChunks re-uploads the handle's whole view as unstamped chunks.
Merged after a peer's change, they tie with a peer's own rewrite or bring
back a range the peer truncated, so that flush fails as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: clip peer chunks at the lowest truncate, not the final size
A handle that truncated and then wrote past the old end ends up larger
than its base, so the merge did not see the truncate and kept the peer's
chunks inside the range the truncate cleared. SetAttr now records the
lowest size truncated to over the current base.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: keep a peer's truncate when the handle left the size alone
The merge took the larger of the handle's and the stored size, so a peer
truncate came back as a zero-filled tail when the handle only overwrote
bytes inside it. The stored size now stands unless the handle changed
its size or wrote past it, and rebasing keeps only the growth writes made
after the flush was taken.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: retry a read view whose merged manifests did not resolve
A failed SetChunks leaves the chunk group answering every read with the
resolution error, and nothing re-ran it while the entry stayed the same.
The handle now marks such a view (on rebase and on SetEntry) and the
next read or lseek resolves the entry's chunks again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: keep a dirty handle's writes when a peer change cannot merge
The new merge refusals sent a peer event on the same file down the old
path, which discarded the dirty pages and adopted the peer's entry, so
close returned OK without the writes. The handle now keeps its writes and
base, and the flush meets the conflict and fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: keep an unpublished truncate across a peer's base change
The truncate mark was tied to the base pointer, so a peer event that moved
a dirty handle's base (a merged content change or a metadata-only change)
dropped it, and a later merge on a regrown file kept peer chunks inside
the truncated range. Those base moves now carry the mark; a flush ack or a
discarded handle state still clears it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: wait for the flush before merging a shrink over buffered writes
A write inside the file leaves its bytes in dirty pages without moving
the size. Merging a peer's truncate then adopted the smaller size, and the
chunk uploaded later sat past it, unreadable through the handle. The event
path and the periodic metadata flush now refuse a merge that shrinks the
handle; the close flush uploads the pages first and merges then.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: tell a shortened base chunk from a whole-file rewrite
A truncate shortens a base chunk in place, so it gets a new reference
key, and an old chunk can carry no timestamp. The rewrite refusal now
applies only to unstamped chunks whose file id is new to the base.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: check the creation time when the base has no inode
An entry without an inode or hard link id matched any replacement at the
same path, so a stale handle could merge into a different file. Without
an inode the merge now needs the same creation time. The filer keeps it
across updates; only a synthesized base has neither and still merges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: keep a dirty handle's writes when a peer replaces the file
A peer entry with another inode at the handle's path went down the
fallback that destroys the dirty pages and installs the replacement, so
close returned OK without the writes. The handle now keeps its writes
and base like it does for an unmergeable change, and its flush refuses to
update the replacement.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: build the content flush tests on freebsd
The invalidation tests, which every platform compiles, now use the
content flush helpers, so the freebsd cross-compile vet failed with
undefined: flushConditionEntry. Nothing in the file is freebsd-specific.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: keep a dirty handle's base when a same-content file replaces it
A replacement with the base's chunks but another inode counted as a
metadata-only change, so the dirty handle adopted it as its base and the
next conditional flush overwrote the new file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: bring a peer's metadata change into a dirty handle
A metadata-only event moved a dirty handle's base but left its live
entry, so the next flush read the peer's chmod as a local edit and
published the old mode back. Replay the handle's own changes onto the
peer's entry instead, as for a content change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: wait for the flush before merging peer inline content over buffered writes
A buffered write is not a chunk yet, so a merge could install a peer's
inline content that later reads return ahead of the write's chunk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: track a whole-file chunk rewrite on the handle
The merge told a rewrite by its unstamped chunks, which a manifest fold
hides: manifests are unstamped either way. Record the rewrite against the
base it was made from, carried across peer base changes like a pending
truncate, and refuse the merge from that.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: refuse a merge that shortens a manifest at the truncate
Clipping a manifest reference leaves its members whole, so peer bytes past
the handle's truncate could reappear once the file grew again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: flush a dirty handle after a peer replaces its file
The replacement test checked only the retained handle state. It now
flushes against a filer holding the replacement, including one with the
base's content, and checks the flush fails and leaves it unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* mount: test an append fsync after a peer's attribute-only change
A second mount's acknowledged chmod, chown, utimes or setxattr leaves
the content alone, so the writer's conditional content flush must still
publish its append and keep that change. On master the refused flush
surfaces as EIO from Fsync.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* mount: fail reads with error when chunk manifest resolution fails
When SetChunks fails to resolve a chunk manifest (e.g. the volume is on a
remote tier with reads disabled), the sections map stays empty and
readDataAtSequential/readDataAtParallel zero-fill every missing section as
if it were a sparse hole. Reads then return all-zero data with no error,
so a plain cp of a large manifest-based file silently produces a
completely zero-filled file.
Remember the resolve error in ChunkGroup (guarded by sectionsLock) and
return it from ReadDataAt. A later successful SetChunks clears it.
Fixes the mount path of #11286.
* filer: propagate manifest resolve errors in streaming read paths
ViewFromChunks discards the chunk manifest resolve error returned by
NonOverlappingVisibleIntervals. On failure the chunk views come back
empty, and the streaming paths zero-fill the entire requested range,
serving HTTP 200 / WebDAV 200 responses whose body is all zeros.
Propagate the error in PrepareStreamContentWithThrottler,
PrepareStreamContentWithPrefetch and the WebDAV read path so these
requests fail with 500 instead.
Fixes the filer HTTP and WebDAV paths of #11286.
* mount: fail lseek with EIO when chunk manifest resolution fails
SearchChunks still consulted the stale section map after SetChunks
recorded a manifest resolution failure, so SEEK_DATA/SEEK_HOLE would
describe the unresolved regions as sparse holes or return ENXIO.
Return the recorded error from SearchChunks and map it to EIO in
Lseek.
Also add regression tests for the stream preparation error paths.
Addresses review feedback on #11287.
* Enable writeback_cache and async_dio FUSE options
Fixes#7978
- Update mount_std.go to use EnableWriteback and EnableAsyncDio from go-fuse
- Add go.mod replace directive to use local go-fuse with capability support
- Remove temporary workaround that disabled these options
This enables proper FUSE kernel capability negotiation for writeback cache
and async direct I/O, improving performance for small writes and concurrent
direct I/O operations.
* Address PR review comments
- Remove redundant nil checks for writebackCache and asyncDio flags
- Update go.mod replace directive to use seaweedfs/go-fuse fork instead of local path
* Add TODO comment for go.mod replace directive
The replace directive must use a local path until seaweedfs/go-fuse#1 is merged.
After merge, this should be updated to use the proper version.
* Use seaweedfs/go-fuse v2.9.0 instead of local repository
Replace local path with seaweedfs/go-fuse v2.9.0 fork which includes
the writeback_cache and async_dio capability support.
* Use github.com/seaweedfs/go-fuse/v2 directly without replace directive
- Updated all imports to use github.com/seaweedfs/go-fuse/v2
- Removed replace directive from go.mod
- Using seaweedfs/go-fuse v2.0.0-20260106181308-87f90219ce09 which includes:
* writeback_cache and async_dio support
* Corrected module path
* Update to seaweedfs/go-fuse v2.9.1
Use v2.9.1 tag which includes the corrected module path
(github.com/seaweedfs/go-fuse/v2) along with writeback_cache
and async_dio support.
* compare chunks by timestamp
* fix slab clearing error
* fix test compilation
* move oldest chunk to sealed, instead of by fullness
* lock on fh.entryViewCache
* remove verbose logs
* revert slat clearing
* less logs
* less logs
* track write and read by timestamp
* remove useless logic
* add entry lock on file handle release
* use mem chunk only, swap file chunk has problems
* comment out code that maybe used later
* add debug mode to compare data read and write
* more efficient readResolvedChunks with linked list
* small optimization
* fix test compilation
* minor fix on writer
* add SeparateGarbageChunks
* group chunks into sections
* turn off debug mode
* fix tests
* fix tests
* tmp enable swap file chunk
* Revert "tmp enable swap file chunk"
This reverts commit 985137ec47.
* simple refactoring
* simple refactoring
* do not re-use swap file chunk. Sealed chunks should not be re-used.
* comment out debugging facilities
* either mem chunk or swap file chunk is fine now
* remove orderedMutex as *semaphore.Weighted
not found impactful
* optimize size calculation for changing large files
* optimize performance to avoid going through the long list of chunks
* still problems with swap file chunk
* rename
* tiny optimization
* swap file chunk save only successfully read data
* fix
* enable both mem and swap file chunk
* resolve chunks with range
* rename
* fix chunk interval list
* also change file handle chunk group when adding chunks
* pick in-active chunk with time-decayed counter
* fix compilation
* avoid nil with empty fh.entry
* refactoring
* rename
* rename
* refactor visible intervals to *list.List
* refactor chunkViews to *list.List
* add IntervalList for generic interval list
* change visible interval to use IntervalList in generics
* cahnge chunkViews to *IntervalList[*ChunkView]
* use NewFileChunkSection to create
* rename variables
* refactor
* fix renaming leftover
* renaming
* renaming
* add insert interval
* interval list adds lock
* incrementally add chunks to readers
Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order
* fix tests compilation
* fix tests compilation