mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-19 02:15:02 +01:00
94de65b3b5d79b85e1112321199e81ccc1273cc1
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c2b2ed769b |
fix(mount): preserve appended chunks during stale attribute updates (#12593)
* fix(mount): preserve acknowledged chunks during attribute updates * mount: keep the WFS field block aligned Adding the publication-lock fields inside the aligned block made gofmt re-indent 25 unrelated lines. Put them in their own group after dirIdleEvict, matching the enterprise layout, so syncs do not conflict there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: finish attribute saves through a FUSE interrupt saveEntryAttributes derived its RPC context from the syscall's cancel channel, so an interrupt while the conditional UpdateEntry was at the owner returned EIO for an update the owner then applied, and skipped the local install and retention. Keep the metadata deadline but drop the cancellation, like the other metadata mutations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: map attribute CAS ids in place instead of cloning the entry saveEntryAttributes cloned the whole entry three times per attempt only to rewrite uid/gid for the filer, and each clone is as large as the chunk list. Switch the ids in place and restore the exact local values after the call, keeping one clone. The test now checks the reply carries the local uid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * mount: compare attribute updates against the filer's own uid/gid With -map.uid/-map.gid, a filer id that is also a mapped local id reads back unchanged and maps forward to another filer id. The IF_ENTRY_EQUAL expected entry then never matched, so chmod, utimes and xattr changes on such files failed with EAGAIN after three attempts, and an update would have rewritten the owner. Keep the filer ids read with the entry for the ids the syscall leaves alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mount): align attribute retention with read lock order * mount: take the handle lock before entryLock in the attribute snapshot attributeEntrySnapshot held entryLock while it queued for the shared handle lock. A Read holding that lock waits on entryLock, the snapshot waits behind a queued exclusive (an invalidation or a Write), and the exclusive waits on the Read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(mount): complete handle lock order and preserve explicit ID intent * mount: re-read a refused attribute update from the file's owner filer The filer checks the IF_ENTRY_EQUAL condition on the owner of the path's write route key. The retry read any filer, so behind one that had not applied the owner's latest update, every attempt sent the same stale base and the syscall failed with EAGAIN. Read from the owner once the lock ring is known, like lookupFilerEntry, but keep the filer's raw ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8db41d0217 |
[Mount] Cache Chunk Manifest Resolution for Repeated File Opens (#11266)
* cache resolved chunk manifests for Mount * Address PR review: per-mount cache, singleflight, reuse ResolveOneChunkManifest - Own the manifest cache per WFS mount instead of a process-global variable, so manifests from one filer backend are never served to another (Devin/CodeRabbit major bug). - Coalesce concurrent cold misses via singleflight so only one fetch runs during a cold burst (Greptile P2). - Copy cached data after releasing the mutex so a large copy does not block concurrent hits, inserts, and evictions (CodeRabbit nitpick). - Reuse the existing ResolveOneChunkManifest function name instead of introducing a new resolveOneChunkManifest wrapper. - Validate (unmarshal) manifest bytes before caching so malformed manifests do not poison the cache. - Add TestChunkGroupManifestResolutionCoalescesColdMisses covering the singleflight cold-miss path. * Address round 2 review: coalesced-miss cancellation, test overlap - Use singleflight.DoChan in fetchOrLoad and select on ctx.Done() so a caller whose context is canceled while waiting for an in-flight fetch returns ctx.Err() promptly instead of blocking for the leader's result (Devin BUG). - Add TestResolveOneChunkManifestCanceledWaiterReturnsDuringCoalescedMiss covering the canceled-waiter path. - Delay the cold-miss fixture response so the leader's fetch is still in flight when concurrent opens join the singleflight, making the one-fetch assertions reliable (CodeRabbit Minor). * Address review: keep ResolveOneChunkManifest four-argument Restore the exported ResolveOneChunkManifest to its original four-argument signature so external callers keep compiling. Move the cache-aware resolution into an unexported resolveOneChunkManifest helper that accepts the per-mount ChunkManifestCache. The exported function delegates to the helper with a nil cache, preserving the historical uncached behavior for every non-Mount caller. The Mount path (ChunkGroup.SetChunks) now calls the unexported helper with the mount-owned cache. Tests and benchmarks that exercise the cache path call the unexported helper directly. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
|
bc06505b40 |
mount: keep metadata operations working on an unlinked open file (#10989)
* mount: serve metadata ops from the open handle of an unlinked file ftruncate on a descriptor whose file was unlinked failed with ENOENT: maybeReadEntry resolved the inode to a path first, and unlink had already dropped it. GetAttr worked around that with its own handle fallback; SetAttr and the xattr handlers had none. Look the handle up first and let it answer whether or not a name still points at the inode. GetAttr keeps reporting nlink 0 there, now off the empty path. Claude-Session: https://claude.ai/code/session_01U1R8BM4bVT46KwPDEj2Ega * mount: read an open handle's attributes under the handle lock too GetAttr held only the LockedEntry lock, which covers the async uploader's chunk appends but not Write or the metadata flush: those rewrite size, times and the whole chunk slice under the handle lock, so FileSize could walk a slice mid-reassignment. The branch this replaced took both locks; take both here, outer handle lock first, as Read and Lseek do. Claude-Session: https://claude.ai/code/session_01U1R8BM4bVT46KwPDEj2Ega * mount: report nlink 0 from SetAttr for an unlinked open file The kernel caches the attributes a SETATTR reply carries, so an ftruncate on an unlinked file left fstat reporting nlink 1 until the cache expired, even though GetAttr had it right. Both replies go through the same rule. Claude-Session: https://claude.ai/code/session_01U1R8BM4bVT46KwPDEj2Ega |
||
|
|
1ddec72707 |
Recover from a dead volume server on the mount read path (#10798)
* mount: re-resolve volume locations after a failed chunk read NewChunkGroup passed nil as the ReaderCache's CacheInvalidator, so retryFetchAfterCacheInvalidation was dead code on the FUSE read path. A mount that cached a volume's locations while one server was down kept retrying that server after it died, then returned EIO, even though the master and filer both resolved the live replica. The S3 gateway already passes its filerClient; do the same for the mount. * test: FUSE integration tests for volume server failover One mount appends while a second tails, and a volume server is killed, started or restarted mid-stream against a 001-replicated cluster of three volume servers. Automates the scenario matrix reported for Docker Swarm mounts, including the large-file variant and a no-chaos control. * test: report the filer's own view when append content mismatches A mismatch between what the writer wrote and what the reader sees can come from either side's cache. Read the file back through the filer's HTTP handler as well, and let the mount verbosity be raised from the environment, so a failing run says which layer lost the data. * test: wait for the reader mount to converge before comparing A mount caches metadata for about a second, so reading the file the instant the writer's last close returned can legitimately come back short. Poll the reader until it matches or the timeout expires; content that is wrong rather than merely late never converges and still fails, now with the writer's mount and the filer's own view alongside it. * test: detect a failover cluster child that exited at startup Signal(0) succeeds for a zombie and nothing reaped these children until shutdown, so a process that died on startup looked alive until the readiness timeout expired. Reap each child as it is started and consult the result. * test: read a file the killed volume server actually holds Placement decides which two of three servers back each volume, so killing volume N and reading readfile-N could pass without the victim ever holding a replica of it. Resolve each file's volumes through the filer and the master, and pick one the victim backs, preferring a file the reader has not cached. * ci: stop persisting checkout credentials in the failover workflow The job does not use the token after cloning. Also tag the README's command block as bash and match the timeout the workflow actually uses. * test: discard the ignored errors errcheck flags in the failover harness * test: resolve manifests when mapping a file to its volumes A manifest chunk's own fid names the volume holding the manifest, not the volumes holding the data, so a large enough file would point the failover victim at the wrong server. * test: pin the stale-location recovery path with a primed reader Reading a file for the first time after a server dies proves nothing: the lookup is fresh and returns the survivor. Kill one holder and wait for the master to drop it, read a file on that volume so the reader caches the lone survivor, restart the first server, then kill the survivor. The reader's only cached location is now dead while the data is live elsewhere, which is the case the invalidator exists for: EIO without it, recovery with it. |
||
|
|
214d3599d3 |
windows mount: cache file data, resolved paths and attributes (#10703)
* benchmark tool for mounted filesystems * ci: on-demand mount benchmark, native WinFsp vs rclone plus a Linux reference * windows mount: let the Windows cache manager cache file data WinFsp only turns the cache manager on for a file when FileInfoTimeout is infinite; at any finite value every application read and write is a synchronous trip into the mount process at whatever size the application issued. Metadata events already reach FspFileSystemNotify, which purges a changed file's cached pages and attributes, so an infinite timeout stays coherent. The dir listing, volume info and EA timeouts are pinned to one second so they do not silently inherit the infinity. * windows mount: cache resolved paths and attributes in the adapter WinFsp addresses every operation by path and has no FORGET, so the adapter walked the whole path through Lookup on each one, and in a directory the filer has not listed yet every walk was a filer round trip; nothing played the part of the kernel's dentry and attribute caches. The path cache owns one lookup reference per entry the way the kernel holds one until FORGET, serves attribute reads for files without an open handle, and is purged by the mount's own mutations and by metadata events, with the timeout as backstop. * windows mount: keep a closed file's attributes cached Open steals the path's cache entry for its handle and Release returned the reference with a purge, so the stat that follows every copied file walked to the filer again. Reading the handle's final attributes before it goes away and moving the reference back into the cache serves that stat locally, the way the kernel's attribute cache does after a close. Only if the path still names that inode, though: WinFsp reports the path the handle opened with, and after a delete-on-close or a rename caching it would resurrect an entry that is gone. * windows mount: persist entries at create, and let the flush stay at close WinFsp posts the cleanup and close that carry the flush after CloseHandle has returned, so deferring the filer entry to the flush let everything that reads through the filer race an unflushed close: a listing missed just-written files, and a directory rename moved a directory on the filer before its newest child existed there, leaving the straggler flush to recreate the child under the dead path. Flush-at-cleanup is not the answer either: it makes every handle's cleanup flush, and those flushes race the unlinks of delete-on-close, re-inserting the entry the unlink just removed. Persisting the entry at create takes the ordering question away. * mount: flush written pages before a truncate shrinks past them The shrink trims chunks, but written pages that have not become chunks yet are invisible to it, so the next flush wrote them back and the file grew again, resurrecting the truncated bytes. Windows hits this on every write-then-shrink because its flush runs after CloseHandle, but the gap is platform-neutral. * mount: order a file's unlink against its in-flight flush Unlink set the handle's deleted flag bare, so a flush already past its own check of that flag wrote the entry back right after the delete removed it, and a delete-on-close file outlived its last handle. The flag is now set under the handle's flush lock and re-checked under it, so a flush either completes before the delete or sees the flag and skips. An eagerly created handle also starts clean: the dirty mark existed to make the deferred filer create happen at flush, and eager creates have nothing to flush. |
||
|
|
68cae26c0b |
mount: fix SetAttr/GetAttr crash from concurrent chunk append under writebackCache (#9667)
* mount: hold the entry lock while reading chunk size in GetAttr/SetAttr Async upload workers append chunks to an open handle's shared entry under the LockedEntry lock (FileHandle.AddChunks), but GetAttr and SetAttr computed FileSize by iterating entry.Chunks without taking it. A concurrent append that reallocated the backing array tore the slice read and crashed in filer.TotalSize. Surfaces with -writebackCache, where handles stay open and flush asynchronously while metadata ops keep arriving. Take the LockedEntry lock for those reads (and SetAttr's truncate rewrite). * mount: re-read entry under the lock in GetAttr/SetAttr If SetEntry swapped the handle's entry pointer between maybeReadEntry and the lock acquisition, the old pointer is orphaned. Re-read fh.entry.Entry under the lock so SetAttr mutates the live entry instead of losing the update, and GetAttr reports the current one. * mount: cover the truncate path in TestAttrChunkRace Alternate SetAttr between mtime-only and a shrinking size so the test also exercises the entry.Chunks rewrite under fh.entry.Lock, not just the read-side size walk. * mount: snapshot chunks under the entry lock on the read path readFromChunks holds fh.entryLock (excludes SetAttr) but not the LockedEntry lock the async uploader appends under, so IsInRemoteOnly, the FileSize fallback, and the RDMA/peer chunk walks read entry.Chunks while AddChunks reallocated it — the same torn-slice crash as GetAttr/SetAttr. Snapshot size, inline content, and the chunk list under a brief LockedEntry RLock, then hand the snapshot to the RDMA/peer helpers instead of holding the lock across network I/O. The captured slice stays valid: append never mutates the old backing array, and truncate is excluded by the fh.entryLock. |