Commit Graph
3 Commits
Author SHA1 Message Date
Chris Lu fd493f2547 mount: refuse to attach opens to a draining file handle (#12219)
* mount: refuse to attach opens to a draining file handle

With the writeback cache, releasing the last open of a file leaves the
shared FileHandle in the map at counter zero while an async flush
drains it; the worker removes the map entries once the flush commits.
AcquireHandle waits for a pending flush and then attaches, but the two
steps are not atomic: a release can slip in between, drop the counter
to zero, and queue the drain while the opener goes on to mint a new
file-handle id on the draining handle. When the drain completes,
RemoveFileHandle deletes every fh id for the inode -- including the one
just minted. The kernel still holds that id, so its next FUSE_READ
gets ENOENT from GetHandle; the failed read leaves the folio not
uptodate and the kernel surfaces EIO to userspace. This was the flaky
TestWritebackCacheConcurrentMixedOps failure.

AcquireFileHandle now refuses to attach when the shared handle has
reached a zero counter and reports it as draining. AcquireHandle and
the Create path retry the whole wait-resolve-attach sequence; the next
pass blocks on the pending flush (flag registration and the counter
drop share pendingAsyncFlushMu) and builds a fresh handle from the
post-flush entry. The DLM-failure release in AcquireHandle also goes
through ReleaseHandle so it cannot strand a counter-zero handle
outside the drain bookkeeping.

Removes the refreshEntryForRead retry from #12206, which worked around
the symptom by refetching the entry after the fh id was already gone.

Test: TestAcquireFileHandleRefusesDrainingHandle covers the refusal
and fresh-handle path; TestWritebackCacheConcurrentMixedOps and an
amplified concurrent-overwrite repro run clean.

* mount: keep refresh-on-error retry for stale-chunk reads
2026-10-29 16:06:57 -07:00
Chris LuandDevin 0ea818a44f mount: drain OFD locks when a file description is released (#12117)
* mount: drain OFD locks when a file description is released

The kernel scopes OFD fcntl locks to the open file description but
reports no lock_owner for them at RELEASE — it only sends one for flock
holders — so an OFD write lock taken on a file handle outlived the
holder and blocked every later open of that file (e.g. etcd failing to
reopen its WAL after a restart). The mount now records the owner of each
granted non-flock lock under its file handle and releases those owners
on RELEASE, which also covers any POSIX locks left over when FLUSH could
not run.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mount: give every open its own fh value over the shared handle

AcquireFileHandle reuses one FileHandle per inode, so every open reported
the same fh to the kernel. A RELEASE for one description could then not
be told apart from a sibling still holding the inode open — draining
lock owners recorded under the shared fh dropped locks belonging to
descriptions that stay open.

Each open now registers its own fh value in fh2inode, all resolving to
the shared FileHandle, and RELEASE removes exactly the id that is
closing. This also stops a duplicated RELEASE on one id from
double-decrementing the open count.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mount: forget flushed lock owners so a later release cannot re-drop locks

POSIX lock owners are process-scoped: Flush releases every lock the
process holds on the inode, but the owner stayed recorded under each
open's fh for the Release-time drain. If the process took a fresh lock
through another open before an old description finished closing, the
stale record made that RELEASE drop the new lock too.

Flush now drops the released owner from every recorded open of the
inode, so the Release drain only ever fires for owners whose locks were
never flushed.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mount: keep close-time lock owner records when the release fails

A routed POSIX owner release that errors keeps the flush hint for a
retry, but Flush also dropped the owner from every open's release
records unconditionally — so the final Release skipped the retry and
the owner filer's lock was stranded until lease reaping. Only forget
the owner once its locks were actually released.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 22:02:13 -07:00
Chris Lu da1e5e714f mount: move the inode table when a rename arrives from the cluster (#10822)
* mount: leave the target alone when a move has no source

MovePath cleared whatever sat at the target before it checked that the source
was still there, so a move it then declined to make had already taken the
target's mapping apart. The same rename reaching the table twice - once for an
open handle, once for the invalidation behind it - was enough to leave the
moved inode with no path at all.

* mount: move the inode table when a rename arrives from the cluster

A rename made by another client reaches this mount only as a metadata event,
and the only table update on that path sat inside the open-file-handle branch.
Every other inode the kernel still addresses by nodeid kept resolving to its
pre-rename path, so the next operation on it went to a path the filer no
longer has.

Move the entry for every rename invalidation. The filer sends one event per
moved entry, so a renamed directory's children follow their parent without a
descendant walk.

* mount: move the inode table exactly once per rename event

Making MovePath return early on a missing source was the wrong half to fix.
The source is also missing when the rename came from a client that never
visited it, and there the destination really was replaced and has to be
unlinked - the early return kept the name resolving to a file the rename
destroyed, so a dirty handle on it could still flush over what took its place.

The two cases are indistinguishable from inside MovePath, so leave it alone and
stop calling it twice: invalidateOpenFileHandle reports whether it moved, and
the handler moves only when it did not. Both paths mark a replaced file's
handle deleted, which the no-handle path previously did not do at all.

* mount: leave a rename alone unless the source is still ours to move

Two ways the fallback move could act on state it did not own. A handle whose
version guard skipped the event never reached RememberPath, so moving the table
under it left the handle flushing to the pre-rename path; the handle path owns
its inode's rename, so the fallback now runs only for an inode without one.

And the subscription can redeliver a rename once it falls out of the 4096-entry
dedup ring. A replay found no source and a live target, unlinked the mapping
the first delivery had just made, and marked the moved file's handle deleted -
worse than the stale path this set out to fix. Only a source still in the table
is moved now.

That gives up unlinking a destination the rename replaced when the source was
never visited here, which is where this started. It is what the mount already
did before this branch, and it is the safer of the two: retaining a stale name
costs a wrong lookup, while unlinking the wrong one costs a file's dirty data.

* mount: decide a rename move inside the table's lock

The source-presence check sat outside MovePath, so two invalidations for one
rename could both see the source and the loser would unlink what the winner had
just placed - the same damage the check was added to prevent. MovePath makes
the decision under its own lock now and reports that nothing moved.

The handle a rename destroyed is also marked from the caller rather than from
inside invalidateOpenFileHandle, which was setting isDeleted bare on a second
handle while holding the first one's lock. markHandleDeleted already takes the
lock the flush reads that flag under, and marking from the caller keeps it to
one handle lock at a time.

The invalidation test harness wires onEntryInvalidation now, the way the mount
does, rather than reaching past it.

* mount: apply a rename ahead of the handle's version fence

The fence exists so an old event cannot roll a handle's entry back to stale
content. A rename carries no content: it says the name the inode answered to is
gone. Skipping one on the strength of the fence left the inode and the handle
both pointing at a name the filer had vacated, and no fallback ran either,
since a handle owns its inode's rename.

Applied before the fence now, and only when MovePath reports the source was
still ours to move - which is what keeps a replayed rename from remembering a
path the handle has already moved past.
2026-08-19 14:57:45 -07:00