mount: keep the shared directory build out of the requester's context (#12206)

* mount: keep the shared directory build out of the requester's context

The shared directory build behind EnsureVisited is a singleflight call: the
first requester for a path owns the build, and every later requester for the
same path waits on that same build. The build ran on the owner's request
context, so a FUSE request that returned -- or that the kernel interrupted --
canceled the build for everybody waiting on it.

A waiter whose own context was still alive then received somebody else's
cancellation. weed/mount/weedfs_dir_read.go classified a canceled read as
EINTR only when the local context was done, so such a waiter fell through to
fuse.EIO: the kernel handed EIO to readdirent, kubelet's applyFSGroup failed,
and the volume never mounted.

Build the shared directory on a context that keeps the request's values but
not its cancellation, and report a canceled read as EINTR even when the local
context is still alive. Waiters still abandon on their own context, so a
canceled request stops waiting immediately.

This follows up #11717, which made the wait abandonable and mapped the local
cancellation to EINTR, but left the shared build itself on the owner's
context. The surrounding build lifecycle calls already use
context.Background() for the same reason.

Test: TestEnsureVisitedSharedBuildSurvivesOwnerRequestCancel reproduces the
cancellation with a fake filer and fails without this change.

* mount: classify gRPC-canceled directory listings as EINTR

* mount: log only real directory read failures, not interruptions

* mount: keep shared section refresh out of the requester's context

* mount: let section-refresh waiters leave on their own context

The shared refresh flight now outlives the requester that started it,
but callers still blocked in singleflight.Do until it finished — an
interrupted directory read could not stop waiting, and the caller's
own deadline never applied to the wait. DoChan + select on the
caller's context restores early abandon while the detached refresh
still completes for the remaining waiters.

* ci: re-run FUSE integration after a flaky writeback-cache failure

* mount: refresh a stale entry once before reporting a read error

A read resolves chunk views from the handle's entry. A concurrent
overwrite commits a new entry whose replaced file ids the filer queues
for deletion; the open handle only learns the new chunk list when the
metadata invalidation lands. A fetch that lands between the volume
delete and the invalidation names data that is already gone and the
read reports EIO. On a chunk read error, wait out any in-flight async
flush, re-read the entry from the filer, install it when it outranks
the handle, and retry once. An unchanged version means the failure is
genuine.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* mount: fence the read-error entry refresh by file identity

Two guards on refreshEntryForRead's install path: the unlocked
dirtyMetadata read raced with Write under the handle lock — drop it,
installAckedEntry already checks under that lock, and it now fences
isDeleted handles the same way. And a saved path can name a different
file after unlink/reuse, so refuse the refresh when the fetched inode
disagrees with the one the handle holds.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-10-29 11:32:57 -07:00
1 parent c7d77eb9a8
commit 3955e7218f
8 files changed
+349 -47

No files matched your search

+24 -11
View File
@@ -91,13 +91,26 @@ const (
)
func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerClient, path util.FullPath, maxCacheableEntries int) error {
// The shared build must not run on the requester's context. A FUSE request
// that returns — or that the kernel interrupts — cancels that context, and
// every other requester waiting on the same build would inherit the
// cancellation. weed/mount/weedfs_dir_read.go only reports EINTR when the
// *local* context is done, so such a waiter falls through to fuse.EIO and
// the kernel hands EIO to readdirent: kubelet's applyFSGroup fails and the
// volume never mounts.
//
// Build on a context that keeps the request's values but not its
// cancellation. Waiters still abandon on their own context below, so a
// canceled request stops waiting immediately.
buildCtx := context.WithoutCancel(ctx)
// Use singleflight to deduplicate concurrent requests for the same path.
// DoChan so a canceled request stops waiting even when another request
// owns the in-flight build.
resultCh := mc.visitGroup.DoChan(string(path), func() (interface{}, error) {
// Check for cancellation before starting
if ctx.Err() != nil {
return nil, ctx.Err()
if buildCtx.Err() != nil {
return nil, buildCtx.Err()
}
// Double-check if already cached (another goroutine may have completed)
@@ -128,7 +141,7 @@ func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerCl
}
}
defer func() {
if !cleanupDone && ctx.Err() != nil {
if !cleanupDone && buildCtx.Err() != nil {
cleanupBuild("canceled")
}
}()
@@ -139,11 +152,11 @@ func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerCl
entryCount = 0
sections = sectionBoundsCollector{}
var batch []*filer.Entry // reset on retry, allow GC of previous entries
if err := mc.deleteFolderChildrenForRebuild(ctx, path); err != nil {
if err := mc.deleteFolderChildrenForRebuild(buildCtx, path); err != nil {
return fmt.Errorf("clear existing entries for %s: %w", path, err)
}
var listErr error
snapshotTsNs, listErr = filer_pb.ReadDirAllEntriesWithSnapshot(ctx, client, path, "", func(pbEntry *filer_pb.Entry, isLast bool) error {
snapshotTsNs, listErr = filer_pb.ReadDirAllEntriesWithSnapshot(buildCtx, client, path, "", func(pbEntry *filer_pb.Entry, isLast bool) error {
entry := filer.FromPbEntry(string(path), pbEntry)
if !mc.includeSystemEntries && IsHiddenSystemEntry(string(path), entry.Name()) {
return nil
@@ -158,7 +171,7 @@ func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerCl
// flush by size, not isLast: hidden entries can return early
if len(batch) >= batchInsertSize {
if err := mc.doBatchInsertEntries(ctx, batch); err != nil {
if err := mc.doBatchInsertEntries(buildCtx, batch); err != nil {
return fmt.Errorf("batch insert for %s: %w", path, err)
}
batch = make([]*filer.Entry, 0, batchInsertSize)
@@ -169,7 +182,7 @@ func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerCl
return listErr
}
if len(batch) > 0 {
if err := mc.doBatchInsertEntries(ctx, batch); err != nil {
if err := mc.doBatchInsertEntries(buildCtx, batch); err != nil {
return fmt.Errorf("batch insert remaining for %s: %w", path, err)
}
}
@@ -198,14 +211,14 @@ func doEnsureVisited(ctx context.Context, mc *MetaCache, client filer_pb.FilerCl
// immediate (a clean-EOF stream glitch clears at once), later ones space out.
// On cancellation the deferred cleanup aborts the build.
for attempt := 0; entryCount == 0 && attempt < emptyRebuildConfirmations; attempt++ {
if ctx.Err() != nil {
return nil, ctx.Err()
if buildCtx.Err() != nil {
return nil, buildCtx.Err()
}
if attempt > 0 {
select {
case <-time.After(emptyRebuildConfirmDelay):
case <-ctx.Done():
return nil, ctx.Err()
case <-buildCtx.Done():
return nil, buildCtx.Err()
}
}
if entryCount, snapshotTsNs, sections, fetchErr = reloadFromFiler(); fetchErr != nil {