Commit Graph
10 Commits
Author SHA1 Message Date
Chris Lu da1e5e714f mount: move the inode table when a rename arrives from the cluster (#10822)
* mount: leave the target alone when a move has no source

MovePath cleared whatever sat at the target before it checked that the source
was still there, so a move it then declined to make had already taken the
target's mapping apart. The same rename reaching the table twice - once for an
open handle, once for the invalidation behind it - was enough to leave the
moved inode with no path at all.

* mount: move the inode table when a rename arrives from the cluster

A rename made by another client reaches this mount only as a metadata event,
and the only table update on that path sat inside the open-file-handle branch.
Every other inode the kernel still addresses by nodeid kept resolving to its
pre-rename path, so the next operation on it went to a path the filer no
longer has.

Move the entry for every rename invalidation. The filer sends one event per
moved entry, so a renamed directory's children follow their parent without a
descendant walk.

* mount: move the inode table exactly once per rename event

Making MovePath return early on a missing source was the wrong half to fix.
The source is also missing when the rename came from a client that never
visited it, and there the destination really was replaced and has to be
unlinked - the early return kept the name resolving to a file the rename
destroyed, so a dirty handle on it could still flush over what took its place.

The two cases are indistinguishable from inside MovePath, so leave it alone and
stop calling it twice: invalidateOpenFileHandle reports whether it moved, and
the handler moves only when it did not. Both paths mark a replaced file's
handle deleted, which the no-handle path previously did not do at all.

* mount: leave a rename alone unless the source is still ours to move

Two ways the fallback move could act on state it did not own. A handle whose
version guard skipped the event never reached RememberPath, so moving the table
under it left the handle flushing to the pre-rename path; the handle path owns
its inode's rename, so the fallback now runs only for an inode without one.

And the subscription can redeliver a rename once it falls out of the 4096-entry
dedup ring. A replay found no source and a live target, unlinked the mapping
the first delivery had just made, and marked the moved file's handle deleted -
worse than the stale path this set out to fix. Only a source still in the table
is moved now.

That gives up unlinking a destination the rename replaced when the source was
never visited here, which is where this started. It is what the mount already
did before this branch, and it is the safer of the two: retaining a stale name
costs a wrong lookup, while unlinking the wrong one costs a file's dirty data.

* mount: decide a rename move inside the table's lock

The source-presence check sat outside MovePath, so two invalidations for one
rename could both see the source and the loser would unlink what the winner had
just placed - the same damage the check was added to prevent. MovePath makes
the decision under its own lock now and reports that nothing moved.

The handle a rename destroyed is also marked from the caller rather than from
inside invalidateOpenFileHandle, which was setting isDeleted bare on a second
handle while holding the first one's lock. markHandleDeleted already takes the
lock the flush reads that flag under, and marking from the caller keeps it to
one handle lock at a time.

The invalidation test harness wires onEntryInvalidation now, the way the mount
does, rather than reaching past it.

* mount: apply a rename ahead of the handle's version fence

The fence exists so an old event cannot roll a handle's entry back to stale
content. A rename carries no content: it says the name the inode answered to is
gone. Skipping one on the strength of the fence left the inode and the handle
both pointing at a name the filer had vacated, and no fallback ran either,
since a handle owns its inode's rename.

Applied before the fence now, and only when MovePath reports the source was
still ours to move - which is what keeps a replayed rename from remembering a
path the handle has already moved past.
2026-08-19 14:57:45 -07:00
Chris Lu bb223967bd mount: fold an inode's single link into its entry (#10818)
InodeEntry held its one path in a slice, so every inode the kernel references
cost a 16-byte backing array and a second heap object on top of the 32-byte
entry. The extra links of a hard-linked file now hang off a pointer instead,
which keeps the struct in the same 32-byte size class and leaves the ordinary
single-link file with nothing to allocate.

Populating the table with 1M children: 237.5 -> 221.5 B/inode at 85-character
paths, 301.3 -> 285.6 at 148.
2026-08-18 20:56:06 -07:00
Chris Lu 65114575eb mount: invalidate hot directory listings by section (#10712)
* mount: invalidate hot directory listings by section

A cached directory used to be dropped whole when it saw 64 changes in
2s: with a continuous writer the listing cycled through wipe, direct
listing and full rebuild for as long as the writer kept going, and
every sibling lookup fell through to the filer in between.

Split each cached listing into name-range sections of 1024 entries. A
burst of foreign changes invalidates just the section it lands in;
entries stay served and events keep applying, and the next readdir
re-lists only that range from the filer, reconciled through the version
gate so it cannot roll back newer applied events. Lookups in an
invalidated section read through until then. The mount's own writes no
longer invalidate anything: they are ground truth for its cache.

* meta_cache: drop the version floor with a deleted or moved directory

The other teardown paths already clear both maps; a floor left behind
here would fence the listing of a directory re-created at the same
path.

* mount: harden section refresh

An unversioned listing (pre-upgrade filer) now only fills gaps instead
of reconciling: without a snapshot to order against, an overwrite or
the deletion sweep could roll back an event applied after the listing.

The section table can be rebuilt or re-split between the listing and
its apply, so the refresh only marks fresh or splits when the section
still covers the range it read. Splicing bounds from a stale range
into a rebuilt table could leave them unsorted.

Bound the wait: a readdir gives a refresh five seconds before serving
the maintained-but-unverified cache. Bound the size: a range grown
past four sections aborts the refresh and drops the directory cache,
re-tiling it with a full rebuild, with that request served direct.

Cover the filer-facing path with a listing server: paging with the
snapshot pinned across pages, the section cutoff, no calls for a
fresh section, and the overgrown-range abort.

* meta_cache: make the section table a self-contained state machine

Churn counting, freshness, stale-range scanning and the refresh
completion with its guard and split now live on dirSections itself,
free of the lock, the store and the apply loop, so they test directly
with synthetic clocks and tables. MetaCache keeps thin wrappers that
hold its mutex and find the directory's table.

* meta_cache: keep section internals out of the apply request

The request now carries the completed build's table and one refresh as
opaque values built by section code, and the boundary-derivation rule
moves out of the build loop into a collector next to the rest of the
section logic.

* mount: fence refreshed sections with a snapshot floor

A refresh versioned the entries it fetched and tombstoned the ones it
swept, but a name absent from both cache and listing kept the old
directory floor, so a delayed event between the two snapshots could
resurrect it into a section already marked fresh. The section now
carries its own floor, consulted next to the directory floor, covering
every name in the range, present or absent — which also retires the
refresh's per-entry version stamps and sweep tombstones.

An unversioned listing sets no floor and vouches for nothing: it may
still fill gaps, but the section stays stale and reads through until a
filer that stamps snapshots re-validates it.

A listing's reach is unknowable up front — a resumed handle can skip
far ahead, and shrunken sections let one batch span many — so a
readdir now re-validates every stale section from its start name to
the end of the directory instead of the next two.

* mount: fence tombstoned names with floors and gate the reconcile

A tombstone answered for its name before the floors were consulted, so
one at an old position let through events the newer listing floor
should have fenced; a build never hit this because it prunes
superseded tombstones, which a section refresh does not. The version
gate now raises a tombstone to the floors like any other record.

With no per-entry versions, only the section floor fences a
reconcile's work, so a range the rebuilt or re-split table no longer
has must not touch the store either: the range check moves ahead of
the mutations, under the same lock the floor install holds.

An unversioned refresh no longer retries: the section is remembered as
unverifiable and skipped by the stale scan, or every batch of every
readdir would re-list the same ranges against a filer that cannot
vouch for them.

* mount: clear beaten unversioned markers and skip refresh mid-build

An unversioned marker outliving the snapshot write that replaced its
content bypassed the section floor the same way an old tombstone did,
letting a delayed pre-snapshot event roll the entry back. The refresh
now clears the marker when its write wins; pinned local-only entries
are not replaced at all, keeping their content and marker.

A rebuild wipes and repopulates the store off the apply loop, so a
refresh reconciling meanwhile could sweep children the build had
already inserted and let it publish the directory incomplete. The
refresh now skips a building directory, as events (buffered) and
purges (skipped) already do; its staleness dies with the build's
fresh table.

* mount: clear the unversioned marker only after its replacement lands

Clearing before the insert meant a failed write left the old local
content claiming the listing floors, fencing the very events that were
still entitled to correct it.

* meta_cache: rename the section state machine to sectionList

dirSections named both the type and the map of them.

* mount: raise the default cacheDirMaxEntries to 100000

The low ceiling guarded against whole-listing rebuild churn: a big
cached directory under writes kept re-streaming everything. Sectioned
invalidation ended that — a burst now costs one range listing — so the
remaining cost of caching a large directory is its one-time build,
comparable to the single direct listing that read-through mode pays on
every enumeration instead.

* meta_cache: cover section border and edge cases

A bound-named entry belongs to the section starting at the bound: the
neighboring refresh's sweep stops before it, its own section's covers
it. Churn past everything the build saw lands in the tail section, a
rename spanning two sections invalidates both, and a listed entry at
the section's end name is cut off with the ones beyond it.
2026-08-11 21:11:18 -07:00
Chris Lu b1fecf3b44 mount: mark windows files archived and ignore a zero timestamp (#10559)
* mount: mark windows files archived and ignore a zero timestamp

Windows synthesises NORMAL when a file reports no attributes at all, which
is not the same as ARCHIVE and is what create_fileattr_test checks.

Utimens also wrote a zero timestamp through. Windows sends zero for a field
it is not setting, and storing it put 1970 in the atime overlay, which then
overrode the entry's real time — so a file created a moment ago reported an
access time of 1970 whenever the caller asked through an open handle.
Reading the path instead went down a different route and looked right,
which is why a probe of a fresh file showed nothing wrong.

* mount: match the file type by its mask, and only treat the epoch as unset

S_IFDIR is part of the multi-bit type field rather than a flag, so masking
against it alone also matched a symlink, which shares the bit. A regular
file is now identified by the type mask.

Rejecting every timestamp at or below zero also rejected a date genuinely
before 1970. Only the epoch itself is what Windows sends for a field it is
not setting, so that is all that is refused.

create_fileattr goes back on the known-failures list: the archive fix
works and the test simply moves on to ask for READONLY too, which needs
Chflags. Taking it off was premature.

* mount: drop the time overlays when an inode is released

atimeMap and dirMtimeMap are keyed by inode and were only ever trimmed by
a random eviction at capacity. Inodes are derived from the path, so a
delete and recreate hands the same number to a different file, which then
reported the previous file's access time — a file created a moment ago
answering with a time from long before it existed.

Cleared when Forget actually releases the inode, not on every decrement:
a partial forget still has users. Forget now reports that so callers
holding state keyed by the inode know when to drop it.

* ci: keep getfileinfo listed while its access time is unexplained

Two causes have been fixed and neither closed it, so the honest state is
listed-with-a-reason rather than removed in hope.

* mount: drop timestamp overlays while the inode table is locked

Forget released the inode under the table's lock but cleaned up the
atime and dir-mtime overlays after returning from it. Inode numbers are
derived from the path, so a lookup arriving in that window is handed the
same number back and can store a time that the cleanup then deletes.

Run the cleanup at the release point instead, as a callback under the
lock. The directory-cache purge stays deferred until after the unlock,
where it has to be.

Claude-Session: https://claude.ai/code/session_01EgY2QA3iiPtiu6ww3P2EBn
2026-08-04 16:42:33 -07:00
Chris Lu f475d60fcf mount: move directory cache state to a side map to shrink InodeEntry (152 to 32 bytes) (#10114)
mount: move directory cache state to a side map to shrink InodeEntry

The mount keeps an InodeEntry alive for every inode the kernel references.
On a mount that is almost entirely regular files, each entry carried the full
directory readdir-cache bookkeeping (four time.Time fields plus counters),
bloating it to 152 bytes whether or not the inode was a directory.

Move that state into a dirState held in a side map keyed by inode, and drop the
isDirectory bool: an inode is a directory iff it has a dirState. InodeEntry is
now just paths + nlookup at 32 bytes, landing in a smaller Go allocator size
class; on a mount with tens of millions of cached file inodes that is several GB
less resident heap. As a side effect the readdir-cache scan helpers iterate only
directories instead of every inode.
2026-06-25 19:17:32 -07:00
Chris LuandCopilot 3f946fc0c0 mount: make metadata cache rebuilds snapshot-consistent (#8531)
* filer: expose metadata events and list snapshots

* mount: invalidate hot directory caches

* mount: read hot directories directly from filer

* mount: add sequenced metadata cache applier

* mount: apply metadata responses through cache applier

* mount: replay snapshot-consistent directory builds

* mount: dedupe self metadata events

* mount: factor directory build cleanup

* mount: replace proto marshal dedup with composite key and ring buffer

The dedup logic was doing a full deterministic proto.Marshal on every
metadata event just to produce a dedup key. Replace with a cheap
composite string key (TsNs|Directory|OldName|NewName).

Also replace the sliding-window slice (which leaked the backing array
unboundedly) with a fixed-size ring buffer that reuses the same array.

* filer: remove mutex and proto.Clone from request-scoped MetadataEventSink

MetadataEventSink is created per-request and only accessed by the
goroutine handling the gRPC call. The mutex and double proto.Clone
(once in Record, once in Last) were unnecessary overhead on every
filer write operation. Store the pointer directly instead.

* mount: skip proto.Clone for caller-owned metadata events

Add ApplyMetadataResponseOwned that takes ownership of the response
without cloning. Local metadata events (mkdir, create, flush, etc.)
are freshly constructed and never shared, so the clone is unnecessary.

* filer: only populate MetadataEvent on successful DeleteEntry

Avoid calling eventSink.Last() on error paths where the sink may
contain a partial event from an intermediate child deletion during
recursive deletes.

* mount: avoid map allocation in collectDirectoryNotifications

Replace the map with a fixed-size array and linear dedup. There are
at most 3 directories to notify (old parent, new parent, new child
if directory), so a 3-element array avoids the heap allocation on
every metadata event.

* mount: fix potential deadlock in enqueueApplyRequest

Release applyStateMu before the blocking channel send. Previously,
if the channel was full (cap 128), the send would block while holding
the mutex, preventing Shutdown from acquiring it to set applyClosed.

* mount: restore signature-based self-event filtering as fast path

Re-add the signature check that was removed when content-based dedup
was introduced. Checking signatures is O(1) on a small slice and
avoids enqueuing and processing events that originated from this
mount instance. The content-based dedup remains as a fallback.

* filer: send snapshotTsNs only in first ListEntries response

The snapshot timestamp is identical for every entry in a single
ListEntries stream. Sending it in every response message wastes
wire bandwidth for large directories. The client already reads
it only from the first response.

* mount: exit read-through mode after successful full directory listing

MarkDirectoryRefreshed was defined but never called, so directories
that entered read-through mode (hot invalidation threshold) stayed
there permanently, hitting the filer on every readdir even when cold.
Call it after a complete read-through listing finishes.

* mount: include event shape and full paths in dedup key

The previous dedup key only used Names, which could collapse distinct
rename targets. Include the event shape (C/D/U/R), source directory,
new parent path, and both entry names so structurally different events
are never treated as duplicates.

* mount: drain pending requests on shutdown in runApplyLoop

After receiving the shutdown sentinel, drain any remaining requests
from applyCh non-blockingly and signal each with errMetaCacheClosed
so callers waiting on req.done are released.

* mount: include IsDirectory in synthetic delete events

metadataDeleteEvent now accepts an isDirectory parameter so the
applier can distinguish directory deletes from file deletes. Rmdir
passes true, Unlink passes false.

* mount: fall back to synthetic event when MetadataEvent is nil

In mknod and mkdir, if the filer response omits MetadataEvent (e.g.
older filer without the field), synthesize an equivalent local
metadata event so the cache is always updated.

* mount: make Flush metadata apply best-effort after successful commit

After filer_pb.CreateEntryWithResponse succeeds, the entry is
persisted. Don't fail the Flush syscall if the local metadata cache
apply fails — log and invalidate the directory cache instead.
Also fall back to a synthetic event when MetadataEvent is nil.

* mount: make Rename metadata apply best-effort

The rename has already succeeded on the filer by the time we apply
the local metadata event. Log failures instead of returning errors
that would be dropped by the caller anyway.

* mount: make saveEntry metadata apply best-effort with fallback

After UpdateEntryWithResponse succeeds, treat local metadata apply
as non-fatal. Log and invalidate the directory cache on failure.
Also fall back to a synthetic event when MetadataEvent is nil.

* filer_pb: preserve snapshotTsNs on error in ReadDirAllEntriesWithSnapshot

Return the snapshot timestamp even when the first page fails, so
callers receive the snapshot boundary when partial data was received.

* filer: send snapshot token for empty directory listings

When no entries are streamed, send a final ListEntriesResponse with
only SnapshotTsNs so clients always receive the snapshot boundary.

* mount: distinguish not-found vs transient errors in lookupEntry

Return fuse.EIO for non-not-found filer errors instead of
unconditionally returning ENOENT, so transient failures don't
masquerade as missing entries.

* mount: make CacheRemoteObject metadata apply best-effort

The file content has already been cached successfully. Don't fail
the read if the local metadata cache update fails.

* mount: use consistent snapshot for readdir in direct mode

Capture the SnapshotTsNs from the first loadDirectoryEntriesDirect
call and store it on the DirectoryHandle. Subsequent batch loads
pass this stored timestamp so all batches use the same snapshot.

Also export DoSeaweedListWithSnapshot so mount can use it directly
with snapshot passthrough.

* filer_pb: fix test fake to send SnapshotTsNs only on first response

Match the server behavior: only the first ListEntriesResponse in a
page carries the snapshot timestamp, subsequent entries leave it zero.

* Fix nil pointer dereference in ListEntries stream consumers

Remove the empty-directory snapshot-only response from ListEntries
that sent a ListEntriesResponse with Entry==nil, which crashed every
raw stream consumer that assumed resp.Entry is always non-nil.

Also add defensive nil checks for resp.Entry in all raw ListEntries
stream consumers across: S3 listing, broker topic lookup, broker
topic config, admin dashboard, topic retention, hybrid message
scanner, Kafka integration, and consumer offset storage.

* Add nil guards for resp.Entry in remaining ListEntries stream consumers

Covers: S3 object lock check, MQ management dashboard (version/
partition/offset loops), and topic retention version loop.

* Make applyLocalMetadataEvent best-effort in Link and Symlink

The filer operations already succeeded; failing the syscall because
the local cache apply failed is wrong. Log a warning and invalidate
the parent directory cache instead.

* Make applyLocalMetadataEvent best-effort in Mkdir/Rmdir/Mknod/Unlink

The filer RPC already committed; don't fail the syscall when the
local metadata cache apply fails. Log a warning and invalidate the
parent directory cache to force a re-fetch on next access.

* flushFileMetadata: add nil-fallback for metadata event and best-effort apply

Synthesize a metadata event when resp.GetMetadataEvent() is nil
(matching doFlush), and make the apply best-effort with cache
invalidation on failure.

* Prevent double-invocation of cleanupBuild in doEnsureVisited

Add a cleanupDone guard so the deferred cleanup and inline error-path
cleanup don't both call DeleteFolderChildren/AbortDirectoryBuild.

* Fix comment: signature check is O(n) not O(1)

* Prevent deferred cleanup after successful CompleteDirectoryBuild

Set cleanupDone before returning from the success path so the
deferred context-cancellation check cannot undo a published build.

* Invalidate parent directory caches on rename metadata apply failure

When applyLocalMetadataEvent fails during rename, invalidate the
source and destination parent directory caches so subsequent accesses
trigger a re-fetch from the filer.

* Add event nil-fallback and cache invalidation to Link and Symlink

Synthesize metadata events when the server doesn't return one, and
invalidate parent directory caches on apply failure.

* Match requested partition when scanning partition directories

Parse the partition range format (NNNN-NNNN) and match against the
requested partition parameter instead of using the first directory.

* Preserve snapshot timestamp across empty directory listings

Initialize actualSnapshotTsNs from the caller-requested value so it
isn't lost when the server returns no entries. Re-add the server-side
snapshot-only response for empty directories (all raw stream consumers
now have nil guards for Entry).

* Fix CreateEntry error wrapping to support errors.Is/errors.As

Use errors.New + %w instead of %v for resp.Error so callers can
unwrap the underlying error.

* Fix object lock pagination: only advance on non-nil entries

Move entriesReceived inside the nil check so nil entries don't
cause repeated ListEntries calls with the same lastFileName.

* Guard Attributes nil check before accessing Mtime in MQ management

* Do not send nil-Entry response for empty directory listings

The snapshot-only ListEntriesResponse (with Entry == nil) for empty
directories breaks consumers that treat any received response as an
entry (Java FilerClient, S3 listing). The Go client-side
DoSeaweedListWithSnapshot already preserves the caller-requested
snapshot via actualSnapshotTsNs initialization, so the server-side
send is unnecessary.

* Fix review findings: subscriber dedup, invalidation normalization, nil guards, shutdown race

- Remove self-signature early-return in processEventFn so all events
  flow through the applier (directory-build buffering sees self-originated
  events that arrive after a snapshot)
- Normalize NewParentPath in collectEntryInvalidations to avoid duplicate
  invalidations when NewParentPath is empty (same-directory update)
- Guard resp.Entry.Attributes for nil in admin_server.go and
  topic_retention.go to prevent panics on entries without attributes
- Fix enqueueApplyRequest race with shutdown by using select on both
  applyCh and applyDone, preventing sends after the apply loop exits
- Add cleanupDone check to deferred cleanup in meta_cache_init.go for
  clarity alongside the existing guard in cleanupBuild
- Add empty directory test case for snapshot consistency

* Propagate authoritative metadata event from CacheRemoteObjectToLocalCluster and generate client-side snapshot for empty directories

- Add metadata_event field to CacheRemoteObjectToLocalClusterResponse
  proto so the filer-emitted event is available to callers
- Use WithMetadataEventSink in the server handler to capture the event
  from NotifyUpdateEvent and return it on the response
- Update filehandle_read.go to prefer the RPC's metadata event over
  a locally fabricated one, falling back to metadataUpdateEvent when
  the server doesn't provide one (e.g., older filers)
- Generate a client-side snapshot cutoff in DoSeaweedListWithSnapshot
  when the server sends no snapshot (empty directory), so callers like
  CompleteDirectoryBuild get a meaningful boundary for filtering
  buffered events

* Skip directory notifications for dirs being built to prevent mid-build cache wipe

When a metadata event is buffered during a directory build,
applyMetadataSideEffects was still firing noteDirectoryUpdate for the
building directory. If the directory accumulated enough updates to
become "hot", markDirectoryReadThrough would call DeleteFolderChildren,
wiping entries that EnsureVisited had already inserted. The build would
then complete and mark the directory cached with incomplete data.

Fix by using applyMetadataSideEffectsSkippingBuildingDirs for buffered
events, which suppresses directory notifications for dirs currently in
buildingDirs while still applying entry invalidations.

* Add test for directory notification suppression during active build

TestDirectoryNotificationsSuppressedDuringBuild verifies that metadata
events targeting a directory under active EnsureVisited build do NOT
fire onDirectoryUpdate for that directory. In production, this prevents
markDirectoryReadThrough from calling DeleteFolderChildren mid-build,
which would wipe entries already inserted by the listing.

The test inserts an entry during a build, sends multiple metadata events
for the building directory, asserts no notifications fired for it,
verifies the entry survives, and confirms buffered events are replayed
after CompleteDirectoryBuild.

* Fix create invalidations, build guard, event shape, context, and snapshot error path

- collectEntryInvalidations: invalidate FUSE kernel cache on pure
  create events (OldEntry==nil && NewEntry!=nil), not just updates
  and deletes
- completeDirectoryBuildNow: only call markCachedFn when an active
  build existed (state != nil), preventing an unpopulated directory
  from being marked as cached
- Add metadataCreateEvent helper that produces a create-shaped event
  (NewEntry only, no OldEntry) and use it in mkdir, mknod, symlink,
  and hardlink create fallback paths instead of metadataUpdateEvent
  which incorrectly set both OldEntry and NewEntry
- applyMetadataResponseEnqueue: use context.Background() for the
  queued mutation so a cancelled caller context cannot abort the
  apply loop mid-write
- DoSeaweedListWithSnapshot: move snapshot initialization before
  ListEntries call so the error path returns the preserved snapshot
  instead of 0

* Fix review findings: test loop, cache race, context safety, snapshot consistency

- Fix build test loop starting at i=1 instead of i=0, missing new-0.txt verification
- Re-check IsDirectoryCached after cache miss to avoid ENOENT race with markDirectoryReadThrough
- Use context.Background() in enqueueAndWait so caller cancellation can't abort build/complete mid-way
- Pass dh.snapshotTsNs in skip-batch loadDirectoryEntriesDirect for snapshot consistency
- Prefer resp.MetadataEvent over fallback in Unlink event derivation
- Add comment on MetadataEventSink.Record single-event assumption

* Fix empty-directory snapshot clock skew and build cancellation race

Empty-directory snapshot: Remove client-side time.Now() synthesis when
the server returns no entries. Instead return snapshotTsNs=0, and in
completeDirectoryBuildNow replay ALL buffered events when snapshot is 0.
This eliminates the clock-skew bug where a client ahead of the filer
would filter out legitimate post-list events.

Build cancellation: Use context.Background() for BeginDirectoryBuild
and CompleteDirectoryBuild calls in doEnsureVisited, so errgroup
cancellation doesn't cause enqueueAndWait to return early and trigger
cleanupBuild while the operation is still queued.

* Add tests for empty-directory build replay and cancellation resilience

TestEmptyDirectoryBuildReplaysAllBufferedEvents: verifies that when
CompleteDirectoryBuild receives snapshotTsNs=0 (empty directory, no
server snapshot), ALL buffered events are replayed regardless of their
TsNs values — no clock-skew-sensitive filtering occurs.

TestBuildCompletionSurvivesCallerCancellation: verifies that once
CompleteDirectoryBuild is enqueued, a cancelled caller context does not
prevent the build from completing. The apply loop runs with
context.Background(), so the directory becomes cached and buffered
events are replayed even when the caller gives up waiting.

* Fix directory subtree cleanup, Link rollback, test robustness

- applyMetadataResponseLocked: when a directory entry is deleted or
  moved, call DeleteFolderChildren on the old path so cached descendants
  don't leak as stale entries.

- Link: save original HardLinkId/Counter before mutation. If
  CreateEntryWithResponse fails after the source was already updated,
  rollback the source entry to its original state via UpdateEntry.

- TestBuildCompletionSurvivesCallerCancellation: replace fixed
  time.Sleep(50ms) with a deadline-based poll that checks
  IsDirectoryCached in a loop, failing only after 2s timeout.

- TestReadDirAllEntriesWithSnapshotEmptyDirectory: assert that
  ListEntries was actually invoked on the mock client so the test
  exercises the RPC path.

- newMetadataEvent: add early return when both oldEntry and newEntry are
  nil to avoid emitting events with empty Directory.

---------

Co-authored-by: Copilot <copilot@github.com>
2026-03-07 09:19:40 -08:00
promalertandChris Lu 9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu d24db396cc fix removing path from inode2path 2022-07-23 18:20:29 -07:00
chrislu 02c2d81cde fix removePathFromInode2Path 2022-07-23 13:21:42 -07:00