3 Commits

Include renames

Author	SHA1	Message	Date
Chris Lu	6a594da739	filer: free orphaned hard-link chunks and remote objects in delete sweeps (#11809) ... * filer: free shared chunks when a folder delete drops the last hard link doBatchDeleteFolderMetaAndData skipped chunk collection for every hard-linked child, relying on maybeDeleteHardLinks to retire the shared inode record. When the delete removes the last link the record is gone and nobody else will ever free those chunks. After the link callback runs, check whether the record survived; if it did not, queue the entry's chunks for deletion. * filer: free orphaned hard-link chunks in the replayed delete sweep The subtree sweep on a replayed directory delete skipped every hard-linked child, so a link that was the inode's last owner leaked its shared chunks. Check the inode record after the store delete; when it is gone, queue the child's chunks like any other swept entry. * filer: mirror remote-object cleanup in the replayed delete sweep Children a lagging peer alone still holds under a replayed directory delete were dropped with plain store deletes: their remote objects leaked and lazy remote fetches could resurrect them. The sweep now records event-stamped tombstones and deletes swept non-directory children through maybeDeleteFromRemoteIfUnchanged, which carries the same stat guard the write-back daemon's processDeleteEvent applies plus a modification-time check — a delayed replay cannot erase an object another writer replaced, even a same-size one. A failed remote delete fails the replay so the event is retried instead of orphaning the object. The directory's own tombstone comes from the event pipeline at the event timestamp and covers the subtree. Metadata-only deletes and cross-cluster events keep the origin's semantics. * filerstore: return real errors from KvGet on MongoDB, ArangoDB, Elastic These stores mapped every lookup failure to ErrKvNotFound, so a database error looked identical to a missing key: hard-link records could appear gone while links still owned their chunks, and path timestamp reads could treat committed state as absent. Return the underlying error and keep ErrKvNotFound for genuinely missing keys. * filer: bind remote-object deletes to the observed object version A guarded stat-then-delete still lets a replacement landing between the two requests go down with the deleted object. Add an optional conditional-delete operation to remote storage clients — S3 If-Match on the live ETag, GCS generation match, Azure IfMatch — and use it everywhere a delete is guarded by a stored stamp: the write-back daemon's delete, rename-source, and pending-rename paths, and the replayed-delete sweep. Backends without it keep the guarded stat-then-delete. * filer: restamp the remote entry after a metadata-only remote update Changing Content-Encoding rewrites the object in place on S3 and advances its LastModified, while the recorded stamp keeps the old mtime. A version-checked delete would then mistake the object for a replacement and keep it forever. Re-stat after UpdateFileMetadata so the saved stamp describes the live object. * filer: mirror the remote-delete ownership checks in the replayed sweep The sweep deleted swept children's remote objects on the listing's snapshot alone. Two gaps: a remote-only entry that took the path since — one a commit record may not cover, such as a lazy fetch materialized from the remote — reads the object as its only copy, and an entry removed while its rename copy was pending leaves the object at the source key orphaned. The per-child delete now runs under the child's entry lock with the commit-time check repeated, keeps a remote-only entry that holds the path, and releases the pending rename's source under the same ownership and stamp conditions the write-back daemon's releasePendingSource applies. * remote.sync: restamp only the object the metadata update touched The live stat after UpdateFileMetadata was saved unconditionally, so a replacement written between the update and the stat stamped the entry with an object it never described — and a later delete would then pass the freshness guard on the replacement's data. The restamp now requires the live object to still match the stored stamp on size and ETag; a different object keeps the old stamp, which the delete guard reads as replaced and keeps. * filer: skip unbudgeted reader-cache test on 32-bit TestReaderCacheUnboundedWithoutBudget proves the reader cache has no implicit memory budget by keeping 96 concurrent 4 MiB downloads in flight (>256 MiB). On linux/386 the weed/filer test binary already runs near the 4 GiB address-space limit, so the burst reliably crashes it with "out of memory" mid-suite. * filer: bail out of a swept child lock instead of waiting on it Replay already holds the event's path locks when the sweep reaches a descendant. A local rename can hold that descendant while waiting on one of the replayed paths, so blocking on the child lock deadlocks both. The sweep now tries the lock and reports ErrLockContended — transient, so the replication retry replays the event once the local write settles. * remote: report the stamp a metadata update produced A metadata write can rewrite the object it touches — S3 changes Content-Encoding through an in-place copy, which can mint a new ETag — so restamping from a fresh stat can reject the very object the update produced, or describe a replacement that landed in between. UpdateFileMetadata now returns the RemoteEntry of the object it wrote (S3 from the copy result, GCS from the update attributes, Azure from a post-update stat); nil means the object was untouched and callers keep the stored stamp. * remote: restamp Azure from the metadata write's own etag	2026-10-18 12:23:30 -07:00
Peter Dodd	5f1a742726	fix: return NoSuchKey and drop stale entries when a remote-mounted object is gone from the remote (#11680) ... A remote-only entry whose object was deleted from the remote storage outside the filer answered GET with 500 and stayed in the filer. The remote's not-found was lost on the way: the backends' ReadFile returned it as an untyped error, so FetchAndWriteNeedle failed with codes.Unknown and nothing downstream could tell it from any other failure. - remote_storage: GCS, S3 (NoSuchKey) and Azure (BlobNotFound) reads return ErrRemoteObjectNotFound. GCS reports a missing bucket the same way as a missing object, so it confirms the bucket with a listing. - volume server and filer: the not-found crosses gRPC as codes.NotFound carrying the sentinel's text, and the filer's cache RPC returns codes.NotFound, which the S3 gateway already maps to NoSuchKey. - s3api: the origin fallback answers NoSuchKey on a confirmed not-found. - filer: a confirmed not-found removes the stale entry, so the lazy remote-metadata cache converges on the remote. Only remote-only files outside .versions and without an active object lock are removed, only if unchanged since the fetch (checked on the object's write owner, under the lock S3 object writes take), and with a metadata-only delete: the filer skips its inline remote delete, and the delete events' entries carry a marker that makes filer.remote.sync and filer.remote.gateway skip their remote delete, while filer.sync still replicates it. The replicated DeleteEntryRequest carries keep_remote_object, so the destination's delete events are marked too. The store drops the marker from every write, so clients cannot plant it.	2026-10-10 11:00:04 +08:00
Chris Lu	60893c5ef3	Classify a filer error before a user-controlled path is wrapped into it (#11004) ... * util, pb: classify a filer error by the status the server sent DoSeaweedListWithSnapshot wrapped a failed ListEntries with %v, dropping the gRPC status, so IsTransientError fell back to matching substrings against a message that now held the caller's path. Keep the status with %w and let it decide, reading the server's own text rather than the wrapper's. Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU * s3: keep the bucket and prefix out of the list retry decision A bucket named transport, or a prefix under logs/unavailable/, made a PermissionDenied listing look transient and got it retried; a key holding the not-found sentence suppressed a retry that should have run. Both checks now read the filer's status, and only fall back to the text when there is none. Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU * filer, s3: classify a delete failure before the path is wrapped into it The filer put the non-empty-folder marker behind its own "delete directory %s" wrapper and the gateway matched it as a substring, so a key named after the marker turned a real delete failure into the demote-the-marker no-op and the request answered 204. Keep the marker leading the message that crosses the wire, turn it back into a sentinel where the response is read, and match that. Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU	2026-08-27 22:30:53 -07:00