filer.remote.gateway: stop zero-byte uploads of remote-only entries (#12110)

* filer.remote.gateway: skip uploading a dropped local copy on update

The bucketed update path uploaded the chunkless entry whenever the update
was not metadata-only. remote.uncache, eviction, and remote.meta.sync drop
a synced entry's chunks and clear its sync stamp, which reads as dirty
when the remote mtime shares the entry's second; the upload then replaced
the object that is the entry's only content with zero bytes. Skip the
upload the same way the mounted-directory sync does.

* filer.remote.gateway: copy a remote-only entry's object on rename

A rename deleted the source object and uploaded the chunkless entry,
replacing the entry's only content with zero bytes — the same loss
remote.uncache caused on plain updates. Complete the rename on the
remote the way the mounted-directory sync does: copy the source object
over (or take the destination when it already describes the entry),
restamp so reads stop resolving through the pending-source marker, and
remove the source only while it is still the object the stamp describes.
A remote-only entry renamed out of the gateway's coverage keeps its
object, and the plain rename's old-key delete only removes the object
the event's stamp still describes.

* filer.remote.gateway: scope the sync-stamp skip to same-path updates

shouldSendToRemote reads the entry's sync stamp, which still covers a
rename's unchanged mtime, so a synced or pending-stamped rename event was
skipped before its copy or upload could run. Only updates in place may
skip on the stamp.

* filer.remote.gateway: keep the source a pending rename moved on from

A rename event whose pending destination has since been renamed again
still names the source a later event reads through its own marker;
deleting the old key here loses the only copy. Mirror the remote.sync
check: leave the source when the event's entry carries the pending-source
marker and no entry holds the destination any more.

* filer.remote.gateway: release a rename's pending source once complete

A rename whose old entry read through a pending-source marker leaves that
earlier source behind: the destination is released by its own key, never
the marker's. Mirror releaseRenamedSource — once the renamed entry no
longer reads through a marker, delete the pending source guarded by the
old entry's stamp, keeping it while a remote-only entry holds its path.

* filer.remote.gateway: finalize a complete rename before failing its source

A rename source outside the gateway's coverage errored before the
destination was checked, and the recorded failure held the event offset.
Stat the destination first: when it already holds the content the entry
describes, refresh the stamp and leave the unmanaged source untouched;
fail only when nothing holds the content.

* filer.remote.sync: document the source-of-truth protocol both processors share

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Chris LuandDevin authored and GitHub committed 2026-10-26 18:03:12 -07:00
1 parent 0f1e73cc3a
commit 67240f538d
3 files changed
+500 -24

No files matched your search

+21
View File
@@ -26,6 +26,27 @@ import (
"google.golang.org/protobuf/proto"
)
// Which side holds an entry's content, and the two processors that keep the
// remote in step with it.
//
// An entry with local data (chunks or inline content) is authoritative: the
// update uploads it. A chunkless entry stamped with a RemoteEntry is
// remote-only: its content is the remote object its stamp describes, so the
// chunkless entry is never uploaded, since it would write zero bytes over
// the only copy, and the object goes only while the stamp still describes it
// (remote_storage.StillCurrent). While the entry carries a
// filer.ExtRemoteRenameSourceKey marker the content is the object at the
// renamed-from path, until the rename's copy or upload restamps the entry
// and clears it. A dropped local copy (remote.uncache, eviction,
// remote.meta.sync) keeps the remote object's size; a size that moved off
// the stamp (a truncate) is new content and uploads.
//
// processUpdateEvent serves filer remote.sync over one mounted directory;
// makeBucketedEventProcessor in filer_remote_gateway_buckets.go serves filer
// remote.gateway over the bucketed space. They implement the same protocol
// and must change together: a guard added to one and not the other is how
// the zero-byte re-upload on remote.uncache happened.
func followUpdatesAndUploadToRemote(option *RemoteSyncOptions, filerSource *source.FilerSource, mountedDir string) error {
// read filer remote storage mount mappings