filer.remote.sync: do not upload an entry whose only content is the remote object (#11890)

* filer.remote.sync: do not upload an entry whose local copy was dropped

remote.uncache and the filer's eviction under storage pressure drop a
synced entry's chunks and clear LastLocalSyncTsNs; remote.meta.sync does
the same when it pulls a newer remote version. When the remote mtime
falls in the entry's mtime second, which is the common case for a small
file synced promptly, shouldSendToRemote falls back to the cleared stamp
and reads the entry as dirty. The dropped chunks make the update a
content change, so the chunkless entry was uploaded and a zero-byte
object replaced the remote one, the entry's only content.

Skip an update that leaves the entry at the same key with no local data
and a sized RemoteEntry: the object at that key already is the content.
The stamp stays cleared, so remote.cache and eviction still see the
entry as uncached. Renames keep going through completeRemoteOnlyRename,
metadata-only edits of a remote-only entry still reach the remote, and a
rewrite in the remote mtime's second holds local data and is uploaded.

* filer.remote.sync: upload a truncate that leaves the entry remote-only

A FUSE truncate of a cached, remote-backed file to zero (truncateEntry,
or SetAttr with size 0) drops the chunks and keeps the sized
RemoteEntry, so the entry reads as remote-only and the dropped-copy skip
left the remote object with its old content.

Skip only while the entry's size is still the remote object's. Uncache
and eviction keep the size and remote.meta.sync sets it from the
object, so those stay skipped; a truncate changes the size and is
uploaded, in the remote mtime's second too.
This commit is contained in:
Daniel García authored and GitHub committed 2026-10-21 11:32:20 -07:00
1 parent f8cfea0dcf
commit 03065155db
2 files changed
+219

No files matched your search

+19
View File
@@ -384,6 +384,16 @@ func processUpdateEvent(
}
glog.V(0).Infof("never replicated, uploading %s", remote_storage.FormatLocation(dest))
}
if proto.Equal(oldDest, dest) && isDroppedLocalCopy(message.NewEntry) {
// The update dropped the local copy: remote.uncache, eviction under
// storage pressure, or remote.meta.sync pulling a newer remote
// version. Each clears the sync stamp, so shouldSendToRemote reads
// the entry as dirty when the remote mtime shares its second. The
// content is the object at this very key, and uploading the chunkless
// entry would replace it with zero bytes.
glog.V(2).Infof("skipping update of remote-only entry: %+v", resp)
return nil
}
glog.V(2).Infof("update: %+v", resp)
if !proto.Equal(oldDest, dest) {
// A renamed entry that holds no local data now (the snapshot was
@@ -488,6 +498,15 @@ func isRemoteOnly(entry *filer_pb.Entry) bool {
return entry != nil && !filer.HasData(entry) && entry.IsInRemoteOnly()
}
// isDroppedLocalCopy reports a remote-only entry whose size is still the
// remote object's: its local copy was dropped and the object describes it.
// Uncache and eviction keep the size, and remote.meta.sync sets it from the
// object. A truncate drops the chunks too but changes the size, and the
// truncated content has to reach the remote.
func isDroppedLocalCopy(entry *filer_pb.Entry) bool {
return isRemoteOnly(entry) && entry.GetAttributes().GetFileSize() == uint64(entry.RemoteEntry.RemoteSize)
}
// completeRemoteOnlyRename finishes a rename whose content exists only on the
// remote. When the destination object already holds what the entry's stamp
// describes (the rename ran before, its offset was not persisted, and