remote.meta.sync: sync directories and remove files deleted from remote (#10184)

* remote.meta.sync: materialize directory entries, including empty ones

Pull metadata by walking the remote tree one directory level at a time
with a delimiter, so subdirectories, including empty ones, are listed as
their own entries and created locally. The previous flat listing only
returned files, so empty remote directories never appeared locally and
non-empty ones only existed as filer-synthesized parents.

* remote.meta.sync: remove local metadata for entries deleted from remote

After reconciling each directory, drop local entries whose remote source
is gone: files are deleted outright, and a directory removed from the
remote is descended into so its remote-backed children are cleaned while
local-only entries are kept. remote.meta.sync exposes -delete (default
on) and remote.mount.buckets reconciles the same way; a plain
remote.mount stays additive.

* remote.meta.sync: reconcile type swaps and prune emptied directories

- when the remote swaps an entry's type (file <-> directory), drop the
  stale local entry and recreate it with the right type; local-only
  entries are left alone
- mark synced directories remote-backed and clean a directory removed
  from the remote locally, deleting it once it holds no local-only
  entries, instead of re-listing the missing remote path
- treat a differing remote size or mtime, not only a newer mtime, as a
  change worth pulling
This commit is contained in:
Chris Lu authored and GitHub committed 2026-07-01 12:14:19 -07:00
1 parent 9d75048594
commit 746ed82662
3 files changed
+226 -57

No files matched your search

+1 -1
View File
@@ -102,7 +102,7 @@ func (c *commandRemoteMount) Do(args []string, commandEnv *CommandEnv, writer io
}
if strategy == MetadataCacheEager {
if err = pullMetadata(commandEnv, writer, util.FullPath(*dir), remoteStorageLocation, util.FullPath(*dir), remoteConf); err != nil {
if err = pullMetadata(commandEnv, writer, util.FullPath(*dir), remoteStorageLocation, util.FullPath(*dir), remoteConf, false); err != nil {
return fmt.Errorf("cache metadata: %w", err)
}
}