From 28fb2f90260ee78f4fe1694cfce783d13395b4b7 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Thu, 15 Jan 2026 00:55:15 -0800 Subject: [PATCH] docs: add remote.copy.local documentation and command comparison table Added comprehensive documentation for the new remote.copy.local command including usage examples, filters, and options. Also added a comparison table showing the differences between remote.cache, remote.uncache, and remote.copy.local commands, along with a typical workflow example. --- Cache-Remote-Storage.md | 53 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/Cache-Remote-Storage.md b/Cache-Remote-Storage.md index 263b662..ee550d8 100644 --- a/Cache-Remote-Storage.md +++ b/Cache-Remote-Storage.md @@ -46,6 +46,59 @@ In `weed shell`, checkout `remote.cache` and `remote.uncache`: ``` +## Copy Local-Only Files to Remote + +The `remote.copy.local` command synchronizes local-only files to remote storage. This is useful when you have files that were created locally and need to be backed up to remote storage. + +``` +> help remote.copy.local + remote.copy.local # copy local-only files to remote storage for mounted directories + + # copy all local-only files in a mounted directory to remote + remote.copy.local -dir=/xxx + + # copy with filters + remote.copy.local -dir=/xxx/some/sub/dir -include=*.pdf + remote.copy.local -dir=/xxx/some/sub/dir -exclude=*.txt + remote.copy.local -minSize=1024000 # copy files larger than 100K + remote.copy.local -maxSize=10240000 # copy files smaller than 10MB + + # force update even if remote file exists + remote.copy.local -dir=/xxx -forceUpdate=true + + # dry run to see what would be copied + remote.copy.local -dir=/xxx -dryRun=true + + This command only copies files that don't have remote entries yet. + Files that are already synchronized with remote storage are skipped. + + The actual data copying goes through volume servers in parallel. +``` + +## Command Comparison + +Here's a comparison of the three main remote storage commands: + +| Feature | `remote.cache` | `remote.uncache` | `remote.copy.local` | +|---------|---------------|------------------|---------------------| +| **Purpose** | Download files from remote to local cache | Remove local cache, keep metadata | Upload local-only files to remote | +| **Data Flow** | Remote → Local | Local → (delete) | Local → Remote | +| **When to Use** | Warm up cache, pre-fetch data | Free up local storage | Backup local files to remote | +| **File Selection** | Files in remote storage | Cached files | Local-only files (no remote entry) | +| **Skips Files** | Already cached files | Files not in remote | Files already in remote | +| **Force Option** | N/A | N/A | `-forceUpdate` to overwrite remote | +| **Common Filters** | `-include`, `-exclude`, `-maxSize`, `-maxAge` | `-include`, `-exclude`, `-minSize`, `-minAge` | `-include`, `-exclude`, `-minSize`, `-maxSize` | +| **Dry Run** | `-dryRun` | `-dryRun` | `-dryRun` | +| **Use Case Example** | Cache frequently accessed files | Uncache old/large files to save space | Upload newly created local files | + +### Typical Workflow + +1. **Mount remote storage**: `remote.mount -dir=/buckets/mybucket -remote=s3_1/bucket` +2. **Create local files**: Write files directly to `/buckets/mybucket/` +3. **Backup to remote**: `remote.copy.local -dir=/buckets/mybucket` (uploads local-only files) +4. **Free up space**: `remote.uncache -dir=/buckets/mybucket -minSize=10240000` (remove large files from cache) +5. **Warm up cache**: `remote.cache -dir=/buckets/mybucket -include=*.pdf` (pre-fetch PDFs) + ## Note If you want some more flexibility to decide which file to cache or uncache, please help to send a PR.