* shell: keep fs.mergeVolumes from reverting entries written during the merge
The walk reads a page of entries and moves their chunks one at a time, then
wrote each entry back unconditionally. A file overwritten in between got its
old version back, and the filer deleted the new version's chunks as garbage,
so an acknowledged write was lost without any error.
The update now carries IF_ENTRY_EQUAL over the entry as the walk read it. When
the filer refuses because the entry changed or was deleted, the copies made on
the target volumes are deleted, their plan room is released, and the entry is
skipped; the source needles stay where the filer still points.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: keep fs.distributeChunks from reverting a file overwritten while it runs
The command looks the entry up once, moves its chunks, and then wrote that
entry back unconditionally, so an overwrite during the redistribution was
reverted and the filer deleted the new version's chunks as garbage.
The update now carries IF_ENTRY_EQUAL over the looked-up entry. When the filer
refuses because the file changed or was deleted, the moved copies and any new
manifest chunks are deleted and the command reports it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: keep s3.versions.audit -heal from clearing a pointer that just moved
The heal cleared the latest-version pointer on the directory entry the walk
read, unconditionally. A version written between the audit and the heal has
already pointed the directory at a live file, and clearing that pointer hid it.
The heal now carries IF_ENTRY_EQUAL over the audited entry and reports a
refusal as a failed heal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: keep fs.meta.changeVolumeId from reverting entries written meanwhile
The walk rewrote each entry's volume ids and wrote back the copy it had read,
unconditionally, so an overwrite between the read and the write was reverted
and its chunks were deleted as garbage. The update now carries IF_ENTRY_EQUAL
over the entry as read, and a refused entry is skipped and reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: keep the target copies another fs.mergeVolumes run committed
moveChunk keeps the key and cookie, so two overlapping merges of the same
source into the same target write the same fid. When one run commits, the
other's update is refused, and deleting its copies deleted the needle the
committed entry now points at.
A refused update now re-reads the entry, resolving its manifests, and deletes
only the copies it no longer references. If the entry cannot be read, the
copies are left for volume.fsck.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: let a later hard-link sibling retry a merge the filer refused
The first sibling of a HardLinkId claims the move for all of them. When its
update is refused, the claim made every later sibling skip, so the shared
chunks stayed on the source volume for the rest of the run. A refused update
now releases the claim, and a sibling listed after the change retries the move
against the entry as it is now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: delete the manifest blobs a failed fold in fs.distributeChunks saved
When re-manifestizing failed partway, the blobs already uploaded were only
counted in a log line and stayed on their volumes with nothing referencing
them. They have freshly assigned fids, so they are now deleted with the
cookie-checked delete the other orphan cleanups use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: check the heal clears the pointer file name too
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: require the shell lock for fs.mergeVolumes -apply
moveChunk keeps the key and cookie, so two overlapping runs that move the same
chunk into the same volume write the same fid. Re-reading the entry before
deleting a refused run's copies covers the run that already committed, but not
one that commits after that read. With the lock, two applying runs can no
longer overlap, as volume.fsck and volume.merge already require. A dry run
still works without it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: check unreferencedTargets reads the refused entry itself
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: check unreferencedTargets keeps a copy a manifest references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* shell: check unreferencedTargets keeps the manifest copy and skips an unreadable manifest
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* shell: expand `~` in local file path arguments
The weed shell parses commands itself instead of going through an OS
shell, so a path like `~/Downloads/foo.meta` was passed verbatim to
`os.Open`, which fails because no `~` directory exists. Users had to
spell out absolute home paths in every command.
Add an `expandHomeDir` helper that resolves a leading `~` or `~/...` to
the user's home directory, and run user-supplied local file paths in
the affected shell commands through it:
fs.meta.load (positional file)
fs.meta.save (-o)
fs.meta.changeVolumeId (-mapping)
s3.iam.export (-file)
s3.iam.import (-file)
s3.policy (-file)
s3tables.bucket (-file)
s3tables.table (-file, -metadata)
volume.fsck (-tempPath)
Filer-namespace path flags (`-dir`, `-path`, `-locationPrefix`, etc.)
are unaffected; they live in the filer, not on the local FS.
* shell: reuse util.ResolvePath instead of a new helper
util.ResolvePath already does tilde expansion; drop the local
expandHomeDir helper and route every shell call site through it.
* Add TraverseBfsWithContext and fix race conditions in error handling
- Add TraverseBfsWithContext function to support context cancellation
- Fix race condition in doTraverseBfsAndSaving using atomic.Bool and sync.Once
- Improve error handling with fail-fast behavior and proper error propagation
- Update command_volume_fsck to use error-returning saveFn callback
- Enhance error messages in readFilerFileIdFile with detailed context
* refactoring
* fix error format
* atomic
* filer_pb: make enqueue return void
* shell: simplify fs.meta.save error handling
* filer_pb: handle enqueue return value
* Revert "atomic"
This reverts commit 712648bc35.
* shell: refine fs.meta.save logic
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* Unify the parameter to disable dry-run on weed shell commands to --apply (instead of --force).
* lint
* refactor
* Execution Order Corrected
* handle deprecated force flag
* fix help messages
* Refactoring]: Using flag.FlagSet.Visit()
* consistent with other commands
* Checks for both flags
* fix toml files
---------
Co-authored-by: chrislu <chris.lu@gmail.com>
