* filer: self-heal fetchWholeChunk on stale volume locations
Upstream #10156/#10800 wired cache invalidation into the buffer-based
read paths, but manifest resolution still goes through fetchWholeChunk,
which returns the raw error on failure. When cached volume locations
are stale (volume tiered to remote storage, server rolled), resolving
a large multipart file fails permanently even though other locations
are healthy.
Thread the ChunkGroup's cacheInvalidator through ResolveChunkManifest /
ResolveOneChunkManifest / fetchWholeChunk, and on failure invalidate,
re-lookup and retry once via the existing retryFetchWithFreshLocations
helper. The streaming bytesBuffer is reset before the retry so partial
bytes from the failed attempt cannot corrupt the manifest
proto.Unmarshal. Non-mount callers pass nil and keep their semantics.
* filer: move the manifest self-heal tests in with the other manifest tests
Also make the stale server stream a prefix and then abort mid-body, which is
what actually leaves partial bytes in the buffer: an HTTP error status returns
before ReadUrlAsStream ever calls the writer, so a 500 never exercised the
Reset the tests claimed to cover.
Claude-Session: https://claude.ai/code/session_01FK3oGC5ZVeJYvNBWgb9JUD
* filer: keep the cached volume locations when a manifest read is cancelled
A cancelled or timed-out read says nothing about where the volume lives, so
dropping the location and going back to the master only costs the next reader
a round trip. PrepareStreamContentWithThrottler already guards its self-heal
this way. The guard also goes inside retryFetchWithFreshLocations, since the
caller can be cancelled between its own check and the invalidation, and that
covers the reader cache and prefetch paths too.
fetchWholeChunk returns the context error rather than the stream failure it
provoked, and ResolveOneChunkManifest wraps with %w so errors.Is still sees it.
That matters even where no invalidator is passed: volume.fsck resolves
manifests with nil and tells its own abort from a corrupt manifest that way,
so the cancellation check sits ahead of the nil-invalidator return.
Claude-Session: https://claude.ai/code/session_01FK3oGC5ZVeJYvNBWgb9JUD
* filer: self-heal manifest reads on the filer and s3 paths too
Every caller that already holds the location cache backing its lookup function
can hand it over: the filer's read, copy and deletion paths and the log cache
have the MasterClient right there, and s3api has the FilerClient. MinusChunks
takes one for the same reason, since the deletion path resolves manifests
through it. Only the shell tools and the replication sinks, whose lookup
functions cache privately with nothing to invalidate, keep passing nil.
Claude-Session: https://claude.ai/code/session_01FK3oGC5ZVeJYvNBWgb9JUD
---------
Co-authored-by: bruce-zzz <bruce.zou@hhy-data.com>
* shell: clean up the target copy when a merge upload fails
A replicated write commits the needle to the local volume before it fans out
to the other replicas, so an upload that reports failure can still have left a
copy on the target. fs.mergeVolumes printed "failed to move" and carried on,
so that copy stayed behind forever: the filer is never re-pointed at it, and
nothing else knows it exists.
One sick replica orphans roughly half the chunks of a merge, two thirds with
three copies, since the entry node is picked at random from the replicas and
the replica upload uses MaxAttempts 1. A volume with a single copy has no such
window: the write is one local append that either succeeds or leaves nothing.
Delete the needle we may have written before continuing. The source side
already did exactly this, so deleteMovedSourceNeedles is renamed to
deleteOrphanedNeedles and reused for both ends.
Claude-Session: https://claude.ai/code/session_01XjiMGK72F4Gs3yWNJhnVjy
* shell: verify the cookie before deleting a merge target needle
The target cleanup deletes a needle an upload may or may not have written, and
BatchDelete matches on the needle id alone. Needle ids come from one global
sequence, so normally nothing else can hold that id — but a volume restored
from elsewhere, or one written either side of a master sequence reset, can, and
then a failed move deletes a live needle out from under its filer entry.
Have the volume server verify the cookie for those. Source needles keep
deleting by id: they are the ones the filer just pointed at, matching every
other filer-driven delete.
Claude-Session: https://claude.ai/code/session_01XjiMGK72F4Gs3yWNJhnVjy
* shell: delete the target copies an abandoned manifest rewrite leaves
rewriteManifestChunk moves sub-chunks one at a time and only then uploads the
rewritten manifest. Every error after the first successful move — a nested
rewrite failing, the marshal, the manifest upload — returned without touching
the copies already written to the target volumes. The filer keeps pointing at
the old manifest, so those copies orphan, one per sub-chunk moved so far.
Track them alongside the sources and delete them on the way out. Nested
rewrites hand theirs up so an outer failure clears the whole subtree.
A failed UpdateEntry deliberately still leaks its copies: that error can also
mean the filer applied the update and lost the response, and deleting there
would turn a leak into data loss.
Claude-Session: https://claude.ai/code/session_01XjiMGK72F4Gs3yWNJhnVjy
* shell: give the plan its room back when a manifest rewrite is abandoned
allocate reserves plannedSize against the chosen target for every move a
multi-target source makes, and release hands it back when the move fails.
Abandoning a manifest rewrite now deletes the copies that did land, so those
reservations stopped matching anything on disk: the plan kept counting bytes
that are gone and refused later chunks with "no target volume has room".
Release them alongside the delete. Nested rewrites hand theirs up so an outer
failure unwinds the whole subtree's accounting.
Claude-Session: https://claude.ai/code/session_01XjiMGK72F4Gs3yWNJhnVjy
* filer_pb: walk a re-delivered directory only once in TraverseBfs
A directory handed back twice by a listing (a page-boundary race with
concurrent renames, or a store whose ordering misbehaves) was enqueued
twice; the second walk re-lists the same subtree and can keep the
traversal from ever terminating.
Claude-Session: https://claude.ai/code/session_01XH7iM88ZqWMEvsLB8tkWPQ
* filer_pb: fail a directory listing whose pagination stops advancing
A full page ending on the very name the cursor started from re-fetches
the same page forever; a store whose listing order does not advance past
the cursor turns any full-directory read into a silent infinite loop.
Return an error naming the stuck cursor instead.
Claude-Session: https://claude.ai/code/session_01XH7iM88ZqWMEvsLB8tkWPQ
* shell: skip foreign-collection manifests in fs.mergeVolumes
Every manifest chunk in the namespace was resolved, downloading its
manifest needle, even when the merge plan only touches one collection.
Sub-chunks live in the manifest's own collection, so a manifest on a
volume outside the plan's collections cannot reference a source volume;
skip it and spare a cluster-wide download pass that looks like a hang
after the real moves finish.
Claude-Session: https://claude.ai/code/session_01XH7iM88ZqWMEvsLB8tkWPQ
* wdclient: bound the wait for a master leader by the caller's context
WithClient waited on GetMaster with context.Background(), so a caller that
arrived while no master leader was known parked in a 200ms poll loop until one
appeared, whatever deadline it had already set on the RPC. Each retry above it
then left another goroutine in the same wait.
Take the context in WithClient and WithClientCustomGetMaster and hand it to
GetMaster, and stop the retry loop once it is done. The dial keeps
context.Background(): fn brings its own RPC context, so a cancellation seen
here cannot be attributed to the shared connection.
Call sites pass whatever they hold: the request context in the filer's
CollectionList, DeleteCollection and Statistics handlers and in the credential
store's propagation, the operation context in the shell's s3.bucket.delete and
the kafka gateway's broker and filer discovery, and context.Background() where
there is none - the shell commands, the admin dashboard wrapper, and the
exclusive locker's initial lease. The locker's release keeps its own
uncancelled context so a slow unlock cannot turn into a ghost lock.
Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU
* wdclient: test that WithClient gives up with the caller's context
Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU
* wdclient: cut the master retry backoff short when the caller gives up
util.Retry sleeps unconditionally between attempts, so a transient error
arriving just before the caller's deadline still cost it a full backoff step.
Use the context-aware util.RetryWithBackoff, the same helper the volume lookup
in this file already uses.
Two call sites went with it: the shell's lock-holder lookup builds its three
second bound before WithClient so it also covers finding the leader, as its
comment already promised, and the filer's post-delete collection cleanup goes
back to an uncancelled context - the entry is already gone, so a caller that
hung up must not leave the collection behind.
Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU
* wdclient: test that a cancel during backoff ends the retry
Claude-Session: https://claude.ai/code/session_01BjDWtZsCoZY6x4pdDmGWxU
* master: stream volume listings
A listing of 800k volumes is 36MB on the wire but 305MB as messages, and the
master built all of it, then held it while grpc encoded it. Two of those at
once is most of a small master's heap, and the maintenance scanner asks every
30 minutes.
The topology goes out first, listing nothing, then its volumes in batches, so
the master holds a batch rather than a cluster: 341MB of live heap for one
listing becomes 4.4MB. It allocates much the same either way -- what changes is
how much of it has to be live at once, which is what sets the heap ceiling.
Batches are built under their disk's lock and sent outside it, so a slow reader
stalls the stream rather than the topology. They therefore do not share one
instant, which a single listing did not either: it takes each disk's lock in
turn, so a volume moving during either can be seen twice or not at all.
The client helper hides which kind of master answered: one too old for the
stream is asked the old way and its reply cut into the same batches. Either way
the topology handed over lists no volumes, so a caller cannot come to depend on
finding them there.
* admin: stream the listing the maintenance scan reads
It asks for every volume in the cluster every 30 minutes. Reassembling it
client-side keeps the scan identical -- ActiveTopology splits disks by the
disk ids on the volumes, so it needs them in the topology -- while the master
no longer builds the whole reply to send it.
* topology: report a disk id that does not depend on map order
A topology disk that fronts several physical disks took its reported id from
whichever volume the map yielded first, so two listings of an unchanged disk
could disagree. Take the smallest instead.
* topology: test that a streamed listing rebuilds to the whole one
The callers that stream now rebuild the listing from a topology sent without
volumes plus the batches after it, so that has to come out the same as being
sent it whole, at every batch size and under a filter.
* clients: stream the volume listings that ask for everything
The dashboard's list and export pages, the collection and ec shard pages, the
topology view, the worker metrics and two shell commands each asked the master
to build all 800k volumes into one reply. They read the same listing as before,
rebuilt on their side, so the master no longer holds it.
The three that already ask for one volume or one collection stay as they are:
their replies are small, and streaming one costs a round trip to say so.
VolumeLocationList.Stats subtracts the deleted figures from the totals to
report live size and needle count. Both deleted figures are maintained as
counters independent of the totals they come off, so either can transiently
exceed its total, and neither subtraction was clamped.
Unclamped, the size wraps to ~16 EB. The count is signed so it merely goes
negative, but VolumeLayout.Stats converts it with uint64(fileCount), which
turns it into ~1.8e19 just the same. Either one swamps the cluster totals
behind /dir/status, /vol/status and Topology.CollectionVolumeStats.
commandFsMergeVolumes.getVolumeSize had the same unclamped subtraction, where
a wrapped size reads as a volume far too large to join any merge plan.
Clamped to zero, matching the guards already in CollectionInfo.LogicalSize
and the admin server's logical-size accumulator.
* fix(shell): honor explicit fs.mergeVolumes from/to direction
mergeVolumes only ever merged a smaller volume into a larger one. When the
user named both -fromVolumeId and -toVolumeId with the source larger than the
target, the planner produced an empty plan and the command printed just
"max volume size: N MB" and moved nothing.
Build the requested pair directly when both ids are given, instead of routing
through the size-descending heuristic. Read-only, empty, and wrong-collection
endpoints are rejected with a clear error rather than a silent no-op.
* fix(shell): allow fs.mergeVolumes into an empty target volume
Merging chunks into an empty volume is valid, e.g. consolidating data into a
freshly created or recently vacuumed volume. Only reject an empty source, which
has nothing to move.
* fix(shell): reject self-map in directed mergeVolumes planner
createMergePlan with from == to returned a {vid: vid} self-merge when called
directly. Guard it in the planner so it is correct independent of the Do
entrypoint.
* fix(mount): sanitize non-UTF-8 filenames; keep marshal errors per-request (#9139)
A single file with invalid-UTF-8 bytes in its name (e.g. a GNOME Trash
"partial" like \x10\x98=\\\x8a\x7f.trashinfo.9a51454f.partial) made every
FUSE-initiated filer RPC fail with:
rpc error: code = Internal desc = grpc: error while marshaling:
string field contains invalid UTF-8
and then produced an avalanche of "connection is closing" errors on
unrelated LookupEntry / ReadDirAll / UpdateEntry calls, causing the
volume-server QPS dips reported in #9139.
Root cause is twofold:
1. Proto3 `string` fields require valid UTF-8, but the FUSE kernel passes
raw name bytes. Create/Mknod/Mkdir/Unlink/Rmdir/Rename/Lookup/Link/
Symlink all forwarded those bytes directly into CreateEntryRequest.Name,
DeleteEntryRequest.Name, StreamRenameEntryRequest.{Old,New}Name and
Entry.Name. saveDataAsChunk also copied the FullPath into
AssignVolumeRequest.Path unchecked.
2. When the marshal failed, shouldInvalidateConnection treated the
resulting codes.Internal as a connection problem and dropped the
shared cached ClientConn — canceling every other in-flight RPC on it.
Fix:
- Add sanitizeFuseName (strings.ToValidUTF8 with '?' replacement, matching
util.FullPath.DirAndName) and make checkName return the sanitized name.
Apply at every FUSE entry point that passes a name to the filer RPC,
including Unlink/Rmdir (which did not previously call checkName) and
both oldName/newName in Rename. Add a backstop scrub for
AssignVolumeRequest.Path so async flush paths cannot reintroduce
invalid bytes from a pre-sanitization cached FullPath.
- In weed/pb.shouldInvalidateConnection, detect client-side marshal
errors via the gRPC library's "error while marshaling" prefix and
return false: the connection is healthy, only the request is bad.
Refs: https://github.com/seaweedfs/seaweedfs/issues/9139#issuecomment-4301184231
* fix(mount,util): use '_' for invalid-UTF-8 replacement (URL-safe)
Sanitized filenames flow downstream into HTTP URLs (volume-server uploads,
filer HTTP API, S3/WebDAV gateways). '?' is the URL query-string
delimiter and would split the path the first time the name lands in one,
so swap every invalid-UTF-8 replacement to '_'. This covers the two
pre-existing sites in weed/util/fullpath.go as well, keeping all paths
sanitized the same way.
* refactor(pb): detect client-side marshal errors via errors.As, not substring
Replace the raw `strings.Contains(err.Error(), ...)` check with a
type-based carve-out: use errors.As against the `GRPCStatus() *Status`
interface to pull the original Status out of any fmt.Errorf("...: %w")
wrapping, then match the library-owned "grpc:" prefix on that Status's
Message.
Why not errors.Is against a proto-level sentinel: gRPC's encode()
collapses the inner proto error with "%v" (stringification) before
wrapping it in a Status, so the original error type does not survive
into the caller. The Status itself is the structural signal that does
survive.
Why not status.FromError: when the caller wraps the Status error with
fmt.Errorf("...: %w", ...), status.FromError rewrites Status.Message
with the full err.Error() of the outermost wrapper, which defeats a
prefix check on the library-owned message. errors.As gives us the
original Status whose Message is still verbatim from the gRPC library.
A new test asserts that a plain errors.New("grpc: error while marshaling: …")
— i.e. the same text attached to something that is NOT a gRPC status —
does not short-circuit invalidation, so we never silently keep a cached
connection alive based on a coincidental substring match.
* refactor(util): centralize UTF-8 sanitization; add FullPath.Sanitized
Addresses review feedback on PR #9207.
Nitpick: every invalid-UTF-8 replacement across the codebase (DirAndName,
Name, mount.sanitizeFuseName, the weedfs_write.go backstop) now goes
through a single util.SanitizeUTF8Name helper, so the replacement char
('_' — URL-safe) is chosen in one place.
Outside-diff: three proto fields took raw FullPath strings that could
break marshaling if an entry ever carried invalid UTF-8
(CreateEntryRequest.Directory in Mkdir, DeleteEntryRequest.Directory in
Unlink, AssignVolumeRequest.Path in command_fs_merge_volumes). The
reviewer's suggested fix — using DirAndName() — would have silently
changed Directory from parent to grandparent, because DirAndName
sanitizes only the trailing component. Added FullPath.Sanitized(), which
scrubs every component, and applied it at the three sites. Exposure is
narrow in practice (FUSE-boundary sanitization and the gRPC-side
isClientSideMarshalError carve-out already cover the #9139 cascade),
but the defense-in-depth is cheap and consistent with the existing
AssignVolume backstop.
New tests in weed/util/fullpath_test.go document:
- SanitizeUTF8Name: valid UTF-8 passes through unchanged; invalid bytes
become '_' (not '?', which is URL-special).
- FullPath.Sanitized: scrubs bytes in any component, not just the last.
- FullPath.DirAndName: dir remains raw on purpose — callers needing a
clean full path must use Sanitized(). The test pins this behavior so
it is not accidentally "fixed" in a way that changes the (dir, name)
semantics callers depend on.
* fix(shell): skip hard-linked entries in fs.mergeVolumes
Hard-linked entries share a chunk list with their siblings, but the
filer's UpdateEntry only rewrites one entry at a time. Moving a chunk
here leaves every other hard-linked sibling pointing at a fid that
either gets deleted by the filer's own garbage step after UpdateEntry
or by deleteMovedSourceNeedles (#9160) — either way, the siblings end
up with dangling references.
Skip the entry with a visible log line so operators know the file was
bypassed and can handle it explicitly (copy-then-unlink, or dedup
before merge). Detected via entry.HardLinkId being non-empty, which is
the same signal the filer itself uses (weed/pb/filer_pb/filer.pb.go:451).
Flagged by coderabbit on #9160 post-merge.
* fix(shell): mergeVolumes suppresses 404 alongside 304 in source cleanup
BatchDelete also returns StatusNotFound (404) for an already-deleted
needle when ReadVolumeNeedle can't find it in the cookie-check path,
not only StatusNotModified (304) from DeleteVolumeNeedle returning
size 0. Both are benign races against a concurrent fsck purge or a
replica that already reconciled, so don't clutter the output with
"delete ... not found" warnings for them.
Flagged by coderabbit on #9160 post-merge.
* fix(shell): mergeVolumes merges hard-linked files via dedup on HardLinkId
Hard-linked siblings share one chunk list through a KV blob keyed by
HardLinkId (see weed/filer/filerstore_hardlink.go). UpdateEntry's
setHardLink rewrites that blob and maybeReadHardLink overrides per-entry
chunks with the blob's on every read, so a single UpdateEntry propagates
new fids to every sibling automatically — the previous skip-hardlinks
bailout was overly conservative and left hard-linked files stuck on
merge-source volumes forever.
Process each HardLinkId exactly once per run with a sync.Map so BFS
workers in different directories synchronize without a global lock.
First sibling carries the chunk move + UpdateEntry; later siblings find
the id in the map and return — preventing the real race, which is two
siblings trying to re-download an already-moved source needle or
double-queue the same fid for deletion.
Also address the log-spam review on deleteMovedSourceNeedles: an
unreachable volume server returns one error per needle, so collapse
multiple failures into a single per-server line with the first error as
an example.
* fix(shell): error on missing volume id in fsck, mergeVolumes, vacuum
Three shell commands silently report success when -volumeId /
-fromVolumeId / -toVolumeId names a volume the master doesn't know
about: typos, already-deleted volumes, and stale scripts all look
identical to a clean no-op, which is what made the confusion in #9116
take as long as it did to diagnose.
- volume.fsck: filter at the per-datanode loop drops unknown ids and
findExtraChunksInVolumeServers ends with totalOrphanChunkCount==0,
printing "no orphan data".
- fs.mergeVolumes: createMergePlan iterates only known volumes, so an
unknown -fromVolumeId produces an empty plan and we print just the
"max volume size: N MB" header (indistinguishable from "nothing to
merge").
- volume.vacuum: the master's VacuumVolume RPC silently iterates
matching volumes; a missing id returns success having done nothing.
Validate the requested ids against the current topology up front and
return an explicit "volume(s) not found on master: [X Y]" error. Also
drop a stale duplicate `if err != nil` in volume.fsck.Do left over from
a prior refactor.
Surfaces #9116 follow-up from madalee-com.
* address review: propagate reloadVolumesInfo error; dedupe vacuum missing ids
- fs.mergeVolumes: c.reloadVolumesInfo's return was ignored. If the
master is unreachable or VolumeList fails, c.volumes stays empty and
the new validation block reports "fromVolumeId X not found on master"
— masking the real connection/RPC failure. Return the wrapped error
instead.
- volume.vacuum: "volume.vacuum -volumeId 5,5,5" on a missing volume 5
listed [5 5 5] in the error. Collect missing ids in a set so each
missing id appears once.
* address review: reject fromVolumeId/toVolumeId values that overflow uint32
flag.Uint produces a uint (64-bit on amd64), and the existing cast to
needle.VolumeId silently truncates to uint32. A typo like
`-fromVolumeId=4294967297` would wrap to volume 1 and slip past every
other validation, so the merge would run against a completely
different volume than the operator intended.
Bail out with an explicit error when the raw flag value exceeds the
uint32 range, before the cast.
* feat(shell): fs.mergeVolumes deletes source needles after filer update
Before this change, mergeVolumes only copied chunks to the destination
volume and updated the filer — the source needle sat untouched on its
original volume as a silent orphan. Operators had to run a separate
volume.fsck + volume.vacuum pass to actually reclaim the space, and
#9116 (comment 4282692876) showed how that pipeline can look exactly
like "mergeVolumes did nothing": the source volume keeps reporting its
original size even though every chunk has been logically moved out.
Clean up the source inline. For each entry, track the pre-move fids as
they're captured, and after the UpdateEntry RPC commits, issue
BatchDelete on every replica of each source volume. Key invariants:
- Source fids are only deleted AFTER UpdateEntry succeeds; if the
filer write fails we skip the cleanup for that entry so we never
delete data the filer still references.
- rewriteManifestChunk grew a fourth return value so nested manifest
and sub-chunk moves propagate their moved-source list back to the
top-level callsite. The outer manifest itself is recorded at the
callsite, since only the callsite sees the pre-rewrite fid.
- deleteMovedSourceNeedles logs errors but never returns them.
Propagating would abort TraverseBfs mid-merge, stranding remaining
entries; logging leaves the fallback path (fsck reconciles later)
intact.
- StatusNotModified from the volume server is expected whenever a
concurrent fsck purge beat us to the delete or a replica already
reconciled — don't warn on it.
Readonly source volumes are already rejected up front by
createMergePlan, so by the time we reach the delete the source is
writable. If a replica's readonly bit has flipped since then the
delete will fail and get logged; the user can re-run once they've
fixed the replica (same failure mode as today's fsck purge).
Fixes the space-not-reclaimed half of #9116.
Related design discussion: #8589.
* address review: cast r.Status to int in StatusNotModified compare
http.StatusNotModified is an untyped constant so the compare works as
written, but the int32/int mixed-type signal trips static analyzers
and PR tooling. Cast explicitly and note why.
* fix(shell): fs.mergeVolumes now rewrites manifest chunks for large files
Previously fs.mergeVolumes skipped any chunk whose IsChunkManifest flag was
true, printing "Change volume id for large file is not implemented yet" and
continuing. Because the BFS traversal only looks at top-level
entry.Chunks, sub-chunks referenced inside a manifest were never
considered either. For any file stored as a chunk manifest (large files
go this path), chunks in the source volume stayed put, leaving behind a
few MB of live data that vacuum and volume.deleteEmpty couldn't clean
up.
This change resolves each manifest chunk recursively, moves any
sub-chunk whose volume id is in the merge plan via the existing
moveChunk path, and re-serializes the manifest. If the manifest chunk
itself lives in a source volume, or any sub-chunk moved, the new
manifest blob is uploaded to a freshly assigned file id (the old
needle becomes orphaned and is reclaimed by vacuum like any other
moved chunk).
Fixes#9116.
* address review: batch UpdateEntry, fix dry-run, defer restore, avoid source volumes
- Call UpdateEntry once per entry after the chunk loop instead of once per
moved chunk (gemini nit).
- In dry-run mode, mark anySubChanged when a sub-chunk in the plan is
encountered and return changed=true after printing "rewrite manifest",
so nested manifests also surface their would-rewrites (gemini nit).
- Defer filer_pb.AfterEntryDeserialization so the manifest chunk list is
restored even when proto.Marshal fails (coderabbit nit).
- Reject AssignVolume results whose file id lands on a volume that is a
source in the merge plan, and retry — otherwise the replacement
manifest could be written to the volume being emptied (coderabbit).
The log message was comparing against the planned size of the destination
volume (including volumes already planned to merge into it) but only
displaying the raw volume size, making the output confusing when the
displayed sizes clearly didn't add up to exceed the limit.
* Add TraverseBfsWithContext and fix race conditions in error handling
- Add TraverseBfsWithContext function to support context cancellation
- Fix race condition in doTraverseBfsAndSaving using atomic.Bool and sync.Once
- Improve error handling with fail-fast behavior and proper error propagation
- Update command_volume_fsck to use error-returning saveFn callback
- Enhance error messages in readFilerFileIdFile with detailed context
* refactoring
* fix error format
* atomic
* filer_pb: make enqueue return void
* shell: simplify fs.meta.save error handling
* filer_pb: handle enqueue return value
* Revert "atomic"
This reverts commit 712648bc35.
* shell: refine fs.meta.save logic
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
