weed/command: skip remote bucket delete when bucket is unmounted (#12418) (#12425)

* weed/command: skip remote bucket delete when bucket is unmounted

* remote_storage: count deleting an already-missing bucket as done

filer.remote.gateway can see the same bucket delete more than once, for
example once per filer when several filers peer their metadata. A copy
that still finds the mount mapping calls DeleteBucket again, and S3
(NoSuchBucket), Azure (ContainerNotFound, ContainerBeingDeleted) and GCS
(404) all failed it, which held the sync offset on the delete. A
missing bucket now counts as deleted, the way deleting a missing object
already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.remote.gateway: keep the mapping table writable after mount.mapping is deleted

Deleting mount.mapping replaced the gateway's mappings with a table whose
map was nil, so the next bucket create panicked writing its mount into
it. Reset to an empty map instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.remote.gateway: count a delete of an absent object as done

filer.remote.sync already treats ErrRemoteObjectNotFound from DeleteFile
as deleted, since GCS reports a missing object that way where S3 and
Azure succeed. The gateway returned it, so a delete event seen twice, as
it is when several filers peer their metadata, held the sync offset on
GCS. Use the same deleteRemoteFile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.remote.gateway: drop cached remote clients after the new gateway tests

GetRemoteStorage caches clients by configuration, so a repeated run
(-count=2) got the first run's recorder back and saw no calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.remote.gateway: keep the default storage across mount.mapping changes

The default storage for new buckets is derived at startup and kept only
in the gateway's copy of the mappings; mount.mapping never stores it.
Replacing or deleting that file dropped it, so a later bucket create
with no -createBucketAt skipped the remote bucket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
authored and GitHub committed 2026-11-05 05:06:41 -08:00
1 parent c971252871
commit 80f32bc074
10 files changed
+235 -2

No files matched your search

@@ -884,6 +884,7 @@ type recordingRemote struct {
// metadataUpdateResult is the RemoteEntry UpdateFileMetadata reports; nil
// means the update left the object untouched
metadataUpdateResult *filer_pb.RemoteEntry
bucketCreates []string
}
func (r *recordingRemote) WriteFile(loc *remote_pb.RemoteStorageLocation, entry *filer_pb.Entry, reader io.Reader) (*filer_pb.RemoteEntry, error) {
@@ -932,6 +933,11 @@ func (r *recordingRemote) DeleteFile(loc *remote_pb.RemoteStorageLocation) error
return r.deleteErr
}
func (r *recordingRemote) CreateBucket(name string) error {
r.bucketCreates = append(r.bucketCreates, name)
return nil
}
func (r *recordingRemote) UpdateFileMetadata(loc *remote_pb.RemoteStorageLocation, oldEntry, newEntry *filer_pb.Entry) (*filer_pb.RemoteEntry, error) {
r.metadataUpdates = append(r.metadataUpdates, loc)
return r.metadataUpdateResult, nil