shell: make every -collection flag parse the same way (#11794)

* shell: make every -collection flag parse the same way

The filter vocabulary (comma-separated names, wildcards, regexes,
"_default" for the empty-named collection) already existed in
wildcard.CompileCollectionMatcher, but several commands still did their
own exact-match comparison or sent the raw flag value to servers.

fs.mergeVolumes used a literal string compare, so -collection=_default
silently produced an empty merge plan; it now uses the shared matcher
while keeping "*" as the match-all default. volume.fsck and volume.mark
filter through the matcher too, and volume.vacuum resolves the filter
client-side into per-collection and per-volume requests, since no
Collection value scopes the VacuumVolume RPC to the unnamed collection
alone.

Commands that put one concrete collection name on the wire normalize
"_default" to "" (volume.grow, fs.configure, mq.topic.compact,
collection.delete, ec.rebuild, ec.balance), and the tier -volumeId
commands resolve the volume's actual collection from topology first so
patterns and named-collection requests pass the volume server's
collection check.

* shell: spell the default-collection keyword via CollectionDefault

* shell: address review findings on collection filter handling

volume.fsck keeps a set of every volume the master knows before any
filtering, so a filer entry whose volume was excluded by -collection or
-volumeId is skipped instead of reported "volume not found" and deleted
by -reallyDeleteFilerEntries. The collection filter also no longer
scopes the filer walk to /buckets/<collection>: fs.configure can bind a
collection to any path prefix, and a narrower walk could mark live
chunks as orphans under -reallyDeleteFromVolume.

ec.rebuild and ec.balance resolve -collection through the new
wildcard.ExpandCollectionFilter, which passes literal names through even
when the master does not list them — ec.rebuild must be able to recover
a collection whose shards all lost their index and never registered.

fs.configure and mq.topic.compact now reject _default outright: an empty
Collection means "unset" in both the path-rule merge and the filer's
storage-option fallback chain, so the keyword would have been silently
ignored.

The unnamed-collection vacuum sweep keeps the master's low-disk-space
exception: a volume whose every replica is read-only is probed with
VacuumVolumeCheck and stays eligible when a replica reports
DiskSpaceLow, instead of being dropped outright.
This commit is contained in:
Chris Lu authored and GitHub committed 2026-10-16 21:07:51 -07:00
1 parent 268424a5e1
commit b0ab4c4e1e
23 files changed
+593 -71

No files matched your search

+29 -4
View File
@@ -2,6 +2,7 @@ package shell
import (
"context"
"fmt"
"strings"
"time"
@@ -142,13 +143,37 @@ func EcBalance(commandEnv *CommandEnv, collections []string, dc string, ecReplic
return ec.EcBalance(commandEnv.ecEnv(), collections, dc, ecReplicaPlacement, diskType, maxParallelization, ioBytePerSecond, applyBalancing, excludeNodes, volumeIds, nil)
}
// compileCollectionPattern compiles the -collection value shared by the ec and
// tier commands: a comma-separated list of collection names, wildcards, and
// regex patterns. An empty value matches the empty-named collection only, the
// same as CollectionDefault.
// compileCollectionPattern compiles the -collection value shared by commands
// where an empty value means the collection with no name: a comma-separated
// list of collection names, wildcards, and regex patterns, or CollectionDefault
// for the empty-named collection.
func compileCollectionPattern(pattern string) (*wildcard.CollectionMatcher, error) {
if strings.TrimSpace(pattern) == "" {
pattern = CollectionDefault
}
return wildcard.CompileCollectionMatcher(pattern)
}
// resolveEcCollections expands the ec commands' -collection value into
// collection names. An empty value means the empty-named collection, the same
// as CollectionDefault. Literal names pass through even when the master does
// not list them — the listing only contains collections with registered
// volumes, and ec.rebuild exists to bring back a collection whose volumes all
// lost their index. Wildcard and regex entries resolve against the listing.
func resolveEcCollections(commandEnv *CommandEnv, filter string) (collections []string, err error) {
if strings.TrimSpace(filter) == "" {
filter = CollectionDefault
}
names, err := ListCollectionNames(commandEnv, false, true)
if err != nil {
return nil, err
}
collections, err = wildcard.ExpandCollectionFilter(filter, names)
if err != nil {
return nil, fmt.Errorf("invalid collection pattern %q: %w", filter, err)
}
if len(collections) == 0 {
return nil, fmt.Errorf("no collections match %q", filter)
}
return collections, nil
}