s3 lifecycle: share daily bucket walk and prune listings (#11797)

* s3 lifecycle: share the daily bucket walk across shards

The daily pass invoked the walker once per shard, and every shard's
walk listed the full bucket tree while keeping only its own 1/16 of
the entries — 16 full listings per pass by default (#11791).

Run now groups the shards by the walk branch they'll take (recovery
vs steady, using the same cursor data and hash checks runShard applies)
and starts one shared walk per group, so a pass lists each bucket at
most twice. WalkerFunc and WalkBuckets take a shard set instead of a
single shard. runShard keeps its own decision and waits on its group's
walk; a shard whose freshly-loaded cursor disagrees with the plan falls
back to a per-shard walk, so mid-pass cursor rewrites can't under-cover
a shard.

* s3 lifecycle: skip the .versions/ pre-pass on non-versioned buckets

walkBucketTree listed every directory twice — once looking for
.versions/ dirs, once for regular entries. On a bucket that never had
versioning enabled the first pass can only find coincidentally-named
user folders, so the extra listing is pure cost (#11791).

FilerListFunc now takes the walk view and consults
Snapshot.BucketVersioned; non-versioned buckets skip pass 1 entirely.
A *-suffixed .versions user folder on such a bucket is recursed into
as a regular directory so real objects inside still surface; a nil
view keeps the two-pass behavior.

* s3 lifecycle: prune walk directories to active rule prefixes

The daily walk listed every directory under a bucket even when no
rule prefix could reach it. Collect the active rule prefixes of each
bucket from the walk view (new Snapshot.BucketPrefixes) and skip
recursing into directories whose keys can never match; a bucket
with no active actions is skipped entirely, saving the listing.

.uploads stays in scope regardless of prefixes because AbortMPU
rules match DestKey, and .versions expansion is pruned only
conservatively (the logical key is a string-prefix of the dir key).

* s3 lifecycle: join planned walks before Run returns

planWalks starts a shared walk goroutine per group, but a shard whose
authoritative cursor load then fails exits runShard without waiting on
it. Run returned while the walk still listed or dispatched deletes
against connections the caller closes on return. Join every planned
walk after the shard waitgroup.

* s3 lifecycle: treat object-locked buckets as versioned in the walk

Object Lock requires versioning and every writer sets both extended
attributes, but IsBucketVersioned only read ExtVersioningKey. A locked
bucket missing that attribute would have its .versions dirs walked as
ordinary folders, hiding version siblings from lifecycle. Classify
ExtObjectLockEnabledKey as versioned too.

weed/s3api/s3lifecycle/DESIGN.md

@@ -19,13 +19,16 @@ dailyrun.Run(ctx, cfg):
     rsh                = engine.ReplayContentHash(snap)
     maxTTL             = engine.MaxEffectiveTTL(snap)
 
+    cursors            = load every shard's persisted cursor
+    walks              = planWalks(cursors) → at most 2 shared walks (recovery, steady)
+
     if rsh != [32]byte{}:                                           // replay-eligible rules present
         globalStartTsNs = min over cfg.Shards of (persisted cursor or runNow - maxTTL)
         reader          = subscribeMeta(ShardPredicate ∈ cfg.Shards, StartTsNs = globalStartTsNs)
         fanOut(reader.Events → shardEvents[shardID]) until ev.TsNs > runNow
 
     spawn one goroutine per shard:
-        runShard(ctx, cfg, snap, runNow, shardID, shardEvents[shardID])
+        runShard(ctx, cfg, snap, runNow, shardID, shardEvents[shardID], walks)
 
     wait all
     teardown reader + fan-out
@@ -37,7 +40,7 @@ runShard(ctx, cfg, snap, runNow, shardID, events):
 
     if rsh == [32]byte{}:                                           // pure walker bucket
         if walkerDue and Walker:
-            cfg.Walker(walkView)                                    // RulesForShard.walk
+            walkForShard(walkView)                                // shared walk, per-shard fallback
             lastWalkedNs = runNow.UnixNano()
         save cursor (TsNs=0, rsh, promoted, lastWalkedNs)
         return
@@ -46,7 +49,7 @@ runShard(ctx, cfg, snap, runNow, shardID, events):
     mustWalkColdStart  = !found
 
     if mustWalkRecovery or mustWalkColdStart:
-        cfg.Walker(engine.RecoveryView(snap))                       // every rule, force-active
+        walkForShard(RecoveryView(snap))                          // every rule, force-active
         walkedThisPass = true
         lastWalkedNs   = runNow.UnixNano()
         if mustWalkRecovery:
@@ -54,7 +57,7 @@ runShard(ctx, cfg, snap, runNow, shardID, events):
             return
 
     if Walker and !walkedThisPass and walkerDue:
-        cfg.Walker(walkView)                                        // steady-state walker
+        walkForShard(walkView)                                    // steady-state walker
         lastWalkedNs = runNow.UnixNano()
 
     startTsNs = !found ? runNow - maxTTL : persisted.TsNs           // steady state honors cursor
@@ -62,7 +65,7 @@ runShard(ctx, cfg, snap, runNow, shardID, events):
     save cursor (TsNs = lastOK, rsh, promoted, lastWalkedNs)
 ```
 
-Walker invocations — three call sites, distinct conditions:
+Walker invocations — three call sites, distinct conditions. `cfg.Walker` takes a shard set, not a single shard: `planWalks` predicts each shard's branch from the pre-loaded cursors and runs one shared walk per group, so a pass lists each bucket at most twice (recovery group + steady group) instead of once per shard. `runShard` waits on its group's walk; a shard whose own freshly-loaded cursor disagrees with the plan falls back to a per-shard walk, so the plan can't under-cover a shard after a cursor is rewritten mid-pass.
 
 | Branch | View | Trigger | Throttle |
 |---|---|---|---|
@@ -230,9 +233,10 @@ The lifecycle evaluator uses `ev.TsNs` for current-version rules and `entry.Nonc
 | `evaluate.go`, `due_at.go`, `rule_hash.go`, `tags.go` | Engine-side rule evaluation |
 | `reader/` | Meta-log subscribe; one subscription per dailyrun.Run pass |
 | `router/router.go` | Per-event rule evaluation |
-| `bootstrap/walker.go` | Bucket walker with `RunForShard(view, shardID)` filter |
+| `bootstrap/walker.go` | Bucket walker; shard-set entry filter |
 | `dispatcher/filer_persister.go` | Filer-backed cursor I/O |
 | `dailyrun/run.go` | Main pass orchestrator: subscription, fan-out, per-shard `runShard` |
+| `dailyrun/walk_plan.go` | Groups shards into shared recovery/steady walks (one listing per group) |
 | `dailyrun/cursor.go` | Cursor type + filer JSON serializer |
 | `dailyrun/walker_dispatcher.go` | Adapter from walker to `LifecycleDelete` RPC |
 
@@ -265,6 +269,6 @@ Worker config:
 Tracked as optimizations rather than blockers:
 
 1. **Long-lived subscription across passes.** Today the subscription is rebuilt per `Run()`. Keeping it alive across passes would eliminate the 7s ctx-timeout per pass and the start/teardown overhead. Requires per-shard pending heap (events whose `DueTime > runNow` would be parked in-memory instead of replayed) and a hot-swappable snapshot for mid-pass config changes. Multi-day refactor; current model works.
-2. **Bucket-coordinated walker.** Phase 4 has each shard walk the full bucket and filter by `ShardID(bucket, key)` — simple but 16× the listing cost. A per-bucket coordinator (the worker owning shard 0 for that bucket lists once, routes matches to other shards) would cut listing cost. Worth doing if listing becomes the bottleneck for very large buckets.
+2. ~~Bucket-coordinated walker~~. Done: `planWalks` shares one listing across the recovery group and one across the steady group (issue #11791). Remaining headroom: directories could be pruned to rule prefixes, and per-shard views could partition work rather than fanning every entry to its shard's dispatcher.
 3. **Per-bucket dispatch lag metric.** Currently only per-shard lag is exposed. Per-bucket would require a per-bucket cursor or a derived metric from `s3_lifecycle_dispatch_total{bucket,kind,outcome}`. Punted on cardinality concerns; revisit when an operator asks for it.
 4. **Meta-log retention plumbing.** If the filer adds GC for `/topics/.system/log`, the `PromotedHash` partition flip needs to consume the filer's actual retention horizon (currently dormant because retention is effectively infinite).