Files
seaweedfs/weed/s3api/s3lifecycle/evaluate.go
T
Chris Lu 89aab30821 feat(s3/lifecycle): wire AbortIncompleteMultipartUpload (Phase 5a) (#9368)
* feat(s3/lifecycle/router): emit ABORT_MPU events for .uploads/<id> init dirs

Detect a meta-log event at exactly .uploads/<upload_id> (a directory)
and build the ObjectInfo from its destination key (entry.Extended[key])
so a rule with Filter.Prefix=foo/ matches an MPU uploading to foo/bar.
Sub-events under .uploads/<id>/<part> ride a different mtime and would
over-fire the ABORT_MPU schedule, so they're rejected explicitly.

m.ObjectKey stays as ev.Key (.uploads/<upload_id>) — the dispatcher
needs the upload directory path, not the destination key, to actually
remove the in-flight upload.

* feat(s3api): wire LifecycleDelete ABORT_MPU to remove the upload dir

Replaces the retryLater stub. Validates the .uploads/<upload_id> shape
of req.ObjectPath (so a malformed event can't escalate to a wider rm),
then deletes the upload directory under <bucket>/.uploads/<id>. Maps
NotFound to NOOP_RESOLVED, transport errors to RETRY_LATER, success to
DONE.

* refactor(s3api): drop redundant exists check before lifecycle ABORT_MPU rm

s3a.rm already does a NotFound-returning lookup, so the pre-check just
adds a round-trip. Map filer_pb.ErrNotFound to NOOP_RESOLVED on rm,
keep transport errors as RETRY_LATER.

* refactor(s3/lifecycle/router): use s3_constants for MPU paths + Extended key

Drop the hardcoded ".uploads/" and "key" string literals; the symbols
already exist as s3_constants.MultipartUploadsFolder and
ExtMultipartObjectKey, and the server side reaches them through the
same constants. Keeping the test helpers tied to those names also makes
the negative-result tests meaningful — they'd otherwise still pass if
the lookup constant drifted.

* fix(s3api): close lifecycle ABORT_MPU traversal + NOT_FOUND gaps

Two issues with the recent ABORT_MPU plumbing:

- "." and ".." passed the no-slash check but resolve to the bucket root
  via util.JoinPath, so .uploads/.. could rm the wrong directory.
- filer.DeleteEntry suppresses ErrNotFound and returns success, so the
  rm path can't distinguish missing from deleted; the previous version
  reported DONE for an already-aborted upload instead of NOOP_RESOLVED.

Reject the two reserved names explicitly and restore the existence
pre-check so the outcome map stays correct. Add a table-test covering
the rejected paths.

* fix(s3/lifecycle/bootstrap): walk MPU init dirs by destination key

A real MPU init record is a directory under .uploads/<id> created by
mkdir; the bootstrap walker was skipping every directory entry, so an
MPU that existed before the meta-log subscription was never aborted.
Even with the skip relaxed, MatchPath used the .uploads/<id> path, so
a rule with Filter.Prefix=logs/ would never fire on an MPU uploading
to logs/foo.txt.

Add Entry.DestKey, let IsMPUInit directories through, and use DestKey
for both MatchPath and ObjectInfo.Key. A bare init directory with no
DestKey means metadata hasn't landed yet — skip rather than guess.

* fix(s3/lifecycle): gate (kind, info) shape so MPU init only fires ABORT_MPU

An MPU init record carries IsMPUInit=true and IsLatest=false. Without
gating, the router and bootstrap walker matched it against every active
ActionKey for the bucket, so NONCURRENT_DAYS / NEWER_NONCURRENT fired
(IsLatest=false reads as a noncurrent version). The dispatcher would
then BLOCK on empty version_id and freeze the cursor.

Add a shape gate at both call sites:
  - IsMPUInit + non-ABORT_MPU kind → continue
  - regular object + ABORT_MPU kind → continue

Plus a defense-in-depth check at the top of EvaluateAction so future
callers can't reintroduce the bug. Tests cover all three layers.

* test(s3/lifecycle): tighten dual-action coverage at the call sites

- Walk multi-action: replace the kinds-as-set check with an exact-shape
  DeepEqual on (path, kind) tuples. The set check would have missed an
  MPU init wrongly firing NONCURRENT_DAYS — exactly the regression the
  (kind, info) gate fixes.
- Router: add a converse case for the dual ExpirationDays +
  AbortIncompleteMultipartUpload rule. A regular current-version object
  must fire only EXPIRATION_DAYS; without the gate the dispatcher would
  also receive ABORT_MPU and rm the object via the MPU code path.
2026-05-08 12:12:42 -07:00

118 lines
3.5 KiB
Go

package s3lifecycle
import (
"strings"
"time"
)
// EvaluateAction returns whether the (rule, kind) action fires for info at
// now. A non-current delete marker is just another non-current version under
// NONCURRENT_DAYS / NEWER_NONCURRENT; only a current sole-survivor marker
// routes to EXPIRED_DELETE_MARKER.
func EvaluateAction(rule *Rule, kind ActionKind, info *ObjectInfo, now time.Time) EvalResult {
none := EvalResult{Action: ActionNone}
if rule == nil || info == nil || rule.Status != StatusEnabled {
return none
}
if !filterMatches(rule, info) {
return none
}
// MPU init records carry IsLatest=false (they are not yet versions);
// without this guard NoncurrentDays / NewerNoncurrent fire on them
// and the dispatcher BLOCKs because version_id is empty, freezing
// the cursor. Only ABORT_MPU is meaningful for an in-flight upload.
if info.IsMPUInit && kind != ActionKindAbortMPU {
return none
}
switch kind {
case ActionKindAbortMPU:
if !info.IsMPUInit || rule.AbortMPUDaysAfterInitiation <= 0 {
return none
}
due := info.ModTime.AddDate(0, 0, rule.AbortMPUDaysAfterInitiation)
if now.Before(due) {
return none
}
return EvalResult{Action: ActionAbortMultipartUpload, RuleID: rule.ID}
case ActionKindExpiredDeleteMarker:
if !info.IsLatest || !info.IsDeleteMarker || !rule.ExpiredObjectDeleteMarker {
return none
}
if info.NumVersions != 1 {
return none
}
return EvalResult{Action: ActionExpireDeleteMarker, RuleID: rule.ID}
case ActionKindExpirationDays:
if !info.IsLatest || info.IsDeleteMarker || rule.ExpirationDays <= 0 {
return none
}
due := info.ModTime.AddDate(0, 0, rule.ExpirationDays)
if now.Before(due) {
return none
}
return EvalResult{Action: ActionDeleteObject, RuleID: rule.ID}
case ActionKindExpirationDate:
if !info.IsLatest || info.IsDeleteMarker || rule.ExpirationDate.IsZero() {
return none
}
if now.Before(rule.ExpirationDate) {
return none
}
return EvalResult{Action: ActionDeleteObject, RuleID: rule.ID}
case ActionKindNoncurrentDays:
if info.IsLatest || rule.NoncurrentVersionExpirationDays <= 0 {
return none
}
base := info.SuccessorModTime
if base.IsZero() {
base = info.ModTime
}
due := base.AddDate(0, 0, rule.NoncurrentVersionExpirationDays)
if now.Before(due) {
return none
}
// nil index = can't evaluate retention; safety-scan revisits.
if rule.NewerNoncurrentVersions > 0 {
if info.NoncurrentIndex == nil || *info.NoncurrentIndex < rule.NewerNoncurrentVersions {
return none
}
}
return EvalResult{Action: ActionDeleteVersion, RuleID: rule.ID}
case ActionKindNewerNoncurrent:
// Count-only; when paired with NoncurrentDays the rule expands to
// NONCURRENT_DAYS instead (RuleActionKinds).
if info.IsLatest || rule.NoncurrentVersionExpirationDays > 0 || rule.NewerNoncurrentVersions <= 0 {
return none
}
if info.NoncurrentIndex == nil || *info.NoncurrentIndex < rule.NewerNoncurrentVersions {
return none
}
return EvalResult{Action: ActionDeleteVersion, RuleID: rule.ID}
}
return none
}
func filterMatches(rule *Rule, info *ObjectInfo) bool {
if rule.Prefix != "" && !strings.HasPrefix(info.Key, rule.Prefix) {
return false
}
if rule.FilterSizeGreaterThan > 0 && info.Size <= rule.FilterSizeGreaterThan {
return false
}
if rule.FilterSizeLessThan > 0 && info.Size >= rule.FilterSizeLessThan {
return false
}
for k, v := range rule.FilterTags {
if got, ok := info.Tags[k]; !ok || got != v {
return false
}
}
return true
}