admin: add bucket lifecycle rule editing (#10860)

* admin: add bucket lifecycle rule editing * address greptile's comments * more small fixes * coderabbit's comments * more comment fixes * more fixes * more * maybe last * last ? * 14850 * 14851 * filer: stamp the content MD5 on every SaveInsideFiler write An entry's ETag falls back to Attributes.Md5, so conditional writers key IF_ETAG_MATCH off it. SaveInsideFiler carried the looked-up attributes forward without refreshing the hash, leaving it describing whatever the previous writer stored: a later conditional write matched the stale hash and overwrote content that had already changed. * s3api: give the bucket lifecycle constants and the write route key one definition each The extended-attribute keys, the XML size cap and the object-write ring key prefix were each spelled out in two places, so the admin dashboard's copies could drift from the gateway's. Move them to the packages both sides already import and alias them where the short local name reads better. * admin: patch the bucket entry's lifecycle keys instead of rewriting the entry The save read the bucket entry, edited its extended map and wrote the whole entry back, guarded by IF_UNMODIFIED_SINCE. Nothing that writes a bucket entry advances its mtime - not the S3 gateway's patchBucketEntry, not SetBucketOwner, not SetBucketQuota - so the guard never fired and the stale snapshot reverted whatever else had changed since the lookup. Send the PATCH_EXTENDED mutation the S3 gateway already uses for these keys: the filer re-reads and merges under the bucket path lock, so only the two lifecycle keys move. That removes the reason for the mtime snapshot, the verification retry loop and the compensating restore of the cleared day-TTL rules, which the migration now logs instead. * s3api: run the delete-lifecycle day-TTL migration through the shared helper DeleteBucketLifecycleHandler kept its own copy of the read-strip-write sequence the put handler now shares, including a missing return that let a ToText failure persist a truncated filer.conf and write a second response. It also wrote the whole file back unconditionally, reverting any concurrent edit; the shared helper writes conditionally. * admin: answer 404 when a lifecycle request names a bucket that does not exist Every SetBucketLifecycle failure came back as 500, including the lookup miss for an unknown bucket, so a client or monitor read a caller error as a server fault and retried it. * s3api: emit lifecycle XML a client would recognize Two changes to what MarshalCanonical writes, both visible through GetBucketLifecycleConfiguration, which replays the stored bytes verbatim: stamp the S3 namespace on the root, and put a size range under <And>. A <Filter> carries one predicate, so two size bounds side by side is a shape AWS does not document. Parsing still accepts either. * admin: fix the lifecycle editor's handling of stored status, deletes and empty saves Four things the editor got wrong: A stored <Status> the S3 API never validated, say 'enabled', left both radio buttons unchecked, so reading the form threw on a null querySelector result and Save did nothing. Collapse anything but an exact 'Enabled' to 'Disabled', which is what the engine already does with it. Deleting a rule re-rendered an open edit form from the snapshot taken when editing began, discarding what had been typed; every other transition folds the form in first. The Transition warning only matched a bare <Transition>, missing the form with attributes, self-closed or namespace-prefixed. Saving an emptied rule list clears the configuration through a path with no prompt, next to a Delete-all-rules button that asks. Also collapses the three divergent copies of formatBytes on this page to one. * filer: stop the day-TTL migration from deleting an operator's path rule The migration removed every rule under the bucket's path that carried a day TTL in the bucket's collection. The add path it is retiring used AddLocationConf, which merged its TTL onto whatever already sat at the prefix, so a rule can hold operator settings the lifecycle path never wrote - a disk type, WORM retention, a read-only flag, a placement pin. Deleting the whole rule to retire its TTL took those with it, leaving objects under that prefix on defaults nobody asked for. Delete only rules shaped like ones the add path created from scratch; anything else keeps its settings and loses just the TTL. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-10-24 06:50:40 +02:00 · 2026-08-21 23:42:26 -07:00
1 parent 301d83cc7a
commit df93d01c06
@@ -1001,6 +1001,133 @@ func toBucketLifecycleRule(rule *s3lifecycle.Rule) BucketLifecycleRule {
 	return out
 }

+// fromBucketLifecycleRule is the inverse of toBucketLifecycleRule, turning a
+// rule edited in the admin UI back into the engine's canonical shape.
+func fromBucketLifecycleRule(rule BucketLifecycleRule) (*s3lifecycle.Rule, error) {
+	out := &s3lifecycle.Rule{
+		ID:                              rule.ID,
+		Status:                          rule.Status,
+		Prefix:                          rule.Prefix,
+		FilterTags:                      rule.Tags,
+		FilterSizeGreaterThan:           rule.SizeGreaterThan,
+		FilterSizeLessThan:              rule.SizeLessThan,
+		ExpirationDays:                  rule.ExpirationDays,
+		ExpiredObjectDeleteMarker:       rule.ExpiredObjectDeleteMarker,
+		NoncurrentVersionExpirationDays: rule.NoncurrentVersionExpirationDays,
+		NewerNoncurrentVersions:         rule.NewerNoncurrentVersions,
+		AbortMPUDaysAfterInitiation:     rule.AbortMultipartDays,
+	}
+	if rule.ExpirationDate != "" {
+		date, err := time.Parse(time.DateOnly, rule.ExpirationDate)
+		if err != nil {
+			return nil, fmt.Errorf("invalid expiration date %q: %w", rule.ExpirationDate, err)
+		}
+		out.ExpirationDate = date
+	}
+	return out, nil
+}
+
+// ErrBucketNotFound reports that the named bucket has no filer entry, so a
+// handler can answer 404 rather than 500.
+var ErrBucketNotFound = errors.New("bucket not found")
+
+// SetBucketLifecycle replaces the lifecycle configuration stored on a
+// bucket's filer entry. An empty rule list clears the configuration
+// entirely, mirroring clearStoredBucketLifecycleConfiguration on the S3 API
+// side. Callers must validate rules before calling this (see
+// validateBucketLifecycleRules) — this only rejects what marshaling itself
+// rejects.
+func (s *AdminServer) SetBucketLifecycle(bucketName string, rules []BucketLifecycleRule) error {
+	canonicalRules := make([]*s3lifecycle.Rule, 0, len(rules))
+	for _, rule := range rules {
+		canonicalRule, err := fromBucketLifecycleRule(rule)
+		if err != nil {
+			return err
+		}
+		canonicalRules = append(canonicalRules, canonicalRule)
+	}
+
+	var lifecycleXML []byte
+	if len(canonicalRules) > 0 {
+		var err error
+		lifecycleXML, err = lifecycle_xml.MarshalCanonical(canonicalRules)
+		if err != nil {
+			return fmt.Errorf("marshal lifecycle configuration: %w", err)
+		}
+		if len(lifecycleXML) > scheduler.MaxBucketLifecycleConfigurationSize {
+			return fmt.Errorf("lifecycle configuration is %d bytes, which exceeds the %d byte limit", len(lifecycleXML), scheduler.MaxBucketLifecycleConfigurationSize)
+		}
+	}
+
+	filerConfig, err := s.getFilerConfig()
+	if err != nil {
+		return fmt.Errorf("get filer configuration: %w", err)
+	}
+	collection := getCollectionName(filerConfig.FilerGroup, bucketName)
+
+	return s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
+		// PATCH_EXTENDED is a no-op on a missing entry, so the existence
+		// check has to happen here rather than fall out of the write.
+		if _, err := filer_pb.LookupEntry(context.Background(), client, &filer_pb.LookupDirectoryEntryRequest{
+			Directory: filerConfig.BucketsPath,
+			Name:      bucketName,
+		}); err != nil {
+			if errors.Is(err, filer_pb.ErrNotFound) {
+				return fmt.Errorf("%w: %s", ErrBucketNotFound, bucketName)
+			}
+			return fmt.Errorf("look up bucket %s: %w", bucketName, err)
+		}
+
+		// Migration: clear any legacy day-TTL filer.conf entries before
+		// writing the new XML, so a failure here leaves the bucket entry
+		// untouched instead of committing the new policy alongside a stale
+		// TTL rule. Same step and ordering as
+		// PutBucketLifecycleConfigurationHandler.
+		if err := filer.ClearBucketLifecycleDayTTLs(context.Background(), client, filerConfig.BucketsPath, bucketName, collection); err != nil {
+			return fmt.Errorf("failed to clear legacy lifecycle TTLs: %w", err)
+		}
+
+		bucketPath := filerConfig.BucketsPath + "/" + bucketName
+		resp, err := client.ObjectTransaction(context.Background(), &filer_pb.ObjectTransactionRequest{
+			LockKey:   bucketPath,
+			RouteKey:  s3_constants.ObjectWriteRouteKeyPrefix + bucketPath,
+			Mutations: []*filer_pb.ObjectMutation{bucketLifecycleMutation(filerConfig.BucketsPath, bucketName, lifecycleXML)},
+		})
+		if err != nil {
+			return fmt.Errorf("failed to update bucket lifecycle: %w", err)
+		}
+		if resp.Error != "" {
+			return fmt.Errorf("failed to update bucket lifecycle: %s", resp.Error)
+		}
+		return nil
+	})
+}
+
+// bucketLifecycleMutation patches the two lifecycle keys rather than writing
+// the whole entry back: the filer re-reads and merges under the bucket path
+// lock, so a concurrent owner/quota/versioning change is preserved instead of
+// being reverted by a stale snapshot. Same mutation the S3 gateway uses for
+// these keys (see patchBucketEntry in s3api_bucket_config.go). Empty XML
+// clears the configuration, transition minimum size included.
+func bucketLifecycleMutation(bucketsPath, bucketName string, lifecycleXML []byte) *filer_pb.ObjectMutation {
+	mutation := &filer_pb.ObjectMutation{
+		Type:      filer_pb.ObjectMutation_PATCH_EXTENDED,
+		Directory: bucketsPath,
+		Name:      bucketName,
+	}
+	if len(lifecycleXML) > 0 {
+		mutation.SetExtended = map[string][]byte{
+			scheduler.BucketLifecycleConfigurationXMLKey: lifecycleXML,
+		}
+		return mutation
+	}
+	mutation.DeleteExtended = []string{
+		scheduler.BucketLifecycleConfigurationXMLKey,
+		scheduler.BucketLifecycleTransitionMinimumObjectSizeKey,
+	}
+	return mutation
+}
+
 // CreateS3Bucket creates a new S3 bucket
 func (s *AdminServer) CreateS3Bucket(bucketName string) error {
 	return s.CreateS3BucketWithQuota(bucketName, 0, false)