admin: add bucket lifecycle rule editing (#10860)

* admin: add bucket lifecycle rule editing

* address greptile's comments

* more small fixes

* coderabbit's comments

* more comment fixes

* more fixes

* more

* maybe last

* last ?

* 14850

* 14851

* filer: stamp the content MD5 on every SaveInsideFiler write

An entry's ETag falls back to Attributes.Md5, so conditional writers key
IF_ETAG_MATCH off it. SaveInsideFiler carried the looked-up attributes
forward without refreshing the hash, leaving it describing whatever the
previous writer stored: a later conditional write matched the stale hash
and overwrote content that had already changed.

* s3api: give the bucket lifecycle constants and the write route key one definition each

The extended-attribute keys, the XML size cap and the object-write ring key
prefix were each spelled out in two places, so the admin dashboard's copies
could drift from the gateway's. Move them to the packages both sides already
import and alias them where the short local name reads better.

* admin: patch the bucket entry's lifecycle keys instead of rewriting the entry

The save read the bucket entry, edited its extended map and wrote the whole
entry back, guarded by IF_UNMODIFIED_SINCE. Nothing that writes a bucket
entry advances its mtime - not the S3 gateway's patchBucketEntry, not
SetBucketOwner, not SetBucketQuota - so the guard never fired and the stale
snapshot reverted whatever else had changed since the lookup.

Send the PATCH_EXTENDED mutation the S3 gateway already uses for these keys:
the filer re-reads and merges under the bucket path lock, so only the two
lifecycle keys move. That removes the reason for the mtime snapshot, the
verification retry loop and the compensating restore of the cleared day-TTL
rules, which the migration now logs instead.

* s3api: run the delete-lifecycle day-TTL migration through the shared helper

DeleteBucketLifecycleHandler kept its own copy of the read-strip-write
sequence the put handler now shares, including a missing return that let a
ToText failure persist a truncated filer.conf and write a second response.
It also wrote the whole file back unconditionally, reverting any concurrent
edit; the shared helper writes conditionally.

* admin: answer 404 when a lifecycle request names a bucket that does not exist

Every SetBucketLifecycle failure came back as 500, including the lookup miss
for an unknown bucket, so a client or monitor read a caller error as a server
fault and retried it.

* s3api: emit lifecycle XML a client would recognize

Two changes to what MarshalCanonical writes, both visible through
GetBucketLifecycleConfiguration, which replays the stored bytes verbatim:
stamp the S3 namespace on the root, and put a size range under <And>. A
<Filter> carries one predicate, so two size bounds side by side is a shape
AWS does not document. Parsing still accepts either.

* admin: fix the lifecycle editor's handling of stored status, deletes and empty saves

Four things the editor got wrong:

A stored <Status> the S3 API never validated, say 'enabled', left both radio
buttons unchecked, so reading the form threw on a null querySelector result
and Save did nothing. Collapse anything but an exact 'Enabled' to 'Disabled',
which is what the engine already does with it.

Deleting a rule re-rendered an open edit form from the snapshot taken when
editing began, discarding what had been typed; every other transition folds
the form in first.

The Transition warning only matched a bare <Transition>, missing the form
with attributes, self-closed or namespace-prefixed.

Saving an emptied rule list clears the configuration through a path with no
prompt, next to a Delete-all-rules button that asks.

Also collapses the three divergent copies of formatBytes on this page to one.

* filer: stop the day-TTL migration from deleting an operator's path rule

The migration removed every rule under the bucket's path that carried a day
TTL in the bucket's collection. The add path it is retiring used
AddLocationConf, which merged its TTL onto whatever already sat at the
prefix, so a rule can hold operator settings the lifecycle path never wrote -
a disk type, WORM retention, a read-only flag, a placement pin. Deleting the
whole rule to retire its TTL took those with it, leaving objects under that
prefix on defaults nobody asked for.

Delete only rules shaped like ones the add path created from scratch;
anything else keeps its settings and loses just the TTL.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
This commit is contained in:
Mathieu Arnold
2026-08-21 23:42:26 -07:00
committed by GitHub
co-authored by Chris Lu Chris Lu
parent 301d83cc7a
commit df93d01c06
17 changed files with 2313 additions and 185 deletions
+127
View File
@@ -1001,6 +1001,133 @@ func toBucketLifecycleRule(rule *s3lifecycle.Rule) BucketLifecycleRule {
return out
}
// fromBucketLifecycleRule is the inverse of toBucketLifecycleRule, turning a
// rule edited in the admin UI back into the engine's canonical shape.
func fromBucketLifecycleRule(rule BucketLifecycleRule) (*s3lifecycle.Rule, error) {
out := &s3lifecycle.Rule{
ID: rule.ID,
Status: rule.Status,
Prefix: rule.Prefix,
FilterTags: rule.Tags,
FilterSizeGreaterThan: rule.SizeGreaterThan,
FilterSizeLessThan: rule.SizeLessThan,
ExpirationDays: rule.ExpirationDays,
ExpiredObjectDeleteMarker: rule.ExpiredObjectDeleteMarker,
NoncurrentVersionExpirationDays: rule.NoncurrentVersionExpirationDays,
NewerNoncurrentVersions: rule.NewerNoncurrentVersions,
AbortMPUDaysAfterInitiation: rule.AbortMultipartDays,
}
if rule.ExpirationDate != "" {
date, err := time.Parse(time.DateOnly, rule.ExpirationDate)
if err != nil {
return nil, fmt.Errorf("invalid expiration date %q: %w", rule.ExpirationDate, err)
}
out.ExpirationDate = date
}
return out, nil
}
// ErrBucketNotFound reports that the named bucket has no filer entry, so a
// handler can answer 404 rather than 500.
var ErrBucketNotFound = errors.New("bucket not found")
// SetBucketLifecycle replaces the lifecycle configuration stored on a
// bucket's filer entry. An empty rule list clears the configuration
// entirely, mirroring clearStoredBucketLifecycleConfiguration on the S3 API
// side. Callers must validate rules before calling this (see
// validateBucketLifecycleRules) — this only rejects what marshaling itself
// rejects.
func (s *AdminServer) SetBucketLifecycle(bucketName string, rules []BucketLifecycleRule) error {
canonicalRules := make([]*s3lifecycle.Rule, 0, len(rules))
for _, rule := range rules {
canonicalRule, err := fromBucketLifecycleRule(rule)
if err != nil {
return err
}
canonicalRules = append(canonicalRules, canonicalRule)
}
var lifecycleXML []byte
if len(canonicalRules) > 0 {
var err error
lifecycleXML, err = lifecycle_xml.MarshalCanonical(canonicalRules)
if err != nil {
return fmt.Errorf("marshal lifecycle configuration: %w", err)
}
if len(lifecycleXML) > scheduler.MaxBucketLifecycleConfigurationSize {
return fmt.Errorf("lifecycle configuration is %d bytes, which exceeds the %d byte limit", len(lifecycleXML), scheduler.MaxBucketLifecycleConfigurationSize)
}
}
filerConfig, err := s.getFilerConfig()
if err != nil {
return fmt.Errorf("get filer configuration: %w", err)
}
collection := getCollectionName(filerConfig.FilerGroup, bucketName)
return s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
// PATCH_EXTENDED is a no-op on a missing entry, so the existence
// check has to happen here rather than fall out of the write.
if _, err := filer_pb.LookupEntry(context.Background(), client, &filer_pb.LookupDirectoryEntryRequest{
Directory: filerConfig.BucketsPath,
Name: bucketName,
}); err != nil {
if errors.Is(err, filer_pb.ErrNotFound) {
return fmt.Errorf("%w: %s", ErrBucketNotFound, bucketName)
}
return fmt.Errorf("look up bucket %s: %w", bucketName, err)
}
// Migration: clear any legacy day-TTL filer.conf entries before
// writing the new XML, so a failure here leaves the bucket entry
// untouched instead of committing the new policy alongside a stale
// TTL rule. Same step and ordering as
// PutBucketLifecycleConfigurationHandler.
if err := filer.ClearBucketLifecycleDayTTLs(context.Background(), client, filerConfig.BucketsPath, bucketName, collection); err != nil {
return fmt.Errorf("failed to clear legacy lifecycle TTLs: %w", err)
}
bucketPath := filerConfig.BucketsPath + "/" + bucketName
resp, err := client.ObjectTransaction(context.Background(), &filer_pb.ObjectTransactionRequest{
LockKey: bucketPath,
RouteKey: s3_constants.ObjectWriteRouteKeyPrefix + bucketPath,
Mutations: []*filer_pb.ObjectMutation{bucketLifecycleMutation(filerConfig.BucketsPath, bucketName, lifecycleXML)},
})
if err != nil {
return fmt.Errorf("failed to update bucket lifecycle: %w", err)
}
if resp.Error != "" {
return fmt.Errorf("failed to update bucket lifecycle: %s", resp.Error)
}
return nil
})
}
// bucketLifecycleMutation patches the two lifecycle keys rather than writing
// the whole entry back: the filer re-reads and merges under the bucket path
// lock, so a concurrent owner/quota/versioning change is preserved instead of
// being reverted by a stale snapshot. Same mutation the S3 gateway uses for
// these keys (see patchBucketEntry in s3api_bucket_config.go). Empty XML
// clears the configuration, transition minimum size included.
func bucketLifecycleMutation(bucketsPath, bucketName string, lifecycleXML []byte) *filer_pb.ObjectMutation {
mutation := &filer_pb.ObjectMutation{
Type: filer_pb.ObjectMutation_PATCH_EXTENDED,
Directory: bucketsPath,
Name: bucketName,
}
if len(lifecycleXML) > 0 {
mutation.SetExtended = map[string][]byte{
scheduler.BucketLifecycleConfigurationXMLKey: lifecycleXML,
}
return mutation
}
mutation.DeleteExtended = []string{
scheduler.BucketLifecycleConfigurationXMLKey,
scheduler.BucketLifecycleTransitionMinimumObjectSizeKey,
}
return mutation
}
// CreateS3Bucket creates a new S3 bucket
func (s *AdminServer) CreateS3Bucket(bucketName string) error {
return s.CreateS3BucketWithQuota(bucketName, 0, false)
+350
View File
@@ -0,0 +1,350 @@
package dash
import (
"errors"
"fmt"
"net/http"
"strings"
"testing"
"time"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/scheduler"
)
func minimalLifecycleRule() BucketLifecycleRule {
return BucketLifecycleRule{
ID: "rule-1",
Status: s3lifecycle.StatusEnabled,
ExpirationDays: 30,
}
}
func TestValidateBucketLifecycleRules_Valid(t *testing.T) {
if err := validateBucketLifecycleRules([]BucketLifecycleRule{minimalLifecycleRule()}); err != nil {
t.Fatalf("expected valid rule set to pass, got: %v", err)
}
}
func TestValidateBucketLifecycleRules_Empty(t *testing.T) {
if err := validateBucketLifecycleRules(nil); err != nil {
t.Fatalf("expected empty rule set to pass, got: %v", err)
}
}
func TestValidateBucketLifecycleRules_TooManyRules(t *testing.T) {
rules := make([]BucketLifecycleRule, MaxBucketLifecycleRules+1)
for i := range rules {
rules[i] = BucketLifecycleRule{Status: s3lifecycle.StatusEnabled, ExpirationDays: 30}
}
if err := validateBucketLifecycleRules(rules); err == nil {
t.Fatal("expected error for exceeding the rule count cap")
}
}
func TestValidateBucketLifecycleRules_DuplicateIDs(t *testing.T) {
rule := minimalLifecycleRule()
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule, rule}); err == nil {
t.Fatal("expected error for duplicate rule IDs")
}
}
func TestValidateBucketLifecycleRules_IDTooLong(t *testing.T) {
rule := minimalLifecycleRule()
longID := ""
for i := 0; i < MaxBucketLifecycleRuleIDLength+1; i++ {
longID += "a"
}
rule.ID = longID
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for an ID longer than the cap")
}
}
func TestValidateBucketLifecycleRules_InvalidStatus(t *testing.T) {
rule := minimalLifecycleRule()
rule.Status = "sort-of-enabled"
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for an invalid status")
}
}
func TestValidateBucketLifecycleRules_ExpirationDaysAndDateMutuallyExclusive(t *testing.T) {
rule := minimalLifecycleRule()
rule.ExpirationDate = "2030-01-01"
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error when both expiration_days and expiration_date are set")
}
}
func TestValidateBucketLifecycleRules_DeleteMarkerWithExpirationDaysRejected(t *testing.T) {
// AWS forbids combining ExpiredObjectDeleteMarker with Days/Date in the
// same Expiration element; ruleFromCanonical would otherwise emit both
// on the same <Expiration>.
rule := minimalLifecycleRule()
rule.ExpiredObjectDeleteMarker = true
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error when expired_object_delete_marker is combined with expiration_days")
}
}
func TestValidateBucketLifecycleRules_DeleteMarkerWithExpirationDateRejected(t *testing.T) {
rule := BucketLifecycleRule{
Status: s3lifecycle.StatusEnabled,
ExpirationDate: "2030-01-01",
ExpiredObjectDeleteMarker: true,
}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error when expired_object_delete_marker is combined with expiration_date")
}
}
func TestValidateBucketLifecycleRules_DeleteMarkerAlone(t *testing.T) {
rule := BucketLifecycleRule{
Status: s3lifecycle.StatusEnabled,
ExpiredObjectDeleteMarker: true,
}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err != nil {
t.Fatalf("expected standalone expired_object_delete_marker to be accepted, got: %v", err)
}
}
func TestValidateBucketLifecycleRules_InvalidExpirationDate(t *testing.T) {
rule := BucketLifecycleRule{Status: s3lifecycle.StatusEnabled, ExpirationDate: "01/01/2030"}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for a malformed expiration_date")
}
}
func TestValidateBucketLifecycleRules_NegativeExpirationDays(t *testing.T) {
rule := minimalLifecycleRule()
rule.ExpirationDays = -1
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for negative expiration_days")
}
}
// TestValidateBucketLifecycleRules_StandaloneNewerNoncurrentVersions guards
// against re-adding a NoncurrentVersionExpirationDays requirement:
// s3lifecycle.RuleActionKinds recognizes a standalone NewerNoncurrentVersions
// (no days) as ActionKindNewerNoncurrent, a valid count-only retention rule
// the S3 API already accepts and stores.
func TestValidateBucketLifecycleRules_StandaloneNewerNoncurrentVersions(t *testing.T) {
rule := BucketLifecycleRule{
Status: s3lifecycle.StatusEnabled,
NewerNoncurrentVersions: 2,
}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err != nil {
t.Fatalf("expected standalone newer_noncurrent_versions to be accepted, got: %v", err)
}
}
func TestValidateBucketLifecycleRules_NegativeAbortMultipartDays(t *testing.T) {
rule := minimalLifecycleRule()
rule.AbortMultipartDays = -5
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for negative abort_multipart_days")
}
}
func TestValidateBucketLifecycleRules_SizeGreaterThanNotLessThanSizeLessThan(t *testing.T) {
rule := minimalLifecycleRule()
rule.SizeGreaterThan = 2048
rule.SizeLessThan = 1024
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error when size_greater_than >= size_less_than")
}
}
func TestValidateBucketLifecycleRules_NegativeSizeBounds(t *testing.T) {
rule := minimalLifecycleRule()
rule.SizeGreaterThan = -1
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for a negative size_greater_than")
}
}
func TestValidateBucketLifecycleRules_EmptyTagKey(t *testing.T) {
rule := minimalLifecycleRule()
rule.Tags = map[string]string{"": "value"}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for an empty tag key")
}
}
func TestValidateBucketLifecycleRules_EmptyTagValue(t *testing.T) {
rule := minimalLifecycleRule()
rule.Tags = map[string]string{"env": " "}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for a blank tag value")
}
}
func TestValidateBucketLifecycleRules_NoAction(t *testing.T) {
rule := BucketLifecycleRule{Status: s3lifecycle.StatusEnabled, Prefix: "logs/"}
if err := validateBucketLifecycleRules([]BucketLifecycleRule{rule}); err == nil {
t.Fatal("expected error for a rule with no action")
}
}
func TestFromBucketLifecycleRule_MapsAllFields(t *testing.T) {
rule := BucketLifecycleRule{
ID: "rule-1",
Status: s3lifecycle.StatusEnabled,
Prefix: "logs/",
Tags: map[string]string{"env": "dev"},
SizeGreaterThan: 1024,
SizeLessThan: 2048,
ExpirationDate: "2030-01-15",
ExpiredObjectDeleteMarker: true,
NoncurrentVersionExpirationDays: 30,
NewerNoncurrentVersions: 2,
AbortMultipartDays: 7,
}
out, err := fromBucketLifecycleRule(rule)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if out.ID != rule.ID {
t.Errorf("ID = %q, want %q", out.ID, rule.ID)
}
if out.Status != rule.Status {
t.Errorf("Status = %q, want %q", out.Status, rule.Status)
}
if out.Prefix != rule.Prefix {
t.Errorf("Prefix = %q, want %q", out.Prefix, rule.Prefix)
}
if len(out.FilterTags) != 1 || out.FilterTags["env"] != "dev" {
t.Errorf("FilterTags = %v, want {env: dev}", out.FilterTags)
}
if out.FilterSizeGreaterThan != rule.SizeGreaterThan {
t.Errorf("FilterSizeGreaterThan = %d, want %d", out.FilterSizeGreaterThan, rule.SizeGreaterThan)
}
if out.FilterSizeLessThan != rule.SizeLessThan {
t.Errorf("FilterSizeLessThan = %d, want %d", out.FilterSizeLessThan, rule.SizeLessThan)
}
wantDate, _ := time.Parse(time.DateOnly, rule.ExpirationDate)
if !out.ExpirationDate.Equal(wantDate) {
t.Errorf("ExpirationDate = %v, want %v", out.ExpirationDate, wantDate)
}
if !out.ExpiredObjectDeleteMarker {
t.Error("expected ExpiredObjectDeleteMarker=true")
}
if out.NoncurrentVersionExpirationDays != rule.NoncurrentVersionExpirationDays {
t.Errorf("NoncurrentVersionExpirationDays = %d, want %d", out.NoncurrentVersionExpirationDays, rule.NoncurrentVersionExpirationDays)
}
if out.NewerNoncurrentVersions != rule.NewerNoncurrentVersions {
t.Errorf("NewerNoncurrentVersions = %d, want %d", out.NewerNoncurrentVersions, rule.NewerNoncurrentVersions)
}
if out.AbortMPUDaysAfterInitiation != rule.AbortMultipartDays {
t.Errorf("AbortMPUDaysAfterInitiation = %d, want %d", out.AbortMPUDaysAfterInitiation, rule.AbortMultipartDays)
}
}
func TestFromBucketLifecycleRule_EmptyExpirationDate(t *testing.T) {
rule := minimalLifecycleRule()
rule.ExpirationDate = ""
out, err := fromBucketLifecycleRule(rule)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !out.ExpirationDate.IsZero() {
t.Errorf("expected zero ExpirationDate, got %v", out.ExpirationDate)
}
}
func TestFromBucketLifecycleRule_InvalidExpirationDate(t *testing.T) {
rule := minimalLifecycleRule()
rule.ExpirationDate = "not-a-date"
if _, err := fromBucketLifecycleRule(rule); err == nil {
t.Fatal("expected error for a malformed expiration date")
}
}
func TestBucketLifecycleMutation_SetsXML(t *testing.T) {
m := bucketLifecycleMutation("/buckets", "mybucket", []byte("<LifecycleConfiguration/>"))
if m.Type != filer_pb.ObjectMutation_PATCH_EXTENDED {
t.Fatalf("expected a PATCH_EXTENDED mutation, got %v", m.Type)
}
if m.Directory != "/buckets" || m.Name != "mybucket" {
t.Fatalf("expected the mutation to target /buckets/mybucket, got %s/%s", m.Directory, m.Name)
}
if got := string(m.SetExtended[scheduler.BucketLifecycleConfigurationXMLKey]); got != "<LifecycleConfiguration/>" {
t.Fatalf("expected the XML key to carry the marshaled config, got %q", got)
}
if len(m.DeleteExtended) != 0 {
t.Fatalf("expected no key deletions when saving rules, got %v", m.DeleteExtended)
}
}
func TestBucketLifecycleMutation_ClearsBothKeys(t *testing.T) {
m := bucketLifecycleMutation("/buckets", "mybucket", nil)
if len(m.SetExtended) != 0 {
t.Fatalf("expected no key writes when clearing, got %v", m.SetExtended)
}
want := map[string]bool{
scheduler.BucketLifecycleConfigurationXMLKey: true,
scheduler.BucketLifecycleTransitionMinimumObjectSizeKey: true,
}
if len(m.DeleteExtended) != len(want) {
t.Fatalf("expected both lifecycle keys to be cleared, got %v", m.DeleteExtended)
}
for _, k := range m.DeleteExtended {
if !want[k] {
t.Fatalf("unexpected key cleared: %s", k)
}
}
}
// A whole-entry write would have carried the rest of the bucket entry with it;
// the patch must name only the keys it owns, so a concurrent owner or quota
// change survives.
func TestBucketLifecycleMutation_TouchesOnlyLifecycleKeys(t *testing.T) {
for _, m := range []*filer_pb.ObjectMutation{
bucketLifecycleMutation("/buckets", "mybucket", []byte("<LifecycleConfiguration/>")),
bucketLifecycleMutation("/buckets", "mybucket", nil),
} {
if m.Entry != nil {
t.Fatal("expected the mutation to carry no entry snapshot")
}
if m.SetContent {
t.Fatal("expected the mutation to leave entry content alone")
}
for k := range m.SetExtended {
if k != scheduler.BucketLifecycleConfigurationXMLKey {
t.Fatalf("unexpected key written: %s", k)
}
}
}
}
func TestSetBucketLifecycle_RejectsOversizedConfiguration(t *testing.T) {
// A single rule ID long enough to blow the cap: this must fail before any
// filer call, which is what makes it testable without one.
rule := minimalLifecycleRule()
rule.ID = strings.Repeat("x", scheduler.MaxBucketLifecycleConfigurationSize+1)
err := (&AdminServer{}).SetBucketLifecycle("mybucket", []BucketLifecycleRule{rule})
if err == nil {
t.Fatal("expected an oversized lifecycle configuration to be rejected")
}
if !strings.Contains(err.Error(), "exceeds") {
t.Fatalf("expected a size-limit error, got: %v", err)
}
}
func TestBucketLifecycleErrorStatus(t *testing.T) {
if got := bucketLifecycleErrorStatus(fmt.Errorf("%w: mybucket", ErrBucketNotFound)); got != http.StatusNotFound {
t.Fatalf("expected a missing bucket to map to 404, got %d", got)
}
if got := bucketLifecycleErrorStatus(errors.New("filer unreachable")); got != http.StatusInternalServerError {
t.Fatalf("expected an unrelated failure to stay 500, got %d", got)
}
}
+155
View File
@@ -2,6 +2,7 @@ package dash
import (
"context"
"errors"
"fmt"
"net/http"
"os"
@@ -13,8 +14,15 @@ import (
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/s3api"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle"
)
// MaxBucketLifecycleRules mirrors AWS S3's limit of 1000 lifecycle rules per bucket.
const MaxBucketLifecycleRules = 1000
// MaxBucketLifecycleRuleIDLength mirrors the S3 API's limit on a lifecycle rule's <ID>.
const MaxBucketLifecycleRuleIDLength = 255
// MaxOwnerNameLength is the maximum allowed length for bucket owner identity names.
// This is a reasonable limit to prevent abuse; AWS IAM user names are limited to 64 chars,
// but we use 256 to allow for more complex identity formats (e.g., email addresses).
@@ -102,6 +110,153 @@ func (s *AdminServer) ShowBucketLifecycle(w http.ResponseWriter, r *http.Request
writeJSON(w, http.StatusOK, lifecycle)
}
// UpdateBucketLifecycle replaces the entire lifecycle configuration for a bucket.
func (s *AdminServer) UpdateBucketLifecycle(w http.ResponseWriter, r *http.Request) {
if !requireSessionCSRFToken(w, r) {
return
}
bucketName := mux.Vars(r)["bucket"]
if bucketName == "" {
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
return
}
var req struct {
Rules []BucketLifecycleRule `json:"rules"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if err := validateBucketLifecycleRules(req.Rules); err != nil {
writeJSONError(w, http.StatusBadRequest, err.Error())
return
}
if err := s.SetBucketLifecycle(bucketName, req.Rules); err != nil {
writeJSONError(w, bucketLifecycleErrorStatus(err), "Failed to update bucket lifecycle: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"message": "Bucket lifecycle updated successfully",
"bucket": bucketName,
})
}
// DeleteBucketLifecycle clears the entire lifecycle configuration for a bucket.
func (s *AdminServer) DeleteBucketLifecycle(w http.ResponseWriter, r *http.Request) {
if !requireSessionCSRFToken(w, r) {
return
}
bucketName := mux.Vars(r)["bucket"]
if bucketName == "" {
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
return
}
if err := s.SetBucketLifecycle(bucketName, nil); err != nil {
writeJSONError(w, bucketLifecycleErrorStatus(err), "Failed to delete bucket lifecycle: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"message": "Bucket lifecycle deleted successfully",
"bucket": bucketName,
})
}
// bucketLifecycleErrorStatus keeps a request for a bucket that does not exist
// out of the 5xx bucket, where a client would retry it.
func bucketLifecycleErrorStatus(err error) int {
if errors.Is(err, ErrBucketNotFound) {
return http.StatusNotFound
}
return http.StatusInternalServerError
}
// validateBucketLifecycleRules rejects a rule set before any write is
// attempted, so a malformed PUT can't half-apply. Mirrors the constraints
// AWS enforces on PutBucketLifecycleConfiguration plus the local rule cap.
func validateBucketLifecycleRules(rules []BucketLifecycleRule) error {
if len(rules) > MaxBucketLifecycleRules {
return fmt.Errorf("a bucket may have at most %d lifecycle rules, got %d", MaxBucketLifecycleRules, len(rules))
}
seenIDs := make(map[string]bool, len(rules))
for i, rule := range rules {
label := fmt.Sprintf("rule %d", i+1)
if rule.ID != "" {
label = fmt.Sprintf("rule %q", rule.ID)
if len(rule.ID) > MaxBucketLifecycleRuleIDLength {
return fmt.Errorf("%s: ID must be %d characters or less", label, MaxBucketLifecycleRuleIDLength)
}
if seenIDs[rule.ID] {
return fmt.Errorf("duplicate rule ID %q", rule.ID)
}
seenIDs[rule.ID] = true
}
switch rule.Status {
case s3lifecycle.StatusEnabled, s3lifecycle.StatusDisabled:
default:
return fmt.Errorf("%s: status must be %q or %q, got %q", label, s3lifecycle.StatusEnabled, s3lifecycle.StatusDisabled, rule.Status)
}
if rule.ExpirationDays > 0 && rule.ExpirationDate != "" {
return fmt.Errorf("%s: expiration_days and expiration_date are mutually exclusive", label)
}
if rule.ExpirationDays < 0 {
return fmt.Errorf("%s: expiration_days must be positive", label)
}
if rule.ExpirationDate != "" {
if _, err := time.Parse(time.DateOnly, rule.ExpirationDate); err != nil {
return fmt.Errorf("%s: invalid expiration_date %q, expected YYYY-MM-DD", label, rule.ExpirationDate)
}
}
if rule.ExpiredObjectDeleteMarker && (rule.ExpirationDays > 0 || rule.ExpirationDate != "") {
return fmt.Errorf("%s: expired_object_delete_marker cannot be combined with expiration_days or expiration_date", label)
}
if rule.NoncurrentVersionExpirationDays < 0 {
return fmt.Errorf("%s: noncurrent_version_expiration_days must be positive", label)
}
if rule.NewerNoncurrentVersions < 0 {
return fmt.Errorf("%s: newer_noncurrent_versions must be positive", label)
}
if rule.AbortMultipartDays < 0 {
return fmt.Errorf("%s: abort_multipart_days must be positive", label)
}
if rule.SizeGreaterThan < 0 || rule.SizeLessThan < 0 {
return fmt.Errorf("%s: size_greater_than and size_less_than must be positive", label)
}
if rule.SizeGreaterThan > 0 && rule.SizeLessThan > 0 && rule.SizeGreaterThan >= rule.SizeLessThan {
return fmt.Errorf("%s: size_greater_than must be less than size_less_than", label)
}
for k, v := range rule.Tags {
if strings.TrimSpace(k) == "" {
return fmt.Errorf("%s: tag keys must not be empty", label)
}
if strings.TrimSpace(v) == "" {
return fmt.Errorf("%s: tag value for key %q must not be empty", label, k)
}
}
hasAction := rule.ExpirationDays > 0 || rule.ExpirationDate != "" || rule.ExpiredObjectDeleteMarker ||
rule.NoncurrentVersionExpirationDays > 0 || rule.NewerNoncurrentVersions > 0 || rule.AbortMultipartDays > 0
if !hasAction {
return fmt.Errorf("%s: must specify at least one action (expiration, noncurrent version expiration, or abort incomplete multipart upload)", label)
}
}
return nil
}
// CreateBucket creates a new S3 bucket
func (s *AdminServer) CreateBucket(w http.ResponseWriter, r *http.Request) {
var req CreateBucketRequest
+2
View File
@@ -179,6 +179,8 @@ func (h *AdminHandlers) registerAPIRoutes(api *mux.Router, enforceWrite bool) {
s3Api.Handle("/buckets/{bucket}", wrapWrite(h.adminServer.DeleteBucket)).Methods(http.MethodDelete)
s3Api.HandleFunc("/buckets/{bucket}", h.adminServer.ShowBucketDetails).Methods(http.MethodGet)
s3Api.HandleFunc("/buckets/{bucket}/lifecycle", h.adminServer.ShowBucketLifecycle).Methods(http.MethodGet)
s3Api.Handle("/buckets/{bucket}/lifecycle", wrapWrite(h.adminServer.UpdateBucketLifecycle)).Methods(http.MethodPut)
s3Api.Handle("/buckets/{bucket}/lifecycle", wrapWrite(h.adminServer.DeleteBucketLifecycle)).Methods(http.MethodDelete)
s3Api.Handle("/buckets/{bucket}/quota", wrapWrite(h.adminServer.UpdateBucketQuota)).Methods(http.MethodPut)
s3Api.Handle("/buckets/{bucket}/owner", wrapWrite(h.adminServer.UpdateBucketOwner)).Methods(http.MethodPut)
@@ -42,6 +42,26 @@ func TestSetupRoutes_RegistersPluginSchedulerStatesAPI_WithAuth(t *testing.T) {
}
}
func TestSetupRoutes_RegistersBucketLifecycleAPI_NoAuth(t *testing.T) {
router := mux.NewRouter()
newRouteTestAdminHandlers().SetupRoutes(router, false, "", "", "", "", true)
assertHasRoute(t, router, http.MethodGet, "/api/s3/buckets/example/lifecycle")
assertHasRoute(t, router, http.MethodPut, "/api/s3/buckets/example/lifecycle")
assertHasRoute(t, router, http.MethodDelete, "/api/s3/buckets/example/lifecycle")
}
func TestSetupRoutes_RegistersBucketLifecycleAPI_WithAuth(t *testing.T) {
router := mux.NewRouter()
newRouteTestAdminHandlers().SetupRoutes(router, true, "admin", "password", "", "", true)
assertHasRoute(t, router, http.MethodGet, "/api/s3/buckets/example/lifecycle")
assertHasRoute(t, router, http.MethodPut, "/api/s3/buckets/example/lifecycle")
assertHasRoute(t, router, http.MethodDelete, "/api/s3/buckets/example/lifecycle")
}
func TestSetupRoutes_RegistersPluginPages_NoAuth(t *testing.T) {
router := mux.NewRouter()
+637 -122
View File
@@ -637,7 +637,7 @@ templ S3Buckets(data dash.S3BucketsData) {
<!-- Bucket Lifecycle Modal -->
<div class="modal fade" id="bucketLifecycleModal" tabindex="-1" aria-labelledby="bucketLifecycleModalLabel" aria-hidden="true">
<div class="modal-dialog modal-lg">
<div class="modal-dialog modal-xl">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title" id="bucketLifecycleModalLabel">
@@ -649,7 +649,13 @@ templ S3Buckets(data dash.S3BucketsData) {
<div id="bucketLifecycleContent"></div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
<button type="button" class="btn btn-outline-danger me-auto" id="lifecycleDeleteAllBtn">
<i class="fas fa-trash me-1"></i>Delete all rules
</button>
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Cancel</button>
<button type="button" class="btn btn-primary" id="lifecycleSaveAllBtn">
<i class="fas fa-save me-1"></i>Save
</button>
</div>
</div>
</div>
@@ -712,6 +718,69 @@ templ S3Buckets(data dash.S3BucketsData) {
let lifecycleRequestSeq = 0;
let cachedUsers = null;
// Working copy of the lifecycle rules currently shown in the modal.
// Edits (add/edit/delete rule) only mutate this local state; nothing is
// sent to the server until "Save" or "Delete all rules" is clicked.
let lifecycleEditor = { bucket: null, rules: [], editingIndex: null, editingRule: null, rawXml: '', hasTransition: false };
function escapeHtml(v) {
return String(v ?? '')
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
// Shadows admin.js's formatBytes for this page, so it keeps that
// signature and unit list; the extra guard is for a size the API left
// out, which the shared one renders as "NaN undefined".
function formatBytes(bytes, decimals = 2) {
if (!bytes) return '0 Bytes';
const k = 1024;
const dm = decimals < 0 ? 0 : decimals;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
}
function formatDays(n) {
return n + (n === 1 ? ' day' : ' days');
}
// csrfHeaders reads the token the layout stamps into every page
// (<meta name="csrf-token">) so mutating lifecycle requests pass it the
// same way the newer admin pages (e.g. s3tables) do.
function csrfHeaders() {
const meta = document.querySelector('meta[name="csrf-token"]');
const token = meta ? (meta.getAttribute('content') || '') : '';
const headers = { 'Content-Type': 'application/json' };
if (token) {
headers['X-CSRF-Token'] = token;
}
return headers;
}
// parseLifecycleResponse reads the body as text before parsing, so a
// non-JSON response (a proxy's HTML error page, an empty body, plain
// text from some middleware) surfaces the HTTP status instead of
// throwing a raw JSON syntax error out of response.json().
function parseLifecycleResponse(response) {
return response.text().then(text => {
let data = {};
if (text) {
try {
data = JSON.parse(text);
} catch (e) {
data = { error: response.statusText || ('HTTP ' + response.status) };
}
} else if (!response.ok) {
data = { error: response.statusText || ('HTTP ' + response.status) };
}
return { ok: response.ok, data: data };
});
}
document.addEventListener('DOMContentLoaded', function() {
// Add click handlers to pagination links
document.querySelectorAll('.pagination-link').forEach(link => {
@@ -1074,6 +1143,10 @@ templ S3Buckets(data dash.S3BucketsData) {
button.addEventListener('click', function() {
const bucketName = this.dataset.bucketName;
// Reset the working copy until the GET below repopulates it,
// so Save/Delete-all can't act on the previous bucket's rules.
lifecycleEditor = { bucket: null, rules: [], editingIndex: null, editingRule: null, rawXml: '', hasTransition: false };
document.getElementById('bucketLifecycleModalLabel').innerHTML =
'<i class="fas fa-recycle me-2"></i>Lifecycle - ' + bucketName;
@@ -1090,14 +1163,14 @@ templ S3Buckets(data dash.S3BucketsData) {
// Drop responses that arrive after another bucket was opened
const requestSeq = ++lifecycleRequestSeq;
fetch(basePath('/api/s3/buckets/' + bucketName + '/lifecycle'))
.then(response => response.json())
.then(data => {
.then(parseLifecycleResponse)
.then(({ ok, data }) => {
if (requestSeq !== lifecycleRequestSeq) return;
if (data.error) {
if (!ok || data.error) {
document.getElementById('bucketLifecycleContent').innerHTML =
'<div class="alert alert-danger">' +
'<i class="fas fa-exclamation-triangle me-2"></i>' +
'Error loading lifecycle rules: ' + data.error +
'Error loading lifecycle rules: ' + (data.error || 'unknown error') +
'<\/div>';
} else {
displayBucketLifecycle(data);
@@ -1114,6 +1187,67 @@ templ S3Buckets(data dash.S3BucketsData) {
});
});
});
// Save the whole working rule set back to the bucket.
document.getElementById('lifecycleSaveAllBtn').addEventListener('click', function() {
if (!lifecycleEditor.bucket) return;
// Fold in any rule the user is still editing so Save doesn't
// silently discard it; abort if that form isn't valid yet.
if (!commitOpenLifecycleRuleForm()) return;
// An empty list clears the configuration, same as Delete all rules,
// so it gets the same prompt.
if (lifecycleEditor.rules.length === 0 &&
!confirm('No rules left: saving removes the lifecycle configuration for ' + lifecycleEditor.bucket + '. This cannot be undone.')) {
return;
}
fetch(basePath('/api/s3/buckets/' + lifecycleEditor.bucket + '/lifecycle'), {
method: 'PUT',
headers: csrfHeaders(),
body: JSON.stringify({ rules: lifecycleEditor.rules })
})
.then(parseLifecycleResponse)
.then(({ ok, data }) => {
if (!ok || data.error) {
alert('Error saving lifecycle rules: ' + (data.error || 'unknown error'));
return;
}
if (lifecycleModalInstance) {
lifecycleModalInstance.hide();
}
setTimeout(() => location.reload(), 500);
})
.catch(error => {
console.error('Error:', error);
alert('Error saving lifecycle rules: ' + error.message);
});
});
// Clear the lifecycle configuration entirely.
document.getElementById('lifecycleDeleteAllBtn').addEventListener('click', function() {
if (!lifecycleEditor.bucket) return;
if (!confirm('Delete all lifecycle rules for ' + lifecycleEditor.bucket + '? This cannot be undone.')) return;
fetch(basePath('/api/s3/buckets/' + lifecycleEditor.bucket + '/lifecycle'), {
method: 'DELETE',
headers: csrfHeaders()
})
.then(parseLifecycleResponse)
.then(({ ok, data }) => {
if (!ok || data.error) {
alert('Error deleting lifecycle rules: ' + (data.error || 'unknown error'));
return;
}
if (lifecycleModalInstance) {
lifecycleModalInstance.hide();
}
setTimeout(() => location.reload(), 500);
})
.catch(error => {
console.error('Error:', error);
alert('Error deleting lifecycle rules: ' + error.message);
});
});
});
function deleteBucket() {
@@ -1145,23 +1279,6 @@ templ S3Buckets(data dash.S3BucketsData) {
function displayBucketDetails(data) {
const bucket = data.bucket;
function escapeHtml(v) {
return String(v ?? '')
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
function formatBytes(bytes) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
}
function formatDate(dateString) {
const date = new Date(dateString);
return date.toLocaleString();
@@ -1232,112 +1349,518 @@ function displayBucketDetails(data) {
document.getElementById('bucketDetailsContent').innerHTML = rows.join('');
}
// displayBucketLifecycle loads the GET response into the local working copy
// and renders the editor. Nothing is sent back to the server until Save or
// Delete all rules is clicked.
function displayBucketLifecycle(data) {
function escapeHtml(v) {
return String(v ?? '')
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
lifecycleEditor.bucket = data.bucket || lifecycleEditor.bucket;
lifecycleEditor.rules = (data.rules || []).map(cloneLifecycleRule);
lifecycleEditor.editingIndex = null;
lifecycleEditor.editingRule = null;
lifecycleEditor.rawXml = data.xml || '';
// SeaweedFS doesn't support Transition rules (PutBucketLifecycleConfiguration
// rejects them for Enabled rules), and the canonical rule shape the admin
// edits has no field for one, so a Transition set by another client would
// silently disappear on the next Save. Warn instead.
// Matches <Transition> and <NoncurrentVersionTransition> however they are
// spelled: with attributes, self-closed, or namespace-prefixed.
lifecycleEditor.hasTransition = /<(\w+:)?(NoncurrentVersion)?Transition[\s>\/]/.test(data.xml || '');
renderLifecycleEditor();
}
// normalizeLifecycleStatus maps a stored status onto the two the editor can
// show. The S3 API stores <Status> unvalidated, and the engine runs a rule
// only on an exact "Enabled", so everything else is Disabled - collapsing it
// here keeps an odd spelling from leaving both radios unchecked, which would
// throw out of readLifecycleRuleFromForm and make the modal look dead.
function normalizeLifecycleStatus(status) {
return status === 'Enabled' ? 'Enabled' : 'Disabled';
}
function cloneLifecycleRule(r) {
return {
id: r.id || '',
status: normalizeLifecycleStatus(r.status),
prefix: r.prefix || '',
tags: Object.assign({}, r.tags || {}),
size_greater_than: r.size_greater_than || null,
size_less_than: r.size_less_than || null,
expiration_days: r.expiration_days || null,
expiration_date: r.expiration_date || '',
expired_object_delete_marker: !!r.expired_object_delete_marker,
noncurrent_version_expiration_days: r.noncurrent_version_expiration_days || null,
newer_noncurrent_versions: r.newer_noncurrent_versions || null,
abort_multipart_days: r.abort_multipart_days || null,
};
}
function blankLifecycleRule() {
return {
id: '', status: 'Enabled', prefix: '', tags: {},
size_greater_than: null, size_less_than: null,
expiration_days: null, expiration_date: '',
expired_object_delete_marker: false,
noncurrent_version_expiration_days: null, newer_noncurrent_versions: null,
abort_multipart_days: null,
};
}
function describeLifecycleScope(rule) {
const parts = [];
if (rule.prefix) {
parts.push('<code>' + escapeHtml(rule.prefix) + '<\/code>');
}
Object.entries(rule.tags || {}).forEach(([k, v]) => {
parts.push('<span class="badge bg-light text-dark border">' + escapeHtml(k) + '=' + escapeHtml(v) + '<\/span>');
});
if (rule.size_greater_than) {
parts.push('size &gt; ' + formatBytes(rule.size_greater_than));
}
if (rule.size_less_than) {
parts.push('size &lt; ' + formatBytes(rule.size_less_than));
}
return parts.length ? parts.join(' ') : '<span class="text-muted">Whole bucket<\/span>';
}
function describeLifecycleActions(rule) {
const actions = [];
if (rule.expiration_days) {
actions.push('Expire after ' + formatDays(rule.expiration_days));
}
if (rule.expiration_date) {
actions.push('Expire on ' + escapeHtml(rule.expiration_date));
}
if (rule.expired_object_delete_marker) {
actions.push('Remove expired object delete markers');
}
if (rule.noncurrent_version_expiration_days) {
let action = 'Expire noncurrent versions after ' + formatDays(rule.noncurrent_version_expiration_days);
if (rule.newer_noncurrent_versions) {
action += ', keep ' + rule.newer_noncurrent_versions + ' newest';
}
actions.push(action);
} else if (rule.newer_noncurrent_versions) {
// Standalone (no expiration_days): the engine still recognizes this
// as an action that expires noncurrent versions beyond the N newest.
actions.push('Keep ' + rule.newer_noncurrent_versions + ' newest noncurrent versions');
}
if (rule.abort_multipart_days) {
actions.push('Abort incomplete multipart uploads after ' + formatDays(rule.abort_multipart_days));
}
if (actions.length === 0) {
return '<span class="text-muted">None<\/span>';
}
return actions.join('<br>');
}
function lifecycleRuleRowHtml(rule, idx) {
const statusHtml = rule.status === 'Enabled'
? '<span class="badge bg-success">Enabled<\/span>'
: '<span class="badge bg-secondary">' + escapeHtml(rule.status) + '<\/span>';
return '<tr>' +
'<td>' + (rule.id ? escapeHtml(rule.id) : '<span class="text-muted">unnamed<\/span>') + '<\/td>' +
'<td>' + statusHtml + '<\/td>' +
'<td>' + describeLifecycleScope(rule) + '<\/td>' +
'<td>' + describeLifecycleActions(rule) + '<\/td>' +
'<td class="text-end text-nowrap">' +
'<button type="button" class="btn btn-sm btn-outline-secondary lifecycle-edit-rule-btn" data-index="' + idx + '" title="Edit rule"><i class="fas fa-pen"><\/i><\/button> ' +
'<button type="button" class="btn btn-sm btn-outline-danger lifecycle-delete-rule-btn" data-index="' + idx + '" title="Delete rule"><i class="fas fa-trash"><\/i><\/button>' +
'<\/td>' +
'<\/tr>';
}
function lifecycleRuleFormHtml() {
const rule = lifecycleEditor.editingRule;
const tagRowHtml = (k, v) =>
'<div class="input-group input-group-sm mb-1 lifecycle-tag-row">' +
'<input type="text" class="form-control lifecycle-tag-key" placeholder="Key" value="' + escapeHtml(k) + '">' +
'<input type="text" class="form-control lifecycle-tag-value" placeholder="Value" value="' + escapeHtml(v) + '">' +
'<button type="button" class="btn btn-outline-danger lifecycle-remove-tag-btn"><i class="fas fa-times"><\/i><\/button>' +
'<\/div>';
const tagRows = Object.entries(rule.tags || {}).map(([k, v]) => tagRowHtml(k, v)).join('');
return '' +
'<div class="card mb-3" id="lifecycleRuleForm">' +
'<div class="card-body">' +
'<h6 class="card-title">' + (lifecycleEditor.editingIndex === 'new' ? 'Add rule' : 'Edit rule') + '<\/h6>' +
'<div class="row g-2 mb-2">' +
'<div class="col-md-6">' +
'<label class="form-label small">ID (optional)<\/label>' +
'<input type="text" class="form-control form-control-sm" id="lifecycleRuleId" value="' + escapeHtml(rule.id) + '">' +
'<\/div>' +
'<div class="col-md-6">' +
'<label class="form-label small d-block">Status<\/label>' +
'<div class="form-check form-check-inline">' +
'<input class="form-check-input" type="radio" name="lifecycleRuleStatus" id="lifecycleStatusEnabled" value="Enabled"' + (rule.status === 'Enabled' ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleStatusEnabled">Enabled<\/label>' +
'<\/div>' +
'<div class="form-check form-check-inline">' +
'<input class="form-check-input" type="radio" name="lifecycleRuleStatus" id="lifecycleStatusDisabled" value="Disabled"' + (rule.status === 'Disabled' ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleStatusDisabled">Disabled<\/label>' +
'<\/div>' +
'<\/div>' +
'<\/div>' +
'<div class="mb-2">' +
'<label class="form-label small">Prefix<\/label>' +
'<input type="text" class="form-control form-control-sm" id="lifecycleRulePrefix" placeholder="logs/" value="' + escapeHtml(rule.prefix) + '">' +
'<\/div>' +
'<div class="mb-2">' +
'<label class="form-label small d-block">Tags<\/label>' +
'<div id="lifecycleTagRows">' + tagRows + '<\/div>' +
'<button type="button" class="btn btn-sm btn-outline-secondary" id="lifecycleAddTagBtn"><i class="fas fa-plus me-1"><\/i>Add tag<\/button>' +
'<\/div>' +
'<div class="row g-2 mb-3">' +
'<div class="col-md-6">' +
'<label class="form-label small">Size greater than (bytes)<\/label>' +
'<input type="number" class="form-control form-control-sm" id="lifecycleRuleSizeGT" min="0" step="1" value="' + (rule.size_greater_than || '') + '">' +
'<\/div>' +
'<div class="col-md-6">' +
'<label class="form-label small">Size less than (bytes)<\/label>' +
'<input type="number" class="form-control form-control-sm" id="lifecycleRuleSizeLT" min="0" step="1" value="' + (rule.size_less_than || '') + '">' +
'<\/div>' +
'<\/div>' +
'<hr>' +
'<h6 class="small text-muted text-uppercase">Actions<\/h6>' +
'<div class="row g-2 align-items-center mb-2">' +
'<div class="col-auto">' +
'<div class="form-check">' +
'<input class="form-check-input" type="checkbox" id="lifecycleExpireDaysEnabled"' + (rule.expiration_days ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleExpireDaysEnabled">Expire after<\/label>' +
'<\/div>' +
'<\/div>' +
'<div class="col-auto">' +
'<input type="number" class="form-control form-control-sm" id="lifecycleExpireDays" min="1" step="1" style="width: 6rem;" value="' + (rule.expiration_days || '') + '"' + (rule.expiration_days ? '' : ' disabled') + '>' +
'<\/div><div class="col-auto">days<\/div>' +
'<\/div>' +
'<div class="row g-2 align-items-center mb-2">' +
'<div class="col-auto">' +
'<div class="form-check">' +
'<input class="form-check-input" type="checkbox" id="lifecycleExpireDateEnabled"' + (rule.expiration_date ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleExpireDateEnabled">Expire on date<\/label>' +
'<\/div>' +
'<\/div>' +
'<div class="col-auto">' +
'<input type="date" class="form-control form-control-sm" id="lifecycleExpireDate" value="' + escapeHtml(rule.expiration_date) + '"' + (rule.expiration_date ? '' : ' disabled') + '>' +
'<\/div>' +
'<\/div>' +
'<div class="form-check mb-2">' +
'<input class="form-check-input" type="checkbox" id="lifecycleDeleteMarker"' + (rule.expired_object_delete_marker ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleDeleteMarker">Remove expired object delete markers<\/label>' +
'<\/div>' +
(() => {
const noncurrentActive = !!(rule.noncurrent_version_expiration_days || rule.newer_noncurrent_versions);
return '<div class="row g-2 align-items-center mb-2">' +
'<div class="col-auto">' +
'<div class="form-check">' +
'<input class="form-check-input" type="checkbox" id="lifecycleNoncurrentEnabled"' + (noncurrentActive ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleNoncurrentEnabled">Limit noncurrent versions<\/label>' +
'<\/div>' +
'<\/div>' +
'<div class="col-auto">after<\/div>' +
'<div class="col-auto">' +
'<input type="number" class="form-control form-control-sm" id="lifecycleNoncurrentDays" min="1" step="1" style="width: 6rem;" value="' + (rule.noncurrent_version_expiration_days || '') + '"' + (noncurrentActive ? '' : ' disabled') + '>' +
'<\/div><div class="col-auto">days (optional), keep<\/div>' +
'<div class="col-auto">' +
'<input type="number" class="form-control form-control-sm" id="lifecycleNoncurrentKeep" min="0" step="1" style="width: 6rem;" value="' + (rule.newer_noncurrent_versions || '') + '"' + (noncurrentActive ? '' : ' disabled') + '>' +
'<\/div><div class="col-auto">newest (optional)<\/div>' +
'<\/div>';
})() +
'<div class="row g-2 align-items-center mb-3">' +
'<div class="col-auto">' +
'<div class="form-check">' +
'<input class="form-check-input" type="checkbox" id="lifecycleAbortMPUEnabled"' + (rule.abort_multipart_days ? ' checked' : '') + '>' +
'<label class="form-check-label" for="lifecycleAbortMPUEnabled">Abort incomplete multipart uploads after<\/label>' +
'<\/div>' +
'<\/div>' +
'<div class="col-auto">' +
'<input type="number" class="form-control form-control-sm" id="lifecycleAbortMPUDays" min="1" step="1" style="width: 6rem;" value="' + (rule.abort_multipart_days || '') + '"' + (rule.abort_multipart_days ? '' : ' disabled') + '>' +
'<\/div><div class="col-auto">days<\/div>' +
'<\/div>' +
'<div class="text-end">' +
'<button type="button" class="btn btn-sm btn-secondary" id="lifecycleCancelRuleBtn">Cancel<\/button> ' +
'<button type="button" class="btn btn-sm btn-primary" id="lifecycleSaveRuleBtn">Save rule<\/button>' +
'<\/div>' +
'<\/div>' +
'<\/div>';
}
function renderLifecycleEditor() {
const container = document.getElementById('bucketLifecycleContent');
const rowsHtml = lifecycleEditor.rules.map(lifecycleRuleRowHtml).join('') ||
'<tr><td colspan="5" class="text-center text-muted py-3">No lifecycle rules<\/td><\/tr>';
let html = '';
if (lifecycleEditor.hasTransition) {
html += '<div class="alert alert-warning small">' +
'<i class="fas fa-triangle-exclamation me-1"><\/i>' +
'This bucket has a Transition rule set outside the admin (SeaweedFS does not support storage class transitions). Saving from here will drop it.' +
'<\/div>';
}
function formatBytes(bytes) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
html += '<div class="table-responsive mb-3">' +
'<table class="table table-sm align-middle">' +
'<thead><tr><th>Rule<\/th><th>Status<\/th><th>Scope<\/th><th>Actions<\/th><th><\/th><\/tr><\/thead>' +
'<tbody>' + rowsHtml + '<\/tbody>' +
'<\/table>' +
'<\/div>' +
'<button type="button" class="btn btn-sm btn-outline-primary mb-3" id="lifecycleAddRuleBtn">' +
'<i class="fas fa-plus me-1"><\/i>Add rule' +
'<\/button>';
if (lifecycleEditor.editingIndex !== null) {
html += lifecycleRuleFormHtml();
}
function days(n) {
return n + (n === 1 ? ' day' : ' days');
if (lifecycleEditor.rawXml) {
html += '<details class="mt-2">' +
'<summary class="text-muted small">Raw XML (as currently stored)<\/summary>' +
'<pre class="bg-light border rounded p-2 mt-2 small mb-0">' + escapeHtml(lifecycleEditor.rawXml) + '<\/pre>' +
'<\/details>';
}
const rules = data.rules || [];
if (rules.length === 0) {
document.getElementById('bucketLifecycleContent').innerHTML =
'<div class="text-center text-muted py-4">No lifecycle configuration<\/div>';
container.innerHTML = html;
bindLifecycleEditorEvents();
}
function bindLifecycleToggle(checkboxId, fieldId, mutuallyExclusiveId) {
const checkbox = document.getElementById(checkboxId);
const field = document.getElementById(fieldId);
if (!checkbox || !field) return;
checkbox.addEventListener('change', function() {
field.disabled = !this.checked;
if (this.checked && mutuallyExclusiveId) {
const other = document.getElementById(mutuallyExclusiveId);
if (other && other.checked) {
other.checked = false;
other.dispatchEvent(new Event('change'));
}
}
});
}
function bindLifecycleEditorEvents() {
const addBtn = document.getElementById('lifecycleAddRuleBtn');
if (addBtn) {
addBtn.addEventListener('click', function() {
// Fold in any rule already being edited before switching forms,
// so an invalid in-progress draft isn't silently discarded.
if (!commitOpenLifecycleRuleForm()) return;
lifecycleEditor.editingIndex = 'new';
lifecycleEditor.editingRule = blankLifecycleRule();
renderLifecycleEditor();
});
}
document.querySelectorAll('.lifecycle-edit-rule-btn').forEach(btn => {
btn.addEventListener('click', function() {
const idx = parseInt(this.dataset.index, 10);
// Same as above: committing never shifts indices (it either
// pushes a new rule at the end or replaces one in place), so
// idx captured above is still valid after this call.
if (!commitOpenLifecycleRuleForm()) return;
lifecycleEditor.editingIndex = idx;
lifecycleEditor.editingRule = cloneLifecycleRule(lifecycleEditor.rules[idx]);
renderLifecycleEditor();
});
});
document.querySelectorAll('.lifecycle-delete-rule-btn').forEach(btn => {
btn.addEventListener('click', function() {
const idx = parseInt(this.dataset.index, 10);
// Fold in an open form first, like every other transition does:
// re-rendering after the splice would otherwise redraw it from the
// snapshot taken when editing began and lose what was typed.
if (lifecycleEditor.editingIndex !== null && lifecycleEditor.editingIndex !== idx) {
if (!commitOpenLifecycleRuleForm()) return;
}
lifecycleEditor.rules.splice(idx, 1);
// Deleting a rule shifts every later index down by one, so an
// open edit form must follow its rule rather than silently
// start targeting whatever now sits at its old index.
if (lifecycleEditor.editingIndex === idx) {
lifecycleEditor.editingIndex = null;
lifecycleEditor.editingRule = null;
} else if (typeof lifecycleEditor.editingIndex === 'number' && lifecycleEditor.editingIndex > idx) {
lifecycleEditor.editingIndex -= 1;
}
renderLifecycleEditor();
});
});
const cancelBtn = document.getElementById('lifecycleCancelRuleBtn');
if (cancelBtn) {
cancelBtn.addEventListener('click', function() {
lifecycleEditor.editingIndex = null;
lifecycleEditor.editingRule = null;
renderLifecycleEditor();
});
}
const addTagBtn = document.getElementById('lifecycleAddTagBtn');
if (addTagBtn) {
addTagBtn.addEventListener('click', function() {
const rows = document.getElementById('lifecycleTagRows');
const row = document.createElement('div');
row.className = 'input-group input-group-sm mb-1 lifecycle-tag-row';
row.innerHTML =
'<input type="text" class="form-control lifecycle-tag-key" placeholder="Key">' +
'<input type="text" class="form-control lifecycle-tag-value" placeholder="Value">' +
'<button type="button" class="btn btn-outline-danger lifecycle-remove-tag-btn"><i class="fas fa-times"><\/i><\/button>';
rows.appendChild(row);
row.querySelector('.lifecycle-remove-tag-btn').addEventListener('click', function() {
row.remove();
});
});
}
document.querySelectorAll('.lifecycle-remove-tag-btn').forEach(btn => {
btn.addEventListener('click', function() {
this.closest('.lifecycle-tag-row').remove();
});
});
bindLifecycleToggle('lifecycleExpireDaysEnabled', 'lifecycleExpireDays', 'lifecycleExpireDateEnabled');
bindLifecycleToggle('lifecycleExpireDateEnabled', 'lifecycleExpireDate', 'lifecycleExpireDaysEnabled');
bindLifecycleToggle('lifecycleNoncurrentEnabled', 'lifecycleNoncurrentDays');
bindLifecycleToggle('lifecycleNoncurrentEnabled', 'lifecycleNoncurrentKeep');
bindLifecycleToggle('lifecycleAbortMPUEnabled', 'lifecycleAbortMPUDays');
const saveRuleBtn = document.getElementById('lifecycleSaveRuleBtn');
if (saveRuleBtn) {
saveRuleBtn.addEventListener('click', saveLifecycleRuleFromForm);
}
}
// readLifecycleRuleFromForm builds and validates a rule from the currently
// open edit form's DOM inputs. Returns { rule: null, error: '...' } when the
// form doesn't hold a valid rule, without mutating any state.
function readLifecycleRuleFromForm() {
const id = document.getElementById('lifecycleRuleId').value.trim();
const status = document.querySelector('input[name="lifecycleRuleStatus"]:checked').value;
const prefix = document.getElementById('lifecycleRulePrefix').value.trim();
const tags = {};
document.querySelectorAll('#lifecycleTagRows .lifecycle-tag-row').forEach(row => {
const key = row.querySelector('.lifecycle-tag-key').value.trim();
const value = row.querySelector('.lifecycle-tag-value').value.trim();
if (key) {
tags[key] = value;
}
});
const sizeGT = parseInt(document.getElementById('lifecycleRuleSizeGT').value, 10) || 0;
const sizeLT = parseInt(document.getElementById('lifecycleRuleSizeLT').value, 10) || 0;
const expireDaysEnabled = document.getElementById('lifecycleExpireDaysEnabled').checked;
const expireDateEnabled = document.getElementById('lifecycleExpireDateEnabled').checked;
const expirationDays = expireDaysEnabled ? (parseInt(document.getElementById('lifecycleExpireDays').value, 10) || 0) : 0;
const expirationDate = expireDateEnabled ? document.getElementById('lifecycleExpireDate').value : '';
const deleteMarker = document.getElementById('lifecycleDeleteMarker').checked;
const noncurrentEnabled = document.getElementById('lifecycleNoncurrentEnabled').checked;
const noncurrentDays = noncurrentEnabled ? (parseInt(document.getElementById('lifecycleNoncurrentDays').value, 10) || 0) : 0;
const noncurrentKeep = noncurrentEnabled ? (parseInt(document.getElementById('lifecycleNoncurrentKeep').value, 10) || 0) : 0;
const abortEnabled = document.getElementById('lifecycleAbortMPUEnabled').checked;
const abortDays = abortEnabled ? (parseInt(document.getElementById('lifecycleAbortMPUDays').value, 10) || 0) : 0;
if (expireDaysEnabled && expirationDays <= 0) {
return { rule: null, error: 'Enter a valid number of days for expiration.' };
}
if (expireDateEnabled && !expirationDate) {
return { rule: null, error: 'Enter a valid expiration date.' };
}
if (noncurrentEnabled && noncurrentDays <= 0 && noncurrentKeep <= 0) {
return { rule: null, error: 'Enter a number of days and/or a newest-versions-to-keep count for noncurrent version handling.' };
}
if (abortEnabled && abortDays <= 0) {
return { rule: null, error: 'Enter a valid number of days for aborting incomplete multipart uploads.' };
}
if (!expireDaysEnabled && !expireDateEnabled && !deleteMarker && !noncurrentEnabled && !abortEnabled) {
return { rule: null, error: 'Select at least one action for this rule.' };
}
// The server rejects this combination too; catching it here names the two
// fields instead of surfacing a 400 after the whole set is submitted.
if (deleteMarker && (expireDaysEnabled || expireDateEnabled)) {
return { rule: null, error: 'Removing expired object delete markers cannot be combined with an expiration.' };
}
if (sizeGT > 0 && sizeLT > 0 && sizeGT >= sizeLT) {
return { rule: null, error: '"Size greater than" must be less than "size less than".' };
}
return {
rule: {
id: id,
status: status,
prefix: prefix,
tags: tags,
size_greater_than: sizeGT > 0 ? sizeGT : null,
size_less_than: sizeLT > 0 ? sizeLT : null,
expiration_days: expirationDays > 0 ? expirationDays : null,
expiration_date: expirationDate || '',
expired_object_delete_marker: deleteMarker,
noncurrent_version_expiration_days: noncurrentDays > 0 ? noncurrentDays : null,
newer_noncurrent_versions: noncurrentKeep > 0 ? noncurrentKeep : null,
abort_multipart_days: abortDays > 0 ? abortDays : null,
},
error: null,
};
}
function saveLifecycleRuleFromForm() {
const { rule, error } = readLifecycleRuleFromForm();
if (error) {
alert(error);
return;
}
function describeScope(rule) {
const parts = [];
if (rule.prefix) {
parts.push('<code>' + escapeHtml(rule.prefix) + '<\/code>');
}
Object.entries(rule.tags || {}).forEach(([k, v]) => {
parts.push('<span class="badge bg-light text-dark border">' + escapeHtml(k) + '=' + escapeHtml(v) + '<\/span>');
});
if (rule.size_greater_than) {
parts.push('size &gt; ' + formatBytes(rule.size_greater_than));
}
if (rule.size_less_than) {
parts.push('size &lt; ' + formatBytes(rule.size_less_than));
}
return parts.length ? parts.join(' ') : '<span class="text-muted">Whole bucket<\/span>';
if (lifecycleEditor.editingIndex === 'new') {
lifecycleEditor.rules.push(rule);
} else {
lifecycleEditor.rules[lifecycleEditor.editingIndex] = rule;
}
lifecycleEditor.editingIndex = null;
lifecycleEditor.editingRule = null;
renderLifecycleEditor();
}
// commitOpenLifecycleRuleForm folds an in-progress add/edit form into
// lifecycleEditor.rules before the whole rule set is sent to the server, so
// clicking the modal's "Save" without first clicking "Save rule" doesn't
// silently drop the open edit. Returns false (and leaves the form open) when
// the open form doesn't hold a valid rule, so the caller can abort the save.
function commitOpenLifecycleRuleForm() {
if (lifecycleEditor.editingIndex === null) {
return true;
}
function describeActions(rule) {
const actions = [];
if (rule.expiration_days) {
actions.push('Expire after ' + days(rule.expiration_days));
}
if (rule.expiration_date) {
actions.push('Expire on ' + escapeHtml(rule.expiration_date));
}
if (rule.expired_object_delete_marker) {
actions.push('Remove expired object delete markers');
}
if (rule.noncurrent_version_expiration_days) {
let action = 'Expire noncurrent versions after ' + days(rule.noncurrent_version_expiration_days);
if (rule.newer_noncurrent_versions) {
action += ', keep ' + rule.newer_noncurrent_versions + ' newest';
}
actions.push(action);
}
if (rule.abort_multipart_days) {
actions.push('Abort incomplete multipart uploads after ' + days(rule.abort_multipart_days));
}
if (actions.length === 0) {
return '<span class="text-muted">None<\/span>';
}
return actions.join('<br>');
const { rule, error } = readLifecycleRuleFromForm();
if (error) {
alert(error);
return false;
}
const rows = [
'<div class="table-responsive">',
'<table class="table table-sm">',
'<thead><tr><th>Rule<\/th><th>Status<\/th><th>Scope<\/th><th>Actions<\/th><\/tr><\/thead>',
'<tbody>'
];
rules.forEach(rule => {
const statusHtml = rule.status === 'Enabled'
? '<span class="badge bg-success">Enabled<\/span>'
: '<span class="badge bg-secondary">' + escapeHtml(rule.status) + '<\/span>';
rows.push(
'<tr>' +
'<td>' + (rule.id ? escapeHtml(rule.id) : '<span class="text-muted">unnamed<\/span>') + '<\/td>' +
'<td>' + statusHtml + '<\/td>' +
'<td>' + describeScope(rule) + '<\/td>' +
'<td>' + describeActions(rule) + '<\/td>' +
'<\/tr>'
);
});
rows.push('<\/tbody>', '<\/table>', '<\/div>');
if (data.xml) {
rows.push(
'<details class="mt-2">',
'<summary class="text-muted small">Raw XML<\/summary>',
'<pre class="bg-light border rounded p-2 mt-2 small mb-0">' + escapeHtml(data.xml) + '<\/pre>',
'<\/details>'
);
if (lifecycleEditor.editingIndex === 'new') {
lifecycleEditor.rules.push(rule);
} else {
lifecycleEditor.rules[lifecycleEditor.editingIndex] = rule;
}
document.getElementById('bucketLifecycleContent').innerHTML = rows.join('');
lifecycleEditor.editingIndex = null;
lifecycleEditor.editingRule = null;
renderLifecycleEditor();
return true;
}
function goToPage(page) {
@@ -1380,14 +1903,6 @@ function displayBucketLifecycle(data) {
return '"' + str + '"';
}
function formatBytes(bytes) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
}
// Fetch all buckets from the API (not just the current page)
fetch(basePath('/api/s3/buckets'))
.then(response => response.json())
File diff suppressed because one or more lines are too long
+171
View File
@@ -3,8 +3,12 @@ package filer
import (
"bytes"
"context"
"crypto/md5"
"errors"
"fmt"
"io"
"strings"
"time"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/wdclient"
@@ -319,6 +323,173 @@ func ClearBucketReadOnly(ctx context.Context, client filer_pb.SeaweedFilerClient
return true, nil
}
// filerConfSnapshot is a read of filer.conf plus enough of the entry (or its
// absence) to make a follow-up write conditional via
// saveFilerConfConditionally, instead of blindly overwriting whatever is
// there by the time the write happens.
type filerConfSnapshot struct {
fc *FilerConf
entry *filer_pb.Entry // nil if filer.conf did not exist at read time
}
func readFilerConfSnapshot(ctx context.Context, client filer_pb.SeaweedFilerClient) (*filerConfSnapshot, error) {
resp, err := filer_pb.LookupEntry(ctx, client, &filer_pb.LookupDirectoryEntryRequest{
Directory: DirectoryEtcSeaweedFS,
Name: FilerConfName,
})
fc := NewFilerConf()
if errors.Is(err, filer_pb.ErrNotFound) {
return &filerConfSnapshot{fc: fc}, nil
}
if err != nil {
return nil, fmt.Errorf("read %s/%s: %v", DirectoryEtcSeaweedFS, FilerConfName, err)
}
if len(resp.Entry.Content) > 0 {
if err := fc.LoadFromBytes(resp.Entry.Content); err != nil {
return nil, fmt.Errorf("parse %s/%s: %v", DirectoryEtcSeaweedFS, FilerConfName, err)
}
}
return &filerConfSnapshot{fc: fc, entry: resp.Entry}, nil
}
// saveFilerConfConditionally writes snap.fc back to filer.conf, conditioned
// on nothing having created or modified the file since snap was read. The
// filer evaluates the condition under its per-path lock, so a racing writer
// fails the precondition instead of silently losing its change.
//
// The condition is an exact content check (IF_ETAG_MATCH against the MD5
// every writer stamps, see SaveInsideFiler), which mtime's one-second
// resolution cannot match. A filer.conf written before that stamp existed
// carries no hash, so that one write falls back to IF_UNMODIFIED_SINCE and
// self-heals from the next write on.
func saveFilerConfConditionally(ctx context.Context, client filer_pb.SeaweedFilerClient, snap *filerConfSnapshot) error {
var buf bytes.Buffer
if err := snap.fc.ToText(&buf); err != nil {
return err
}
content := buf.Bytes()
contentMd5 := md5.Sum(content)
if snap.entry == nil {
err := filer_pb.CreateEntry(ctx, client, &filer_pb.CreateEntryRequest{
Directory: DirectoryEtcSeaweedFS,
Entry: &filer_pb.Entry{
Name: FilerConfName,
IsDirectory: false,
Attributes: &filer_pb.FuseAttributes{
Mtime: time.Now().Unix(),
Crtime: time.Now().Unix(),
FileMode: uint32(0644),
FileSize: uint64(len(content)),
Md5: contentMd5[:],
},
Content: content,
},
Condition: &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{
{Kind: filer_pb.WriteCondition_IF_NOT_EXISTS},
}},
})
if err != nil {
return fmt.Errorf("create %s/%s: %w", DirectoryEtcSeaweedFS, FilerConfName, err)
}
return nil
}
entry := snap.entry
var condition *filer_pb.WriteCondition
if entry.Attributes != nil && len(entry.Attributes.Md5) > 0 {
condition = &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{
{Kind: filer_pb.WriteCondition_IF_ETAG_MATCH, Etags: []string{fmt.Sprintf("%x", entry.Attributes.Md5)}},
}}
} else {
var unmodifiedSince int64
if entry.Attributes != nil {
unmodifiedSince = entry.Attributes.Mtime
} else {
entry.Attributes = &filer_pb.FuseAttributes{}
}
condition = &filer_pb.WriteCondition{Clauses: []*filer_pb.WriteCondition_Clause{
{Kind: filer_pb.WriteCondition_IF_UNMODIFIED_SINCE, UnixTime: unmodifiedSince},
}}
}
entry.Content = content
entry.Attributes.Mtime = time.Now().Unix()
entry.Attributes.FileSize = uint64(len(content))
entry.Attributes.Md5 = contentMd5[:]
err := filer_pb.UpdateEntry(ctx, client, &filer_pb.UpdateEntryRequest{
Directory: DirectoryEtcSeaweedFS,
Entry: entry,
Condition: condition,
})
if err != nil {
return fmt.Errorf("update %s/%s: %w", DirectoryEtcSeaweedFS, FilerConfName, err)
}
return nil
}
// ClearBucketLifecycleDayTTLs removes any day-TTL filer.conf rules a legacy
// PutBucketLifecycleConfiguration handler installed under the bucket's path.
// Per-write TTL is now driven by the LifecycleTTLResolver built off the
// stored lifecycle XML, so a lingering day-TTL rule would double-stamp
// expiration (volume server expires under the old rule) or contradict a
// newly saved XML. The write is conditioned on filer.conf being unchanged
// since it was read here, so a concurrent writer fails this call rather than
// silently losing one side's change.
func ClearBucketLifecycleDayTTLs(ctx context.Context, client filer_pb.SeaweedFilerClient, bucketsPath, bucketName, collection string) error {
snap, err := readFilerConfSnapshot(ctx, client)
if err != nil {
return err
}
if snap.entry == nil {
return nil
}
changed := false
bucketPrefix := fmt.Sprintf("%s/%s/", bucketsPath, bucketName)
for prefix, ttl := range snap.fc.GetCollectionTtls(collection) {
if !strings.HasPrefix(prefix, bucketPrefix) || !strings.HasSuffix(ttl, "d") {
continue
}
locConf, found := snap.fc.GetLocationConf(prefix)
if !found {
continue
}
// Logged either way: this is a one-way migration, and the prefix and
// TTL are all an operator needs to put a rule back by hand.
if isLifecycleOwnedPathConf(locConf) {
glog.V(0).Infof("lifecycle migration: dropping legacy day-TTL rule %s ttl=%s", prefix, ttl)
snap.fc.DeleteLocationConf(prefix)
} else {
glog.V(0).Infof("lifecycle migration: clearing legacy day-TTL %s on operator rule %s", ttl, prefix)
updated := ClonePathConf(locConf)
updated.Ttl = ""
if err := snap.fc.SetLocationConf(updated); err != nil {
return err
}
}
changed = true
}
if !changed {
return nil
}
return saveFilerConfConditionally(ctx, client, snap)
}
// isLifecycleOwnedPathConf reports whether a rule looks like one the removed
// PutBucketLifecycleConfiguration add path created, which set only the
// routing and TTL fields. That path merged onto whatever already sat at the
// prefix, so anything else here - a disk type, WORM settings, a read-only
// flag, a placement pin - is an operator's, and deleting the whole rule to
// retire its TTL would take their configuration with it.
func isLifecycleOwnedPathConf(c *filer_pb.FilerConf_PathConf) bool {
return c.GetDiskType() == "" && !c.GetFsync() && !c.GetReadOnly() &&
c.GetDataCenter() == "" && c.GetRack() == "" && c.GetDataNode() == "" &&
c.GetMaxFileNameLength() == 0 && !c.GetDisableChunkDeletion() &&
c.Worm == nil && c.GetWormGracePeriodSeconds() == 0 && c.GetWormRetentionTimeSeconds() == 0
}
func (fc *FilerConf) GetCollectionTtls(collection string) (ttls map[string]string) {
ttls = make(map[string]string)
fc.rules.Walk(func(key []byte, value *filer_pb.FilerConf_PathConf) bool {
+341
View File
@@ -2,11 +2,17 @@ package filer
import (
"bytes"
"context"
"errors"
"fmt"
"reflect"
"sync"
"testing"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"google.golang.org/protobuf/proto"
)
@@ -248,3 +254,338 @@ func TestClearReadOnly(t *testing.T) {
assert.False(t, rule.ReadOnly)
assert.Equal(t, "7d", rule.Ttl)
}
// fakeFilerConfClient is a minimal in-memory filer_pb.SeaweedFilerClient that
// only supports the single-file round trip ReadInsideFiler/SaveInsideFiler
// need: lookup, create-if-absent, update. Embedding the interface satisfies
// the rest of it; calling any other method panics on the nil embedded value.
type fakeFilerConfClient struct {
filer_pb.SeaweedFilerClient
mu sync.Mutex
entries map[string]*filer_pb.Entry // key: dir+"/"+name
}
func newFakeFilerConfClient() *fakeFilerConfClient {
return &fakeFilerConfClient{entries: make(map[string]*filer_pb.Entry)}
}
func (c *fakeFilerConfClient) key(dir, name string) string { return dir + "/" + name }
func (c *fakeFilerConfClient) LookupDirectoryEntry(_ context.Context, in *filer_pb.LookupDirectoryEntryRequest, _ ...grpc.CallOption) (*filer_pb.LookupDirectoryEntryResponse, error) {
c.mu.Lock()
defer c.mu.Unlock()
e, ok := c.entries[c.key(in.Directory, in.Name)]
if !ok {
return nil, filer_pb.ErrNotFound
}
// A real gRPC round trip always hands back an independent copy (proto
// marshal/unmarshal), so mutating what the caller gets back (as
// saveFilerConfConditionally does before re-sending it) must not affect
// what conditionHoldsLocked below compares against.
return &filer_pb.LookupDirectoryEntryResponse{Entry: cloneFakeEntry(e)}, nil
}
func cloneFakeEntry(e *filer_pb.Entry) *filer_pb.Entry {
if e == nil {
return nil
}
// proto.Clone rather than a struct copy: filer_pb.Entry embeds
// protoimpl.MessageState (a sync.Mutex), which a plain `*e` copy would
// duplicate by value — exactly what go vet's copylocks check flags.
return proto.Clone(e).(*filer_pb.Entry)
}
func (c *fakeFilerConfClient) CreateEntry(_ context.Context, in *filer_pb.CreateEntryRequest, _ ...grpc.CallOption) (*filer_pb.CreateEntryResponse, error) {
c.mu.Lock()
defer c.mu.Unlock()
key := c.key(in.Directory, in.Entry.Name)
if !conditionHoldsLocked(in.Condition, c.entries[key]) {
return nil, errors.New("precondition failed")
}
c.entries[key] = in.Entry
return &filer_pb.CreateEntryResponse{}, nil
}
func (c *fakeFilerConfClient) UpdateEntry(_ context.Context, in *filer_pb.UpdateEntryRequest, _ ...grpc.CallOption) (*filer_pb.UpdateEntryResponse, error) {
c.mu.Lock()
defer c.mu.Unlock()
key := c.key(in.Directory, in.Entry.Name)
if !conditionHoldsLocked(in.Condition, c.entries[key]) {
return nil, errors.New("precondition failed")
}
c.entries[key] = in.Entry
return &filer_pb.UpdateEntryResponse{}, nil
}
// conditionHoldsLocked is a minimal stand-in for the filer server's
// writeConditionSatisfied (weed/server/filer_grpc_server_condition.go),
// covering just the clause kinds saveFilerConfConditionally uses, so tests
// can exercise the CAS path without a real filer server.
func conditionHoldsLocked(cond *filer_pb.WriteCondition, current *filer_pb.Entry) bool {
if cond == nil {
return true
}
for _, clause := range cond.Clauses {
switch clause.Kind {
case filer_pb.WriteCondition_IF_NOT_EXISTS:
if current != nil {
return false
}
case filer_pb.WriteCondition_IF_UNMODIFIED_SINCE:
if current != nil && current.Attributes != nil && current.Attributes.Mtime > clause.UnixTime {
return false
}
case filer_pb.WriteCondition_IF_ETAG_MATCH:
if current == nil {
return false
}
stored := ""
if current.Attributes != nil {
stored = fmt.Sprintf("%x", current.Attributes.Md5)
}
matched := false
for _, want := range clause.Etags {
if want == stored {
matched = true
break
}
}
if !matched {
return false
}
}
}
return true
}
// putFilerConf seeds the fake client's filer.conf with the given rules.
func putFilerConf(t *testing.T, client *fakeFilerConfClient, rules ...*filer_pb.FilerConf_PathConf) {
t.Helper()
fc := NewFilerConf()
for _, r := range rules {
require.NoError(t, fc.SetLocationConf(r))
}
var buf bytes.Buffer
require.NoError(t, fc.ToText(&buf))
client.entries[client.key(DirectoryEtcSeaweedFS, FilerConfName)] = &filer_pb.Entry{
Name: FilerConfName,
Content: buf.Bytes(),
Attributes: &filer_pb.FuseAttributes{},
}
}
// readFilerConfText returns the current filer.conf content stored in the fake
// client, or "" if none exists yet.
func readFilerConfText(client *fakeFilerConfClient) string {
e, ok := client.entries[client.key(DirectoryEtcSeaweedFS, FilerConfName)]
if !ok {
return ""
}
return string(e.Content)
}
func TestClearBucketLifecycleDayTTLs_NoFilerConf(t *testing.T) {
client := newFakeFilerConfClient()
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
assert.Empty(t, readFilerConfText(client))
}
func TestClearBucketLifecycleDayTTLs_NoMatchingRule(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/other/",
Collection: "other",
Ttl: "7d",
})
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
assert.Contains(t, readFilerConfText(client), "other")
}
func TestClearBucketLifecycleDayTTLs_RemovesDayTTLUnderBucket(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/mybucket/",
Collection: "mybucket",
Ttl: "7d",
})
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
reloaded := NewFilerConf()
require.NoError(t, reloaded.LoadFromBytes([]byte(readFilerConfText(client))))
_, found := reloaded.GetLocationConf("/buckets/mybucket/")
assert.False(t, found, "day-TTL rule should have been removed")
}
func TestClearBucketLifecycleDayTTLs_KeepsNonDayTTL(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/mybucket/",
Collection: "mybucket",
Ttl: "7m", // minutes, not days: not a legacy lifecycle TTL rule
})
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
reloaded := NewFilerConf()
require.NoError(t, reloaded.LoadFromBytes([]byte(readFilerConfText(client))))
_, found := reloaded.GetLocationConf("/buckets/mybucket/")
assert.True(t, found, "non-day TTL rule should be left alone")
}
func TestClearBucketLifecycleDayTTLs_KeepsOperatorSettingsAndClearsOnlyTheTTL(t *testing.T) {
// The removed add path merged its TTL onto whatever already sat at the
// prefix, so a rule carrying operator settings must survive with only its
// TTL retired - deleting it outright would drop the disk type and the
// read-only flag with it.
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/mybucket/logs/",
Collection: "mybucket",
Ttl: "30d",
DiskType: "hdd",
ReadOnly: true,
})
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
reloaded := NewFilerConf()
require.NoError(t, reloaded.LoadFromBytes([]byte(readFilerConfText(client))))
rule, found := reloaded.GetLocationConf("/buckets/mybucket/logs/")
require.True(t, found, "a rule carrying operator settings must not be deleted")
assert.Equal(t, "", rule.Ttl, "the legacy day TTL should be gone")
assert.Equal(t, "hdd", rule.DiskType, "the operator's disk type must survive")
assert.True(t, rule.ReadOnly, "the operator's read-only flag must survive")
}
func TestClearBucketLifecycleDayTTLs_KeepsWormSettings(t *testing.T) {
worm := true
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/mybucket/locked/",
Collection: "mybucket",
Ttl: "7d",
Worm: &worm,
WormRetentionTimeSeconds: 3600,
})
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
reloaded := NewFilerConf()
require.NoError(t, reloaded.LoadFromBytes([]byte(readFilerConfText(client))))
rule, found := reloaded.GetLocationConf("/buckets/mybucket/locked/")
require.True(t, found, "a WORM rule must not be deleted to retire a TTL")
assert.Equal(t, "", rule.Ttl)
assert.True(t, rule.GetWorm())
assert.Equal(t, uint64(3600), rule.WormRetentionTimeSeconds)
}
func TestClearBucketLifecycleDayTTLs_KeepsOtherBucketsAndCollections(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client,
&filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/mybucket/", Collection: "mybucket", Ttl: "7d"},
&filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/other/", Collection: "other", Ttl: "7d"},
// nested under the target bucket's path but tagged to a different
// collection (e.g. a filer-group-prefixed name): must not be swept up
// by this bucket's cleanup, since GetCollectionTtls filters by collection.
&filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/mybucket/nested/", Collection: "group_mybucket", Ttl: "7d"},
)
require.NoError(t, ClearBucketLifecycleDayTTLs(context.Background(), client, "/buckets", "mybucket", "mybucket"))
reloaded := NewFilerConf()
require.NoError(t, reloaded.LoadFromBytes([]byte(readFilerConfText(client))))
_, found := reloaded.GetLocationConf("/buckets/mybucket/")
assert.False(t, found, "the target bucket's day-TTL rule should be gone")
text := readFilerConfText(client)
assert.Contains(t, text, "other")
assert.Contains(t, text, "group_mybucket")
}
// bumpMtime models a writer that landed after unmodifiedSince, so an
// IF_UNMODIFIED_SINCE precondition built on that timestamp no longer holds.
func bumpMtime(client *fakeFilerConfClient, unmodifiedSince int64) {
client.mu.Lock()
defer client.mu.Unlock()
e := client.entries[client.key(DirectoryEtcSeaweedFS, FilerConfName)]
if e == nil {
return
}
if e.Attributes == nil {
e.Attributes = &filer_pb.FuseAttributes{}
}
e.Attributes.Mtime = unmodifiedSince + 1
}
func TestSaveFilerConfConditionally_RejectsUpdateModifiedConcurrently(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/a/", Collection: "a", Ttl: "7d"})
snap, err := readFilerConfSnapshot(context.Background(), client)
require.NoError(t, err)
require.NotNil(t, snap.entry)
require.Empty(t, snap.entry.Attributes.Md5, "test setup should model a pre-existing filer.conf with no stamped Md5")
// Someone else writes filer.conf after our read but before our write.
bumpMtime(client, snap.entry.Attributes.Mtime)
err = saveFilerConfConditionally(context.Background(), client, snap)
assert.Error(t, err, "expected a concurrent modification to fail the conditional write")
}
// TestSaveFilerConfConditionally_ExactContentCheckRejectsConcurrentModification
// exercises the primary path: once an entry has been written through this
// function (so it carries a content Md5), a later write with a stale
// snapshot is rejected by the exact IF_ETAG_MATCH check even when the
// concurrent writer happened to leave mtime unchanged — something a
// mtime-only precondition would have missed.
func TestSaveFilerConfConditionally_ExactContentCheckRejectsConcurrentModification(t *testing.T) {
client := newFakeFilerConfClient()
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/a/", Collection: "a", Ttl: "7d"})
// First write through saveFilerConfConditionally stamps Md5, moving
// subsequent reads onto the exact-content check.
snap, err := readFilerConfSnapshot(context.Background(), client)
require.NoError(t, err)
require.NoError(t, saveFilerConfConditionally(context.Background(), client, snap))
snap2, err := readFilerConfSnapshot(context.Background(), client)
require.NoError(t, err)
require.NotEmpty(t, snap2.entry.Attributes.Md5, "expected Md5 to have been stamped by the prior write")
// A concurrent writer goes through SaveInsideFiler, the path every
// other filer.conf writer uses. It lands in the same wall-clock second,
// so only the content hash can catch it.
concurrentFc := NewFilerConf()
require.NoError(t, concurrentFc.LoadFromBytes([]byte(readFilerConfText(client))))
require.NoError(t, concurrentFc.SetLocationConf(&filer_pb.FilerConf_PathConf{
LocationPrefix: "/buckets/other/", Collection: "other", Ttl: "3d",
}))
var buf bytes.Buffer
require.NoError(t, concurrentFc.ToText(&buf))
require.NoError(t, SaveInsideFiler(context.Background(), client, DirectoryEtcSeaweedFS, FilerConfName, buf.Bytes()))
client.entries[client.key(DirectoryEtcSeaweedFS, FilerConfName)].Attributes.Mtime = snap2.entry.Attributes.Mtime
err = saveFilerConfConditionally(context.Background(), client, snap2)
assert.Error(t, err, "expected the exact-content check to reject a write whose baseline content hash no longer matches")
assert.Contains(t, readFilerConfText(client), "/buckets/other/", "the concurrent writer's content must survive")
}
func TestSaveFilerConfConditionally_RejectsCreateWhenCreatedConcurrently(t *testing.T) {
client := newFakeFilerConfClient()
snap, err := readFilerConfSnapshot(context.Background(), client)
require.NoError(t, err)
require.Nil(t, snap.entry, "filer.conf should not exist yet")
// Someone else creates filer.conf after our read but before our write.
putFilerConf(t, client, &filer_pb.FilerConf_PathConf{LocationPrefix: "/buckets/b/", Collection: "b", Ttl: "3d"})
err = saveFilerConfConditionally(context.Background(), client, snap)
assert.Error(t, err, "expected a concurrent create to fail the conditional write")
assert.Contains(t, readFilerConfText(client), "/buckets/b/", "the concurrent writer's content must survive")
}
+10
View File
@@ -3,6 +3,7 @@ package filer
import (
"bytes"
"context"
"crypto/md5"
"time"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
@@ -41,6 +42,11 @@ func ReadInsideFiler(ctx context.Context, filerClient filer_pb.SeaweedFilerClien
return
}
// SaveInsideFiler writes content into the entry's inline Content. The content
// MD5 is stamped on every write so the entry's ETag always describes what is
// actually stored: conditional writers key IF_ETAG_MATCH off it, and a hash
// left behind by an earlier write would make that condition pass against
// content that has since changed.
func SaveInsideFiler(ctx context.Context, client filer_pb.SeaweedFilerClient, dir, name string, content []byte) error {
resp, err := filer_pb.LookupEntry(ctx, client, &filer_pb.LookupDirectoryEntryRequest{
@@ -48,6 +54,8 @@ func SaveInsideFiler(ctx context.Context, client filer_pb.SeaweedFilerClient, di
Name: name,
})
contentMd5 := md5.Sum(content)
if err == filer_pb.ErrNotFound {
err = filer_pb.CreateEntry(ctx, client, &filer_pb.CreateEntryRequest{
Directory: dir,
@@ -59,6 +67,7 @@ func SaveInsideFiler(ctx context.Context, client filer_pb.SeaweedFilerClient, di
Crtime: time.Now().Unix(),
FileMode: uint32(0644),
FileSize: uint64(len(content)),
Md5: contentMd5[:],
},
Content: content,
},
@@ -69,6 +78,7 @@ func SaveInsideFiler(ctx context.Context, client filer_pb.SeaweedFilerClient, di
entry.Content = content
entry.Attributes.Mtime = time.Now().Unix()
entry.Attributes.FileSize = uint64(len(content))
entry.Attributes.Md5 = contentMd5[:]
err = filer_pb.UpdateEntry(ctx, client, &filer_pb.UpdateEntryRequest{
Directory: dir,
Entry: entry,
+141
View File
@@ -3,6 +3,7 @@ package lifecycle_xml
import (
"bytes"
"encoding/xml"
"sort"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle"
)
@@ -81,6 +82,146 @@ func ruleToCanonical(r *Rule) *s3lifecycle.Rule {
return out
}
// CanonicalToLifecycle is the inverse of LifecycleToCanonical: it builds a
// marshalable Lifecycle from the engine's flat Rule shape. Only the fields
// s3lifecycle.Rule can represent are populated — there is no way back to
// Transition / NoncurrentVersionTransition, which the canonical form never
// carries.
func CanonicalToLifecycle(rules []*s3lifecycle.Rule) *Lifecycle {
lc := &Lifecycle{
Rules: make([]Rule, 0, len(rules)),
}
for _, r := range rules {
lc.Rules = append(lc.Rules, ruleFromCanonical(r))
}
return lc
}
// s3XMLNamespace is the namespace every S3 client stamps on a lifecycle
// document it PUTs. GetBucketLifecycleConfiguration replays the stored bytes
// verbatim, so a document written here has to carry it too or it would come
// back stripped of a namespace the client-written ones have.
const s3XMLNamespace = "http://s3.amazonaws.com/doc/2006-03-01/"
// MarshalCanonical serializes the canonical rules straight to a
// BucketLifecycleConfiguration XML document, mirroring ParseCanonical.
func MarshalCanonical(rules []*s3lifecycle.Rule) ([]byte, error) {
lc := CanonicalToLifecycle(rules)
out, err := xml.Marshal(struct {
Lifecycle
XMLNS string `xml:"xmlns,attr"`
}{Lifecycle: *lc, XMLNS: s3XMLNamespace})
if err != nil {
return nil, err
}
return append([]byte(xml.Header), out...), nil
}
func ruleFromCanonical(r *s3lifecycle.Rule) Rule {
out := Rule{
ID: r.ID,
Status: RuleStatus(r.Status),
Filter: filterFromCanonical(r.Prefix, r.FilterTags, r.FilterSizeGreaterThan, r.FilterSizeLessThan),
}
if r.ExpirationDays > 0 || !r.ExpirationDate.IsZero() || r.ExpiredObjectDeleteMarker {
out.Expiration = Expiration{
set: true,
Days: r.ExpirationDays,
}
if !r.ExpirationDate.IsZero() {
out.Expiration.Date = ExpirationDate{Time: r.ExpirationDate}
}
if r.ExpiredObjectDeleteMarker {
out.Expiration.DeleteMarker = ExpireDeleteMarker{val: true, set: true}
}
}
if r.NoncurrentVersionExpirationDays > 0 || r.NewerNoncurrentVersions > 0 {
out.NoncurrentVersionExpiration = NoncurrentVersionExpiration{
set: true,
NoncurrentDays: r.NoncurrentVersionExpirationDays,
NewerNoncurrentVersions: r.NewerNoncurrentVersions,
}
}
if r.AbortMPUDaysAfterInitiation > 0 {
out.AbortIncompleteMultipartUpload = AbortIncompleteMultipartUpload{
set: true,
DaysAfterInitiation: r.AbortMPUDaysAfterInitiation,
}
}
return out
}
// filterFromCanonical is the inverse of flattenFilter: it picks the
// narrowest Filter shape that represents the given prefix/tags/size bounds,
// matching what Filter.MarshalXML (single Prefix|Tag branch plus optional
// size bounds) and its And branch can each express.
func filterFromCanonical(prefix string, tags map[string]string, sizeGT, sizeLT int64) Filter {
hasSize := sizeGT > 0 || sizeLT > 0
// <Filter> holds exactly one predicate; anything more goes under <And>,
// two size bounds included, which is the form AWS documents for a range.
discriminants := 0
if prefix != "" {
discriminants++
}
discriminants += len(tags)
if sizeGT > 0 {
discriminants++
}
if sizeLT > 0 {
discriminants++
}
f := Filter{set: true, ObjectSizeGreaterThan: sizeGT, ObjectSizeLessThan: sizeLT}
switch {
case discriminants > 1:
f.andSet = true
f.And = And{
ObjectSizeGreaterThan: sizeGT,
ObjectSizeLessThan: sizeLT,
}
if prefix != "" {
f.And.Prefix = NewPrefix(prefix)
}
if len(tags) > 0 {
keys := make([]string, 0, len(tags))
for k := range tags {
keys = append(keys, k)
}
sort.Strings(keys)
f.And.Tags = make([]Tag, 0, len(keys))
for _, k := range keys {
f.And.Tags = append(f.And.Tags, Tag{Key: k, Value: tags[k]})
}
}
// The And branch carries its own size bounds; the enclosing
// Filter only emits them on the non-And path (see
// Filter.MarshalXML), so clear them here to avoid duplication.
f.ObjectSizeGreaterThan = 0
f.ObjectSizeLessThan = 0
case len(tags) == 1:
f.tagSet = true
for k, v := range tags {
f.Tag = Tag{Key: k, Value: v}
}
case hasSize:
// Single discriminant and it's a size range: the bounds set above
// already cover it — no <Prefix> (not even an empty one; nothing
// was requested) and no <And> (nothing else to combine with).
default:
// Either a single prefix or no discriminant at all (whole-bucket
// filter) — both are expressed as a <Prefix> element, empty or not.
f.Prefix = NewPrefix(prefix)
}
return f
}
func flattenFilter(f *Filter) (prefix string, tags map[string]string, sizeGT, sizeLT int64) {
if !f.set {
return
+323
View File
@@ -4,6 +4,9 @@ import (
"encoding/xml"
"strings"
"testing"
"time"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle"
)
func TestLifecycleXMLRoundTrip_NoncurrentVersionExpiration(t *testing.T) {
@@ -274,3 +277,323 @@ func TestLifecycleXMLRoundTrip_CompleteRule(t *testing.T) {
}
}
}
// assertCanonicalRoundTrip drives a canonical rule through
// CanonicalToLifecycle -> MarshalCanonical -> ParseCanonical and checks the
// result matches the input, proving the admin write path (which only ever
// has the canonical form) produces XML the S3 API can read back unchanged.
func assertCanonicalRoundTrip(t *testing.T, in *s3lifecycle.Rule) *s3lifecycle.Rule {
t.Helper()
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
out, err := ParseCanonical(xmlBytes)
if err != nil {
t.Fatalf("ParseCanonical(%s): %v", xmlBytes, err)
}
if len(out) != 1 {
t.Fatalf("expected 1 rule after round trip, got %d: %s", len(out), xmlBytes)
}
return out[0]
}
func TestCanonicalRoundTrip_WholeBucket(t *testing.T) {
in := &s3lifecycle.Rule{ID: "whole-bucket", Status: s3lifecycle.StatusEnabled, ExpirationDays: 30}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "" {
t.Errorf("expected empty prefix, got %q", out.Prefix)
}
if out.ExpirationDays != 30 {
t.Errorf("expected ExpirationDays=30, got %d", out.ExpirationDays)
}
}
func TestCanonicalRoundTrip_PrefixOnly(t *testing.T) {
in := &s3lifecycle.Rule{ID: "prefix-only", Status: s3lifecycle.StatusEnabled, Prefix: "logs/", ExpirationDays: 7}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "logs/" {
t.Errorf("expected prefix 'logs/', got %q", out.Prefix)
}
if len(out.FilterTags) != 0 {
t.Errorf("expected no tags, got %v", out.FilterTags)
}
}
func TestCanonicalRoundTrip_TagOnly(t *testing.T) {
in := &s3lifecycle.Rule{
ID: "tag-only",
Status: s3lifecycle.StatusEnabled,
FilterTags: map[string]string{"env": "dev"},
ExpirationDays: 7,
}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "" {
t.Errorf("expected empty prefix, got %q", out.Prefix)
}
if len(out.FilterTags) != 1 || out.FilterTags["env"] != "dev" {
t.Errorf("expected tags {env:dev}, got %v", out.FilterTags)
}
}
func TestCanonicalRoundTrip_PrefixAndTags(t *testing.T) {
in := &s3lifecycle.Rule{
ID: "prefix-and-tags",
Status: s3lifecycle.StatusEnabled,
Prefix: "logs/",
FilterTags: map[string]string{"env": "dev", "tier": "hot"},
ExpirationDays: 7,
}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "logs/" {
t.Errorf("expected prefix 'logs/', got %q", out.Prefix)
}
if len(out.FilterTags) != 2 || out.FilterTags["env"] != "dev" || out.FilterTags["tier"] != "hot" {
t.Errorf("expected tags {env:dev, tier:hot}, got %v", out.FilterTags)
}
}
func TestCanonicalRoundTrip_SizeBounds(t *testing.T) {
in := &s3lifecycle.Rule{
ID: "size-bounds",
Status: s3lifecycle.StatusEnabled,
FilterSizeGreaterThan: 512,
FilterSizeLessThan: 1048576,
ExpirationDays: 30,
}
out := assertCanonicalRoundTrip(t, in)
if out.FilterSizeGreaterThan != 512 {
t.Errorf("expected FilterSizeGreaterThan=512, got %d", out.FilterSizeGreaterThan)
}
if out.FilterSizeLessThan != 1048576 {
t.Errorf("expected FilterSizeLessThan=1048576, got %d", out.FilterSizeLessThan)
}
}
func TestCanonicalRoundTrip_PrefixAndTagsWithSizeBounds(t *testing.T) {
// Multiple discriminants (prefix + tags) force the <And> branch, which
// carries its own size bounds distinct from the single-branch Filter.
in := &s3lifecycle.Rule{
ID: "and-with-size",
Status: s3lifecycle.StatusEnabled,
Prefix: "logs/",
FilterTags: map[string]string{"env": "dev"},
FilterSizeGreaterThan: 1024,
FilterSizeLessThan: 2048,
ExpirationDays: 7,
}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "logs/" || len(out.FilterTags) != 1 {
t.Errorf("expected prefix 'logs/' and 1 tag, got prefix=%q tags=%v", out.Prefix, out.FilterTags)
}
if out.FilterSizeGreaterThan != 1024 || out.FilterSizeLessThan != 2048 {
t.Errorf("expected size bounds 1024/2048, got %d/%d", out.FilterSizeGreaterThan, out.FilterSizeLessThan)
}
}
func TestCanonicalRoundTrip_ExpirationDate(t *testing.T) {
date := time.Date(2030, 1, 15, 0, 0, 0, 0, time.UTC)
in := &s3lifecycle.Rule{ID: "expire-on-date", Status: s3lifecycle.StatusEnabled, ExpirationDate: date}
out := assertCanonicalRoundTrip(t, in)
if !out.ExpirationDate.Equal(date) {
t.Errorf("expected ExpirationDate=%v, got %v", date, out.ExpirationDate)
}
}
func TestCanonicalRoundTrip_ExpiredObjectDeleteMarker(t *testing.T) {
in := &s3lifecycle.Rule{ID: "delete-marker", Status: s3lifecycle.StatusEnabled, ExpiredObjectDeleteMarker: true}
out := assertCanonicalRoundTrip(t, in)
if !out.ExpiredObjectDeleteMarker {
t.Error("expected ExpiredObjectDeleteMarker=true")
}
}
func TestCanonicalRoundTrip_NoncurrentVersionExpiration(t *testing.T) {
in := &s3lifecycle.Rule{
ID: "noncurrent",
Status: s3lifecycle.StatusEnabled,
NoncurrentVersionExpirationDays: 30,
NewerNoncurrentVersions: 2,
}
out := assertCanonicalRoundTrip(t, in)
if out.NoncurrentVersionExpirationDays != 30 {
t.Errorf("expected NoncurrentVersionExpirationDays=30, got %d", out.NoncurrentVersionExpirationDays)
}
if out.NewerNoncurrentVersions != 2 {
t.Errorf("expected NewerNoncurrentVersions=2, got %d", out.NewerNoncurrentVersions)
}
}
func TestCanonicalRoundTrip_AbortMultipartUpload(t *testing.T) {
in := &s3lifecycle.Rule{ID: "abort-mpu", Status: s3lifecycle.StatusEnabled, AbortMPUDaysAfterInitiation: 7}
out := assertCanonicalRoundTrip(t, in)
if out.AbortMPUDaysAfterInitiation != 7 {
t.Errorf("expected AbortMPUDaysAfterInitiation=7, got %d", out.AbortMPUDaysAfterInitiation)
}
}
func TestCanonicalRoundTrip_Disabled(t *testing.T) {
in := &s3lifecycle.Rule{ID: "disabled-rule", Status: s3lifecycle.StatusDisabled, ExpirationDays: 30}
out := assertCanonicalRoundTrip(t, in)
if out.Status != s3lifecycle.StatusDisabled {
t.Errorf("expected Status=Disabled, got %q", out.Status)
}
}
func TestCanonicalRoundTrip_TagOrderIsStable(t *testing.T) {
// FilterTags is a map; And.Tags must be emitted in a deterministic
// (sorted) order so re-saving an unchanged rule doesn't churn the
// stored XML.
in := &s3lifecycle.Rule{
ID: "stable-order",
Status: s3lifecycle.StatusEnabled,
Prefix: "logs/",
FilterTags: map[string]string{"zeta": "1", "alpha": "2", "mu": "3"},
}
var firstXML []byte
for i := 0; i < 5; i++ {
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if i == 0 {
firstXML = xmlBytes
continue
}
if string(xmlBytes) != string(firstXML) {
t.Fatalf("marshal output is not stable across runs:\n%s\nvs\n%s", firstXML, xmlBytes)
}
}
if !strings.Contains(string(firstXML), "<Tag><Key>alpha</Key>") {
t.Errorf("expected tags sorted alphabetically (alpha first), got: %s", firstXML)
}
}
func TestCanonicalRoundTrip_PrefixWithSizeBoundsUsesAnd(t *testing.T) {
// A size range paired with a prefix is two different attributes, which
// requires <And> — unlike a size range alone (see
// TestCanonicalRoundTrip_SizeBounds), which doesn't need one.
in := &s3lifecycle.Rule{
ID: "prefix-with-size",
Status: s3lifecycle.StatusEnabled,
Prefix: "logs/",
FilterSizeGreaterThan: 1024,
ExpirationDays: 7,
}
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if !strings.Contains(string(xmlBytes), "<And>") {
t.Errorf("expected prefix+size to be wrapped in <And>, got: %s", xmlBytes)
}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "logs/" {
t.Errorf("expected prefix 'logs/', got %q", out.Prefix)
}
if out.FilterSizeGreaterThan != 1024 {
t.Errorf("expected FilterSizeGreaterThan=1024, got %d", out.FilterSizeGreaterThan)
}
}
func TestCanonicalRoundTrip_TagWithSizeBoundsUsesAnd(t *testing.T) {
in := &s3lifecycle.Rule{
ID: "tag-with-size",
Status: s3lifecycle.StatusEnabled,
FilterTags: map[string]string{"env": "dev"},
FilterSizeLessThan: 2048,
ExpirationDays: 7,
}
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if !strings.Contains(string(xmlBytes), "<And>") {
t.Errorf("expected tag+size to be wrapped in <And>, got: %s", xmlBytes)
}
out := assertCanonicalRoundTrip(t, in)
if len(out.FilterTags) != 1 || out.FilterTags["env"] != "dev" {
t.Errorf("expected tags {env:dev}, got %v", out.FilterTags)
}
if out.FilterSizeLessThan != 2048 {
t.Errorf("expected FilterSizeLessThan=2048, got %d", out.FilterSizeLessThan)
}
}
func TestCanonicalRoundTrip_SizeOnlyOmitsPrefixElement(t *testing.T) {
// A size-only filter must not stamp a spurious empty <Prefix> — nothing
// was requested, so nothing besides the size bounds should appear.
in := &s3lifecycle.Rule{
ID: "size-only-no-prefix",
Status: s3lifecycle.StatusEnabled,
FilterSizeGreaterThan: 512,
ExpirationDays: 7,
}
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if strings.Contains(string(xmlBytes), "<Prefix>") {
t.Errorf("expected no <Prefix> element for a size-only filter, got: %s", xmlBytes)
}
out := assertCanonicalRoundTrip(t, in)
if out.Prefix != "" {
t.Errorf("expected empty prefix, got %q", out.Prefix)
}
if out.FilterSizeGreaterThan != 512 {
t.Errorf("expected FilterSizeGreaterThan=512, got %d", out.FilterSizeGreaterThan)
}
}
func TestCanonicalRoundTrip_SizeRangeUsesAnd(t *testing.T) {
// <Filter> holds one predicate; a range is two, so it belongs under <And>,
// which is the form AWS documents.
in := &s3lifecycle.Rule{
ID: "size-range",
Status: s3lifecycle.StatusEnabled,
FilterSizeGreaterThan: 512,
FilterSizeLessThan: 1048576,
ExpirationDays: 30,
}
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{in})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if !strings.Contains(string(xmlBytes), "<And><ObjectSizeGreaterThan>512</ObjectSizeGreaterThan><ObjectSizeLessThan>1048576</ObjectSizeLessThan></And>") {
t.Errorf("expected both size bounds under <And>, got: %s", xmlBytes)
}
out := assertCanonicalRoundTrip(t, in)
if out.FilterSizeGreaterThan != 512 || out.FilterSizeLessThan != 1048576 {
t.Errorf("expected the size range to survive the round trip, got %+v", out)
}
}
func TestMarshalCanonical_CarriesTheS3Namespace(t *testing.T) {
// GetBucketLifecycleConfiguration replays these bytes verbatim, so a
// document written here has to look like one a client PUT.
xmlBytes, err := MarshalCanonical([]*s3lifecycle.Rule{{
ID: "ns",
Status: s3lifecycle.StatusEnabled,
ExpirationDays: 1,
}})
if err != nil {
t.Fatalf("MarshalCanonical: %v", err)
}
if !strings.Contains(string(xmlBytes), `<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">`) {
t.Errorf("expected the S3 namespace on the root element, got: %s", xmlBytes)
}
if _, err := ParseCanonical(xmlBytes); err != nil {
t.Errorf("the namespaced document must still parse: %v", err)
}
}
+5
View File
@@ -3,4 +3,9 @@ package s3_constants
const (
// DefaultBucketsPath is the default path for S3 buckets in the filer
DefaultBucketsPath = "/buckets"
// ObjectWriteRouteKeyPrefix namespaces a full path into the ring key used to
// resolve and forward writes to it. Every writer of an object or bucket
// entry hashes this same key, so they serialize on one filer's path lock.
ObjectWriteRouteKeyPrefix = "s3.object.write:"
)
+9 -56
View File
@@ -1132,37 +1132,13 @@ func (s3a *S3ApiServer) PutBucketLifecycleConfigurationHandler(w http.ResponseWr
// (volume server expires under the old rule) or contradict the new
// XML after a rule change. The add path is gone — this loop only
// shrinks the conf, never grows it.
fc, err := filer.ReadFilerConfFromFilers(s3a.option.Filers, s3a.option.GrpcDialOption, nil)
if err != nil {
glog.Errorf("PutBucketLifecycleConfigurationHandler read filer config: %s", err)
if err := s3a.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return filer.ClearBucketLifecycleDayTTLs(context.Background(), client, s3a.option.BucketsPath, bucket, s3a.getCollectionName(bucket))
}); err != nil {
glog.Errorf("PutBucketLifecycleConfigurationHandler clear legacy day-TTLs: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
return
}
collectionTtls := fc.GetCollectionTtls(s3a.getCollectionName(bucket))
changed := false
bucketPrefix := fmt.Sprintf("%s/%s/", s3a.option.BucketsPath, bucket)
for prefix, ttl := range collectionTtls {
if !strings.HasPrefix(prefix, bucketPrefix) || !strings.HasSuffix(ttl, "d") {
continue
}
fc.DeleteLocationConf(prefix)
changed = true
}
if changed {
var buf bytes.Buffer
if err := fc.ToText(&buf); err != nil {
glog.Errorf("PutBucketLifecycleConfigurationHandler save config to text: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
}
if err := s3a.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return filer.SaveInsideFiler(context.Background(), client, filer.DirectoryEtcSeaweedFS, filer.FilerConfName, buf.Bytes())
}); err != nil {
glog.Errorf("PutBucketLifecycleConfigurationHandler save config inside filer: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
return
}
}
if errCode := s3a.storeBucketLifecycleConfiguration(bucket, lifecycleXML, r.Header.Get(bucketLifecycleTransitionMinimumObjectSizeHeader)); errCode != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, errCode)
@@ -1184,37 +1160,14 @@ func (s3a *S3ApiServer) DeleteBucketLifecycleHandler(w http.ResponseWriter, r *h
return
}
fc, err := filer.ReadFilerConfFromFilers(s3a.option.Filers, s3a.option.GrpcDialOption, nil)
if err != nil {
glog.Errorf("DeleteBucketLifecycleHandler read filer config: %s", err)
// Same legacy day-TTL migration as the PUT handler.
if err := s3a.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return filer.ClearBucketLifecycleDayTTLs(context.Background(), client, s3a.option.BucketsPath, bucket, s3a.getCollectionName(bucket))
}); err != nil {
glog.Errorf("DeleteBucketLifecycleHandler clear legacy day-TTLs: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
return
}
collectionTtls := fc.GetCollectionTtls(s3a.getCollectionName(bucket))
changed := false
bucketPrefix := fmt.Sprintf("%s/%s/", s3a.option.BucketsPath, bucket)
for prefix, ttl := range collectionTtls {
if !strings.HasPrefix(prefix, bucketPrefix) || !strings.HasSuffix(ttl, "d") {
continue
}
fc.DeleteLocationConf(prefix)
changed = true
}
if changed {
var buf bytes.Buffer
if err := fc.ToText(&buf); err != nil {
glog.Errorf("DeleteBucketLifecycleHandler save config to text: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
}
if err := s3a.WithFilerClient(false, func(client filer_pb.SeaweedFilerClient) error {
return filer.SaveInsideFiler(context.Background(), client, filer.DirectoryEtcSeaweedFS, filer.FilerConfName, buf.Bytes())
}); err != nil {
glog.Errorf("DeleteBucketLifecycleHandler save config inside filer: %s", err)
s3err.WriteErrorResponse(w, r, s3err.ErrInternalError)
return
}
}
if errCode := s3a.clearStoredBucketLifecycleConfiguration(bucket); errCode != s3err.ErrNone {
s3err.WriteErrorResponse(w, r, errCode)
+7 -3
View File
@@ -4,14 +4,18 @@ import (
"strings"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3err"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/scheduler"
)
const (
bucketLifecycleConfigurationXMLKey = "s3-bucket-lifecycle-configuration-xml"
bucketLifecycleTransitionMinimumObjectSizeKey = "s3-bucket-lifecycle-transition-default-minimum-object-size"
// The stored form is defined in the scheduler package, which callers
// outside the gateway also read; these are local shorthands for it.
bucketLifecycleConfigurationXMLKey = scheduler.BucketLifecycleConfigurationXMLKey
bucketLifecycleTransitionMinimumObjectSizeKey = scheduler.BucketLifecycleTransitionMinimumObjectSizeKey
maxBucketLifecycleConfigurationSize = scheduler.MaxBucketLifecycleConfigurationSize
bucketLifecycleTransitionMinimumObjectSizeHeader = "X-Amz-Transition-Default-Minimum-Object-Size"
defaultLifecycleTransitionMinimumObjectSize = "all_storage_classes_128K"
maxBucketLifecycleConfigurationSize = 1 << 20
)
func normalizeBucketLifecycleTransitionMinimumObjectSize(value string) string {
+2 -2
View File
@@ -17,8 +17,8 @@ import (
// objectWriteRouteKeyPrefix namespaces an object's full path into the ring key
// used to resolve and forward its writes. Shared by every routed builder so the
// gateway and filer hash the same key.
const objectWriteRouteKeyPrefix = "s3.object.write:"
// gateway, the admin dashboard and the filer hash the same key.
const objectWriteRouteKeyPrefix = s3_constants.ObjectWriteRouteKeyPrefix
// objectRouteKey is the ring key the gateway hashes to resolve an object's owner
// filer. It is also sent as route_key on each routed transaction, so a non-owner
+12 -1
View File
@@ -11,7 +11,18 @@ import (
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle/engine"
)
const BucketLifecycleConfigurationXMLKey = "s3-bucket-lifecycle-configuration-xml"
// The bucket lifecycle configuration lives here rather than in weed/s3api so
// callers outside the gateway (the admin dashboard, the scheduler itself) and
// the gateway share one definition of the extended-attribute keys and the
// size cap. weed/s3api aliases these; nothing redeclares the values.
const (
BucketLifecycleConfigurationXMLKey = "s3-bucket-lifecycle-configuration-xml"
BucketLifecycleTransitionMinimumObjectSizeKey = "s3-bucket-lifecycle-transition-default-minimum-object-size"
// MaxBucketLifecycleConfigurationSize caps the serialized XML, mirroring
// the S3 API's limit.
MaxBucketLifecycleConfigurationSize = 1 << 20
)
// ParseError is returned alongside successfully-loaded inputs so callers can
// surface malformed bucket configs rather than silently dropping them.