Add S3 Lifecycle wiki page

New page documenting all supported lifecycle features: - Expiration (Days, Date), NoncurrentVersionExpiration, ExpiredObjectDeleteMarker, AbortIncompleteMultipartUpload - Filter support (Prefix, Tag, And, ObjectSize) - Two-tier architecture (TTL fast path + worker scan) - Terraform and AWS CLI examples - Worker configuration reference - Versioning integration notes Update S3 API table from "Partial" to "Yes" for lifecycle endpoints. Add S3 Lifecycle link to sidebar under AWS S3 API section.
2026-10-09 07:47:54 +02:00 · 2026-03-28 14:01:48 -07:00
1 parent 3e45126d00
commit c66632f27b
@@ -65,9 +65,9 @@ For IAM action names used in bucket policies (e.g., `s3:ListBucketMultipartUploa
 | GetBucketEncryption | Yes | See [[Server-Side Encryption]] |
 | PutBucketEncryption | Yes | |
 | DeleteBucketEncryption | Yes | |
-| GetBucketLifecycleConfiguration | Partial | TTL-based expiration only |
-| PutBucketLifecycleConfiguration | Partial | TTL-based expiration only |
-| DeleteBucketLifecycle | Partial | TTL-based expiration only |
+| GetBucketLifecycleConfiguration | Yes | See [[S3 Lifecycle]] |
+| PutBucketLifecycleConfiguration | Yes | Transition rules not supported |
+| DeleteBucketLifecycle | Yes | |
 | GetBucketLocation | Yes | Returns default location |
 | GetBucketOwnershipControls | Yes | |
 | PutBucketOwnershipControls | Yes | |
@@ -0,0 +1,146 @@
+# S3 Lifecycle Configuration
+
+SeaweedFS supports S3 bucket lifecycle configuration for automated object expiration, non-current version cleanup, delete marker removal, and incomplete multipart upload abortion.
+
+## Supported Features
+
+| Feature | Status | Notes |
+|---|---|---|
+| `Expiration.Days` | Supported | Fast path via TTL + RocksDB compaction filter |
+| `Expiration.Date` | Supported | Evaluated by lifecycle worker at scan time |
+| `ExpiredObjectDeleteMarker` | Supported | Removes delete markers that are the sole remaining version |
+| `NoncurrentVersionExpiration.NoncurrentDays` | Supported | Requires bucket versioning enabled |
+| `NoncurrentVersionExpiration.NewerNoncurrentVersions` | Supported | Keep N newest non-current versions |
+| `AbortIncompleteMultipartUpload.DaysAfterInitiation` | Supported | Per-rule prefix scoping |
+| `Filter.Prefix` | Supported | |
+| `Filter.Tag` | Supported | Evaluated at scan time |
+| `Filter.And` (Prefix + Tags + Size) | Supported | Evaluated at scan time |
+| `Filter.ObjectSizeGreaterThan` | Supported | Evaluated at scan time |
+| `Filter.ObjectSizeLessThan` | Supported | Evaluated at scan time |
+| `Transition` | Not supported | Requires storage class tiers |
+| `NoncurrentVersionTransition` | Not supported | Requires storage class tiers |
+
+## API Endpoints
+
+```
+PUT    /{bucket}?lifecycle   # PutBucketLifecycleConfiguration
+GET    /{bucket}?lifecycle   # GetBucketLifecycleConfiguration
+DELETE /{bucket}?lifecycle   # DeleteBucketLifecycle
+```
+
+## Example: Terraform
+
+```hcl
+resource "aws_s3_bucket_lifecycle_configuration" "example" {
+  bucket = aws_s3_bucket.example.id
+
+  rule {
+    id     = "expire-logs"
+    status = "Enabled"
+
+    filter {
+      prefix = "logs/"
+    }
+
+    expiration {
+      days = 30
+    }
+
+    noncurrent_version_expiration {
+      noncurrent_days          = 7
+      newer_noncurrent_versions = 2
+    }
+
+    abort_incomplete_multipart_upload {
+      days_after_initiation = 7
+    }
+  }
+}
+```
+
+## Example: AWS CLI
+
+```bash
+# Set lifecycle configuration
+aws s3api put-bucket-lifecycle-configuration \
+  --endpoint-url http://localhost:8333 \
+  --bucket my-bucket \
+  --lifecycle-configuration '{
+    "Rules": [
+      {
+        "ID": "expire-old",
+        "Status": "Enabled",
+        "Filter": { "Prefix": "" },
+        "Expiration": { "Days": 90 },
+        "NoncurrentVersionExpiration": {
+          "NoncurrentDays": 30
+        },
+        "AbortIncompleteMultipartUpload": {
+          "DaysAfterInitiation": 7
+        }
+      }
+    ]
+  }'
+
+# Get lifecycle configuration
+aws s3api get-bucket-lifecycle-configuration \
+  --endpoint-url http://localhost:8333 \
+  --bucket my-bucket
+
+# Delete lifecycle configuration
+aws s3api delete-bucket-lifecycle \
+  --endpoint-url http://localhost:8333 \
+  --bucket my-bucket
+```
+
+## How It Works
+
+### Two-Tier Expiration Architecture
+
+**Tier 1 — TTL fast path:** Simple `Expiration.Days` rules with prefix-only filters are translated into TTL entries in `filer.conf`. The RocksDB compaction filter automatically removes expired entries during normal compaction at zero additional cost. This is the most efficient path for the common case.
+
+**Tier 2 — Scan-time evaluation:** Rules with tag filters, size filters, date-based expiration, non-current version expiration, and delete marker cleanup are evaluated by the lifecycle plugin worker. The worker periodically scans buckets and evaluates each object against the stored lifecycle XML configuration.
+
+### Which rules use which tier?
+
+| Rule Type | Tier | Why |
+|---|---|---|
+| `Expiration.Days` (prefix only) | TTL fast path | Can be expressed as per-entry TTL |
+| `Expiration.Days` (with tags/size) | Worker scan | TTL can't express tag/size constraints |
+| `Expiration.Date` | Worker scan | Absolute date, not relative TTL |
+| `NoncurrentVersionExpiration` | Worker scan | Requires version enumeration |
+| `ExpiredObjectDeleteMarker` | Worker scan | Requires version counting |
+| `AbortIncompleteMultipartUpload` | Worker scan | Scans `.uploads` directory |
+
+### Lifecycle Worker
+
+The lifecycle plugin worker runs as part of the SeaweedFS plugin system. It:
+
+1. **Detects** buckets with lifecycle rules (from stored lifecycle XML or filer.conf TTLs)
+2. **Scans** bucket contents and evaluates lifecycle rules against each object
+3. **Executes** actions: delete expired objects, remove old non-current versions, clean up delete markers, abort stale multipart uploads
+
+Worker configuration options:
+
+| Setting | Default | Description |
+|---|---|---|
+| `batch_size` | 1000 | Entries per filer listing page |
+| `max_deletes_per_bucket` | 10000 | Max expired objects to delete per run |
+| `dry_run` | false | Detect but don't delete |
+| `delete_marker_cleanup` | true | Remove expired delete markers |
+| `abort_mpu_days` | 7 | Fallback for buckets without lifecycle XML MPU rules |
+| `bucket_filter` | (all) | Wildcard pattern to scope lifecycle to specific buckets |
+
+## Versioning Integration
+
+Lifecycle rules interact with [S3 Object Versioning](S3-Object-Versioning):
+
+- **`NoncurrentVersionExpiration`** only applies to versioned buckets. Non-current versions are deleted after `NoncurrentDays` days since they were superseded. `NewerNoncurrentVersions` retains the N newest non-current versions.
+- **`ExpiredObjectDeleteMarker`** removes delete markers that are the sole remaining version of an object (no non-current versions behind them).
+- **`Expiration.Days`** on a versioned bucket creates a delete marker when the current version expires; it does not permanently delete the object.
+
+## Limitations
+
+- **Transition rules** (`Transition`, `NoncurrentVersionTransition`) are not supported. SeaweedFS does not have S3-equivalent storage class tiers.
+- **Detection interval**: The lifecycle worker scans on a configurable interval (default 5 minutes). Objects may persist slightly beyond their configured expiration until the next scan.
+- **TTL fast path** applies only to `Expiration.Days` rules with prefix-only filters. Rules with tag or size constraints are always evaluated at scan time.
@@ -74,6 +74,7 @@
 ### AWS S3 API
 * [[Amazon S3 API]]
 * [[Supported APIs vs Minio]]
+* [[S3 Lifecycle]]
 * [[S3 Conditional Operations]]
 * [[S3 CORS]]
 * [[S3 Object Lock and Retention]]