Table of Contents
S3 Lifecycle — Recipes
Worked, end-to-end configurations for the common lifecycle scenarios, each with how to apply it and how to confirm it ran. For the feature reference and validation rules see S3 Lifecycle; for config knobs see the Operator Guide.
All examples use the AWS CLI against the S3 endpoint. Set it once:
export S3_ENDPOINT=http://localhost:8333
Each rule needs a Status and a Filter (use {} to match the whole bucket). The CLI accepts the JSON shown here; the bytes on the wire are the equivalent LifecycleConfiguration XML.
Verifying any rule
Two independent signals tell you a rule worked:
-
The dispatch counter advances. Each delete increments
SeaweedFS_s3_lifecycle_dispatch_total{bucket,kind,outcome}. A successful expiry shows up asoutcome="DONE". Thekindlabel is one of:kindSet by expiration_daysExpiration.Daysexpiration_dateExpiration.Datenoncurrent_daysNoncurrentVersionExpiration.NoncurrentDaysnewer_noncurrentNoncurrentVersionExpiration.NewerNoncurrentVersions(stand-alone)abort_mpuAbortIncompleteMultipartUploadexpired_delete_markerExpiration.ExpiredObjectDeleteMarkersum by (kind, outcome) (SeaweedFS_s3_lifecycle_dispatch_total{bucket="my-bucket"}) -
The object is gone.
aws s3api head-objectreturns404, orlist-object-versionsno longer shows the version.
Timing. Replay-eligible rules (Expiration.Days, NoncurrentDays, AbortIncompleteMultipartUpload) fire on the next worker pass. Version-list-aware rules (Expiration.Date, NewerNoncurrentVersions, ExpiredObjectDeleteMarker) fire on the next walker pass — additionally up to one walker_interval_minutes window. With the default daily schedule, budget up to 24h. To check a rule immediately without waiting, drive the worker by hand with s3.lifecycle.run-shard (see the Operator Guide).
Expire objects under a prefix
Delete everything under logs/ 30 days after it was written.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "expire-logs-30d",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Expiration": { "Days": 30 }
}]
}'
Replay path (kind="expiration_days"). The 30-day clock starts at the object's latest-version PUT. On a versioned bucket this writes a delete marker rather than removing data — pair it with the cleanup recipes below.
Verify:
aws --endpoint-url "$S3_ENDPOINT" s3api head-object --bucket my-bucket --key logs/old.log
# expected: An error occurred (404)
Abort incomplete multipart uploads
Reclaim parts from uploads that were started but never completed, 7 days after initiation.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "abort-stuck-mpu",
"Status": "Enabled",
"Filter": {},
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}]
}'
Replay path (kind="abort_mpu"). The clock starts at the CreateMultipartUpload time. Safe to run on every bucket — it only touches in-flight uploads, never completed objects.
Verify:
aws --endpoint-url "$S3_ENDPOINT" s3api list-multipart-uploads --bucket my-bucket
# expected: no Uploads older than 7 days
Expire objects by tag
Delete objects tagged temp=true, one day after write, regardless of prefix.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "expire-temp-tagged",
"Status": "Enabled",
"Filter": { "Tag": { "Key": "temp", "Value": "true" } },
"Expiration": { "Days": 1 }
}]
}'
Replay path. The tag is read from the live object at evaluation time, so re-tagging an object changes whether it matches. (This mutability is also why the TTL fast path refuses to stamp tag-filtered rules.)
Expire only large objects
Combine predicates with And. Here: delete objects under tmp/ that are larger than 5 MiB.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "expire-large-tmp",
"Status": "Enabled",
"Filter": {
"And": {
"Prefix": "tmp/",
"ObjectSizeGreaterThan": 5242880
}
},
"Expiration": { "Days": 7 }
}]
}'
And is required whenever a filter has more than one predicate. Size bounds are strict: ObjectSizeGreaterThan matches objects strictly larger than the value, ObjectSizeLessThan strictly smaller. Set both to target a size band.
Versioned bucket: keep the N newest versions
On a versioned bucket, retain the 5 most recent noncurrent versions and expire older noncurrent versions 30 days after they were superseded.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "prune-noncurrent",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"NoncurrentVersionExpiration": {
"NoncurrentDays": 30,
"NewerNoncurrentVersions": 5
}
}]
}'
A noncurrent version is removed only when both conditions hold: it is older than NoncurrentDays and there are at least NewerNoncurrentVersions newer noncurrent versions ahead of it. The five newest noncurrent versions are always kept, however old they get; everything behind them ages out at 30 days. The noncurrent clock starts at the PUT that demoted the version, not the version's own mtime.
Because the retain-N cap needs the full version list, this combination is evaluated on the walker pass, so it also waits up to one walker_interval_minutes window.
Use NoncurrentDays alone for a pure age policy, or NewerNoncurrentVersions alone for a pure count cap (keep exactly the N newest, no age requirement).
Verify:
aws --endpoint-url "$S3_ENDPOINT" s3api list-object-versions --bucket my-bucket --prefix my-key
# expected: at most 5 noncurrent versions remain
Versioned bucket: clean up fully after deletes
A DELETE on a versioned bucket leaves a delete marker, and previous versions remain as noncurrent. To reclaim everything for deleted objects, use two rules — one to expire the old versions, one to remove the leftover delete marker once it is the only thing left:
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [
{
"ID": "expire-noncurrent",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
},
{
"ID": "expire-orphan-delete-markers",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
]
}'
ExpiredObjectDeleteMarker removes a delete marker only when it is the sole remaining version of the key (no noncurrent versions behind it). With the first rule clearing the noncurrent versions, the second eventually finds each delete marker orphaned and removes it. Both run on the walker pass.
One-time cleanup at a fixed date
Delete everything under archive/2024/ on a specific date, rather than relative to each object's age.
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [{
"ID": "drop-2024-archive",
"Status": "Enabled",
"Filter": { "Prefix": "archive/2024/" },
"Expiration": { "Date": "2026-07-01T00:00:00Z" }
}]
}'
Walker path (kind="expiration_date"). The rule fires on the first walker pass at or after the date; a date already in the past triggers on the next walk. Remove the rule afterward — it stays active and would expire anything later written under the prefix.
A combined production config
Rules are independent and evaluated together, so one document can carry the whole policy for a bucket:
aws --endpoint-url "$S3_ENDPOINT" s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration '{
"Rules": [
{
"ID": "expire-logs-30d",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Expiration": { "Days": 30 }
},
{
"ID": "abort-stuck-mpu",
"Status": "Enabled",
"Filter": {},
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
},
{
"ID": "prune-noncurrent",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 5 }
},
{
"ID": "expire-orphan-delete-markers",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
]
}'
GET returns the stored document verbatim. To temporarily disable a single rule without deleting it, set its Status to Disabled and re-PUT; the worker reads the change on its next pass.
See also
S3 Lifecycle · S3 Lifecycle Operator Guide · S3 Lifecycle Monitoring · S3 Lifecycle Troubleshooting · S3 Lifecycle vs Volume TTL · S3 Object Versioning
Introduction
- Quick Start with weed mini
- Simplest S3 Bucket and User Setup
- Components
- Blob Store Architecture
- Getting Started
- Production Setup
- A typical step‐by‐step example
- Benchmarks
- FAQ
- Applications
API
Configuration
- Replication
- Store file with a Time To Live
- Failover Master Server
- Erasure coding for warm storage
- EC Bitrot Detection
- Server Startup via Systemd
- Environment Variables
Filer
- Filer Setup
- Directories and Files
- File Operations Quick Reference
- Data Structure for Large Files
- Filer Data Encryption
- Filer Commands and Operations
- Filer JWT Use
- TUS Resumable Uploads
Filer Stores
- Filer Cassandra Setup
- Filer Redis Setup
- Super Large Directories
- Path-Specific Filer Store
- Choosing a Filer Store
- Customize Filer Store
Management
Advanced Filer Configurations
- Migrate to Filer Store
- Add New Filer Store
- Filer Store Replication
- Filer Active Active cross cluster continuous synchronization
- Filer as a Key-Large-Value Store
- Path Specific Configuration
- Filer Change Data Capture
- Filer Operation Serialization
FUSE Mount
- Mount on Windows
- FIO benchmark
- fstab and systemd mount
- POSIX Compliance
- Distributed POSIX Locks
- P2P reading in weed mount
- Mount over the Internet
WebDAV
SFTP Server
Cloud Drive
- Cloud Drive Benefits
- Cloud Drive Architecture
- Configure Remote Storage
- Azure Blob Storage Authentication
- Mount Remote Storage
- Cache Remote Storage
- Cloud Drive Quick Setup
- Gateway to Remote Object Storage
AWS S3 API
- Amazon S3 API
- Supported APIs vs Minio
- S3 Lifecycle
- S3 Lifecycle vs Volume TTL
- S3 Conditional Operations
- S3 CORS
- S3 Object Lock and Retention
- S3 Object Versioning
- S3 RenameObject
- S3 API Benchmark
- S3 API FAQ
- S3 Bucket Quota
- S3 Rate Limiting
- S3 API Audit log
- S3 Nginx Proxy
- Docker Compose for S3
S3 Table Bucket
- S3 Table Bucket
- S3 Table Bucket Commands
- S3 Tables Security
- SeaweedFS Iceberg Catalog
- Iceberg REST Catalog API
- Iceberg Table Maintenance
- SeaweedFS Lance Catalog
- Lance Maintenance Worker
Iceberg Integrations
- Spark Iceberg Integration
- Trino Iceberg Integration
- Dremio Iceberg Integration
- DuckDB Iceberg Integration
- Doris Iceberg Integration
- RisingWave Iceberg Integration
- Lakekeeper Iceberg Integration
Lance Integrations
S3 Authentication & IAM
- S3 Configuration - Start Here
- S3 Credentials (
-s3.config) - OIDC Integration (
-s3.iam.config) - Kubernetes ServiceAccount Authentication (IRSA-style)
- S3 Policy Variables
- S3 Policy Conditions
- S3 Bucket Policies
- Amazon IAM API
- AWS IAM CLI
- weed shell - Shell IAM Commands
Server-Side Encryption
S3 Client Tools
- AWS CLI with SeaweedFS
- s3cmd with SeaweedFS
- rclone with SeaweedFS
- restic with SeaweedFS
- nodejs with Seaweed S3
Machine Learning
HDFS
- Hadoop Compatible File System
- run Spark on SeaweedFS
- run HBase on SeaweedFS
- Run Trino on SeaweedFS
- Hadoop Benchmark
- HDFS via S3 connector
Replication and Backup
- Async Replication to another Filer [Deprecated]
- Async Backup
- Async Filer Metadata Backup
- Async Replication to Cloud [Deprecated]
- Kubernetes Backups and Recovery with K8up
Metadata Change Events
Messaging
- Structured Data Lake with SMQ and SQL
- Seaweed Message Queue
- SQL Queries on Message Queue
- SQL Quick Reference
- PostgreSQL-compatible Server weed db
- Pub-Sub to SMQ to SQL
- Kafka to Kafka Gateway to SMQ to SQL
Use Cases
Operations
- System Metrics
- weed shell
- Data Backup
- Deployment to Kubernetes and Minikube
- Helm Chart Recipes
- Deployment with seaweed-up
Rust Volume Server
Advanced
- Large File Handling
- Optimization
- Optimization for Many Small Buckets
- Volume Management
- Tiered Storage
- Cloud Tier
- Cloud Monitoring
- Load Command Line Options from a file
- SRV Service Discovery
- Volume Files Structure
Security
- Security Overview
- Security Configuration
- Cryptography and FIPS Compliance
- Run Blob Storage on Public Internet