3

Helm Chart Recipes

Chris Lu edited this page 2026-09-03 00:14:53 -07:00

Table of Contents

• Three-node S3 cluster
• Single node for development
• Lakehouse: S3 Tables with the Iceberg REST catalog
• Filer metadata on PostgreSQL
• Bare metal: node-local disks
• Cross-cluster replication

Copy-paste values.yaml files for the shapes people ask about most. Every recipe renders against the current chart. Install any of them with:

helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm
helm install seaweedfs seaweedfs/seaweedfs -n seaweedfs --create-namespace -f values.yaml

The chart README in k8s/charts/seaweedfs lists every key. For the operator instead of the chart, see Deployment to Kubernetes and Minikube.

Three-node S3 cluster

Three masters for Raft failover, three volume servers, two filers, two S3 gateways, every write stored twice, and a bucket ready to use.

global:
  seaweedfs:
    enableReplication: true
    replicationPlacement: "001"   # one extra copy on another server; "002" for two

master:
  replicas: 3
  data:
    type: persistentVolumeClaim   # the cluster's default storage class; add storageClass to pick one
    size: 1Gi

volume:
  replicas: 3                     # at least 1 + the sum of the replication digits
  dataDirs:
    - name: data
      type: persistentVolumeClaim
      size: 500Gi
      maxVolumes: 0               # size the volume count from the disk

filer:
  replicas: 2
  data:
    type: persistentVolumeClaim
    size: 20Gi

s3:
  enabled: true
  replicas: 2
  enableAuth: true
  credentials:
    admin:
      accessKey: admin
      secretKey: change-me
  createBuckets:
    - name: app-storage

How the pieces fit:

• replicationPlacement is three digits, XYZ: X extra copies in other data centers, Y in other racks of the same data center, Z on other servers of the same rack. Total copies are 1 + X + Y + Z, and you need at least that many volume servers in the matching places. Without rack or data center labels every volume server is in the same rack, so 001 needs two volume servers and 002 needs three. enableReplication sets both master.defaultReplication and filer.defaultReplicaPlacement to this value. See Replication.
• volume.replicas is the number of volume server pods, one per node with the chart's default anti-affinity. It is independent of the replication digits except for the minimum above.
• maxVolumes: 0 lets each volume server compute how many volumes fit on its disk. The chart's master defaults to 1000MB volumes (master.volumeSizeLimitMB); raise it to 30000 for large disks with few buckets, keep it small when you expect many buckets, since each bucket grows its own volumes.
• Master and filer data on claims keep the cluster identity and the metadata when a pod is rescheduled. The chart defaults both to hostPath, which does not follow the pod. The older filer.enablePVC key is ignored once filer.data is set, so use filer.data.
• Claims use the cluster's default storage class. Add storageClass under a data block or a dataDirs entry to choose another, for example local-path on k3s.
• Two filers on the default LevelDB store sync with each other. For three or more filers, or a lot of metadata, put the store in a database (next recipes).
• s3.enableAuth with s3.credentials writes the identities into the S3 config secret; the read identity gets random keys unless you set it too. createBuckets runs once as a post-install hook.

The endpoint inside the cluster is seaweedfs-s3.seaweedfs.svc:8333. From a laptop:

kubectl -n seaweedfs port-forward svc/seaweedfs-s3 8333:8333
AWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=change-me \
  aws --endpoint-url http://localhost:8333 s3 ls s3://app-storage/

Expose it with s3.ingress or a LoadBalancer under s3.service.type. Grow the cluster by raising volume.replicas; new volume servers take new writes and nothing moves until you run volume.balance in weed shell.

Single node for development

One of everything, no authentication.

master:
  data:
    type: persistentVolumeClaim
    size: 1Gi

volume:
  dataDirs:
    - name: data
      type: persistentVolumeClaim
      size: 20Gi
      maxVolumes: 0

filer:
  data:
    type: persistentVolumeClaim
    size: 5Gi

s3:
  enabled: true
  createBuckets:
    - name: test

For a laptop without Kubernetes, weed mini or the Docker image does the same in one process, see Quick Start with weed mini.

Lakehouse: S3 Tables with the Iceberg REST catalog

The three-node recipe plus the catalog port on the S3 gateway. Table buckets are created through the S3 Tables API rather than createBuckets.

global:
  seaweedfs:
    enableReplication: true
    replicationPlacement: "001"

master:
  replicas: 3
  data:
    type: persistentVolumeClaim   # the cluster's default storage class; add storageClass to pick one
    size: 1Gi

volume:
  replicas: 3
  dataDirs:
    - name: data
      type: persistentVolumeClaim
      size: 500Gi
      maxVolumes: 0

filer:
  replicas: 2
  data:
    type: persistentVolumeClaim
    size: 20Gi

s3:
  enabled: true
  replicas: 2
  enableAuth: true
  icebergPort: 8181
  credentials:
    admin:
      accessKey: admin
      secretKey: change-me

Create a table bucket, then point engines at the S3 endpoint and the catalog:

export S3_ENDPOINT=http://seaweedfs-s3.seaweedfs.svc:8333
aws s3tables --region us-east-1 --endpoint-url $S3_ENDPOINT create-table-bucket --name warehouse

The Iceberg REST catalog is http://seaweedfs-s3.seaweedfs.svc:8181, or set s3.icebergIngress to reach it from outside. Engine setup is in Spark Iceberg Integration, Trino Iceberg Integration, DuckDB Iceberg Integration, and the other pages under S3 Table Bucket. s3.lancePort (default 9101) serves the SeaweedFS Lance Catalog the same way.

Filer metadata on PostgreSQL

For three or more filers, or when metadata outgrows a local LevelDB, keep it in a database the filers share. Any store from Filer Stores works the same way through WEED_<STORE>_* environment variables; this is postgres2.

filer:
  replicas: 3
  extraEnvironmentVars:
    WEED_LEVELDB2_ENABLED: "false"
    WEED_POSTGRES2_ENABLED: "true"
    WEED_POSTGRES2_HOSTNAME: "postgres.database.svc"
    WEED_POSTGRES2_PORT: "5432"
    WEED_POSTGRES2_DATABASE: "seaweedfs"
    WEED_POSTGRES2_SSLMODE: "disable"
  secretExtraEnvironmentVars:
    WEED_POSTGRES2_USERNAME:
      secretKeyRef:
        name: postgres-credentials
        key: username
    WEED_POSTGRES2_PASSWORD:
      secretKeyRef:
        name: postgres-credentials
        key: password

s3:
  enabled: true
  replicas: 3

The filer creates its table on first start. WEED_LEVELDB2_ENABLED must be turned off, since the chart enables it by default and two enabled stores is an error. The filers no longer need a claim, so filer.data can stay at its default.

Bare metal: node-local disks

Skip the storage class and write straight to each node's disks. One volume server per node, every listed directory becomes a -dir on that pod.

volume:
  replicas: 3
  dataDirs:
    - name: nvme0
      type: hostPath
      hostPathPrefix: /mnt/nvme0
      maxVolumes: 0
    - name: hdd0
      type: hostPath
      hostPathPrefix: /mnt/hdd0
      maxVolumes: 0

The chart mounts <hostPathPrefix>/object_store/ for each entry. Pin the pods with volume.nodeSelector if only some nodes have the disks. To steer buckets between disk types, tag the directories in the same order with volume.extraArgs: ["-disk", "ssd,hdd"], see Tiered Storage. Master and filer data default to hostPath too; give them claims as in the recipes above, or accept that they stay on the node they started on.

Cross-cluster replication

Two clusters from any recipe above can replicate to each other continuously with weed filer.sync, active-active or active-passive, from any pod or host that reaches both filers. The command, the operator sidecar form, and the topology rules are in Filer Active Active cross cluster continuous synchronization.

Introduction

• Quick Start with weed mini
• Simplest S3 Bucket and User Setup
• Components
• Blob Store Architecture
• Getting Started
• Production Setup
• A typical step‐by‐step example
• Benchmarks
• FAQ
• Applications

API

• Master Server API
• Volume Server API
• Filer Server API
• Client Libraries
• SeaweedFS Java Client

Configuration

• Replication
• Store file with a Time To Live
• Failover Master Server
• Erasure coding for warm storage
• EC Bitrot Detection
• Server Startup via Systemd
• Environment Variables

Filer

• Filer Setup
• Directories and Files
• File Operations Quick Reference
• Data Structure for Large Files
• Filer Data Encryption
• Filer Commands and Operations
• Filer JWT Use
• TUS Resumable Uploads

Filer Stores

• Filer Cassandra Setup
• Filer Redis Setup
• Super Large Directories
• Path-Specific Filer Store
• Choosing a Filer Store
• Customize Filer Store

Management

• Admin UI
• Admin UI OIDC
• Worker
• Plugin Worker Scheduling

Advanced Filer Configurations

• Migrate to Filer Store
• Add New Filer Store
• Filer Store Replication
• Filer Active Active cross cluster continuous synchronization
• Filer as a Key-Large-Value Store
• Path Specific Configuration
• Filer Change Data Capture
• Filer Operation Serialization

FUSE Mount

• Mount on Windows
• FIO benchmark
• fstab and systemd mount
• POSIX Compliance
• Distributed POSIX Locks
• P2P reading in weed mount
• Mount over the Internet

WebDAV

SFTP Server

Cloud Drive

• Cloud Drive Benefits
• Cloud Drive Architecture
• Configure Remote Storage
• Azure Blob Storage Authentication
• Mount Remote Storage
• Cache Remote Storage
• Cloud Drive Quick Setup
• Gateway to Remote Object Storage

AWS S3 API

• Amazon S3 API
• Supported APIs vs Minio
• S3 Lifecycle
  • S3 Lifecycle Recipes
  • S3 Lifecycle Operator Guide
  • S3 Lifecycle Monitoring
  • S3 Lifecycle Troubleshooting
  • S3 Lifecycle Architecture
• S3 Lifecycle vs Volume TTL
• S3 Conditional Operations
• S3 CORS
• S3 Object Lock and Retention
• S3 Object Versioning
• S3 RenameObject
• S3 API Benchmark
• S3 API FAQ
• S3 Bucket Quota
• S3 Rate Limiting
• S3 API Audit log
• S3 Nginx Proxy
• Docker Compose for S3

S3 Table Bucket

• S3 Table Bucket
• S3 Table Bucket Commands
• S3 Tables Security
• SeaweedFS Iceberg Catalog
• Iceberg REST Catalog API
• Iceberg Table Maintenance
• SeaweedFS Lance Catalog
• Lance Maintenance Worker

Iceberg Integrations

• Spark Iceberg Integration
• Trino Iceberg Integration
• Dremio Iceberg Integration
• DuckDB Iceberg Integration
• Doris Iceberg Integration
• RisingWave Iceberg Integration
• Lakekeeper Iceberg Integration

Lance Integrations

• LanceDB Integration
• Spark Lance Integration
• DuckDB Lance Integration

S3 Authentication & IAM

• S3 Configuration - Start Here
• S3 Credentials (-s3.config)
• OIDC Integration (-s3.iam.config)
• Kubernetes ServiceAccount Authentication (IRSA-style)
• S3 Policy Variables
• S3 Policy Conditions
• S3 Bucket Policies
• Amazon IAM API
• AWS IAM CLI
• weed shell - Shell IAM Commands

Server-Side Encryption

• Server-Side Encryption
• Server-Side Encryption SSE-KMS
• Server-Side Encryption SSE-C

S3 Client Tools

• AWS CLI with SeaweedFS
• s3cmd with SeaweedFS
• rclone with SeaweedFS
• restic with SeaweedFS
• nodejs with Seaweed S3

Machine Learning

• TensorFlow with SeaweedFS
• Distributing AI Model Files for Multi-GPU Loading

HDFS

• Hadoop Compatible File System
• run Spark on SeaweedFS
• run HBase on SeaweedFS
• Run Trino on SeaweedFS
• Hadoop Benchmark
• HDFS via S3 connector

Replication and Backup

• Async Replication to another Filer [Deprecated]
• Async Backup
• Async Filer Metadata Backup
• Async Replication to Cloud [Deprecated]
• Kubernetes Backups and Recovery with K8up

Metadata Change Events

• Filer Metadata Events
• Filer Notification Webhook

Messaging

• Structured Data Lake with SMQ and SQL
• Seaweed Message Queue
• SQL Queries on Message Queue
• SQL Quick Reference
• PostgreSQL-compatible Server weed db
• Pub-Sub to SMQ to SQL
• Kafka to Kafka Gateway to SMQ to SQL

Use Cases

• Use Cases
• Actual Users

Operations

• System Metrics
• weed shell
• Data Backup
• Deployment to Kubernetes and Minikube
• Helm Chart Recipes
• Deployment with seaweed-up
  • seaweed-up Command Reference
  • Cluster Plan Workflow
  • Cluster Plan Inventory Reference
  • Cluster Plan Day 2 Operations

Rust Volume Server

• Rust Volume Server

Advanced

• Large File Handling
• Optimization
• Optimization for Many Small Buckets
• Volume Management
• Tiered Storage
• Cloud Tier
• Cloud Monitoring
• Load Command Line Options from a file
• SRV Service Discovery
• Volume Files Structure

Security

• Security Overview
• Security Configuration
• Cryptography and FIPS Compliance
• Run Blob Storage on Public Internet

Misc Use Case Examples

• UrBackup with SeaweedFS
• Docker Image Registry with SeaweedFS
• SeaweedFS in Docker Swarm
• Words from SeaweedFS Users
• Independent Benchmarks
• Hardware