mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-08 15:41:15 +02:00
docs(note): add production-setup slide deck
Marp-based markdown deck walking through the three-layer production topology (masters, volumes, filers + DB, S3 gateways, admin + workers) plus an erasure-coding note. Uses the object-store-layout diagram on the overview slide. Makefile renders PDF/HTML/PPTX via marp-cli.
This commit is contained in:
@@ -0,0 +1,31 @@
|
|||||||
|
MARP ?= npx @marp-team/marp-cli
|
||||||
|
SRC := seaweedfs-production-setup.md
|
||||||
|
PDF := $(SRC:.md=.pdf)
|
||||||
|
HTML := $(SRC:.md=.html)
|
||||||
|
PPTX := $(SRC:.md=.pptx)
|
||||||
|
FLAGS := --allow-local-files
|
||||||
|
|
||||||
|
.PHONY: all pdf html pptx watch clean
|
||||||
|
|
||||||
|
all: pdf
|
||||||
|
|
||||||
|
pdf: $(PDF)
|
||||||
|
|
||||||
|
html: $(HTML)
|
||||||
|
|
||||||
|
pptx: $(PPTX)
|
||||||
|
|
||||||
|
$(PDF): $(SRC)
|
||||||
|
$(MARP) $(SRC) --pdf $(FLAGS) -o $@
|
||||||
|
|
||||||
|
$(HTML): $(SRC)
|
||||||
|
$(MARP) $(SRC) --html $(FLAGS) -o $@
|
||||||
|
|
||||||
|
$(PPTX): $(SRC)
|
||||||
|
$(MARP) $(SRC) --pptx $(FLAGS) -o $@
|
||||||
|
|
||||||
|
watch:
|
||||||
|
$(MARP) $(SRC) --watch --html $(FLAGS)
|
||||||
|
|
||||||
|
clean:
|
||||||
|
rm -f $(PDF) $(HTML) $(PPTX)
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 738 KiB |
@@ -0,0 +1,118 @@
|
|||||||
|
---
|
||||||
|
marp: true
|
||||||
|
theme: default
|
||||||
|
paginate: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# SeaweedFS Production Setup
|
||||||
|
|
||||||
|
A walkthrough of the recommended deployment topology
|
||||||
|
|
||||||
|
- Storage layer
|
||||||
|
- File access layer
|
||||||
|
- Backend operations
|
||||||
|
- Erasure coding for durability
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Storage Layer
|
||||||
|
|
||||||
|
**3 Masters + Volume Servers**
|
||||||
|
|
||||||
|
- **Masters (run 3)**
|
||||||
|
- Form a Raft quorum; tolerate 1 failure
|
||||||
|
- Track volume locations and cluster topology
|
||||||
|
- Lightweight; not on the data path for reads/writes after lookup
|
||||||
|
|
||||||
|
- **Volume Servers (scale out)**
|
||||||
|
- Hold the actual needle data on local disks
|
||||||
|
- Add more servers for capacity and throughput
|
||||||
|
- Heartbeat to masters; clients talk to them directly for I/O
|
||||||
|
|
||||||
|
Rule of thumb: start with 3 masters, then grow volume servers with data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# File Access Layer
|
||||||
|
|
||||||
|
**Filer + DB for metadata**
|
||||||
|
|
||||||
|
- Filer serves POSIX-like directory and file metadata
|
||||||
|
- Metadata store options: PostgreSQL, MySQL, Redis, Cassandra, TiKV, etc.
|
||||||
|
- Run **3 filers** for HA
|
||||||
|
- Filers are stateless relative to each other — state lives in the DB
|
||||||
|
|
||||||
|
**S3 Servers**
|
||||||
|
|
||||||
|
- Expose the S3 API on top of the filer
|
||||||
|
- Scale the count based on throughput requirements
|
||||||
|
- Can be co-located with filers or run independently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Backend Operations
|
||||||
|
|
||||||
|
**1 Admin Server + a few Workers**
|
||||||
|
|
||||||
|
- **Admin server (1 is enough)**
|
||||||
|
- Coordinates background jobs: balancing, EC encoding, vacuum, replication fixes
|
||||||
|
- **Not on the data path** — a restart does not affect object store reads/writes
|
||||||
|
- No HA required for normal operation
|
||||||
|
|
||||||
|
- **Workers (a few)**
|
||||||
|
- Execute long-running tasks issued by the admin server
|
||||||
|
- Horizontally scalable based on maintenance workload
|
||||||
|
|
||||||
|
Keeps housekeeping off the hot path.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Production Topology Summary
|
||||||
|
|
||||||
|
| Component | Count | Role |
|
||||||
|
|----------------|----------------------|-----------------------------------|
|
||||||
|
| Master | 3 | Raft quorum, topology, volume map |
|
||||||
|
| Volume Server | N (scale out) | Stores data needles |
|
||||||
|
| Filer | 3 | Metadata gateway |
|
||||||
|
| Metadata DB | HA cluster | Persists filer metadata |
|
||||||
|
| S3 Server | N (by throughput) | S3 API frontend |
|
||||||
|
| Admin | 1 | Background job coordinator |
|
||||||
|
| Worker | A few | Executes admin-scheduled tasks |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Erasure Coding for Durability
|
||||||
|
|
||||||
|
Prefer **many volume servers** so EC shards spread across failure domains.
|
||||||
|
|
||||||
|
**Example: 8 volume servers, EC 5+3**
|
||||||
|
|
||||||
|
- Each volume becomes 5 data shards + 3 parity shards
|
||||||
|
- Shards are placed on 8 distinct volume servers
|
||||||
|
- **Tolerates up to 3 simultaneous volume server failures**
|
||||||
|
- Storage overhead: 1.6x vs 3x for full replication
|
||||||
|
|
||||||
|
```
|
||||||
|
Volume -> [D1][D2][D3][D4][D5] + [P1][P2][P3]
|
||||||
|
| | | | | | | |
|
||||||
|
VS1 VS2 VS3 VS4 VS5 VS6 VS7 VS8
|
||||||
|
```
|
||||||
|
|
||||||
|
More volume servers = more EC layouts available (e.g. 10+4, 6+3).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# Getting Started Checklist
|
||||||
|
|
||||||
|
1. Provision 3 master nodes (small boxes are fine)
|
||||||
|
2. Provision volume servers sized to your data footprint
|
||||||
|
3. Stand up an HA metadata DB, then 3 filers pointing at it
|
||||||
|
4. Add S3 servers sized to required throughput
|
||||||
|
5. Run 1 admin + a few workers for background maintenance
|
||||||
|
6. Once you have enough volume servers, enable EC (e.g. 5+3 on 8 servers)
|
||||||
|
|
||||||
|
Scale each layer independently as usage grows.
|
||||||
Reference in New Issue
Block a user