diff --git a/Getting-Started.md b/Getting-Started.md index 123fe90..88c1262 100644 --- a/Getting-Started.md +++ b/Getting-Started.md @@ -1,28 +1,49 @@ # Getting Started -## Quick Start with `weed mini` (Recommended for Beginners) +## Quick Start with `weed mini` (Recommended for Most Single-Node and Small-Cluster Setups) -> **⚠️ Warning**: The `weed mini` command is for learning and development only. It may not guarantee backward compatibility and **should not be used in production**. For production use, see [Multi-Component Setup](#multi-component-setup-for-production-and-advanced-users) below. - -The easiest way to get started with SeaweedFS is using the `weed mini` command, which starts all components in one process with sensible defaults: +The easiest way to get started with SeaweedFS is the `weed mini` command, which starts all components (master, volume, filer, S3, WebDAV, Admin UI, and the maintenance worker for vacuum/EC/balance) in one process with auto-tuned defaults: ```bash weed mini -dir=/data ``` -This gives you a complete SeaweedFS setup including S3 compatibility, all accessible via simple web interfaces. See [Quick-Start-with-weed-mini.md](Quick-Start-with-weed-mini.md) for detailed instructions. +This gives you a complete SeaweedFS setup including S3 compatibility and a built-in Admin UI. See [Quick-Start-with-weed-mini.md](Quick-Start-with-weed-mini.md) for detailed instructions. -**Perfect for:** -- Learning SeaweedFS -- Development and testing -- S3 beginners -- Prototyping +**Good for:** +- Learning SeaweedFS, development, and prototyping +- Single-node production setups (S3 gateway, presigned URLs, file storage) +- Small clusters where one node runs `weed mini` and additional `weed volume` servers connect to it for extra capacity + +**When to choose [Multi-Component Setup](#multi-component-setup-for-larger-clusters-and-advanced-users) instead:** +- You need multiple master servers (HA / Raft quorum) — `weed mini` is single-master by design +- You want to run components on separate hosts for isolation or scaling +- You need fine-grained per-component tuning that the auto-tuned defaults don't cover + +> **Note on backward compatibility**: `weed mini` defaults (volume size, port layout, bundled components) may evolve over releases. Pin a SeaweedFS version if you depend on specific defaults, or pass them explicitly as flags. + +### `weed mini` vs `weed server -filer -s3` + +Both run multiple components in one process, but they serve different goals: + +| | `weed mini` | `weed server -filer -s3` | +|---|---|---| +| Components | master + volume + filer + S3 + WebDAV + **Admin UI** + **maintenance worker** | master + volume (+ optional filer / s3 / iam / webdav / sftp / mq) | +| Master mode | Single master (no peers) | Supports multi-master peers | +| Volume size limit | Auto-tuned (64–1024 MB) | Fixed default (~30 GB) | +| Volume max count | Auto (based on free disk) | 8 by default | +| Default volume port | 9340 | 8080 | +| Iceberg REST catalog | Enabled (8181) | Off | +| Pre-stop seconds | 1 (fast shutdown) | 10 | +| Best for | Single-node, small clusters, dev | Building block for multi-master clusters, custom tuning | + +For a single-node S3 setup (e.g. presigned upload/download URLs), `weed mini` is the simpler choice — you also get the Admin UI and built-in maintenance for vacuum/EC/balance without extra processes. --- -## Multi-Component Setup (For Production and Advanced Users) +## Multi-Component Setup (For Larger Clusters and Advanced Users) -For production deployments or if you prefer separating components, follow the manual setup instructions below. +For multi-master clusters, deployments where components live on separate hosts, or setups that need fine-grained tuning beyond `weed mini`'s auto-defaults, follow the manual setup instructions below. Decompress the downloaded file. You will only find one executable file, either "weed" on most systems or "weed.exe" on windows. @@ -62,7 +83,7 @@ If you are using a custom gRPC port for master, the address format for `-master` ### Cheat Sheet: Setup One Master Server and One Volume Server -Actually, forget about previous commands. You can setup one master server and one volume server in one shot: +You can set up one master and one volume server in one shot: ```bash ./weed server -dir="./data" @@ -71,6 +92,14 @@ Actually, forget about previous commands. You can setup one master server and on ./weed server -master.port=9333 -volume.port=8080 -dir="./data" ``` +To also start a filer and S3 gateway in the same process: + +```bash +./weed server -dir="./data" -filer -s3 +``` + +If you want all of that plus the Admin UI, the maintenance worker (vacuum / erasure coding / balance), and auto-tuned volume sizing on a single node, use [`weed mini`](Quick-Start-with-weed-mini.md) instead. + ## Testing SeaweedFS With the master and volume server up, now what? Let's pump in a lot of files into the system! diff --git a/Quick-Start-with-weed-mini.md b/Quick-Start-with-weed-mini.md index 78b15ac..4f67472 100644 --- a/Quick-Start-with-weed-mini.md +++ b/Quick-Start-with-weed-mini.md @@ -1,16 +1,18 @@ # Quick Start with `weed mini` -> **Note**: The `weed mini` command is designed for learning, development, and testing only. It may not guarantee backward compatibility between versions. For production deployments, use the [Multi-Component Setup](Getting-Started.md#multi-component-setup-for-production-and-advanced-users). +`weed mini` is an all-in-one SeaweedFS command that starts all essential components in a single process with auto-tuned defaults. It is appropriate for development and testing **and** for many single-node and small-cluster production setups — for example, an S3 gateway that issues presigned upload/download URLs. -## Overview - -`weed mini` is an all-in-one SeaweedFS command designed specifically for S3 beginners, small deployments, and development environments. It starts all essential SeaweedFS components in a single process with optimized defaults. If there are enough disk space, `weed mini` should be trouble-free in most cases. - -**Perfect for:** +**Good for:** - Learning SeaweedFS and S3-compatible storage -- Development and testing -- Prototyping -- Small deployments with minimal configuration +- Development, testing, and prototyping +- Single-node production setups (S3 gateway, file storage, presigned URLs) +- Small clusters: run `weed mini` on the primary node and connect additional `weed volume` servers to it for extra capacity + +**When to use [Multi-Component Setup](Getting-Started.md#multi-component-setup-for-larger-clusters-and-advanced-users) instead:** +- You need multiple master servers (HA / Raft quorum) — `weed mini` runs a single master by design +- You want components on separate hosts for isolation, or fine-grained tuning beyond the auto-defaults + +> **Backward compatibility**: `weed mini` defaults (volume size, port layout, bundled components) may evolve between releases. Pin a SeaweedFS version or pass the relevant flags explicitly if you depend on specific defaults. ## Features @@ -18,18 +20,36 @@ The `weed mini` command starts: - **Master Server** - Cluster coordination - **Volume Server** - Data storage - **Filer** - Hierarchical file system interface -- **S3 Gateway** - S3-compatible API endpoint +- **S3 Gateway** - S3-compatible API endpoint (with embedded IAM) +- **Iceberg REST Catalog** - Iceberg table catalog - **WebDAV Gateway** - WebDAV file access - **Admin UI** - Web-based management console - **Maintenance Worker** - Background tasks (vacuuming, erasure coding, balancing) All components run in a single process with optimized settings: -- Auto-configured volume size limit (suitable for most use cases) -- Auto-scaling volumes based on available disk space -- Single master mode (no cluster coordination overhead) -- 1-second pre-stop time (faster startup/shutdown) +- Auto-configured volume size limit (64MB–1024MB, based on disk capacity) +- Auto-scaling volume count based on available disk space +- Single master mode (no Raft peers / quorum overhead) +- 1-second pre-stop time (faster shutdown) - Embedded IAM for S3 credential management +## How `weed mini` differs from `weed server -filer -s3` + +Both commands run multiple SeaweedFS components in a single process, but they target different use cases. + +| | `weed mini` | `weed server -filer -s3` | +|---|---|---| +| Components started | master + volume + filer + S3 + WebDAV + **Admin UI** + **maintenance worker** | master + volume + filer + S3 (admin UI and worker not bundled) | +| Master mode | Single master (no peers) | Supports multi-master peers (HA) | +| Volume size limit | Auto-tuned (64–1024 MB based on disk) | Fixed ~30 GB default | +| Volume max count | Auto (based on free disk space) | 8 by default | +| Default volume port | 9340 | 8080 | +| Iceberg REST catalog | Enabled on port 8181 | Not bundled | +| Pre-stop seconds | 1 (fast shutdown) | 10 | +| Intent | Single-node and small-cluster all-in-one | Composable building block for clusters | + +For a typical single-node S3 setup, `weed mini` is the simpler choice — you get the Admin UI and built-in vacuum/EC/balance maintenance without running extra processes. Choose `weed server` when you want explicit control over which components run, plan to run multiple master peers, or are building a larger cluster from individual `weed master` / `weed volume` / `weed filer` processes. + ## Quick Start ### Basic Usage @@ -261,6 +281,17 @@ weed mini -dir=/data -s3.port=8334 weed mini -dir=/data -s3.iam.config=/path/to/iam.json ``` +## Growing Into a Small Cluster + +`weed mini` runs a single master, but the embedded master accepts heartbeats from additional volume servers on other hosts. To add storage capacity to a `weed mini` deployment, run extra `weed volume` processes pointing at the mini node's master: + +```bash +# On the additional storage node: +weed volume -dir=/data -max=0 -master=":9333" -port=8080 +``` + +This keeps the all-in-one simplicity of `weed mini` while spreading data across multiple machines. If you reach a point where you need a multi-master HA setup, migrate to the [Multi-Component Setup](Getting-Started.md#multi-component-setup-for-larger-clusters-and-advanced-users). + ## Stopping the Server ```bash @@ -277,13 +308,13 @@ weed mini -dir=/data -s3.iam.config=/path/to/iam.json ## Performance Considerations -The `mini` command is optimized for development and small deployments. For production workloads: +`weed mini` is tuned for single-node and small-cluster deployments. As your workload grows, consider: -- **Separate components** on different servers (see [Production-Setup.md](Production-Setup.md)) -- **Increase volume size limits** based on your needs -- **Use dedicated object storage** for cloud integration -- **Configure replication** for data safety -- **Set up monitoring** and alerting +- **Configure replication** for data safety (set `-master.defaultReplication`, e.g. `010` to keep a copy on another rack) +- **Add volume servers** on additional hosts for more capacity (see [Growing Into a Small Cluster](#growing-into-a-small-cluster)) +- **Increase the volume size limit** with `-master.volumeSizeLimitMB` if the auto-tuned default is too small for your file mix +- **Set up monitoring** with `-metricsPort` for Prometheus scraping +- **Move to multi-master / multi-component** when you need master HA or want to isolate components on separate hosts ## Related Documentation