mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 05:20:49 +02:00
Create Migrate-Maintenance-Scripts-to-Admin-Script-Plugin.md
Chris Lu
@@ -0,0 +1,145 @@
|
||||
# Migrate Maintenance Scripts to Admin Script Plugin
|
||||
|
||||
This guide explains how to migrate from the legacy `[master.maintenance]` scripts in `master.toml` to the **admin script plugin worker**.
|
||||
|
||||
## Why migrate?
|
||||
|
||||
The legacy approach runs maintenance scripts directly on the master server process. This has several drawbacks:
|
||||
|
||||
- Scripts compete for CPU and memory with the master server.
|
||||
- No visibility into script execution progress or errors.
|
||||
- No UI for editing scripts — requires restarting the master to change them.
|
||||
- Erasure coding detection and encoding runs serially on a single node.
|
||||
|
||||
The admin script plugin worker addresses all of these:
|
||||
|
||||
- Scripts run on a separate `weed worker` process, offloading the master.
|
||||
- The admin UI at `/plugin` provides real-time progress, run history, and error reporting.
|
||||
- Script text and run interval are editable from the UI without restarts.
|
||||
- Erasure coding has a dedicated `erasure_coding` plugin worker with per-volume detection and parallel execution across multiple workers.
|
||||
|
||||
## Migration steps
|
||||
|
||||
### 1. Note your current scripts
|
||||
|
||||
Find your `master.toml` (in `./`, `~/.seaweedfs/`, or `/etc/seaweedfs/`) and note the `[master.maintenance]` section:
|
||||
|
||||
```toml
|
||||
[master.maintenance]
|
||||
scripts = """
|
||||
lock
|
||||
ec.encode -fullPercent=95 -quietFor=1h
|
||||
ec.rebuild -apply
|
||||
ec.balance -apply
|
||||
fs.log.purge -daysAgo=7
|
||||
volume.deleteEmpty -quietFor=24h -apply
|
||||
volume.balance -apply
|
||||
volume.fix.replication -apply
|
||||
s3.clean.uploads -timeAgo=24h
|
||||
unlock
|
||||
"""
|
||||
sleep_minutes = 17
|
||||
```
|
||||
|
||||
### 2. Start the admin server
|
||||
|
||||
```bash
|
||||
weed admin -master=localhost:9333
|
||||
```
|
||||
|
||||
The admin server connects to the master. Once connected, the master automatically skips its built-in maintenance scripts (you will see `Skipping master maintenance scripts because admin server is connected` in the master log).
|
||||
|
||||
### 3. Start a worker
|
||||
|
||||
```bash
|
||||
weed worker -admin=localhost:23646
|
||||
```
|
||||
|
||||
The worker registers with the admin server and begins handling plugin jobs. By default it handles all available job types including `admin_script` and `erasure_coding`.
|
||||
|
||||
### 4. Configure the admin script in the UI
|
||||
|
||||
Open the admin UI at `http://<admin-host>:23646/plugin` and find the **Admin Script** job type.
|
||||
|
||||
The default admin script is:
|
||||
|
||||
```
|
||||
ec.balance -apply
|
||||
fs.log.purge -daysAgo=7
|
||||
volume.deleteEmpty -quietFor=24h -apply
|
||||
volume.fix.replication -apply
|
||||
s3.clean.uploads -timeAgo=24h
|
||||
```
|
||||
|
||||
The default run interval is **17 minutes**, matching the legacy default.
|
||||
|
||||
#### Adjustments to make
|
||||
|
||||
Compare your old `master.toml` scripts with the defaults above and customize as needed:
|
||||
|
||||
| Old command | What to do |
|
||||
|---|---|
|
||||
| `lock` / `unlock` | **Remove.** The admin script worker handles locking automatically. |
|
||||
| `ec.encode -fullPercent=95 -quietFor=1h` | **Remove.** This is now handled by the dedicated `erasure_coding` plugin worker, which detects and encodes eligible volumes automatically. Configure its thresholds (fullness ratio, quiet period, etc.) separately in the UI. |
|
||||
| `ec.rebuild -apply` | Add to the admin script if you still want periodic EC shard repair via the script. |
|
||||
| `ec.balance -apply` | Already in the default script. |
|
||||
| `volume.balance -apply` | Add to the admin script if you had it. |
|
||||
| `volume.fix.replication -apply` | Already in the default script. |
|
||||
| `volume.deleteEmpty -quietFor=24h -apply` | Already in the default script. |
|
||||
| `fs.log.purge -daysAgo=7` | Already in the default script. |
|
||||
| `s3.clean.uploads -timeAgo=24h` | Already in the default script. |
|
||||
| `volume.tier.upload -dest s3 ...` | Add to the admin script if you had tiered storage upload. |
|
||||
| Any custom commands | Add them to the script in the UI. |
|
||||
|
||||
If you had a custom `sleep_minutes` value, set the **Run Interval (minutes)** field to match.
|
||||
|
||||
### 5. Verify
|
||||
|
||||
After the first run interval elapses, check the admin UI at `/plugin` for:
|
||||
|
||||
- **Run history**: confirms the script executed successfully.
|
||||
- **Activity stream**: shows each command that was executed.
|
||||
- **Error reporting**: highlights any commands that failed.
|
||||
|
||||
You can also trigger a manual run from the UI to verify immediately.
|
||||
|
||||
### 6. Clean up master.toml (optional)
|
||||
|
||||
Once you have confirmed the admin script plugin worker is running correctly, you can remove or blank out the `[master.maintenance]` section in `master.toml`:
|
||||
|
||||
```toml
|
||||
[master.maintenance]
|
||||
scripts = ""
|
||||
sleep_minutes = 17
|
||||
```
|
||||
|
||||
This is optional — the master already skips its scripts when the admin server is connected. But removing them avoids confusion and prevents them from running if the admin server is temporarily unavailable.
|
||||
|
||||
## Command flag differences
|
||||
|
||||
Some shell commands use different flag names between the legacy scripts and the admin script worker. The admin script worker executes the same `weed shell` commands, so the flags are identical. The only difference is:
|
||||
|
||||
- **`-force`** flag on older scripts (e.g. `ec.balance -force`) has been renamed to **`-apply`** in current versions. Both are accepted, but `-apply` is preferred.
|
||||
|
||||
## Architecture overview
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ weed master │◄────────│ weed admin │────────►│ weed worker │
|
||||
│ │ gRPC │ :23646 │ gRPC │ │
|
||||
│ master.toml │ │ Admin UI │ │ admin_script │
|
||||
│ [maintenance│ │ /plugin │ │ erasure_coding│
|
||||
│ scripts │ │ │ │ vacuum │
|
||||
│ SKIPPED] │ │ │ │ balance │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
- The **master** still has the legacy maintenance code but skips it when an admin server is connected.
|
||||
- The **admin server** manages plugin configuration, scheduling, and dispatching.
|
||||
- The **worker** executes admin script commands and dedicated plugin jobs (erasure coding, vacuum, balance).
|
||||
|
||||
## See also
|
||||
|
||||
- [[Worker]] — `weed worker` command reference
|
||||
- [[Volume Management]] — volume management scripts overview
|
||||
- [[Erasure Coding for warm storage]] — erasure coding details and the `erasure_coding` plugin worker
|
||||
Reference in New Issue
Block a user