mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-20 04:54:24 +02:00
admin: default to port 12535, keep 23646 as a best-effort compat listener (#11816)
* admin: default to port 12535, keep 23646 as a best-effort compat listener The default admin HTTP port 23646 derives a worker gRPC port of 33646 (port + GrpcPortOffset), inside the Linux default ephemeral range (32768+). One of our own startup dials can claim 33646 before the admin binds it, killing startup. This is the same failure #10209 fixed for the test port allocator, still reachable via the mini default (#10838) and via standalone `weed admin` defaults. Move the default admin port to 12535 (gRPC 22535) everywhere, keeping the http+10000 convention so worker -admin derivation, the Rust worker and host:http.grpc addressing keep working: - weed admin -port, weed mini -admin.port - weed worker -admin, the plugin worker fallback, Rust worker and weed-lance-worker defaults - help text, README, helm values, docker compose, tests So old workers, bookmarks and scripts keep working, admin also serves the same handler and the worker gRPC server on the previous default (-port.legacy / -admin.port.legacy, default 23646, 0 disables). The legacy gRPC port binds best-effort: it lives in the ephemeral range, so a failed bind only warns instead of reintroducing the startup failure. weed mini reserves it early like the primary port and keeps it out of its own port relocation. grpc.Server tracks every listener handed to Serve, so the compat listener stops with the primary. Also warn when an explicitly configured admin port puts the derived gRPC port at or above the ephemeral floor, the other fix suggested in the issue. Supersedes the pinned-grpc approach in #11814: pinning -admin.port.grpc off the +10000 convention breaks external workers that derive the admin gRPC port from the HTTP address. ClickHouse/ClickHouse#115512 pins -admin.port on their side; this makes that pin unnecessary. Fixes #10838 * admin: let services win the legacy ports, close compat listener on failure Review findings on the compat listeners: - Do not put the legacy ports in mini's reservedPorts: an explicit -s3.port=23646 or -master.port.grpc=33646 must be honored, not rejected or stolen by the early reservation. Mini now checks the finalized port set and skips only the conflicting compat listener, so HTTP and gRPC compat degrade independently and the service keeps its port. - Also skip the legacy HTTP listener when a mini service owns the port: a wildcard-bound service can still leave loopback bindable, which would let the compat listener shadow it for loopback clients. - Close legacyServer's raw listener in the deferred cleanup: Serve may not have registered it yet when startAdminServer returns early. - Helm: add admin.legacyPort (default 23646) and expose the legacy http/grpc ports through the admin Service, container ports and NetworkPolicy so older workers resolved via the Service keep working. * helm: only pass -port.legacy when it differs from the binary default ct install deploys the released image with the new chart, and 4.48 does not know -port.legacy, so the admin pod crash-loops. Omitting the flag at the default value is behavior-identical on new binaries (their default is also 23646) and keeps older releases installable; legacyPort: 0 or a custom port still renders the flag.
This commit is contained in:
1 parent
d3ee817b2e
commit
8ac5133abe
33 files changed
+316
-101
No files matched your search
@@ -21,7 +21,7 @@ The testing environment uses **official SeaweedFS commands** and includes:
|
||||
- **1 Master Server** (port 9333) - Coordinates the cluster with 50MB volume size limit
|
||||
- **6 Volume Servers** (ports 8080-8085) - Distributed across 2 data centers and 3 racks for diversity
|
||||
- **1 Filer** (port 8888) - Provides file system interface
|
||||
- **1 Admin Server** (port 23646) - Detects volumes needing EC and manages workers using official `admin` command
|
||||
- **1 Admin Server** (port 12535) - Detects volumes needing EC and manages workers using official `admin` command
|
||||
- **3 EC Workers** - Execute erasure coding tasks using official `worker` command with task-specific working directories
|
||||
- **1 Load Generator** - Continuously writes and deletes files using SeaweedFS shell commands
|
||||
- **1 Monitor** - Tracks cluster health and EC progress using shell scripts
|
||||
@@ -47,7 +47,7 @@ This provides:
|
||||
- Docker and Docker Compose installed
|
||||
- GNU Make installed
|
||||
- At least 4GB RAM available for containers
|
||||
- Ports 8080-8085, 8888, 9333, 23646 available
|
||||
- Ports 8080-8085, 8888, 9333, 12535 available
|
||||
|
||||
### Start the Environment
|
||||
|
||||
@@ -127,7 +127,7 @@ Run `make help` to see all available targets:
|
||||
|---------|-----|-------------|
|
||||
| Master UI | http://localhost:9333 | Cluster status and topology |
|
||||
| Filer | http://localhost:8888 | File operations |
|
||||
| Admin Server | http://localhost:23646/ | Task management |
|
||||
| Admin Server | http://localhost:12535/ | Task management |
|
||||
| Monitor | http://localhost:9999/status | Complete cluster monitoring |
|
||||
| Volume Servers | http://localhost:8080-8085/status | Individual volume server stats |
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ build-and-restart: build ## Build with latest changes and restart services
|
||||
@echo "2️⃣ Recreating workers to reconnect..."
|
||||
@docker-compose -f $(COMPOSE_FILE) up -d worker1 worker2 worker3
|
||||
@echo "✅ All services recreated with latest changes!"
|
||||
@echo "🌐 Admin UI: http://localhost:23646/"
|
||||
@echo "🌐 Admin UI: http://localhost:12535/"
|
||||
@echo "💡 Workers will reconnect to the new admin server"
|
||||
|
||||
restart-workers: ## Restart all workers to reconnect to admin server
|
||||
@@ -64,7 +64,7 @@ start: ## Start the complete SeaweedFS cluster with admin and workers
|
||||
@echo "✅ Cluster started!"
|
||||
@echo ""
|
||||
@echo "📊 Access points:"
|
||||
@echo " • Admin UI: http://localhost:23646/"
|
||||
@echo " • Admin UI: http://localhost:12535/"
|
||||
@echo " • Master UI: http://localhost:9333/"
|
||||
@echo " • Filer: http://localhost:8888/"
|
||||
@echo ""
|
||||
@@ -107,7 +107,7 @@ start-staged: ## Start services in proper order with delays
|
||||
@echo "✅ All services started!"
|
||||
@echo ""
|
||||
@echo "📊 Access points:"
|
||||
@echo " • Admin UI: http://localhost:23646/"
|
||||
@echo " • Admin UI: http://localhost:12535/"
|
||||
@echo " • Master UI: http://localhost:9333/"
|
||||
@echo " • Filer: http://localhost:8888/"
|
||||
@echo ""
|
||||
@@ -136,7 +136,7 @@ status: ## Check the status of all services
|
||||
@echo "Master:"
|
||||
@curl -s http://localhost:9333/cluster/status | jq '.IsLeader' 2>/dev/null || echo " ❌ Master not ready"
|
||||
@echo "Admin:"
|
||||
@curl -s http://localhost:23646/ | grep -q "Admin" && echo " ✅ Admin ready" || echo " ❌ Admin not ready"
|
||||
@curl -s http://localhost:12535/ | grep -q "Admin" && echo " ✅ Admin ready" || echo " ❌ Admin not ready"
|
||||
|
||||
logs: ## Show logs from all services
|
||||
@echo "📜 Following logs from all services..."
|
||||
@@ -157,7 +157,7 @@ master-logs: ## Show logs from master server
|
||||
|
||||
admin-ui: ## Open admin UI in browser (macOS)
|
||||
@echo "🌐 Opening admin UI in browser..."
|
||||
@open http://localhost:23646/ || echo "💡 Manually open: http://localhost:23646/"
|
||||
@open http://localhost:12535/ || echo "💡 Manually open: http://localhost:12535/"
|
||||
|
||||
test: ## Run integration test to verify task assignment and completion
|
||||
@echo "🧪 Running Admin-Worker Integration Test"
|
||||
@@ -166,11 +166,11 @@ test: ## Run integration test to verify task assignment and completion
|
||||
@echo "1️⃣ Checking cluster health..."
|
||||
@sleep 5
|
||||
@curl -s http://localhost:9333/cluster/status | jq '.IsLeader' > /dev/null && echo "✅ Master healthy" || echo "❌ Master not ready"
|
||||
@curl -s http://localhost:23646/ | grep -q "Admin" && echo "✅ Admin healthy" || echo "❌ Admin not ready"
|
||||
@curl -s http://localhost:12535/ | grep -q "Admin" && echo "✅ Admin healthy" || echo "❌ Admin not ready"
|
||||
@echo ""
|
||||
@echo "2️⃣ Checking worker registration..."
|
||||
@sleep 10
|
||||
@echo "💡 Check admin UI for connected workers: http://localhost:23646/"
|
||||
@echo "💡 Check admin UI for connected workers: http://localhost:12535/"
|
||||
@echo ""
|
||||
@echo "3️⃣ Generating load to trigger EC tasks..."
|
||||
@echo "📝 Creating test files to fill volumes..."
|
||||
@@ -193,13 +193,13 @@ test: ## Run integration test to verify task assignment and completion
|
||||
@echo " • EC shards being distributed"
|
||||
@echo ""
|
||||
@echo "✅ Integration test setup complete!"
|
||||
@echo "📊 Monitor progress at: http://localhost:23646/"
|
||||
@echo "📊 Monitor progress at: http://localhost:12535/"
|
||||
|
||||
quick-test: ## Quick verification that core services are running
|
||||
@echo "⚡ Quick Health Check"
|
||||
@echo "===================="
|
||||
@echo "Master: $$(curl -s http://localhost:9333/cluster/status | jq -r '.IsLeader // "not ready"')"
|
||||
@echo "Admin: $$(curl -s http://localhost:23646/ | grep -q "Admin" && echo "ready" || echo "not ready")"
|
||||
@echo "Admin: $$(curl -s http://localhost:12535/ | grep -q "Admin" && echo "ready" || echo "not ready")"
|
||||
@echo "Workers: $$(docker-compose -f $(COMPOSE_FILE) ps worker1 worker2 worker3 | grep -c Up) running"
|
||||
|
||||
validate: ## Validate integration test configuration
|
||||
@@ -223,7 +223,7 @@ demo: start ## Start cluster and run demonstration
|
||||
@echo " • Load generator (creates files continuously)"
|
||||
@echo ""
|
||||
@echo "📊 Watch the process:"
|
||||
@echo " 1. Visit: http://localhost:23646/"
|
||||
@echo " 1. Visit: http://localhost:12535/"
|
||||
@echo " 2. Observe workers connecting"
|
||||
@echo " 3. Watch tasks being created and assigned"
|
||||
@echo " 4. See tasks progress from pending → completed"
|
||||
@@ -244,13 +244,13 @@ vacuum-test: ## Create test data with garbage and verify vacuum detection
|
||||
@echo ""
|
||||
@echo "1️⃣ Checking cluster health..."
|
||||
@curl -s http://localhost:9333/cluster/status | jq '.IsLeader' > /dev/null && echo "✅ Master ready" || (echo "❌ Master not ready. Run 'make start' first." && exit 1)
|
||||
@curl -s http://localhost:23646/ | grep -q "Admin" && echo "✅ Admin ready" || (echo "❌ Admin not ready. Run 'make start' first." && exit 1)
|
||||
@curl -s http://localhost:12535/ | grep -q "Admin" && echo "✅ Admin ready" || (echo "❌ Admin not ready. Run 'make start' first." && exit 1)
|
||||
@echo ""
|
||||
@echo "2️⃣ Creating test data with garbage..."
|
||||
@docker-compose -f $(COMPOSE_FILE) exec vacuum-tester go run create_vacuum_test_data.go -files=25 -delete=0.5 -size=200
|
||||
@echo ""
|
||||
@echo "3️⃣ Configuration Instructions:"
|
||||
@echo " Visit: http://localhost:23646/maintenance/config/vacuum"
|
||||
@echo " Visit: http://localhost:12535/maintenance/config/vacuum"
|
||||
@echo " Set for testing:"
|
||||
@echo " • Enable Vacuum Tasks: ✅ Checked"
|
||||
@echo " • Garbage Threshold: 0.20 (20%)"
|
||||
@@ -258,7 +258,7 @@ vacuum-test: ## Create test data with garbage and verify vacuum detection
|
||||
@echo " • Min Volume Age: [0] [Minutes]"
|
||||
@echo " • Max Concurrent: 2"
|
||||
@echo ""
|
||||
@echo "4️⃣ Monitor vacuum tasks at: http://localhost:23646/maintenance"
|
||||
@echo "4️⃣ Monitor vacuum tasks at: http://localhost:12535/maintenance"
|
||||
@echo ""
|
||||
@echo "💡 Use 'make vacuum-status' to check volume garbage ratios"
|
||||
|
||||
@@ -335,8 +335,8 @@ vacuum-help: ## Show vacuum testing help and examples
|
||||
@echo " make vacuum-demo # Full guided demonstration"
|
||||
@echo ""
|
||||
@echo "Configuration:"
|
||||
@echo " Visit: http://localhost:23646/maintenance/config/vacuum"
|
||||
@echo " Monitor: http://localhost:23646/maintenance"
|
||||
@echo " Visit: http://localhost:12535/maintenance/config/vacuum"
|
||||
@echo " Monitor: http://localhost:12535/maintenance"
|
||||
@echo ""
|
||||
@echo "Custom Parameters:"
|
||||
@echo " make vacuum-data FILES=50 DELETE=0.8 SIZE=200"
|
||||
|
||||
@@ -243,7 +243,7 @@ func printTestingInstructions() {
|
||||
fmt.Println("🧪 Testing Instructions:")
|
||||
fmt.Println()
|
||||
fmt.Println("1. Configure Vacuum for Testing:")
|
||||
fmt.Println(" Visit: http://localhost:23646/maintenance/config/vacuum")
|
||||
fmt.Println(" Visit: http://localhost:12535/maintenance/config/vacuum")
|
||||
fmt.Println(" Set:")
|
||||
fmt.Printf(" - Garbage Percentage Threshold: 20 (20%% - lower than default 30)\n")
|
||||
fmt.Printf(" - Scan Interval: [30] [Seconds] (faster than default)\n")
|
||||
@@ -253,7 +253,7 @@ func printTestingInstructions() {
|
||||
fmt.Println()
|
||||
|
||||
fmt.Println("2. Monitor Vacuum Tasks:")
|
||||
fmt.Println(" Visit: http://localhost:23646/maintenance")
|
||||
fmt.Println(" Visit: http://localhost:12535/maintenance")
|
||||
fmt.Println(" Watch for vacuum tasks to appear in the queue")
|
||||
fmt.Println()
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ echo ""
|
||||
# Check if SeaweedFS is running
|
||||
echo "📋 Checking SeaweedFS status..."
|
||||
MASTER_URL="${MASTER_HOST:-master:9333}"
|
||||
ADMIN_URL="${ADMIN_HOST:-admin:23646}"
|
||||
ADMIN_URL="${ADMIN_HOST:-admin:12535}"
|
||||
|
||||
if ! curl -s http://$MASTER_URL/cluster/status > /dev/null; then
|
||||
echo "❌ SeaweedFS master not running at $MASTER_URL"
|
||||
@@ -42,7 +42,7 @@ echo ""
|
||||
|
||||
# Phase 3: Configure vacuum
|
||||
echo "⚙️ Phase 3: Vacuum configuration instructions..."
|
||||
echo " 1. Visit: http://localhost:23646/maintenance/config/vacuum"
|
||||
echo " 1. Visit: http://localhost:12535/maintenance/config/vacuum"
|
||||
echo " 2. Set these values for testing:"
|
||||
echo " - Enable Vacuum Tasks: ✅ Checked"
|
||||
echo " - Garbage Threshold: 0.30"
|
||||
@@ -57,7 +57,7 @@ echo ""
|
||||
|
||||
# Phase 4: Monitor tasks
|
||||
echo "🎯 Phase 4: Monitoring vacuum tasks..."
|
||||
echo " Visit: http://localhost:23646/maintenance"
|
||||
echo " Visit: http://localhost:12535/maintenance"
|
||||
echo " You should see vacuum tasks appear within 30 seconds"
|
||||
echo ""
|
||||
|
||||
@@ -93,9 +93,9 @@ echo ""
|
||||
echo "🎉 Demo Complete!"
|
||||
echo ""
|
||||
echo "🔍 Things to check:"
|
||||
echo " 1. Maintenance Queue: http://localhost:23646/maintenance"
|
||||
echo " 1. Maintenance Queue: http://localhost:12535/maintenance"
|
||||
echo " 2. Volume Status: http://localhost:9333/vol/status"
|
||||
echo " 3. Admin Dashboard: http://localhost:23646"
|
||||
echo " 3. Admin Dashboard: http://localhost:12535"
|
||||
echo ""
|
||||
echo "💡 Next Steps:"
|
||||
echo " - Try different garbage thresholds (0.10, 0.50, 0.80)"
|
||||
|
||||
@@ -114,9 +114,9 @@ services:
|
||||
admin:
|
||||
image: chrislusf/seaweedfs:local
|
||||
ports:
|
||||
- "23646:23646" # HTTP admin interface (default port)
|
||||
- "33646:33646" # gRPC worker communication (23646 + 10000)
|
||||
command: "-v=2 admin -port=23646 -masters=master:9333 -dataDir=/data"
|
||||
- "12535:12535" # HTTP admin interface (default port)
|
||||
- "22535:22535" # gRPC worker communication (12535 + 10000)
|
||||
command: "-v=2 admin -port=12535 -masters=master:9333 -dataDir=/data"
|
||||
depends_on:
|
||||
- master
|
||||
- filer
|
||||
@@ -127,7 +127,7 @@ services:
|
||||
|
||||
worker1:
|
||||
image: chrislusf/seaweedfs:local
|
||||
command: "-v=2 worker -admin=admin:23646 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
command: "-v=2 worker -admin=admin:12535 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
depends_on:
|
||||
- admin
|
||||
volumes:
|
||||
@@ -139,7 +139,7 @@ services:
|
||||
|
||||
worker2:
|
||||
image: chrislusf/seaweedfs:local
|
||||
command: "-v=2 worker -admin=admin:23646 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
command: "-v=2 worker -admin=admin:12535 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
depends_on:
|
||||
- admin
|
||||
volumes:
|
||||
@@ -151,7 +151,7 @@ services:
|
||||
|
||||
worker3:
|
||||
image: chrislusf/seaweedfs:local
|
||||
command: "-v=2 worker -admin=admin:23646 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
command: "-v=2 worker -admin=admin:12535 -capabilities=erasure_coding,vacuum -maxConcurrent=2"
|
||||
depends_on:
|
||||
- admin
|
||||
volumes:
|
||||
@@ -199,7 +199,7 @@ services:
|
||||
curl -s http://master:9333/cluster/status | jq '.IsLeader, .Peers' || echo 'Master not ready';
|
||||
echo;
|
||||
echo 'Admin status:';
|
||||
curl -s http://admin:23646/ | grep -o 'Admin.*Interface' || echo 'Admin not ready';
|
||||
curl -s http://admin:12535/ | grep -o 'Admin.*Interface' || echo 'Admin not ready';
|
||||
echo;
|
||||
echo 'Volume count by server:';
|
||||
curl -s http://master:9333/vol/status | jq '.Volumes | length' || echo 'Volumes not ready';
|
||||
@@ -237,4 +237,4 @@ services:
|
||||
- seaweed_net
|
||||
environment:
|
||||
- MASTER_HOST=master:9333
|
||||
- ADMIN_HOST=admin:23646
|
||||
- ADMIN_HOST=admin:12535
|
||||
@@ -23,7 +23,7 @@ else
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}2. Checking if required ports are available...${NC}"
|
||||
for port in 9333 8080 8081 8082 8083 8084 8085 8888 23646; do
|
||||
for port in 9333 8080 8081 8082 8083 8084 8085 8888 12535; do
|
||||
if lsof -i :$port > /dev/null 2>&1; then
|
||||
echo -e "${YELLOW}⚠️ Port $port is in use${NC}"
|
||||
else
|
||||
@@ -41,7 +41,7 @@ else
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}4. Verifying admin server configuration...${NC}"
|
||||
if docker-compose -f docker-compose-ec-test.yml config | grep -q "admin:23646"; then
|
||||
if docker-compose -f docker-compose-ec-test.yml config | grep -q "admin:12535"; then
|
||||
echo -e "${GREEN}✅ Admin server port configuration is correct${NC}"
|
||||
else
|
||||
echo -e "${RED}❌ Admin server port configuration is incorrect${NC}"
|
||||
|
||||
Reference in new issue
Block a user