mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 13:30:46 +02:00
chrislu
00a672d12e
MAJOR ENHANCEMENT: Complete Docker-based integration testing infrastructure ## New Docker Compose Infrastructure: - docker-compose.yml: Complete multi-service setup with health checks - Apache Kafka + Zookeeper - Confluent Schema Registry - SeaweedFS full stack (Master, Volume, Filer, MQ Broker, MQ Agent) - Kafka Gateway service - Test setup and utility services ## Docker Services: - Dockerfile.kafka-gateway: Custom Kafka Gateway container - Dockerfile.test-setup: Schema registration and test data setup - kafka-gateway-start.sh: Service startup script with dependency waiting - wait-for-services.sh: Comprehensive service readiness verification ## Test Setup Utility: - cmd/setup/main.go: Automated schema registration utility - Registers User, UserEvent, and LogEntry Avro schemas - Handles service discovery and health checking ## Integration Tests: - docker_integration_test.go: Comprehensive Docker-based integration tests - Kafka connectivity and topic operations - Schema Registry integration - Kafka Gateway functionality - Sarama and kafka-go client compatibility - Cross-client message compatibility - Performance benchmarking ## Build and Test Infrastructure: - Makefile: 30+ targets for development and testing - setup, test-unit, test-integration, test-e2e - Performance testing and benchmarking - Individual service management - Debugging and monitoring tools - CI/CD integration targets ## Documentation: - README.md: Comprehensive documentation - Architecture overview and service descriptions - Quick start guide and development workflow - Troubleshooting and performance tuning - CI/CD integration examples ## Key Features: ✅ Complete service orchestration with health checks ✅ Automated schema registration and test data setup ✅ Multi-client compatibility testing (Sarama, kafka-go) ✅ Performance benchmarking and monitoring ✅ Development-friendly debugging tools ✅ CI/CD ready with proper cleanup ✅ Comprehensive documentation and examples ## Usage: make setup-schemas # Start all services and register schemas make test-e2e # Run end-to-end integration tests make clean # Clean up environment This provides a production-ready testing infrastructure that ensures Kafka Gateway compatibility with real Kafka ecosystems and validates schema registry integration in realistic deployment scenarios.
42 lines
1005 B
Bash
Executable File
42 lines
1005 B
Bash
Executable File
#!/bin/sh
|
|
|
|
# Kafka Gateway Startup Script for Integration Testing
|
|
|
|
set -e
|
|
|
|
echo "Starting Kafka Gateway..."
|
|
|
|
# Wait for dependencies
|
|
echo "Waiting for SeaweedFS Filer..."
|
|
while ! nc -z ${SEAWEEDFS_FILER%:*} ${SEAWEEDFS_FILER#*:}; do
|
|
sleep 1
|
|
done
|
|
echo "SeaweedFS Filer is ready"
|
|
|
|
echo "Waiting for SeaweedFS MQ Broker..."
|
|
while ! nc -z ${SEAWEEDFS_MQ_BROKER%:*} ${SEAWEEDFS_MQ_BROKER#*:}; do
|
|
sleep 1
|
|
done
|
|
echo "SeaweedFS MQ Broker is ready"
|
|
|
|
echo "Waiting for Schema Registry..."
|
|
while ! curl -f ${SCHEMA_REGISTRY_URL}/subjects > /dev/null 2>&1; do
|
|
sleep 1
|
|
done
|
|
echo "Schema Registry is ready"
|
|
|
|
# Create offset database directory
|
|
mkdir -p /data/offsets
|
|
|
|
# Start Kafka Gateway
|
|
echo "Starting Kafka Gateway on port ${KAFKA_PORT:-9093}..."
|
|
exec /usr/bin/weed kafka.gateway \
|
|
-filer=${SEAWEEDFS_FILER} \
|
|
-mq.broker=${SEAWEEDFS_MQ_BROKER} \
|
|
-schema.registry=${SCHEMA_REGISTRY_URL} \
|
|
-port=${KAFKA_PORT:-9093} \
|
|
-ip=0.0.0.0 \
|
|
-offset.db=/data/offsets/kafka-offsets.db \
|
|
-log.level=1 \
|
|
-v=2
|