mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-15 02:28:31 +02:00
chrislu
00a672d12e
MAJOR ENHANCEMENT: Complete Docker-based integration testing infrastructure ## New Docker Compose Infrastructure: - docker-compose.yml: Complete multi-service setup with health checks - Apache Kafka + Zookeeper - Confluent Schema Registry - SeaweedFS full stack (Master, Volume, Filer, MQ Broker, MQ Agent) - Kafka Gateway service - Test setup and utility services ## Docker Services: - Dockerfile.kafka-gateway: Custom Kafka Gateway container - Dockerfile.test-setup: Schema registration and test data setup - kafka-gateway-start.sh: Service startup script with dependency waiting - wait-for-services.sh: Comprehensive service readiness verification ## Test Setup Utility: - cmd/setup/main.go: Automated schema registration utility - Registers User, UserEvent, and LogEntry Avro schemas - Handles service discovery and health checking ## Integration Tests: - docker_integration_test.go: Comprehensive Docker-based integration tests - Kafka connectivity and topic operations - Schema Registry integration - Kafka Gateway functionality - Sarama and kafka-go client compatibility - Cross-client message compatibility - Performance benchmarking ## Build and Test Infrastructure: - Makefile: 30+ targets for development and testing - setup, test-unit, test-integration, test-e2e - Performance testing and benchmarking - Individual service management - Debugging and monitoring tools - CI/CD integration targets ## Documentation: - README.md: Comprehensive documentation - Architecture overview and service descriptions - Quick start guide and development workflow - Troubleshooting and performance tuning - CI/CD integration examples ## Key Features: ✅ Complete service orchestration with health checks ✅ Automated schema registration and test data setup ✅ Multi-client compatibility testing (Sarama, kafka-go) ✅ Performance benchmarking and monitoring ✅ Development-friendly debugging tools ✅ CI/CD ready with proper cleanup ✅ Comprehensive documentation and examples ## Usage: make setup-schemas # Start all services and register schemas make test-e2e # Run end-to-end integration tests make clean # Clean up environment This provides a production-ready testing infrastructure that ensures Kafka Gateway compatibility with real Kafka ecosystems and validates schema registry integration in realistic deployment scenarios.
57 lines
1.3 KiB
Docker
57 lines
1.3 KiB
Docker
# Dockerfile for Kafka Gateway Integration Testing
|
|
FROM golang:1.21-alpine AS builder
|
|
|
|
# Install build dependencies
|
|
RUN apk add --no-cache git make gcc musl-dev sqlite-dev
|
|
|
|
# Set working directory
|
|
WORKDIR /app
|
|
|
|
# Copy go mod files
|
|
COPY go.mod go.sum ./
|
|
|
|
# Download dependencies
|
|
RUN go mod download
|
|
|
|
# Copy source code
|
|
COPY . .
|
|
|
|
# Build the weed binary with Kafka gateway support
|
|
RUN CGO_ENABLED=1 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o weed ./weed
|
|
|
|
# Final stage
|
|
FROM alpine:latest
|
|
|
|
# Install runtime dependencies
|
|
RUN apk --no-cache add ca-certificates wget curl netcat-openbsd sqlite
|
|
|
|
# Create non-root user
|
|
RUN addgroup -g 1000 seaweedfs && \
|
|
adduser -D -s /bin/sh -u 1000 -G seaweedfs seaweedfs
|
|
|
|
# Set working directory
|
|
WORKDIR /usr/bin
|
|
|
|
# Copy binary from builder
|
|
COPY --from=builder /app/weed .
|
|
|
|
# Create data directory
|
|
RUN mkdir -p /data && chown seaweedfs:seaweedfs /data
|
|
|
|
# Copy startup script
|
|
COPY test/kafka/scripts/kafka-gateway-start.sh /usr/bin/kafka-gateway-start.sh
|
|
RUN chmod +x /usr/bin/kafka-gateway-start.sh
|
|
|
|
# Switch to non-root user
|
|
USER seaweedfs
|
|
|
|
# Expose Kafka protocol port
|
|
EXPOSE 9093
|
|
|
|
# Health check
|
|
HEALTHCHECK --interval=10s --timeout=5s --start-period=30s --retries=3 \
|
|
CMD nc -z localhost 9093 || exit 1
|
|
|
|
# Default command
|
|
CMD ["/usr/bin/kafka-gateway-start.sh"]
|