chrislu 00a672d12e Add comprehensive Docker Compose setup for Kafka integration tests ...

MAJOR ENHANCEMENT: Complete Docker-based integration testing infrastructure

## New Docker Compose Infrastructure:
- docker-compose.yml: Complete multi-service setup with health checks
  - Apache Kafka + Zookeeper
  - Confluent Schema Registry
  - SeaweedFS full stack (Master, Volume, Filer, MQ Broker, MQ Agent)
  - Kafka Gateway service
  - Test setup and utility services

## Docker Services:
- Dockerfile.kafka-gateway: Custom Kafka Gateway container
- Dockerfile.test-setup: Schema registration and test data setup
- kafka-gateway-start.sh: Service startup script with dependency waiting
- wait-for-services.sh: Comprehensive service readiness verification

## Test Setup Utility:
- cmd/setup/main.go: Automated schema registration utility
- Registers User, UserEvent, and LogEntry Avro schemas
- Handles service discovery and health checking

## Integration Tests:
- docker_integration_test.go: Comprehensive Docker-based integration tests
  - Kafka connectivity and topic operations
  - Schema Registry integration
  - Kafka Gateway functionality
  - Sarama and kafka-go client compatibility
  - Cross-client message compatibility
  - Performance benchmarking

## Build and Test Infrastructure:
- Makefile: 30+ targets for development and testing
  - setup, test-unit, test-integration, test-e2e
  - Performance testing and benchmarking
  - Individual service management
  - Debugging and monitoring tools
  - CI/CD integration targets

## Documentation:
- README.md: Comprehensive documentation
  - Architecture overview and service descriptions
  - Quick start guide and development workflow
  - Troubleshooting and performance tuning
  - CI/CD integration examples

## Key Features:
✅ Complete service orchestration with health checks
✅ Automated schema registration and test data setup
✅ Multi-client compatibility testing (Sarama, kafka-go)
✅ Performance benchmarking and monitoring
✅ Development-friendly debugging tools
✅ CI/CD ready with proper cleanup
✅ Comprehensive documentation and examples

## Usage:
make setup-schemas  # Start all services and register schemas
make test-e2e      # Run end-to-end integration tests
make clean         # Clean up environment

This provides a production-ready testing infrastructure that ensures
Kafka Gateway compatibility with real Kafka ecosystems and validates
schema registry integration in realistic deployment scenarios.

2025-09-12 08:46:03 -07:00

..

main.go

Add comprehensive Docker Compose setup for Kafka integration tests

2025-09-12 08:46:03 -07:00