Commit Graph
4 Commits
Author SHA1 Message Date
chrislu 3570027656 feat: Enable publishers to use Parquet logical types
Enhanced MQ publishers to utilize the new logical types:
- Updated convertToRecordValue() to use TimestampValue instead of string RFC3339
- Added DateValue support for birth_date field (days since epoch)
- Added DecimalValue support for precise_amount field with configurable precision/scale
- Enhanced UserEvent struct with PreciseAmount and BirthDate fields
- Added convertToDecimal() helper using big.Rat for precise decimal conversion
- Updated test data generator to produce varied birth dates (1970-2005) and precise amounts

Publishers now generate structured data with proper logical types:
-  TIMESTAMP: Microsecond precision UTC timestamps
-  DATE: Birth dates as days since Unix epoch
-  DECIMAL: Precise amounts with 18-digit precision, 4-decimal scale

Successfully tested with PostgreSQL integration - all topics created with logical type data.
2025-09-03 07:26:36 -07:00
chrislu 900bd94456 address some comments 2025-09-02 16:54:34 -07:00
chrislu 586a795b02 return fast on error 2025-09-02 14:09:38 -07:00
chrislu c9e093194d setup integration test for postgres 2025-09-02 10:51:37 -07:00