Files
seaweedfs/weed
chrislu 6c19e548d3 feat: implement working Kafka consumer functionality with stored record batches
- Fixed Produce v2+ handler to properly store messages in ledger and update high water mark
- Added record batch storage system to cache actual Produce record batches
- Modified Fetch handler to return stored record batches instead of synthetic ones
- Consumers can now successfully fetch and decode messages with correct CRC validation
- Sarama consumer successfully consumes messages (1/3 working, investigating offset handling)

Key improvements:
- Produce handler now calls AssignOffsets() and AppendRecord() correctly
- High water mark properly updates from 0 → 1 → 2 → 3
- Record batches stored during Produce and retrieved during Fetch
- CRC validation passes because we return exact same record batch data
- Debug logging shows 'Using stored record batch for offset X'

TODO: Fix consumer offset handling when fetchOffset == highWaterMark
2025-09-12 13:20:33 -07:00
..
2025-08-09 21:47:29 -07:00
2025-07-19 23:56:17 -07:00
2025-08-30 11:15:48 -07:00
2025-08-31 23:31:28 -07:00
2025-08-22 01:15:42 -07:00
2025-09-10 11:04:42 -07:00
2025-08-30 11:15:48 -07:00
2024-06-25 09:18:11 -07:00
2025-08-30 11:15:48 -07:00
2024-02-14 08:26:38 -08:00
2025-07-19 21:43:34 -07:00
2025-03-17 23:13:27 -07:00