mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-31 09:20:50 +01:00
fix: Add topic cache invalidation and auto-creation on metadata requests
Add InvalidateTopicExistsCache method to SeaweedMQHandlerInterface and impl ement cache refresh logic in metadata response handler. When a consumer requests metadata for a topic that doesn't appear in the cache (but was just created by a producer), force a fresh broker check and auto-create the topic if needed with default partitions. This fix attempts to address the consumer stalling issue by: 1. Invalidating stale cache entries before checking broker 2. Automatically creating topics on metadata requests (like Kafka's auto.create.topics.enable=true) 3. Returning topics to consumers more reliably However, testing shows consumers still can't find topics even after creation, suggesting a deeper issue with topic persistence or broker client communication. Added InvalidateTopicExistsCache to mock handler as no-op for testing. Note: Integration testing reveals that consumers get 'topic does not exist' errors even when producers successfully create topics. This suggests the real issue is either: - Topics created by producers aren't visible to broker client queries - Broker client TopicExists() doesn't work correctly - There's a race condition in topic creation/registration Requires further investigation of broker client implementation and SMQ topic persistence logic.
This commit is contained in:
1 parent
3c1df04c9a
commit
a5db489f82
4 files changed
+27
-3
No files matched your search
@@ -107,8 +107,6 @@ func TestEmptyFetchShouldNotStopConsumer(t *testing.T) {
|
||||
nextFetchOffset := committedOffset + 1
|
||||
|
||||
// First attempt: get empty (transient - data might not be available yet)
|
||||
firstFetchResult := 0 // bytes returned
|
||||
|
||||
// WRONG behavior (bug): Consumer sees 0 bytes and stops
|
||||
// wrongConsumerLogic := (firstFetchResult == 0) // gives up!
|
||||
|
||||
|
||||
Reference in new issue
Block a user