Files
seaweedfs/weed/mq/kafka/integration/broker_client_test.go
T
fcd28031e7 mq/kafka: wait for a filer to register instead of panicking on an empty list (#12019)
* mq/kafka: wait for a filer to register instead of panicking on an empty list

NewSeaweedMQBrokerHandler queried the masters for filers once, ~2 s after
start, and handed an empty list to NewFilerClientAccessor, which panics. A
fresh filer registers only after its empty-membership confirm window
(10 s), so a gateway started with a fresh cluster crashed (Kafka Quick
Test, kafka-gateway exit 2). Retry while the list is empty for up to 60 s,
then return an error.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* mq/kafka: wait out a transient-empty broker listing too

A fresh SMQ broker registers with the masters on the same announce
delay as a filer, so the one-shot broker lookup could fail startup the
same way. Generalize waitForFilers into waitForClusterNodes and bound
both discoveries by the same window; the empty-broker check becomes
unreachable and drops away.

* mq/kafka: wait out the broker health check like node discovery

A broker announces to the masters before it has acquired its balancer
lock, so the gateway's three-attempt probe could still fail fast on a
fresh cluster. Retry FindBrokerLeader for the same window the caller
waits out empty listings.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 07:09:43 -07:00

47 lines
1.4 KiB
Go

package integration
import (
"context"
"errors"
"testing"
"time"
"google.golang.org/grpc"
"github.com/seaweedfs/seaweedfs/weed/pb/mq_pb"
)
type stubMessagingClient struct {
mq_pb.SeaweedMessagingClient
calls int
failFor int
err error
}
func (s *stubMessagingClient) FindBrokerLeader(ctx context.Context, in *mq_pb.FindBrokerLeaderRequest, opts ...grpc.CallOption) (*mq_pb.FindBrokerLeaderResponse, error) {
s.calls++
if s.calls <= s.failFor {
return nil, s.err
}
return &mq_pb.FindBrokerLeaderResponse{}, nil
}
func TestHealthCheckRetriesPastTransientBrokerErrors(t *testing.T) {
stub := &stubMessagingClient{failFor: 2, err: errors.New("lock broker_balancer not found")}
bc := &BrokerClient{ctx: context.Background(), client: stub}
if err := bc.HealthCheck(); err != nil || stub.calls != 3 {
t.Fatalf("err %v after %d calls, want success on the 3rd", err, stub.calls)
}
}
func TestHealthCheckGivesUpAtDeadline(t *testing.T) {
defer func(orig time.Duration) { clusterNodeDiscoveryTimeout = orig }(clusterNodeDiscoveryTimeout)
clusterNodeDiscoveryTimeout = 300 * time.Millisecond
stub := &stubMessagingClient{failFor: 1 << 30, err: errors.New("lock broker_balancer not found")}
bc := &BrokerClient{ctx: context.Background(), client: stub}
if err := bc.HealthCheck(); err == nil || stub.calls < 2 {
t.Fatalf("err %v after %d calls, want repeated retries then an error", err, stub.calls)
}
}