mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-09-20 13:30:46 +02:00

Files

T

chrislu 71769da3b4 Phase 3: Fix ApiVersions matrix accuracy and version validation

ApiVersions Matrix Accuracy completed:

## Critical Fixes
- **OffsetFetch API**: Updated advertised from v0-v2 to v0-v5 (MAJOR fix)
  - Implementation already supported v3+ throttle_time_ms and v5+ leader_epoch
  - Clients can now use advanced OffsetFetch features
- **CreateTopics API**: Updated advertised from v0-v4 to v0-v5 (minor fix)
  - Implementation already routed v5 requests to v2+ handler
  - Better client compatibility for v5 CreateTopics requests

## Implementation
- **handleApiVersions()**: Corrected advertised max versions
- **validateAPIVersion()**: Updated validation ranges to match advertisements
- **Consistency**: Eliminated mismatch between advertised vs implemented versions

## Testing & Verification
- **Comprehensive test suite**: 6 new tests in api_versions_test.go
- **Version validation tests**: OffsetFetch v3-v5 and CreateTopics v5 now accepted
- **End-to-end verification**: E2E tests still pass, no regressions
- **API audit documentation**: Complete version matrix in API_VERSION_MATRIX.md

## Impact
- **Client compatibility**: Higher-version clients can now connect properly
- **Feature utilization**: Advanced features like leader epoch, throttle time accessible
- **Protocol compliance**: Advertised versions now match actual implementation
- **Future-proofing**: Clear process for managing API version accuracy

Ready for Phase 4: Consumer group protocol metadata parsing

2025-09-13 13:50:10 -07:00

3.6 KiB

Raw Blame History

Kafka API Version Matrix Audit

Summary

This document audits the advertised API versions in handleApiVersions() against actual implementation support and identifies mismatches that need correction.

Current Status: MISMATCHES FOUND ⚠️

API Version Discrepancies

API Key	API Name	Advertised	Actually Implemented	Status	Action Needed
18	ApiVersions	v0-v3	v0-v3	✅ Match	None
3	Metadata	v0-v7	v0-v7	✅ Match	None
0	Produce	v0-v7	v0-v7	✅ Match	None
1	Fetch	v0-v7	v0-v7	✅ Match	None
2	ListOffsets	v0-v2	v0-v2	✅ Match	None
19	CreateTopics	v0-v4	v0-v5	❌ Mismatch	Update advertised to v0-v5
20	DeleteTopics	v0-v4	v0-v4	✅ Match	None
11	JoinGroup	v0-v7	v0-v7	✅ Match	None
14	SyncGroup	v0-v5	v0-v5	✅ Match	None
8	OffsetCommit	v0-v2	v0-v2	✅ Match	None
9	OffsetFetch	v0-v2	v0-v5+	❌ MAJOR Mismatch	Update advertised to v0-v5
10	FindCoordinator	v0-v2	v0-v2	✅ Match	None
12	Heartbeat	v0-v4	v0-v4	✅ Match	None
13	LeaveGroup	v0-v4	v0-v4	✅ Match	None

Detailed Analysis

1. OffsetFetch API (Key 9) - CRITICAL MISMATCH

Advertised: v0-v2 (max version 2)
Actually Implemented: Up to v5+
- Evidence: buildOffsetFetchResponse() includes if apiVersion >= 5 for leader epoch
- Evidence: if apiVersion >= 3 for throttle time
Impact: Clients may not use advanced features available in v3-v5
Action: Update advertised max version from 2 to 5

2. CreateTopics API (Key 19) - MINOR MISMATCH

Advertised: v0-v4 (max version 4)
Actually Implemented: v0-v5
- Evidence: handleCreateTopics() routes v5 requests to handleCreateTopicsV2Plus()
- Evidence: Tests validate v0-v5 versions
Impact: v5 clients may not connect expecting v5 support
Action: Update advertised max version from 4 to 5

3. Validation vs Advertisement Inconsistency

The validateAPIVersion() function matches the advertised versions, which means it will incorrectly reject valid v3-v5 OffsetFetch requests that the handler can actually process.

Implementation Details

OffsetFetch Version Features:

v0-v2: Basic offset fetch
v3+: Includes throttle_time_ms
v5+: Includes leader_epoch for each partition

CreateTopics Version Features:

v0-v1: Regular array format
v2-v5: Compact array format, tagged fields

Recommendations

Immediate Fix: Update handleApiVersions() to advertise correct max versions
Consistency Check: Update validateAPIVersion() to match the corrected advertised versions
Testing: Verify that higher version clients can successfully connect and use advanced features
Documentation: Update any client guidance to reflect the correct supported versions

Test Verification Needed

After fixes:

Test OffsetFetch v3, v4, v5 with real Kafka clients
Test CreateTopics v5 with real Kafka clients
Verify throttle_time_ms and leader_epoch are correctly populated
Ensure version validation doesn't reject valid higher version requests

Conservative Alternative

If we want to be conservative and not advertise versions we haven't thoroughly tested:

OffsetFetch: Limit to v3 (throttle time support) instead of v5
CreateTopics: Keep at v4 unless v5 is specifically needed

This would still fix the main discrepancy while being more cautious about untested version features.

3.6 KiB Raw Blame History