chrislu 94615996ed workaround: increase Spark task retries for eventual consistency ...

Issue: EOF exceptions when reading immediately after write
- Files appear truncated by ~78 bytes on first read
- SeaweedOutputStream.close() does wait for all chunks via Future.get()
- But distributed file systems can have eventual consistency delays

Workaround:
- Increase spark.task.maxFailures from default 1 to 4
- Allows Spark to automatically retry failed read tasks
- If file becomes consistent after 1-2 seconds, retry succeeds

This is a pragmatic solution for testing. The proper fix would be:
1. Ensure SeaweedOutputStream.close() waits for volume server acknowledgment
2. Or add explicit sync/flush mechanism in SeaweedFS client
3. Or investigate if metadata is updated before data is fully committed

For CI tests, automatic retries should mask the consistency delay.

2025-11-22 23:48:22 -08:00

..

spark

workaround: increase Spark task retries for eventual consistency

2025-11-22 23:48:22 -08:00