* iceberg maintenance: an unreadable parquet file fails the job, not the process
parquet.NewReader panics on any file it cannot open (wrong magic, a
truncated footer, zero bytes), and parquet-go's row decoder panics on a
corrupt page under a valid footer. Maintenance runs in the worker's
goroutines with no recover, so one bad delete or data file in any table
ended the whole process.
- Every parquet open in compaction goes through newParquetReader, which
opens with parquet.OpenFile and returns the error.
- A panic while decoding a delete file or a data file's rows is recovered
into an error naming the file (recoverRead/guardRead).
- As a backstop, detection recovers per table and Execute per operation:
the table is skipped or the operation fails, and the worker keeps
running.
A failed delete file still fails the table's compaction (no commit), as a
read error did before; a failed data file still skips only its bin.
* iceberg maintenance: test a corrupt page inside a merge fails it naming the file
* iceberg maintenance: assert a skipped bin's files stay in the snapshot
---------
Co-authored-by: user.email <chrislusf@users.noreply.github.com>
* deps: upgrade parquet-go to v0.32.0
Iceberg compaction writes the merged file with the schema of its first
input, encodings included. parquet-go before v0.31.0 took the deprecated
PLAIN_DICTIONARY encoding that DuckDB writes at face value and encoded
those pages as plain int32 indices, but the spec gives PLAIN_DICTIONARY
the same bit-width-prefixed RLE layout as RLE_DICTIONARY. Every
dictionary-encoded column in a compacted file then decoded onto a single
dictionary entry, and anything past one page failed to decode at all.
* iceberg: cover compaction of dictionary-encoded input
The fixture is a DuckDB-written file, so it carries the PLAIN_DICTIONARY
encoding a Go writer will not produce.
* iceberg: tally whole rows in the dictionary merge test
Counting each column on its own passes a merge that remaps names while
leaving their cardinality intact.
