mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-29 16:30:44 +01:00
Javier de la Torreanduser.email
e9c280e095
* iceberg maintenance: bound decimal, time, uuid, fixed and rescaled timestamp columns on compacted files The column statistics written for a compacted data file give lower and upper bounds only for booleans, integers, floats, dates, strings, binaries and timestamps already stored in the Iceberg type's unit. Every other column is left unbounded, so a reader can never skip a compacted file on it: a DECIMAL(18,2) column written by DuckDB (stored as INT64) loses its bounds after compaction, as does a millisecond timestamp, a TIME or a UUID. boundLiteral now also converts: - DECIMAL stored as INT32 or INT64 at the column's scale. A decimal stored as a byte array still gets no bound: Parquet compares those statistics as bytes, which does not order negative two's-complement values; - TIME, TIMESTAMP, TIMESTAMPTZ and TIMESTAMP_NS from any Parquet time unit, rescaled to the Iceberg unit; a value rescaled to a coarser unit is rounded down for a lower bound and up for an upper bound so it stays a bound, and a value that would overflow gets none; - UUID and FIXED[n] from a FIXED_LEN_BYTE_ARRAY of the right length; - a DOUBLE column read from a FLOAT file. arrow-go becomes a direct dependency (decimal128, already in the module graph through iceberg-go). * iceberg maintenance: bound TIME(MILLIS) columns stored as INT32 * iceberg maintenance: bound timestamptz_ns columns too --------- Co-authored-by: user.email <chrislusf@users.noreply.github.com>