* s3tables: read Iceberg manifest lists that omit the Avro format version
The Iceberg spec pins the Avro header metadata of manifest files but says
nothing about manifest lists, so writers disagree. Java and PyIceberg record
"format-version"; DuckDB writes no header metadata at all. iceberg-go reads a
missing entry as v1, so every v2 manifest listed in a DuckDB-written list is
rejected with
manifest file's 'format-version' metadata indicates version 2,
but entry from manifest list indicates version 1
and, because v1 has no "content" field, delete manifests silently decode as
data manifests.
ReadManifestList derives the version from the record schema the writer
embedded - v2 added "content" and the sequence numbers, v3 added
"first_row_id" - and splices it into the header before handing the bytes to
iceberg-go. Lists that already carry the entry, and input that is not a
parseable Avro container, go through untouched.
* iceberg: parse DuckDB-written manifest lists in maintenance and data preview
Every manifest list read - the four maintenance operations and the admin
table data preview - went straight to iceberg-go, so tables written by DuckDB
failed detection and all of compact, remove_orphans, rewrite_manifests and
expire_snapshots before they touched anything. Route them through
s3tables.ReadManifestList, which recovers the format version the writer left
out of the Avro header.
This also restores the manifest content type on those tables: with the list
read as v1 every delete manifest looked like a data manifest, which hid
deletes from the compaction guard and made the preview report a table with
position deletes as having none.
* admin: move volume-server read JWT helper into dash
The Iceberg data preview page needs the same per-fileId read token the
file browser uses when streaming chunks from volume servers.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
* admin: add Iceberg table data preview page
The admin UI browses the Iceberg catalog down to table details but not
the data itself. Add a Browse Data page per table that walks the
selected snapshot's manifests and shows sample rows from its Parquet
data files, plus the data file list with per-file preview, a snapshot
switcher, and a row limit selector.
Rows are read through a ranged ReaderAt over stream-content so only
the Parquet footer and needed pages are fetched, with the volume read
JWT applied when configured. Iceberg locations resolve into /buckets
with traversal guards, and the file parameter must match a
manifest-listed data file. Snapshots with delete files get a warning
that raw rows are shown.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
* admin: integration test for Iceberg catalog and data preview pages
Starts a weed mini cluster with the admin UI, creates a table bucket,
namespace, and tables via the S3 Tables manager, uploads real Parquet
files via S3, writes manifests and snapshots with iceberg-go, and
asserts on the rendered pages: catalog browsing, table details,
current and historical snapshot previews, per-file preview, row
limits, unknown snapshot and file errors, and a metadata-less table.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
* admin: write Iceberg preview chunk reads straight into the caller slice
ReadAt wrapped the caller's buffer in a bytes.Buffer, which would
silently allocate a fresh backing array and drop bytes if it ever grew.
Copy directly into the destination slice and reject negative offsets so
the ReaderAt contract holds.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
* admin: link to snapshot history when the preview switcher truncates
The snapshot switcher caps at 25 entries; add a trailing item pointing
at the table details page so older snapshots stay reachable.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
* test: hoist mini cluster context assignment out of the goroutine
Set MiniClusterCtx before launching the cluster goroutine and clear it
in stop(), so the assignment is not buried in the command loop.
Claude-Session: https://claude.ai/code/session_015n3oKLTjnPjcnZtfigNKur
Chris Lu