Sniff identifies TS assets for the coming policy-driven alignment even
though nothing reaches it through repack today. Compile-time assertions
now state each adapter's capabilities explicitly.
Sniff sat in the mandatory adapter interface but has exactly one
caller, the repack gate, which only Indexer adapters can reach - the
hls-ts implementation was dead code. Move it to a Sniffer capability
discovered by assertion like the others: parquet keeps it, hls-ts
drops it, and repack skips the gate when an adapter cannot sniff.
Ingest admits mediaSequence up to MaxInt64-(count-1), but the payload
decoder rejected anything above MaxInt64-count, so a boundary playlist
ingested successfully and then failed every view. Mirror the ingest
bound, covered by a round-trip test at the boundary.
- namespace the query parameters as format.ingest, format.repack and
format.view, following the mv.from/cp.from dotted convention, so the
general endpoints cannot collide with pass-through client parameters;
requests naming both ingest and repack are rejected
- state Accept-Ranges: none on view responses, which always answer with
whole documents or whole extents
- derive the small-content permission from the boundary source instead
of a second positional bool that a call site could silently swap
- validate the hls-ts layout before returning it, making the formattest
invariant enforced rather than emergent
DecodeLayout rejected names over 256 bytes while Encode accepted them,
so an oversized name encoded fine and then failed every decode. Enforce
the bound in Validate, shared by both directions.
The cutter materialized every interior cut up front, so a tiny sidecar
declaring one enormous extent could allocate gigabytes of cut offsets
before any media byte arrived. Keep only the extent start offsets,
bounded by the extent count, and derive each cut arithmetically.
The footer already names every row-group byte range, so the adapter
only reads metadata: one extent per row group, the leading magic riding
with the first, and a trailing extent for the page indexes and footer.
Engines that fetch row groups by offset then read exactly the covering
chunks. Parquet needs no view; alignment alone delivers the benefit.
The ingest sidecar is an FFmpeg-style EXT-X-BYTERANGE media playlist;
its segments become the extents and 188 becomes the align quantum, so
every storage chunk holds whole TS packets of one segment. The view
renders a playback playlist with plain numbered segment URLs for
clients that do not speak byte-range HLS, and maps ?seq=N to the
segment's extent. Playlist state the generated playlist cannot
reproduce (EXT-X-KEY, MAP, DISCONTINUITY, GAP, I-frame-only) is
rejected at ingest.
A format adapter reduces one container format to three things the core
understands: extent sizes, an alignment quantum, and an opaque payload.
Capabilities beyond identity (Indexer, SidecarIndexer, Viewer) are
discovered by type assertion. The layout persists in one compact
extended attribute keyed by extent sizes rather than chunk ids, so it
survives chunk manifest folding, and the Cutter turns it into upload
chunk boundaries clamped by maxMB and the align quantum. The formattest
kit holds every adapter to no-panic parsing of truncated input.
Chris Lu