mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-30 17:00:48 +01:00
Javier de la Torreanduser.email
edf2b0f5ee
* iceberg maintenance: an unreadable parquet file fails the job, not the process parquet.NewReader panics on any file it cannot open (wrong magic, a truncated footer, zero bytes), and parquet-go's row decoder panics on a corrupt page under a valid footer. Maintenance runs in the worker's goroutines with no recover, so one bad delete or data file in any table ended the whole process. - Every parquet open in compaction goes through newParquetReader, which opens with parquet.OpenFile and returns the error. - A panic while decoding a delete file or a data file's rows is recovered into an error naming the file (recoverRead/guardRead). - As a backstop, detection recovers per table and Execute per operation: the table is skipped or the operation fails, and the worker keeps running. A failed delete file still fails the table's compaction (no commit), as a read error did before; a failed data file still skips only its bin. * iceberg maintenance: test a corrupt page inside a merge fails it naming the file * iceberg maintenance: assert a skipped bin's files stay in the snapshot --------- Co-authored-by: user.email <chrislusf@users.noreply.github.com>
29 lines
968 B
Go
29 lines
968 B
Go
package iceberg
|
|
|
|
import (
|
|
"fmt"
|
|
"runtime/debug"
|
|
|
|
"github.com/seaweedfs/seaweedfs/weed/glog"
|
|
)
|
|
|
|
// recoverRead turns a panic into *errp, naming what was being read (a file,
|
|
// or a table operation).
|
|
// parquet-go panics on more than a file it cannot open: a corrupt page under
|
|
// a valid footer panics in its row decoder. Nothing above a maintenance job
|
|
// recovers, so such a panic would end the process hosting the worker; with
|
|
// this the job fails for that file instead. Use as
|
|
// `defer recoverRead(&err, what)` in a function with a named error result.
|
|
func recoverRead(errp *error, what string) {
|
|
if r := recover(); r != nil {
|
|
glog.Errorf("iceberg maintenance: recovered panic in %s: %v\n%s", what, r, debug.Stack())
|
|
*errp = fmt.Errorf("%s: panic: %v", what, r)
|
|
}
|
|
}
|
|
|
|
// guardRead runs fn, turning a panic into an error naming what.
|
|
func guardRead[T any](what string, fn func() (T, error)) (v T, err error) {
|
|
defer recoverRead(&err, what)
|
|
return fn()
|
|
}
|