Commit Graph
2 Commits
Author SHA1 Message Date
81c90cb446 filer.backup: retry a failed checkpoint read instead of replaying from the epoch (#12036)
* filer.backup: retry a failed checkpoint read instead of replaying from the epoch

Fixes #12035

* test: reset sink.local.directory in backup offset test cleanup

The new test sets sink.local.directory on the shared viper but only
reset the enabled flag, leaking a key that points at a deleted
t.TempDir() to later tests in the package.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: cover the retried attempt re-reading the checkpoint

The fake filer now fails only its first KvGet, and the test runs a
second doFilerBackup attempt asserting the checkpoint is re-read and
the attempt proceeds past the offset stage. LookupDirectoryEntry is
stubbed so the SSE-S3 KEK probe returns a miss instead of retrying
10x2s.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: user.email <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-24 08:22:29 -07:00
Chris Lu b3be2f5449 filer.backup, filer.sync: stop sharing resume checkpoints across destinations (#10934)
* filer.backup: key the checkpoint by source path and sink destination

The checkpoint id hashed only sink name + directory, so two backups to
different buckets or endpoints sharing a directory layout advanced one
checkpoint: whichever job was running pushed the shared offset forward,
and a stopped or failing job later resumed from the other's position,
silently skipping changes. Backups of different source paths to the same
destination shared a checkpoint the same way.

Each sink now reports a destination identity (endpoint or account,
bucket or container, directory) and the checkpoint is keyed by the
source path plus that identity. Reads fall back to the historical
name+directory key when the new key has no value, so existing backups
resume where they left off; writes go only to the new key.

* filer.sync: include the target path in the offset key

The offset stored on the target filer was keyed by source path and
source filer signature only, so two syncs from the same source cluster
and path to different directories on the same target cluster advanced
one shared checkpoint, and the slower one could resume past events it
never applied. The target path now participates in the key; "/" keeps
the historical form, and a sync with a non-root target path falls back
to the historical key once when its own key has no value yet.

* join checkpoint key fields with NUL so they cannot alias

A path or configuration value spelling out the separator could
concatenate two different field tuples to the same checkpoint key.
NUL cannot appear in a CLI path argument or any sane configuration
value, making the encoding injective.
2026-08-24 19:30:20 -07:00