* filer.remote.sync: bounded filer lookups, one chunk cache per process
The filer lookups on the sync path (currentEntry, isSuperseded,
uploadCurrentEntry, liveRemoteEntry, the mount directory read) ran with no
deadline. A lookup that never answers keeps its job alive, the job keeps
its path in the MetadataProcessor conflict index, AddSyncJob waits on the
condition and the daemon stops consuming events with no log line. Each
lookup now carries a 30 s deadline, so the job fails and the processor's
stop-and-resubscribe path runs.
remoteUploadReader built a chunk cache per upload; the cache's worker
goroutine is never stopped, so each upload leaked one goroutine and its
cache (21k goroutines, ~150 MB/h under lock-file churn, OOM at the memory
limit). One chunk cache per process.
(cherry picked from commit ff77fd222772d4f3b68f34d5ee39a66684f90e78)
* filer.remote.sync: a failed mount lookup at startup never resolves to now
collectLastSyncOffset returned time.Now() when the mounted directory lookup
failed, which skips every event since the saved offset. The saved offset is
used instead; without one the daemon exits so a restart retries.
(cherry picked from commit 8fd3124f5b3d3f8c3a291263a710047b4aff81ae)
* filer.remote.sync: no upload on an unanswered supersession lookup; bounded offset read
retriedWriteFile returns the supersession lookup's error instead of
treating an unanswered lookup as "not superseded": in a replay an older
snapshot could otherwise overwrite a newer remote object. The processor
retries the event. isSuperseded keeps its boolean form for the other call
sites. GetSyncOffset's KvGet carries a 30 s deadline so a startup that falls
back to the saved offset cannot hang on it.
TestGatewayUncachedRewriteIsStillUploaded asserted the upload on a failed
lookup; it now asserts the withheld upload and the lookup error.
(cherry picked from commit 8fc2e718f7cd8322615b4e40007be903ade2dbfa)
* filer.remote.sync test: keep the slices import the other gateway tests use
* filer.remote.sync: an unanswered lookup after a failed attempt ends the retry loop; an unread offset never resolves to now minus timeAgo
After a failed upload attempt the supersession lookup's error ended in
isSuperseded's boolean and the loop could upload the snapshot again
unverified. The lookup error now ends the loop (errLookupUnanswered, not
retried) and the processor retries the event. With -timeAgo set, a failed
read of the saved offset resolved to now minus timeAgo, skipping everything
older; the daemon exits instead so a restart retries.
(cherry picked from commit 55ca2a88e01d7b556d494a404b9cc263c27b2566)
* filer.remote.sync: an unresolved start offset returns to the retry loop instead of exiting
resolveLastSyncOffset returns the lookup or offset-read error; both callers
(directory and buckets) hand it to util.RetryUntil, which retries with
backoff instead of the process exiting.
* filer.remote.sync: keep doc comments on the functions they describe
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 952d80c4d3)
* filer.remote.sync: isSuperseded returns the lookup error itself
Its only production callers needed the error, so the boolean wrapper was
left to the tests. The tests now assert the error on a failed lookup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 51f603b5ec)
* filer.remote.sync: bound the stamp written after each upload
updateLocalEntry's UpdateEntry was the last filer call on the sync path
without a deadline. A filer that never answers the stamp keeps the job and
its path in the conflict index the same way an unanswered lookup did. The
stamp carries IF_ENTRY_EQUAL, so a retried event after a timeout either
stamps again or skips a stamp that already landed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c7338fb68f)
* filer.remote.sync: an unread offset never resolves to the mount's creation time
With the offset read now bounded, a timed-out read fell through to the
directory's creation time and replayed every event already synced. The
error goes back to the startup retry loop, as with -timeAgo; a missing
offset still reads as zero without an error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 0a1b6c2944)
* filer.remote.sync: stamp a copy of the event entry
updateLocalEntry set the RemoteEntry and dropped the rename marker on the
caller's entry before the RPC. A transient stamp failure retries the same
event in place, which then read as already synced and skipped both the
upload and the stamp. The stub filer now applies a successful update the
way the filer does, since tests had relied on that shared object.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1f332e5866)
* filer.remote.sync: a directory not created yet starts now, as before
A gateway started before the first bucket looked up a missing buckets
directory, had no saved offset, and returned an error to the retry loop
on every attempt until a bucket appeared. A missing directory with no
saved offset has nothing to replay, so it starts now as it did before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c31497a9a3)
* filer.remote.sync: take the start time before the mount lookup
A bucket created between a not-found lookup and the end of the offset
read had its creation before a start taken afterwards, so the gateway
never created its remote bucket.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit f535fca8f6)
* filer.remote.sync: save the resume position when a subscription ends
A failed event ends the subscription with ErrResubscribe, and the retry
loop runs startup again, recomputing -timeAgo from a later now. The
periodic checkpoint only runs as events arrive, so a failure before the
first one could start the retry after the event it failed on. The resume
position is now saved whenever the subscription returns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7be919ded1)
* remote_storage: bound the sync offset write like the read
The offset write runs in the subscription's checkpoint callback and, now,
when a subscription ends; a filer that never answers it stalled either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit f888ac6b8f)
* filer.remote.sync: say that any saved offset wins over the creation time
The creation time in seconds was compared with the offset in
milliseconds, which made any saved offset win. That is the behaviour to
keep: remote.unmount recreates the directory and then saves the offset,
usually in the same second, so a seconds comparison would start at the
creation second and replay the purge. The comparison now states it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit a0d60ee00f)
* filer.remote.sync: keep the resume position for the retry when its write fails
The retry in the same process now resumes from the position the ended
subscription returned, as filer.backup does, so a failed offset write
followed by an empty read cannot recompute -timeAgo past a failed event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 39ef551bcf)
* filer.remote.sync test: give the same-second saved offset a fractional part
With whole seconds, returning the creation second resolved to the same
time and passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4bb74382a6)
---------
Co-authored-by: james <james@medable.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add GitHub Actions workflow for codespell on master
* Add rudimentary codespell config
* Tune codespell config: skip generated code, ignore camelCase, whitelist domain terms
Add camelCase/PascalCase regex to ignore common Go/Rust/JS identifiers
like allLocations, publishErr, ReadInside, FlushInterval. Also skip
templ-generated *_templ.go files, and whitelist a handful of
short/domain-specific words (visibles, fo, te, ser, bject, unparseable,
keep-alives, tread, anc, ue) that show up as false positives across the
tree.
Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix ambiguous typos and protect false positives
Fixes typos that codespell reports with multiple candidate suggestions
(so `codespell -w` cannot auto-apply them), plus one inline pragma and
one config entry to protect legitimate identifiers.
Manual fixes (single correct answer chosen from context):
- pattens -> patterns (5x) in filer/upload/shell flag help strings
- finded -> found (2x) in tarantool storage.lua comment
- spacify -> specify (2x) in helm chart values.yaml comment
- wether -> whether in skiplist.go docstring
- simpe -> simple in mq schema test case name
False-positive protection:
- Add `//codespell:ignore` next to `source GET's` (possessive of HTTP
verb) in s3api_object_handlers_copy_stream.go
- Whitelist `auther` in .codespellrc — it's a local variable meaning
"authenticator" in weed/security/tls.go, not a typo of "author".
Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Extend codespell ignore list: .git-meta path and thirdparty groupId
Also skip `.git-meta` (scratch dir for commit messages that may contain
typo words verbatim) and whitelist `thirdparty` — it appears as the
literal Maven groupId `org.apache.hadoop.thirdparty` in hdfs3 poms
and cannot be renamed.
Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [DATALAD RUNCMD] Fix non-ambiguous typos with codespell -w
Auto-applied fixes to the 44 remaining single-suggestion typos across
docs, comments, log messages, tests, config, and one Java pom.
=== Do not change lines below ===
{
"chain": [],
"cmd": "uvx codespell -w",
"exit": 0,
"extra_inputs": [],
"inputs": [],
"outputs": [],
"pwd": "."
}
^^^ Do not change lines above ^^^
* Revert breaking codespell fixes; whitelist unknwon and atleast
Two of the auto-applied `codespell -w` fixes were false positives that
would break the build/tests:
- go.mod: `github.com/unknwon/goconfig` is a real Go module path — the
upstream author's GitHub handle is literally `unknwon`. Renaming to
`unknown` would fail dependency resolution.
- test/benchmark/fuse_db/bin/{sqlite_verify.py,run_mysql.sh,run_sqlite.sh}:
`atleast` is a literal CLI mode value (a string constant compared and
passed as a positional argument). Rewriting to `at least` splits it
into two arguments and breaks the mode check.
Reverted those files and whitelisted both words in .codespellrc so
future runs won't re-suggest the same broken fixes.
Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
