mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-07 12:37:41 +01:00
2fe480a6e2a7cf8de199d12a9bfdd1cfdc6caaaf
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
206d7b1418 |
filer.remote.sync: bounded filer lookups, one chunk cache per process (#12441)
* filer.remote.sync: bounded filer lookups, one chunk cache per process The filer lookups on the sync path (currentEntry, isSuperseded, uploadCurrentEntry, liveRemoteEntry, the mount directory read) ran with no deadline. A lookup that never answers keeps its job alive, the job keeps its path in the MetadataProcessor conflict index, AddSyncJob waits on the condition and the daemon stops consuming events with no log line. Each lookup now carries a 30 s deadline, so the job fails and the processor's stop-and-resubscribe path runs. remoteUploadReader built a chunk cache per upload; the cache's worker goroutine is never stopped, so each upload leaked one goroutine and its cache (21k goroutines, ~150 MB/h under lock-file churn, OOM at the memory limit). One chunk cache per process. (cherry picked from commit ff77fd222772d4f3b68f34d5ee39a66684f90e78) * filer.remote.sync: a failed mount lookup at startup never resolves to now collectLastSyncOffset returned time.Now() when the mounted directory lookup failed, which skips every event since the saved offset. The saved offset is used instead; without one the daemon exits so a restart retries. (cherry picked from commit 8fd3124f5b3d3f8c3a291263a710047b4aff81ae) * filer.remote.sync: no upload on an unanswered supersession lookup; bounded offset read retriedWriteFile returns the supersession lookup's error instead of treating an unanswered lookup as "not superseded": in a replay an older snapshot could otherwise overwrite a newer remote object. The processor retries the event. isSuperseded keeps its boolean form for the other call sites. GetSyncOffset's KvGet carries a 30 s deadline so a startup that falls back to the saved offset cannot hang on it. TestGatewayUncachedRewriteIsStillUploaded asserted the upload on a failed lookup; it now asserts the withheld upload and the lookup error. (cherry picked from commit 8fc2e718f7cd8322615b4e40007be903ade2dbfa) * filer.remote.sync test: keep the slices import the other gateway tests use * filer.remote.sync: an unanswered lookup after a failed attempt ends the retry loop; an unread offset never resolves to now minus timeAgo After a failed upload attempt the supersession lookup's error ended in isSuperseded's boolean and the loop could upload the snapshot again unverified. The lookup error now ends the loop (errLookupUnanswered, not retried) and the processor retries the event. With -timeAgo set, a failed read of the saved offset resolved to now minus timeAgo, skipping everything older; the daemon exits instead so a restart retries. (cherry picked from commit 55ca2a88e01d7b556d494a404b9cc263c27b2566) * filer.remote.sync: an unresolved start offset returns to the retry loop instead of exiting resolveLastSyncOffset returns the lookup or offset-read error; both callers (directory and buckets) hand it to util.RetryUntil, which retries with backoff instead of the process exiting. * filer.remote.sync: keep doc comments on the functions they describe Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit |
||
|
Chris LuandDevin
|
2f6c237238 |
filer: keep lazy remote reads from resurrecting deleted paths (#11452)
* filer: keep lazy remote reads from resurrecting deleted paths Under a remote mount with filer.remote.sync as write-back, a path that was deleted or renamed away could come back as a chunkless remote-only entry: between the local delete and the daemon's remote delete, a store miss made maybeLazyFetchFromRemote trust a bucket that was behind the filer. The ghost then outlived the remote object -- HEAD answered 200, GET failed, and nothing cleaned it up. The filer now tombstones paths it deletes under a remote mount, learned both synchronously from its own delete path and from peer metadata events. The lazy fetch and the lazy listing skip a tombstoned path until the path is written again, until the mount's persisted write-back sync offset has passed the delete event (the remote delete has landed), or until a generous TTL covers a mount without a daemon. Fixes #11440 Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: cover recursive remote deletes with an ancestor tombstone A recursive delete now records the directory tombstone before walking children, so a partial traversal or a store that drops the subtree without listing it still leaves every descendant covered. Directory tombstones also subsume older descendant entries on add, descendant adds covered by a standing ancestor are skipped, and an existing tombstone can be refreshed even at capacity. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: scope remote tombstones to the deleted object's generation A remote object whose own mtime postdates the local delete is a new generation, not the one the tombstone hides, so a recreated directory can surface remote writes made after its delete while old-generation objects stay hidden. Lazy fetch now stats the remote object before deciding, listings pass each child's remote mtime, and a sync offset releases a tombstone once it reaches the delete's own timestamp. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: rebuild remote deletion tombstones after restart In-memory tombstones are lost on restart while remote write-back offsets persist, so a filer boot replays the persisted metadata log from the oldest mount offset and folds deletes back into the tombstone set through the same event handler. Lazy remote reads hold off while the replay runs so a pending delete cannot resurrect in the gap. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: release remote tombstones only after their delete event lands The write-back offset orders against event timestamps, but the synchronous delete path recorded tombstones with the local clock before its event was emitted — a later unrelated event could already have pushed the mount's watermark past that guess, releasing the tombstone before the daemon applied the delete. Tombstones recorded ahead of their event are now marked pending and can only be lifted by the event confirming them or by TTL; event-stamped tombstones release through the offset as before. The remote-mtime generation bypass is dropped: remote and filer clocks are independent, and a pending remote delete removes whatever object sits at the path, so a "newer" remote object would only resurrect as a phantom. Tombstoned lookups now skip the remote stat entirely. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: drop dir tombstone when recursive delete fails before listing The ancestor tombstone is recorded before the child listing; if that listing fails nothing was deleted, and the leftover tombstone would hide still-existing remote children for the whole TTL. Tombstones for children already deleted stay, since their remote deletes are still owed. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: block lazy remote reads on startup tombstone rebuild The rebuild gate is now a done-channel set synchronously before the replay goroutine starts, so no lazy read can slip through in between. Reads wait on it with context cancellation instead of returning an empty miss that makes remote-only objects look deleted. The replay start is floored at now-TTL: mounts without a recorded write-back offset previously replayed the whole persisted history, and events older than the TTL would only build already-expired tombstones. The gate check now runs after the mount lookup so replaying the meta log's own directory listings does not deadlock on the gate, and the replay retries with backoff until it succeeds instead of failing open. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: mark restamped tombstone pending until its delete event lands When a local delete raises an existing tombstone's timestamp, the new value is only a local clock guess ahead of that delete's event. Leaving the tombstone un-pending lets a write-back offset release it before the event is actually consumed, reopening the resurrection window. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: bound tombstone replay to the tombstone TTL Persisted-log replay retried forever, keeping lazy remote reads gated indefinitely when the log cannot be read. Cap retries at the tombstone TTL measured from replay start: past that point every tombstone would have expired anyway, so opening the gate loses no protection. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: re-check deletion tombstone before persisting lazy fetch A delete landing while StatFile is in flight passed the earlier tombstone check but still persisted the fetched entry, resurrecting a path whose remote delete is pending. Re-check right before CreateEntry. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: retract a lazily persisted entry when a delete raced the insert The pre-insert tombstone check still leaves a window between the check and the store insert. Since deletes always record the tombstone before removing the entry, a tombstone visible right after a successful insert means the delete already ran: delete the entry back out so the tombstoned path stays deleted. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: note why the replay deadline can safely open the gate Deletes made after startup are captured by the live delete and event paths, so a stalled replay can only be missing pre-restart deletes, all of which are past the tombstone TTL by the deadline. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: retract only the entry a lazy remote read materialized Deleting by path after a raced delete could remove a legitimate rewrite that replaced the fetched entry. Verify the stored entry still matches the remote object (or the just-created directory shape) before deleting, and apply the same post-insert check to lazy listing children. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * filer: require full-entry equality before retracting a lazy entry Remote-only matching still removed a write that had updated the fetched entry, e.g. appended chunks. Compare the persisted entry against what this read materialized; any change means a real update owns the path. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
promalertandChris Lu
|
9012069bd7 |
chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code Signed-off-by: promalert <promalert@outlook.com> * goimports -w . --------- Signed-off-by: promalert <promalert@outlook.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
chrislu
|
81fdf3651b | grpc connection to filer add sw-client-id header | ||
|
chrislu
|
26dbc6c905 | move to https://github.com/seaweedfs/seaweedfs | ||
|
chrislu
|
9f9ef1340c |
use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed. |
||
|
Chris Lu
|
e5fc35ed0c | change server address from string to a type | ||
|
Chris Lu
|
3bd48c4f29 |
filer.remote.sync: exit when directory is unmounted
this will not propagate the deletions back to the cloud |