mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-18 12:08:32 +02:00
b2eefb8a73b8a44c7fcd640bd008ed247148f5f6
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Peter Dodd
|
5f1a742726 |
fix: return NoSuchKey and drop stale entries when a remote-mounted object is gone from the remote (#11680)
A remote-only entry whose object was deleted from the remote storage outside the filer answered GET with 500 and stayed in the filer. The remote's not-found was lost on the way: the backends' ReadFile returned it as an untyped error, so FetchAndWriteNeedle failed with codes.Unknown and nothing downstream could tell it from any other failure. - remote_storage: GCS, S3 (NoSuchKey) and Azure (BlobNotFound) reads return ErrRemoteObjectNotFound. GCS reports a missing bucket the same way as a missing object, so it confirms the bucket with a listing. - volume server and filer: the not-found crosses gRPC as codes.NotFound carrying the sentinel's text, and the filer's cache RPC returns codes.NotFound, which the S3 gateway already maps to NoSuchKey. - s3api: the origin fallback answers NoSuchKey on a confirmed not-found. - filer: a confirmed not-found removes the stale entry, so the lazy remote-metadata cache converges on the remote. Only remote-only files outside .versions and without an active object lock are removed, only if unchanged since the fetch (checked on the object's write owner, under the lock S3 object writes take), and with a metadata-only delete: the filer skips its inline remote delete, and the delete events' entries carry a marker that makes filer.remote.sync and filer.remote.gateway skip their remote delete, while filer.sync still replicates it. The replicated DeleteEntryRequest carries keep_remote_object, so the destination's delete events are marked too. The store drops the marker from every write, so clients cannot plant it. |
||
|
|
1df165d514 |
fix(volume): keep the TTL clock across a vacuum commit instead of rescanning (#11630)
* fix(volume): keep the TTL clock across a vacuum commit instead of rescanning CommitCompact reloads the swapped files while holding dataFileAccessLock, and for a vacuumed TTL volume that reload re-derived lastModifiedTsSeconds by reading every live needle's append timestamp from the .dat: two random reads per needle, with every read of the volume blocked behind them. The in-memory clock is already current at that point. Every write since the volume loaded moved it, and makeupDiff only replays writes that went through that path. Carry it across the reload instead. This also stops an over-budget scan from falling back to the new .dat's mtime and restarting an expiring volume's TTL at the commit. * volume: carry the append watermark as the TTL clock across a vacuum commit The running append watermark is the clock the reload's recovery scan recomputes, so the commit can keep it directly. Client-supplied needle modified times can run ahead of or behind the append time; keeping lastModifiedTsSeconds itself would let a forged or stale timestamp move expiry through a vacuum, where the scan it replaces used server-side append timestamps. * storage: test that a vacuum commit keeps the append clock A write's client supplied modified time can lie ahead of or behind its append time; the commit must land the TTL clock on the append watermark, the same value the recovery scan would have recomputed. * volume: carry the append watermark as the TTL clock across a vacuum commit Mirrors the Go volume server: the running append watermark is the clock the reload's recovery scan recomputes, so the commit keeps it instead of rescanning live needles under the write lock. * volume: commit carries the last-write append time, not the latest append lastAppendAtNs counts tombstone appends and is reseeded from the .dat tail at every load, so it can sit ahead of the last write -- a delete freshens the commit clock -- or behind it: a restarted vacuumed volume's tail needle is not its newest write, and the commit would move the TTL clock backward into premature expiry. Track lastWriteAppendAtNs instead, bumped only on needle appends and seeded by the recovery scan, so the commit lands the clock on the same live-write maximum the rescan would have recomputed. * volume: rescan at commit when the newest write was deleted lastWriteAppendAtNs can hold a write the index no longer holds, so carrying it extends the TTL clock past what recovery over the compacted index would compute. Remember the key behind the watermark so its tombstone or index rollback can send the reload back through recoverLastModifiedTs, landing on the newest surviving write. * volume: a tombstone retires the write rows beneath it in the last-write scan A needle deleted after the compaction copy leaves its write row followed by a tombstone in the committed index. The reverse scan skipped the tombstone row but then counted the dead write, reseeding the watermark and flag as if it were alive. Track keys whose latest row is a tombstone so their earlier write rows stop counting, and exercise the delete-inside-the-commit-window ordering in the tests. --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Peter Dodd
|
8ebef03509 |
helm: expose s3 update strategy, lifecycle, and termination grace period (#11602)
The standalone s3 Deployment hardcoded terminationGracePeriodSeconds and offered no way to set a container lifecycle or the Deployment strategy, so operators could not add a preStop delay to drain endpoints before SIGTERM or hold maxUnavailable at 0 during rollouts. Add s3.updateStrategy, s3.lifecycle and s3.terminationGracePeriodSeconds. Defaults render the same manifest as before. |
||
|
|
c739e5cf78 |
fix(s3): drain in-flight requests on SIGTERM in standalone weed s3 (#11603)
Standalone weed s3 only shut its servers down when shutdownCtx was set, which only weed mini does. On SIGTERM the interrupt hooks ran and the process exited with requests still in flight, so clients saw connection resets during rolling restarts. Register an interrupt hook that drains every S3 HTTP(S) listener (including the local and unix socket ones, and the Iceberg and Lance servers) for up to 15s alongside a bounded gRPC GracefulStop, then closes the S3 API server, reusing the filer's shutdown helper. Serve exits now join that shutdown so the process does not exit mid-drain, and the secondary listeners tolerate http.ErrServerClosed instead of exiting fatally. Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Peter Dodd
|
52c6df3bec |
fix(filer): leave the lock ring before stopping gRPC on shutdown (#11604)
A draining filer stayed in the master's lock ring until its process exited, while gRPC GracefulStop was already refusing new connections. S3 gateways and peer filers kept routing object-write locks and owner-routed writes to it for the whole graceful-stop window. On shutdown the filer now sends leave_lock_ring on its open KeepConnected stream. The master removes it from the lock ring only; it stays a cluster member so peers keep following its metadata log through the drain. Once the ring update without it arrives, the filer has already transferred its locks to the new owners, and it keeps serving through the prior-owner window (plus a second for peers that apply the update later) before gRPC and HTTP begin draining. The whole leave is bounded at 10s so slow lock transfers or a stuck stream cannot hold up the drain. A lone filer, or a master that ignores the message, falls back to the previous behavior. |
||
|
Peter DoddandChris Lu
|
52fb9f93ff |
s3: track filer joins and leaves pushed by the master (#11563)
* fix(s3): track filer joins and leaves pushed by the master The S3 FilerClient replaced its -filer seed with a master snapshot of filer IPs at boot and refreshed it only every 5 minutes. A rolling restart replaces every filer well inside that window, leaving S3 servers with only dead addresses and failing every write until the next poll. Apply the master's ClusterNodeUpdate pushes to the filer list as they arrive, keeping the poll as a backstop. The last filer is never removed, and a poll snapshot requested before a push was applied is discarded rather than overwriting newer membership. * Defer last-filer leaves; bump the generation only on real changes * fix(s3): cancel deferred filer leaves on rejoin and on discovery A deferred last-filer leave outlived the filer it was recorded for: a rejoin at the same address looked like a duplicate add, and a discovery snapshot left the entry behind. The next join then removed a live filer until the following poll. A join now cancels any deferred leave for its address, and an applied snapshot clears them, since it is the master's current membership. * Bump the push generation when a rejoin cancels a deferred leave --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
Peter DoddandClaude Opus 5
|
f7ae2d4dd5 |
fix(redis2): remove orphaned directory index members on listing (#10735)
* fix(redis2): remove orphaned directory index members on listing ListDirectoryEntries skipped index members whose value key was gone and left them in the ZSET, so the per-directory child index grew without bound under any TTL workload. Mirror the ZRem the logical-expiry branch already performs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(redis2): keep the index member when a concurrent insert recreates the value The orphan cleanup removed the member unconditionally, so an InsertEntry landing between FindEntry and the ZRem left a live value with no index member, invisible to listings until another InsertEntry on that path. UpdateEntry does not re-add it, so the loss persisted. Restore the member when the value is present again after the removal. The value key and the directory index key hash to different slots, so a Lua script or MULTI over both is not available to the cluster store. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
9fd7075bea |
filer: retry and surface metadata replay failures from peers (#10714)
* filer: stop silently dropping metadata replay failures from peers When two filers do not share a store (e.g. one leveldb3 per pod), each subscribes to its peers' metadata streams and replays their events locally (meta_aggregator.go's maybeReplicateMetadataChange, wired into doSubscribeToOneFiler). A failed Replay() was logged and then treated as done anyway: processEventFn always returned nil regardless of the replay outcome, and processOne advanced lastTsNs unconditionally. The offset is the only record of subscription progress, so a dropped event was gone for good - no retry, and nothing else ever observed it. An entry that fails to replay this way diverges from its peer permanently. This is how a bucket's quota (entry.Quota, carried on peer events like everything else - see entry_codec.go's EqualEntry comparing Quota, and FromPbEntry copying it in entry.go) can end up different across filers indefinitely: one replay hiccup on one filer, and its enforcement and any metric reading its own store diverges from the others' with no signal anything went wrong. Fix: replicateMetadataChange now retries a failure with util.Retry, which already distinguishes transient errors (timeouts, connection resets, throttling, ...) from everything else and bounds the backoff. That covers the common case - a busy store, a blip talking to a remote-backed backend - without changing behavior when replay succeeds. An error that is not transient, or outlives the retry budget, is not retried further: propagating it so the offset never advances would stall this peer's entire stream behind one event that may never replay, which is worse than the one entry staying stale. Instead it is skipped, loudly - counted in a new stats.FilerMetaAggregatorReplayFailures metric and logged at error level - so the divergence is discoverable instead of silent. Tested: go build ./... and go test ./weed/filer/... ./weed/server/... Added meta_aggregator_replay_test.go: one test fails against the old one-shot Replay call (a single transient failure is never retried, so the store never converges) and passes with the fix; a second covers a permanently-failing event completing quickly and being counted instead of retried forever. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * filer: keep the test quota constant int64 for 32-bit builds An untyped shift constant passed to t.Fatalf's ...any defaults to int and overflows on 32-bit, failing go vet there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * filer: name the diverged entry in the give-up log line event.Directory is only the parent (typically /buckets), so for any directory with more than one child the previous log line could not say which entry failed to replay - the exact thing the change exists to make discoverable. Name comes from NewEntry, falling back to OldEntry for deletes; both getters are nil-safe. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(filer): trim metadata replay comments to the non-obvious why Compress the added comments on replicateMetadataChange and its tests down to the reasoning a maintainer cannot get from the code: why a retry-exhausted failure is skipped rather than propagated, what the old one-shot Replay body did that the test pins, and why the quota constant is typed int64. Drops deployment-specific narration and restatement of the code. No behaviour change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: restore load-bearing clauses trimmed in the comment pass Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * filer: document and test the multi-step DeleteEntry replay hazard CodeRabbit flagged that FilerStoreWrapper.DeleteEntry skips the delete once FindEntry reports the path already gone, and that the redis store families remove the primary key before parent-directory membership. Chained together, a delete that fails between those two steps is retried as a no-op: the stale membership is never revisited, and replicateMetadataChange now reports overall success for it without incrementing FilerMetaAggregatorReplayFailures, whereas before this PR every such failure was unconditionally logged. The underlying store inconsistency is pre-existing (a single non-retried Replay already leaves the same stale membership behind); what retry adds is that this one case no longer surfaces it. Making Replay atomic or teaching every store to repair secondary mutations on retry is out of scope here. Instead: document the hazard at Replay, filerstore_wrapper.go's DeleteEntry, and replicateMetadataChange, and add a test against the real FilerStoreWrapper (not a strawman) that pins down the current, documented behavior. * filer: trim replay retry comments and tests Drop the comment-only hunks documenting the pre-existing DeleteEntry partial-failure hazard, the test that asserted that hazard still exists, and the second hand-rolled fake store. Reuse stubFilerStore for the two retry tests. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Peter Dodd
|
bb5e6e40df | fix(remote/gcs): tolerate already-deleted object in DeleteFile (#10151) | ||
|
Peter DoddandCursor
|
4476cb282b |
feat(filer): add atime to FuseAttributes + TouchAccessTime RPC (#9556)
* feat(filer): add atime field and TouchAccessTime RPC to filer proto
Introduce POSIX-style access-time tracking on the filer:
- FuseAttributes gains atime (field 22) and atime_ns (field 23).
- New TouchAccessTime RPC (and Touch{Access,Time}{Request,Response})
lets read paths bump atime without going through UpdateEntry's
chunk-rewrite/EqualEntry short-circuit.
Additive proto changes only; zero atime is treated as unset and
existing clients are unaffected. Java client proto is kept in lock
step.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(filer): wire Atime through Attr codec with mtime fallback
Add Attr.Atime and round-trip it through EntryAttributeToPb /
EntryAttributeToExistingPb / PbToEntryAttribute. A zero proto atime
decodes as Mtime, so legacy entries report a sensible value and
freshly-created/updated entries default Atime to Mtime when callers
do not set it explicitly.
CreateEntry and UpdateEntry stamp Atime = Mtime (or Crtime) when it
is zero. TouchAccessTime later bypasses this path to write atime
alone via Store.UpdateEntry.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(filer): preserve atime in first epoch second on decode
The Atime decode branch previously treated any attr.Atime == 0 as
unset and overwrote it with Mtime, which drops valid timestamps in
the first second of the unix epoch where attr.Atime is 0 but
attr.AtimeNs > 0. Check both fields so we only fall back to Mtime
when both are zero.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Peter DoddandChris Lu
|
0e570d6a8f |
feat(remote.mount): add -metadataStrategy flag to control metadata caching (#8568)
* feat(remote): add -noSync flag to skip upfront metadata pull on mount Made-with: Cursor * refactor(remote): split mount setup from metadata sync Extract ensureMountDirectory for create/validate; call pullMetadata directly when sync is needed. Caller controls sync step for -noSync. Made-with: Cursor * fix(remote): validate mount root when -noSync so bad bucket/creds fail fast When -noSync is used, perform a cheap remote check (ListBuckets and verify bucket exists) instead of skipping all remote I/O. Invalid buckets or credentials now fail at mount time. Made-with: Cursor * test(remote): add TestRemoteMountNoSync for -noSync mount and persisted mapping Made-with: Cursor * test(remote): assert no upfront metadata after -noSync mount After remote.mount -noSync, run fs.ls on the mount dir and assert empty listing so the test fails if pullMetadata was invoked eagerly. Made-with: Cursor * fix(remote): propagate non-ErrNotFound lookup errors in ensureMountDirectory Return lookupErr immediately for any LookupDirectoryEntry failure that is not filer_pb.ErrNotFound, so only the not-found case creates the entry and other lookup failures are reported to the caller. Made-with: Cursor * fix(remote): use errors.Is for ErrNotFound in ensureMountDirectory Replace fragile strings.Contains(lookupErr.Error(), ...) with errors.Is(lookupErr, filer_pb.ErrNotFound) before calling CreateEntry. Made-with: Cursor * fix(remote): use LookupEntry so ErrNotFound is recognised after gRPC Raw gRPC LookupDirectoryEntry returns a status error, not the sentinel, so errors.Is(lookupErr, filer_pb.ErrNotFound) was always false. Use filer_pb.LookupEntry which normalises not-found to ErrNotFound so the mount directory is created when missing. Made-with: Cursor * test(remote): ignore weed shell banner in TestRemoteMountNoSync fs.ls count Exclude master/filer and prompt lines from entry count so the assertion checks only actual fs.ls output for empty -noSync mount. Made-with: Cursor * fix(remote.mount): use 0755 for mount dir, document bucket-less early return Made-with: Cursor * feat(remote.mount): replace -noSync with -metadataStrategy=lazy|eager - Add -metadataStrategy flag (eager default, lazy skips upfront metadata pull) - Accept lazy/eager case-insensitively; reject invalid values with clear error - Rename TestRemoteMountNoSync to TestRemoteMountMetadataStrategyLazy - Add TestRemoteMountMetadataStrategyEager and TestRemoteMountMetadataStrategyInvalid Made-with: Cursor * fix(remote.mount): validate strategy and remote before creating mount directory Move strategy validation and validateMountRoot (lazy path) before ensureMountDirectory so that invalid strategies or bad bucket/credentials fail without leaving orphaned directory entries in the filer. * refactor(remote.mount): remove unused remote param from ensureMountDirectory The remote *RemoteStorageLocation parameter was left over from the old syncMetadata signature. Only remoteConf.Name is used inside the function. * doc(remote.mount): add TODO for HeadBucket-style validation validateMountRoot currently lists all buckets to verify one exists. Note the need for a targeted BucketExists method in the interface. * refactor(remote.mount): use MetadataStrategy type and constants Replace raw string comparisons with a MetadataStrategy type and MetadataStrategyEager/MetadataStrategyLazy constants for clarity and compile-time safety. * refactor(remote.mount): rename MetadataStrategy to MetadataCacheStrategy More precisely describes the purpose: controlling how metadata is cached from the remote, not metadata handling in general. * fix(remote.mount): remove validateMountRoot from lazy path Lazy mount's purpose is to skip remote I/O. Validating via ListBuckets contradicts that, especially on accounts with many buckets. Invalid buckets or credentials will surface on first lazy access instead. * fix(test): handle shell exit 0 in TestRemoteMountMetadataStrategyInvalid The weed shell process exits with code 0 even when individual commands fail — errors appear in stdout. Check output instead of requiring a non-nil error. * test(remote.mount): remove metadataStrategy shell integration tests These tests only verify string output from a shell process that always exits 0 — they cannot meaningfully validate eager vs lazy behavior without a real remote backend. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Peter DoddandChris Lu
|
146a090754 |
filer: propagate lazy metadata deletes to remote mounts (#8522)
* filer: propagate lazy metadata deletes to remote mounts Delete operations now call the remote backend for mounted remote-only entries before removing filer metadata, keeping remote state aligned and preserving retry semantics on remote failures. Made-with: Cursor * filer: harden remote delete metadata recovery Persist remote-delete metadata pendings so local entry removal can be retried after failures, and return explicit errors when remote client resolution fails to prevent silent local-only deletes. Made-with: Cursor * filer: streamline remote delete client lookup and logging Avoid a redundant mount trie traversal by resolving the remote client directly from the matched mount location, and add parity logging for successful remote directory deletions. Made-with: Cursor * filer: harden pending remote metadata deletion flow Retry pending-marker writes before local delete, fail closed when marking cannot be persisted, and start remote pending reconciliation only after the filer store is initialised to avoid nil store access. Made-with: Cursor * filer: avoid lazy fetch in pending metadata reconciliation Use a local-only entry lookup during pending remote metadata reconciliation so cache misses do not trigger remote lazy fetches. Made-with: Cursor * filer: serialise concurrent index read-modify-write in pending metadata deletion Add remoteMetadataDeletionIndexMu to Filer and acquire it for the full read→mutate→commit sequence in markRemoteMetadataDeletionPending and clearRemoteMetadataDeletionPending, preventing concurrent goroutines from overwriting each other's index updates. Made-with: Cursor * filer: start remote deletion reconciliation loop in NewFiler Move the background goroutine for pending remote metadata deletion reconciliation from SetStore (where it was gated by sync.Once) to NewFiler alongside the existing loopProcessingDeletion goroutine. The sync.Once approach was problematic: it buried a goroutine launch as a side effect of a setter, was unrecoverable if the goroutine panicked, could race with store initialisation, and coupled its lifecycle to unrelated shutdown machinery. The existing nil-store guard in reconcilePendingRemoteMetadataDeletions handles the window before SetStore is called. * filer: skip remote delete for replicated deletes from other filers When isFromOtherCluster is true the delete was already propagated to the remote backend by the originating filer. Repeating the remote delete on every replica doubles API calls, and a transient remote failure on the replica would block local metadata cleanup — leaving filers inconsistent. * filer: skip pending marking for directory remote deletes Directory remote deletes are idempotent and do not need the pending/reconcile machinery that was designed for file deletes where the local metadata delete might fail after the remote object is already removed. * filer: propagate remote deletes for children in recursive folder deletion doBatchDeleteFolderMetaAndData iterated child files but only called NotifyUpdateEvent and collected chunks — it never called maybeDeleteFromRemote for individual children. This left orphaned objects in the remote backend when a directory containing remote-only files was recursively deleted. Also fix isFromOtherCluster being hardcoded to false in the recursive call to doBatchDeleteFolderMetaAndData for subdirectories. * filer: simplify pending remote deletion tracking to single index key Replace the double-bookkeeping scheme (individual KV entry per path + newline-delimited index key) with a single index key that stores paths directly. This removes the per-path KV writes/deletes, the base64 encoding round-trip, and the transaction overhead that was only needed to keep the two representations in sync. * filer: address review feedback on remote deletion flow - Distinguish missing remote config from client initialization failure in maybeDeleteFromRemote error messages. - Use a detached context (30s timeout) for pending-mark and pending-clear KV writes so they survive request cancellation after the remote object has already been deleted. - Emit NotifyUpdateEvent in reconcilePendingRemoteMetadataDeletions after a successful retry deletion so downstream watchers and replicas learn about the eventual metadata removal. * filer: remove background reconciliation for pending remote deletions The pending-mark/reconciliation machinery (KV index, mutex, background loop, detached contexts) handled the narrow case where the remote object was deleted but the subsequent local metadata delete failed. The client already receives the error and can retry — on retry the remote not-found is treated as success and the local delete proceeds normally. The added complexity (and new edge cases around NotifyUpdateEvent, multi-filer consistency during reconciliation, and context lifetime) is not justified for a transient store failure the caller already handles. Remove: loopProcessingRemoteMetadataDeletionPending, reconcilePendingRemoteMetadataDeletions, markRemoteMetadataDeletionPending, clearRemoteMetadataDeletionPending, listPendingRemoteMetadataDeletionPaths, encodePendingRemoteMetadataDeletionIndex, FindEntryLocal, and all associated constants, fields, and test infrastructure. * filer: fix test stubs and add early exit on child remote delete error - Refactor stubFilerStore to release lock before invoking callbacks and propagate callback errors, preventing potential deadlocks in tests - Implement ListDirectoryPrefixedEntries with proper prefix filtering instead of delegating to the unfiltered ListDirectoryEntries - Add continue after setting err on child remote delete failure in doBatchDeleteFolderMetaAndData to skip further processing of the failed entry * filer: propagate child remote delete error instead of silently continuing Replace `continue` with early `break` when maybeDeleteFromRemote fails for a child entry during recursive folder deletion. The previous `continue` skipped the error check at the end of the loop body, so a subsequent successful entry would overwrite err and the remote delete error was silently lost. Now the loop breaks, the existing error check returns the error, and NotifyUpdateEvent / chunk collection are correctly skipped for the failed entry. * filer: delete remote file when entry has Remote pointer, not only when remote-only Replace IsInRemoteOnly() guard with entry.Remote == nil check in maybeDeleteFromRemote. IsInRemoteOnly() requires zero local chunks and RemoteSize > 0, which incorrectly skips remote deletion for cached files (local chunks exist) and zero-byte remote objects (RemoteSize 0). The correct condition is whether the entry has a remote backing object at all. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Peter Dodd
|
16f2269a33 |
feat(filer): lazy metadata pulling (#8454)
* Add remote storage index for lazy metadata pull Introduces remoteStorageIndex, which maintains a map of filer directory to remote storage client/location, refreshed periodically from the filer's mount mappings. Provides lazyFetchFromRemote, ensureRemoteEntryInFiler, and isRemoteBacked on S3ApiServer as integration points for handler-level work in a follow-up PR. Nothing is wired into the server yet. Made-with: Cursor * Add unit tests for remote storage index and wire field into S3ApiServer Adds tests covering isEmpty, findForPath (including longest-prefix resolution), and isRemoteBacked. Also removes a stray PR review annotation from the index file and adds the remoteStorageIdx field to S3ApiServer so the package compiles ahead of the wiring PR. Made-with: Cursor * Address review comments on remote storage index - Use filer_pb.CreateEntry helper so resp.Error is checked, not just the RPC error - Extract keepPrev closure to remove duplicated error-handling in refresh loop - Add comment explaining availability-over-consistency trade-off on filer save failure Made-with: Cursor * Move lazy metadata pull from S3 API to filer - Add maybeLazyFetchFromRemote in filer: on FindEntry miss, stat remote and CreateEntry when path is under a remote mount - Use singleflight for dedup; context guard prevents CreateEntry recursion - Availability-over-consistency: return in-memory entry if CreateEntry fails - Add longest-prefix test for nested mounts in remote_storage_test.go - Remove remoteStorageIndex, lazyFetchFromRemote, ensureRemoteEntryInFiler, doLazyFetch from s3api; filer now owns metadata operations - Add filer_lazy_remote_test.go with tests for hit, miss, not-found, CreateEntry failure, longest-prefix, and FindEntry integration Made-with: Cursor * Address review: fix context guard test, add FindMountDirectory comment, remove dead code Made-with: Cursor * Nitpicks: restore prev maker in registerStubMaker, instance-scope lazyFetchGroup, nil-check remoteEntry Made-with: Cursor * Fix remotePath when mountDir is root: ensure relPath has leading slash Made-with: Cursor * filer: decouple lazy-fetch persistence from caller context Use context.Background() inside the singleflight closure for CreateEntry so persistence is not cancelled when the winning request's context is cancelled. Fixes CreateEntry failing for all waiters when the first caller times out. Made-with: Cursor * filer: remove redundant Mode bitwise OR with zero Made-with: Cursor * filer: use bounded context for lazy-fetch persistence Replace context.Background() with context.WithTimeout(30s) and defer cancel() to prevent indefinite blocking and release resources. Made-with: Cursor * filer: use checked type assertion for singleflight result Made-with: Cursor * filer: rename persist context vars to avoid shadowing function parameter Made-with: Cursor |
||
|
Peter DoddandCursor
|
0910252e31 |
feat: add statfile remote storage (#8443)
* feat: add statfile; add error for remote storage misses * feat: statfile implementations for storage providers * test: add unit tests for StatFile method across providers Add comprehensive unit tests for the StatFile implementation covering: - S3: interface compliance and error constant accessibility - Azure: interface compliance, error constants, and field population - GCS: interface compliance, error constants, error detection, and field population Also fix variable shadowing issue in S3 and Azure StatFile implementations where named return parameters were being shadowed by local variable declarations. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: address StatFile review feedback - Use errors.New for ErrRemoteObjectNotFound sentinel - Fix S3 HeadObject 404 detection to use awserr.Error code check - Remove hollow field-population tests that tested nothing - Remove redundant stdlib error detection tests - Trim verbose doc comment on ErrRemoteObjectNotFound Co-authored-by: Cursor <cursoragent@cursor.com> * fix: address second round of StatFile review feedback - Rename interface assertion tests to TestXxxRemoteStorageClientImplementsInterface - Delegate readFileRemoteEntry to StatFile in all three providers - Revert S3 404 detection to RequestFailure.StatusCode() check - Fix double-slash in GCS error message format string - Add storage type prefix to S3 error message for consistency Co-authored-by: Cursor <cursoragent@cursor.com> * fix: comments --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Peter Dodd
|
f4af1cc0ba | feat(helm): annotations for service account (#8429) | ||
|
|
4d513a2b3d |
feat(gcs): add application default credentials fallback support (#8161)
* feat(gcs): add application default credentials fallback support * refactor * Update weed/remote_storage/gcs/gcs_storage_client.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |