filer: batch exact lookup RPC, authoritative volume lookup, VolumeDelete status codes (#11122)

* storage: make DeleteVolume errors inspectable with errors.Is

An absent volume wraps ErrVolumeNotFound and an only-empty refusal now
wraps ErrVolumeNotEmpty with %w instead of %v, so callers no longer have
to match on the message.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* volume server: return NotFound and FailedPrecondition from VolumeDelete

An absent volume maps to codes.NotFound and a non-empty volume under
only_empty to codes.FailedPrecondition, so a caller retiring a volume can
treat NotFound as already done. The store message is kept in the status
description because the EC empty-replica sweep still matches on it.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* wdclient: add LookupVolumeIdsAuthoritative

Bypasses the vid map and asks the provider directly, for callers where a
stale positive location is unsafe.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: add LookupDirectoryEntries batch lookup RPC

Up to 4096 exact-path lookups in one call, resolved concurrently with
results in request order, plus one deduplicated location lookup for every
volume the returned entries reference and per-fid read tokens when the
filer signs reads. unavailable_volume_is_miss lets cache-style callers
take an entry whose volume has no live location as a miss, resolved
against the master rather than the filer's location cache.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: test that an expired file entry is deleted on read

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: test that AssignVolume and CreateEntry resolve the same TTL rule

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* master: refuse partial lookups while warming up

LookupVolume returned Unavailable during warm-up only when every requested
volume was missing. A batch mixing a reported volume with one whose server
has not reconnected yet came back as a partial answer with a per-volume
not-found, which a caller treating the master as authoritative reads as
gone. Any not-found during warm-up is now Unavailable, which callers
already retry.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: build batch test requests instead of copying a proto message

Copying a generated message copies its internal mutex, which go vet's
copylocks check rejects.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: match ErrNotFound with errors.Is and state the miss rule's contract

A wrapped not-found from the store would otherwise be reported as an
error rather than a miss. The comments now say why a nil location map is
the only sign of an unanswered lookup: the provider returns nil when it
got no answer and a populated map, with unserved volumes reported as
errors, when the master did answer.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* volume server: map absent and non-empty VolumeDelete errors in the Rust server

Matches the Go server: an absent volume is NotFound and an only_empty
refusal is FailedPrecondition instead of Internal, with the messages the
EC empty-replica sweep matches on.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm

* filer: test that a malformed entry keeps its error outside cache mode

Same test file as the enterprise tree, so the next sync sees one version.

Claude-Session: https://claude.ai/code/session_01T4MEV3ETqFFKN46Uu2ZrUm
This commit is contained in:
Chris Lu
2026-09-03 14:52:05 -07:00
committed by GitHub
parent eb9f54cebb
commit 8112f2733a
18 changed files with 2378 additions and 110 deletions
@@ -13,6 +13,9 @@ service SeaweedFiler {
rpc LookupDirectoryEntry (LookupDirectoryEntryRequest) returns (LookupDirectoryEntryResponse) {
}
rpc LookupDirectoryEntries (LookupDirectoryEntriesRequest) returns (LookupDirectoryEntriesResponse) {
}
rpc ListEntries (ListEntriesRequest) returns (stream ListEntriesResponse) {
}
@@ -581,6 +584,7 @@ message Location {
string public_url = 2;
uint32 grpc_port = 3;
string data_center = 4;
bool data_in_remote = 5;
}
message LookupVolumeResponse {
map<string, Locations> locations_map = 1;
@@ -912,3 +916,40 @@ message MountInfo {
int64 last_seen_ns = 3;
string data_center = 4;
}
// LookupDirectoryEntriesRequest batches independent exact-path lookups into
// one bounded RPC. Results preserve this order. Empty batches and batches over
// 4096 requests are rejected. Declared last to keep generated message indices
// stable.
message LookupDirectoryEntriesRequest {
repeated LookupDirectoryEntryRequest requests = 1;
// Cache callers may treat an Entry whose Volume the master no longer
// reports as a miss. A lookup the filer could not complete still marks the
// Entry with an error. Ordinary filesystem callers keep the fail-closed
// error in both cases.
bool unavailable_volume_is_miss = 2;
}
message LookupDirectoryEntryResult {
// found reports metadata presence. An error makes this item unusable even
// when entry is present, for example when a chunk volume cannot be located.
bool found = 1;
Entry entry = 2;
string error = 3;
// Per-entry read fence. Every filer event at or below this position is
// reflected in found/entry/error for this item.
int64 log_ts_ns = 4;
int32 log_signature = 5;
}
message LookupDirectoryEntriesResponse {
repeated LookupDirectoryEntryResult results = 1;
// One deduplicated location set for every volume referenced by returned
// Entry.chunks. A missing/unavailable volume has an empty Locations value
// and marks each affected result with an error.
map<string, Locations> locations_map = 2;
// Short-lived, exact-FID read capabilities. Direct data-plane clients must
// present the matching token to a Volume instead of treating a shared
// service credential as authority to read arbitrary needles.
map<string, string> read_auth = 3;
}