s3: keep transient object-entry errors out of ErrObjectNotFound (#12083)

* s3: keep transient object-entry errors out of ErrObjectNotFound

getObjectEntry wrapped every fetch error as ErrObjectNotFound, so a
transient filer failure (busy filer, ctx deadline, transport blip) was
indistinguishable from a real miss. LifecycleDelete resolves
ErrObjectNotFound permanently as NOOP_RESOLVED, so one flaky lookup at
dispatch time silently skipped the object's expiration - errors=0, no
retry, no log. This matches the flaky s3tests
test_lifecyclev2_expiration signature: exactly one object of a
same-rule pair surviving past its window.

Only collapse genuine misses (filer not-found, version/latest-version
sentinels); everything else propagates so the worker retries it as
RETRY_LATER instead of resolving it.

Also seed bucketConfigCache in the retention test server and teach the
fake filer to inject per-path lookup errors.

* s3: keep invalid version ids on the not-found path in getObjectEntry

A malformed versionId used to collapse into ErrObjectNotFound like every
other fetch error, so the handlers answered NoSuchKey. With transient
errors now propagated, an invalid id would surface as InternalError;
fold it back into the miss classification.

* filer_pb: keep Unavailable out of LookupEntry's not-found normalization

* s3: reject Unavailable in isObjectEntryMiss before message matching

* s3: keep transient fallback errors out of latest-version resolution

* filer_pb: limit LookupEntry's not-found text match to miss statuses

* s3: limit isFilerNotFound text matching to miss statuses

weed/s3api/s3_error_utils.go

@@ -14,8 +14,10 @@ import (
 // isFilerNotFound reports whether a filer error is a not-found.
 // Unlike lookups (normalized in filer_pb.LookupEntry), list and cache errors
 // cross gRPC as raw status errors, so the sentinel survives as codes.NotFound
-// or only as text. The text is matched last and only on what the filer itself
-// said, since callers wrap these errors with a path the client chose.
+// or as codes.Unknown carrying only its text. The text is matched only on
+// miss-shaped statuses and only on what the filer itself said: the same
+// message on another status is a failed read, and callers wrap these errors
+// with a path the client chose.
 func isFilerNotFound(err error) bool {
 	if err == nil {
 		return false
@@ -24,7 +26,8 @@ func isFilerNotFound(err error) bool {
 		return true
 	}
 	if st, ok := util.ServerStatus(err); ok {
-		return st.Code() == codes.NotFound || strings.Contains(st.Message(), filer_pb.ErrNotFound.Error())
+		return st.Code() == codes.NotFound ||
+			(st.Code() == codes.Unknown && strings.Contains(st.Message(), filer_pb.ErrNotFound.Error()))
 	}
 	return strings.Contains(err.Error(), filer_pb.ErrNotFound.Error())
 }