Files
seaweedfs/weed/s3api/s3api_head_bucket_test.go
T
Chris LuandDevin 5d80ac39b8 s3api: verify under the write lock before re-committing an ambiguous routed PUT (#11649)
* s3api: verify under the write lock before re-committing an ambiguous routed PUT

A routed PUT whose response was lost after the owner committed, or whose
transaction returned a store error, fell straight into the lock path and
re-sent the same entry. A concurrent PUT that had superseded the commit in
the meantime had already deleted this entry's chunks as old, so the
re-commit restored metadata pointing at dead needles and the object read
back 404 permanently.

The lock path now resolves the routed attempt's outcome inside the write
lock first: a stored entry with the uploaded chunks means the route
landed; a different stored entry or an unresolved lookup refuses the
re-commit with ServiceUnavailable so the client retries with a fresh
upload; only a proven-absent entry falls through to the normal create.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: tighten ambiguous routed PUT recovery

- a match found only after an unanswered filer is not authoritative;
  the unreachable filer may hold a newer entry, so refuse instead of
  finalizing on it (both recovery paths)
- a failed post-recovery finalization now rolls the recovered entry
  back, matching the create path's undo
- a proven-absent entry is only re-committed after probing that the
  uploaded chunks' needles are still alive; a committed-and-deleted
  PUT would otherwise write back an entry pointing at reclaimed
  needles
- the .versions latest pointer no longer flips back to an older
  version when a newer one committed while the write was uncertain

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: stamp late versions noncurrent when a newer latest pointer wins

The keep-newer early return in updateLatestVersionInDirectory left the
version just stored without ExtNoncurrentSinceNsKey, so the lifecycle
engine could never age it out.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: only a failure past mutation 0 makes a routed PUT ambiguous

A deterministic refusal at the PUT mutation (e.g. "existing entry is a
directory") applied nothing, but marking it ambiguous sent the lock-path
fallback through conservative recovery, which found the directory entry
and refused with 503 instead of the correct 409.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: keep all transaction errors ambiguous; route directory conflicts through the lock path

* s3: verify uploaded chunks before re-committing over a directory

A stored directory does not prove the routed PUT never committed: the
route can land, a delete can remove the entry and its chunks, and a
nested write can recreate the directory before recovery takes the lock.
Re-committing then stores an entry pointing at dead needles. Probe the
uploaded chunks first and refuse with ServiceUnavailable when they can
no longer be verified.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-10 08:52:34 +08:00

97 lines
3.0 KiB
Go

package s3api
import (
"context"
"net/http"
"net/http/httptest"
"path"
"strings"
"testing"
"github.com/gorilla/mux"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/wdclient"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/status"
)
type fakeLookupFiler struct {
filer_pb.UnimplementedSeaweedFilerServer
entry *filer_pb.Entry
lookupErr error
deleted []string
}
func (f *fakeLookupFiler) LookupDirectoryEntry(ctx context.Context, req *filer_pb.LookupDirectoryEntryRequest) (*filer_pb.LookupDirectoryEntryResponse, error) {
if f.lookupErr != nil {
return nil, f.lookupErr
}
return &filer_pb.LookupDirectoryEntryResponse{Entry: f.entry}, nil
}
func (f *fakeLookupFiler) DeleteEntry(ctx context.Context, req *filer_pb.DeleteEntryRequest) (*filer_pb.DeleteEntryResponse, error) {
f.deleted = append(f.deleted, path.Join(req.Directory, req.Name))
f.entry = nil
return &filer_pb.DeleteEntryResponse{}, nil
}
func newHeadBucketTestServer(t *testing.T, impl filer_pb.SeaweedFilerServer) *S3ApiServer {
t.Helper()
filers := []pb.ServerAddress{startFakeFiler(t, impl)}
dialOption := grpc.WithTransportCredentials(insecure.NewCredentials())
return &S3ApiServer{
option: &S3ApiServerOption{Filers: filers, GrpcDialOption: dialOption, BucketsPath: "/buckets"},
filerClient: wdclient.NewFilerClient(filers, dialOption, ""),
}
}
// A lookup that fails for a reason other than "not found" must not be reported
// as a missing bucket: a 404 is a definite answer and stops the client retrying.
func TestHeadBucketSeparatesLookupFailureFromMissingBucket(t *testing.T) {
const bucket = "head-bucket"
cases := []struct {
name string
filer *fakeLookupFiler
wantCode int
wantBody string
}{
{
name: "transient lookup failure",
filer: &fakeLookupFiler{lookupErr: status.Error(codes.Internal, "filer store unavailable")},
wantCode: http.StatusInternalServerError,
wantBody: "<Code>InternalError</Code>",
},
{
name: "missing bucket",
filer: &fakeLookupFiler{},
wantCode: http.StatusNotFound,
wantBody: "<Code>NoSuchBucket</Code>",
},
{
name: "existing bucket",
filer: &fakeLookupFiler{entry: &filer_pb.Entry{Name: bucket, IsDirectory: true}},
wantCode: http.StatusOK,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s3a := newHeadBucketTestServer(t, tc.filer)
req := httptest.NewRequest(http.MethodHead, "/"+bucket, nil)
req = mux.SetURLVars(req, map[string]string{"bucket": bucket})
rr := httptest.NewRecorder()
s3a.HeadBucketHandler(rr, req)
if rr.Code != tc.wantCode {
t.Fatalf("status = %d, want %d: %s", rr.Code, tc.wantCode, rr.Body.String())
}
if tc.wantBody != "" && !strings.Contains(rr.Body.String(), tc.wantBody) {
t.Fatalf("body = %s, want %s", rr.Body.String(), tc.wantBody)
}
})
}
}