mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-11 17:10:40 +02:00
* admin: treat a missing S3 Tables policy as an empty load, not an error
The bucket/table policy GET relayed the backend's 404 NoSuchPolicy to the
dialog, whose loader treats any non-OK response as a load failure and
keeps Save and Delete blocked. A bucket or table without a policy could
never be given one. Return policy null instead, the same contract
ShowBucketPolicy uses for classic buckets.
* admin: reject policy documents the structured editor would misread
A top-level JSON array passed the object guard (typeof [] is 'object')
and loaded as a zero-statement policy, which the next commit would
rewrite to an empty document. Object elements in Action/Resource were
coerced to '[object Object]' and saved that way on the s3tables surface,
which stores policies verbatim. Both now throw, which routes the
document to the JSON tab like other unrepresentable shapes.
* admin: let the JSON tab save documents the structured editor can't model
Save with the JSON tab active required a round-trip through
policyDocToEditorState, so exactly the documents the dialogs shunt to
'JSON tab only' mode (unrepresentable Effect, Resource+NotResource, and
the like) could never be saved - Delete was the only mutation left.
Invalid JSON still blocks; an unrepresentable document now saves and the
editor state stays marked unparsed.
* admin: pin the policy editor to what each consumer's backend supports
The s3tables evaluator has no NotResource/NotPrincipal fields - it
silently drops them, turning Allow+NotResource into allow-everything and
making Deny+NotPrincipal inert - and it only matches s3tables: actions
against s3tables ARNs, while the editor suggested s3: actions and
arn:aws:s3::: resources. New registerPolicyEditor knobs: allowNegation
hides the Not* modes and routes documents using them to the JSON tab;
resourceSuggestions pins the Resource autocomplete to the open
resource's ARN; the S3 Tables dialogs get an s3tables-only action
datalist. requirePrincipal now also hides NotPrincipal, which
policy_engine.ValidateBucketPolicy always rejects, and the client-side
check requires Principal specifically to match that server rule.
* admin: save S3 Tables policies from a button, not form submission
The multi-input structured editor sits inside a form whose Save button
was type=submit, so Enter in any single-line editor input - accepting an
autocomplete suggestion, say - implicitly submitted whatever half-built
statement the editor held, and the backend stores the document verbatim.
A lone statement with no Principal matches nobody, locking out every
non-owner. Save is now an ordinary button and the form ignores
submission.
* admin: block zero-statement policy saves
Committing the active tab before the emptiness check made 'Policy JSON
is required' dead code: an empty editor serializes to {"Statement":[]},
which the s3tables backend stores verbatim - evaluated default-deny for
every non-owner, while the statement-count column keeps showing 'Not
configured'. All three policy dialogs now refuse a save with no
statements and point at Delete instead. The classic bucket modal only
gained a clearer message; the server already rejected the document.
* admin: guard S3 Tables policy mutations against stale and overlapping requests
The save/delete completions ran against whatever resource the shared
modal happened to show by then: a slow PUT for one bucket would hide the
modal mid-edit of another and misattribute its alerts, a late DELETE
cleared the shared textarea over the newly opened resource with its
loaded flag set, and nothing stopped a double-click from firing two
overlapping mutations. Ported the classic modal's pattern: capture the
target on start, flag the mutation in flight with the buttons disabled,
and only touch the UI when the completion still matches the open
resource. Success now reloads the page, which also keeps the Policy
column's statement count honest.
* admin: confirm before deleting an S3 Tables policy
Delete Policy sat next to Save and fired on a single click; with
default-allow enabled one stray click silently dropped the resource
policy and left the bucket open to every principal. Same confirmation
the classic bucket modal already has.
* admin: let a corrupt stored bucket policy be shown, fixed, and deleted
A stored document the decoder rejects made the policy GET 500, and with
the loaded flag never set the modal blocked both Save and Delete - the
one policy an operator most needs to remove was the one they couldn't,
even though the delete path never reads the document. The GET now
returns the raw bytes alongside a null policy; the dialog hands them to
the JSON tab and unblocks the buttons.
* admin: url-encode the bucket name in the policy API calls
The filer lists any directory under the buckets path, names S3 would
never allow included; one carrying '#' or '%' broke the fetch URL or
addressed a different name than the modal shows.
* admin: drop stale edit-policy responses on the IAM policies page
The same race the bucket and S3 Tables dialogs already guard against:
open one policy's editor while its GET stalls, open another, and the
late response populates the editor under the second policy's name -
Update then saves the first policy's statements over the second.
* admin: warn before a bucket policy save drops unsupported fields
The editor tracks unmodeled top-level keys precisely so
confirmPolicyFieldDiscard can warn before the server's Version+Statement
decode discards them, but only the IAM page called it; the bucket modal
saved a pasted document with e.g. a console-generated Id without a word
while the editor kept displaying the field.
* s3: enforce the bucket policy size cap on both surfaces
The 20KB cap lived only in the admin UI, so a larger policy stored via
the S3 API displayed there but could never be re-saved, desyncing the
two writers the cap comment claimed could not desync. The constant now
lives in policy_engine next to the shared validator and PutBucketPolicy
rejects oversized documents with PolicyTooLarge, matching AWS.
* admin: ship the policy editor's fieldset styles with the editor
The .policy-stmt-* rules that undo Bootstrap's full-width legend reset
stayed behind in policies.templ when the editor markup moved to the
shared script, so the bucket and S3 Tables dialogs rendered Actions/
Resource/Principal as full-width jumbo headings. PolicyDatalists is the
component every consumer already renders once; the styles live there
now.
* s3: mirror bucket policy changes into the IAM store from the metadata subscription
The advanced-IAM path appends the bucket-policy:<bucket> document to
every STS/session evaluation, but only this gateway's own PutBucketPolicy
maintained that mirror - a policy tightened or created through the admin
UI (or another gateway) never reached it, so revoked access stayed live
indefinitely, and the delete side was an unimplemented TODO in any case.
The metadata subscription now diffs the stored policy on every bucket
entry change and updates or removes the mirror, covering all writers and
deletion with one mechanism; IAMManager gains the missing
RemoveBucketPolicy.
* admin: deduplicate the bucket policy write path
Set and Delete carried line-for-line identical filer closures;
bucketPolicyMutation already treats nil as clear-the-key. The shared
helper sits below Set's validation, since ValidatePolicy cannot take the
nil document Delete passes.
* s3: drop ValidateBucketPolicy's re-checks of ValidatePolicy rules
Both callers run ValidatePolicy first, which already enforces the
version and at-least-one-statement rules; the duplicates were dead code
with drifted error text.
* admin: seed a new statement's Resource from the pinned suggestions
A fresh statement on the S3 Tables dialogs started with no resource row
at all; seed it with the broadest pinned ARN the same way cfg.bucket
already seeds the classic modal.
* admin: refuse to save Not* fields the backend would silently drop
Hiding the NotResource/NotPrincipal modes was not enough where negation
is disallowed: the JSON tab accepts any valid document (that is its
job), and a statement's Advanced-fields box can reintroduce the keys, so
an s3tables save could still store fields the evaluator drops - turning
Allow+NotResource into allow-everything. commitPolicyActiveTab now runs
a final document-level check over what would actually be saved; Delete
stays available for cleanup.
* s3: move the IAM bucket policy mirror on a bucket rename
A same-directory rename delivers one event carrying both entries, and
the byte-equality short-circuit skipped the new name's mirror when the
policy was unchanged - while the replayed delete for the old name
removed its mirror, leaving the renamed bucket unmirrored. The mirror
decision is now a pure function that removes the old name and writes the
new one regardless of byte equality, with the rename cases unit tested.
* s3: backfill the IAM bucket policy mirror on lazy bucket loads
The metadata subscription only mirrors changes, so a policy that
predates the IAM integration never reached the bucket-policy:<bucket>
mirror and its grants did not bind on the IAM path until the policy was
next modified. The gateway is deliberately lazy at startup (nothing
lists all buckets), so the backfill hooks the same place a bucket's
policy first becomes known: the cold bucket-config load. EnsureBucketPolicy
writes only when no mirror is stored, so repeat loads cost one cached
read.
* s3: reconcile the bucket policy backfill against concurrent changes
The backfill's check-then-write could race an event-driven mirror update
or removal and re-store bytes that were already stale, with no later
event to heal it. EnsureBucketPolicy now reports whether it wrote, and a
write is reconciled against a fresh authoritative entry read: a changed
policy is re-mirrored, a removed one is removed. Anything changing after
that read fires its own event, which finds the backfill's write already
present and supersedes it. The backfill also carries the entry's raw
bytes rather than a re-marshaled document, so the reconcile can
byte-compare.
* s3: prime the bucket policy mirror before advanced-IAM authorization
The backfill ran from the lazy bucket-config load, but IAM authorization
evaluates the bucket-policy:<bucket> mirror before any handler runs - a
grant carried only by a not-yet-mirrored policy denied forever, and the
denied request never reached the code that would have loaded the bucket.
authorizeWithIAM now primes the bucket config first (an in-memory cache
hit once warm), and the backfill runs synchronously on the cold load so
the very first authorization already sees the mirror.
804 lines
26 KiB
Go
804 lines
26 KiB
Go
package dash
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"net/http"
|
|
"os"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/gorilla/mux"
|
|
"github.com/seaweedfs/seaweedfs/weed/filer"
|
|
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/policy_engine"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
|
|
"github.com/seaweedfs/seaweedfs/weed/s3api/s3lifecycle"
|
|
)
|
|
|
|
// MaxBucketLifecycleRules mirrors AWS S3's limit of 1000 lifecycle rules per bucket.
|
|
const MaxBucketLifecycleRules = 1000
|
|
|
|
// MaxBucketLifecycleRuleIDLength mirrors the S3 API's limit on a lifecycle rule's <ID>.
|
|
const MaxBucketLifecycleRuleIDLength = 255
|
|
|
|
// MaxOwnerNameLength is the maximum allowed length for bucket owner identity names.
|
|
// This is a reasonable limit to prevent abuse; AWS IAM user names are limited to 64 chars,
|
|
// but we use 256 to allow for more complex identity formats (e.g., email addresses).
|
|
const MaxOwnerNameLength = 256
|
|
|
|
// S3 Bucket management data structures for templates
|
|
type S3BucketsData struct {
|
|
Username string `json:"username"`
|
|
Buckets []S3Bucket `json:"buckets"`
|
|
TotalBuckets int `json:"total_buckets"`
|
|
TotalSize int64 `json:"total_size"`
|
|
LastUpdated time.Time `json:"last_updated"`
|
|
|
|
// Pagination
|
|
CurrentPage int `json:"current_page"`
|
|
TotalPages int `json:"total_pages"`
|
|
PageSize int `json:"page_size"`
|
|
|
|
// Sorting
|
|
SortBy string `json:"sort_by"`
|
|
SortOrder string `json:"sort_order"`
|
|
}
|
|
|
|
type CreateBucketRequest struct {
|
|
Name string `json:"name"` // validated manually in CreateBucket
|
|
Region string `json:"region"`
|
|
QuotaSize int64 `json:"quota_size"` // Quota size in bytes
|
|
QuotaUnit string `json:"quota_unit"` // Unit: MB, GB, TB
|
|
QuotaEnabled bool `json:"quota_enabled"` // Whether quota is enabled
|
|
VersioningEnabled bool `json:"versioning_enabled"` // Whether versioning is enabled
|
|
ObjectLockEnabled bool `json:"object_lock_enabled"` // Whether object lock is enabled
|
|
ObjectLockMode string `json:"object_lock_mode"` // Object lock mode: "GOVERNANCE" or "COMPLIANCE"
|
|
SetDefaultRetention bool `json:"set_default_retention"` // Whether to set default retention
|
|
ObjectLockDuration int32 `json:"object_lock_duration"` // Default retention duration in days
|
|
Owner string `json:"owner"` // Bucket owner identity (for S3 IAM authentication)
|
|
}
|
|
|
|
// S3 Bucket Management Handlers
|
|
|
|
// ShowS3Buckets displays the Object Store buckets management page
|
|
func (s *AdminServer) ShowS3Buckets(w http.ResponseWriter, r *http.Request) {
|
|
username := UsernameFromContext(r.Context())
|
|
|
|
data, err := s.GetS3BucketsData(1, 100, "name", "asc")
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to get Object Store buckets: "+err.Error())
|
|
return
|
|
}
|
|
|
|
data.Username = username
|
|
writeJSON(w, http.StatusOK, data)
|
|
}
|
|
|
|
// ShowBucketDetails displays detailed information about a specific bucket
|
|
func (s *AdminServer) ShowBucketDetails(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
details, err := s.GetBucketDetails(bucketName)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to get bucket details: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, details)
|
|
}
|
|
|
|
// ShowBucketLifecycle returns the lifecycle configuration for a specific bucket
|
|
func (s *AdminServer) ShowBucketLifecycle(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
lifecycle, err := s.GetBucketLifecycle(bucketName)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to get bucket lifecycle: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, lifecycle)
|
|
}
|
|
|
|
// UpdateBucketLifecycle replaces the entire lifecycle configuration for a bucket.
|
|
func (s *AdminServer) UpdateBucketLifecycle(w http.ResponseWriter, r *http.Request) {
|
|
if !requireSessionCSRFToken(w, r) {
|
|
return
|
|
}
|
|
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
var req struct {
|
|
Rules []BucketLifecycleRule `json:"rules"`
|
|
}
|
|
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
|
|
return
|
|
}
|
|
|
|
if err := validateBucketLifecycleRules(req.Rules); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
|
|
if err := s.SetBucketLifecycle(bucketName, req.Rules); err != nil {
|
|
writeJSONError(w, bucketLifecycleErrorStatus(err), "Failed to update bucket lifecycle: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket lifecycle updated successfully",
|
|
"bucket": bucketName,
|
|
})
|
|
}
|
|
|
|
// DeleteBucketLifecycle clears the entire lifecycle configuration for a bucket.
|
|
func (s *AdminServer) DeleteBucketLifecycle(w http.ResponseWriter, r *http.Request) {
|
|
if !requireSessionCSRFToken(w, r) {
|
|
return
|
|
}
|
|
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
if err := s.SetBucketLifecycle(bucketName, nil); err != nil {
|
|
writeJSONError(w, bucketLifecycleErrorStatus(err), "Failed to delete bucket lifecycle: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket lifecycle deleted successfully",
|
|
"bucket": bucketName,
|
|
})
|
|
}
|
|
|
|
// bucketLifecycleErrorStatus keeps a request for a bucket that does not exist
|
|
// out of the 5xx bucket, where a client would retry it.
|
|
func bucketLifecycleErrorStatus(err error) int {
|
|
if errors.Is(err, ErrBucketNotFound) {
|
|
return http.StatusNotFound
|
|
}
|
|
return http.StatusInternalServerError
|
|
}
|
|
|
|
// validateBucketLifecycleRules rejects a rule set before any write is
|
|
// attempted, so a malformed PUT can't half-apply. Mirrors the constraints
|
|
// AWS enforces on PutBucketLifecycleConfiguration plus the local rule cap.
|
|
func validateBucketLifecycleRules(rules []BucketLifecycleRule) error {
|
|
if len(rules) > MaxBucketLifecycleRules {
|
|
return fmt.Errorf("a bucket may have at most %d lifecycle rules, got %d", MaxBucketLifecycleRules, len(rules))
|
|
}
|
|
|
|
seenIDs := make(map[string]bool, len(rules))
|
|
for i, rule := range rules {
|
|
label := fmt.Sprintf("rule %d", i+1)
|
|
if rule.ID != "" {
|
|
label = fmt.Sprintf("rule %q", rule.ID)
|
|
if len(rule.ID) > MaxBucketLifecycleRuleIDLength {
|
|
return fmt.Errorf("%s: ID must be %d characters or less", label, MaxBucketLifecycleRuleIDLength)
|
|
}
|
|
if seenIDs[rule.ID] {
|
|
return fmt.Errorf("duplicate rule ID %q", rule.ID)
|
|
}
|
|
seenIDs[rule.ID] = true
|
|
}
|
|
|
|
switch rule.Status {
|
|
case s3lifecycle.StatusEnabled, s3lifecycle.StatusDisabled:
|
|
default:
|
|
return fmt.Errorf("%s: status must be %q or %q, got %q", label, s3lifecycle.StatusEnabled, s3lifecycle.StatusDisabled, rule.Status)
|
|
}
|
|
|
|
if rule.ExpirationDays > 0 && rule.ExpirationDate != "" {
|
|
return fmt.Errorf("%s: expiration_days and expiration_date are mutually exclusive", label)
|
|
}
|
|
if rule.ExpirationDays < 0 {
|
|
return fmt.Errorf("%s: expiration_days must be positive", label)
|
|
}
|
|
if rule.ExpirationDate != "" {
|
|
if _, err := time.Parse(time.DateOnly, rule.ExpirationDate); err != nil {
|
|
return fmt.Errorf("%s: invalid expiration_date %q, expected YYYY-MM-DD", label, rule.ExpirationDate)
|
|
}
|
|
}
|
|
if rule.ExpiredObjectDeleteMarker && (rule.ExpirationDays > 0 || rule.ExpirationDate != "") {
|
|
return fmt.Errorf("%s: expired_object_delete_marker cannot be combined with expiration_days or expiration_date", label)
|
|
}
|
|
|
|
if rule.NoncurrentVersionExpirationDays < 0 {
|
|
return fmt.Errorf("%s: noncurrent_version_expiration_days must be positive", label)
|
|
}
|
|
if rule.NewerNoncurrentVersions < 0 {
|
|
return fmt.Errorf("%s: newer_noncurrent_versions must be positive", label)
|
|
}
|
|
if rule.AbortMultipartDays < 0 {
|
|
return fmt.Errorf("%s: abort_multipart_days must be positive", label)
|
|
}
|
|
|
|
if rule.SizeGreaterThan < 0 || rule.SizeLessThan < 0 {
|
|
return fmt.Errorf("%s: size_greater_than and size_less_than must be positive", label)
|
|
}
|
|
if rule.SizeGreaterThan > 0 && rule.SizeLessThan > 0 && rule.SizeGreaterThan >= rule.SizeLessThan {
|
|
return fmt.Errorf("%s: size_greater_than must be less than size_less_than", label)
|
|
}
|
|
|
|
for k, v := range rule.Tags {
|
|
if strings.TrimSpace(k) == "" {
|
|
return fmt.Errorf("%s: tag keys must not be empty", label)
|
|
}
|
|
if strings.TrimSpace(v) == "" {
|
|
return fmt.Errorf("%s: tag value for key %q must not be empty", label, k)
|
|
}
|
|
}
|
|
|
|
hasAction := rule.ExpirationDays > 0 || rule.ExpirationDate != "" || rule.ExpiredObjectDeleteMarker ||
|
|
rule.NoncurrentVersionExpirationDays > 0 || rule.NewerNoncurrentVersions > 0 || rule.AbortMultipartDays > 0
|
|
if !hasAction {
|
|
return fmt.Errorf("%s: must specify at least one action (expiration, noncurrent version expiration, or abort incomplete multipart upload)", label)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// ShowBucketPolicy returns the policy document for a specific bucket, or
|
|
// {"bucket": ..., "policy": null} if the bucket has none.
|
|
func (s *AdminServer) ShowBucketPolicy(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
policy, raw, err := s.GetBucketPolicy(bucketName)
|
|
if err != nil {
|
|
writeJSONError(w, bucketPolicyErrorStatus(err), "Failed to get bucket policy: "+err.Error())
|
|
return
|
|
}
|
|
|
|
resp := map[string]interface{}{
|
|
"bucket": bucketName,
|
|
"policy": policy,
|
|
}
|
|
if policy == nil && len(raw) > 0 {
|
|
// Stored bytes the decoder rejects: hand them to the JSON tab so
|
|
// the operator can fix or delete the document.
|
|
resp["policy_text"] = string(raw)
|
|
}
|
|
writeJSON(w, http.StatusOK, resp)
|
|
}
|
|
|
|
// UpdateBucketPolicy replaces the bucket policy for a bucket.
|
|
func (s *AdminServer) UpdateBucketPolicy(w http.ResponseWriter, r *http.Request) {
|
|
if !requireSessionCSRFToken(w, r) {
|
|
return
|
|
}
|
|
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
var req struct {
|
|
Policy *policy_engine.PolicyDocument `json:"policy"`
|
|
}
|
|
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
|
|
return
|
|
}
|
|
if req.Policy == nil {
|
|
writeJSONError(w, http.StatusBadRequest, "policy is required; use DELETE to clear a bucket policy")
|
|
return
|
|
}
|
|
|
|
if err := s.SetBucketPolicy(bucketName, req.Policy); err != nil {
|
|
writeJSONError(w, bucketPolicyErrorStatus(err), "Failed to update bucket policy: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket policy updated successfully",
|
|
"bucket": bucketName,
|
|
})
|
|
}
|
|
|
|
// RemoveBucketPolicy clears the bucket policy for a bucket. Named
|
|
// "Remove", not "Delete", because (*AdminServer).DeleteBucketPolicy is the
|
|
// data-layer method this handler calls.
|
|
func (s *AdminServer) RemoveBucketPolicy(w http.ResponseWriter, r *http.Request) {
|
|
if !requireSessionCSRFToken(w, r) {
|
|
return
|
|
}
|
|
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
if err := s.DeleteBucketPolicy(bucketName); err != nil {
|
|
writeJSONError(w, bucketPolicyErrorStatus(err), "Failed to delete bucket policy: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket policy deleted successfully",
|
|
"bucket": bucketName,
|
|
})
|
|
}
|
|
|
|
// bucketPolicyErrorStatus keeps a request for a bucket that does not exist,
|
|
// or an invalid policy document, out of the 5xx bucket where a client
|
|
// would retry it. Mirrors bucketLifecycleErrorStatus.
|
|
func bucketPolicyErrorStatus(err error) int {
|
|
if errors.Is(err, ErrBucketNotFound) {
|
|
return http.StatusNotFound
|
|
}
|
|
if errors.Is(err, ErrInvalidBucketPolicy) {
|
|
return http.StatusBadRequest
|
|
}
|
|
return http.StatusInternalServerError
|
|
}
|
|
|
|
// CreateBucket creates a new S3 bucket
|
|
func (s *AdminServer) CreateBucket(w http.ResponseWriter, r *http.Request) {
|
|
var req CreateBucketRequest
|
|
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
|
|
return
|
|
}
|
|
if strings.TrimSpace(req.Name) == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
// Validate bucket name (basic validation)
|
|
if len(req.Name) < 3 || len(req.Name) > 63 {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name must be between 3 and 63 characters")
|
|
return
|
|
}
|
|
|
|
// Validate object lock settings
|
|
if req.ObjectLockEnabled {
|
|
// Object lock requires versioning to be enabled
|
|
req.VersioningEnabled = true
|
|
|
|
// Validate object lock mode
|
|
if req.ObjectLockMode != "GOVERNANCE" && req.ObjectLockMode != "COMPLIANCE" {
|
|
writeJSONError(w, http.StatusBadRequest, "Object lock mode must be either GOVERNANCE or COMPLIANCE")
|
|
return
|
|
}
|
|
|
|
// Validate retention duration if default retention is enabled
|
|
if req.SetDefaultRetention {
|
|
if req.ObjectLockDuration <= 0 {
|
|
writeJSONError(w, http.StatusBadRequest, "Object lock duration must be greater than 0 days when default retention is enabled")
|
|
return
|
|
}
|
|
}
|
|
}
|
|
|
|
normalizedUnit, err := normalizeQuotaUnit(req.QuotaUnit)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
req.QuotaUnit = normalizedUnit
|
|
quotaBytes := convertQuotaToBytes(req.QuotaSize, normalizedUnit)
|
|
|
|
// Validate quota: if enabled, size must be greater than 0
|
|
if req.QuotaEnabled && quotaBytes <= 0 {
|
|
writeJSONError(w, http.StatusBadRequest, "Quota size must be greater than 0 when quota is enabled")
|
|
return
|
|
}
|
|
|
|
// Sanitize owner: trim whitespace and enforce max length
|
|
owner := strings.TrimSpace(req.Owner)
|
|
if len(owner) > MaxOwnerNameLength {
|
|
writeJSONError(w, http.StatusBadRequest, fmt.Sprintf("Owner name must be %d characters or less", MaxOwnerNameLength))
|
|
return
|
|
}
|
|
|
|
err = s.CreateS3BucketWithObjectLock(req.Name, quotaBytes, req.QuotaEnabled, req.VersioningEnabled, req.ObjectLockEnabled, req.ObjectLockMode, req.SetDefaultRetention, req.ObjectLockDuration, owner)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to create bucket: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusCreated, map[string]interface{}{
|
|
"message": "Bucket created successfully",
|
|
"bucket": req.Name,
|
|
"quota_size": req.QuotaSize,
|
|
"quota_unit": req.QuotaUnit,
|
|
"quota_enabled": req.QuotaEnabled,
|
|
"versioning_enabled": req.VersioningEnabled,
|
|
"object_lock_enabled": req.ObjectLockEnabled,
|
|
"object_lock_mode": req.ObjectLockMode,
|
|
"object_lock_duration": req.ObjectLockDuration,
|
|
"owner": owner,
|
|
})
|
|
}
|
|
|
|
// UpdateBucketQuota updates the quota settings for a bucket
|
|
func (s *AdminServer) UpdateBucketQuota(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
var req struct {
|
|
QuotaSize int64 `json:"quota_size"`
|
|
QuotaUnit string `json:"quota_unit"`
|
|
QuotaEnabled bool `json:"quota_enabled"`
|
|
}
|
|
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
|
|
return
|
|
}
|
|
|
|
if req.QuotaEnabled && req.QuotaSize <= 0 {
|
|
writeJSONError(w, http.StatusBadRequest, "quota_size must be > 0 when quota_enabled is true")
|
|
return
|
|
}
|
|
|
|
normalizedUnit, err := normalizeQuotaUnit(req.QuotaUnit)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, err.Error())
|
|
return
|
|
}
|
|
req.QuotaUnit = normalizedUnit
|
|
// Convert quota to bytes
|
|
quotaBytes := convertQuotaToBytes(req.QuotaSize, normalizedUnit)
|
|
|
|
err = s.SetBucketQuota(bucketName, quotaBytes, req.QuotaEnabled)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to update bucket quota: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket quota updated successfully",
|
|
"bucket": bucketName,
|
|
"quota_size": req.QuotaSize,
|
|
"quota_unit": req.QuotaUnit,
|
|
"quota_enabled": req.QuotaEnabled,
|
|
})
|
|
}
|
|
|
|
// DeleteBucket deletes an S3 bucket
|
|
func (s *AdminServer) DeleteBucket(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
err := s.DeleteS3Bucket(bucketName)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to delete bucket: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket deleted successfully",
|
|
"bucket": bucketName,
|
|
})
|
|
}
|
|
|
|
// UpdateBucketOwner updates the owner of an S3 bucket
|
|
func (s *AdminServer) UpdateBucketOwner(w http.ResponseWriter, r *http.Request) {
|
|
bucketName := mux.Vars(r)["bucket"]
|
|
if bucketName == "" {
|
|
writeJSONError(w, http.StatusBadRequest, "Bucket name is required")
|
|
return
|
|
}
|
|
|
|
// Use pointer to detect if owner field was explicitly provided
|
|
var req struct {
|
|
Owner *string `json:"owner"`
|
|
}
|
|
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
|
|
return
|
|
}
|
|
|
|
// Require owner field to be explicitly provided
|
|
if req.Owner == nil {
|
|
writeJSONError(w, http.StatusBadRequest, "Owner field is required (use empty string to clear owner)")
|
|
return
|
|
}
|
|
|
|
// Trim and validate owner
|
|
owner := strings.TrimSpace(*req.Owner)
|
|
if len(owner) > MaxOwnerNameLength {
|
|
writeJSONError(w, http.StatusBadRequest, fmt.Sprintf("Owner name must be %d characters or less", MaxOwnerNameLength))
|
|
return
|
|
}
|
|
|
|
err := s.SetBucketOwner(bucketName, owner)
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to update bucket owner: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"message": "Bucket owner updated successfully",
|
|
"bucket": bucketName,
|
|
"owner": owner,
|
|
})
|
|
}
|
|
|
|
// SetBucketOwner sets the owner of a bucket
|
|
func (s *AdminServer) SetBucketOwner(bucketName string, owner string) error {
|
|
return s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
|
// Get the current bucket entry
|
|
lookupResp, err := client.LookupDirectoryEntry(context.Background(), &filer_pb.LookupDirectoryEntryRequest{
|
|
Directory: "/buckets",
|
|
Name: bucketName,
|
|
})
|
|
if err != nil {
|
|
return fmt.Errorf("lookup bucket %s: %w", bucketName, err)
|
|
}
|
|
|
|
bucketEntry := lookupResp.Entry
|
|
|
|
// Initialize Extended map if nil
|
|
if bucketEntry.Extended == nil {
|
|
bucketEntry.Extended = make(map[string][]byte)
|
|
}
|
|
|
|
// Set or remove the owner. The account id recorded by the S3 API goes with
|
|
// it: the S3 API derives it from the owning identity when it is absent, so
|
|
// leaving the old one behind would keep the previous owner of the objects.
|
|
delete(bucketEntry.Extended, s3_constants.ExtAmzOwnerKey)
|
|
if owner == "" {
|
|
delete(bucketEntry.Extended, s3_constants.AmzIdentityId)
|
|
} else {
|
|
bucketEntry.Extended[s3_constants.AmzIdentityId] = []byte(owner)
|
|
}
|
|
|
|
// Update the entry
|
|
_, err = client.UpdateEntry(context.Background(), &filer_pb.UpdateEntryRequest{
|
|
Directory: "/buckets",
|
|
Entry: bucketEntry,
|
|
})
|
|
if err != nil {
|
|
return fmt.Errorf("failed to update bucket owner: %w", err)
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|
|
|
|
// ListBucketsAPI returns the list of buckets as JSON
|
|
func (s *AdminServer) ListBucketsAPI(w http.ResponseWriter, r *http.Request) {
|
|
buckets, err := s.GetS3Buckets()
|
|
if err != nil {
|
|
writeJSONError(w, http.StatusInternalServerError, "Failed to get buckets: "+err.Error())
|
|
return
|
|
}
|
|
|
|
writeJSON(w, http.StatusOK, map[string]interface{}{
|
|
"buckets": buckets,
|
|
"total": len(buckets),
|
|
})
|
|
}
|
|
|
|
// Helper function to convert quota size and unit to bytes
|
|
func convertQuotaToBytes(size int64, unit string) int64 {
|
|
if size <= 0 {
|
|
return 0
|
|
}
|
|
|
|
switch unit {
|
|
case "TB":
|
|
return size * 1024 * 1024 * 1024 * 1024
|
|
case "GB":
|
|
return size * 1024 * 1024 * 1024
|
|
case "MB":
|
|
return size * 1024 * 1024
|
|
case "KB":
|
|
return size * 1024
|
|
case "B":
|
|
return size
|
|
default:
|
|
return 0
|
|
}
|
|
}
|
|
|
|
func normalizeQuotaUnit(unit string) (string, error) {
|
|
normalized := strings.ToUpper(strings.TrimSpace(unit))
|
|
if normalized == "" {
|
|
return "MB", nil
|
|
}
|
|
switch normalized {
|
|
case "B", "KB", "MB", "GB", "TB":
|
|
return normalized, nil
|
|
default:
|
|
return "", fmt.Errorf("unsupported quota unit: %s", unit)
|
|
}
|
|
}
|
|
|
|
// SetBucketQuota sets the quota for a bucket
|
|
func (s *AdminServer) SetBucketQuota(bucketName string, quotaBytes int64, quotaEnabled bool) error {
|
|
return s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
|
// Get the current bucket entry
|
|
lookupResp, err := client.LookupDirectoryEntry(context.Background(), &filer_pb.LookupDirectoryEntryRequest{
|
|
Directory: "/buckets",
|
|
Name: bucketName,
|
|
})
|
|
if err != nil {
|
|
return fmt.Errorf("bucket not found: %w", err)
|
|
}
|
|
|
|
bucketEntry := lookupResp.Entry
|
|
|
|
// Determine quota value (negative if disabled)
|
|
var quota int64
|
|
if quotaEnabled && quotaBytes > 0 {
|
|
quota = quotaBytes
|
|
} else if !quotaEnabled && quotaBytes > 0 {
|
|
quota = -quotaBytes
|
|
} else {
|
|
quota = 0
|
|
}
|
|
|
|
// Update the quota
|
|
bucketEntry.Quota = quota
|
|
|
|
// Update the entry
|
|
_, err = client.UpdateEntry(context.Background(), &filer_pb.UpdateEntryRequest{
|
|
Directory: "/buckets",
|
|
Entry: bucketEntry,
|
|
})
|
|
if err != nil {
|
|
return fmt.Errorf("failed to update bucket quota: %w", err)
|
|
}
|
|
|
|
if quota <= 0 {
|
|
// with no active quota, quota enforcement can no longer clear a
|
|
// read-only flag it turned on, so lift it here
|
|
if _, err := filer.ClearBucketReadOnly(context.Background(), client, "/buckets", bucketName); err != nil {
|
|
return fmt.Errorf("failed to clear bucket read-only flag: %w", err)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|
|
|
|
// CreateS3BucketWithQuota creates a new S3 bucket with quota settings
|
|
func (s *AdminServer) CreateS3BucketWithQuota(bucketName string, quotaBytes int64, quotaEnabled bool) error {
|
|
return s.CreateS3BucketWithObjectLock(bucketName, quotaBytes, quotaEnabled, false, false, "", false, 0, "")
|
|
}
|
|
|
|
// CreateS3BucketWithObjectLock creates a new S3 bucket with quota, versioning, object lock settings, and owner
|
|
func (s *AdminServer) CreateS3BucketWithObjectLock(bucketName string, quotaBytes int64, quotaEnabled, versioningEnabled, objectLockEnabled bool, objectLockMode string, setDefaultRetention bool, objectLockDuration int32, owner string) error {
|
|
return s.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
|
|
// First ensure /buckets directory exists
|
|
_, err := client.CreateEntry(context.Background(), &filer_pb.CreateEntryRequest{
|
|
Directory: "/",
|
|
Entry: &filer_pb.Entry{
|
|
Name: "buckets",
|
|
IsDirectory: true,
|
|
Attributes: &filer_pb.FuseAttributes{
|
|
FileMode: uint32(0755 | os.ModeDir), // Directory mode
|
|
Uid: uint32(1000),
|
|
Gid: uint32(1000),
|
|
Crtime: time.Now().Unix(),
|
|
Mtime: time.Now().Unix(),
|
|
TtlSec: 0,
|
|
},
|
|
},
|
|
})
|
|
// Ignore error if directory already exists
|
|
if err != nil && !strings.Contains(err.Error(), "already exists") && !strings.Contains(err.Error(), "existing entry") {
|
|
return fmt.Errorf("failed to create /buckets directory: %w", err)
|
|
}
|
|
|
|
// Check if bucket already exists
|
|
_, err = client.LookupDirectoryEntry(context.Background(), &filer_pb.LookupDirectoryEntryRequest{
|
|
Directory: "/buckets",
|
|
Name: bucketName,
|
|
})
|
|
if err == nil {
|
|
return fmt.Errorf("bucket %s already exists", bucketName)
|
|
}
|
|
|
|
// Determine quota value (negative if disabled)
|
|
var quota int64
|
|
if quotaEnabled && quotaBytes > 0 {
|
|
quota = quotaBytes
|
|
} else if !quotaEnabled && quotaBytes > 0 {
|
|
quota = -quotaBytes
|
|
} else {
|
|
quota = 0
|
|
}
|
|
|
|
// Prepare bucket attributes with versioning and object lock metadata
|
|
attributes := &filer_pb.FuseAttributes{
|
|
FileMode: uint32(0755 | os.ModeDir), // Directory mode
|
|
Uid: filer_pb.OS_UID,
|
|
Gid: filer_pb.OS_GID,
|
|
Crtime: time.Now().Unix(),
|
|
Mtime: time.Now().Unix(),
|
|
TtlSec: 0,
|
|
}
|
|
|
|
// Create extended attributes map for versioning and owner
|
|
extended := make(map[string][]byte)
|
|
|
|
// Set bucket owner if specified
|
|
if owner != "" {
|
|
extended[s3_constants.AmzIdentityId] = []byte(owner)
|
|
}
|
|
|
|
// Create bucket entry
|
|
bucketEntry := &filer_pb.Entry{
|
|
Name: bucketName,
|
|
IsDirectory: true,
|
|
Attributes: attributes,
|
|
Extended: extended,
|
|
Quota: quota,
|
|
}
|
|
|
|
// Handle versioning using shared utilities
|
|
if err := s3api.StoreVersioningInExtended(bucketEntry, versioningEnabled); err != nil {
|
|
return fmt.Errorf("failed to store versioning configuration: %w", err)
|
|
}
|
|
|
|
// Handle Object Lock configuration using shared utilities
|
|
if objectLockEnabled {
|
|
var duration int32 = 0
|
|
var mode string = ""
|
|
|
|
if setDefaultRetention {
|
|
// Validate Object Lock parameters only when setting default retention
|
|
if err := s3api.ValidateObjectLockParameters(objectLockEnabled, objectLockMode, objectLockDuration); err != nil {
|
|
return fmt.Errorf("invalid Object Lock parameters: %w", err)
|
|
}
|
|
duration = objectLockDuration
|
|
mode = objectLockMode
|
|
}
|
|
|
|
// Create Object Lock configuration using shared utility
|
|
objectLockConfig := s3api.CreateObjectLockConfigurationFromParams(objectLockEnabled, mode, duration)
|
|
|
|
// Store Object Lock configuration in extended attributes using shared utility
|
|
if err := s3api.StoreObjectLockConfigurationInExtended(bucketEntry, objectLockConfig); err != nil {
|
|
return fmt.Errorf("failed to store Object Lock configuration: %w", err)
|
|
}
|
|
}
|
|
|
|
// Create bucket directory under /buckets
|
|
_, err = client.CreateEntry(context.Background(), &filer_pb.CreateEntryRequest{
|
|
Directory: "/buckets",
|
|
Entry: bucketEntry,
|
|
})
|
|
if err != nil {
|
|
return fmt.Errorf("failed to create bucket directory: %w", err)
|
|
}
|
|
|
|
return nil
|
|
})
|
|
}
|