mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-12 09:17:44 +02:00
Chris Lu
9f6feef299
* feat(s3api): add bucket quota S3 extension via ?seaweedfs-quota
Add a SeaweedFS-specific S3 subresource for bucket quota management:
PUT /{bucket}?seaweedfs-quota — set bucket quota (s3:PutBucketQuota)
GET /{bucket}?seaweedfs-quota — get bucket quota (s3:GetBucketQuota)
The request/response body is JSON:
{"quota_size": 100, "quota_unit": "GB", "quota_enabled": true}
Quota is stored on the bucket's filer entry (positive = enabled,
negative = disabled but retained, zero = no quota), matching the
existing admin REST API behavior. When quota is cleared, the bucket's
read-only flag is also lifted.
Authentication uses the existing S3 SigV4 flow — no new global secret
is needed. Authorization uses two new dedicated IAM permissions:
s3:PutBucketQuota
s3:GetBucketQuota
This allows integrations like Apache CloudStack to manage per-bucket
quotas through the S3 endpoint with a scoped credential, without
exposing the broad admin REST API or requiring a separate admin token.
The credential can be limited to s3:PutBucketQuota/s3:GetBucketQuota
only, preventing bucket deletion, user management, or cluster topology
changes.
The coarse-grained ACTION_PUT_BUCKET_QUOTA/ACTION_GET_BUCKET_QUOTA
constants are added to s3_constants, and the action resolver maps the
seaweedfs-quota query parameter to the fine-grained s3: actions for
policy evaluation.
* docs: update design for S3 ?seaweedfs-quota extension approach
Replace the broad admin REST API + bearer-token design with the narrow,
scoped S3 ?seaweedfs-quota extension. Update quota, usage reporting, and
SeaweedFS-side changes sections to reflect PR #11279.
* fix(s3api): address review comments on quota handler
Fix four issues identified by Devin, Greptile, and CodeRabbit reviews:
1. Integer overflow in convertQuotaToBytes: large quota_size values
(e.g. 8388608 TB) could overflow int64, wrapping to negative and
being silently treated as zero quota. Now returns an error when
size * multiplier would exceed math.MaxInt64.
2. Disabled quotas returned negative sizes in GET: the GET handler
returned entry.Quota directly, which is negative for disabled-but-
retained quotas. Now returns the absolute magnitude as quota_size
and derives quota_enabled from the sign, making the response
round-trippable.
3. Missing buckets returned 500 instead of NoSuchBucket: the PUT
handler treated all lookup failures as internal errors. Now
distinguishes filer_pb.ErrNotFound and returns ErrNoSuchBucket.
4. Trailing JSON was silently accepted: the decoder read only the
first JSON object without checking for trailing data. Now
requires EOF after the object, rejecting malformed payloads.
Also add tests for overflow detection and trailing data rejection.
* fix(s3api): cast math.MaxInt64 to int64 for 32-bit vet
On 32-bit platforms, math.MaxInt64 is an untyped int constant that
overflows int (32-bit) when used directly in fmt.Errorf with %d.
Cast to int64 explicitly to fix Go Vet 32-bit.
* docs: reconcile design doc with implementation and add AWS tools note
- Resolve open question about IAM endpoint path: driver accepts optional
iamUrl and defaults to <s3Url>/iam
- Add note explaining ?seaweedfs-quota is not callable by standard AWS tools
(aws s3api, s3cmd, rclone), and how this compares to MinIO and Ceph quota
APIs which also live outside the standard S3 API
* docs: fix IAM endpoint default — SeaweedFS IAM is at POST / on S3 endpoint
SeaweedFS registers its embedded IAM API at POST / on the same S3
endpoint (UnifiedPostHandler), not under /iam. The design doc
previously said the driver defaults iamUrl to <s3Url>/iam, which would
send IAM operations to an unregistered path. Correct the default to
s3Url.
Found by Greptile review on PR #11279.
* docs: fix credential model, signer, and GET response shape in design doc
Three issues found by CodeRabbit review on PR #11279:
1. Credential-scope contradiction: the doc claimed the service credential
is scoped to only s3:PutBucketQuota/s3:GetBucketQuota, but the
implementation uses it as the admin credential for all operations
(bucket CRUD, IAM user provisioning, quota). Document the actual
model.
2. S3Signer -> AWSS3V4Signer: the doc said 'S3Signer for SigV4 signing'
but S3Signer is legacy SigV2. Correct to AWSS3V4Signer.
3. GET response shape: the doc showed a single JSON example with 'GB'
for both PUT and GET, but GET always returns quota_unit 'B' and the
absolute byte count. Document PUT input and GET response separately.
173 lines
7.5 KiB
Go
173 lines
7.5 KiB
Go
package s3_constants
|
|
|
|
// S3 action strings for bucket policy evaluation
|
|
// These match the official AWS S3 action format used in IAM and bucket policies
|
|
const (
|
|
// Object operations
|
|
S3_ACTION_GET_OBJECT = "s3:GetObject"
|
|
S3_ACTION_PUT_OBJECT = "s3:PutObject"
|
|
S3_ACTION_DELETE_OBJECT = "s3:DeleteObject"
|
|
S3_ACTION_DELETE_OBJECT_VERSION = "s3:DeleteObjectVersion"
|
|
S3_ACTION_GET_OBJECT_VERSION = "s3:GetObjectVersion"
|
|
S3_ACTION_GET_OBJECT_ATTRIBUTES = "s3:GetObjectAttributes"
|
|
|
|
// Object ACL operations
|
|
S3_ACTION_GET_OBJECT_ACL = "s3:GetObjectAcl"
|
|
S3_ACTION_PUT_OBJECT_ACL = "s3:PutObjectAcl"
|
|
|
|
// Object tagging operations
|
|
S3_ACTION_GET_OBJECT_TAGGING = "s3:GetObjectTagging"
|
|
S3_ACTION_PUT_OBJECT_TAGGING = "s3:PutObjectTagging"
|
|
S3_ACTION_DELETE_OBJECT_TAGGING = "s3:DeleteObjectTagging"
|
|
|
|
// Object retention and legal hold
|
|
S3_ACTION_GET_OBJECT_RETENTION = "s3:GetObjectRetention"
|
|
S3_ACTION_PUT_OBJECT_RETENTION = "s3:PutObjectRetention"
|
|
S3_ACTION_GET_OBJECT_LEGAL_HOLD = "s3:GetObjectLegalHold"
|
|
S3_ACTION_PUT_OBJECT_LEGAL_HOLD = "s3:PutObjectLegalHold"
|
|
S3_ACTION_BYPASS_GOVERNANCE = "s3:BypassGovernanceRetention"
|
|
|
|
// Multipart upload operations
|
|
S3_ACTION_CREATE_MULTIPART = "s3:CreateMultipartUpload"
|
|
S3_ACTION_UPLOAD_PART = "s3:UploadPart"
|
|
S3_ACTION_COMPLETE_MULTIPART = "s3:CompleteMultipartUpload"
|
|
S3_ACTION_ABORT_MULTIPART = "s3:AbortMultipartUpload"
|
|
S3_ACTION_UPLOAD_PART_COPY = "s3:UploadPartCopy"
|
|
S3_ACTION_LIST_PARTS = "s3:ListMultipartUploadParts"
|
|
S3_ACTION_LIST_MULTIPART_UPLOADS = "s3:ListBucketMultipartUploads"
|
|
|
|
// Bucket operations
|
|
S3_ACTION_CREATE_BUCKET = "s3:CreateBucket"
|
|
S3_ACTION_DELETE_BUCKET = "s3:DeleteBucket"
|
|
S3_ACTION_LIST_BUCKET = "s3:ListBucket"
|
|
S3_ACTION_LIST_BUCKET_VERSIONS = "s3:ListBucketVersions"
|
|
|
|
// Bucket ACL operations
|
|
S3_ACTION_GET_BUCKET_ACL = "s3:GetBucketAcl"
|
|
S3_ACTION_PUT_BUCKET_ACL = "s3:PutBucketAcl"
|
|
|
|
// Bucket policy operations
|
|
S3_ACTION_GET_BUCKET_POLICY = "s3:GetBucketPolicy"
|
|
S3_ACTION_PUT_BUCKET_POLICY = "s3:PutBucketPolicy"
|
|
S3_ACTION_DELETE_BUCKET_POLICY = "s3:DeleteBucketPolicy"
|
|
|
|
// Bucket tagging operations
|
|
S3_ACTION_GET_BUCKET_TAGGING = "s3:GetBucketTagging"
|
|
S3_ACTION_PUT_BUCKET_TAGGING = "s3:PutBucketTagging"
|
|
S3_ACTION_DELETE_BUCKET_TAGGING = "s3:DeleteBucketTagging"
|
|
|
|
// Bucket CORS operations
|
|
S3_ACTION_GET_BUCKET_CORS = "s3:GetBucketCors"
|
|
S3_ACTION_PUT_BUCKET_CORS = "s3:PutBucketCors"
|
|
S3_ACTION_DELETE_BUCKET_CORS = "s3:DeleteBucketCors"
|
|
|
|
// Bucket lifecycle operations
|
|
// Note: Both PUT and DELETE lifecycle operations use s3:PutLifecycleConfiguration
|
|
S3_ACTION_GET_BUCKET_LIFECYCLE = "s3:GetLifecycleConfiguration"
|
|
S3_ACTION_PUT_BUCKET_LIFECYCLE = "s3:PutLifecycleConfiguration"
|
|
|
|
// Bucket versioning operations
|
|
S3_ACTION_GET_BUCKET_VERSIONING = "s3:GetBucketVersioning"
|
|
S3_ACTION_PUT_BUCKET_VERSIONING = "s3:PutBucketVersioning"
|
|
|
|
// Bucket location
|
|
S3_ACTION_GET_BUCKET_LOCATION = "s3:GetBucketLocation"
|
|
|
|
// Bucket notification
|
|
S3_ACTION_GET_BUCKET_NOTIFICATION = "s3:GetBucketNotification"
|
|
S3_ACTION_PUT_BUCKET_NOTIFICATION = "s3:PutBucketNotification"
|
|
|
|
// Bucket object lock operations
|
|
S3_ACTION_GET_BUCKET_OBJECT_LOCK = "s3:GetBucketObjectLockConfiguration"
|
|
S3_ACTION_PUT_BUCKET_OBJECT_LOCK = "s3:PutBucketObjectLockConfiguration"
|
|
|
|
// Bucket encryption operations
|
|
// Note: DELETE bucket encryption uses s3:PutEncryptionConfiguration, matching AWS
|
|
S3_ACTION_GET_BUCKET_ENCRYPTION = "s3:GetEncryptionConfiguration"
|
|
S3_ACTION_PUT_BUCKET_ENCRYPTION = "s3:PutEncryptionConfiguration"
|
|
|
|
// Bucket request payment operations
|
|
S3_ACTION_GET_BUCKET_REQUEST_PAYMENT = "s3:GetBucketRequestPayment"
|
|
S3_ACTION_PUT_BUCKET_REQUEST_PAYMENT = "s3:PutBucketRequestPayment"
|
|
|
|
// Bucket public access block operations
|
|
// Note: DELETE uses s3:PutBucketPublicAccessBlock, matching AWS
|
|
S3_ACTION_GET_BUCKET_PUBLIC_ACCESS_BLOCK = "s3:GetBucketPublicAccessBlock"
|
|
S3_ACTION_PUT_BUCKET_PUBLIC_ACCESS_BLOCK = "s3:PutBucketPublicAccessBlock"
|
|
|
|
// Bucket ownership controls operations
|
|
// Note: DELETE uses s3:PutBucketOwnershipControls, matching AWS
|
|
S3_ACTION_GET_BUCKET_OWNERSHIP_CONTROLS = "s3:GetBucketOwnershipControls"
|
|
S3_ACTION_PUT_BUCKET_OWNERSHIP_CONTROLS = "s3:PutBucketOwnershipControls"
|
|
|
|
// SeaweedFS extension: bucket quota operations
|
|
// PUT /{bucket}?seaweedfs-quota and GET /{bucket}?seaweedfs-quota
|
|
// These are SeaweedFS-specific subresources, not part of the AWS S3 API.
|
|
S3_ACTION_PUT_BUCKET_QUOTA = "s3:PutBucketQuota"
|
|
S3_ACTION_GET_BUCKET_QUOTA = "s3:GetBucketQuota"
|
|
|
|
// Wildcard for all S3 actions
|
|
S3_ACTION_ALL = "s3:*"
|
|
)
|
|
|
|
// S3 Tables action strings for policy evaluation.
|
|
// Source of truth for the operation names: the dispatch switch in
|
|
// weed/s3api/s3tables/handler.go. Keep this list in sync with that switch
|
|
// when operations are added, renamed, or removed.
|
|
const (
|
|
// Table bucket operations
|
|
S3TABLES_ACTION_CREATE_TABLE_BUCKET = "s3tables:CreateTableBucket"
|
|
S3TABLES_ACTION_GET_TABLE_BUCKET = "s3tables:GetTableBucket"
|
|
S3TABLES_ACTION_LIST_TABLE_BUCKETS = "s3tables:ListTableBuckets"
|
|
S3TABLES_ACTION_DELETE_TABLE_BUCKET = "s3tables:DeleteTableBucket"
|
|
|
|
// Table bucket policy operations
|
|
S3TABLES_ACTION_PUT_TABLE_BUCKET_POLICY = "s3tables:PutTableBucketPolicy"
|
|
S3TABLES_ACTION_GET_TABLE_BUCKET_POLICY = "s3tables:GetTableBucketPolicy"
|
|
S3TABLES_ACTION_DELETE_TABLE_BUCKET_POLICY = "s3tables:DeleteTableBucketPolicy"
|
|
|
|
// Namespace operations
|
|
S3TABLES_ACTION_CREATE_NAMESPACE = "s3tables:CreateNamespace"
|
|
S3TABLES_ACTION_GET_NAMESPACE = "s3tables:GetNamespace"
|
|
S3TABLES_ACTION_UPDATE_NAMESPACE = "s3tables:UpdateNamespace"
|
|
S3TABLES_ACTION_LIST_NAMESPACES = "s3tables:ListNamespaces"
|
|
S3TABLES_ACTION_DELETE_NAMESPACE = "s3tables:DeleteNamespace"
|
|
|
|
// Table operations
|
|
S3TABLES_ACTION_CREATE_TABLE = "s3tables:CreateTable"
|
|
S3TABLES_ACTION_REGISTER_TABLE = "s3tables:RegisterTable"
|
|
S3TABLES_ACTION_GET_TABLE = "s3tables:GetTable"
|
|
S3TABLES_ACTION_LIST_TABLES = "s3tables:ListTables"
|
|
S3TABLES_ACTION_UPDATE_TABLE = "s3tables:UpdateTable"
|
|
S3TABLES_ACTION_DELETE_TABLE = "s3tables:DeleteTable"
|
|
S3TABLES_ACTION_RENAME_TABLE = "s3tables:RenameTable"
|
|
|
|
// View operations
|
|
S3TABLES_ACTION_CREATE_VIEW = "s3tables:CreateView"
|
|
S3TABLES_ACTION_GET_VIEW = "s3tables:GetView"
|
|
S3TABLES_ACTION_LIST_VIEWS = "s3tables:ListViews"
|
|
S3TABLES_ACTION_UPDATE_VIEW = "s3tables:UpdateView"
|
|
S3TABLES_ACTION_DELETE_VIEW = "s3tables:DeleteView"
|
|
S3TABLES_ACTION_RENAME_VIEW = "s3tables:RenameView"
|
|
|
|
// Table policy operations
|
|
S3TABLES_ACTION_PUT_TABLE_POLICY = "s3tables:PutTablePolicy"
|
|
S3TABLES_ACTION_GET_TABLE_POLICY = "s3tables:GetTablePolicy"
|
|
S3TABLES_ACTION_DELETE_TABLE_POLICY = "s3tables:DeleteTablePolicy"
|
|
|
|
// Maintenance configuration operations
|
|
S3TABLES_ACTION_PUT_TABLE_BUCKET_MAINTENANCE_CONFIGURATION = "s3tables:PutTableBucketMaintenanceConfiguration"
|
|
S3TABLES_ACTION_GET_TABLE_BUCKET_MAINTENANCE_CONFIGURATION = "s3tables:GetTableBucketMaintenanceConfiguration"
|
|
S3TABLES_ACTION_PUT_TABLE_MAINTENANCE_CONFIGURATION = "s3tables:PutTableMaintenanceConfiguration"
|
|
S3TABLES_ACTION_GET_TABLE_MAINTENANCE_CONFIGURATION = "s3tables:GetTableMaintenanceConfiguration"
|
|
S3TABLES_ACTION_GET_TABLE_MAINTENANCE_JOB_STATUS = "s3tables:GetTableMaintenanceJobStatus"
|
|
|
|
// Tagging operations
|
|
S3TABLES_ACTION_TAG_RESOURCE = "s3tables:TagResource"
|
|
S3TABLES_ACTION_LIST_TAGS_FOR_RESOURCE = "s3tables:ListTagsForResource"
|
|
S3TABLES_ACTION_UNTAG_RESOURCE = "s3tables:UntagResource"
|
|
|
|
// Wildcard for all S3 Tables actions
|
|
S3TABLES_ACTION_ALL = "s3tables:*"
|
|
)
|