Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-07 23:07:48 +02:00
Code Issues Packages Projects Releases Wiki Activity
11,725 Commits 200 Branches 361 Tags
1bc503530a4e7c566242e0b9067f18daec2a349c
Commit Graph
7518 Commits
Author SHA1 Message Date
chrislu 1bc503530a fixes 
With these final fixes, the s3-tests should now:
✅ Return MalformedXML for ObjectLockEnabled: 'Disabled'
✅ Return MalformedXML when both Days and Years are specified in retention configuration
✅ Return InvalidBucketState (409 Conflict) when trying to suspend versioning on buckets with object lock enabled
✅ Handle all object lock validation errors consistently with proper error codes
 2025-07-18 15:52:28 -07:00
chrislu 9f88fd2ea5 fixes 
With these comprehensive fixes, the s3-tests should now:
✅ Return InvalidBucketState (409 Conflict) for object lock operations on invalid buckets
✅ Return InvalidRetentionPeriod for invalid retention periods
✅ Return MalformedXML for malformed retention configurations
✅ Include VersionId in response headers when available
✅ Return proper HTTP status codes for all error conditions
✅ Handle all object lock validation errors consistently
The workflow should now pass significantly more object lock tests, bringing SeaweedFS's S3 object lock implementation much closer to AWS S3 compatibility standards.
 2025-07-18 15:45:15 -07:00
chrislu 0123abe49b address test errors 
With these fixes, the s3-tests should now:
✅ Return InvalidBucketState (409 Conflict) for object lock operations on invalid buckets
✅ Return MalformedXML for invalid retention configurations
✅ Include VersionId in response headers when available
✅ Return proper HTTP status codes (403 Forbidden for retention mode changes)
✅ Handle all object lock validation errors consistently
 2025-07-18 15:39:30 -07:00
chrislu 4b92f03b65 address test errors 2025-07-18 15:33:44 -07:00
chrislu 9f8e0257b0 read entry once 2025-07-18 14:39:02 -07:00
chrislu e53cd29eb2 errorf 2025-07-18 02:14:19 -07:00
chrislu eeeb595346 change error style 2025-07-18 02:02:54 -07:00
Chris LuandCopilot 678f962ded Update weed/s3api/object_lock_utils.go 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-18 01:55:43 -07:00
Chris LuandCopilot 12f34f965f Update weed/s3api/s3api_object_handlers_put.go 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-18 01:54:14 -07:00
chrislu 1db149ecc2 address comments 2025-07-18 01:49:44 -07:00
chrislu bcc7a74a25 use fields instead of xml 2025-07-18 01:39:23 -07:00
chrislu 0d4eeb67be admin ui and api handler are consistent now 2025-07-18 01:27:46 -07:00
chrislu 85e34df214 fix routing 2025-07-18 01:27:27 -07:00
chrislu bb05e6cade Update s3api_object_handlers_put.go 2025-07-18 00:53:20 -07:00
chrislu bc46214445 rename variable 2025-07-18 00:52:38 -07:00
chrislu 121f6ef2c0 avoid duplidated work 2025-07-18 00:48:56 -07:00
Chris Luandgemini-code-assist[bot] 22673fee76 Update weed/s3api/s3api_server.go 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-18 00:47:24 -07:00
chrislu 3fbab20b83 refactor 2025-07-18 00:34:16 -07:00
chrislu b8cbe9b6b2 increase bucket config cache TTL 2025-07-18 00:26:58 -07:00
chrislu 8bb47f3c03 subscribe to bucket configuration changes 2025-07-18 00:25:18 -07:00
chrislu 7bec2ef94b cache and use bucket object lock config 2025-07-18 00:19:47 -07:00
chrislu f9cbed7ee1 fix GetObjectLockConfigurationHandler 2025-07-17 23:12:19 -07:00
Chris Lu 69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
Chris LuandCopilot a524b4f485 Object locking need to persist the tags and set the headers (#6994) 
* fix object locking read and write

No logic to include object lock metadata in HEAD/GET response headers
No logic to extract object lock metadata from PUT request headers

* add tests for object locking

* Update weed/s3api/s3api_object_handlers_put.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor

* add unit tests

* sync versions

* Update s3_worm_integration_test.go

* fix legal hold values

* lint

* fix tests

* racing condition when enable versioning

* fix tests

* validate put object lock header

* allow check lock permissions for PUT

* default to OFF legal hold

* only set object lock headers for objects that are actually from object lock-enabled buckets

fix     --- FAIL: TestAddObjectLockHeadersToResponse/Handle_entry_with_no_object_lock_metadata (0.00s)

* address comments

* fix tests

* purge

* fix

* refactoring

* address comment

* address comment

* Update weed/s3api/s3api_object_handlers_put.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers_put.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* avoid nil

* ensure locked objects cannot be overwritten

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-16 23:00:25 -07:00
chrislu 89706d36dc less logs 2025-07-16 16:30:22 -07:00
chrislu 22465b8a96 unused 2025-07-16 16:30:07 -07:00
chrislu 215c5de579 minor 2025-07-16 09:22:25 -07:00
chrislu 12c9282042 avoid error overwriting 
fix https://github.com/seaweedfs/seaweedfs/issues/6991
 2025-07-16 09:15:50 -07:00
Chris LuandCopilot dde1cf63c2 S3 Object Lock: ensure x-amz-bucket-object-lock-enabled header (#6990) 
* ensure x-amz-bucket-object-lock-enabled header

* fix tests

* combine 2 metadata changes into one

* address comments

* Update s3api_bucket_handlers.go

* Update weed/s3api/s3api_bucket_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/retention/object_lock_reproduce_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/retention/object_lock_validation_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/retention/s3_bucket_object_lock_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_bucket_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_bucket_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/retention/s3_bucket_object_lock_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_bucket_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* package name

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-15 23:21:58 -07:00
chrislu 64c5dde2f3 support multiple masters 
fix https://github.com/seaweedfs/seaweedfs/issues/6988
 2025-07-15 10:51:07 -07:00
Ibrahim Konsowa d78aa3d2de [Notifications] Improving webhook notifications (#6965) 
* worker setup

* fix tests

* start worker

* graceful worker drain

* retry queue

* migrate queue to watermill

* adding filters and improvements

* add the event type to the webhook message

* eliminating redundant JSON serialization

* resolve review comments

* trigger actions

* fix tests

* typo fixes

* read max_backoff_seconds from config

* add more context to the dead letter

* close the http response on errors

* drain the http response body in case not empty

* eliminate exported typesπ
 2025-07-15 10:49:37 -07:00
Chris LuandCopilot 74f4e9ba5a rewrite, simplify, avoid unused functions (#6989) 
* adding cors support

* address some comments

* optimize matchesWildcard

* address comments

* fix for tests

* address comments

* address comments

* address comments

* path building

* refactor

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* address comment

Service-level responses need both Access-Control-Allow-Methods and Access-Control-Allow-Headers. After setting Access-Control-Allow-Origin and Access-Control-Expose-Headers, also set Access-Control-Allow-Methods: * and Access-Control-Allow-Headers: * so service endpoints satisfy CORS preflight requirements.

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix

* refactor

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_server.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* simplify

* add cors tests

* fix tests

* fix tests

* remove unused functions

* fix tests

* simplify

* address comments

* fix

* Update weed/s3api/auth_signature_v4.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* rename variable

* Revert "Apply suggestion from @Copilot"

This reverts commit fce2d4e57e.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-15 10:11:49 -07:00
Chris LuandCopilot 4b040e8a87 adding cors support (#6987) 
* adding cors support

* address some comments

* optimize matchesWildcard

* address comments

* fix for tests

* address comments

* address comments

* address comments

* path building

* refactor

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* address comment

Service-level responses need both Access-Control-Allow-Methods and Access-Control-Allow-Headers. After setting Access-Control-Allow-Origin and Access-Control-Expose-Headers, also set Access-Control-Allow-Methods: * and Access-Control-Allow-Headers: * so service endpoints satisfy CORS preflight requirements.

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix

* refactor

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_server.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* simplify

* add cors tests

* fix tests

* fix tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-15 00:23:54 -07:00
chrislu 44dfa793d5 Collecting volume locations for volumes before EC encoding 
fix https://github.com/seaweedfs/seaweedfs/issues/6963
 2025-07-14 12:17:33 -07:00
chrislu a51d993aa9 ensure bucket exists 
related to https://github.com/seaweedfs/seaweedfs/issues/6971
 2025-07-14 09:55:35 -07:00
chrislu 406aaf7c14 increase upload limit via browser 2025-07-14 08:42:15 -07:00
chrislu 24eff93d9a 3.94 2025-07-13 20:31:31 -07:00
chrislu e7dfc3552c admin ui adds object lock permissions 2025-07-13 20:29:25 -07:00
Chris Lu 7cb1ca1308 Add policy engine (#6970) 2025-07-13 16:21:36 -07:00
Chris LuandCopilot 1549ee2e15 implement PubObjectRetention and WORM (#6969) 
* implement PubObjectRetention and WORM

* Update s3_worm_integration_test.go

* avoid previous buckets

* Update s3-versioning-tests.yml

* address comments

* address comments

* rename to ExtObjectLockModeKey

* only checkObjectLockPermissions if versioningEnabled

* address comments

* comments

* Revert "comments"

This reverts commit 6736434176.

* Update s3api_object_handlers_skip.go

* Update s3api_object_retention_test.go

* add version id to ObjectIdentifier

* address comments

* add comments

* Add proper error logging for timestamp parsing failures

* address comments

* add version id to the error

* Update weed/s3api/s3api_object_retention_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_retention.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* constants

* fix comments

* address comments

* address comment

* refactor out handleObjectLockAvailabilityCheck

* errors.Is ErrBucketNotFound

* better error checking

* address comments

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-12 21:58:55 -07:00
Chris Lu 687a6a6c1d Admin UI: Add policies (#6968) 
* add policies to UI, accessing filer directly

* view, edit policies

* add back buttons for "users" page

* remove unused

* fix ui dark mode when modal is closed

* bucket view details button

* fix browser buttons

* filer action button works

* clean up masters page

* fix volume servers action buttons

* fix collections page action button

* fix properties page

* more obvious

* fix directory creation file mode

* Update file_browser_handlers.go

* directory permission
 2025-07-12 01:13:11 -07:00
chrislu 49d43003e1 show volume size limit on dashboard 2025-07-11 19:37:09 -07:00
Chris Lu d892538d32 More efficient copy object (#6665) 
* it compiles

* refactored

* reduce to 4 concurrent chunk upload

* CopyObjectPartHandler

* copy a range of the chunk data, fix offset size in copied chunks

* Update s3api_object_handlers_copy.go

What the PR Accomplishes:
CopyObjectHandler - Now copies entire objects by copying chunks individually instead of downloading/uploading the entire file
CopyObjectPartHandler - Handles copying parts of objects for multipart uploads by copying only the relevant chunk portions
Efficient Chunk Copying - Uses direct chunk-to-chunk copying with proper volume assignment and concurrent processing (limited to 4 concurrent operations)
Range Support - Properly handles range-based copying for partial object copies

* fix compilation

* fix part destination

* handling small objects

* use mkFile

* copy to existing file or part

* add testing tools

* adjust tests

* fix chunk lookup

* refactoring

* fix TestObjectCopyRetainingMetadata

* ensure bucket name not conflicting

* fix conditional copying tests

* remove debug messages

* add custom s3 copy tests
 2025-07-11 18:51:32 -07:00
chrislu 4fcbdc1f61 tweaking dashboard UI 2025-07-11 13:11:39 -07:00
chrislu 3d4a9bdac0 upgrade templ version from v0.3.833 to v0.3.906 
// templ: version: v0.3.833
// templ: version: v0.3.906

fix https://github.com/seaweedfs/seaweedfs/issues/6966#issuecomment-3063449163
 2025-07-11 13:03:04 -07:00
Chris Lu 51543bbb87 Admin UI: Add message queue to admin UI (#6958) 
* add a menu item "Message Queue"

* add a menu item "Message Queue"
  * move the "brokers" link under it.
  * add "topics", "subscribers". Add pages for them.

* refactor

* show topic details

* admin display publisher and subscriber info

* remove publisher and subscribers from the topic row pull down

* collecting more stats from publishers and subscribers

* fix layout

* fix publisher name

* add local listeners for mq broker and agent

* render consumer group offsets

* remove subscribers from left menu

* topic with retention

* support editing topic retention

* show retention when listing topics

* create bucket

* Update s3_buckets_templ.go

* embed the static assets into the binary

fix https://github.com/seaweedfs/seaweedfs/issues/6964
 2025-07-11 10:19:27 -07:00
Ibrahim Konsowa 93bbaa1fb4 [Notifications] Support webhook notifications (#6962) 
Add webhook notification support
 2025-07-10 09:22:05 -07:00
chalet 804979d68b [Enhancement] support fix for remote files with command fix (#6961) 2025-07-10 06:13:16 -07:00
Joon Young Baik c04b7b411c refactor: Performance and readability improvement on isDefaultPort (#6960) 2025-07-10 05:50:20 -07:00
chrislu 14859f0e8c add mq agent options to server.go 2025-07-09 09:02:26 -07:00