Commit Graph
7550 Commits
Author SHA1 Message Date
chrislu b80971e85e refactor to use BaseUIProvider 2025-07-26 22:22:50 -07:00
chrislu f7140d4577 reduce duplication 2025-07-26 22:16:24 -07:00
chrislu 6659c213c3 config 2025-07-26 22:06:31 -07:00
chrislu a38465d04c use schema defined defaults 2025-07-26 21:27:02 -07:00
chrislu 319aa61dab MaintenanceConfig use ConfigField 2025-07-26 21:23:19 -07:00
chrislu 4e21b3de78 ensure correct default value 2025-07-26 20:10:22 -07:00
chrislu a24561797f show default values 2025-07-26 19:16:25 -07:00
chrislu 638ab0d6f4 Update ui.go 2025-07-26 19:06:31 -07:00
chrislu 68d86af501 fix ec volumes list 2025-07-26 19:06:14 -07:00
chrislu 13c0826f2f display ec shard sizes 2025-07-26 16:04:18 -07:00
chrislu 965ec4f51b garbage percentage threshold 2025-07-26 13:28:52 -07:00
chrislu 600bfbf671 rename 2025-07-26 13:23:13 -07:00
chrislu 019de6dc7a interval field UI component 2025-07-26 09:22:59 -07:00
chrislu 8dd9b7cccf show ec volumes 2025-07-26 08:55:39 -07:00
chrislu 3b2c75d2bd ec is mostly working now
* distribution of ec shards needs improvement
* need configuration to enable ec
2025-07-25 22:02:28 -07:00
chrislu 379d585cc7 local copy, ec. Need to distribute. 2025-07-25 20:49:48 -07:00
chrislu 0247bf2873 listing ec shards 2025-07-25 19:46:21 -07:00
chrislu 41cbbefb0b move ec task logic 2025-07-25 16:49:12 -07:00
chrislu ffd91383c6 worker can run ec work and report status
but:
1. one volume should not be repeatedly worked on.
2. ec shards needs to be distributed and source data should be deleted.
2025-07-25 15:24:31 -07:00
chrislu 19df3fcc5d worker register itself first 2025-07-25 13:22:33 -07:00
chrislu a3d727034a clean up logs 2025-07-25 13:16:59 -07:00
chrislu c557ec0062 worker reconnect to admin 2025-07-25 02:12:51 -07:00
chrislu 63817fe7ee test canScheduleTaskNow 2025-07-25 01:59:54 -07:00
chrislu afdca513ac sticky worker id 2025-07-25 01:59:22 -07:00
chrislu 14e95186d6 skip task ID column 2025-07-25 00:53:54 -07:00
chrislu 960acc031e rendering pending tasks 2025-07-25 00:48:19 -07:00
chrislu 6cf78754b2 build and restart 2025-07-24 23:50:22 -07:00
chrislu 01e8cc4600 worker retries connecting to admin 2025-07-24 23:24:01 -07:00
chrislu e7084c5b7e compiles 2025-07-24 22:55:22 -07:00
chrislu 16a29b57f9 compiles, detection uses volumeSizeLimitMB from master 2025-07-24 22:37:09 -07:00
chrislu a1966e9692 working worker and admin. Task detection is not working yet. 2025-07-24 21:58:08 -07:00
chrislu cd28ce7786 admin start wtih grpc port. worker has its own working directory 2025-07-24 19:43:10 -07:00
chrislu a4cf55a741 remove "enhanced" reference 2025-07-24 08:07:50 -07:00
chrislu 30ad65bb5f integration test. ec worker task 2025-07-24 07:50:03 -07:00
chrislu 0d8e1f8a17 reorganized the codebase to move the simulation framework and tests into their own dedicated package 2025-07-24 01:08:10 -07:00
chrislu a0874d201c added simulation as tests 2025-07-24 01:00:25 -07:00
chrislu add122484c initial design 2025-07-24 00:37:02 -07:00
chrislu 7ab85c3748 return proper default value for locking and versioning
fix https://github.com/seaweedfs/seaweedfs/issues/6971
fix https://github.com/seaweedfs/seaweedfs/issues/7028
2025-07-23 22:20:48 -07:00
chrislu 4f72a1778f minor 2025-07-23 21:59:50 -07:00
Chris Lu 5ac037f763 change priority of admin credentials from env varaibles (#7032)
* change priority of admin credentials from env varaibles

* address comment
2025-07-23 11:44:36 -07:00
Chris Luandgemini-code-assist[bot] e3d3c495ab S3 API: simpler way to start s3 with credentials (#7030)
* simpler way to start s3 with credentials

* AWS_ACCESS_KEY_ID=access_key AWS_SECRET_ACCESS_KEY=secret_key weed s3

* last adding credentials from env variables

* Update weed/s3api/auth_credentials.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* simplify

* adjust doc

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-07-23 02:05:26 -07:00
Chris Lu 33b9017b48 fix listing objects (#7008)
* fix listing objects

* add more list testing

* address comments

* fix next marker

* fix isTruncated in listing

* fix tests

* address tests

* Update s3api_object_handlers_multipart.go

* fixes

* store json into bucket content, for tagging and cors

* switch bucket metadata from json to proto

* fix

* Update s3api_bucket_config.go

* fix test issue

* fix test_bucket_listv2_delimiter_prefix

* Update cors.go

* skip special characters

* passing listing

* fix test_bucket_list_delimiter_prefix

* ok. fix the xsd generated go code now

* fix cors tests

* fix test

* fix test_bucket_list_unordered and test_bucket_listv2_unordered

do not accept the allow-unordered and delimiter parameter combination

* fix test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous

The tests test_bucket_list_objects_anonymous and test_bucket_listv2_objects_anonymous were failing because they try to set bucket ACL to public-read, but SeaweedFS only supported private ACL.

Updated PutBucketAclHandler to use the existing ExtractAcl function which already supports all standard S3 canned ACLs
Replaced the hardcoded check for only private ACL with proper ACL parsing that handles public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control, etc.
Added unit tests to verify all standard canned ACLs are accepted

* fix list unordered

The test is expecting the error code to be InvalidArgument instead of InvalidRequest

* allow anonymous listing( and head, get)

* fix test_bucket_list_maxkeys_invalid

Invalid values: max-keys=blah → Returns ErrInvalidMaxKeys (HTTP 400)

* updating IsPublicRead when parsing acl

* more logs

* CORS Test Fix

* fix test_bucket_list_return_data

* default to private

* fix test_bucket_list_delimiter_not_skip_special

* default no acl

* add debug logging

* more logs

* use basic http client

remove logs also

* fixes

* debug

* Update stats.go

* debugging

* fix anonymous test expectation

anonymous user can read, as configured in s3 json.
2025-07-22 01:07:15 -07:00
Chris Luandgemini-code-assist[bot] 3a5ee18265 Fix versioning list only (#7015)
* fix listing objects

* address comments

* Update weed/s3api/s3api_object_versioning.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update test/s3/versioning/s3_directory_versioning_test.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-07-21 10:35:21 -07:00
Chris Lu c196d03951 fix listing object versions (#7006)
* fix listing object versions

* Update s3api_object_versioning.go

* Update s3_directory_versioning_test.go

* check previous skipped tests

* fix test_versioning_stack_delete_merkers

* address test_bucket_list_return_data_versioning

* Update s3_directory_versioning_test.go

* fix test_versioning_concurrent_multi_object_delete

* fix test_versioning_obj_suspend_versions test

* fix empty owner

* fix listing versioned objects

* default owner

* fix path
2025-07-21 00:23:22 -07:00
chrislu bfe68984d5 fix logging 2025-07-20 20:02:44 -07:00
Chris Lu 377f1f24c7 add basic object ACL (#7004)
* add back tests

* get put object acl

* check permission to put object acl

* rename file

* object list versions now contains owners

* set object owner

* refactoring

* Revert "add back tests"

This reverts commit 9adc507c45.
2025-07-20 14:15:25 -07:00
Chris Lu 85036936d1 Read write directory object (#7003)
* read directory object

* address comments

* address comments

* name should not have "/" prefix

* fix compilation

* refactor
2025-07-20 13:28:17 -07:00
Chris Lu 41b5bac063 read directory object (#7002)
* read directory object

* address comments

* address comments
2025-07-20 09:40:47 -07:00
chrislu 394e42cd51 3.95 2025-07-19 23:57:36 -07:00
Chris Lu 530b6e5ef1 add CORS tests (#7001)
* add CORS tests

* parallel tests

* Always attempt compaction when compactSnapshots is called

* start servers

* fix port

* revert

* debug ports

* fix ports

* debug

* Update s3tests.yml

* Update s3tests.yml

* Update s3tests.yml

* Update s3tests.yml

* Update s3tests.yml
2025-07-19 23:56:17 -07:00