Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-22 05:54:24 +02:00
Code Issues Packages Projects Releases Wiki Activity
11,839 Commits 212 Branches 361 Tags
5c1e6e904d3ba902c9d6f11b19aa2c6b33d39c28
Commit Graph
11839 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu 5c1e6e904d CRITICAL: Restore task conflict definitions to prevent data integrity issues 
- Restore conflicts between vacuum, balance, erasure_coding, and ec_vacuum tasks
- Prevent dangerous concurrent operations on same volumes/resources
- Add comprehensive task conflict matrix to avoid race conditions
- This addresses a serious safety regression where all conflicts were removed

Critical conflicts restored:
- vacuum ↔ balance, erasure_coding, ec_vacuum
- balance ↔ vacuum, erasure_coding, ec_vacuum
- erasure_coding ↔ vacuum, balance, ec_vacuum
- ec_vacuum ↔ vacuum, balance, erasure_coding
- replication ↔ vacuum, balance (destructive ops)
 2025-08-10 17:56:13 -07:00
chrislu 04e3a68b61 Replace deprecated ioutil with os package functions 
- Replace ioutil.WriteFile with os.WriteFile
- Replace ioutil.ReadFile with os.ReadFile
- Remove deprecated io/ioutil import
- Aligns with Go 1.16+ best practices and modern Go standards
 2025-08-10 17:53:48 -07:00
chrislu c220ad1e69 Replace bubble sort with idiomatic sort.Slice in EC shard management 
- Replace O(n²) bubble sort implementation with efficient sort.Slice
- More concise, readable, and performant for larger slices
- Uses idiomatic Go sorting pattern
 2025-08-10 17:52:30 -07:00
chrislu c9ca213d4c Fix linter error in master_grpc_ec_generation_test.go 
Remove orphaned code that was causing syntax error
 2025-08-10 17:51:28 -07:00
chrislu 4ec743583d Address PR #7116 review comments 
- Fix CodeQL security issue: Add bounds checking for int64 to uint8 conversion in disk_location_ec.go
- Replace goto with idiomatic map approach in ec_shard_management.go
- Fix EC volume handling in maintenance_scanner.go: add support for EC-only volumes
- Fix test failures in master_grpc_ec_generation_test.go: handle raft leadership issues
 2025-08-10 17:50:10 -07:00
chrislu 5bb475c572 Fixed CodeQL Security Issue 2025-08-10 17:48:29 -07:00
chrislu cd75202da8 Replaced goto with Idiomatic Map Approach 2025-08-10 17:48:18 -07:00
chrislu cdba261913 add safety check 2025-08-10 17:41:55 -07:00
chrislu 179bcb777e Delete ec_vacuum_integration_test.go 2025-08-10 17:34:03 -07:00
chrislu 802267b281 validates all critical upgrade scenarios for the generation-aware EC vacuum system 2025-08-10 17:33:08 -07:00
chrislu d6e335d6e2 Integration test: end-to-end vacuum G→G+1 with live reads (zero downtime) 2025-08-10 17:23:17 -07:00
chrislu 884da0496c testing mixed generation 2025-08-10 17:12:20 -07:00
chrislu 60f975aa20 add unit tests 2025-08-10 17:07:29 -07:00
chrislu a3d0509c92 logs 2025-08-10 16:57:08 -07:00
chrislu 2e51e1dab2 ec volume UI rendering version 2025-08-10 16:54:44 -07:00
chrislu a187f103d1 normal volume CompactionRevision 2025-08-10 16:22:36 -07:00
chrislu 3087da07db metrics with generation 2025-08-10 16:17:46 -07:00
chrislu 3ef8a9f3b2 Mixed-version cluster compatibility 2025-08-10 15:54:30 -07:00
chrislu d326affc4f default generation 2025-08-10 15:44:08 -07:00
chrislu d341db8861 Update ec_vacuum_task.go 2025-08-10 15:39:26 -07:00
chrislu cf5f0b8e74 cleanupGracePeriod 2025-08-10 15:13:12 -07:00
chrislu 0bcb64140f Add activation step to EC vacuum: call ActivateEcGeneration when G+1 ready 2025-08-10 15:07:57 -07:00
chrislu 62d89fa60b EC vacuum distribution updated for generation-aware mount/copy RPCs 2025-08-10 15:04:55 -07:00
chrislu de9399761b EC vacuum task updated for generation-aware operation 2025-08-10 15:00:22 -07:00
chrislu 8797e73523 cachedLookupEcShardLocations updated for generation-specific caching 2025-08-10 14:51:55 -07:00
chrislu f00dc46607 VolumeEcShardRead to read from correct (vid, generation) EcVolume 2025-08-10 14:44:27 -07:00
chrislu 1b41544f97 detecting ec volumes 2025-08-10 14:41:06 -07:00
chrislu e9033136aa todo 2025-08-10 14:24:59 -07:00
chrislu 56d1ed77c1 copy file 2025-08-10 14:24:30 -07:00
chrislu 9e2e600b6d VolumeEcShardsGenerate updated for generation-specific file creation 2025-08-10 14:14:37 -07:00
chrislu 99f132729c MountEcShards/UnmountEcShards updated for generation support 2025-08-10 14:09:07 -07:00
chrislu 8c31d5e331 EcVolume creation properly refactored 2025-08-10 13:45:08 -07:00
chrislu ef5f9f629a Generation file layout 2025-08-10 13:32:20 -07:00
chrislu 50cc17e8fa ActivateEcGeneration RPC implemented 2025-08-10 13:11:47 -07:00
chrislu 870b2ffe07 LookupEcVolume handle generation parameter 2025-08-10 13:06:37 -07:00
chrislu e4f266d927 Active generation tracking implemented 2025-08-10 13:03:52 -07:00
chrislu 5650b6f505 Add master_pb.ActivateEcGenerationRequest/Response RPC definition 2025-08-10 12:53:10 -07:00
chrislu 09d740db47 Add generation field to volume_server_pb.VolumeEcShardReadRequest 2025-08-10 12:51:27 -07:00
chrislu 8c54b72af3 Add generation field to volume_server_pb VolumeEcShards{Copy,Generate,Rebuild,ToVolume}Request 2025-08-10 12:51:01 -07:00
chrislu 312fee4ddf Add generation field to volume_server_pb VolumeEcShards{Mount,Unmount}Request 2025-08-10 12:49:43 -07:00
chrislu 03ad8f2490 Add generation field to master_pb.VolumeEcShardInformationMessage 2025-08-10 12:47:47 -07:00
chrislu fc666e2e48 collect ec volume deleted bytes 2025-08-10 12:37:18 -07:00
chrislu 3a9ec6f30c Update worker.go 2025-08-10 02:16:23 -07:00
chrislu 8603a3a1e9 ec vacuum task need to register itself 2025-08-10 02:16:15 -07:00
chrislu bea9ca3616 generate ec data and deleted data 2025-08-10 02:15:55 -07:00
chrislu 32e7ba2f9d adding ec vacuum 2025-08-10 00:58:26 -07:00
chrislu 05a0cc156b Self-Contained Design 
To prove the system is truly self-contained:
To add a new task:
Create a task package (e.g., worker/tasks/compression/)
Import it: _ "github.com/.../worker/tasks/compression"
That's it! No other changes needed.
To remove a task:
Delete the task package directory
Remove the import line
That's it! No other changes needed.
 2025-08-10 00:15:26 -07:00
chrislu 96d6d27607 remove ❌ Vacuum - Completely removed ❌ Balance - Completely removed 2025-08-10 00:00:46 -07:00
chrislu c7f291e4e4 simplify logging 2025-08-09 23:49:45 -07:00
Chris LuCopilotgemini-code-assist[bot]
 		25bbf4c3d4 Admin UI: Fetch task logs (#7114) 
* show task details

* loading tasks

* task UI works

* generic rendering

* rendering the export link

* removing placementConflicts from task parameters

* remove TaskSourceLocation

* remove "Server ID" column

* rendering balance task source

* sources and targets

* fix ec task generation

* move info

* render timeline

* simplified worker id

* simplify

* read task logs from worker

* isValidTaskID

* address comments

* Update weed/worker/tasks/balance/execution.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/erasure_coding/ec_task.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/task_log_handler.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix shard ids

* plan distributing shard id

* rendering planned shards in task details

* remove Conflicts

* worker logs correctly

* pass in dc and rack

* task logging

* Update weed/admin/maintenance/maintenance_queue.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* display log details

* logs have fields now

* sort field keys

* fix link

* fix collection filtering

* avoid hard coded ec shard counts

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-09 21:47:29 -07:00