Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-24 06:50:40 +02:00
Code Issues Packages Projects Releases Wiki Activity
11,060 Commits 224 Branches 362 Tags
eac37e41ef4e2c19e02b1046aafdbbc4ce749b84
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu f11567816e Revert "fix: volume data integrity check (#6118)" 
This reverts commit e12b9ab746.
 2024-10-17 18:52:30 -07:00
chrislu 804e765397 remove unused 2024-10-13 23:12:10 -07:00
chrislu 6845e25318 set proxied request content length 2024-10-10 13:55:32 -07:00
chrislu 35fd1e1c9a optimize memory usage for large number of volumes 
1. unwrap the map to avoid extra map object creation
2. fix ec shard counting in UpdateEcShards
 2024-10-10 10:00:30 -07:00
chrislu b28b1a3402 3.77 2024-10-10 01:28:06 -07:00
chrislu 07df5c53de avoid nil 
I1010 03:10:25.220345 mount_std.go:281 This is SeaweedFS version 30GB 3.76 82ed61c655 linux amd64
^[OR^[OR2024/10/10 03:12:56 Unimplemented opcode POLL
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3983b41]
goroutine 112 [running]:
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2.mergeProceesors.1(0xc000513540)
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:30 +0x41
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata.makeSubscribeMetadataFunc.func1({0x514a560, 0xc0009b06d0})
/github/workspace/weed/pb/filer_pb_tail.go:87 +0x293
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1.1(0xc000d9e008)
/github/workspace/weed/mount/wfs_filer_client.go:27 +0x68
github.com/seaweedfs/seaweedfs/weed/pb.WithGrpcClient(0xa9?, 0x9d1f90fb?, 0xc00098fbe0, {0xc000e1c078, 0x15}, 0x0?, {0xc000a76bd0?, 0x1?, 0x1?
})
/github/workspace/weed/pb/grpc_client_server.go:155 +0x33a
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient.func1()
/github/workspace/weed/mount/wfs_filer_client.go:25 +0x14c
github.com/seaweedfs/seaweedfs/weed/util.Retry({0x489a10c, 0xa}, 0xc00098fcf8)
/github/workspace/weed/util/retry.go:16 +0xa7
github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).WithFilerClient(0x0?, 0x30?, 0xc00056a568?)
/github/workspace/weed/mount/wfs_filer_client.go:18 +0x65
github.com/seaweedfs/seaweedfs/weed/pb.WithFilerClientFollowMetadata({0x50f5898, 0xc000e2c180}, 0xc00101e000, 0xc000d22030)
/github/workspace/weed/pb/filer_pb_tail.go:51 +0x87
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func2()
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:115 +0xcb
github.com/seaweedfs/seaweedfs/weed/util.RetryUntil({0x48b8b35, 0x11}, 0xc00098fef0, 0x4af0048)
/github/workspace/weed/util/retry.go:64 +0x4e
github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents(0xc000d1d980, 0x9d1f90fb, {0x50f5898, 0xc000e2c180}, {0x7ffdf6f98dca,
0x4}, 0x17fcf8b7c7ecf8cf, {0xc0000ac208, 0x1, 0x1})
/github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:113 +0x39b
created by github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).StartBackgroundTasks in goroutine 1
/github/workspace/weed/mount/weedfs.go:150 +0x146
 2024-10-09 22:46:00 -07:00
chrislu 82ed61c655 3.76 2024-10-06 19:37:03 -07:00
chrislu c0e36231ad use only one metadata follow process 2024-10-06 12:55:19 -07:00
chrislu a3a8f6217e fix 2024-10-03 09:03:17 -07:00
chrislu 117c3aebda 3.75 2024-10-03 08:33:35 -07:00
chrislu ff47956530 add helper functions 2024-10-03 00:13:46 -07:00
chrislu 11e53dd8ef skip error if conf file is not created 2024-10-01 22:46:48 -07:00
chrislu f747767aa1 support load balancer in front of s3 2024-10-01 12:57:45 -07:00
chrislu 7dd480acad handle missing leading zero padding in replica 2024-10-01 10:16:21 -07:00
chrislu e3fa87bcc1 3.74 2024-09-30 11:29:01 -07:00
chrislu cdd7fa81ab fix help message 2024-09-29 21:53:07 -07:00
chrislu 20929f2a57 adjust resource heavy for volume.fix.replication 2024-09-29 11:32:18 -07:00
chrislu 6564ceda91 skip resource heavy commands from running on master nodes 2024-09-29 10:51:17 -07:00
chrislu ec30a504ba refactor 2024-09-29 10:38:22 -07:00
chrislu 9cd263b2ce refactor 2024-09-29 10:35:53 -07:00
chrislu 701abbb9df add IsResourceHeavy() to command interface 2024-09-28 20:23:01 -07:00
chrislu 4ec6eab004 fix max volume count auto setting 
related to https://github.com/seaweedfs/seaweedfs/issues/6041#issuecomment-2360448182
 2024-09-19 09:13:08 -07:00
chrislu 50e22864cf refactor 2024-09-17 23:10:41 -07:00
chrislu 99ae99e3b9 fix nil filer conf 
fix https://github.com/seaweedfs/seaweedfs/issues/6034
 2024-09-17 11:07:29 -07:00
chrislu a9c9e1bcb3 refactor 2024-09-12 22:45:30 -07:00
chrislu 6063a889ed 3.73 2024-09-12 13:11:55 -07:00
chrislu d660d5c7d4 increasing default cache size 2024-09-10 10:30:19 -07:00
chrislu f23015ac5b weed mount caching 2024-09-10 10:30:05 -07:00
chrislu 93decd1114 remove unused 2024-09-10 10:29:25 -07:00
chrislu 69fcdd0840 adjust logging 2024-09-10 10:28:49 -07:00
chrislu 43d9f68219 more consistent server header for s3 apis 2024-09-09 09:07:58 -07:00
chrislu ff3d46637d better logging for volume growth 2024-09-07 12:38:34 -07:00
chrislu accba3070a refactor 2024-09-07 11:54:22 -07:00
chrislu eb02946c97 support write once read many 
fix https://github.com/seaweedfs/seaweedfs/issues/5954
 2024-09-04 02:25:07 -07:00
chrislu 18afdb15b6 Revert "weed mount, weed dav add option to force cache" 
This reverts commit 7367b976b0.
 2024-09-04 01:38:29 -07:00
chrislu 7367b976b0 weed mount, weed dav add option to force cache 2024-09-04 01:19:14 -07:00
chrislu 66ac82bb8f default cacheDirWrite to cacheDir 2024-09-04 00:05:58 -07:00
chrislu 8d5a6d7e74 fix for spreading ec shards 2024-09-03 22:30:11 -07:00
chrislu 8e4bffc66b copy ec shards to disks already having ec volumes 
fix https://github.com/seaweedfs/seaweedfs/issues/5615
 2024-09-03 21:19:10 -07:00
chrislu 3c0854e986 unnecessary skipping 2024-08-30 16:38:01 -07:00
chrislu 654b8210f7 parameter name 2024-08-30 14:52:07 -07:00
chrislu 8679870008 fix typo 2024-08-30 14:51:55 -07:00
chrislu 22d5d2ea96 adjust warning log 2024-08-30 14:51:42 -07:00
chrislu a4b25a642d math/rand => math/rand/v2 2024-08-29 09:52:21 -07:00
chrislu ded5e084ea ensure none zero lastGrowCount 2024-08-27 09:03:11 -07:00
chrislu 0cf2c15828 rename 2024-08-27 09:02:48 -07:00
chrislu bebc385607 randomly pick one replica to write 2024-08-27 09:02:39 -07:00
chrislu 3b7bb62e38 logs on error 2024-08-26 09:09:11 -07:00
chrislu 4463296811 add parallel vacuuming 2024-08-21 22:53:54 -07:00
chrislu 5535b60e8c fix replication range check 
fix for https://github.com/seaweedfs/seaweedfs/wiki/Replication#replication-string
 2024-08-21 16:58:43 -07:00
chrislu 6ce27affa4 also use /healthz for most consistent health check 2024-08-21 10:55:39 -07:00
chrislu b3696024d1 add warning for not enough copies when skipping vacuuming volumes 
fix https://github.com/seaweedfs/seaweedfs/issues/5906
 2024-08-20 09:39:35 -07:00
chrislu e50d85c0f3 3.72 2024-08-18 21:51:53 -07:00
chrislu e8462ba3ad prevent compacting on the same volume 2024-08-18 12:08:43 -07:00
chrislu 8f1f1730e9 minor 2024-08-15 19:12:11 -07:00
chrislu fdf7193ae7 rename 2024-08-13 13:59:24 -07:00
chrislu 07f4998188 add dat file size into vif for EC 2024-08-13 13:56:00 -07:00
chrislu 3a2e21fee7 fix EC reading on nLargeBlockRows 
fix https://github.com/seaweedfs/seaweedfs/issues/5465
 2024-08-13 12:01:46 -07:00
chrislu 915f9f5054 update java client to 3.71, also adjust the groupId 2024-08-12 17:30:21 -07:00
chrislu e14b6e02df load ec volume id locations 
fix https://github.com/seaweedfs/seaweedfs/issues/5713
 2024-08-12 11:54:25 -07:00
chrislu 480fcb1d03 fix build 2024-08-12 09:45:51 -07:00
chrislu ff66155302 fix tests 2024-08-12 09:30:42 -07:00
chrislu 83fe2bfc36 java 3.71 2024-08-11 21:04:51 -07:00
chrislu c055ee7334 fix reading chunk length calculation 2024-08-11 20:59:32 -07:00
chrislu 270e91b0be adds locking 2024-08-11 13:06:01 -07:00
chrislu b6fd1ff4ce refactor 2024-08-10 12:39:16 -07:00
chrislu 7438648d1c Merge branch 'master' into mq 2024-08-10 10:01:57 -07:00
chrislu e825dd7496 fix opposite logic 
fix https://github.com/seaweedfs/seaweedfs/issues/5871
 2024-08-08 23:16:38 -07:00
chrislu e069009f09 [remote.cache, remote.uncache] fix min/max age 
fix https://github.com/seaweedfs/seaweedfs/issues/5871
 2024-08-08 21:25:52 -07:00
chrislu 9ca30e52d5 fuse mount handles ttl entries 
fix https://github.com/seaweedfs/seaweedfs/issues/5527
 2024-08-08 08:18:54 -07:00
chrislu 2074e7db67 fix invalid toml 2024-08-08 00:03:42 -07:00
chrislu e86b0bcaaa simplify 2024-08-07 10:46:29 -07:00
chrislu 57dc39c451 randomizing next file handle id 2024-08-07 10:42:00 -07:00
chrislu 8378a5b70b rename 2024-08-01 23:54:42 -07:00
chrislu d49ecde535 rename functions 2024-08-01 23:27:23 -07:00
chrislu 4fee496b49 conditional delete 2024-08-01 13:00:54 -07:00
chrislu e2a07d11d5 Revert "Check ShouldGrowVolumes before returning error in assign. (#5819)" 
This reverts commit 98d66338d0333cd955f7840c64ef95e3c4807a17.
 2024-07-26 11:21:50 -07:00
chrislu ec9455dab7 same change as #5819 
follow up with https://github.com/seaweedfs/seaweedfs/pull/5819
 2024-07-26 11:06:47 -07:00
chrislu ed7e721efe 3.71 2024-07-21 21:03:27 -07:00
chrislu 9265be43c0 avoid nil 2024-07-21 21:01:29 -07:00
chrislu 4020149690 3.70 2024-07-21 20:27:39 -07:00
chrislu 9dd008f8f1 add version to filer 2024-07-21 20:13:51 -07:00
chrislu 9dcc576499 Revert "add collection for buckets" 
This reverts commit 96af571219.
 2024-07-20 12:00:50 -07:00
chrislu aed2a18223 get, set and add path conf 2024-07-20 11:27:11 -07:00
chrislu 96af571219 add collection for buckets 2024-07-17 23:14:36 -07:00
chrislu a8fa78b892 refactoring 2024-07-17 23:14:36 -07:00
chrislu 7a75928e7d Azure sink: avoid overwriting existing files 2024-07-17 23:14:36 -07:00
chrislu c6dec11ea5 [filer.sync] skip overwriting existing fresh entry 2024-07-16 09:38:10 -07:00
chrislu 19d14b4c71 minor 2024-07-16 09:21:01 -07:00
chrislu 3a82f5ffad ensure metadata follow a specific folder 
fix https://github.com/seaweedfs/seaweedfs/issues/5774
 2024-07-12 11:17:30 -07:00
chrislu 4e7d8eb3f1 small refactoring 2024-07-11 14:50:11 -07:00
chrislu 2addcd1623 fix test 2024-07-11 07:53:02 -07:00
chrislu 590397229e fix heap init 2024-07-11 00:26:18 -07:00
chrislu 06e3dd306f clean up 2024-07-11 00:19:39 -07:00
chrislu c59517e284 upgrade to github.com/redis/go-redis/v9 
fix https://github.com/seaweedfs/seaweedfs/issues/4532
 2024-07-11 00:15:19 -07:00
chrislu 33b11f92bc Squashed commit of the following: 
commit 465b47b8f72668dfc12eaa72befb232513c9e1db
Merge: f39272b54 9c440d472
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:56:51 2024 -0700

    Merge branch 'master' into merged-persisted-logs

commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:49:26 2024 -0700

    merge meta logs from multiple files
 2024-07-11 00:04:24 -07:00
chrislu 182f6d00a9 always copy ecj files 2024-07-10 01:25:57 -07:00
chrislu b8121e4b24 simplify 2024-07-08 11:10:41 -07:00
chrislu f618121e63 Fix S3 deletion in deep folders, and names with empty spaces 
fix https://github.com/seaweedfs/seaweedfs/issues/5748
 2024-07-08 10:20:09 -07:00
chrislu f3d708d3b6 add a test case 2024-07-07 23:44:36 -07:00