Github-Mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-09-08 15:41:15 +02:00
Code Issues Packages Projects Releases Wiki Activity
6,778 Commits 177 Branches 359 Tags
5c87fcc6d28b230154db35cbe7735a5f1b84024f
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
chrislu 5c87fcc6d2 add client id for all metadata listening clients 2021-12-30 00:23:57 -08:00
chrislu fb434318e3 dynamically adjust connection timeout 
better fix for https://github.com/chrislusf/seaweedfs/issues/2541
 2021-12-29 22:44:39 -08:00
chrislu 5788bf2270 s3: increase timeout limit 
https://github.com/chrislusf/seaweedfs/issues/2541
 2021-12-29 22:21:02 -08:00
chrislu d351541757 upgrade fuse version 2021-12-28 18:18:54 -08:00
chrislu 498661e3bb mount: remove limits on number of parallel requests 2021-12-28 17:41:01 -08:00
chrislu 9a00c17555 reader: avoid wrong pattern detection due to lock waiting 2021-12-28 16:30:33 -08:00
chrislu f7a6f6b4c0 if this is enabled, there are some "bus error" with git clone 2021-12-28 13:44:52 -08:00
chrislu 0da2dfd640 fuse: change to direct io mode 
before and after:

chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 4.534068 secs (236816430 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192  0.01s user 3.86s system 84% cpu 4.561 total
chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192
8192+0 records in
8192+0 records out
1073741824 bytes transferred in 3.824072 secs (280784948 bytes/sec)
dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192  0.01s user 3.22s system 83% cpu 3.857 total
 2021-12-28 12:22:56 -08:00
chrislu 3fd4da34a4 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2021-12-28 01:10:38 -08:00
chrislu 2422556456 monitor write pattern: avoid timing due to locking 2021-12-28 01:10:35 -08:00
chrislu 80db8b13d8 bug: cleanup function was called twice 2021-12-27 20:53:02 -08:00
chrislu 67b723f74e Filer Server API support fsync 
fix https://github.com/chrislusf/seaweedfs/issues/2528
 2021-12-26 17:28:47 -08:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls 
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
 2021-12-26 00:15:03 -08:00
chrislu c935b9669e 2.83 2021-12-25 01:01:34 -08:00
chrislu eb4ad2546f use proper chunk size limit option 2021-12-24 22:52:18 -08:00
chrislu 41bbf320bb use 2MB chunk size. cache size is the wrong option 2021-12-24 22:50:19 -08:00
chrislu 982ea85d81 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2021-12-24 22:40:07 -08:00
chrislu 083d8e9ece add stream writer 
this should improve streaming write performance, which is common in many cases, e.g., copying large files.

This is additional to improved random read write operations: https://github.com/chrislusf/seaweedfs/wiki/FUSE-Mount/_compare/3e69d193805c79802f4f8f6cc63269b7a9a911f3...19084d87918f297cac15e2471c19306176e0771f
 2021-12-24 22:38:22 -08:00
chrislu 255a1c7dcd refactor type names 2021-12-23 18:23:18 -08:00
chrislu f77ca41769 refactor 2021-12-23 17:48:34 -08:00
chrislu 1d36884845 rename files 2021-12-23 17:47:58 -08:00
chrislu 2d1a1f5e03 rename variables and functions 2021-12-23 17:35:57 -08:00
chrislu 7bf48ee135 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2021-12-23 17:23:26 -08:00
chrislu 6de331b014 clean up 2021-12-23 17:23:21 -08:00
chrislu 032df784ed chunked file works now 2021-12-23 17:17:32 -08:00
chrislu c2aad1c7ff detect non streaming mode on first write request 2021-12-22 17:20:44 -08:00
chrislu b541e39a2c fix tests 2021-12-22 16:17:30 -08:00
chrislu 2bc6fa90ff Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2021-12-22 16:05:43 -08:00
chrislu 0ec7bc6710 detect non streaming mode on the first read 2021-12-22 16:05:38 -08:00
chrislu 4c1368d621 fix test 2021-12-22 16:05:08 -08:00
chrislu 7b78fc72b0 add page chunk interval list 2021-12-22 02:53:33 -08:00
chrislu b7cd52636b Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2021-12-21 22:24:43 -08:00
chrislu 3981d65b68 remove println 2021-12-21 22:24:38 -08:00
chrislu 9a73319b45 mount: different write strategy for streaming write and random write 2021-12-21 17:28:55 -08:00
chrislu 4b8dcff448 reverting default admin scripts 
fix https://github.com/chrislusf/seaweedfs/issues/2525

this new default value was introduced in 2.80
this affects production environments, e.g., EC is not desired, volume balancing is not preferred, etc.
 2021-12-20 13:34:57 -08:00
chrislu b21a67bbe6 add writer pattern object for later use 2021-12-20 11:53:48 -08:00
chrislu 4fd29dad86 remove writeOnly flag 2021-12-20 01:11:43 -08:00
chrislu bc96682760 refactor, change file locations 2021-12-20 01:02:23 -08:00
chrislu 866c2657f0 avoid FUSE cache only for the first 512 bytes 2021-12-19 23:13:36 -08:00
chrislu 0cb9036f66 mount: only cache the first chunk on stream read 2021-12-19 23:06:03 -08:00
chrislu a152f17937 mount: improve read performance on random reads 2021-12-19 22:43:14 -08:00
chrislu 85c526c583 s3: bind to a specific IP 
fix https://github.com/chrislusf/seaweedfs/issues/2516
 2021-12-17 11:34:37 -08:00
chrislu 5eacff9d4f log message adds server name 
address https://github.com/chrislusf/seaweedfs/issues/2514#issuecomment-995925733
 2021-12-16 10:46:26 -08:00
chrislu 50ddd8c8e2 remove debug messages 
fix https://github.com/chrislusf/seaweedfs/issues/2514
 2021-12-16 00:58:15 -08:00
chrislu 8329cf86a7 Merge branch 'content_disposition_download' 2021-12-15 13:19:40 -08:00
chrislu 7210558c7b s3: pass through s3 presigned headers 
fix https://github.com/chrislusf/seaweedfs/discussions/2502
 2021-12-15 13:18:53 -08:00
chrislu bf4d7affc0 gateway to remote object store: skip replicating multipart upload part files 
fix https://github.com/chrislusf/seaweedfs/issues/2509
 2021-12-14 19:48:31 -08:00
chrislu 8dca561bd1 adjust the name to be consistent 
comes from https://github.com/chrislusf/seaweedfs/pull/2511
 2021-12-14 19:39:08 -08:00
chrislu 316f326464 add more help message 2021-12-13 13:14:36 -08:00
chrislu c3b73ec23b 2.82 2021-12-12 23:25:24 -08:00
chrislu 7bf14452f7 s3: handle Response-Content-Disposition 2021-12-12 23:05:40 -08:00
chrislu 94e5c0a454 skip purging from ec volumes 2021-12-12 22:55:27 -08:00
Chris Lu 689f5513a9 redis3 supports sentinel 2021-11-29 01:09:51 -08:00
Chris Lu 7227cfddf5 2.80 2021-11-29 00:57:08 -08:00
Chris Lu 3d7390302d add s3.clean.uploads -timeAgo=24h 2021-11-29 00:49:49 -08:00
Chris Lu 2f72c24498 skip the rest logic 2021-11-29 00:32:21 -08:00
Chris Lu 88ff8fc27b ensure uploaded chunks are deleted on error 2021-11-29 00:28:26 -08:00
Chris Lu bea26549f2 avoid empty extended attribute value 2021-11-28 23:50:52 -08:00
Chris Lu ce2af0082e revert 2021-11-28 23:35:22 -08:00
Chris Lu 1c9f3c7ac0 read deleted chunks when replcating data 2021-11-28 23:34:34 -08:00
Chris Lu 96c66ca2aa read deleted chunks when replicating data 2021-11-28 23:33:03 -08:00
Chris Lu f3a334965d Update command_remote_uncache.go 
minor
 2021-11-28 23:10:32 -08:00
Chris Lu 9ccfc1cfee Update command_volume_fsck.go 
minor
 2021-11-28 22:39:24 -08:00
Chris Lu 71a94267c9 Update command_volume_fsck.go 
remove unused code
 2021-11-28 22:37:11 -08:00
Chris Lu ad16221a35 adjust error log 2021-11-28 22:06:17 -08:00
Chris Lu cf1586a34d add logs for writing to remote file 2021-11-27 22:09:23 -08:00
Chris Lu 3a19eea97c allocate memory by slabs 2021-11-27 12:13:00 -08:00
Chris Lu da1c755f6a fix build issue 
fix https://github.com/wangyoucao577/go-release-action/issues/66
 2021-11-23 07:39:43 -08:00
Chris Lu 6c27845be0 add retries when writing to remote s3 
fix https://github.com/chrislusf/seaweedfs/issues/2465
 2021-11-22 21:48:04 -08:00
Chris Lu f3c789d662 2.79 2021-11-21 18:40:24 -08:00
Chris Lu 3d87aa767d fix same dc and other dc 2021-11-16 09:14:01 -08:00
Chris Lu a0ef6e3611 prevent nil response 
fix https://github.com/chrislusf/seaweedfs/issues/2452
 2021-11-15 08:46:01 -08:00
Chris Lu 791aa3cb44 update to wangyoucao577/go-release-action@v1.21 2021-11-15 08:33:46 -08:00
Chris Lu 320a08f250 adjust go release action version 2021-11-14 23:45:34 -08:00
Chris Lu 100c654ec3 2.78 2021-11-14 23:29:59 -08:00
Chris Lu 947fe4cc97 docker: entry point change working directory from root to /data 2021-11-12 17:21:49 -08:00
Chris Lu 7bf891c00a randomize same-dc servers and other-dc servers 2021-11-12 11:30:11 -08:00
Chris Lu 1f75f1f9dc filer: fix mysql2 SQL template 2021-11-11 22:28:28 -08:00
Chris Lu 3abbaccb70 filer: fix mysql command to upsert 2021-11-11 22:27:13 -08:00
Chris Lu 177129013c releases in github actions disable http2client 2021-11-11 21:28:06 -08:00
Chris Lu c77c0de418 make install 2021-11-09 12:34:57 -08:00
Chris Lu 1d4a61af5e add brokers 2021-11-09 08:50:55 -08:00
Chris Lu 59d1435d65 s3: avoid possible attacks by version 2021-11-08 17:47:56 -08:00
Chris Lu 5c99b8acc3 rename variables 2021-11-08 17:47:56 -08:00
Chris Lu 00fdab1284 adjust client name 2021-11-08 17:47:56 -08:00
Chris Lu 4729a57cc0 use constants 2021-11-08 17:47:56 -08:00
Chris Lu d9dd72ea56 rename pacakge 2021-11-08 17:47:56 -08:00
Chris Lu 5cf332357b 2.77 2021-11-07 13:52:45 -08:00
Chris Lu 244841f080 adjust logs 2021-11-07 13:27:57 -08:00
Chris Lu 6e999f1176 s3: fix ListObject if more than 10000 objects 2021-11-07 12:39:36 -08:00
Chris Lu a8b0f8864d fix help message 2021-11-07 12:38:35 -08:00
Chris Lu 55973e8572 log unknown access key 2021-11-07 12:37:46 -08:00
Chris Lu c4e22b5a9a filer: deprecate "-peers" option 2021-11-06 14:36:45 -07:00
Chris Lu 751a7073e3 Merge branch 'flexible_filer_addition' 2021-11-06 14:26:26 -07:00
Chris Lu e0fc2898e9 auto updated filer peer list 2021-11-06 14:23:35 -07:00
Chris Lu 04663c3611 remote.mount: print out metadata sync errors 2021-11-06 11:29:50 -07:00
Chris Lu 330d1fde7f send peers info to filers 2021-11-06 11:29:50 -07:00
Chris Lu 84bb8e7365 send peers info to filers 2021-11-06 04:07:38 -07:00
Chris Lu 4b9c42996a refactor grpc API 2021-11-05 18:11:40 -07:00
Chris Lu 5ea86ef1da Revert "master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()" 
This reverts commit af71ae11aa.
 2021-11-05 17:52:15 -07:00