Commit Graph
100 Commits
Author SHA1 Message Date
chrislu b98c0b4bf4 disable symlink translation for now 2022-07-24 18:46:32 -07:00
chrislu 2dc5d4adac disable symlink translation for now 2022-07-24 18:24:34 -07:00
chrislu 3d2bbf96d2 adjust glog format 2022-07-24 17:29:08 -07:00
chrislu 2807fe08fc adjust glog format 2022-07-24 16:44:58 -07:00
chrislu 2b64916d3e set status to fuse.OK 2022-07-24 15:30:55 -07:00
chrislu 7386cf6a06 mount: resolve symlink 2022-07-24 15:11:24 -07:00
chrislu 40089b6753 maintain node lookup counter 2022-07-24 13:43:11 -07:00
chrislu d24db396cc fix removing path from inode2path 2022-07-23 18:20:29 -07:00
chrislu 41eeb4deef do not add new inode during link 2022-07-23 13:37:07 -07:00
chrislu 02c2d81cde fix removePathFromInode2Path 2022-07-23 13:21:42 -07:00
chrislu 4d5dc55766 mount: better support hard links 2022-07-23 12:57:00 -07:00
chrislu ebda0689ab use linux default max name length 2022-07-23 10:51:06 -07:00
chrislu 64f3d6fb6e metadata subscription uses client epoch 2022-07-23 10:50:28 -07:00
chrislu 2c8818351f mount: comment out stuck fuse inode notify during renaming 2022-07-22 18:17:34 -07:00
chrislu a5c94acd4f mount: check possible nil entry case 2022-07-22 15:33:37 -07:00
chrislu 7a6c559ab4 fix Change replication via volume.configure.replication by collection
fix https://github.com/chrislusf/seaweedfs/issues/3346
2022-07-21 22:01:05 -07:00
chrislu fc8241fb5e leveldb3: add instant dropping bucket 2022-07-21 18:48:51 -07:00
chrislu c93f7ffa44 explicit bucket aware declaration 2022-07-21 18:23:53 -07:00
chrislu 6147b61b0a mount: fix concurrent map read and map write
fix https://github.com/chrislusf/seaweedfs/issues/3344
2022-07-21 01:38:26 -07:00
chrislu 2ae3f812f8 minor 2022-07-19 11:43:31 -07:00
chrislu 56ec89625a 3.16 2022-07-18 00:53:05 -07:00
chrislu 1db012485f mount: fix bug during busy writes
fix https://github.com/chrislusf/seaweedfs/issues/3315
2022-07-15 01:03:17 -07:00
chrislu aca20cd9f4 more logging related to filer metadata subscription 2022-07-15 00:42:13 -07:00
chrislu fa61074513 add clientId logging 2022-07-14 12:27:34 -07:00
chrislu fbd8f868a1 filer may have trouble to re-connect
clientId is used twice: one for local metadata subscription, one for combined metadata subscription.
2022-07-14 12:15:31 -07:00
chrislu 289402a741 writer pattern: similar changes to reader pattern 2022-07-13 02:30:44 -07:00
chrislu 6e90f7bdd0 detect sequential mode with a counter 2022-07-13 02:20:03 -07:00
chrislu 578c8aea0e typo 2022-07-13 02:18:31 -07:00
chrislu 1451b389a4 Avoid fatal error: sync: Unlock of unlocked RWMutex
fix https://github.com/chrislusf/seaweedfs/issues/3306
2022-07-13 00:58:15 -07:00
chrislu 860c207c94 typo 2022-07-12 02:56:19 -07:00
chrislu 37578929d4 fix compilation 2022-07-11 01:30:00 -07:00
chrislu 93ccc6e05f 3.15 2022-07-11 00:50:07 -07:00
chrislu f87952a411 remove locks 2022-07-11 00:25:20 -07:00
chrislu 3d0defa663 fix compilation 2022-07-08 19:16:58 -07:00
chrislu 6f00c7f061 comments 2022-07-08 19:07:31 -07:00
chrislu d685b94102 mount: rename also invalidate source inode 2022-07-08 19:07:03 -07:00
chrislu 48382676d2 fix filtering by volume id 2022-07-08 10:29:24 -07:00
chrislu e2ecf137a2 fix resetting fh.reader 2022-07-08 01:04:15 -07:00
chrislu 185332b91f Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-07-08 00:29:41 -07:00
chrislu 28add5a534 mount: fix racing conditions
prevent wrong reading when the SingleChunkCacher is started, but not finished yet
2022-07-08 00:29:39 -07:00
chrislu a85ed3fe8f minor 2022-07-07 22:27:24 -07:00
chrislu 1f8e73ede7 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-07-07 22:26:06 -07:00
chrislu 2ea18cdcc8 remove dead code 2022-07-07 22:26:03 -07:00
chrislu b7de5c6c43 shell: remove unused filer parameter from shell.toml 2022-07-07 15:01:23 -07:00
chrislu f2f0482dd3 mount: random read also try to use the local cache first 2022-07-07 11:50:28 -07:00
chrislu 05c53820b9 mount: file handler release memory 2022-07-07 11:49:40 -07:00
chrislu 69ef6459c3 S3 ListObjectsV2 fix pagination under bucket root
fix https://github.com/chrislusf/seaweedfs/issues/3166
2022-07-06 02:28:34 -07:00
chrislu 57975b93ff adjust used size reporting 2022-07-04 13:47:42 -07:00
chrislu a79f5bd0d5 adjust used size reporting 2022-07-04 13:46:32 -07:00
chrislu 3c79c77056 3.14 2022-07-04 13:15:49 -07:00
chrislu d0eb65ae7b update go-fuse library version
Make OSXFUSE handle timeouts the same way as Linux
2022-07-03 16:09:03 -07:00
chrislu 141f662734 edge case checking when volume server does not have capacity to balance
fix https://github.com/chrislusf/seaweedfs/issues/3257
2022-07-01 12:16:18 -07:00
chrislu bee3a7c798 fix tests 2022-07-01 12:09:42 -07:00
chrislu 06a8b174b5 also remove Sync() for idx file 2022-06-30 13:50:53 -07:00
chrislu 52580743b9 stuck with file.Sync()
starting to get weed/storage () - (master) > go test ./...
I0630 12:19:54 65819 needle_map_metric_test.go:26] FileCount expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:27] DeletedSize expected 1648 actual 1648
I0630 12:19:54 65819 needle_map_metric_test.go:28] ContentSize expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:29] DeletedCount expected 1648 actual 1959
I0630 12:19:54 65819 needle_map_metric_test.go:30] MaxFileKey expected 10000 actual 10000
I0630 12:19:54 65819 volume_loading.go:136] loading index /var/folders/jv/0rlhdck17jzgt7b3hcnq63mc0000gp/T/TestCompaction909350722/001/1.idx to memory
I0630 12:20:36 65819 volume_vacuum.go:98] Committing volume 1 vacuuming...
panic: test timed out after 10m0s

goroutine 61 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:2029 +0x8e
created by time.goFunc
	/usr/local/go/src/time/sleep.go:176 +0x32

goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc0000cd520, {0x1c09bde?, 0x5ab798ea5c102?}, 0x1c67bd8)
	/usr/local/go/src/testing/testing.go:1487 +0x37a
testing.runTests.func1(0xc0000cd520?)
	/usr/local/go/src/testing/testing.go:1839 +0x6e
testing.tRunner(0xc0000cd520, 0xc00039fcd8)
	/usr/local/go/src/testing/testing.go:1439 +0x102
testing.runTests(0xc0000dadc0?, {0x25b4460, 0x6, 0x6}, {0x2855108?, 0x40?, 0x25c4b80?})
	/usr/local/go/src/testing/testing.go:1837 +0x457
testing.(*M).Run(0xc0000dadc0)
	/usr/local/go/src/testing/testing.go:1719 +0x5d9
main.main()
	_testmain.go:57 +0x1aa

goroutine 21 [chan receive]:
github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).flushDaemon(0x0?)
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:883 +0x6a
created by github.com/chrislusf/seaweedfs/weed/glog.init.0
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:410 +0x1bf

goroutine 55 [syscall]:
syscall.syscall(0x44ac0f96?, 0xc00007ad80?, 0xc00098b480?, 0x10588df?)
	/usr/local/go/src/runtime/sys_darwin.go:22 +0x4e
syscall.fcntl(0x100000001?, 0xc00098b4b0?, 0x1069a7c?)
	/usr/local/go/src/syscall/zsyscall_darwin_amd64.go:319 +0x30
internal/poll.(*FD).Fsync.func1(...)
	/usr/local/go/src/internal/poll/fd_fsync_darwin.go:18
internal/poll.ignoringEINTR(...)
	/usr/local/go/src/internal/poll/fd_posix.go:74
internal/poll.(*FD).Fsync(0xc00038f1e0?)
	/usr/local/go/src/internal/poll/fd_fsync_darwin.go:17 +0xfc
os.(*File).Sync(0xc00012a030)
	/usr/local/go/src/os/file_posix.go:168 +0x4e
github.com/chrislusf/seaweedfs/weed/storage/backend.(*DiskFile).Sync(...)
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/backend/disk_file.go:84
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).makeupDiff(0xc0000bb440, {0xc000633a40, 0x52}, {0xc000633aa0, 0x52}, {0xc000633b00, 0x52}, {0xc000633b60, 0x52})
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:295 +0x12fa
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).CommitCompact(0xc0000bb440)
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:119 +0x3e8
github.com/chrislusf/seaweedfs/weed/storage.TestCompaction(0xc00025a000)
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum_test.go:89 +0x305
testing.tRunner(0xc00025a000, 0x1c67bd8)
	/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1486 +0x35f

goroutine 56 [chan receive, 10 minutes]:
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker.func1()
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:244 +0x41a
created by github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker
	/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:234 +0x56
FAIL	github.com/chrislusf/seaweedfs/weed/storage	600.194s
?   	github.com/chrislusf/seaweedfs/weed/storage/backend	[no test files]
ok  	github.com/chrislusf/seaweedfs/weed/storage/backend/memory_map	(cached)
?   	github.com/chrislusf/seaweedfs/weed/storage/backend/s3_backend	[no test files]
ok  	github.com/chrislusf/seaweedfs/weed/storage/erasure_coding	(cached)
?   	github.com/chrislusf/seaweedfs/weed/storage/idx	[no test files]
ok  	github.com/chrislusf/seaweedfs/weed/storage/needle	(cached)
ok  	github.com/chrislusf/seaweedfs/weed/storage/needle_map	(cached)
ok  	github.com/chrislusf/seaweedfs/weed/storage/super_block	(cached)
?   	github.com/chrislusf/seaweedfs/weed/storage/types	[no test files]
?   	github.com/chrislusf/seaweedfs/weed/storage/volume_info	[no test files]
FAIL
2022-06-30 12:32:55 -07:00
chrislu ffaa2163e0 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-29 12:50:33 -07:00
chrislu f3c4b07b8a Update backers.md 2022-06-29 12:50:31 -07:00
chrislu a8abab2412 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-29 02:44:14 -07:00
chrislu 449ecc8890 remove max connection age
related to https://github.com/chrislusf/seaweedfs/pull/3226
2022-06-29 02:44:12 -07:00
chrislu 4b1f48a399 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-27 12:40:50 -07:00
chrislu 4a65159250 fix reading time 2022-06-27 12:40:47 -07:00
chrislu b9f7b6fb9a adjust log message 2022-06-26 23:12:16 -07:00
chrislu d003bb0166 java 3.13 2022-06-26 20:41:51 -07:00
chrislu 40a9634c3d 3.13 2022-06-26 20:13:52 -07:00
chrislu 9dfea44aeb fix unit 2022-06-26 12:21:38 -07:00
chrislu 3142e77eca mount: stats report physical size instead of logical size 2022-06-26 12:14:34 -07:00
chrislu f77eda4b4f Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-25 09:22:52 -07:00
chrislu 3ce46c3566 fix entry full url lookup 2022-06-25 09:22:49 -07:00
chrislu 9c517d2b35 masterclient: fallback to directly querying master in case of missing volume id location 2022-06-24 02:08:57 -07:00
chrislu 509a9047db test compact map with snowflake sequencer 2022-06-23 21:45:51 -07:00
chrislu 217d7dc377 verify seq is always unique 2022-06-23 21:22:05 -07:00
chrislu 46ae28d64c upgrade github.com/chrislusf/raft
fix https://github.com/chrislusf/seaweedfs/issues/3223
2022-06-23 12:10:23 -07:00
chrislu 28f615dcae java: fix data encryption
fix https://github.com/chrislusf/seaweedfs/issues/3211
2022-06-23 03:05:59 -07:00
chrislu 0c13fc873c fix compilation 2022-06-23 00:45:11 -07:00
chrislu 96496d5286 master: broadcast new volume locations to clients to avoid possible racing condition
fix https://github.com/chrislusf/seaweedfs/issues/3220
2022-06-23 00:41:33 -07:00
chrislu 52c44d646e Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-22 00:34:27 -07:00
chrislu 530da0cc89 remove unused code 2022-06-22 00:34:25 -07:00
chrislu 6b3beb709d fix format 2022-06-21 12:21:02 -07:00
chrislu 4dc27e1ed5 3.12 2022-06-20 19:07:00 -07:00
chrislu 492da3dbce master: put metadata under instance specific folder 2022-06-20 19:04:49 -07:00
chrislu 41dfe27102 Revert "remove max connection age"
This reverts commit b9b684194f.
2022-06-20 14:49:39 -07:00
chrislu 2295766e2c Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-20 14:16:18 -07:00
chrislu 1d0c53ea56 remote storage: stop supporting hdfs as a remote storage 2022-06-20 14:15:59 -07:00
chrislu bf5a5abfb1 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-20 13:18:11 -07:00
chrislu 69a2e503ab non-recursive directory deletion
fix a problem during git checkout a different version, where updated directories are deleted even though the directory is not empty.
2022-06-20 13:18:08 -07:00
chrislu 36931babf9 update go-fuse to latest version 2022-06-20 13:15:20 -07:00
chrislu 76f26c4b99 mount: delete target entry first during move 2022-06-20 13:15:02 -07:00
chrislu 625fd16a2e reduce upfront memory usage for low density volume 2022-06-16 15:39:29 -07:00
chrislu aaa9938353 Revert "adjust conditions"
This reverts commit e024586ff1.
2022-06-16 01:26:36 -07:00
chrislu e024586ff1 adjust conditions 2022-06-15 23:19:39 -07:00
chrislu d4ef06cdcf 3.11 2022-06-14 12:14:52 -07:00
chrislu b347b2fb54 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-14 11:16:51 -07:00
chrislu 36d2756978 volume: avoid writing too much data for range requests in large files
related to https://github.com/chrislusf/seaweedfs/issues/3178
2022-06-14 11:16:47 -07:00
chrislu e437ec5a68 3.10 2022-06-12 19:30:27 -07:00
chrislu d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
chrislu 444ac21050 go fmt 2022-06-11 09:51:11 -07:00
chrislu a74e926063 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-11 09:51:02 -07:00
chrislu 61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
chrislu 9f8b72a54d Revert "Merge pull request #3159 from shichanglin5/_duplicateUUID"
This reverts commit 37da689319, reversing
changes made to 00d53c34c4.
2022-06-10 06:38:17 -07:00
chrislu 8a1166c021 auto remove previous leftover socket 2022-06-08 00:32:09 -07:00