Chris Lu
6e477ae00f
filer: keep the internal .system folder out of the per-bucket store path ( #10248 )
...
* filer: keep .system internal folder in the default SQL table
Bucket-table SQL stores read the first path segment under /buckets as a
bucket name. The ListBuckets owner index lives at /buckets/.system/..., so
every write there hit isValidBucket(".system") == false and returned
"invalid bucket name .system", flooding the filer log on the postgres/mysql
backends. Route dot-prefixed internal folders to the default table by their
full path, like any other non-bucket entry.
* filer: keep .system internal folder in the default leveldb3 DB
Same guard as the SQL stores: leveldb3 would otherwise open a separate DB
for the .system owner-index folder instead of keeping it in the default DB.
* filer: keep .system internal folder under the default ydb prefix
Skips a DescribeTable round trip per operation on the .system owner-index
path, which never resolves to a real bucket table.
* filer: keep .system internal folder in the default arangodb collection
Avoids creating a stray collection for the .system owner-index folder.
2026-07-06 11:25:50 -07:00
Chris Lu
83195fc111
filer: reuse the caller's fetched entry in CreateEntry ( #9645 )
...
CreateEntry starts with a FindEntry to load the current entry. A conditional
CreateEntry already fetched that entry to evaluate the precondition under the
per-path lock, so the create repeated the lookup.
Add an existing *Entry parameter: when non-nil it is used as the current entry
and the internal lookup is skipped; nil keeps the lookup. The gRPC CreateEntry
handler passes the entry it fetched for the precondition, removing the redundant
read while the lock is held. All other callers pass nil.
2026-05-23 21:40:41 -07:00
promalert and Chris Lu
9012069bd7
chore: execute goimports to format the code ( #7983 )
...
* chore: execute goimports to format the code
Signed-off-by: promalert <promalert@outlook.com >
* goimports -w .
---------
Signed-off-by: promalert <promalert@outlook.com >
Co-authored-by: Chris Lu <chris.lu@gmail.com >
2026-01-07 13:06:08 -08:00
b669607fcd
Add error list each entry func ( #7485 )
...
* added error return in type ListEachEntryFunc
* return error if errClose
* fix fmt.Errorf
* fix return errClose
* use %w fmt.Errorf
* added entry in messege error
* add callbackErr in ListDirectoryEntries
* fix error
* add log
* clear err when the scanner stops on io.EOF, so returning err doesn’t surface EOF as a failure.
* more info in error
* add ctx to logs, error handling
* fix return eachEntryFunc
* fix
* fix log
* fix return
* fix foundationdb test s
* fix eachEntryFunc
* fix return resEachEntryFuncErr
* Update weed/filer/filer.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update weed/filer/elastic/v7/elastic_store.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update weed/filer/hbase/hbase_store.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update weed/filer/foundationdb/foundationdb_store.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update weed/filer/ydb/ydb_store.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix
* add scanErr
---------
Co-authored-by: Roman Tamarov <r.tamarov@kryptonite.ru >
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com >
Co-authored-by: chrislu <chris.lu@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-25 19:35:19 -08:00
Chris Lu
69553e5ba6
convert error fromating to %w everywhere ( #6995 )
2025-07-16 23:39:27 -07:00
Aleksey Kosov and akosov
4511c2cc1f
Changes logging function ( #6919 )
...
* updated logging methods for stores
* updated logging methods for stores
* updated logging methods for filer
* updated logging methods for uploader and http_util
* updated logging methods for weed server
---------
Co-authored-by: akosov <a.kosov@kryptonite.ru >
2025-06-24 08:44:06 -07:00
chrislu
cb67137a03
optionally open the leveldb in readonly mode
2024-06-25 21:07:04 -07:00
Konstantin Lebedev and Konstantin Lebedev
1cac5d983d
fix: disallow file name too long when writing a file ( #4881 )
...
* fix: disallow file name too long when writing a file
* bool LongerName to MaxFilenameLength
---------
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co >
2023-10-12 14:29:55 -07:00
Nico D'Cotta
796b7508f3
Implement SRV lookups for filer ( #4767 )
2023-08-24 07:08:56 -07:00
chrislu
70a4c98b00
refactor filer_pb.Entry and filer.Entry to use GetChunks()
...
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
chrislu
fc8241fb5e
leveldb3: add instant dropping bucket
2022-07-21 18:48:51 -07:00
Konstantin Lebedev
d8925b4e83
Merge branch 'new_master' into ydb
...
# Conflicts:
# go.mod
# go.sum
2022-05-03 00:13:57 +05:00
chrislu
94635e9b5c
filer: add filer group
2022-05-01 21:59:16 -07:00
Konstantin Lebedev
21033ff4c3
refactor use const CountEntryChunksForGzip
2022-05-01 22:28:55 +05:00
chrislu
4042fdf3bb
rename to skipCheckParentDir
...
related to https://github.com/chrislusf/seaweedfs/pull/2761
It's better to default to false.
2022-03-16 23:55:31 -07:00
zzq09494
40b0033fa7
go fmt
2022-03-17 14:19:48 +08:00
zzq09494
ee5417a5b3
fix: ut test error
2022-03-17 11:16:46 +08:00
chrislu
56da3494cb
leveldb3: adjust memory allocation for each bucket
2022-02-20 13:40:13 -08:00
Eng Zer Jun
b92df1654c
test: use T.TempDir to create temporary test directory
...
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com >
2022-02-14 10:38:13 +08:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com >
2021-10-14 12:27:58 +08:00
Chris Lu
96514f0f00
fix tests
2021-09-13 02:19:48 -07:00
joshuafc
5654d0d60d
CompactionTableSizeMultiplier of leveldb use default value. #2325
...
To improve performance of leveldb find key in condition of large directory(millions of files) which use uuid as filename.
2021-09-09 10:42:34 +08:00
Chris Lu
bdb632fa62
Merge pull request #2211 from qieqieplus/filer-bloom-filter
...
add bloom filter for filer leveldb/rocksdb
2021-07-22 23:35:54 -07:00
qieqieplus
2b46df13f8
add bloom filter for filer leveldb/rocksdb
2021-07-23 14:05:59 -06:00
Chris Lu
182288f860
filer: fix mysql, postgres batch delete error
2021-07-22 08:23:20 -07:00
Konstantin Lebedev
6aa1a56ec8
avoid crashes Galera Cluster
...
https://github.com/chrislusf/seaweedfs/issues/2125
2021-06-15 18:12:39 +05:00
Chris Lu
ddc8643ee0
filer: directory listing adds namePatternExclude
...
fix https://github.com/chrislusf/seaweedfs/issues/2023
2021-04-24 11:49:03 -07:00
Chris Lu
2d4c2db81d
filer: leveldb, rocksdb auto create store directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1901
2021-03-14 13:20:14 -07:00
Chris Lu
4be51c0701
filer: leveldb and hbase may miss files when listing large directories more than 1024
...
fix https://github.com/chrislusf/seaweedfs/issues/1768
2021-01-31 20:11:44 -08:00
Chris Lu
a4063a5437
add stream list directory entries
2021-01-15 23:56:24 -08:00
Chris Lu
f002e668de
change limit to int64 in case of overflow
2021-01-14 23:10:37 -08:00
Chris Lu
b5ceffe188
implement leveldb changes
2021-01-14 22:33:05 -08:00
Chris Lu
ca73013453
leveldb3: ensure repeatable adding removing buckets
2021-01-13 13:48:48 -08:00
Chris Lu
5cd9c9e642
remove println
2021-01-12 18:47:38 -08:00
Chris Lu
def6d8e565
fix test
2021-01-12 12:32:54 -08:00
Chris Lu
629c9962e7
filer: add leveldb3
...
support dedicated leveldb instance for each bucket
2021-01-12 02:29:44 -08:00
Chris Lu