Commit Graph
29 Commits
Author SHA1 Message Date
Chris Lu e383ee47cb filer: use bind variables for request-controlled values in the arangodb store (#10795)
* arangodb: bind list prefix, start file name and collection into the AQL query

Concatenating them into the query text let a caller-supplied prefix or
start name close the string literal and append arbitrary AQL, which runs
with the filer's ArangoDB credentials against any collection.

* arangodb: bind the folder path and collection into the recursive delete query

A trailing-slash S3 key reaches DeleteFolderChildren through the
directory-marker cleanup, so quotes in the path could turn the filter
into a match-everything REMOVE over the whole bucket collection.

* arangodb: match the real directory prefix in the recursive delete

The prefix was built by re-joining the path segments with commas, so it
never matched a stored directory and the subtree sweep did nothing.
2026-08-17 15:15:26 -07:00
Chris Lu 6e477ae00f filer: keep the internal .system folder out of the per-bucket store path (#10248)
* filer: keep .system internal folder in the default SQL table

Bucket-table SQL stores read the first path segment under /buckets as a
bucket name. The ListBuckets owner index lives at /buckets/.system/..., so
every write there hit isValidBucket(".system") == false and returned
"invalid bucket name .system", flooding the filer log on the postgres/mysql
backends. Route dot-prefixed internal folders to the default table by their
full path, like any other non-bucket entry.

* filer: keep .system internal folder in the default leveldb3 DB

Same guard as the SQL stores: leveldb3 would otherwise open a separate DB
for the .system owner-index folder instead of keeping it in the default DB.

* filer: keep .system internal folder under the default ydb prefix

Skips a DescribeTable round trip per operation on the .system owner-index
path, which never resolves to a real bucket table.

* filer: keep .system internal folder in the default arangodb collection

Avoids creating a stray collection for the .system owner-index folder.
2026-07-06 11:25:50 -07:00
promalertandChris Lu 9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
b669607fcd Add error list each entry func (#7485)
* added error return in type ListEachEntryFunc

* return error if errClose

* fix fmt.Errorf

* fix return errClose

* use %w fmt.Errorf

* added entry in messege error

* add callbackErr in ListDirectoryEntries

* fix error

* add log

* clear err when the scanner stops on io.EOF, so returning err doesn’t surface EOF as a failure.

* more info in error

* add ctx to logs, error handling

* fix return eachEntryFunc

* fix

* fix log

* fix return

* fix foundationdb test s

* fix eachEntryFunc

* fix return resEachEntryFuncErr

* Update weed/filer/filer.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/filer/elastic/v7/elastic_store.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/filer/hbase/hbase_store.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/filer/foundationdb/foundationdb_store.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update weed/filer/ydb/ydb_store.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* fix

* add scanErr

---------

Co-authored-by: Roman Tamarov <r.tamarov@kryptonite.ru>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-11-25 19:35:19 -08:00
Chris Lu 69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
Aleksey Kosovandakosov 4511c2cc1f Changes logging function (#6919)
* updated logging methods for stores

* updated logging methods for stores

* updated logging methods for filer

* updated logging methods for uploader and http_util

* updated logging methods for weed server

---------

Co-authored-by: akosov <a.kosov@kryptonite.ru>
2025-06-24 08:44:06 -07:00
chrislu 70a4c98b00 refactor filer_pb.Entry and filer.Entry to use GetChunks()
for later locking on reading chunks
2022-11-15 06:33:36 -08:00
gfxanda 48db56ddad arangodb s3 bucket name compatibility (#3588)
* Update arangodb_store.go

* update readme, properly escape queries, add name patching

* use underscore

* use underscore

* better comment

* fix readme

Co-authored-by: a <a@a.a>
2022-09-09 09:43:42 -07:00
gfx bf844d8e46 Fix crash in arangodb filer when attempting to access a deleted bucket (#3587)
* Update helpers.go

* Update arangodb_store_bucket.go
2022-09-04 20:32:10 -07:00
gfx 3621987727 Correctly pass arangodb driver Transaction ID into the context (#3586)
Update arangodb_store.go
2022-09-04 19:29:52 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev 21033ff4c3 refactor use const CountEntryChunksForGzip 2022-05-01 22:28:55 +05:00
chrislu cb1e7aa27d fix build 2022-04-20 16:21:56 -07:00
a 7e92517571 change user and pass to username and password 2022-04-01 14:09:25 -05:00
elee beb406bbbb fix ls, onBucketDelete still not triggering 2022-03-18 22:21:57 -05:00
elee 411c0df3fe switch to multi collection, change readme 2022-03-18 21:51:16 -05:00
elee 1cea6c73d3 update readme 2022-03-18 00:34:19 -05:00
elee 25be96832a update readme with index info 2022-03-18 00:33:08 -05:00
elee 06f23aa675 put in delete folder children query 2022-03-18 00:29:48 -05:00
elee 0701feeb17 added more to readme 2022-03-18 00:26:11 -05:00
elee 171c27ddf1 zz 2022-03-17 22:35:30 -05:00
elee 2f0cdcdceb update readme 2022-03-17 21:26:16 -05:00
elee 423ce57cde prefix search, bucket implemented 2022-03-17 21:12:25 -05:00
elee bf745bdccb revise comment 2022-03-17 14:09:34 -05:00
elee a0df993cef cleanup, add a few comments 2022-03-17 14:08:36 -05:00
elee 94883e2fad ok then 2022-03-17 05:23:12 -05:00
elee f3ab6769e9 make start from indexing work 2022-03-17 05:09:13 -05:00
elee 31571dd96e fix file listing 2022-03-17 05:07:40 -05:00
elee 921535001a arangodb adapter 2022-03-17 04:49:26 -05:00