Commit Graph
13 Commits
Author SHA1 Message Date
Chris Lu 5b519489c1 remote_storage: build all S3-compatible clients through one constructor (#10720)
* remote_storage: build S3-compatible clients through one constructor

The eight non-s3 S3-SDK providers each duplicated the AWS session setup
and only the s3 maker could take a custom *http.Client. Route every
S3-compatible type (s3, wasabi, b2, aliyun, tencent, baidu, filebase,
storj, contabo) through MakeWithHTTPClient with a single options table,
and add S3CompatibleEndpoint so callers can resolve the endpoint a given
type dials. No behavior change.

* volume: apply the remote-endpoint check to all S3-compatible providers

FetchAndWriteNeedle validated the endpoint and used the pinned dialer only
for type "s3". Every S3-SDK backend (wasabi, b2, aliyun, tencent, baidu,
filebase, storj, contabo) dials a caller-supplied endpoint through the same
client, so gate on S3CompatibleEndpoint to apply the same check uniformly.
-volume.allowUntrustedRemoteEndpoints still opts out.

* volume: don't route the guarded remote-endpoint client through a proxy

The guarded client exists to dial the validated endpoint directly and
re-check the resolved IP at connect time. With http.ProxyFromEnvironment
set, the dialer only validates the proxy's address while the proxy
re-resolves the endpoint host, which reopens the rebinding window. Drop
the proxy on this path; operators that need one can opt out with
-volume.allowUntrustedRemoteEndpoints.
2026-08-11 19:06:12 -07:00
Chris Lu e696c2585e s3 remote: honor s3.support_tagging in UpdateFileMetadata (#10532)
* s3 remote: honor s3.support_tagging in UpdateFileMetadata

The write path already skips tagging when the remote is configured
without tagging support, but the metadata-update path sent
PutObjectTagging or DeleteObjectTagging unconditionally. On remotes
that reject tagging requests, any metadata update failed -- including
updates with no tags at all, which land on the DeleteObjectTagging
branch.

* s3 remote: drop the never-read supportTagging field

Every maker set it, nothing read it: the tagging decision is made from
conf.S3SupportTagging. Keeping a field that looks like the switch but
is not invites exactly the inconsistency the previous commit fixed.
2026-08-01 20:18:59 -07:00
promalertandChris Lu 9012069bd7 chore: execute goimports to format the code (#7983)
* chore: execute goimports to format the code

Signed-off-by: promalert <promalert@outlook.com>

* goimports -w .

---------

Signed-off-by: promalert <promalert@outlook.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-01-07 13:06:08 -08:00
Chris Lu 69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
Muhammad Hallaj bin SuberyandMuhammad Hallaj bin Subery 9bd422d2c9 adding support for B2 region (#4177)
Co-authored-by: Muhammad Hallaj bin Subery <hallaj@tuta.io>
2023-02-05 21:24:21 -08:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
chrislu 61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
Chris Lu 7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu 001a472057 cloud mount: remote storage support hdfs 2021-08-29 18:41:29 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu a7a914f120 cloud drive: add support for Baidu BOS 2021-08-24 23:46:33 -07:00
Chris Lu 47d775cf68 cloud drive: add support for BackBlaze 2021-08-24 22:30:06 -07:00