refactor: centralize genUploadUrl in UploadOption (#10164)

* refactor: centralize genUploadUrl in UploadOption

Replace inline genFileUrlFn closures with operation.GenUploadUrl field:

- Add GenUploadUrl func(host, fileId) string to UploadOption struct
- Add GenUploadUrlProxy(filerAddress string) utility function
- Remove genFileUrlFn parameter from UploadWithRetry signature
- Update all callers: mount, gateway, mq, filer_copy, filer_sync

This matches the weed mount -filerProxy pattern exactly,
factorizing the URL generation logic across all consumers.

Co-Authored-By: Athena 🏛️ <hermes-agent@local> (custom / Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf)

* docker release: run all platform jobs in one wave, cache rocksdb compile

Drop max-parallel so the 13 per-platform builds run together instead of two
waves of 8 (rocksdb was queuing behind the cap and starting ~8 min late).

Keep cache-to mode=max for rocksdb: its RocksDB static_lib compile is
sha-independent, so it caches across releases and stops being the ~16-min
long-pole that gates the merge fan-in. go-build variants stay mode=min.

Co-Authored-By: Athena 🏛️ <hermes-agent@local> (custom / Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf)

* refactor: centralize genUploadUrl in UploadOption

Replace inline genFileUrlFn closures with operation.GenUploadUrl field:

- Add GenUploadUrl func(host, fileId) string to UploadOption struct
- Add GenUploadUrlProxy(filerAddress string) utility function
- Remove genFileUrlFn parameter from UploadWithRetry signature
- Update all callers: mount, gateway, mq, filer_copy, filer_sync

This matches the weed mount -filerProxy pattern exactly,
factorizing the URL generation logic across all consumers.

Co-Authored-By: Athena 🏛️ <hermes-agent@local> (custom / Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf)

* Remove accidental ROCmFPX submodule reference

* gofmt chunk upload option block

* Preserve broker cipher and re-read proxy filer per upload attempt

Chunk uploads must keep the configured Cipher, and both the mount and broker current filer can change on failover, so build the proxy upload URL inside the closure instead of capturing the address once.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
This commit is contained in:
authored and GitHub committed 2026-06-30 20:45:43 -07:00
1 parent ce3ba31bcd
commit bdcc3154ed
9 files changed
+84 -82

No files matched your search

-9
View File
@@ -371,9 +371,6 @@ func (worker *FileCopyWorker) uploadFileAsOne(task FileCopyTask, f *os.File) err
MimeType: mimeType,
PairMap: nil,
},
func(host, fileId string) string {
return fmt.Sprintf("http://%s/%s", host, fileId)
},
util.NewBytesReader(data),
)
if flushErr != nil {
@@ -456,9 +453,6 @@ func (worker *FileCopyWorker) uploadFileInChunks(task FileCopyTask, f *os.File,
MimeType: "",
PairMap: nil,
},
func(host, fileId string) string {
return fmt.Sprintf("http://%s/%s", host, fileId)
},
io.NewSectionReader(f, i*chunkSize, chunkSize),
)
@@ -576,9 +570,6 @@ func (worker *FileCopyWorker) saveDataAsChunk(reader io.Reader, name string, off
MimeType: "",
PairMap: nil,
},
func(host, fileId string) string {
return fmt.Sprintf("http://%s/%s", host, fileId)
},
reader,
)