This addresses issue #7504 where a single weed mount FUSE instance does
not fully utilize node network bandwidth when reading large files.
The root cause is that HTTP GET streaming reads were getting serialized
even with multiple concurrent goroutines, due to:
1. HTTP/2 multiplexing all requests over a single TCP connection
2. The streaming loop using a 64KB intermediate buffer with extra copies
Changes:
http/client: Enable truly parallel connections
- Set ForceAttemptHTTP2: false to disable HTTP/2 multiplexing
- Set MaxConnsPerHost: 0 (unlimited) to allow parallel connections per host
- This lets multiple goroutines use separate TCP connections to the same
volume server
filer/stream: Add direct-to-buffer fast path
- For full, uncompressed, unencrypted chunks, use RetriedFetchChunkData
to read directly into the destination buffer
- This skips the 64KB staged streaming loop and avoids an extra memory copy
- Falls back to streaming path on partial fill or error
These changes should improve bandwidth utilization when reading large files
through a single mount instance.
Ref: #7504
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
chrislu