filer.sync: sign proxied chunk I/O from the per-side security file (#11645)

* filer.sync: sign proxied chunk I/O from the per-side security file

The -a.security / -b.security files were used for gRPC TLS and the
HTTPS client but not for jwt.filer_signing, so filer-proxied chunk
reads and writes carried a token signed with the process-wide key and
failed authorization whenever the two clusters' keys differ.

LoadFilerJwtFromFile returns a FilerJwtProvider for each side's file,
which FilerSource and FilerSink now accept for proxied chunk reads and
writes. With no keys in the file or no flag, both fall back to the
process-wide jwt.filer_signing configuration as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* replication: use the side filer read key for manifest downloads and fall back per access level

Manifest chunk resolution still signed proxied downloads with the
process-wide read key, so a source filer requiring its own key 401'd on
manifest-bearing files. A side security file that set only one access
level also produced empty tokens for the other instead of inheriting the
process-wide key, and the side file loader ignored the WEED_ environment
overrides the filer itself honors.

ResolveChunkManifest/ResolveOneChunkManifest keep their signatures;
FilerJwt-aware variants thread the provider down to fetchWholeChunk,
which prefers it on proxy URLs. The side loader now applies the same
environment precedence and falls back to the process-wide signer per
missing access level.

* security: verify the configured filer token lifetimes

* security: reject negative filer token lifetimes

A negative expires_after_seconds reached GenJwtForFilerServer and produced
a token with no expiration claim. Also synchronize the Authorization-header
capture in the proxy test and restore the prior viper key on cleanup.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
Chris LuandDevin authored and GitHub committed 2026-10-08 22:04:06 +08:00
1 parent 14fdd61aea
commit 4d1f49c638
13 files changed
+348 -31

No files matched your search

+16 -1
View File
@@ -34,6 +34,7 @@ type FilerSource struct {
dataCenter string
signature int32
httpClient *util_http_client.HTTPClient
jwtForFiler security.FilerJwtProvider
}
func (fs *FilerSource) Initialize(configuration util.Configuration, prefix string) error {
@@ -67,6 +68,16 @@ func (fs *FilerSource) SetHttpClient(client *util_http_client.HTTPClient) {
fs.httpClient = client
}
func (fs *FilerSource) SetFilerJwtProvider(provider security.FilerJwtProvider) {
fs.jwtForFiler = provider
}
// FilerJwt returns the side-specific filer API credential, or nil when the
// process-wide jwt.filer_signing configuration applies.
func (fs *FilerSource) FilerJwt() security.FilerJwtProvider {
return fs.jwtForFiler
}
func (fs *FilerSource) LookupFileId(ctx context.Context, part string) (fileUrls []string, err error) {
vid2Locations := make(map[string]*filer_pb.Locations)
@@ -126,7 +137,11 @@ func (fs *FilerSource) ReadPart(fileId string, offset int64) (filename string, h
if fs.proxyByFiler {
fileUrl := util_http.ProxyChunkUrl(fs.address, fileId)
filename, header, resp, err = downloadFn(fileUrl, util_http.JwtForFilerServer(false), offset)
jwt := util_http.JwtForFilerServer(false)
if fs.jwtForFiler != nil {
jwt = string(fs.jwtForFiler(false))
}
filename, header, resp, err = downloadFn(fileUrl, jwt, offset)
if err == nil {
err = readPartStatusError(fileUrl, resp)
}