Files
seaweedfs/weed/security/metadata_subscribe.go
T
c1c95e127f filer: scoped read-only tokens for metadata subscriptions (#12767)
* security: scoped read-only token for filer metadata subscriptions

A token signed with jwt.filer_signing.subscribe.key names the subscribe
methods and the path prefixes its holder may see. mTLS peers and admin
tokens stay unrestricted. Internal subscribers attach an admin token
when both keys are configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: enforce subscription token scope on metadata streams

With jwt.filer_signing.subscribe.key set, SubscribeMetadata and
SubscribeLocalMetadata authorize the caller first. A scoped caller gets
a stream that sends only events whose paths are all under its prefixes,
filters batches, and never carries log file refs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: send an admin token on internal metadata subscriptions

Peer filers, metadata followers and the lifecycle reader attach an
admin token when the subscribe key is configured, so they keep the full
stream without mTLS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: add fs.meta.token to mint metadata subscription tokens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: end scoped metadata subscriptions at token expiry

The scope now carries the token's exp. The scoped stream's context ends
at that time and the call returns Unauthenticated, so an open stream no
longer outlives its token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.sync: sign metadata subscriptions with the source cluster's key

MetadataFollowOption takes a FilerJwtProvider; filer.sync passes the
one loaded from -a.security or -b.security, so clusters with different
admin keys both accept the subscription.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: report fs.meta.token flag errors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: give scoped metadata subscribers their own client id

A scoped caller could reuse another subscriber's ClientId with a higher
epoch and replace that registration, disconnecting it. Scoped streams now
register under a server-chosen id that no other subscriber holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* security: accept only HS256 metadata subscription tokens

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* security: test HS384 metadata subscription token rejection

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: never let a scoped subscriber replace another registration

The server-chosen id can be taken by another subscriber between choosing
and registering it. Registering scoped streams with the lowest epoch makes
addClient refuse them rather than replace the other registration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: send position markers for events a scoped token withholds

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test a scoped stream withholding a whole event batch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: filter SendMsg on scoped metadata streams

The scoped wrappers only overrode Send, so a SendMsg on the embedded
stream would reach the client unfiltered. Route it through the same
filter and refuse any other message type. Ported from #12790.

Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test a scoped subscription through the real subscribe loop

Run SubscribeLocalMetadata with a scoped token over a flushed backlog,
the in-memory tail and live events: only in-scope events arrive, sibling,
"..", encoded and cross-boundary rename paths are dropped, and no log
file references are sent although the client asks for them. Without the
subscribe key every event still arrives. Ported from #12790.

Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test that the HTTP API refuses metadata subscription tokens

GET, HEAD, PUT, POST and DELETE all refuse a subscription token, also
when the subscribe key equals the read or write key. Ported from #12790.

Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mindaeon <30555587+mindaeon@users.noreply.github.com>
2026-11-19 12:41:16 -08:00

149 lines
5.3 KiB
Go

package security
import (
"context"
"path"
"strings"
"time"
jwt "github.com/golang-jwt/jwt/v5"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/util"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
// The filer gRPC methods a scoped subscription token may name in allowed_methods.
const (
MethodSubscribeMetadata = "SubscribeMetadata"
MethodSubscribeLocalMetadata = "SubscribeLocalMetadata"
)
// GenJwtForMetadataSubscription mints a read-only token for the filer's
// metadata subscription calls, limited to events under prefixes. It carries
// allowed_prefixes and allowed_methods, so it is never an admin token.
// Returns an empty string without a key, an expiry, or an absolute prefix.
func GenJwtForMetadataSubscription(signingKey SigningKey, expiresAfterSec int, prefixes []string) EncodedJwt {
prefixes, ok := cleanMetadataPrefixes(prefixes)
if len(signingKey) == 0 || expiresAfterSec <= 0 || !ok {
return ""
}
claims := SeaweedFilerClaims{
AllowedPrefixes: prefixes,
AllowedMethods: []string{MethodSubscribeMetadata, MethodSubscribeLocalMetadata},
RegisteredClaims: jwt.RegisteredClaims{
ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Second * time.Duration(expiresAfterSec))),
},
}
encoded, err := jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString([]byte(signingKey))
if err != nil {
glog.V(0).Infof("Failed to sign metadata subscription claims: %v", err)
return ""
}
return EncodedJwt(encoded)
}
// cleanMetadataPrefixes cleans prefixes; ok is false when the list is empty
// or any prefix is not absolute.
func cleanMetadataPrefixes(prefixes []string) (cleaned []string, ok bool) {
for _, p := range prefixes {
if !strings.HasPrefix(p, "/") {
return nil, false
}
cleaned = append(cleaned, path.Clean(p))
}
return cleaned, len(cleaned) > 0
}
// MetadataPathInScope reports whether fullPath is one of the prefixes or below
// one, by whole path segments: "/buckets/a" holds "/buckets/a/x", not "/buckets/ab".
func MetadataPathInScope(fullPath string, prefixes []string) bool {
fullPath = path.Clean("/" + fullPath)
for _, p := range prefixes {
if p == "/" || fullPath == p || strings.HasPrefix(fullPath, p+"/") {
return true
}
}
return false
}
// MetadataSubscriptionScope limits a subscription to events under Prefixes
// until ExpiresAt.
type MetadataSubscriptionScope struct {
Prefixes []string
ExpiresAt time.Time
}
func metadataSubscriptionScope(signingKey SigningKey, token EncodedJwt, method string) (*MetadataSubscriptionScope, bool) {
parsed, err := decodeJwt(signingKey, token, &SeaweedFilerClaims{}, jwt.WithValidMethods([]string{jwt.SigningMethodHS256.Alg()}))
if err != nil || parsed == nil || !parsed.Valid {
return nil, false
}
claims, ok := parsed.Claims.(*SeaweedFilerClaims)
if !ok || claims.SessionId != "" || claims.ExpiresAt == nil {
return nil, false
}
allowsMethod := false
for _, m := range claims.AllowedMethods {
if m != MethodSubscribeMetadata && m != MethodSubscribeLocalMetadata {
return nil, false
}
allowsMethod = allowsMethod || m == method
}
if !allowsMethod {
return nil, false
}
prefixes, ok := cleanMetadataPrefixes(claims.AllowedPrefixes)
if !ok {
return nil, false
}
return &MetadataSubscriptionScope{Prefixes: prefixes, ExpiresAt: claims.ExpiresAt.Time}, true
}
// CheckGrpcMetadataSubscription authorizes a metadata subscription call.
// Without subscribeKey every caller is unrestricted (nil scope). With it, an
// mTLS-verified peer or an admin token signed by adminKey is unrestricted,
// and a token signed by subscribeKey is limited to the returned scope.
func CheckGrpcMetadataSubscription(ctx context.Context, subscribeKey, adminKey SigningKey, method string) (scope *MetadataSubscriptionScope, err error) {
if len(subscribeKey) == 0 || isMtlsVerifiedPeer(ctx) {
return nil, nil
}
md, _ := metadata.FromIncomingContext(ctx)
authHeaders := md.Get("authorization")
if len(authHeaders) > maxGrpcAdminAuthValues {
authHeaders = authHeaders[:maxGrpcAdminAuthValues]
}
for _, raw := range authHeaders {
parts := strings.Fields(raw)
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") || parts[1] == "" {
continue
}
token := EncodedJwt(parts[1])
if IsFilerAdminToken(adminKey, token) {
return nil, nil
}
if scope, ok := metadataSubscriptionScope(subscribeKey, token, method); ok {
return scope, nil
}
}
return nil, status.Error(codes.Unauthenticated, "metadata subscription requires mTLS, an admin token, or a subscription token for this method")
}
// WithMetadataSubscriptionCredentials adds an admin token to a subscription
// call: from jwtProvider when the caller has per-cluster keys, else when this
// process has both jwt.filer_signing.key and jwt.filer_signing.subscribe.key,
// so internal subscribers keep full access.
func WithMetadataSubscriptionCredentials(ctx context.Context, jwtProvider FilerJwtProvider) context.Context {
var token EncodedJwt
if jwtProvider != nil {
token = jwtProvider(true)
} else if v := util.GetViper(); v.GetString("jwt.filer_signing.subscribe.key") != "" {
token = GenJwtForFilerAdmin(SigningKey(v.GetString("jwt.filer_signing.key")), v.GetInt("jwt.filer_signing.expires_after_seconds"))
}
if token == "" {
return ctx
}
return metadata.AppendToOutgoingContext(ctx, "authorization", BearerPrefix+string(token))
}