Files
seaweedfs/weed/s3api/iceberg/handlers_oauth.go
T
ad89f003a5 s3: end a catalog-vended session with the access key it was vended for (#11945)
* s3: end a catalog-vended session with the access key it was vended for

A table credential the Iceberg catalog vends is a self-contained STS session:
it stays valid until it expires (an hour by default) even after the access key
that loaded the table is deleted or deactivated, or its identity is removed
or disabled. An operator revoking a leaked key cannot cut off the table
credentials that key already obtained.

Sessions are stateless JWTs, so there is no revocation list to consult. This
change follows the same pattern as role sessions, which are already refused
once their role's ID no longer matches (checkSessionRoleBinding): a vended
session carries the access key it was vended for (src_ak, recorded by the
previous change), and validateSTSSessionToken checks it against the identity
table held in memory. A session of provider iceberg-catalog that carries a
source key is refused (InvalidAccessKeyId, as for an unknown key) once its
identity is gone or disabled, or the key is no longer one of that identity's
active, unexpired credentials.

Scope:
- Only catalog-vended sessions are checked; web-identity, AssumeRole and
  every other STS session are validated as before.
- The vendor binds a session to the caller's key only when the caller is a
  stored identity holding that key. A caller that signed with another STS
  session's key, an OIDC user, or a request that recorded no key gets an
  unbound session, as today; binding it would refuse every request made with
  it. A session minted before this change carries no key and is unbound.
- The check is one map lookup under the read lock and a scan of one
  identity's keys (about 11 ns, 0 allocations, with 1,000 identities).

Tests (s3api_iceberg_vended_session_test.go) mint real sessions through the
catalog vendor and validate them the way a data-path request is validated:
deactivating, expiring or deleting key A ends A's session while the session
vended through key B on the same identity keeps working; disabling or
deleting the identity ends it; callers that are not a stored identity holding
the key get unbound sessions that keep working; other STS sessions are not
checked. The first two tests and the third fail without this change.

* s3: keep the vended-session key binding when the key dies before vend

vendedSessionKeyLive gated the binding, so a key revoked between the
request's authentication and the vend skipped it and minted an unbound
session that stayed usable until it expired. Every catalog auth path
pairs the identity name in context with the key that authenticated, so
a stored-identity principal always means the request signed with that
identity's own key. Bind whenever the principal is a stored identity: a
session bound to a now-dead key is refused on its first request, which
is what the binding is for.

* s3: gate the vended-session key binding on the authenticated caller type

lookupByIdentityName(principal) named only that some identity exists, not
that the request signed with a stored key: an STS session or OIDC principal
that collides with an identity name got a session bound to its temporary
key, which vendedSessionKeyLive then refused on every request. Verifiers
now record the verified key as a stored credential, and the vendor binds
only when that marker is present — keeping the binding even when the key
was revoked between authentication and the vend.

* s3: enforce the vended key binding under a dedicated claim

Sessions minted before the binding existed carry the caller's key in the
audit-only SourceAccessKeyId — including temporary STS keys that can never
resolve in the identity table. Enforcing on that field would end live table
credentials at upgrade; the binding now travels under BoundAccessKeyId,
leaving legacy sessions unbound as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: bind vended sessions to a credential fingerprint

The key id alone cannot identify a credential: an operator can delete a
key and re-create it under the same id with a different secret, reviving
sessions minted for the original. Verifiers now record a fingerprint of
the secret at authentication; the session dies unless it still matches.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: keep recording the caller's key on every catalog session

SourceAccessKeyId was unconditional audit metadata before the binding
existed; gating it on the stored-credential marker dropped the record
for callers that authenticated another way. With the binding under its
own claim there is no ambiguity left, so the field records the caller's
key again regardless of how it authenticated.

* s3: key the vended-session credential fingerprint with the signing key

The bound-credential fingerprint travels inside the vended session
token. As a plain SHA-256 of the secret it gave any token holder an
offline guessing oracle for weak secrets, and CodeQL flags it twice.

Store the verified secret in the request marker instead and HMAC it
with the STS signing key at vend and at validation. The claim can no
longer be mined, and the per-request auth path stops hashing secrets.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-23 03:58:06 -07:00

408 lines
15 KiB
Go

package iceberg
import (
"crypto/hmac"
"crypto/sha256"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"time"
jwt "github.com/golang-jwt/jwt/v5"
"github.com/seaweedfs/seaweedfs/weed/glog"
)
// OAuthTokenResponse is the response for POST /v1/oauth/tokens.
type OAuthTokenResponse struct {
AccessToken string `json:"access_token"`
TokenType string `json:"token_type"`
ExpiresIn int `json:"expires_in"`
Scope string `json:"scope,omitempty"`
// IssuedTokenType is set to the access-token URN on token-exchange
// responses, where RFC 8693 requires it.
IssuedTokenType string `json:"issued_token_type,omitempty"`
}
// OAuthErrorResponse is the error response for the OAuth endpoint.
type OAuthErrorResponse struct {
Error string `json:"error"`
Description string `json:"error_description,omitempty"`
}
// IcebergClaims are JWT claims for Iceberg catalog OAuth tokens.
type IcebergClaims struct {
IdentityName string `json:"identity_name"`
AccessKey string `json:"access_key"`
jwt.RegisteredClaims
}
const defaultOauthTokenExpiry = 3600
// maxOauthTokenExpiry bounds the configured TTL so seconds-to-Duration
// conversions cannot overflow into already-expired tokens.
const maxOauthTokenExpiry = 365 * 24 * 3600
// accessTokenTokenType is the RFC 8693 token-type URN for access tokens,
// required in issued_token_type on token-exchange responses.
const accessTokenTokenType = "urn:ietf:params:oauth:token-type:access_token"
// grant types accepted by POST /v1/oauth/tokens. Iceberg Java 1.10.x
// refreshes tokens via token-exchange (exchangeEnabled defaults true), so a
// server that only accepts client_credentials leaves those clients unable
// to refresh.
const (
grantTypeClientCredentials = "client_credentials"
grantTypeTokenExchange = "urn:ietf:params:oauth:grant-type:token-exchange"
)
// oauthExpirySeconds returns the OAuth token TTL. Deployments whose
// clients cannot refresh tokens on 401 can raise this to survive client
// restart cycles instead of dying every hour.
func oauthExpirySeconds() int {
if v := os.Getenv("ICEBERG_OAUTH_TOKEN_EXPIRY"); v != "" {
// Parse in 64-bit space: on 32-bit platforms Atoi overflows and
// errors on oversized values, which would silently fall back to
// the default instead of clamping.
if n, err := strconv.ParseInt(v, 10, 64); err == nil && n > 0 {
if n > maxOauthTokenExpiry {
return maxOauthTokenExpiry
}
return int(n)
}
}
return defaultOauthTokenExpiry
}
// tokenExchangeGrace is how long an expired subject token may still be
// exchanged for a fresh access token. Signature verification is the real
// gate; the grace exists so a client holding a token that expired while the
// exchange grant was unsupported (or during a server outage) recovers
// without a process restart.
func tokenExchangeGrace() time.Duration {
grace := 2 * oauthExpirySeconds()
if grace < 3600 {
grace = 3600
}
if grace > 86400 {
grace = 86400
}
return time.Duration(grace) * time.Second
}
// handleOAuthTokens implements the OAuth2 client_credentials flow.
// POST /v1/oauth/tokens
func (s *Server) handleOAuthTokens(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
writeOAuthError(w, http.StatusBadRequest, "invalid_request", "Could not parse form body")
return
}
// Reject credentials in query string to prevent leaking secrets into logs and caches.
if r.URL.Query().Get("client_secret") != "" {
writeOAuthError(w, http.StatusBadRequest, "invalid_request", "client_secret must not be sent in the URL")
return
}
grantType := r.PostFormValue("grant_type")
switch grantType {
case grantTypeClientCredentials:
// handled below
case grantTypeTokenExchange, "token_exchange":
s.handleTokenExchange(w, r)
return
default:
writeOAuthError(w, http.StatusBadRequest, "unsupported_grant_type",
fmt.Sprintf("Unsupported grant_type: %s", grantType))
return
}
clientID := r.PostFormValue("client_id")
clientSecret := r.PostFormValue("client_secret")
// Also support HTTP Basic auth per OAuth2 spec
if clientID == "" && clientSecret == "" {
var ok bool
clientID, clientSecret, ok = r.BasicAuth()
if !ok {
writeOAuthError(w, http.StatusUnauthorized, "invalid_client", "Missing client credentials")
return
}
}
if clientID == "" || clientSecret == "" {
writeOAuthError(w, http.StatusUnauthorized, "invalid_client", "Missing client_id or client_secret")
return
}
if s.credentialValidator == nil {
writeOAuthError(w, http.StatusInternalServerError, "server_error", "Credential validation not configured")
return
}
identityName, _, err := s.credentialValidator.ValidateS3Credential(clientID, clientSecret)
if err != nil {
glog.V(2).Infof("Iceberg OAuth: credential validation failed for client_id=%s: %v", clientID, err)
writeOAuthError(w, http.StatusUnauthorized, "invalid_client", "Invalid client credentials")
return
}
// Generate a JWT signed with a key derived from the client secret.
// Include the access key in claims so we can look up the exact credential for verification.
tokenString, err := mintIcebergToken(identityName, clientID, clientSecret, oauthExpirySeconds())
if err != nil {
glog.Errorf("Iceberg OAuth: failed to sign token: %v", err)
writeOAuthError(w, http.StatusInternalServerError, "server_error", "Failed to generate token")
return
}
scope := r.PostFormValue("scope")
resp := OAuthTokenResponse{
AccessToken: tokenString,
TokenType: "bearer",
ExpiresIn: oauthExpirySeconds(),
Scope: scope,
}
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, http.StatusOK, resp)
}
// handleTokenExchange implements RFC 8693 token exchange for Iceberg REST
// clients: a previously issued access token (subject_token) is exchanged for
// a fresh one. Iceberg Java's OAuth2Manager refreshes via this grant
// (exchangeEnabled defaults true), so supporting it lets those clients
// self-heal before their token expires — no client restart needed.
//
// Client authentication (RFC 8693 §2.1) is accepted but not required:
// Iceberg Java's refreshExpiredToken sends Basic credentials with the
// exchange, while its proactive scheduled refresh sends only the Bearer
// session headers. An expired subject_token is only exchangeable with valid
// client credentials, and an unauthenticated exchange never extends the
// token's lifetime beyond the subject_token's own expiry.
func (s *Server) handleTokenExchange(w http.ResponseWriter, r *http.Request) {
if s.credentialValidator == nil {
writeOAuthError(w, http.StatusInternalServerError, "server_error", "Credential validation not configured")
return
}
subjectToken := r.PostFormValue("subject_token")
if subjectToken == "" {
writeOAuthError(w, http.StatusBadRequest, "invalid_request", "Missing subject_token")
return
}
// Verify the subject token by signature (exp checked separately against
// the recovery grace).
unverified := &IcebergClaims{}
parser := jwt.NewParser(jwt.WithoutClaimsValidation())
if _, _, err := parser.ParseUnverified(subjectToken, unverified); err != nil {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "Invalid subject_token")
return
}
if unverified.AccessKey == "" || unverified.Issuer != "seaweedfs-iceberg" {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "Invalid subject_token")
return
}
identityName, _, secretKey, err := s.credentialValidator.GetCredentialByAccessKey(unverified.AccessKey)
if err != nil {
glog.V(2).Infof("Iceberg OAuth: token exchange failed to get credential for access key: %v", err)
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "Invalid subject_token")
return
}
// Optional client authentication. When present, the client must
// authenticate with the same credential that issued the subject_token.
clientID, clientSecret, hasBasic := r.BasicAuth()
if !hasBasic {
clientID = r.PostFormValue("client_id")
clientSecret = r.PostFormValue("client_secret")
hasBasic = clientID != "" && clientSecret != ""
}
clientAuthenticated := false
if hasBasic {
if _, _, err := s.credentialValidator.ValidateS3Credential(clientID, clientSecret); err != nil {
glog.V(2).Infof("Iceberg OAuth: token exchange client authentication failed for client_id=%s: %v", clientID, err)
writeOAuthError(w, http.StatusUnauthorized, "invalid_client", "Invalid client credentials")
return
}
if clientID != unverified.AccessKey {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "subject_token was issued to a different client")
return
}
clientAuthenticated = true
}
signingKey := deriveSigningKey(unverified.AccessKey, secretKey)
claims := &IcebergClaims{}
parsed, err := jwt.ParseWithClaims(subjectToken, claims, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
return signingKey, nil
}, jwt.WithoutClaimsValidation())
if err != nil || !parsed.Valid {
glog.V(2).Infof("Iceberg OAuth: token exchange signature verification failed: %v", err)
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "Invalid subject_token")
return
}
if claims.Issuer != "seaweedfs-iceberg" {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "Invalid subject_token")
return
}
if claims.ExpiresAt == nil {
// Tokens minted by this server always carry an expiry; a subject
// token without one must not be exchangeable forever.
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "subject_token has no expiry")
return
}
ttlSeconds := oauthExpirySeconds()
if time.Now().After(claims.ExpiresAt.Time) {
// Expired subject tokens are only exchangeable by an authenticated
// client, within the recovery grace.
if !clientAuthenticated {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant",
"subject_token is expired; client authentication is required to exchange it")
return
}
if time.Since(claims.ExpiresAt.Time) > tokenExchangeGrace() {
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "subject_token expired beyond the exchange grace window")
return
}
} else if remaining := int(time.Until(claims.ExpiresAt.Time).Seconds()); !clientAuthenticated && remaining < ttlSeconds {
// Unauthenticated exchange (proactive refresh carrying only the
// Bearer) must not extend the lifetime past the subject's expiry;
// otherwise a leaked token could chain-refresh forever. An
// authenticated client gets a fresh full TTL — it could mint one
// via client_credentials anyway.
ttlSeconds = remaining
}
if ttlSeconds <= 0 {
// A subject token with under a second left would otherwise be
// exchanged for a token that is born expired.
writeOAuthError(w, http.StatusBadRequest, "invalid_grant", "subject_token has no remaining lifetime")
return
}
tokenString, err := mintIcebergToken(identityName, unverified.AccessKey, secretKey, ttlSeconds)
if err != nil {
glog.Errorf("Iceberg OAuth: failed to sign exchanged token: %v", err)
writeOAuthError(w, http.StatusInternalServerError, "server_error", "Failed to generate token")
return
}
resp := OAuthTokenResponse{
AccessToken: tokenString,
TokenType: "bearer",
ExpiresIn: ttlSeconds,
Scope: r.PostFormValue("scope"),
IssuedTokenType: accessTokenTokenType,
}
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, http.StatusOK, resp)
}
// mintIcebergToken issues a signed access token for the given identity and
// credential, valid for ttlSeconds.
func mintIcebergToken(identityName, accessKey, secret string, ttlSeconds int) (string, error) {
signingKey := deriveSigningKey(accessKey, secret)
now := time.Now()
claims := IcebergClaims{
IdentityName: identityName,
AccessKey: accessKey,
RegisteredClaims: jwt.RegisteredClaims{
IssuedAt: jwt.NewNumericDate(now),
ExpiresAt: jwt.NewNumericDate(now.Add(time.Duration(ttlSeconds) * time.Second)),
Issuer: "seaweedfs-iceberg",
},
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
return token.SignedString(signingKey)
}
// authenticateBearer validates a Bearer token from the Authorization header.
// Returns the identity name, identity object, and whether auth succeeded.
func (s *Server) authenticateBearer(r *http.Request) (string, interface{}, bool) {
identityName, identity, _, _, ok := s.authenticateBearerKey(r)
return identityName, identity, ok
}
// authenticateBearerKey is authenticateBearer that also returns the access key
// the token was minted from (its verified access_key claim), so the catalog
// can tell apart the holders of several keys on one identity.
func (s *Server) authenticateBearerKey(r *http.Request) (string, interface{}, string, string, bool) {
auth := r.Header.Get("Authorization")
if auth == "" {
return "", nil, "", "", false
}
if !strings.HasPrefix(strings.ToLower(auth), "bearer ") {
return "", nil, "", "", false
}
tokenString := strings.TrimSpace(auth[7:])
if tokenString == "" {
return "", nil, "", "", false
}
if s.credentialValidator == nil {
return "", nil, "", "", false
}
// Parse the token without verification first to get the access key,
// then look up the exact credential to verify the signature.
parser := jwt.NewParser(jwt.WithoutClaimsValidation())
unverified := &IcebergClaims{}
_, _, err := parser.ParseUnverified(tokenString, unverified)
if err != nil {
glog.V(2).Infof("Iceberg OAuth: failed to parse token: %v", err)
return "", nil, "", "", false
}
if unverified.AccessKey == "" {
return "", nil, "", "", false
}
// Look up the credential by access key to get the signing key for verification
identityName, identity, secretKey, err := s.credentialValidator.GetCredentialByAccessKey(unverified.AccessKey)
if err != nil {
glog.V(2).Infof("Iceberg OAuth: failed to get credential for access key: %v", err)
return "", nil, "", "", false
}
signingKey := deriveSigningKey(unverified.AccessKey, secretKey)
claims := &IcebergClaims{}
verified, err := jwt.ParseWithClaims(tokenString, claims, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
return signingKey, nil
})
if err != nil || !verified.Valid {
glog.V(2).Infof("Iceberg OAuth: token verification failed: %v", err)
return "", nil, "", "", false
}
// The token's signing key is derived from this access key, so a verified
// token was minted from it.
return identityName, identity, unverified.AccessKey, secretKey, true
}
// deriveSigningKey derives a signing key from the access key and secret using HMAC-SHA256.
// Including the access key prevents cross-credential token forgery when two
// credentials happen to share the same secret.
func deriveSigningKey(accessKey, secret string) []byte {
h := hmac.New(sha256.New, []byte("seaweedfs-iceberg-oauth"))
h.Write([]byte(accessKey))
h.Write([]byte{0}) // null separator
h.Write([]byte(secret))
return h.Sum(nil)
}
func writeOAuthError(w http.ResponseWriter, status int, errCode, description string) {
resp := OAuthErrorResponse{
Error: errCode,
Description: description,
}
writeJSON(w, status, resp)
}