Files
Warp/server/internal/storage/storage.go
T
tajniak81andClaude Opus 5.5 92836f8dcd
CI / Test (ubuntu-latest) (push) Canceled after 0s
CI / Test (windows-latest) (push) Canceled after 0s
CI / Storage tests on PostgreSQL (push) Canceled after 0s
CI / Complement (federation off) (push) Canceled after 0s
CI / golangci-lint (push) Canceled after 0s
CI / Smoke test the binary (push) Canceled after 0s
Phase 6: media
Implements PLAN.md Phase 6: the media repository. Clients upload files,
reserve an mxc:// URI and upload to it later, download and ask for
thumbnails, all through the authenticated media endpoints of Matrix 1.11;
other servers download this server's media through the federation media
endpoints, and media uploaded elsewhere is fetched over federation on
first use and kept.

internal/mediaapi is the component. Content goes into a BlobStore named
by its SHA-256, so the same file uploaded twice is stored once and the
hash doubles as the ETag; the metadata that maps an mxc:// URI onto a
blob is in media_repository and media_thumbnails. The local-filesystem
store writes to a temporary file and renames it into place once hashed,
so nothing ever refers to half a file. A storage driver can bring its own
store through storage.BlobProvider, which is where PocketBase's
filesystem abstraction (local or S3) plugs in with Phase 10.

Reserved uploads belong to the user who made them, take one upload,
lapse after a day, and are capped per user; a download of one waits for
its content, woken in memory the moment it arrives, for as long as
timeout_ms allows.

Thumbnails come from a fixed set of configured sizes, so a thousand
requested sizes cannot mean a thousand resizes. They are made on first
request and kept, never enlarge, honour a JPEG's EXIF orientation, and
point at the original when it already fits. The pixel count is checked
from the header before anything is decoded, at most four resizes run at
once, and concurrent requests for the same thumbnail share one.

Remote media is fetched through the signed client: the multipart/mixed
answer of Matrix 1.11, with a fallback to the legacy download endpoint
for servers that answer M_UNRECOGNIZED, and Location redirects followed
by a dialer that refuses loopback, private and link-local addresses.
Concurrent requests share one fetch, detached from the request that
started it. With federation off nothing is fetched, and what was cached
before is still served.

Downloads carry a sandboxing CSP, nosniff, inline Content-Disposition
only for the types the spec lists as safe, and private immutable caching.
The unauthenticated legacy download endpoints are off by default behind
media.legacy_unauthenticated_downloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:26:49 +02:00

146 lines
4.4 KiB
Go

// Package storage defines the interfaces every Warp storage driver
// implements, plus the registry that turns a config into a Database.
//
// Drivers live in sub-packages (sqlite, postgres, pocketbase) and register
// themselves from an init function, the database/sql way. Nothing in this
// package imports a driver, so components can depend on the interfaces
// without pulling in every database engine.
package storage
import (
"context"
"errors"
"fmt"
"log/slog"
"maps"
"slices"
"sync"
"time"
)
// Driver names. These are the values accepted by database.driver in the
// config file.
const (
DriverSQLite = "sqlite"
DriverPostgres = "postgres"
DriverPocketBase = "pocketbase"
)
// ErrNotImplemented is returned by a driver that is compiled in but not
// finished — currently PocketBase.
var ErrNotImplemented = errors.New("not implemented by this storage driver")
// ErrNoSuchSetting is returned by SettingsStore.Get for an unknown key.
var ErrNoSuchSetting = errors.New("no such setting")
// Config is the subset of the file configuration a driver needs.
type Config struct {
Driver string
ConnectionString string
MaxOpenConns int
MaxIdleConns int
ConnMaxLifetime time.Duration
}
// Database is the handle every component gets. Component-specific stores
// (userapi, roomserver, syncapi...) hang off it as they are implemented.
type Database interface {
// Driver reports which driver this is, for logs and admin output.
Driver() string
// Ping checks the connection is alive; used by /healthz.
Ping(ctx context.Context) error
// Migrate applies pending migrations.
Migrate(ctx context.Context) error
// PendingMigrations lists migrations that Migrate would apply.
PendingMigrations(ctx context.Context) ([]string, error)
// Settings is the server_settings key/value store backing the runtime
// settings service (federation and registration switches).
Settings() SettingsStore
// Users is the userapi store: accounts, devices, tokens, account data.
Users() UserStore
// Rooms is the roomserver store: rooms, events, room state, aliases.
Rooms() RoomStore
// Sync is the syncapi store: receipts, to-device messages, presence and
// uploaded filters.
Sync() SyncStore
// Federation is the federation store: other servers' keys, the
// outbound queue and its backoff, and the inbound transaction log.
Federation() FederationStore
// Media is the mediaapi store: the metadata of uploaded and cached
// remote media, and of their thumbnails.
Media() MediaStore
// Close releases the connection pool.
Close() error
}
// SettingsStore persists the handful of settings that are mutable at
// runtime through the admin API.
type SettingsStore interface {
// Get returns the raw value for key. The boolean reports presence.
Get(ctx context.Context, key string) (string, bool, error)
// GetAll returns every setting.
GetAll(ctx context.Context) (map[string]string, error)
// Set writes key, overwriting any previous value.
Set(ctx context.Context, key, value string) error
// Delete removes key, reverting it to its configured default. Deleting
// a key that is not there is not an error.
Delete(ctx context.Context, key string) error
}
// OpenFunc opens a database for a driver.
type OpenFunc func(ctx context.Context, cfg Config, log *slog.Logger) (Database, error)
var (
registryMu sync.RWMutex
registry = map[string]OpenFunc{}
)
// Register makes a driver available to Open. It panics on a duplicate
// registration, which can only be a programming error.
func Register(name string, open OpenFunc) {
registryMu.Lock()
defer registryMu.Unlock()
if _, dup := registry[name]; dup {
panic("storage: driver " + name + " registered twice")
}
registry[name] = open
}
// Drivers lists the registered driver names, sorted.
func Drivers() []string {
registryMu.RLock()
defer registryMu.RUnlock()
return slices.Sorted(maps.Keys(registry))
}
// Open connects using the driver named in cfg.
func Open(ctx context.Context, cfg Config, log *slog.Logger) (Database, error) {
registryMu.RLock()
open, ok := registry[cfg.Driver]
registryMu.RUnlock()
if !ok {
return nil, fmt.Errorf("storage: unknown driver %q (compiled in: %v)", cfg.Driver, Drivers())
}
if log == nil {
log = slog.Default()
}
db, err := open(ctx, cfg, log.With("driver", cfg.Driver))
if err != nil {
return nil, fmt.Errorf("storage: open %s: %w", cfg.Driver, err)
}
return db, nil
}