Files
DriverVault/API Server/internal/plugins/store.go
T
tajniak81andClaude Opus 5 9bd5c523c4 Plugins: the global layer moves into the database, beside the other two
The integration cascade stored its top layer differently from the two below
it: org (L2) and user (L3) plugin config lived in PocketBase, in a
pluginSettings field, while the global (L1) layer sat in a plugins.json
next to the binary. That split was accretion rather than design - the file
was the whole store in the v1 MVP, and the per-tenant layers were later
built on PocketBase and layered on top of it instead of replacing it.

It also cost something real. plugins.json was a second state store with
different durability from pb_data: its own volume, its own ownership, its
own backup. Losing pb_data is unmissable; losing api_data was silent, which
is how "every plugin comes back disabled after a redeploy" happened.

L1 now lives in the app_settings collection - one record keyed "global",
holding its settings in a pluginSettings field, the same mechanism and the
same field name the layers below use. The documents still differ in shape,
because only L1 carries enable state and the registration of external
plugins, but the storage is no longer a special case.

The Manager grows a Store seam (PocketBase in production, file for the
import, memory for tests) and, more importantly, a loaded gate. Settings in
a database mean the store can be unreachable at boot - a cold stack, or a
service account still to be set from the panel. That must not read as "no
plugins configured", or the first save would write emptiness over real
settings. So until a read succeeds the Manager stays unloaded, every
mutation is refused, /api/admin/plugins* answers 503, and a background
retry backs off to two minutes. The same gate covers a document that will
not parse: it is never replaced by one built from an empty map, which is a
stronger guarantee than the .corrupt backup it replaces.

Writing to a store also revealed a hole in the previous fix. Classifying a
save failure as errPersist was left to each Store, and a store that
returned a plain error would fall through to the "saved, but the plugin
failed to start" branch and be reported as a 200 - the same silent-success
bug through a different door. The Manager now classifies, whatever the
Store returns; a test pins it.

Upgrades are automatic: on the first boot that finds no settings in the
database, an existing plugins.json is imported and renamed to
plugins.json.migrated. The import is refused if the store is merely
unreachable, or if the file does not parse, so a stale or broken file can
never overwrite live settings. /data is still needed - the panel rewrites
.env there when it retargets PocketBase - but plugin settings no longer
depend on it.

21 tests in internal/plugins cover both stores, including the production
path against a fake PocketBase: create-then-update of the singleton,
round-trip across a restart, an outage that leaves settings intact, a
missing collection reading as not-ready rather than empty, and the import
running exactly once. go build, go vet and go test ./... pass. Schema
changes are mirrored into scripts/setup-pocketbase.mjs as that file
requires. Not verified: no Docker CLI here, so no image was built and the
bootstrap of app_settings against a real PocketBase is untested outside the
fake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 16:52:47 +02:00

330 lines
9.7 KiB
Go

package plugins
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"path/filepath"
"sync"
"drivervault/apiserver/internal/pb"
)
// Store is where a Manager keeps plugin enable-state and global (L1) config.
//
// It deals in the raw JSON document rather than in records, so the same Manager
// logic works whichever backing it has: PocketBase in production, a file for the
// one-time import of the legacy plugins.json, memory in tests.
type Store interface {
// Load returns the stored document. found is false when the store holds no
// document at all — a fresh install — which callers must tell apart from a
// document that exists and is empty, because only the former may be seeded
// from a legacy file.
//
// A returned error means the store could not be reached or read. It must
// NOT be taken as "no settings": the Manager stays unloaded and refuses to
// save, so a transient outage cannot overwrite settings it never read.
Load(ctx context.Context) (data []byte, found bool, err error)
// Save replaces the stored document.
Save(ctx context.Context, data []byte) error
// Describe names the store for log and error messages.
Describe() string
}
// PluginSettingsField is the record field the document lives in. The org (L2)
// and user (L3) layers of the cascade keep their plugin config in a field of
// this name too (see internal/api/integrations.go), so the global layer is
// stored the same way they are rather than being a special case.
//
// The documents are not identical in shape: L2/L3 hold per-plugin user config
// only, while L1 additionally carries the enable state and the registration of
// external plugins. Same mechanism and same field, different payload.
const PluginSettingsField = "pluginSettings"
// globalSettingsKey identifies the singleton record holding the global layer.
// It is a fixed constant, never caller input.
const globalSettingsKey = "global"
// --- PocketBase-backed store (production) -----------------------------------
// pbStore keeps the document in a single record of the app_settings collection,
// identified by key="global". This is the production store.
type pbStore struct {
client *pb.Client
collection string
mu sync.Mutex
id string // cached id of the singleton record, resolved on first Load/Save
}
// NewPocketBaseStore returns a Store backed by the singleton settings record in
// the given collection.
func NewPocketBaseStore(client *pb.Client, collection string) Store {
return &pbStore{client: client, collection: collection}
}
func (s *pbStore) Describe() string {
return "PocketBase " + s.collection + " (key=" + globalSettingsKey + ")"
}
// settingsRecord is the slice of the singleton record this store reads.
type settingsRecord struct {
ID string `json:"id"`
PluginSettings json.RawMessage `json:"pluginSettings"`
}
func (s *pbStore) Load(ctx context.Context) ([]byte, bool, error) {
if !s.client.Configured() {
return nil, false, fmt.Errorf("%w: no PocketBase service account configured", errNotReady)
}
rec, found, err := s.find(ctx)
if err != nil {
return nil, false, err
}
if !found {
return nil, false, nil
}
s.mu.Lock()
s.id = rec.ID
s.mu.Unlock()
return rec.PluginSettings, true, nil
}
// find resolves the singleton record. A missing record is not an error.
func (s *pbStore) find(ctx context.Context) (settingsRecord, bool, error) {
q := url.Values{}
q.Set("filter", "key='"+globalSettingsKey+"'")
q.Set("perPage", "1")
res, err := s.client.List(ctx, s.collection, q)
if err != nil {
// A missing collection means bootstrap has not run yet — still "not
// ready" rather than "no settings", so nothing gets overwritten.
return settingsRecord{}, false, fmt.Errorf("%w: %v", errNotReady, err)
}
var items []settingsRecord
if err := json.Unmarshal(res.Items, &items); err != nil {
return settingsRecord{}, false, fmt.Errorf("%w: %v", errNotReady, err)
}
if len(items) == 0 {
return settingsRecord{}, false, nil
}
return items[0], true, nil
}
func (s *pbStore) Save(ctx context.Context, data []byte) error {
if !s.client.Configured() {
return fmt.Errorf("%w: no PocketBase service account configured", errPersist)
}
s.mu.Lock()
id := s.id
s.mu.Unlock()
// Resolve the record id if this process has not seen it yet (or if the
// record was recreated behind our back).
if id == "" {
rec, found, err := s.find(ctx)
if err != nil {
return fmt.Errorf("%w: %v", errPersist, err)
}
if found {
id = rec.ID
}
}
payload := map[string]any{
"key": globalSettingsKey,
PluginSettingsField: json.RawMessage(data),
}
if id != "" {
if err := s.client.Update(ctx, s.collection, id, payload, nil); err != nil {
if !isNotFound(err) {
return fmt.Errorf("%w: %v", errPersist, err)
}
// The record was deleted since we resolved it; fall through and
// recreate it rather than losing the save.
s.mu.Lock()
s.id = ""
s.mu.Unlock()
id = ""
} else {
s.mu.Lock()
s.id = id
s.mu.Unlock()
return nil
}
}
var created settingsRecord
if err := s.client.Create(ctx, s.collection, payload, &created); err != nil {
return fmt.Errorf("%w: %v", errPersist, err)
}
s.mu.Lock()
s.id = created.ID
s.mu.Unlock()
return nil
}
// isNotFound reports whether err is a PocketBase 404.
func isNotFound(err error) bool {
var apiErr *pb.APIError
return errors.As(err, &apiErr) && apiErr.Status == 404
}
// --- File-backed store (legacy import, and tests) ---------------------------
// fileStore keeps the document in a local JSON file. This was the only store
// before the global layer moved into PocketBase; it survives as the source of
// the one-time import, and as a convenient store for tests.
type fileStore struct{ path string }
// NewFileStore returns a Store backed by the JSON file at path.
func NewFileStore(path string) Store { return &fileStore{path: path} }
func (s *fileStore) Describe() string { return s.path }
func (s *fileStore) Load(ctx context.Context) ([]byte, bool, error) {
data, err := os.ReadFile(s.path)
switch {
case err == nil:
return data, true, nil
case errors.Is(err, os.ErrNotExist):
return nil, false, nil
default:
return nil, false, fmt.Errorf("%w: %v", errNotReady, err)
}
}
// Save writes through a temporary file and renames it into place, so an
// interrupted write cannot leave a half-written document behind.
func (s *fileStore) Save(ctx context.Context, data []byte) error {
tmp, err := os.CreateTemp(filepath.Dir(s.path), ".plugins-*.json")
if err != nil {
return fmt.Errorf("%w: %v", errPersist, err)
}
tmpName := tmp.Name()
defer func() {
if tmpName != "" {
_ = os.Remove(tmpName)
}
}()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return fmt.Errorf("%w: %v", errPersist, err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
return fmt.Errorf("%w: %v", errPersist, err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("%w: %v", errPersist, err)
}
if err := os.Rename(tmpName, s.path); err != nil {
return fmt.Errorf("%w: %v", errPersist, err)
}
tmpName = ""
return nil
}
// --- In-memory store (tests) ------------------------------------------------
// memoryStore holds the document in memory. failLoad/failSave let a test drive
// the not-ready and failed-save paths.
type memoryStore struct {
mu sync.Mutex
data []byte
found bool
failLoad error
failSave error
}
// NewMemoryStore returns a Store that keeps the document in memory.
func NewMemoryStore() Store { return &memoryStore{} }
func (s *memoryStore) Describe() string { return "memory" }
func (s *memoryStore) Load(ctx context.Context) ([]byte, bool, error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.failLoad != nil {
return nil, false, s.failLoad
}
return s.data, s.found, nil
}
func (s *memoryStore) Save(ctx context.Context, data []byte) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.failSave != nil {
return s.failSave
}
s.data = append([]byte(nil), data...)
s.found = true
return nil
}
// --- Legacy import ----------------------------------------------------------
// MigrateLegacyFile seeds store from the pre-PocketBase plugins.json at path,
// and is a no-op unless every condition holds: the store has no document at all,
// the file exists, and it parses. On success the file is renamed to
// <path>.migrated so a later boot cannot import it a second time or let the two
// copies drift apart.
//
// It reports whether an import happened. A store that could not be read returns
// an error and imports nothing — seeding on a failed read would overwrite live
// settings with a stale file.
func MigrateLegacyFile(ctx context.Context, store Store, path string) (bool, error) {
if path == "" {
return false, nil
}
_, found, err := store.Load(ctx)
if err != nil {
return false, err
}
if found {
return false, nil // already living in the store; the file is history
}
data, err := os.ReadFile(path)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return false, nil // fresh install, nothing to import
}
return false, err
}
// Only import something we can actually parse, so a corrupt file is left on
// disk for inspection rather than written into the database.
recs := map[string]*record{}
if err := decodeRecords(data, recs); err != nil {
return false, fmt.Errorf("legacy %s not imported: %v", path, err)
}
if len(recs) == 0 {
return false, nil
}
encoded, err := encodeRecords(recs)
if err != nil {
return false, err
}
if err := store.Save(ctx, encoded); err != nil {
return false, err
}
if err := os.Rename(path, path+".migrated"); err != nil {
// The settings are safely in the store; failing to rename only risks a
// confusing leftover file, so report it without undoing the import.
return true, fmt.Errorf("imported %s but could not rename it: %v", path, err)
}
return true, nil
}