driver-vault/DriverVault
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plugins: the global layer moves into the database, beside the other two

Browse Source 
The integration cascade stored its top layer differently from the two below
it: org (L2) and user (L3) plugin config lived in PocketBase, in a
pluginSettings field, while the global (L1) layer sat in a plugins.json
next to the binary. That split was accretion rather than design - the file
was the whole store in the v1 MVP, and the per-tenant layers were later
built on PocketBase and layered on top of it instead of replacing it.

It also cost something real. plugins.json was a second state store with
different durability from pb_data: its own volume, its own ownership, its
own backup. Losing pb_data is unmissable; losing api_data was silent, which
is how "every plugin comes back disabled after a redeploy" happened.

L1 now lives in the app_settings collection - one record keyed "global",
holding its settings in a pluginSettings field, the same mechanism and the
same field name the layers below use. The documents still differ in shape,
because only L1 carries enable state and the registration of external
plugins, but the storage is no longer a special case.

The Manager grows a Store seam (PocketBase in production, file for the
import, memory for tests) and, more importantly, a loaded gate. Settings in
a database mean the store can be unreachable at boot - a cold stack, or a
service account still to be set from the panel. That must not read as "no
plugins configured", or the first save would write emptiness over real
settings. So until a read succeeds the Manager stays unloaded, every
mutation is refused, /api/admin/plugins* answers 503, and a background
retry backs off to two minutes. The same gate covers a document that will
not parse: it is never replaced by one built from an empty map, which is a
stronger guarantee than the .corrupt backup it replaces.

Writing to a store also revealed a hole in the previous fix. Classifying a
save failure as errPersist was left to each Store, and a store that
returned a plain error would fall through to the "saved, but the plugin
failed to start" branch and be reported as a 200 - the same silent-success
bug through a different door. The Manager now classifies, whatever the
Store returns; a test pins it.

Upgrades are automatic: on the first boot that finds no settings in the
database, an existing plugins.json is imported and renamed to
plugins.json.migrated. The import is refused if the store is merely
unreachable, or if the file does not parse, so a stale or broken file can
never overwrite live settings. /data is still needed - the panel rewrites
.env there when it retargets PocketBase - but plugin settings no longer
depend on it.

21 tests in internal/plugins cover both stores, including the production
path against a fake PocketBase: create-then-update of the singleton,
round-trip across a restart, an outage that leaves settings intact, a
missing collection reading as not-ready rather than empty, and the import
running exactly once. go build, go vet and go test ./... pass. Schema
changes are mirrored into scripts/setup-pocketbase.mjs as that file
requires. Not verified: no Docker CLI here, so no image was built and the
bootstrap of app_settings against a real PocketBase is untested outside the
fake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
tajniak81
2026-08-21 16:52:47 +02:00
co-authored by Claude Opus 5
parent d2803bbd93
commit 9bd5c523c4
28 changed files with 1161 additions and 299 deletions
Download Patch File Download Diff File Expand all files Collapse all files
API Server/internal/api/control_e2e_test.go
+6 -1
View File
@@ -16,6 +16,7 @@ import (
"drivervault/apiserver/internal/config"
"drivervault/apiserver/internal/ocpp"
"drivervault/apiserver/internal/pb"
"drivervault/apiserver/internal/plugins"
)
// This test exercises the two PocketBase-touching control paths end-to-end
@@ -131,7 +132,11 @@ func TestControlStepUpAndAuditE2E(t *testing.T) {
PluginsFile: pluginsFile,
OCPPRequireTLS: false, // httptest is plaintext; TLS enforcement covered elsewhere
}, pb.New(pbSrv.URL, "admin@test.local", "pw"))
if err := s.plugins.Load(); err != nil {
// The global layer normally lives in PocketBase; point it at the file above
// so this test does not have to stand up an app_settings collection too.
s.pluginStore = plugins.NewFileStore(pluginsFile)
s.plugins = plugins.NewManager(s.pluginStore)
if err := s.plugins.Load(context.Background()); err != nil {
t.Fatalf("load plugins: %v", err)
}
// Seed the control-token index (mirrors what generating a token does).
API Server/internal/api/plugins.go
+35 -2
View File
@@ -8,13 +8,32 @@ import (
"drivervault/apiserver/internal/plugins"
)
// pluginsReady guards every plugin endpoint. Until the settings have been read
// from PocketBase the server knows of no plugins — reporting that as an empty or
// all-disabled list would be a lie the panel could then save back over the real
// settings, so the endpoints answer 503 instead.
func (s *Server) pluginsReady(w http.ResponseWriter) bool {
if s.plugins.Ready() {
return true
}
writeError(w, http.StatusServiceUnavailable,
"plugin settings are not loaded yet — the database is unreachable; retrying")
return false
}
// GET /api/admin/plugins — every known plugin (registry ∪ persisted), secrets masked.
func (s *Server) handleListPlugins(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"plugins": s.plugins.List()})
}
// GET /api/admin/plugins/{name} — one plugin's view.
func (s *Server) handleGetPlugin(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
v, ok := s.plugins.Get(r.PathValue("name"))
if !ok {
writeError(w, http.StatusNotFound, "unknown plugin")
@@ -26,6 +45,9 @@ func (s *Server) handleGetPlugin(w http.ResponseWriter, r *http.Request) {
// PUT /api/admin/plugins/{name} — enable/disable + merge config. Body:
// {enabled?, config?}. A secret left at the mask keeps its stored value.
func (s *Server) handleUpdatePlugin(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
name := r.PathValue("name")
current, ok := s.plugins.Get(name)
if !ok {
@@ -51,8 +73,10 @@ func (s *Server) handleUpdatePlugin(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{"plugin": v})
case plugins.IsUnknown(err):
writeError(w, http.StatusNotFound, "unknown plugin")
case plugins.IsNotReady(err):
writeError(w, http.StatusServiceUnavailable, err.Error())
case plugins.IsPersist(err):
// The change never reached plugins.json and has been rolled back.
// The change never reached the store and has been rolled back.
// Reporting this as a 200-with-warning is what let a plugin look
// enabled in the panel and come back disabled after a redeploy.
writeError(w, http.StatusInternalServerError, err.Error())
@@ -65,6 +89,9 @@ func (s *Server) handleUpdatePlugin(w http.ResponseWriter, r *http.Request) {
// POST /api/admin/plugins — register an external (remote HTTP) plugin. Body:
// {name, baseURL, provider?}. This is the "add a plugin without a rebuild" path.
func (s *Server) handleRegisterPlugin(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
var body struct {
Name string `json:"name"`
BaseURL string `json:"baseURL"`
@@ -79,7 +106,7 @@ func (s *Server) handleRegisterPlugin(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "name and baseURL are required")
return
}
if err := s.plugins.RegisterExternal(body.Name, body.BaseURL, body.Provider); err != nil {
if err := s.plugins.RegisterExternal(r.Context(), body.Name, body.BaseURL, body.Provider); err != nil {
if plugins.IsPersist(err) {
writeError(w, http.StatusInternalServerError, err.Error())
return
@@ -94,6 +121,9 @@ func (s *Server) handleRegisterPlugin(w http.ResponseWriter, r *http.Request) {
// DELETE /api/admin/plugins/{name} — remove an external plugin (builtins can
// only be disabled).
func (s *Server) handleDeletePlugin(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
if err := s.plugins.Remove(r.Context(), r.PathValue("name")); err != nil {
if plugins.IsPersist(err) {
writeError(w, http.StatusInternalServerError, err.Error())
@@ -108,6 +138,9 @@ func (s *Server) handleDeletePlugin(w http.ResponseWriter, r *http.Request) {
// POST /api/admin/plugins/{name}/health — run a health check now. Works on
// disabled plugins too, so a config can be verified before enabling it.
func (s *Server) handlePluginHealth(w http.ResponseWriter, r *http.Request) {
if !s.pluginsReady(w) {
return
}
h, err := s.plugins.HealthCheck(r.Context(), r.PathValue("name"))
if err != nil {
if plugins.IsUnknown(err) {
API Server/internal/api/server.go
+85 -23
View File
@@ -148,6 +148,7 @@ const (
colDocuments = "car_documents"
colReminders = "reminders"
colControlAudit = "control_audit"
colAppSettings = "app_settings"
)
// Server wires together the HTTP handlers and their dependencies.
@@ -157,6 +158,13 @@ type Server struct {
pb *pb.Client
plugins *plugins.Manager
// pluginStore is the Manager's backing store, kept here so the one-time
// import of a legacy plugins.json can address it directly. legacyPlugins is
// the path that import reads; pluginsStop ends the background load retry.
pluginStore plugins.Store
legacyPlugins string
pluginsStop context.CancelFunc
// ocpp is the OCPP 1.6J Central System that Anker Solix chargers connect to
// when their owner picks a control mode of own/proxy (see internal/ocpp and
// integrations_ankersolix_control.go). Nil-safe: control endpoints report a
@@ -168,41 +176,95 @@ type Server struct {
// New constructs a Server around an already-built PocketBase client.
func New(cfg config.Config, client *pb.Client) *Server {
// The global (L1) plugin layer lives in PocketBase alongside the org (L2)
// and user (L3) layers, rather than in a file beside the binary.
store := plugins.NewPocketBaseStore(client, colAppSettings)
return &Server{
cfg: cfg,
pb: client,
plugins: plugins.NewManager(cfg.PluginsFile),
ocpp: ocpp.NewCSMS(func(f string, a ...any) { log.Printf("ocpp: "+f, a...) }),
control: newControlIndex(),
ctlRL: newRateLimiter(30, time.Minute), // 30 control commands / min / charger
cfg: cfg,
pb: client,
pluginStore: store,
legacyPlugins: cfg.PluginsFile,
plugins: plugins.NewManager(store),
ocpp: ocpp.NewCSMS(func(f string, a ...any) { log.Printf("ocpp: "+f, a...) }),
control: newControlIndex(),
ctlRL: newRateLimiter(30, time.Minute), // 30 control commands / min / charger
}
}
// StartPlugins loads persisted plugin state and initialises enabled plugins. It
// also warms the OCPP control-token index from PocketBase (its source of truth),
// so the first charger to reconnect after a restart resolves immediately instead
// of triggering a lazy rebuild mid-handshake. The warm-up is best-effort and
// non-blocking; if PocketBase is not yet configured it no-ops and the lazy path
// rebuilds on first connect.
// StartPlugins reads the plugin settings from PocketBase and initialises every
// enabled plugin. It also warms the OCPP control-token index from PocketBase (its
// source of truth), so the first charger to reconnect after a restart resolves
// immediately instead of triggering a lazy rebuild mid-handshake. The warm-up is
// best-effort and non-blocking; if PocketBase is not yet configured it no-ops and
// the lazy path rebuilds on first connect.
//
// The settings now live in the database, so at boot the database may not be
// reachable yet — a cold stack, or a service account still to be configured from
// the panel. That is not fatal and, crucially, not treated as "no plugins
// configured": the Manager stays unloaded, the admin endpoints answer 503, and a
// background retry keeps trying until the read succeeds. Nothing is written
// until something has been read, so an outage cannot erase the settings.
func (s *Server) StartPlugins() error {
// Surface an unwritable state directory at boot. Without this the first
// symptom is a superadmin enabling plugins, seeing them work, and finding
// them all disabled after the next redeploy — because every save failed.
if err := s.plugins.CheckWritable(); err != nil {
log.Printf("WARNING: %v", err)
log.Printf("WARNING: plugin changes will NOT survive a restart — make the directory holding PLUGINS_FILE writable by the container user")
}
err := s.plugins.Load()
ctx, cancel := context.WithCancel(context.Background())
s.pluginsStop = cancel
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
s.ensureControlIndex(ctx)
warmCtx, warmCancel := context.WithTimeout(context.Background(), 15*time.Second)
defer warmCancel()
s.ensureControlIndex(warmCtx)
}()
err := s.loadPlugins(ctx)
if err != nil {
log.Printf("plugins: settings unavailable, retrying in the background (%v)", err)
go s.retryLoadPlugins(ctx)
}
return err
}
// loadPlugins imports a pre-PocketBase plugins.json if one is still lying around
// and the database holds no settings yet, then reads the settings.
func (s *Server) loadPlugins(ctx context.Context) error {
switch migrated, err := plugins.MigrateLegacyFile(ctx, s.pluginStore, s.legacyPlugins); {
case err != nil:
// Not fatal: the read below reports the real problem if there is one.
log.Printf("plugins: legacy import skipped: %v", err)
case migrated:
log.Printf("plugins: imported %s into PocketBase; renamed it to %s.migrated",
s.legacyPlugins, s.legacyPlugins)
}
return s.plugins.Load(ctx)
}
// retryLoadPlugins keeps reading until the settings load or the server stops.
// The backoff caps at two minutes, so a long outage costs at most one log line
// every two minutes rather than a tight spin.
func (s *Server) retryLoadPlugins(ctx context.Context) {
const maxBackoff = 2 * time.Minute
backoff := 5 * time.Second
for {
select {
case <-ctx.Done():
return
case <-time.After(backoff):
}
if err := s.loadPlugins(ctx); err == nil {
log.Println("plugins: settings loaded")
return
} else {
log.Printf("plugins: still unavailable, retrying in %s (%v)", backoff, err)
}
if backoff < maxBackoff {
backoff *= 2
}
}
}
// Stop releases server-held resources (plugin instances and OCPP sessions).
func (s *Server) Stop(ctx context.Context) {
if s.pluginsStop != nil {
s.pluginsStop()
}
s.ocpp.Shutdown(ctx)
s.plugins.Shutdown(ctx)
}
API Server/internal/api/vehicleproviders_test.go
+7 -1
View File
@@ -1,6 +1,7 @@
package api
import (
"context"
"encoding/json"
"io"
"net/http"
@@ -13,6 +14,7 @@ import (
"drivervault/apiserver/internal/config"
"drivervault/apiserver/internal/pb"
"drivervault/apiserver/internal/plugins"
)
// The vehicle-provider layer deliberately searches payloads by key name instead
@@ -460,7 +462,11 @@ func newProviderTestServer(t *testing.T, fake *fakeProviderPB) *httptest.Server
}
s := New(config.Config{UsersCollection: "users", PluginsFile: pluginsFile},
pb.New(pbSrv.URL, "admin@test.local", "pw"))
if err := s.plugins.Load(); err != nil {
// The global layer normally lives in PocketBase; point it at the file above
// so this test does not have to stand up an app_settings collection too.
s.pluginStore = plugins.NewFileStore(pluginsFile)
s.plugins = plugins.NewManager(s.pluginStore)
if err := s.plugins.Load(context.Background()); err != nil {
t.Fatalf("load plugins: %v", err)
}
srv := httptest.NewServer(s.Handler())