normalize negative pool size

.github/SECURITY.md

@@ -114,3 +114,15 @@ Third, even if that issue is still available, with PocketBase it would have been
 
 In the future I may consider eventually replacing the library because it is no longer actively maintained but as of now it is working correctly and as expected for our use case and you can safely flag the security warning as false-positive.
 </details>
+
+<details>
+<summary><strong>JSVM "sandboxing"</strong></summary>
+
+This is another very common report but **there is no such thing as JSVM "sandboxing" in PocketBase**.
+
+The JS `pb_hooks` (or JSVM for short) are NOT supposed to run untrusted or client provided JavaScript code _(the same way you are not supposed to run untrusted code in your Node.js server)_.
+
+Once interpreted the `pb_hooks` run as part of the same application process together with the rest of the Go code. There are no additional filesystem, network, memory, etc. restrictions. This means that it is OK for developers to be able to access environment variables, perform network calls to any URLs they want, invoke shell commands or even sleep/block the script execution.
+
+So if you are security researcher and not sure if something is a security fault in the JS hooks, ask yourself - "Can I do the same when using PocketBase as Go framework?" and if the answer is "Yes" then it is not a security issue with the JSVM.
+</details>

.github/workflows/release.yaml

@@ -26,9 +26,9 @@ jobs:
           node-version: '>=25.2.1'
 
       - name: Set up Go
-        uses: actions/setup-go@v6
+        uses: actions/setup-go@v7
         with:
-          go-version: '>=1.26.5'
+          go-version: '>=1.27.1'
 
       # This step usually is not needed because the /ui/dist is pregenerated locally
       # but its here to ensure that each release embeds the latest admin ui artifacts.

CHANGELOG_16_22.md

@@ -2,6 +2,60 @@
 > For the most recent versions, please refer to [CHANGELOG.md](./CHANGELOG.md)
 ---
 
+## v0.22.55
+
+- (_Backported from v0.40.3_) Fixed collection index validator to allow expressions with parenthesis in the optional `WHERE` clause.
+
+- (_Backported from v0.40.3_) Fixed nested cascade delete of self-referenced relation records.
+
+- (_Backported from v0.40.3_) Bumped `golang.org/x/*` dependencies to silence security scanners ([#7829](https://github.com/pocketbase/pocketbase/discussions/7829)).
+
+
+## v0.22.54
+
+- (_Backported from v0.40.2_) Bumped goja and its related dependencies _(regex unescaped dash error fix and base64 optimizations)_.
+
+- (_Backported from v0.40.2_) Return an error when filter params fallback fails to json serialize and optimized params replacement to execute in a single pass.
+
+- (_Backported from v0.40.2_) Bumped the min Go GitHub action version to 1.27.1 as it includes some [minor `database/sql` and `enconding/json/v2` bug fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.27.1).
+
+
+## v0.22.53
+
+- Workaround `encoding/json` infinite recursion when used with Go 1.27 ([#7808](https://github.com/pocketbase/pocketbase/issues/7808)).
+
+
+## v0.22.52
+
+- (_Backported from v0.39.11_) Bumped `golang.org/x/*` deps and the min Go GitHub action version to 1.26.6 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.6).
+
+
+## v0.22.51
+
+- (_Backported from v0.39.10_) Reverted the auto panic recover handling for the cli commands to preserve the old behavior and allow panic to force exit with non-zero code ([#7781](https://github.com/pocketbase/pocketbase/issues/7781)).
+
+
+## v0.22.50
+
+- (_Backported from v0.39.9_) Bumped goja, fexpr and their related deps.
+
+
+## v0.22.49
+
+- (_Backported from v0.39.8_) Bumped `golang.org/x/*` indirect dependencies as there are some minor security fixes.
+
+- (_Backported from v0.39.8_) Updated `modernc.org/sqlite` to v1.54.0 ([SQLite 3.53.3](https://sqlite.org/src/timeline?from=version-3.53.2&to=version-3.53.3&to2=branch-3.53)).
+
+
+## v0.22.48
+
+- (_Backported from v0.39.7_) Replaced `github.com/go-ozzo/ozzo-validation` with the fork `github.com/pocketbase/ozzo-validation` since the original library has recently changed ownership and the new maintainer cannot be trusted.
+
+- (_Backported from v0.39.7_) Fixed View collection `*` validator and added more friendly error messages ([#7761](https://github.com/pocketbase/pocketbase/issues/7761)).
+
+- (_Backported from v0.39.7_) ⚠️ Security fix for unhandled panic in internal worker goroutines ([#7762](https://github.com/pocketbase/pocketbase/discussions/7762)).
+
+
 ## v0.22.47
 
 - (_Backported from v0.39.6_) Bumped the min Go GitHub action version to 1.26.5 as it includes some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.5).

CONTRIBUTING.md

@@ -13,7 +13,7 @@ This document describes how to prepare a PR for a change in the main repository.
 
 ## Prerequisites
 
-- Go 1.25+ (for making changes in the Go code)
+- Go 1.27+ (for making changes in the Go code)
 - Node 24+ (for making changes in the Superuser UI)
 
 If you haven't already, you can fork the main repository and clone your fork so that you can work locally:

README.md

@@ -49,7 +49,7 @@ your own custom app specific business logic and still have a single portable exe
 
 Here is a minimal example:
 
-0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 
 1. Create a new project directory with the following `main.go` file inside it:
     ```go
@@ -92,32 +92,37 @@ _For more details please refer to [Extend with Go](https://pocketbase.io/docs/go
 
 To build the minimal standalone executable, like the prebuilt ones in the releases page, you can simply run `go build` inside the `examples/base` directory:
 
-0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 1. Clone/download the repo
 2. Navigate to `examples/base`
-3. Run `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`
-   (_https://go.dev/doc/install/source#environment_)
+3. Run `CGO_ENABLED=0 go build` to build a binary for your current environment
+   _(or to target other platforms use `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`; see https://go.dev/doc/install/source#environment)_
 4. Start the created executable by running `./base serve`.
 
 Note that the supported build targets by the pure Go SQLite driver at the moment are:
 
-```
-darwin  amd64
-darwin  arm64
-freebsd amd64
-freebsd arm64
-linux   386
-linux   amd64
-linux   arm
-linux   arm64
-linux   loong64
-linux   ppc64le
-linux   riscv64
-linux   s390x
-windows 386
-windows amd64
-windows arm64
-```
+| GOOS    | GOARCH  |
+|---------|---------|
+| darwin  | amd64   |
+| darwin  | arm64   |
+| freebsd | 386     |
+| freebsd | amd64   |
+| freebsd | arm     |
+| freebsd | arm64   |
+| linux   | 386     |
+| linux   | amd64   |
+| linux   | arm     |
+| linux   | arm64   |
+| linux   | loong64 |
+| linux   | ppc64le |
+| linux   | riscv64 |
+| linux   | s390x   |
+| netbsd  | amd64   |
+| openbsd | amd64   |
+| openbsd | arm64   |
+| windows | 386     |
+| windows | amd64   |
+| windows | arm64   |
 
 ### Testing
 
@@ -134,7 +139,7 @@ Check also the [Testing guide](http://pocketbase.io/docs/testing) to learn how t
 
 If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
 
-All reports will be promptly addressed and you'll be credited in the fix release notes.
+You could find more details in the project [Security policy](https://github.com/pocketbase/pocketbase/security/policy).
 
 ## Contributing
 

apis/backup_create.go

@@ -5,7 +5,7 @@ import (
 	"net/http"
 	"regexp"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/backup_upload.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/filesystem"

apis/batch.go

@@ -2,7 +2,7 @@ package apis
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"io"
 	"mime/multipart"
@@ -14,10 +14,11 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/types"
 	"github.com/spf13/cast"
 )
@@ -195,7 +196,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			p.stopCh <- struct{}{}
 		}()
 
-		go func() {
+		routine.FireAndForget(func() {
 			err := p.process(txApp, batch, 0)
 
 			if err != nil {
@@ -216,7 +217,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			}
 
 			p.errCh <- err
-		}()
+		})
 
 		select {
 		case responseErr := <-p.errCh:

apis/collection.go

@@ -6,7 +6,7 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/router"

apis/collection_import.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/collection_test.go

@@ -1225,6 +1225,71 @@ func TestCollectionUpdate(t *testing.T) {
 				"OnModelValidate":              1,
 			},
 		},
+		{
+			Name:   "add another OAuth2 provider to an auth collection",
+			Method: http.MethodPatch,
+			URL:    "/api/collections/users",
+			Body: strings.NewReader(`{
+				"oauth2": {
+					"providers": [
+						{"name": "apple", "clientId": "a", "clientSecret": "b"},
+						{
+							"pkce": null,
+							"name": "google",
+							"authURL": "",
+							"displayName": "existing",
+							"extra": {}
+						}
+					]
+				}
+			}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				// verify that the collection has google and gitlab as OAuth2 providers
+				users, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if v := len(users.OAuth2.Providers); v != 2 {
+					t.Fatalf("Expected 2 OAuth2 providers, got %d", v)
+				}
+
+				if v := users.OAuth2.Providers[0].Name; v != "gitlab" {
+					t.Fatalf("Expected provider 0 to be %s, got %s", "gitlab", v)
+				}
+
+				if v := users.OAuth2.Providers[1].Name; v != "google" {
+					t.Fatalf("Expected provider 1 to be %s, got %s", "google", v)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"name":"google"`,
+				`"name":"apple"`,
+				`"displayName":"existing"`,
+				`"clientId":"test"`,
+				`"clientId":"a"`,
+			},
+			NotExpectedContent: []string{
+				`"name":"gitlab"`,
+				`clientSecret`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                              0,
+				"OnCollectionUpdateRequest":      1,
+				"OnCollectionUpdate":             1,
+				"OnCollectionUpdateExecute":      1,
+				"OnCollectionAfterUpdateSuccess": 1,
+				"OnCollectionValidate":           1,
+				"OnModelUpdate":                  1,
+				"OnModelUpdateExecute":           1,
+				"OnModelAfterUpdateSuccess":      1,
+				"OnModelValidate":                1,
+			},
+		},
 
 		// view
 		// -----------------------------------------------------------
@@ -1617,7 +1682,7 @@ func TestCollectionOAuth2Providers(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
-				`{"name":"oidc3","displayName":"OIDC","logo":"\u003csvg`,
+				`{"name":"oidc3","displayName":"OIDC","logo":"<svg`,
 			},
 			NotExpectedContent: []string{
 				`"order":`,

apis/cron_test.go

@@ -77,10 +77,13 @@ func TestCronsRun(t *testing.T) {
 		app.Cron().Add("test", "* * * * *", func() {
 			app.Store().Set("testJobCalls", cast.ToInt(app.Store().Get("testJobCalls"))+1)
 		})
+		app.Cron().Stop()
 	}
 
 	expectedCalls := func(expected int) func(t testing.TB, app *tests.TestApp, res *http.Response) {
 		return func(t testing.TB, app *tests.TestApp, res *http.Response) {
+			time.Sleep(50 * time.Millisecond)
+
 			total := cast.ToInt(app.Store().Get("testJobCalls"))
 			if total != expected {
 				t.Fatalf("Expected total testJobCalls %d, got %d", expected, total)
@@ -93,7 +96,6 @@ func TestCronsRun(t *testing.T) {
 			Name:            "unauthorized",
 			Method:          http.MethodPost,
 			URL:             "/api/crons/test",
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  401,
@@ -107,7 +109,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
 			},
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  403,
@@ -121,7 +122,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
 			},
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  404,
@@ -135,7 +135,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
 			},
-			Delay:          50 * time.Millisecond,
 			BeforeTestFunc: beforeTestFunc,
 			AfterTestFunc:  expectedCalls(1),
 			ExpectedStatus: 204,

apis/health.go

@@ -37,7 +37,7 @@ func healthCheck(e *core.RequestEvent) error {
 		headersToCheck := append(
 			slices.Clone(e.App.Settings().TrustedProxy.Headers),
 			// common proxy headers
-			"CF-Connecting-IP", "Fly-Client-IP", "X‑Forwarded-For",
+			"CF-Connecting-IP", "Fly-Client-IP", "X-Forwarded-For",
 		)
 		for _, header := range headersToCheck {
 			if e.Request.Header.Get(header) != "" {

apis/logs.go

@@ -11,8 +11,13 @@ import (
 
 // bindLogsApi registers the request logs api endpoints.
 func bindLogsApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
-	sub := rg.Group("/logs").Bind(RequireSuperuserAuth(), SkipSuccessActivityLog())
+	sub := rg.Group("/logs").Bind(
+		RequireSuperuserAuth(),
+		SkipSuccessActivityLog(),
+	)
+
 	sub.GET("", logsList)
+	sub.DELETE("", logsTruncate)
 	sub.GET("/stats", logsStats)
 	sub.GET("/{id}", logsView)
 }
@@ -71,3 +76,19 @@ func logsView(e *core.RequestEvent) error {
 
 	return e.JSON(http.StatusOK, log)
 }
+
+func logsTruncate(e *core.RequestEvent) error {
+	// delete all rows directly (aka. no model hooks will be fired)
+	_, err := e.App.AuxNonconcurrentDB().Delete((&core.Log{}).TableName(), nil).Execute()
+	if err != nil {
+		return e.InternalServerError("Failed to truncate all logs.", err)
+	}
+
+	// try to free the unused disk space
+	err = e.App.AuxVacuum()
+	if err != nil {
+		e.App.Logger().Warn("Failed to VACUUM aux database", "error", err)
+	}
+
+	return e.NoContent(http.StatusNoContent)
+}

apis/logs_test.go

@@ -210,3 +210,59 @@ func TestLogsStats(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestLogsTruncate(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodDelete,
+			URL:             "/api/logs",
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := tests.StubLogsData(app); err != nil {
+					t.Fatal(err)
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				var found []core.Log
+
+				if err := app.LogQuery().All(&found); err != nil {
+					t.Fatal(err)
+				}
+
+				if len(found) > 0 {
+					t.Fatalf("Expected all logs to be deleted, found: %v", found)
+				}
+			},
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/middlewares.go

@@ -293,6 +293,7 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
 			e.Response.Header().Set("X-XSS-Protection", "1; mode=block")
 			e.Response.Header().Set("X-Content-Type-Options", "nosniff")
 			e.Response.Header().Set("X-Frame-Options", "SAMEORIGIN")
+			e.Response.Header().Set("Cross-Origin-Opener-Policy", "same-origin")
 
 			// @todo consider a default HSTS?
 			// (see also https://webkit.org/blog/8146/protecting-against-hsts-abuse/)

apis/middlewares_body_limit.go

@@ -84,49 +84,73 @@ func applyBodyLimit(e *core.RequestEvent, limitBytes int64) error {
 	}
 
 	// replace the request body
-	//
-	// note: we don't use sync.Pool since the size of the elements could vary too much
-	// and it might not be efficient (see https://github.com/golang/go/issues/23199)
-	e.Request.Body = &limitedReader{ReadCloser: e.Request.Body, limit: limitBytes}
+	e.Request.Body = newMaxBytesReader(e.Request.Body, limitBytes)
 
 	return nil
 }
 
-type limitedReader struct {
-	io.ReadCloser
-	limit     int64
-	totalRead int64
+func newMaxBytesReader(body io.ReadCloser, limitBytes int64) *maxBytesReader {
+	return &maxBytesReader{
+		ReadCloser: body,
+		limit:      limitBytes,
+		remaining:  limitBytes,
+	}
 }
 
-func (r *limitedReader) Read(b []byte) (int, error) {
+// maxBytesReader is very similar to the http.MaxBytesReader but support
+// rereads and doesn't try to prematurely close the related response
+// to allow consequent middlewares to operate correctly.
+type maxBytesReader struct {
+	io.ReadCloser
+	limit     int64
+	remaining int64
+	stickyErr error
+}
+
+func (r *maxBytesReader) Read(b []byte) (int, error) {
+	if r.stickyErr != nil {
+		return 0, r.stickyErr
+	}
+
+	if len(b) == 0 {
+		return 0, nil
+	}
+
+	// if possible no need to read the entire chunk since
+	// remaining+1 is enough to determine whether it exceed the limit
+	if int64(len(b))-1 > r.remaining {
+		b = b[:r.remaining+1]
+	}
+
 	n, err := r.ReadCloser.Read(b)
-	if err != nil {
+
+	if int64(n) <= r.remaining {
+		r.remaining -= int64(n)
+		r.stickyErr = err
 		return n, err
 	}
 
-	r.totalRead += int64(n)
-	if r.totalRead > r.limit {
-		return n, ErrRequestEntityTooLarge
-	}
+	n = int(r.remaining)
 
-	return n, nil
+	r.remaining = 0
+	r.stickyErr = ErrRequestEntityTooLarge
+
+	return n, r.stickyErr
 }
 
 // explicit casts to ensure that the main struct methods will be invoked
 // (extra precautions in case of nested interface wrapping erasure)
 // ---
 
-func (r *limitedReader) Reread() {
+func (r *maxBytesReader) Reread() {
 	rereader, ok := r.ReadCloser.(router.Rereader)
 	if ok {
 		rereader.Reread()
+		r.remaining = r.limit
+		r.stickyErr = nil
 	}
 }
 
-func (r *limitedReader) Close() error {
-	closer, ok := r.ReadCloser.(io.Closer)
-	if ok {
-		return closer.Close()
-	}
-	return nil
+func (r *maxBytesReader) Close() error {
+	return r.ReadCloser.Close()
 }

apis/middlewares_body_limit_test.go

@@ -1,9 +1,10 @@
 package apis_test
 
 import (
-	"bytes"
-	"fmt"
+	"io"
+	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/apis"
@@ -19,13 +20,35 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	pbRouter.POST("/a", func(e *core.RequestEvent) error {
-		return e.String(200, "a")
-	}) // default global BodyLimit check
 
-	pbRouter.POST("/b", func(e *core.RequestEvent) error {
-		return e.String(200, "b")
-	}).Bind(apis.BodyLimit(20))
+	testHandler := func(e *core.RequestEvent) error {
+		// read the body multiple times to ensure that the limited
+		// reader guards and rereads are invoked
+		var result any
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		return e.JSON(200, result)
+	}
+
+	const customLimit = 20
+
+	pbRouter.POST("/a", testHandler) // default global BodyLimit check
+	pbRouter.POST("/b", testHandler).Bind(apis.BodyLimit(customLimit))
+	pbRouter.POST("/iof", func(e *core.RequestEvent) error {
+		// ensure that normal io methods still operate correctly
+		b, err := io.ReadAll(e.Request.Body)
+		if err != nil {
+			return err
+		}
+
+		return e.String(http.StatusOK, string(b))
+	}).Bind(apis.BodyLimit(customLimit))
 
 	mux, err := pbRouter.BuildMux()
 	if err != nil {
@@ -33,20 +56,90 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	}
 
 	scenarios := []struct {
-		url            string
-		size           int64
-		expectedStatus int
+		name              string
+		url               string
+		body              string
+		lazyContentLength bool
+		expectedStatus    int
 	}{
-		{"/a", 21, 200},
-		{"/a", apis.DefaultMaxBodySize + 1, 413},
-		{"/b", 20, 200},
-		{"/b", 21, 413},
+		{
+			"(eager content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"(eager content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"io.ReadAll io.EOF exact limit check",
+			"/iof",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
 	}
 
 	for _, s := range scenarios {
-		t.Run(fmt.Sprintf("%s_%d", s.url, s.size), func(t *testing.T) {
+		t.Run(s.name, func(t *testing.T) {
 			rec := httptest.NewRecorder()
-			req := httptest.NewRequest("POST", s.url, bytes.NewReader(make([]byte, s.size)))
+
+			req := httptest.NewRequest("POST", s.url, strings.NewReader(s.body))
+			req.Header.Set("Content-Type", "application/json")
+
+			if s.lazyContentLength {
+				req.ContentLength = -1
+			}
+
 			mux.ServeHTTP(rec, req)
 
 			result := rec.Result()

apis/middlewares_rate_limit_test.go

@@ -1,8 +1,10 @@
 package apis_test
 
 import (
+	"fmt"
 	"net/http/httptest"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"github.com/pocketbase/pocketbase/apis"
@@ -127,27 +129,29 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 		{"/rate/guest", 0, true, 429},
 	}
 
-	for _, s := range scenarios {
-		t.Run(s.url, func(t *testing.T) {
+	synctest.Test(t, func(t *testing.T) {
+		for i, s := range scenarios {
+			prefix := fmt.Sprintf("[%s:%d] ", s.url, i+1)
+
 			rec := httptest.NewRecorder()
 			req := httptest.NewRequest("GET", s.url, nil)
 
 			if s.authenticated {
 				auth, err := app.FindAuthRecordByEmail("users", "test@example.com")
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				token, err := auth.NewAuthToken()
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				req.Header.Add("Authorization", token)
 			}
 
 			if s.wait > 0 {
-				time.Sleep(time.Duration(s.wait) * time.Millisecond)
+				synctest.Sleep(time.Duration(s.wait) * time.Millisecond)
 			}
 
 			mux.ServeHTTP(rec, req)
@@ -155,10 +159,10 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 			result := rec.Result()
 
 			if result.StatusCode != s.expectedStatus {
-				t.Fatalf("Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
+				t.Fatalf(prefix+"Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
 			}
-		})
-	}
+		}
+	})
 }
 
 func TestDefaultRateLimitMiddlewareSkipChecks(t *testing.T) {

apis/realtime.go

@@ -2,7 +2,7 @@ package apis
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"log/slog"
@@ -10,8 +10,8 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/picker"
@@ -260,7 +260,7 @@ func realtimeUpdateClientsAuth(app core.App, authRecord *core.Record) error {
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
@@ -275,7 +275,7 @@ func realtimeUpdateClientsAuth(app core.App, authRecord *core.Record) error {
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -288,7 +288,7 @@ func realtimeUnsetClientsAuthByRecordModelOrProxy(app core.App, authModel core.M
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
@@ -299,7 +299,7 @@ func realtimeUnsetClientsAuthByRecordModelOrProxy(app core.App, authModel core.M
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -312,7 +312,7 @@ func realtimeUnsetClientsAuthByCollection(app core.App, collection *core.Collect
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil && clientAuth.Collection().Name == collection.Name {
@@ -321,7 +321,7 @@ func realtimeUnsetClientsAuthByCollection(app core.App, collection *core.Collect
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -623,7 +623,7 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 	}
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			var clientAuth *core.Record
 
 			for _, client := range chunk {
@@ -766,7 +766,7 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -782,7 +782,7 @@ func realtimeBroadcastDryCacheKey(app core.App, key string) error {
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				messages, ok := client.Get(key).([]subscriptions.Message)
 				if !ok {
@@ -801,7 +801,7 @@ func realtimeBroadcastDryCacheKey(app core.App, key string) error {
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -817,7 +817,7 @@ func realtimeUnsetDryCacheKey(app core.App, key string) error {
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				if client.Get(key) != nil {
 					client.Unset(key)
@@ -825,7 +825,7 @@ func realtimeUnsetDryCacheKey(app core.App, key string) error {
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()

apis/realtime_test.go

@@ -2,7 +2,7 @@ package apis_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"net/http"
@@ -1171,7 +1171,9 @@ func TestRealtimeRecordResolve(t *testing.T) {
 
 			var mu sync.Mutex
 			notify := func(clientId string, eventData []byte) {
-				data := struct{ Action string }{}
+				data := struct {
+					Action string `json:"action"`
+				}{}
 				_ = json.Unmarshal(eventData, &data)
 
 				mu.Lock()

apis/record_auth_email_change_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 )

apis/record_auth_email_change_request.go

@@ -3,8 +3,8 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 )

apis/record_auth_impersonate.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/record_auth_methods_test.go

@@ -54,7 +54,7 @@ func TestRecordAuthMethodsList(t *testing.T) {
 				`"providers":[{`,
 				`"name":"google"`,
 				`"name":"gitlab"`,
-				`"logo":"\u003csvg`,
+				`"logo":"<svg`,
 				`"logo":""`, // for the legacy fields
 				`"state":`,
 				`"displayName":`,

apis/record_auth_otp_request.go

@@ -6,8 +6,8 @@ import (
 	"fmt"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_password_reset_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/security"

apis/record_auth_password_reset_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_verification_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"

apis/record_auth_verification_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_with_oauth2.go

@@ -4,7 +4,7 @@ import (
 	"bytes"
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"io"
@@ -17,8 +17,8 @@ import (
 	"syscall"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/dbutils"

apis/record_auth_with_oauth2_redirect.go

@@ -1,7 +1,7 @@
 package apis
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"net/http"
 	"strings"

apis/record_auth_with_otp.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"fmt"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/record_auth_with_password.go

@@ -6,9 +6,9 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/list"

apis/record_helpers_test.go

@@ -1,7 +1,7 @@
 package apis_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"net/http"
 	"net/http/httptest"

apis/settings.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/tools/router"

apis/sql.go

@@ -8,7 +8,7 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/router"
 )

cmd/superuser.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 
 	"github.com/fatih/color"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cobra"

core/app.go

@@ -54,11 +54,16 @@ type App interface {
 	// Bootstrap initializes the application
 	// (aka. create data dir, open db connections, load settings, etc.).
 	//
-	// It will call ResetBootstrapState() if the application was already bootstrapped.
+	// It calls ClearBootstrap() if the application was already bootstrapped.
 	Bootstrap() error
 
-	// ResetBootstrapState releases the initialized core app resources
+	// ClearBootstrap releases the initialized core app resources
 	// (closing db connections, stopping cron ticker, etc.).
+	//
+	// This method is no-op if the application is not bootstrapped yet.
+	ClearBootstrap() error
+
+	// Deprecated: use ClearBootstrap().
 	ResetBootstrapState() error
 
 	// DataDir returns the app data directory path.
@@ -713,6 +718,13 @@ type App interface {
 	// resources (db, app settings, etc).
 	OnBootstrap() *hook.Hook[*BootstrapEvent]
 
+	// OnBootstrapClear hook is triggered when clearing the main application
+	// resources (db connections, cron, logger, etc.)
+	//
+	// It is usually invoked automatically right before app termination
+	// or when manually calling app.ClearBootstrap().
+	OnBootstrapClear() *hook.Hook[*BootstrapEvent]
+
 	// OnServe hook is triggered when the app web server is started
 	// (after starting the TCP listener but before initializing the blocking serve task),
 	// allowing you to adjust its options and attach new routes or middlewares.
@@ -1262,6 +1274,20 @@ type App interface {
 	// triggered and called only if their event data origin matches the tags.
 	OnMailerRecordOTPSend(tags ...string) *hook.TaggedHook[*MailerRecordEvent]
 
+	// ---------------------------------------------------------------
+	// Filesystem event hooks
+	// (not publicly exposed until file_field refactoring)
+	// ---------------------------------------------------------------
+
+	// onFilesystemNewWriter is an internal hook for app.NewFilesystem()
+	// instances that is triggered on every storage filesystem writer initialization
+	// (aka. whenever attempting to create a new file).
+	onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent]
+
+	// onFilesystemDelete is an internal hook for app.NewFilesystem()
+	// instances that is triggered for every storage file delete call.
+	onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent]
+
 	// ---------------------------------------------------------------
 	// Realtime API event hooks
 	// ---------------------------------------------------------------

core/auth_origin_model.go

@@ -5,8 +5,8 @@ import (
 	"errors"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/backup.go

@@ -0,0 +1,137 @@
+package core
+
+import (
+	"context"
+	"fmt"
+	"log/slog"
+	"sort"
+	"time"
+
+	"github.com/pocketbase/pocketbase/tools/inflector"
+)
+
+const (
+	StoreKeyActiveBackup = "@activeBackup"
+)
+
+// generateBackupName generates a new backup name based on the app name and current date.
+func generateBackupName(app App, prefix string) string {
+	appName := inflector.Snakecase(app.Settings().Meta.AppName)
+	if len(appName) > 50 {
+		appName = appName[:50]
+	}
+
+	return fmt.Sprintf(
+		"%s%s_%s.zip",
+		prefix,
+		appName,
+		time.Now().UTC().Format("20060102150405"),
+	)
+}
+
+// registerAutobackupHooks registers the autobackup app serve hooks.
+func (app *BaseApp) registerAutobackupHooks() {
+	const jobId = "__pbAutoBackup__"
+
+	loadJob := func() {
+		rawSchedule := app.Settings().Backups.Cron
+		if rawSchedule == "" {
+			app.Cron().Remove(jobId)
+			return
+		}
+
+		app.Cron().Add(jobId, rawSchedule, func() {
+			const autoPrefix = "@auto_pb_backup_"
+
+			name := generateBackupName(app, autoPrefix)
+
+			if err := app.CreateBackup(context.Background(), name); err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to create backup",
+					slog.String("name", name),
+					slog.String("error", err.Error()),
+				)
+
+				alertError := sendSystemAlertToAllSuperusers(
+					app,
+					"Autobackup failure",
+					"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
+				)
+				if alertError != nil {
+					app.Logger().Warn(
+						"[Backup cron] Failed to send backup error alerts",
+						slog.String("name", name),
+						slog.String("error", alertError.Error()),
+					)
+				}
+			}
+
+			maxKeep := app.Settings().Backups.CronMaxKeep
+
+			if maxKeep == 0 {
+				return // no explicit limit
+			}
+
+			fsys, err := app.NewBackupsFilesystem()
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to initialize the backup filesystem",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+			defer fsys.Close()
+
+			files, err := fsys.List(autoPrefix)
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to list autogenerated backups",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+
+			if maxKeep >= len(files) {
+				return // nothing to remove
+			}
+
+			// sort desc
+			sort.Slice(files, func(i, j int) bool {
+				return files[i].ModTime.After(files[j].ModTime)
+			})
+
+			// keep only the most recent n auto backup files
+			toRemove := files[maxKeep:]
+
+			for _, f := range toRemove {
+				if err := fsys.Delete(f.Key); err != nil {
+					app.Logger().Error(
+						"[Backup cron] Failed to remove old autogenerated backup",
+						slog.String("key", f.Key),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+		})
+	}
+
+	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+
+	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+}

core/backup_create.go

@@ -0,0 +1,437 @@
+package core
+
+import (
+	"archive/zip"
+	"compress/flate"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"log/slog"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
+	"github.com/pocketbase/pocketbase/tools/hook"
+	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/store"
+)
+
+var errIsDir = errors.New("the specified path is a directory and not a regular file")
+
+// CreateBackup creates a new backup of the current app pb_data directory.
+//
+// If name is empty, it will be autogenerated.
+// If backup with the same name exists, the new backup file will replace it.
+//
+// To safely perform the backup, it is recommended to have free disk space
+// for at least 2x the size of the pb_data directory.
+//
+// By default backups are stored in pb_data/backups
+// (the backups directory itself is excluded from the generated backup).
+//
+// Backups can be stored on S3 if it is configured in app.Settings().Backups.
+// When using S3 storage for the uploaded collection files, you have to
+// take care manually to backup those since they are not part of the pb_data.
+//
+// DB write locks are minimal and isolated only for the duration of the
+// VACUUM INTO statement that creates a live copy of the app database.
+//
+// The backup works as follow:
+//
+//  1. Start listening for DELETED storage files.
+//     In case a file is being deleted while the backup is still ongoing,
+//     we directly copy it in the zip before the deletion and mark it as "excluded".
+//
+//  2. Copy the main database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  3. Stop listening for DELETED files.
+//
+//  4. Start listening for NEW storage files and mark all new files from this point as "excluded".
+//
+//  5. Copy the logs database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  6. Copy the rest of the pb_data files in the zip while ignoring the "excluded" list (it should be concurrent safe).
+//
+//  7. Stop listening for NEW storage files.
+//
+// While there is a risk for a race condition between steps 1, 2 and 3, it is an
+// acceptable trade-off between performance and correctness because in
+// the worst case there will be some unused storage files in the backup that don't do any harm.
+func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
+	if app.Store().Has(StoreKeyActiveBackup) {
+		return errors.New("try again later - another backup/restore operation has already been started")
+	}
+
+	app.Store().Set(StoreKeyActiveBackup, name)
+	defer app.Store().Remove(StoreKeyActiveBackup)
+
+	startTime := time.Now()
+
+	event := new(BackupEvent)
+	event.App = app
+	event.Context = ctx
+	event.Name = name
+	// default root dir entries to exclude from the backup generation
+	event.Exclude = []string{
+		LocalBackupsDirName,
+		LocalTempDirName,
+		LocalNotifyDirName,
+		LocalAutocertCacheDirName,
+		lostFoundDirName,
+	}
+
+	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
+		if e.Name == "" {
+			e.Name = generateBackupName(e.App, "pb_backup_")
+		}
+
+		app.Logger().Debug("[" + e.Name + "] zip archive started")
+
+		// create backup zip
+		// (it needs to be inside the current pb_data to avoid "cross-device link" errors)
+		// -----------------------------------------------------------
+		tempZipPath := filepath.Join(app.DataDir(), LocalTempDirName, "pb_backup_"+security.PseudorandomString(6))
+		err := createZip(e, tempZipPath)
+		if err != nil {
+			return err
+		}
+		defer os.Remove(tempZipPath)
+
+		// persist the backup in the backups filesystem
+		// -----------------------------------------------------------
+		fsys, err := e.App.NewBackupsFilesystem()
+		if err != nil {
+			return err
+		}
+		defer fsys.Close()
+
+		fsys.SetContext(e.Context)
+
+		file, err := filesystem.NewFileFromPath(tempZipPath)
+		if err != nil {
+			return err
+		}
+		file.OriginalName = e.Name
+		file.Name = file.OriginalName
+
+		err = fsys.UploadFile(file, file.Name)
+		if err != nil {
+			return err
+		}
+
+		app.Logger().Debug(
+			"["+e.Name+"] zip archive completed",
+			slog.Float64("execTime", float64(time.Since(startTime))/float64(time.Millisecond)),
+		)
+
+		return nil
+	})
+}
+
+func createZip(be *BackupEvent, tempZipPath string) error {
+	logPrefix := "[" + be.Name + "] "
+
+	// make sure that the special temp directory exists
+	localTempDir := filepath.Dir(tempZipPath)
+	if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
+		return fmt.Errorf(logPrefix+"failed to create temp dir: %w", err)
+	}
+
+	const tempFilesHookId = "__pbTempBackupFilesystemWatcher__"
+	defer func() {
+		// unbind again in cacase of an error
+		be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+		be.App.onFilesystemNewWriter().Unbind(tempFilesHookId)
+	}()
+
+	zipper, err := newZipWriter(tempZipPath)
+	if err != nil {
+		return err
+	}
+	defer zipper.close()
+
+	excluded := store.New[string, struct{}](nil)
+	for _, name := range be.Exclude {
+		excluded.Set(normalizePathExclude(name), struct{}{})
+	}
+
+	// init deleted files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemDelete().Bind(&hook.Handler[*FilesystemDeleteEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemDeleteEvent) error {
+			// note: the zip header name allow only forward slashes
+			zipPath := path.Join(LocalStorageDirName, e.FileKey)
+
+			if excluded.Has(normalizePathExclude(zipPath)) || be.App.Settings().S3.Enabled {
+				return e.Next()
+			}
+
+			localPath := filepath.Join(
+				be.App.DataDir(),
+				LocalStorageDirName,
+				e.FileKey,
+			)
+
+			// copy to zip before delete
+			err := zipper.copyFileToZip(localPath, zipPath)
+			if err != nil {
+				// it is ok to ignore directories
+				if !errors.Is(err, errIsDir) {
+					be.App.Logger().Warn(
+						logPrefix+"failed to copy file in backup zip before delete",
+						slog.Any("error", err),
+						slog.String("file", e.FileKey),
+					)
+				}
+			} else {
+				// mark that it was already copied
+				excluded.Set(normalizePathExclude(zipPath), struct{}{})
+			}
+
+			// proceed with the normal deletion
+			return e.Next()
+		},
+	})
+
+	// copy data.db
+	// ---------------------------------------------------------------
+	dataStartTime := time.Now()
+	tempDataDBPath := filepath.Join(localTempDir, dataDBFilename)
+
+	_, err = be.App.ConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempDataDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	// eagerly stop listening for deleted files since we already have what we needed
+	be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+
+	be.App.Logger().Debug(
+		logPrefix+dataDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(dataStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempDataDBPath, dataDBFilename)
+	if err != nil {
+		_ = os.Remove(tempDataDBPath)
+		return err
+	}
+	_ = os.Remove(tempDataDBPath)
+
+	excluded.Set(normalizePathExclude(dataDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-shm"), struct{}{})
+
+	// init to-be-created files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemNewWriter().Bind(&hook.Handler[*FilesystemNewWriterEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemNewWriterEvent) error {
+			if !be.App.Settings().S3.Enabled {
+				// mark for exclude even if the writer eventually fails
+				// (all record files have random name so collisions are unlikely)
+				name := normalizePathExclude(filepath.Join(LocalStorageDirName, e.FileKey))
+				excluded.Set(name, struct{}{})
+			}
+
+			return e.Next()
+		},
+	})
+
+	// copy auxiliary.db
+	// ---------------------------------------------------------------
+	auxStartTime := time.Now()
+	tempAuxDBPath := filepath.Join(localTempDir, auxDBFilename)
+
+	_, err = be.App.AuxConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempAuxDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	be.App.Logger().Debug(
+		logPrefix+auxDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(auxStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempAuxDBPath, auxDBFilename)
+	if err != nil {
+		_ = os.Remove(tempAuxDBPath)
+		return err
+	}
+	_ = os.Remove(tempAuxDBPath)
+
+	excluded.Set(normalizePathExclude(auxDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-shm"), struct{}{})
+
+	// try to run manual checkpoints to ensure that all wal writes during the
+	// previous VACUUM INTO are transferred and don't accumulate
+	// (errors are ignore because some drivers may not support the wal_checkpoint pragma)
+	// ---------------------------------------------------------------
+	_, _ = be.App.NonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+	_, _ = be.App.AuxNonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+
+	// copy the rest of the pb_data
+	// ---------------------------------------------------------------
+	err = zipper.copyDirToZip(os.DirFS(be.App.DataDir()), excluded)
+	if err != nil {
+		return err
+	}
+
+	return zipper.close()
+}
+
+// normalize the provided file path to always end with forward slash
+func normalizePathExclude(filePath string) string {
+	return path.Clean(filePath) + "/"
+}
+
+type zipWriter struct {
+	mu     sync.Mutex
+	w      *zip.Writer
+	f      *os.File
+	closed bool
+}
+
+func newZipWriter(zipFilePath string) (*zipWriter, error) {
+	f, err := os.Create(zipFilePath)
+	if err != nil {
+		return nil, err
+	}
+
+	w := zip.NewWriter(f)
+	w.RegisterCompressor(zip.Deflate, func(out io.Writer) (io.WriteCloser, error) {
+		return flate.NewWriter(out, flate.BestSpeed)
+	})
+
+	return &zipWriter{
+		w: w,
+		f: f,
+	}, nil
+}
+
+func (z *zipWriter) close() error {
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return nil
+	}
+
+	z.closed = true
+
+	return errors.Join(z.w.Close(), z.f.Close())
+}
+
+func (z *zipWriter) copyFileToZip(localPath string, zipPath string) error {
+	info, err := os.Stat(localPath)
+	if err != nil {
+		return err
+	}
+
+	if info.IsDir() {
+		return errIsDir
+	}
+
+	h, err := zip.FileInfoHeader(info)
+	if err != nil {
+		return err
+	}
+
+	h.Name = zipPath
+	h.Method = zip.Deflate
+
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return errors.New("zip writer is already closed")
+	}
+
+	fw, err := z.w.CreateHeader(h)
+	if err != nil {
+		return err
+	}
+
+	f, err := os.Open(localPath)
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+
+	_, err = io.Copy(fw, f)
+
+	return err
+}
+
+func (z *zipWriter) copyDirToZip(fsys fs.FS, excludedPrefixes *store.Store[string, struct{}]) error {
+	return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+
+		// skip excluded prefixes
+		if excludedPrefixes != nil {
+			check := normalizePathExclude(name)
+			prefixes := excludedPrefixes.Keys() // refetch to avoid races
+			for _, prefix := range prefixes {
+				if strings.HasPrefix(check, prefix) {
+					if d.IsDir() {
+						return fs.SkipDir
+					}
+					return nil
+				}
+			}
+		}
+
+		if d.IsDir() {
+			return nil
+		}
+
+		info, err := d.Info()
+		if err != nil {
+			return err
+		}
+
+		h, err := zip.FileInfoHeader(info)
+		if err != nil {
+			return err
+		}
+
+		h.Name = name
+		h.Method = zip.Deflate
+
+		z.mu.Lock()
+		defer z.mu.Unlock()
+
+		if z.closed {
+			// note: fs.WalkDir perform direct comparison with the value
+			return fs.SkipAll
+		}
+
+		fw, err := z.w.CreateHeader(h)
+		if err != nil {
+			return err
+		}
+
+		f, err := fsys.Open(name)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+
+		_, err = io.Copy(fw, f)
+
+		return err
+	})
+}

core/backup_restore.go

@@ -9,117 +9,12 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
-	"sort"
-	"time"
 
 	"github.com/pocketbase/pocketbase/tools/archive"
-	"github.com/pocketbase/pocketbase/tools/filesystem"
-	"github.com/pocketbase/pocketbase/tools/inflector"
 	"github.com/pocketbase/pocketbase/tools/osutils"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
 
-const (
-	StoreKeyActiveBackup = "@activeBackup"
-)
-
-// CreateBackup creates a new backup of the current app pb_data directory.
-//
-// If name is empty, it will be autogenerated.
-// If backup with the same name exists, the new backup file will replace it.
-//
-// The backup is executed within a transaction, meaning that new writes
-// will be temporary "blocked" until the backup file is generated.
-//
-// To safely perform the backup, it is recommended to have free disk space
-// for at least 2x the size of the pb_data directory.
-//
-// By default backups are stored in pb_data/backups
-// (the backups directory itself is excluded from the generated backup).
-//
-// When using S3 storage for the uploaded collection files, you have to
-// take care manually to backup those since they are not part of the pb_data.
-//
-// Backups can be stored on S3 if it is configured in app.Settings().Backups.
-func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
-	if app.Store().Has(StoreKeyActiveBackup) {
-		return errors.New("try again later - another backup/restore operation has already been started")
-	}
-
-	app.Store().Set(StoreKeyActiveBackup, name)
-	defer app.Store().Remove(StoreKeyActiveBackup)
-
-	event := new(BackupEvent)
-	event.App = app
-	event.Context = ctx
-	event.Name = name
-	// default root dir entries to exclude from the backup generation
-	event.Exclude = []string{
-		LocalBackupsDirName,
-		LocalTempDirName,
-		LocalNotifyDirName,
-		LocalAutocertCacheDirName,
-		lostFoundDirName,
-	}
-
-	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
-		// generate a default name if missing
-		if e.Name == "" {
-			e.Name = generateBackupName(e.App, "pb_backup_")
-		}
-
-		// make sure that the special temp directory exists
-		// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
-		localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
-		if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
-			return fmt.Errorf("failed to create a temp dir: %w", err)
-		}
-
-		// archive pb_data in a temp directory, excluding the "backups" and the temp dirs
-		//
-		// run in transaction to temporary block other writes (transactions uses the NonconcurrentDB connection)
-		// ---
-		tempPath := filepath.Join(localTempDir, "pb_backup_"+security.PseudorandomString(6))
-		createErr := e.App.RunInTransaction(func(txApp App) error {
-			return txApp.AuxRunInTransaction(func(txApp App) error {
-				// run manual checkpoint and truncate the WAL files
-				// (errors are ignored because it is not that important and the PRAGMA may not be supported by the used driver)
-				txApp.DB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-				txApp.AuxDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-
-				return archive.Create(txApp.DataDir(), tempPath, e.Exclude...)
-			})
-		})
-		if createErr != nil {
-			return createErr
-		}
-		defer os.Remove(tempPath)
-
-		// persist the backup in the backups filesystem
-		// ---
-		fsys, err := e.App.NewBackupsFilesystem()
-		if err != nil {
-			return err
-		}
-		defer fsys.Close()
-
-		fsys.SetContext(e.Context)
-
-		file, err := filesystem.NewFileFromPath(tempPath)
-		if err != nil {
-			return err
-		}
-		file.OriginalName = e.Name
-		file.Name = file.OriginalName
-
-		if err := fsys.UploadFile(file, file.Name); err != nil {
-			return err
-		}
-
-		return nil
-	})
-}
-
 // RestoreBackup restores the backup with the specified name and restarts
 // the current running application process.
 //
@@ -299,124 +194,3 @@ func (app *BaseApp) RestoreBackup(ctx context.Context, name string) error {
 		return nil
 	})
 }
-
-// registerAutobackupHooks registers the autobackup app serve hooks.
-func (app *BaseApp) registerAutobackupHooks() {
-	const jobId = "__pbAutoBackup__"
-
-	loadJob := func() {
-		rawSchedule := app.Settings().Backups.Cron
-		if rawSchedule == "" {
-			app.Cron().Remove(jobId)
-			return
-		}
-
-		app.Cron().Add(jobId, rawSchedule, func() {
-			const autoPrefix = "@auto_pb_backup_"
-
-			name := generateBackupName(app, autoPrefix)
-
-			if err := app.CreateBackup(context.Background(), name); err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to create backup",
-					slog.String("name", name),
-					slog.String("error", err.Error()),
-				)
-
-				alertError := sendSystemAlertToAllSuperusers(
-					app,
-					"Autobackup failure",
-					"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
-				)
-				if alertError != nil {
-					app.Logger().Warn(
-						"[Backup cron] Failed to send backup error alerts",
-						slog.String("name", name),
-						slog.String("error", alertError.Error()),
-					)
-				}
-			}
-
-			maxKeep := app.Settings().Backups.CronMaxKeep
-
-			if maxKeep == 0 {
-				return // no explicit limit
-			}
-
-			fsys, err := app.NewBackupsFilesystem()
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to initialize the backup filesystem",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-			defer fsys.Close()
-
-			files, err := fsys.List(autoPrefix)
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to list autogenerated backups",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-
-			if maxKeep >= len(files) {
-				return // nothing to remove
-			}
-
-			// sort desc
-			sort.Slice(files, func(i, j int) bool {
-				return files[i].ModTime.After(files[j].ModTime)
-			})
-
-			// keep only the most recent n auto backup files
-			toRemove := files[maxKeep:]
-
-			for _, f := range toRemove {
-				if err := fsys.Delete(f.Key); err != nil {
-					app.Logger().Error(
-						"[Backup cron] Failed to remove old autogenerated backup",
-						slog.String("key", f.Key),
-						slog.String("error", err.Error()),
-					)
-				}
-			}
-		})
-	}
-
-	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-
-	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-}
-
-func generateBackupName(app App, prefix string) string {
-	appName := inflector.Snakecase(app.Settings().Meta.AppName)
-	if len(appName) > 50 {
-		appName = appName[:50]
-	}
-
-	return fmt.Sprintf(
-		"%s%s_%s.zip",
-		prefix,
-		appName,
-		time.Now().UTC().Format("20060102150405"),
-	)
-}

core/backup_test.go

@@ -126,11 +126,7 @@ func verifyBackupContent(app core.App, path string) error {
 	expectedRootEntries := []string{
 		"storage",
 		"data.db",
-		"data.db-shm",
-		"data.db-wal",
 		"auxiliary.db",
-		"auxiliary.db-shm",
-		"auxiliary.db-wal",
 		".gitignore",
 	}
 

core/base.go

@@ -12,6 +12,7 @@ import (
 	"regexp"
 	"runtime"
 	"strings"
+	"sync/atomic"
 	"time"
 
 	"github.com/fatih/color"
@@ -44,6 +45,9 @@ const (
 
 	// @todo consider removing after backups refactoring
 	lostFoundDirName string = "lost+found"
+
+	dataDBFilename string = "data.db"
+	auxDBFilename  string = "auxiliary.db"
 )
 
 // FilesManager defines an interface with common methods that files manager models should implement.
@@ -86,11 +90,12 @@ type BaseApp struct {
 	auxNonconcurrentDB  dbx.Builder
 
 	// app event hooks
-	onBootstrap     *hook.Hook[*BootstrapEvent]
-	onServe         *hook.Hook[*ServeEvent]
-	onTerminate     *hook.Hook[*TerminateEvent]
-	onBackupCreate  *hook.Hook[*BackupEvent]
-	onBackupRestore *hook.Hook[*BackupEvent]
+	onBootstrap      *hook.Hook[*BootstrapEvent]
+	onBootstrapClear *hook.Hook[*BootstrapEvent]
+	onServe          *hook.Hook[*ServeEvent]
+	onTerminate      *hook.Hook[*TerminateEvent]
+	onBackupCreate   *hook.Hook[*BackupEvent]
+	onBackupRestore  *hook.Hook[*BackupEvent]
 
 	// db model hooks
 	onModelValidate           *hook.Hook[*ModelEvent]
@@ -146,6 +151,17 @@ type BaseApp struct {
 	onMailerRecordOTPSend           *hook.Hook[*MailerRecordEvent]
 	onMailerRecordAuthAlertSend     *hook.Hook[*MailerRecordEvent]
 
+	// filesystem event hooks
+	//
+	// @todo 1:
+	// intentionally not exposed since the events are too "chatty" and
+	// can cause unnecessary userland tests breaking changes;
+	// reevaluate once refactoring the file_field
+	//
+	// @todo 2: if exposed consider registering the same for the backup filesystem
+	_onFilesystemNewWriter *hook.Hook[*FilesystemNewWriterEvent]
+	_onFilesystemDelete    *hook.Hook[*FilesystemDeleteEvent]
+
 	// realtime api event hooks
 	onRealtimeConnectRequest   *hook.Hook[*RealtimeConnectRequestEvent]
 	onRealtimeMessageSend      *hook.Hook[*RealtimeMessageEvent]
@@ -235,6 +251,7 @@ func NewBaseApp(config BaseAppConfig) *BaseApp {
 func (app *BaseApp) initHooks() {
 	// app event hooks
 	app.onBootstrap = &hook.Hook[*BootstrapEvent]{}
+	app.onBootstrapClear = &hook.Hook[*BootstrapEvent]{}
 	app.onServe = &hook.Hook[*ServeEvent]{}
 	app.onTerminate = &hook.Hook[*TerminateEvent]{}
 	app.onBackupCreate = &hook.Hook[*BackupEvent]{}
@@ -294,6 +311,10 @@ func (app *BaseApp) initHooks() {
 	app.onMailerRecordOTPSend = &hook.Hook[*MailerRecordEvent]{}
 	app.onMailerRecordAuthAlertSend = &hook.Hook[*MailerRecordEvent]{}
 
+	// filesystem event hooks
+	app._onFilesystemNewWriter = &hook.Hook[*FilesystemNewWriterEvent]{}
+	app._onFilesystemDelete = &hook.Hook[*FilesystemDeleteEvent]{}
+
 	// realtime API event hooks
 	app.onRealtimeConnectRequest = &hook.Hook[*RealtimeConnectRequestEvent]{}
 	app.onRealtimeMessageSend = &hook.Hook[*RealtimeMessageEvent]{}
@@ -387,14 +408,14 @@ func (app *BaseApp) IsBootstrapped() bool {
 // Bootstrap initializes the application
 // (aka. create data dir, open db connections, load settings, etc.).
 //
-// It will call ResetBootstrapState() if the application was already bootstrapped.
+// It calls ClearBootstrap() if the application was already bootstrapped.
 func (app *BaseApp) Bootstrap() error {
 	event := &BootstrapEvent{}
 	event.App = app
 
 	err := app.OnBootstrap().Trigger(event, func(e *BootstrapEvent) error {
-		// clear resources of previous core state (if any)
-		if err := app.ResetBootstrapState(); err != nil {
+		// clear previous bootstrap state (if any)
+		if err := app.ClearBootstrap(); err != nil {
 			return err
 		}
 
@@ -442,41 +463,55 @@ func (app *BaseApp) Bootstrap() error {
 	return err
 }
 
-type closer interface {
-	Close() error
+// Deprecated: use [ClearBootstrap].
+func (app *BaseApp) ResetBootstrapState() error {
+	return app.ClearBootstrap()
 }
 
-// ResetBootstrapState releases the initialized core app resources
+// ClearBootstrap releases the initialized core app resources
 // (closing db connections, stopping cron ticker, etc.).
-func (app *BaseApp) ResetBootstrapState() error {
-	app.Cron().Stop()
-
-	var errs []error
-
-	dbs := []*dbx.Builder{
-		&app.concurrentDB,
-		&app.nonconcurrentDB,
-		&app.auxConcurrentDB,
-		&app.auxNonconcurrentDB,
+//
+// This method is no-op if the application is not bootstrapped yet.
+func (app *BaseApp) ClearBootstrap() error {
+	if !app.IsBootstrapped() {
+		return nil
 	}
 
-	for _, db := range dbs {
-		if db == nil {
-			continue
+	event := &BootstrapEvent{}
+	event.App = app
+
+	return app.OnBootstrapClear().Trigger(event, func(e *BootstrapEvent) error {
+		type closer interface {
+			Close() error
 		}
-		if v, ok := (*db).(closer); ok {
-			if err := v.Close(); err != nil {
-				errs = append(errs, err)
+
+		var errs []error
+
+		dbs := []*dbx.Builder{
+			&app.concurrentDB,
+			&app.nonconcurrentDB,
+			&app.auxConcurrentDB,
+			&app.auxNonconcurrentDB,
+		}
+
+		for _, db := range dbs {
+			if db == nil {
+				continue
 			}
+			if v, ok := (*db).(closer); ok {
+				if err := v.Close(); err != nil {
+					errs = append(errs, err)
+				}
+			}
+			*db = nil
 		}
-		*db = nil
-	}
 
-	if len(errs) > 0 {
-		return errors.Join(errs...)
-	}
+		if len(errs) > 0 {
+			return errors.Join(errs...)
+		}
 
-	return nil
+		return nil
+	})
 }
 
 // DB returns the default app data.db builder instance.
@@ -712,9 +747,10 @@ func (app *BaseApp) NewMailClient() mailer.Mailer {
 //
 // NB! Make sure to call Close() on the returned result
 // after you are done working with it.
-func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
+func (app *BaseApp) NewFilesystem() (fsys *filesystem.System, err error) {
 	if app.settings != nil && app.settings.S3.Enabled {
-		return filesystem.NewS3(
+		// S3
+		fsys, err = filesystem.NewS3(
 			app.settings.S3.Bucket,
 			app.settings.S3.Region,
 			app.settings.S3.Endpoint,
@@ -722,10 +758,41 @@ func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
 			app.settings.S3.Secret,
 			app.settings.S3.ForcePathStyle,
 		)
+	} else {
+		// local filesystem
+		fsys, err = filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	}
+	if err != nil {
+		return nil, err
 	}
 
-	// fallback to local filesystem
-	return filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	// attach delete hook
+	if app._onFilesystemDelete.Length() > 0 {
+		fsys.OnDelete().BindFunc(func(originalEvent *filesystem.DeleteEvent) error {
+			appEvent := new(FilesystemDeleteEvent)
+			appEvent.DeleteEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemDelete.Trigger(appEvent, func(fde *FilesystemDeleteEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	// attach write hook
+	if app._onFilesystemNewWriter.Length() > 0 {
+		fsys.OnNewWriter().BindFunc(func(originalEvent *filesystem.NewWriterEvent) error {
+			appEvent := new(FilesystemNewWriterEvent)
+			appEvent.NewWriterEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemNewWriter.Trigger(appEvent, func(fwe *FilesystemNewWriterEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	return fsys, nil
 }
 
 // NewBackupsFilesystem creates a new local or S3 filesystem instance
@@ -767,7 +834,7 @@ func (app *BaseApp) Restart() error {
 	event.IsRestart = true
 
 	return app.OnTerminate().Trigger(event, func(e *TerminateEvent) error {
-		_ = e.App.ResetBootstrapState()
+		_ = e.App.ClearBootstrap()
 
 		// attempt to restart the bootstrap process in case execve returns an error for some reason
 		defer func() {
@@ -810,6 +877,10 @@ func (app *BaseApp) OnBootstrap() *hook.Hook[*BootstrapEvent] {
 	return app.onBootstrap
 }
 
+func (app *BaseApp) OnBootstrapClear() *hook.Hook[*BootstrapEvent] {
+	return app.onBootstrapClear
+}
+
 func (app *BaseApp) OnServe() *hook.Hook[*ServeEvent] {
 	return app.onServe
 }
@@ -1016,6 +1087,18 @@ func (app *BaseApp) OnMailerRecordAuthAlertSend(tags ...string) *hook.TaggedHook
 	return hook.NewTaggedHook(app.onMailerRecordAuthAlertSend, tags...)
 }
 
+// -------------------------------------------------------------------
+// Filesystem event hooks
+// -------------------------------------------------------------------
+
+func (app *BaseApp) onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent] {
+	return app._onFilesystemNewWriter
+}
+
+func (app *BaseApp) onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent] {
+	return app._onFilesystemDelete
+}
+
 // -------------------------------------------------------------------
 // Realtime API event hooks
 // -------------------------------------------------------------------
@@ -1173,7 +1256,7 @@ func (app *BaseApp) OnBatchRequest() *hook.Hook[*BatchRequestEvent] {
 // -------------------------------------------------------------------
 
 func (app *BaseApp) initDataDB() error {
-	dbPath := filepath.Join(app.DataDir(), "data.db")
+	dbPath := filepath.Join(app.DataDir(), dataDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1235,7 +1318,7 @@ func normalizeSQLLog(sql string) string {
 func (app *BaseApp) initAuxDB() error {
 	// note: renamed to "auxiliary" because "aux" is a reserved Windows filename
 	// (see https://github.com/pocketbase/pocketbase/issues/5607)
-	dbPath := filepath.Join(app.DataDir(), "auxiliary.db")
+	dbPath := filepath.Join(app.DataDir(), auxDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1351,7 +1434,15 @@ func (app *BaseApp) registerBaseHooks() {
 		Id: "__pbCronStart__",
 		Func: func(e *ServeEvent) error {
 			app.Cron().Start()
+			return e.Next()
+		},
+		Priority: 999,
+	})
 
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbCronStop__",
+		Func: func(e *BootstrapEvent) error {
+			app.Cron().Stop()
 			return e.Next()
 		},
 		Priority: 999,
@@ -1408,9 +1499,41 @@ func getLoggerMinLevel(app App) slog.Level {
 }
 
 func (app *BaseApp) initLogger() error {
+	var stopped atomic.Bool
+
 	duration := 3 * time.Second
 	ticker := time.NewTicker(duration)
-	done := make(chan bool, 1)
+
+	done := make(chan struct{}, 1)
+
+	runLogsWrite := func(logs []*logger.Log) {
+		if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
+			return
+		}
+
+		// write the accumulated logs
+		//
+		// note: based on several local tests there is no
+		// significant performance difference between small number
+		// of separate write queries vs 1 big INSERT
+		app.AuxRunInTransaction(func(txApp App) error {
+			model := &Log{}
+			for _, l := range logs {
+				model.MarkAsNew()
+				model.Id = GenerateDefaultRandomId()
+				model.Level = int(l.Level)
+				model.Message = l.Message
+				model.Data = l.Data
+				model.Created, _ = types.ParseDateTime(l.Time)
+
+				if err := txApp.AuxSave(model); err != nil {
+					log.Println("Failed to write log", model, err)
+				}
+			}
+
+			return nil
+		})
+	}
 
 	handler := logger.NewBatchHandler(logger.BatchOptions{
 		Level:     getLoggerMinLevel(app),
@@ -1425,40 +1548,30 @@ func (app *BaseApp) initLogger() error {
 				}
 			}
 
-			ticker.Reset(duration)
+			if !stopped.Load() {
+				ticker.Reset(duration)
+			}
 
 			return app.Settings().Logs.MaxDays > 0
 		},
 		WriteFunc: func(ctx context.Context, logs []*logger.Log) error {
-			if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
-				return nil
+			// don't block and wait for the write transaction to complete
+			// when we can't be sure if the logs write wasn't triggered while
+			// inside another AUX db transaction (ticker or batch threshold reached)
+			// which can block indefinitely and cause deadlock
+			// (https://github.com/pocketbase/pocketbase/issues/7836)
+			shouldBlock, _ := ctx.Value(logger.BlockKey).(bool)
+			if shouldBlock {
+				runLogsWrite(logs)
+			} else {
+				routine.FireAndForget(func() { runLogsWrite(logs) })
 			}
 
-			// write the accumulated logs
-			// (note: based on several local tests there is no significant performance difference between small number of separate write queries vs 1 big INSERT)
-			app.AuxRunInTransaction(func(txApp App) error {
-				model := &Log{}
-				for _, l := range logs {
-					model.MarkAsNew()
-					model.Id = GenerateDefaultRandomId()
-					model.Level = int(l.Level)
-					model.Message = l.Message
-					model.Data = l.Data
-					model.Created, _ = types.ParseDateTime(l.Time)
-
-					if err := txApp.AuxSave(model); err != nil {
-						log.Println("Failed to write log", model, err)
-					}
-				}
-
-				return nil
-			})
-
 			return nil
 		},
 	})
 
-	go func() {
+	routine.FireAndForget(func() {
 		ctx := context.Background()
 
 		for {
@@ -1469,21 +1582,25 @@ func (app *BaseApp) initLogger() error {
 				handler.WriteAll(ctx)
 			}
 		}
-	}()
+	})
 
 	app.logger = slog.New(handler)
 
-	// write all remaining logs before ticker.Stop to avoid races with ResetBootstrap user calls
-	app.OnTerminate().Bind(&hook.Handler[*TerminateEvent]{
-		Id: "__pbAppLoggerOnTerminate__",
-		Func: func(e *TerminateEvent) error {
-			handler.WriteAll(context.Background())
+	// attempt to write all queued logs before clearing the application bootstrap state
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbAppLoggerFlushBeforeStop__",
+		Func: func(e *BootstrapEvent) error {
+			// extra precaution in case the hook was manually triggered while inside aux db transaction
+			_, isTx := e.App.AuxNonconcurrentDB().(*dbx.Tx)
+			ctx := context.WithValue(context.Background(), logger.BlockKey, !isTx)
+			handler.WriteAll(ctx)
 
+			stopped.Store(true)
 			ticker.Stop()
 
-			// don't block in case OnTerminate is triggered more than once
+			// don't block in case the hook is triggered more than once
 			select {
-			case done <- true:
+			case done <- struct{}{}:
 			default:
 			}
 

core/base_test.go

@@ -7,6 +7,7 @@ import (
 	"os"
 	"slices"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	_ "unsafe"
@@ -64,7 +65,7 @@ func TestBaseAppBootstrap(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if app.IsBootstrapped() {
 		t.Fatal("Didn't expect the application to be bootstrapped.")
@@ -114,7 +115,7 @@ func TestBaseAppBootstrap(t *testing.T) {
 	runNilChecks(nilChecksBeforeReset)
 
 	// reset
-	if err := app.ResetBootstrapState(); err != nil {
+	if err := app.ClearBootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
@@ -140,7 +141,7 @@ func TestNewBaseAppTx(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if err := app.Bootstrap(); err != nil {
 		t.Fatal(err)
@@ -184,7 +185,7 @@ func TestBaseAppNewMailClient(t *testing.T) {
 		DataDir:       testDataDir,
 		EncryptionEnv: "pb_test_env",
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	client1 := app.NewMailClient()
 	m1, ok := client1.(*mailer.Sendmail)
@@ -214,7 +215,7 @@ func TestBaseAppNewFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewFilesystem()
@@ -243,7 +244,7 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewBackupsFilesystem()
@@ -265,68 +266,172 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	}
 }
 
+const logsThreshold = 200
+
+func assertLogsCount(t *testing.T, app core.App, expected int) {
+	var total int
+
+	err := app.LogQuery().Select("count(*)").Row(&total)
+	if err != nil {
+		t.Fatalf("Failed to fetch total logs: %v", err)
+	}
+
+	if total != expected {
+		t.Fatalf("Expected %d log(s), got %d", expected, total)
+	}
+}
+
 func TestBaseAppLoggerWrites(t *testing.T) {
 	t.Parallel()
 
+	// note: outside of synctest because the bootstrap tickers could deadlock
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Run("disabled logs retention", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 0
+
+			for i := 0; i < logsThreshold+1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, 0)
+		})
+	})
+
+	t.Run("test batch logs writes", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 2
+
+			for i := 0; i < logsThreshold-1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			// below threshold
+			assertLogsCount(t, app, 0)
+
+			// threshold reached -> should trigger batch write
+			app.Logger().Error("test")
+
+			// should be skipped from this batch and added for the next
+			app.Logger().Error("test")
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, logsThreshold)
+
+			// note: we can't test the flush timer here because the ticker
+			// was started out of the synctest buble to avoid deadlocks
+			// (see TestBaseAppLoggerWritesAwaited for a flaky but real timer test)
+		})
+	})
+}
+
+func TestBaseAppLoggerWritesAwaited(t *testing.T) {
+	t.Parallel()
+
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
 
-	// reset
-	if err := app.DeleteOldLogs(time.Now()); err != nil {
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
 		t.Fatal(err)
 	}
 
-	const logsThreshold = 200
-
-	totalLogs := func(app core.App, t *testing.T) int {
-		var total int
-
-		err := app.LogQuery().Select("count(*)").Row(&total)
-		if err != nil {
-			t.Fatalf("Failed to fetch total logs: %v", err)
-		}
-
-		return total
+	// enable logs persistence
+	app.Settings().Logs.MaxDays = 1
+	err = app.Save(app.Settings())
+	if err != nil {
+		t.Fatal(err)
 	}
 
-	t.Run("disabled logs retention", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 0
+	t.Run("flush on timer tick", func(t *testing.T) {
+		timeout := time.After(5 * time.Second)
+		done := make(chan struct{})
 
-		for i := 0; i < logsThreshold+1; i++ {
-			app.Logger().Error("test")
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			done <- struct{}{}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.Logger().Error("test")
+
+		// short wait to ensure that there is no non-blocking write
+		time.Sleep(500 * time.Millisecond)
+
+		assertLogsCount(t, app, 0)
+
+		// wait for the ticker to write the db record
+		select {
+		case <-timeout:
+			t.Fatal("ticker wait timeout")
+		case <-done:
 		}
 
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
+		assertLogsCount(t, app, 1)
 	})
 
-	t.Run("test batch logs writes", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 1
-
-		for i := 0; i < logsThreshold-1; i++ {
-			app.Logger().Error("test")
-		}
-
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
-
-		// should trigger batch write
+	t.Run("before ClearBootstrap flush", func(t *testing.T) {
 		app.Logger().Error("test")
 
-		// should be added for the next batch write
-		app.Logger().Error("test")
+		app.Bootstrap()
 
-		if total := totalLogs(app, t); total != logsThreshold {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold, total)
+		assertLogsCount(t, app, 2)
+	})
+
+	t.Run("batch flush inside aux transaction shouldn't hang", func(t *testing.T) {
+		timeout := time.After(1 * time.Second)
+		done := make(chan struct{})
+		totalCreated := 0
+
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			totalCreated++
+			if totalCreated == 200 {
+				done <- struct{}{}
+			}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.AuxRunInTransaction(func(txApp core.App) error {
+			for range logsThreshold {
+				txApp.Logger().Error("test")
+			}
+
+			return nil
+		})
+
+		// wait for the non-blocking write
+		select {
+		case <-timeout:
+			t.Fatal("non-blocking write timeout")
+		case <-done:
 		}
 
-		// wait for ~3 secs to check the timer trigger
-		time.Sleep(3200 * time.Millisecond)
-		if total := totalLogs(app, t); total != logsThreshold+1 {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold+1, total)
-		}
+		assertLogsCount(t, app, 202)
+
+		// force clear to ensure that there are no other logs
+		app.Bootstrap()
+
+		assertLogsCount(t, app, 202)
 	})
 }
 
@@ -369,7 +474,7 @@ func TestBaseAppRefreshSettingsLoggerMinLevelEnabled(t *testing.T) {
 				DataDir: testDataDir,
 				IsDev:   s.isDev,
 			})
-			defer app.ResetBootstrapState()
+			defer app.ClearBootstrap()
 
 			if err := app.Bootstrap(); err != nil {
 				t.Fatal(err)

core/collection_import.go

@@ -4,12 +4,13 @@ import (
 	"cmp"
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/jsontext"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/spf13/cast"
 )
 
@@ -186,7 +187,11 @@ func (app *BaseApp) ImportCollections(toImport []map[string]any, deleteMissing b
 			)
 			if err := validator.run(); err != nil {
 				// serialize the validation error(s)
-				serializedErr, _ := json.MarshalIndent(err, "", "  ")
+				serializedErr, _ := json.Marshal(
+					err,
+					jsontext.WithIndentPrefix(""),
+					jsontext.WithIndent("  "),
+				)
 
 				return validation.Errors{"collections": validation.NewError(
 					"validation_collections_import_failure",

core/collection_import_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"strings"
 	"testing"
 
@@ -449,7 +449,7 @@ func TestImportCollectionsCreateRules(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	raw, err := json.Marshal(collection)
+	raw, err := json.Marshal(collection, json.Deterministic(true))
 	if err != nil {
 		t.Fatal(err)
 	}

core/collection_model.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strconv"
 	"strings"
@@ -522,8 +522,6 @@ func (m *Collection) unmarshalRawOptions() error {
 // For new/"blank" Collection models it replaces the model with a factory
 // instance and then unmarshal the provided data one on top of it.
 func (m *Collection) UnmarshalJSON(b []byte) error {
-	type alias *Collection
-
 	// initialize the default fields
 	// (e.g. in case the collection was NOT created using the designated factories)
 	if m.IsNew() && m.Type == "" {
@@ -540,7 +538,8 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
 		*m = *blank
 	}
 
-	return json.Unmarshal(b, alias(m))
+	type alias Collection
+	return json.Unmarshal(b, (*alias)(m))
 }
 
 // MarshalJSON implements the [json.Marshaler] interface.
@@ -550,10 +549,12 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
 func (m Collection) MarshalJSON() ([]byte, error) {
 	switch m.Type {
 	case CollectionTypeView:
-		return json.Marshal(struct {
+		alias := struct {
 			baseCollection
 			collectionViewOptions
-		}{m.baseCollection, m.collectionViewOptions})
+		}{m.baseCollection, m.collectionViewOptions}
+
+		return json.Marshal(alias, json.Deterministic(true))
 	case CollectionTypeAuth:
 		alias := struct {
 			baseCollection
@@ -582,15 +583,15 @@ func (m Collection) MarshalJSON() ([]byte, error) {
 			alias.OAuth2.Providers = redactedProviders
 		}
 
-		return json.Marshal(alias)
+		return json.Marshal(alias, json.Deterministic(true))
 	default:
-		return json.Marshal(m.baseCollection)
+		return json.Marshal(m.baseCollection, json.Deterministic(true))
 	}
 }
 
 // String returns a string representation of the current collection.
 func (m Collection) String() string {
-	raw, _ := json.Marshal(m)
+	raw, _ := m.MarshalJSON()
 	return string(raw)
 }
 
@@ -925,8 +926,14 @@ func onCollectionSaveExecute(e *CollectionEvent) error {
 	}
 
 	// trigger an update for all views with changed fields as a result of the current collection save
-	// (ignoring view errors to allow users to update the query from the UI)
-	resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	// (only log the error to allow users to adjust the problematic view queries from the UI)
+	depViewsErr := resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	if depViewsErr != nil {
+		e.App.Logger().Warn(
+			"Dependent view collection(s) may need to be updated after "+e.Collection.Name+" collection change",
+			"error", depViewsErr,
+		)
+	}
 
 	return nil
 }

core/collection_model_auth_options.go

@@ -1,12 +1,15 @@
 package core
 
 import (
+	"encoding/json/v2"
+	"errors"
+	"slices"
 	"strconv"
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/security"
@@ -409,6 +412,71 @@ type OAuth2Config struct {
 	Enabled bool `form:"enabled" json:"enabled"`
 }
 
+// UnmarshalJSON implements the [json.Unmarshaler] interface.
+//
+// The main difference from the standard unmarshalization is that
+// instead of replacing the entire providers config slice, we ensure
+// that partially submitted provider data (e.g. without clientSecret)
+// is merged on per config level based on the provider name
+// (https://github.com/pocketbase/pocketbase/issues/7815).
+func (c *OAuth2Config) UnmarshalJSON(b []byte) error {
+	originalProviders := slices.Clone(c.Providers)
+
+	type alias OAuth2Config
+	err := json.Unmarshal(b, (*alias)(c))
+	if err != nil {
+		return err
+	}
+
+	if len(c.Providers) == 0 {
+		return nil
+	}
+
+	// unmarshal again but this time into a plain array of objects
+	// so that we have only the submitted fields and no zero defaults
+	plain := struct {
+		Providers []map[string]any `json:"providers"`
+	}{}
+	err = json.Unmarshal(b, &plain)
+	if err != nil {
+		return err
+	}
+
+	// no providers were submitted
+	if len(plain.Providers) == 0 {
+		return nil
+	}
+
+	if len(c.Providers) != len(plain.Providers) {
+		return errors.New("the length of the plain unmarshalized providers and the ones from the config doesn't match")
+	}
+
+ProvidersMergeLoop:
+	for i, plain := range plain.Providers {
+		for _, original := range originalProviders {
+			if original.Name == plain["name"] {
+				raw, err := json.Marshal(plain)
+				if err != nil {
+					return err
+				}
+
+				// unmarshal the new plain data on top of the original one
+				err = json.Unmarshal(raw, &original)
+				if err != nil {
+					return err
+				}
+
+				// reassigne to the updated original
+				c.Providers[i] = original
+
+				continue ProvidersMergeLoop
+			}
+		}
+	}
+
+	return nil
+}
+
 // GetProviderConfig returns the first OAuth2ProviderConfig that matches the specified name.
 //
 // Returns false and zero config if no such provider is available in c.Providers.

core/collection_model_auth_options_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strings"
 	"testing"
@@ -712,6 +712,76 @@ func TestPasswordAuthConfigValidate(t *testing.T) {
 	}
 }
 
+func TestOAuth2ConfigUnmarshalJSON(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []struct {
+		name     string
+		newJSON  string
+		expected string
+	}{
+		{
+			"missing",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"}
+			}`,
+			`{"providers":[{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"b","clientId":"b_clientId","clientSecret":"b_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": []
+			}`,
+			`{"providers":[],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"non-empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": [
+					{"name": "c", "clientId": "c_clientId", "clientSecret": "c_clientSecret"},
+					{"name": "a", "displayName": "a_displayName"}
+				]
+			}`,
+			`{"providers":[{"pkce":null,"name":"c","clientId":"c_clientId","clientSecret":"c_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"a_displayName","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			config := core.OAuth2Config{
+				Enabled: false,
+				MappedFields: core.OAuth2KnownFields{
+					Name: "name_test",
+				},
+				Providers: []core.OAuth2ProviderConfig{
+					{Name: "a", ClientId: "a_clientId", ClientSecret: "a_clientSecret"},
+					{Name: "b", ClientId: "b_clientId", ClientSecret: "b_clientSecret"},
+				},
+			}
+
+			err := json.Unmarshal([]byte(s.newJSON), &config)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			raw, err := json.Marshal(config, json.Deterministic(true))
+			if err != nil {
+				t.Fatal(err)
+			}
+			rawStr := string(raw)
+
+			if rawStr != s.expected {
+				t.Fatalf("Expected OAuth2ProviderConfig\n%s\ngot\n%s", s.expected, rawStr)
+			}
+		})
+	}
+}
+
 func TestOAuth2ConfigGetProviderConfig(t *testing.T) {
 	scenarios := []struct {
 		name           string
@@ -1016,8 +1086,8 @@ func TestOAuth2ProviderConfigInitProvider(t *testing.T) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}
 
-			rawMeta, _ := json.Marshal(provider.Extra())
-			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra)
+			rawMeta, _ := json.Marshal(provider.Extra(), json.Deterministic(true))
+			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra, json.Deterministic(true))
 			if !bytes.Equal(rawMeta, expectedMeta) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}

core/collection_model_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -618,7 +618,7 @@ func TestCollectionUnmarshalJSON(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			rawResult, err := json.Marshal(collection)
+			rawResult, err := json.Marshal(collection, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -829,7 +829,7 @@ func TestCollectionDBExport(t *testing.T) {
 		},
 		{
 			core.CollectionTypeAuth,
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":null,"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":null},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":[],"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":[]},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 	}
 
@@ -860,7 +860,7 @@ func TestCollectionDBExport(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			raw, err := json.Marshal(result)
+			raw, err := json.Marshal(result, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}

core/collection_model_view_options.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 )
 
 var _ optionsValidator = (*collectionViewOptions)(nil)

core/collection_query.go

@@ -3,7 +3,7 @@ package core
 import (
 	"bytes"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -250,52 +250,6 @@ func (app *BaseApp) TruncateCollection(collection *Collection) error {
 
 // -------------------------------------------------------------------
 
-// saveViewCollection persists the provided View collection changes:
-//   - deletes the old related SQL view (if any)
-//   - creates a new SQL view with the latest newCollection.Options.Query
-//   - generates new feilds list  based on newCollection.Options.Query
-//   - updates newCollection.Fields based on the generated view table info and query
-//   - saves the newCollection
-//
-// This method returns an error if newCollection is not a "view".
-func saveViewCollection(app App, newCollection, oldCollection *Collection) error {
-	if !newCollection.IsView() {
-		return errors.New("not a view collection")
-	}
-
-	return app.RunInTransaction(func(txApp App) error {
-		query := newCollection.ViewQuery
-
-		// generate collection fields from the query
-		viewFields, err := txApp.CreateViewFields(query)
-		if err != nil {
-			return err
-		}
-
-		// delete old renamed view
-		if oldCollection != nil {
-			if err := txApp.DeleteView(oldCollection.Name); err != nil {
-				return err
-			}
-		}
-
-		// wrap view query if necessary
-		query, err = normalizeViewQueryId(txApp, query)
-		if err != nil {
-			return fmt.Errorf("failed to normalize view query id: %w", err)
-		}
-
-		// (re)create the view
-		if err := txApp.SaveView(newCollection.Name, query); err != nil {
-			return err
-		}
-
-		newCollection.Fields = viewFields
-
-		return txApp.Save(newCollection)
-	})
-}
-
 // normalizeViewQueryId wraps (if necessary) the provided view query
 // with a subselect to ensure that the id column is a text since
 // currently we don't support non-string model ids
@@ -342,50 +296,59 @@ func resaveViewsWithChangedFields(app App, excludeIds ...string) error {
 	}
 
 	return app.RunInTransaction(func(txApp App) error {
+		var collectionErrors []error
+
 		for _, collection := range collections {
 			if len(excludeIds) > 0 && list.ExistInSlice(collection.Id, excludeIds) {
 				continue
 			}
 
-			// clone the existing fields for temp modifications
-			oldFields, err := collection.Fields.Clone()
-			if err != nil {
-				return err
+			check := func() error {
+				// clone the existing fields for temp modifications
+				oldFields, err := collection.Fields.Clone()
+				if err != nil {
+					return err
+				}
+
+				// generate new fields from the query
+				newFields, err := txApp.CreateViewFields(collection.ViewQuery)
+				if err != nil {
+					return err
+				}
+
+				// unset the fields' ids to exclude from the comparison
+				for _, f := range oldFields {
+					f.SetId("")
+				}
+				for _, f := range newFields {
+					f.SetId("")
+				}
+
+				encodedNewFields, err := json.Marshal(newFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				encodedOldFields, err := json.Marshal(oldFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				if bytes.EqualFold(encodedNewFields, encodedOldFields) {
+					return nil // no changes
+				}
+
+				return txApp.Save(collection)
 			}
 
-			// generate new fields from the query
-			newFields, err := txApp.CreateViewFields(collection.ViewQuery)
-			if err != nil {
-				return err
-			}
-
-			// unset the fields' ids to exclude from the comparison
-			for _, f := range oldFields {
-				f.SetId("")
-			}
-			for _, f := range newFields {
-				f.SetId("")
-			}
-
-			encodedNewFields, err := json.Marshal(newFields)
-			if err != nil {
-				return err
-			}
-
-			encodedOldFields, err := json.Marshal(oldFields)
-			if err != nil {
-				return err
-			}
-
-			if bytes.EqualFold(encodedNewFields, encodedOldFields) {
-				continue // no changes
-			}
-
-			if err := saveViewCollection(txApp, collection, nil); err != nil {
-				return err
+			if err := check(); err != nil {
+				collectionErrors = append(
+					collectionErrors,
+					fmt.Errorf("[%s] %w", collection.Name, err),
+				)
 			}
 		}
 
-		return nil
+		return errors.Join(collectionErrors...)
 	})
 }

core/collection_record_table_sync.go

@@ -6,8 +6,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/security"
 )

core/collection_record_table_sync_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"testing"
 
 	"github.com/pocketbase/dbx"
@@ -278,12 +278,12 @@ func TestSingleVsMultipleValuesNormalization(t *testing.T) {
 				t.Fatalf("Failed to load record: %v", err)
 			}
 
-			encodedResult, err := json.Marshal(result)
+			encodedResult, err := json.Marshal(result, json.Deterministic(true))
 			if err != nil {
 				t.Fatalf("Failed to encode result: %v", err)
 			}
 
-			encodedExpectation, err := json.Marshal(s.expected)
+			encodedExpectation, err := json.Marshal(s.expected, json.Deterministic(true))
 			if err != nil {
 				t.Fatalf("Failed to encode expectation: %v", err)
 			}

core/collection_validate.go

@@ -7,8 +7,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/list"

core/db.go

@@ -10,8 +10,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"
 )

core/db_connect.go

@@ -11,7 +11,7 @@ func DefaultDBConnect(dbPath string) (*dbx.DB, error) {
 	// Note: the busy_timeout pragma must be first because
 	// the connection needs to be set to block on busy before WAL mode
 	// is set in case it hasn't been already set by another connection.
-	pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)"
+	pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)&_defensive=1"
 
 	db, err := dbx.Open("sqlite", dbPath+pragmas)
 	if err != nil {

core/db_table_test.go

@@ -3,7 +3,7 @@ package core_test
 import (
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"slices"
 	"testing"
@@ -120,7 +120,11 @@ func TestTableInfo(t *testing.T) {
 		t.Run(fmt.Sprintf("%d_%s", i, s.tableName), func(t *testing.T) {
 			rows, _ := app.TableInfo(s.tableName)
 
-			raw, err := json.Marshal(rows)
+			raw, err := json.Marshal(
+				rows,
+				json.Deterministic(true),
+				json.FormatNilSliceAsNull(true),
+			)
 			if err != nil {
 				t.Fatal(err)
 			}

core/event_request_batch.go

@@ -3,7 +3,7 @@ package core
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/hook"
 )
 

core/event_request_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"net/http"
 	"strings"
 	"testing"
@@ -183,7 +183,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
 			t.Fatalf("Failed to resolve request info: %v", err)
 		}
 
-		raw, err := json.Marshal(info)
+		raw, err := json.Marshal(info, json.Deterministic(true))
 		if err != nil {
 			t.Fatalf("Failed to serialize request info: %v", err)
 		}
@@ -205,7 +205,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
 			t.Fatalf("Failed to resolve request info: %v", err)
 		}
 
-		raw, err := json.Marshal(info)
+		raw, err := json.Marshal(info, json.Deterministic(true))
 		if err != nil {
 			t.Fatalf("Failed to serialize request info: %v", err)
 		}
@@ -308,7 +308,7 @@ func TestRequestInfoClone(t *testing.T) {
 
 	// check the original data
 	// ---
-	originalRaw, err := json.Marshal(info)
+	originalRaw, err := json.Marshal(info, json.Deterministic(true))
 	if err != nil {
 		t.Fatalf("Failed to serialize original request info: %v", err)
 	}
@@ -321,7 +321,7 @@ func TestRequestInfoClone(t *testing.T) {
 
 	// check the clone data
 	// ---
-	cloneRaw, err := json.Marshal(clone)
+	cloneRaw, err := json.Marshal(clone, json.Deterministic(true))
 	if err != nil {
 		t.Fatalf("Failed to serialize clone request info: %v", err)
 	}

core/events.go

@@ -8,6 +8,7 @@ import (
 	"time"
 
 	"github.com/pocketbase/pocketbase/tools/auth"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/mailer"
 	"github.com/pocketbase/pocketbase/tools/router"
@@ -188,6 +189,24 @@ type MailerRecordEvent struct {
 	Meta map[string]any
 }
 
+// -------------------------------------------------------------------
+// Filesystem events data
+// -------------------------------------------------------------------
+
+type FilesystemNewWriterEvent struct {
+	hook.Event
+	*filesystem.NewWriterEvent
+
+	App App
+}
+
+type FilesystemDeleteEvent struct {
+	hook.Event
+	*filesystem.DeleteEvent
+
+	App App
+}
+
 // -------------------------------------------------------------------
 // Model events data
 // -------------------------------------------------------------------

core/external_auth_model.go

@@ -4,7 +4,7 @@ import (
 	"context"
 	"errors"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"

core/field.go

@@ -6,7 +6,7 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/list"
 )

core/field_autodate.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_bool.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_date.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_editor.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_email.go

@@ -5,8 +5,8 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_file.go

@@ -9,7 +9,7 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
@@ -732,12 +732,16 @@ func (f *FileField) toSliceValue(raw any) []any {
 	case nil:
 		// nothing to cast
 	case *filesystem.File:
-		result = append(result, value)
+		if value != nil {
+			result = append(result, value)
+		}
 	case filesystem.File:
 		result = append(result, &value)
 	case []*filesystem.File:
 		for _, v := range value {
-			result = append(result, v)
+			if v != nil {
+				result = append(result, v)
+			}
 		}
 	case []filesystem.File:
 		for _, v := range value {

core/field_file_test.go

@@ -3,7 +3,7 @@ package core_test
 import (
 	"bytes"
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -98,11 +98,13 @@ func TestFileFieldPrepareValue(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	f1Raw, err := json.Marshal(f1)
+	f1Raw, err := json.Marshal(f1, json.Deterministic(true))
 	if err != nil {
 		t.Fatal(err)
 	}
 
+	var nilFile *filesystem.File
+
 	scenarios := []struct {
 		raw      any
 		field    *core.FileField
@@ -114,8 +116,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
 		{123, &core.FileField{MaxSelect: 1}, `"123"`},
 		{"a", &core.FileField{MaxSelect: 1}, `"a"`},
 		{`["a"]`, &core.FileField{MaxSelect: 1}, `"a"`},
-		{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
 		{f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
+		{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
+		{nilFile, &core.FileField{MaxSelect: 1}, `""`},
 		{[]string{}, &core.FileField{MaxSelect: 1}, `""`},
 		{[]string{"a", "b"}, &core.FileField{MaxSelect: 1}, `"b"`},
 
@@ -126,8 +129,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
 		{"a", &core.FileField{MaxSelect: 2}, `["a"]`},
 		{`["a"]`, &core.FileField{MaxSelect: 2}, `["a"]`},
 		{[]any{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
-		{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
 		{[]filesystem.File{*f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
+		{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
+		{[]any{nilFile, f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
 		{[]string{}, &core.FileField{MaxSelect: 2}, `[]`},
 		{[]string{"a", "b", "c"}, &core.FileField{MaxSelect: 2}, `["a","b","c"]`},
 	}
@@ -139,7 +143,7 @@ func TestFileFieldPrepareValue(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -208,7 +212,7 @@ func TestFileFieldDriverValue(t *testing.T) {
 				}
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -695,7 +699,7 @@ func TestFileFieldFindGetter(t *testing.T) {
 
 			v := getter(record)
 
-			raw, err := json.Marshal(v)
+			raw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -812,7 +816,7 @@ func TestFileFieldFindSetter(t *testing.T) {
 
 			setter(record, s.value)
 
-			raw, err := json.Marshal(record.Get(s.field.GetName()))
+			raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -911,8 +915,8 @@ func TestFileFieldIntercept(t *testing.T) {
 
 		tests.TestValidationErrors(t, err, []string{"text"})
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f1.Name, f3})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f1.Name, f3}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -931,8 +935,8 @@ func TestFileFieldIntercept(t *testing.T) {
 			t.Fatalf("Expected save to succeed, got %v", err)
 		}
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -952,8 +956,8 @@ func TestFileFieldIntercept(t *testing.T) {
 			t.Fatalf("Expected save to succeed, got %v", err)
 		}
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -1063,8 +1067,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
 					t.Fatalf("Expected save to succeed, got %v", err)
 				}
 
-				raw, _ := json.Marshal(record.GetRaw("file_many"))
-				expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
+				raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+				expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
 				if !bytes.Equal(expectedRaw, raw) {
 					t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 				}
@@ -1086,8 +1090,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
 					t.Fatalf("Expected save to succeed, got %v", err)
 				}
 
-				raw, _ := json.Marshal(record.GetRaw("file_many"))
-				expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
+				raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+				expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
 				if !bytes.Equal(expectedRaw, raw) {
 					t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 				}

core/field_geo_point.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_geo_point_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"testing"
 
@@ -57,7 +57,7 @@ func TestGeoPointFieldPrepareValue(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			raw, err := json.Marshal(v)
+			raw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}

core/field_json.go

@@ -2,12 +2,14 @@ package core
 
 import (
 	"context"
+	"encoding/json/jsontext"
+	"encoding/json/v2"
 	"slices"
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )
@@ -171,6 +173,18 @@ func (f *JSONField) ValidateValue(ctx context.Context, app App, record *Record)
 		return validation.NewError("validation_invalid_json", "Must be a valid json value")
 	}
 
+	// temp extra encoding/json/v2 check since the above validator is
+	// still using the v1 semantics
+	//
+	// @todo remove after updating the string validator
+	if len(raw) > 0 {
+		var dummy any
+		err := json.Unmarshal(raw, &dummy, jsontext.AllowInvalidUTF8(true))
+		if err != nil {
+			return validation.NewError("validation_invalid_json", "Must be a valid json value")
+		}
+	}
+
 	rawStr := strings.TrimSpace(raw.String())
 
 	if f.Required && slices.Contains(emptyJSONValues, rawStr) {

core/field_json_test.go

@@ -101,6 +101,16 @@ func TestJSONFieldValidateValue(t *testing.T) {
 			},
 			true,
 		},
+		{
+			"invalid raw value (v2 semantics)",
+			&core.JSONField{Name: "test"},
+			func() *core.Record {
+				record := core.NewRecord(collection)
+				record.SetRaw("test", types.JSONRaw(`{"a": 1, "a": 2}`))
+				return record
+			},
+			true,
+		},
 		{
 			"zero field value (not required)",
 			&core.JSONField{Name: "test"},

core/field_number.go

@@ -4,7 +4,7 @@ import (
 	"context"
 	"math"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_password.go

@@ -7,7 +7,7 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 	"golang.org/x/crypto/bcrypt"

core/field_password_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"strings"
@@ -496,7 +496,7 @@ func TestPasswordFieldFindSetter(t *testing.T) {
 
 			setter(record, s.value)
 
-			raw, err := json.Marshal(record.Get(s.field.GetName()))
+			raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}

core/field_relation.go

@@ -4,8 +4,8 @@ import (
 	"context"
 	"database/sql/driver"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_relation_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"testing"
 
@@ -119,7 +119,7 @@ func TestRelationFieldPrepareValue(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -181,7 +181,7 @@ func TestRelationFieldDriverValue(t *testing.T) {
 				}
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -590,7 +590,7 @@ func TestRelationFieldFindSetter(t *testing.T) {
 
 			setter(record, s.value)
 
-			raw, err := json.Marshal(record.Get(s.field.GetName()))
+			raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}

core/field_select.go

@@ -5,7 +5,7 @@ import (
 	"database/sql/driver"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_select_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"testing"
 
@@ -119,7 +119,7 @@ func TestSelectFieldPrepareValue(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -181,7 +181,7 @@ func TestSelectFieldDriverValue(t *testing.T) {
 				}
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -503,7 +503,7 @@ func TestSelectFieldFindSetter(t *testing.T) {
 
 			setter(record, s.value)
 
-			raw, err := json.Marshal(record.Get(s.field.GetName()))
+			raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}

core/field_test.go

@@ -2,12 +2,12 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"reflect"
 	"strings"
 	"testing"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 )

core/field_text.go

@@ -8,8 +8,8 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"

core/field_url.go

@@ -5,8 +5,8 @@ import (
 	"net/url"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/fields_list.go

@@ -2,7 +2,7 @@ package core
 
 import (
 	"database/sql/driver"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"slices"
 	"strconv"
@@ -277,7 +277,7 @@ func (l *FieldsList) add(pos int, newField Field) {
 
 // String returns the string representation of the current list.
 func (l FieldsList) String() string {
-	v, _ := json.Marshal(l)
+	v, _ := l.MarshalJSON()
 	return string(v)
 }
 
@@ -355,12 +355,12 @@ func (l FieldsList) MarshalJSON() ([]byte, error) {
 		wrapper = append(wrapper, data)
 	}
 
-	return json.Marshal(wrapper)
+	return json.Marshal(wrapper, json.Deterministic(true))
 }
 
 // Value implements the [driver.Valuer] interface.
 func (l FieldsList) Value() (driver.Value, error) {
-	data, err := json.Marshal(l)
+	data, err := l.MarshalJSON()
 
 	return string(data), err
 }

core/fields_list_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"slices"
 	"strconv"
 	"strings"

core/log_model.go

@@ -1,13 +1,23 @@
 package core
 
-import "github.com/pocketbase/pocketbase/tools/types"
+import (
+	"encoding/json/v2"
+
+	"github.com/pocketbase/pocketbase/tools/types"
+)
 
 var (
-	_ Model = (*Log)(nil)
+	_ Model      = (*Log)(nil)
+	_ DBExporter = (*Log)(nil)
 )
 
 const LogsTableName = "_logs"
 
+const (
+	defaultMaxLogDataSize    = 16 << 10 // ~16kb
+	defaultMaxLogMessageSize = 8000
+)
+
 type Log struct {
 	BaseModel
 
@@ -17,6 +27,63 @@ type Log struct {
 	Level   int                `db:"level" json:"level"`
 }
 
-func (m *Log) TableName() string {
+func (l *Log) TableName() string {
 	return LogsTableName
 }
+
+// DBExport prepares and exports the current log model for db persistence.
+//
+// It also truncates the log's message and data to ensure that it is
+// under app.Settings().Logs.MaxDataSize.
+func (l *Log) DBExport(app App) (map[string]any, error) {
+	result := map[string]any{
+		"id":      l.Id,
+		"created": l.Created,
+		"level":   l.Level,
+	}
+
+	// truncate the raw message bytes
+	// (this is expected to be very rare so it is ok even if multi-byte chars)
+	if int64(len(l.Message)) > defaultMaxLogMessageSize {
+		result["message"] = l.Message[:defaultMaxLogMessageSize]
+	} else {
+		result["message"] = l.Message
+	}
+
+	// @todo once added in the standard library consider replacing with
+	// WithByteLimit and WithDepthLimit as suggested in https://github.com/golang/go/issues/56733
+	if len(l.Data) == 0 {
+		result["data"] = l.Data
+	} else {
+		maxDataSize := app.Settings().Logs.MaxDataSize
+		if maxDataSize == 0 {
+			maxDataSize = defaultMaxLogDataSize
+		}
+
+		rawData, err := l.Data.MarshalJSON()
+		if err != nil {
+			return nil, err
+		}
+
+		if int64(len(rawData)) > maxDataSize {
+			truncatedData := types.JSONMap[any]{}
+
+			// ignore syntax errors in case of truncated incomplete json
+			//
+			// jsonv2 stream decodes and all "valid" attrs read up to the
+			// invalid part will be populated in truncatedData
+			_ = json.Unmarshal(rawData[:maxDataSize], &truncatedData)
+
+			truncatedData["__pb_truncated__"] = true
+
+			rawData, err = truncatedData.MarshalJSON()
+			if err != nil {
+				return nil, err
+			}
+		}
+
+		result["data"] = types.JSONRaw(rawData)
+	}
+
+	return result, nil
+}

core/log_model_test.go

@@ -0,0 +1,119 @@
+package core_test
+
+import (
+	"bytes"
+	"encoding/json/v2"
+	"strings"
+	"testing"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/types"
+)
+
+func TestLogTableName(t *testing.T) {
+	t.Parallel()
+
+	var log core.Log
+
+	if name := log.TableName(); name != core.LogsTableName {
+		t.Fatalf("Expected Log table name %q, got %q", core.LogsTableName, name)
+	}
+}
+
+func TestLogDBExport(t *testing.T) {
+	t.Parallel()
+
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	date, err := types.ParseDateTime("2026-08-18 10:20:30.456Z")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	messageLimit := 8000
+	dataLimit := 16 << 10
+
+	scenarios := []struct {
+		name       string
+		log        core.Log
+		limit      int64 // 0 -> use default
+		expectJSON string
+	}{
+		{
+			"empty log",
+			core.Log{},
+			0,
+			`{"created":"","data":{},"id":"","level":0,"message":""}`,
+		},
+		{
+			"with message and data below the default limits",
+			core.Log{
+				BaseModel: core.BaseModel{Id: "test_id"},
+				Created:   date,
+				Level:     123,
+				Message:   "test_message",
+				Data:      types.JSONMap[any]{"a": "test1", "b": "test2"},
+			},
+			0,
+			`{"created":"2026-08-18 10:20:30.456Z","data":{"a":"test1","b":"test2"},"id":"test_id","level":123,"message":"test_message"}`,
+		},
+		{
+			"with message and data exactly the default limits",
+			core.Log{
+				BaseModel: core.BaseModel{Id: "test_id"},
+				Created:   date,
+				Level:     123,
+				Message:   strings.Repeat("a", messageLimit),
+				Data:      types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", dataLimit-32)},
+			},
+			0,
+			`{"created":"2026-08-18 10:20:30.456Z","data":{"a":"test1","b":"test2","c":"` + strings.Repeat("a", dataLimit-32) + `"},"id":"test_id","level":123,"message":"` + strings.Repeat("a", messageLimit) + `"}`,
+		},
+		{
+			"with message and data above the default limits",
+			core.Log{
+				BaseModel: core.BaseModel{Id: "test_id"},
+				Created:   date,
+				Level:     123,
+				Message:   strings.Repeat("a", messageLimit) + "x",                                                      // "x" should be omitted
+				Data:      types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", dataLimit-32) + "x"}, // the end will be incomplete and something like `"c":"...aaaaaax`
+			},
+			0,
+			`{"created":"2026-08-18 10:20:30.456Z","data":{"__pb_truncated__":true,"a":"test1","b":"test2","c":"` + strings.Repeat("a", dataLimit-32) + `x"},"id":"test_id","level":123,"message":"` + strings.Repeat("a", messageLimit) + `"}`,
+		},
+		{
+			"with data above custom limit",
+			core.Log{
+				BaseModel: core.BaseModel{Id: "test_id"},
+				Created:   date,
+				Level:     123,
+				Message:   "test_message",
+				Data:      types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", (2<<10)-32) + "x"}, // the end will be incomplete and something like `"c":"...aaaaaax`
+			},
+			2 << 10,
+			`{"created":"2026-08-18 10:20:30.456Z","data":{"__pb_truncated__":true,"a":"test1","b":"test2","c":"` + strings.Repeat("a", (2<<10)-32) + `x"},"id":"test_id","level":123,"message":"test_message"}`,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			testApp.Settings().Logs.MaxDataSize = s.limit
+
+			result, err := s.log.DBExport(testApp)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			raw, err := json.Marshal(result, json.Deterministic(true))
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			if !bytes.Equal(raw, []byte(s.expectJSON)) {
+				t.Fatalf("Expected export data\n%s\ngot\n%s", s.expectJSON, raw)
+			}
+		})
+	}
+}

core/log_printer_test.go

@@ -3,17 +3,24 @@ package core
 import (
 	"context"
 	"database/sql"
+	"io"
 	"log/slog"
 	"os"
 	"testing"
 	"time"
 
+	"github.com/fatih/color"
 	"github.com/pocketbase/dbx"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/logger"
 )
 
 func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
+	// temp unset to avoid littering the stdout if the test fails when in dev mode
+	colorOutput := color.Output
+	color.Output = io.Discard
+	defer func() { color.Output = colorOutput }()
+
 	testLogLevel := 4
 
 	scenarios := []struct {
@@ -48,7 +55,7 @@ func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
 				DataDir: testDataDir,
 				IsDev:   s.isDev,
 			})
-			defer app.ResetBootstrapState()
+			defer app.ClearBootstrap()
 
 			if err := app.Bootstrap(); err != nil {
 				t.Fatal(err)
@@ -68,7 +75,7 @@ func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
 			var printedLevels []int
 			var persistedLevels []int
 
-			ctx := context.Background()
+			ctx := context.WithValue(context.Background(), logger.BlockKey, true)
 
 			// track printed logs
 			originalPrintLog := printLog

core/log_query_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"testing"
 	"time"
@@ -63,7 +63,7 @@ func TestLogsStats(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	encoded, _ := json.Marshal(result)
+	encoded, _ := json.Marshal(result, json.Deterministic(true))
 	if string(encoded) != expected {
 		t.Fatalf("Expected\n%q\ngot\n%q", expected, string(encoded))
 	}

core/migrations_runner_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"testing"
 	"time"

core/notify_watcher.go

@@ -10,6 +10,7 @@ import (
 	"github.com/fatih/color"
 	"github.com/fsnotify/fsnotify"
 	"github.com/pocketbase/pocketbase/tools/hook"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
 
@@ -152,7 +153,7 @@ func createNotifyDirWatcher(app App, instanceId string, localNotifyDirPath strin
 	}
 
 	// watch
-	go func() {
+	routine.FireAndForget(func() {
 		defer stopDebounceTimer()
 
 		for {
@@ -204,7 +205,7 @@ func createNotifyDirWatcher(app App, instanceId string, localNotifyDirPath strin
 				}
 			}
 		}
-	}()
+	})
 
 	return watcher, err
 }

core/record_field_resolver.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -241,6 +241,7 @@ func (r *RecordFieldResolver) updateQueryWithDeduplicateConstraint(query *dbx.Se
 	// }
 }
 
+//nolint:unused
 func preferGroupBy(info *dbx.QueryInfo, fullUnquotedGroupByCol string) bool {
 	if len(info.GroupBy) != 0 {
 		return false
@@ -353,7 +354,7 @@ func (r *RecordFieldResolver) resolveStaticRequestField(path ...string) (*search
 
 		// if that doesn't work, try encoding it
 		if castErr != nil {
-			encoded, jsonErr := json.Marshal(v)
+			encoded, jsonErr := json.Marshal(v, json.Deterministic(true))
 			if jsonErr == nil {
 				val = string(encoded)
 			}

core/record_field_resolver_runner.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"reflect"
@@ -333,7 +333,7 @@ func (r *runner) processRequestBodyEachModifier(bodyField Field) (*search.Resolv
 	}
 
 	bodyItems := toSlice(r.resolver.requestInfo.Body[bodyField.GetName()])
-	bodyItemsRaw, err := json.Marshal(bodyItems)
+	bodyItemsRaw, err := json.Marshal(bodyItems, json.Deterministic(true))
 	if err != nil {
 		return nil, fmt.Errorf("cannot serialize the data for field %q", r.activeProps[2])
 	}

core/record_field_resolver_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"regexp"
 	"slices"
 	"strings"
@@ -940,7 +940,7 @@ func TestRecordFieldResolverResolveStaticRequestInfoFields(t *testing.T) {
 				t.Fatalf("Expected parameter r.Identifier %q, got %q", paramName, r.Identifier)
 			}
 
-			encodedParamValue, _ := json.Marshal(paramValue)
+			encodedParamValue, _ := json.Marshal(paramValue, json.Deterministic(true))
 			if string(encodedParamValue) != s.expectParamValue {
 				t.Fatalf("Expected r.Params %#v for %s, got %#v", s.expectParamValue, r.Identifier, string(encodedParamValue))
 			}