normalize negative pool size

.github/SECURITY.md

@@ -2,19 +2,26 @@
 
 **Keep in mind that PocketBase is a non-commercial open source project, maintained entirely on volunteer basis (there is no company or dedicated team behind it), and there are no bounties!**
 
-If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io** or submit a private [GitHub Security advisory](https://github.com/pocketbase/pocketbase/security/advisories).
+If you want to responsibly report a security issue you'll have to reach out as a human to **support at pocketbase.io**.
 
-I try to be as responsive as possible and usually address security reports within a day or two, but if you didn't receive a reply from me for more than 5 days it is very likely that your email was flagged and in that case please open a GitHub issue or discussion just mentioning that you found a vulnerability and want to report it so that I can see the notification and will try to contact you for more details.
+This means:
+- no overconfident and arrogant tone
+- no threatening deadlines
+- no requirement for me to login in your security platform just to read the report
+- no inflated severity (we can discuss the CVSS score after confirming the issue)
+- no LLMs usage as part of your report description or followup communication
 
-In case the vulnerability is confirmed, within another couple days I'll try to submit a fix, GitHub security advisory and CVE with remediation steps and **minimal details** regarding the found exploit to minimize giving too much hints to malicious actors (you'll be credited both in the fix release notes and in the public report).
+Reports that don't follow the above will NOT be reviewed no matter of their validity _(you are of course free to publish whatever you want; see also [#7718](https://github.com/pocketbase/pocketbase/discussions/7718))_.
 
-### Please:
+**Or in other words - a simple _"Hey I think I found a security issue when I do X"_ is enough.**
 
-- DO NOT use LLM as part of your report or email communication - it is extremely frustrating to spend an hour or more reading a wall of generated text, writing an elaborate reply and then to receive another generic LLM prompt response in return.
+I try to be as responsive as possible and usually address security issues within couple days but if you didn't receive a reply from me for more than a week it is very likely that your email was flagged and in that case please open a GitHub issue or discussion just mentioning that you found a vulnerability and want to report it so that I can see the notification and will try to contact you for more details.
 
-- DO NOT reserve and publish MITRE CVE number on your own _(I prefer to do it through the GitHub Security advisory)_ and try to communicate first privately the details to better understand how the code is being used and whether the supposed vulnerability can be actually exploited in any real practical scenarios. Otherwise you are risking needlessly causing scaremongering and annoyance for users that rely on security scanners as part of their CI/CD pipeline.
+In case the vulnerability is confirmed:
 
-- Wait before publicly disclosing and sharing details about the found vulnerability, **ideally at least 5 days after the fix**, to make it harder to exploit and give enough time for users to patch their instances _(you are free to provide a PoC and as much details as you want in your own blog/gist/etc.)_.
+- I'll start working on a local fix.
+- Once the fix is implemented locally, I'll publish a pre-announcement with a scheduled release date _(and when possible an approximate release time)_.
+- After the release, I'll publish a GitHub security advisory and CVE with remediation steps and **minimal** details regarding the found exploit _(you are free to publish PoC and more details in your own blog, gist, etc. but it is advised to wait at least a week after the release to allow enough time for people to patch their instances before making it more publicly known)_.
 
 ### Below is a short list of previous reports that are NOT considered security issues:
 
@@ -72,6 +79,26 @@ If someone is able to tamper with the OAuth2 responses then the entire OAuth2 fl
 ~Nonetheless, in future PocketBase releases there will be [extra `localhost` domain like checks](https://github.com/orgs/pocketbase/projects/2/views/1?pane=issue&itemId=159545722) when assigning the OAuth2 avatar URL to a `file` field that will further minimize the risk of internal network probing requests in case of a vulnerable OAuth2 provider.~ _Done._
 </details>
 
+<details>
+<summary><strong>Users enumeration</strong></summary>
+
+This is a common and usually valid report but there is no easy solution without confusing and degrading the users experience.
+
+Some endpoints, like the user create/register, can be used for usernames or emails enumeration based on various response heuristics - timing, specific error messages, etc.
+
+In many places where applicable we've tried to minimize the impact by using constant time checks, returning non-descriptive error messages, applying an internal rate limit for some operations, etc. but it is not bulletproof and if somebody wants to find out if a user is registered they will be able to do it one way or another.
+
+If you think that there is a place where we can improve the handling without hurting too much the user experience, feel free to open a regular public issue and it will be considered.
+</details>
+
+<details>
+<summary><strong>Attack-vectors relying on social engineering</strong></summary>
+
+Reports for attacks relying on various social engineering tactics _(e.g. tricking someone to click on a link)_ are valid concerns but usually out of the security scope of the project as there are a lot of cases where the APIs are deliberately designed for minimal friction.
+
+If you have concerns for such attack, feel free to open a regular public issue and we can eventually try to reconsider adding extra guards when feasible _(or at least properly document the existing behavior)_.
+</details>
+
 <details>
 <summary><strong><code>disintegration/imaging</code> CVE-2023-36308</strong></summary>
 
@@ -87,3 +114,15 @@ Third, even if that issue is still available, with PocketBase it would have been
 
 In the future I may consider eventually replacing the library because it is no longer actively maintained but as of now it is working correctly and as expected for our use case and you can safely flag the security warning as false-positive.
 </details>
+
+<details>
+<summary><strong>JSVM "sandboxing"</strong></summary>
+
+This is another very common report but **there is no such thing as JSVM "sandboxing" in PocketBase**.
+
+The JS `pb_hooks` (or JSVM for short) are NOT supposed to run untrusted or client provided JavaScript code _(the same way you are not supposed to run untrusted code in your Node.js server)_.
+
+Once interpreted the `pb_hooks` run as part of the same application process together with the rest of the Go code. There are no additional filesystem, network, memory, etc. restrictions. This means that it is OK for developers to be able to access environment variables, perform network calls to any URLs they want, invoke shell commands or even sleep/block the script execution.
+
+So if you are security researcher and not sure if something is a security fault in the JS hooks, ask yourself - "Can I do the same when using PocketBase as Go framework?" and if the answer is "Yes" then it is not a security issue with the JSVM.
+</details>

.github/workflows/release.yaml

@@ -26,9 +26,9 @@ jobs:
           node-version: '>=25.2.1'
 
       - name: Set up Go
-        uses: actions/setup-go@v6
+        uses: actions/setup-go@v7
         with:
-          go-version: '>=1.26.2'
+          go-version: '>=1.27.1'
 
       # This step usually is not needed because the /ui/dist is pregenerated locally
       # but its here to ensure that each release embeds the latest admin ui artifacts.

CHANGELOG_16_22.md

@@ -2,6 +2,93 @@
 > For the most recent versions, please refer to [CHANGELOG.md](./CHANGELOG.md)
 ---
 
+## v0.22.55
+
+- (_Backported from v0.40.3_) Fixed collection index validator to allow expressions with parenthesis in the optional `WHERE` clause.
+
+- (_Backported from v0.40.3_) Fixed nested cascade delete of self-referenced relation records.
+
+- (_Backported from v0.40.3_) Bumped `golang.org/x/*` dependencies to silence security scanners ([#7829](https://github.com/pocketbase/pocketbase/discussions/7829)).
+
+
+## v0.22.54
+
+- (_Backported from v0.40.2_) Bumped goja and its related dependencies _(regex unescaped dash error fix and base64 optimizations)_.
+
+- (_Backported from v0.40.2_) Return an error when filter params fallback fails to json serialize and optimized params replacement to execute in a single pass.
+
+- (_Backported from v0.40.2_) Bumped the min Go GitHub action version to 1.27.1 as it includes some [minor `database/sql` and `enconding/json/v2` bug fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.27.1).
+
+
+## v0.22.53
+
+- Workaround `encoding/json` infinite recursion when used with Go 1.27 ([#7808](https://github.com/pocketbase/pocketbase/issues/7808)).
+
+
+## v0.22.52
+
+- (_Backported from v0.39.11_) Bumped `golang.org/x/*` deps and the min Go GitHub action version to 1.26.6 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.6).
+
+
+## v0.22.51
+
+- (_Backported from v0.39.10_) Reverted the auto panic recover handling for the cli commands to preserve the old behavior and allow panic to force exit with non-zero code ([#7781](https://github.com/pocketbase/pocketbase/issues/7781)).
+
+
+## v0.22.50
+
+- (_Backported from v0.39.9_) Bumped goja, fexpr and their related deps.
+
+
+## v0.22.49
+
+- (_Backported from v0.39.8_) Bumped `golang.org/x/*` indirect dependencies as there are some minor security fixes.
+
+- (_Backported from v0.39.8_) Updated `modernc.org/sqlite` to v1.54.0 ([SQLite 3.53.3](https://sqlite.org/src/timeline?from=version-3.53.2&to=version-3.53.3&to2=branch-3.53)).
+
+
+## v0.22.48
+
+- (_Backported from v0.39.7_) Replaced `github.com/go-ozzo/ozzo-validation` with the fork `github.com/pocketbase/ozzo-validation` since the original library has recently changed ownership and the new maintainer cannot be trusted.
+
+- (_Backported from v0.39.7_) Fixed View collection `*` validator and added more friendly error messages ([#7761](https://github.com/pocketbase/pocketbase/issues/7761)).
+
+- (_Backported from v0.39.7_) ⚠️ Security fix for unhandled panic in internal worker goroutines ([#7762](https://github.com/pocketbase/pocketbase/discussions/7762)).
+
+
+## v0.22.47
+
+- (_Backported from v0.39.6_) Bumped the min Go GitHub action version to 1.26.5 as it includes some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.5).
+
+
+## v0.22.46
+
+- (_Backported from v0.39.1_) Bumped the min Go GitHub action version to 1.26.4 as it includes some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.4).
+
+
+## v0.22.45
+
+- (_Backported from v0.38.2_) Updated all `golang.org/x/` packages to cover the recent [security fixes](https://groups.google.com/g/golang-announce/c/PdiGK3xulk4) _(none of them should be a critical issue in PocketBase but nonetheless it is advised to update)_.
+
+
+## v0.22.44
+
+- (_Backported from v0.38.1_) Force unset the auth state of existing realtime connections on user tokenKey change.
+
+
+## v0.22.43
+
+- (_Backported from v0.38.0_) Bumped min Go GitHub action version to 1.26.3 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.3).
+
+
+## v0.22.42
+
+- (_Backported from v0.37.4_) Adjusted Bitbucket, GitHub, GitLab and Gitea/Forgejo OAuth2 providers to better reflect recent API updates and doc references.
+    _In case the userinfo data is not sufficient, some of the providers now send a separate list emails request in order to minimize eventual linking security issues caused by custom onpremise setups (e.g. Gitea/Forgejo allows skipping the email verification if an ENV variable is configured)._
+
+- (_Backported from v0.37.4_) ⚠️ Fixed a pre-hijacking OAuth2 linking vulnerability ([#7662](https://github.com/pocketbase/pocketbase/discussions/7662)).
+
+
 ## v0.22.41
 
 - (_Backported from v0.36.9_) Updated the Discord `AuthUser.Name` field to use `global_name`.

CONTRIBUTING.md

@@ -13,7 +13,7 @@ This document describes how to prepare a PR for a change in the main repository.
 
 ## Prerequisites
 
-- Go 1.25+ (for making changes in the Go code)
+- Go 1.27+ (for making changes in the Go code)
 - Node 24+ (for making changes in the Superuser UI)
 
 If you haven't already, you can fork the main repository and clone your fork so that you can work locally:

README.md

@@ -49,7 +49,7 @@ your own custom app specific business logic and still have a single portable exe
 
 Here is a minimal example:
 
-0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 
 1. Create a new project directory with the following `main.go` file inside it:
     ```go
@@ -92,32 +92,37 @@ _For more details please refer to [Extend with Go](https://pocketbase.io/docs/go
 
 To build the minimal standalone executable, like the prebuilt ones in the releases page, you can simply run `go build` inside the `examples/base` directory:
 
-0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 1. Clone/download the repo
 2. Navigate to `examples/base`
-3. Run `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`
-   (_https://go.dev/doc/install/source#environment_)
+3. Run `CGO_ENABLED=0 go build` to build a binary for your current environment
+   _(or to target other platforms use `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`; see https://go.dev/doc/install/source#environment)_
 4. Start the created executable by running `./base serve`.
 
 Note that the supported build targets by the pure Go SQLite driver at the moment are:
 
-```
-darwin  amd64
-darwin  arm64
-freebsd amd64
-freebsd arm64
-linux   386
-linux   amd64
-linux   arm
-linux   arm64
-linux   loong64
-linux   ppc64le
-linux   riscv64
-linux   s390x
-windows 386
-windows amd64
-windows arm64
-```
+| GOOS    | GOARCH  |
+|---------|---------|
+| darwin  | amd64   |
+| darwin  | arm64   |
+| freebsd | 386     |
+| freebsd | amd64   |
+| freebsd | arm     |
+| freebsd | arm64   |
+| linux   | 386     |
+| linux   | amd64   |
+| linux   | arm     |
+| linux   | arm64   |
+| linux   | loong64 |
+| linux   | ppc64le |
+| linux   | riscv64 |
+| linux   | s390x   |
+| netbsd  | amd64   |
+| openbsd | amd64   |
+| openbsd | arm64   |
+| windows | 386     |
+| windows | amd64   |
+| windows | arm64   |
 
 ### Testing
 
@@ -134,7 +139,7 @@ Check also the [Testing guide](http://pocketbase.io/docs/testing) to learn how t
 
 If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
 
-All reports will be promptly addressed and you'll be credited in the fix release notes.
+You could find more details in the project [Security policy](https://github.com/pocketbase/pocketbase/security/policy).
 
 ## Contributing
 

apis/backup.go

@@ -70,8 +70,10 @@ func backupDownload(e *core.RequestEvent) error {
 		return e.ForbiddenError("Insufficient permissions to access the resource.", err)
 	}
 
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
-	defer cancel()
+	allowedIPs := e.App.Settings().SuperuserIPs
+	if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+		return e.ForbiddenError("Insufficient permissions to access the resource.", nil)
+	}
 
 	fsys, err := e.App.NewBackupsFilesystem()
 	if err != nil {
@@ -79,6 +81,9 @@ func backupDownload(e *core.RequestEvent) error {
 	}
 	defer fsys.Close()
 
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
+	defer cancel()
+
 	fsys.SetContext(ctx)
 
 	key := e.Request.PathValue("key")

apis/backup_create.go

@@ -5,7 +5,7 @@ import (
 	"net/http"
 	"regexp"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/backup_test.go

@@ -528,6 +528,58 @@ func TestBackupsDownload(t *testing.T) {
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
+		{
+			Name:    "with valid superuser file token AND whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/backups/test1.zip?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := createTestBackups(app); err != nil {
+					t.Fatal(err)
+				}
+
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"127.0.0.1"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				"storage/",
+				"data.db",
+				"auxiliary.db",
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:    "with valid superuser file token BUT non-whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/backups/test1.zip?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := createTestBackups(app); err != nil {
+					t.Fatal(err)
+				}
+
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/backup_upload.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/filesystem"

apis/base.go

@@ -31,6 +31,7 @@ func NewRouter(app core.App) (*router.Router[*core.RequestEvent], error) {
 	pbRouter.Bind(panicRecover())
 	pbRouter.Bind(rateLimit())
 	pbRouter.Bind(loadAuthToken())
+	pbRouter.Bind(superuserIPsWhitelist())
 	pbRouter.Bind(securityHeaders())
 	pbRouter.Bind(BodyLimit(DefaultMaxBodySize))
 
@@ -47,6 +48,7 @@ func NewRouter(app core.App) (*router.Router[*core.RequestEvent], error) {
 	bindBatchApi(app, apiGroup)
 	bindRealtimeApi(app, apiGroup)
 	bindHealthApi(app, apiGroup)
+	bindSQLApi(app, apiGroup)
 
 	// UI routes
 	bindUIExtensions(app)

apis/batch.go

@@ -2,7 +2,7 @@ package apis
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"io"
 	"mime/multipart"
@@ -14,10 +14,11 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/types"
 	"github.com/spf13/cast"
 )
@@ -195,7 +196,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			p.stopCh <- struct{}{}
 		}()
 
-		go func() {
+		routine.FireAndForget(func() {
 			err := p.process(txApp, batch, 0)
 
 			if err != nil {
@@ -216,7 +217,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			}
 
 			p.errCh <- err
-		}()
+		})
 
 		select {
 		case responseErr := <-p.errCh:

apis/batch_test.go

@@ -224,7 +224,7 @@ func TestBatchRequest(t *testing.T) {
 			},
 		},
 		{
-			Name:   "mixed create/update/delete (rules failure)",
+			Name:   "mixed create/update/delete (non-superuser rule failure)",
 			Method: http.MethodPost,
 			URL:    "/api/batch",
 			Body: strings.NewReader(`{
@@ -284,6 +284,70 @@ func TestBatchRequest(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "mixed create/update/delete (superuser rule failure)",
+			Method: http.MethodPost,
+			URL:    "/api/batch",
+			Headers: map[string]string{
+				// test@example.com, clients
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
+			Body: strings.NewReader(`{
+				"requests": [
+					{"method":"DELETE", "url":"/api/collections/demo2/records/achvryl401bhse3", "headers": {"Authorization": "ignored"}},
+					{"method":"PATCH", "url":"/api/collections/demo3/records/1tmknxy2868d869", "body": {"title": "batch_update"}, "headers": {"Authorization": "ignored"}},
+					{"method":"POST", "url":"/api/collections/_superusers/records", "body": {"email":"test_batch@example.com","password":"1234567890"}}
+				]
+			}`),
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"requests":{`,
+				`"2":{"code":"batch_request_failed"`,
+				`403`,
+			},
+			NotExpectedContent: []string{
+				`"0":`,
+				`"1":`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                        0,
+				"OnBatchRequest":           1,
+				"OnModelUpdate":            1,
+				"OnModelUpdateExecute":     1,
+				"OnModelAfterUpdateError":  1,
+				"OnModelDelete":            1,
+				"OnModelDeleteExecute":     1,
+				"OnModelAfterDeleteError":  1,
+				"OnModelValidate":          1,
+				"OnRecordUpdateRequest":    1,
+				"OnRecordUpdate":           1,
+				"OnRecordUpdateExecute":    1,
+				"OnRecordAfterUpdateError": 1,
+				"OnRecordDeleteRequest":    1,
+				"OnRecordDelete":           1,
+				"OnRecordDeleteExecute":    1,
+				"OnRecordAfterDeleteError": 1,
+				"OnRecordEnrich":           1,
+				"OnRecordValidate":         1,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				_, err = app.FindRecordById("demo2", "achvryl401bhse3")
+				if err != nil {
+					t.Fatal("Expected record to not be deleted")
+				}
+
+				_, err = app.FindFirstRecordByFilter("demo3", `title="batch_update"`)
+				if err == nil {
+					t.Fatal("Expected record to not be updated")
+				}
+
+				_, err = app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test_batch@example.com")
+				if err == nil {
+					t.Fatal("Expected superuser to not be created")
+				}
+			},
+		},
 		{
 			Name:   "mixed create/update/delete (rules success)",
 			Method: http.MethodPost,

apis/collection.go

@@ -6,7 +6,7 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/router"

apis/collection_import.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/collection_test.go

@@ -1225,6 +1225,71 @@ func TestCollectionUpdate(t *testing.T) {
 				"OnModelValidate":              1,
 			},
 		},
+		{
+			Name:   "add another OAuth2 provider to an auth collection",
+			Method: http.MethodPatch,
+			URL:    "/api/collections/users",
+			Body: strings.NewReader(`{
+				"oauth2": {
+					"providers": [
+						{"name": "apple", "clientId": "a", "clientSecret": "b"},
+						{
+							"pkce": null,
+							"name": "google",
+							"authURL": "",
+							"displayName": "existing",
+							"extra": {}
+						}
+					]
+				}
+			}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				// verify that the collection has google and gitlab as OAuth2 providers
+				users, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if v := len(users.OAuth2.Providers); v != 2 {
+					t.Fatalf("Expected 2 OAuth2 providers, got %d", v)
+				}
+
+				if v := users.OAuth2.Providers[0].Name; v != "gitlab" {
+					t.Fatalf("Expected provider 0 to be %s, got %s", "gitlab", v)
+				}
+
+				if v := users.OAuth2.Providers[1].Name; v != "google" {
+					t.Fatalf("Expected provider 1 to be %s, got %s", "google", v)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"name":"google"`,
+				`"name":"apple"`,
+				`"displayName":"existing"`,
+				`"clientId":"test"`,
+				`"clientId":"a"`,
+			},
+			NotExpectedContent: []string{
+				`"name":"gitlab"`,
+				`clientSecret`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                              0,
+				"OnCollectionUpdateRequest":      1,
+				"OnCollectionUpdate":             1,
+				"OnCollectionUpdateExecute":      1,
+				"OnCollectionAfterUpdateSuccess": 1,
+				"OnCollectionValidate":           1,
+				"OnModelUpdate":                  1,
+				"OnModelUpdateExecute":           1,
+				"OnModelAfterUpdateSuccess":      1,
+				"OnModelValidate":                1,
+			},
+		},
 
 		// view
 		// -----------------------------------------------------------
@@ -1617,7 +1682,7 @@ func TestCollectionOAuth2Providers(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
-				`{"name":"oidc3","displayName":"OIDC","logo":"\u003csvg`,
+				`{"name":"oidc3","displayName":"OIDC","logo":"<svg`,
 			},
 			NotExpectedContent: []string{
 				`"order":`,

apis/cron_test.go

@@ -77,10 +77,13 @@ func TestCronsRun(t *testing.T) {
 		app.Cron().Add("test", "* * * * *", func() {
 			app.Store().Set("testJobCalls", cast.ToInt(app.Store().Get("testJobCalls"))+1)
 		})
+		app.Cron().Stop()
 	}
 
 	expectedCalls := func(expected int) func(t testing.TB, app *tests.TestApp, res *http.Response) {
 		return func(t testing.TB, app *tests.TestApp, res *http.Response) {
+			time.Sleep(50 * time.Millisecond)
+
 			total := cast.ToInt(app.Store().Get("testJobCalls"))
 			if total != expected {
 				t.Fatalf("Expected total testJobCalls %d, got %d", expected, total)
@@ -93,7 +96,6 @@ func TestCronsRun(t *testing.T) {
 			Name:            "unauthorized",
 			Method:          http.MethodPost,
 			URL:             "/api/crons/test",
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  401,
@@ -107,7 +109,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
 			},
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  403,
@@ -121,7 +122,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
 			},
-			Delay:           50 * time.Millisecond,
 			BeforeTestFunc:  beforeTestFunc,
 			AfterTestFunc:   expectedCalls(0),
 			ExpectedStatus:  404,
@@ -135,7 +135,6 @@ func TestCronsRun(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
 			},
-			Delay:          50 * time.Millisecond,
 			BeforeTestFunc: beforeTestFunc,
 			AfterTestFunc:  expectedCalls(1),
 			ExpectedStatus: 204,

apis/extensions.go

@@ -81,7 +81,8 @@ func copyExtensionMainjs(buf *bytes.Buffer, ext core.UIExtension) error {
 	defer f.Close()
 
 	// wrap in a self-executing function to avoid scope and concatenation issues
-	_, _ = buf.WriteString("(function(){")
+	// (the await/async is for top-level await)
+	_, _ = buf.WriteString("await (async function(){")
 
 	_, err = io.Copy(buf, f)
 	if err != nil {

apis/extensions_test.go

@@ -72,7 +72,7 @@ func TestUIExtensions_Mainjs(t *testing.T) {
 			},
 			AfterTestFunc:   successAfterTestFunc,
 			ExpectedStatus:  200,
-			ExpectedContent: []string{"(function(){ext1_main})();(function(){ext3_main})();"},
+			ExpectedContent: []string{"await (async function(){ext1_main})();await (async function(){ext3_main})();"},
 			ExpectedEvents:  map[string]int{"*": 0},
 		},
 	}

apis/file.go

@@ -60,6 +60,7 @@ type fileApi struct {
 }
 
 func (api *fileApi) fileToken(e *core.RequestEvent) error {
+	// extra check for just in case the handler is called in a different context
 	if e.Auth == nil {
 		return e.UnauthorizedError("Missing auth context.", nil)
 	}
@@ -114,6 +115,15 @@ func (api *fileApi) download(e *core.RequestEvent) error {
 		token := e.Request.URL.Query().Get("token")
 		authRecord, _ := e.App.FindAuthRecordByToken(token, core.TokenTypeFile)
 
+		// reset the auth state if it is superuser and it is not whitelisted
+		// (not critical because file tokens are short-lived but checked nonetheless as an extra precaution)
+		if authRecord != nil && authRecord.IsSuperuser() {
+			allowedIPs := e.App.Settings().SuperuserIPs
+			if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+				authRecord = nil
+			}
+		}
+
 		// create a shallow copy of the cached request data and adjust it to the current auth record (if any)
 		requestInfo := *originalRequestInfo
 		requestInfo.Context = core.RequestInfoContextProtectedFile

apis/file_test.go

@@ -353,6 +353,50 @@ func TestFileDownload(t *testing.T) {
 				"OnFileDownloadRequest": 1,
 			},
 		},
+		{
+			Name:    "protected file - superuser with non-whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/files/demo1/al1h9ijdeojtsjy/300_Jsjq7RdBgA.png?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+				err := app.Save(app.Settings())
+				if err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:    "protected file - superuser with whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/files/demo1/al1h9ijdeojtsjy/300_Jsjq7RdBgA.png?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"127.0.0.1"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"PNG"},
+			ExpectedEvents: map[string]int{
+				"*":                     0,
+				"OnFileDownloadRequest": 1,
+			},
+		},
 		{
 			Name:            "protected file - guest without view access",
 			Method:          http.MethodGet,

apis/health.go

@@ -37,7 +37,7 @@ func healthCheck(e *core.RequestEvent) error {
 		headersToCheck := append(
 			slices.Clone(e.App.Settings().TrustedProxy.Headers),
 			// common proxy headers
-			"CF-Connecting-IP", "Fly-Client-IP", "X‑Forwarded-For",
+			"CF-Connecting-IP", "Fly-Client-IP", "X-Forwarded-For",
 		)
 		for _, header := range headersToCheck {
 			if e.Request.Header.Get(header) != "" {

apis/logs.go

@@ -11,8 +11,13 @@ import (
 
 // bindLogsApi registers the request logs api endpoints.
 func bindLogsApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
-	sub := rg.Group("/logs").Bind(RequireSuperuserAuth(), SkipSuccessActivityLog())
+	sub := rg.Group("/logs").Bind(
+		RequireSuperuserAuth(),
+		SkipSuccessActivityLog(),
+	)
+
 	sub.GET("", logsList)
+	sub.DELETE("", logsTruncate)
 	sub.GET("/stats", logsStats)
 	sub.GET("/{id}", logsView)
 }
@@ -71,3 +76,19 @@ func logsView(e *core.RequestEvent) error {
 
 	return e.JSON(http.StatusOK, log)
 }
+
+func logsTruncate(e *core.RequestEvent) error {
+	// delete all rows directly (aka. no model hooks will be fired)
+	_, err := e.App.AuxNonconcurrentDB().Delete((&core.Log{}).TableName(), nil).Execute()
+	if err != nil {
+		return e.InternalServerError("Failed to truncate all logs.", err)
+	}
+
+	// try to free the unused disk space
+	err = e.App.AuxVacuum()
+	if err != nil {
+		e.App.Logger().Warn("Failed to VACUUM aux database", "error", err)
+	}
+
+	return e.NoContent(http.StatusNoContent)
+}

apis/logs_test.go

@@ -210,3 +210,59 @@ func TestLogsStats(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestLogsTruncate(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodDelete,
+			URL:             "/api/logs",
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := tests.StubLogsData(app); err != nil {
+					t.Fatal(err)
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				var found []core.Log
+
+				if err := app.LogQuery().All(&found); err != nil {
+					t.Fatal(err)
+				}
+
+				if len(found) > 0 {
+					t.Fatalf("Expected all logs to be deleted, found: %v", found)
+				}
+			},
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/middlewares.go

@@ -42,6 +42,9 @@ const (
 	DefaultLoadAuthTokenMiddlewarePriority = DefaultRateLimitMiddlewarePriority - 20
 	DefaultLoadAuthTokenMiddlewareId       = "pbLoadAuthToken"
 
+	DefaultSuperuserIPsWhitelistMiddlewarePriority = DefaultLoadAuthTokenMiddlewarePriority + 5
+	DefaultSuperuserIPsWhitelistMiddlewareId       = "pbSuperuserIPsWhitelist"
+
 	DefaultSecurityHeadersMiddlewarePriority = DefaultRateLimitMiddlewarePriority - 10
 	DefaultSecurityHeadersMiddlewareId       = "pbSecurityHeaders"
 
@@ -290,6 +293,7 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
 			e.Response.Header().Set("X-XSS-Protection", "1; mode=block")
 			e.Response.Header().Set("X-Content-Type-Options", "nosniff")
 			e.Response.Header().Set("X-Frame-Options", "SAMEORIGIN")
+			e.Response.Header().Set("Cross-Origin-Opener-Policy", "same-origin")
 
 			// @todo consider a default HSTS?
 			// (see also https://webkit.org/blog/8146/protecting-against-hsts-abuse/)
@@ -299,6 +303,28 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
 	}
 }
 
+// superuserIPsWhitelist middleware checks the current authenticated superuser IP
+// against the configured SuperuserIPs whitelist setting.
+//
+// This middleware is registered by default for all routes.
+func superuserIPsWhitelist() *hook.Handler[*core.RequestEvent] {
+	return &hook.Handler[*core.RequestEvent]{
+		Id:       DefaultSuperuserIPsWhitelistMiddlewareId,
+		Priority: DefaultSuperuserIPsWhitelistMiddlewarePriority,
+		Func: func(e *core.RequestEvent) error {
+			if e.HasSuperuserAuth() {
+				ips := e.App.Settings().SuperuserIPs
+
+				if len(ips) > 0 && !isIPInList(ips, e.RealIP()) {
+					return e.ForbiddenError("", errors.New("superuser IP is not whitelisted"))
+				}
+			}
+
+			return e.Next()
+		},
+	}
+}
+
 // SkipSuccessActivityLog is a helper middleware that instructs the global
 // activity logger to log only requests that have failed/returned an error.
 func SkipSuccessActivityLog() *hook.Handler[*core.RequestEvent] {

apis/middlewares_body_limit.go

@@ -84,49 +84,73 @@ func applyBodyLimit(e *core.RequestEvent, limitBytes int64) error {
 	}
 
 	// replace the request body
-	//
-	// note: we don't use sync.Pool since the size of the elements could vary too much
-	// and it might not be efficient (see https://github.com/golang/go/issues/23199)
-	e.Request.Body = &limitedReader{ReadCloser: e.Request.Body, limit: limitBytes}
+	e.Request.Body = newMaxBytesReader(e.Request.Body, limitBytes)
 
 	return nil
 }
 
-type limitedReader struct {
-	io.ReadCloser
-	limit     int64
-	totalRead int64
+func newMaxBytesReader(body io.ReadCloser, limitBytes int64) *maxBytesReader {
+	return &maxBytesReader{
+		ReadCloser: body,
+		limit:      limitBytes,
+		remaining:  limitBytes,
+	}
 }
 
-func (r *limitedReader) Read(b []byte) (int, error) {
+// maxBytesReader is very similar to the http.MaxBytesReader but support
+// rereads and doesn't try to prematurely close the related response
+// to allow consequent middlewares to operate correctly.
+type maxBytesReader struct {
+	io.ReadCloser
+	limit     int64
+	remaining int64
+	stickyErr error
+}
+
+func (r *maxBytesReader) Read(b []byte) (int, error) {
+	if r.stickyErr != nil {
+		return 0, r.stickyErr
+	}
+
+	if len(b) == 0 {
+		return 0, nil
+	}
+
+	// if possible no need to read the entire chunk since
+	// remaining+1 is enough to determine whether it exceed the limit
+	if int64(len(b))-1 > r.remaining {
+		b = b[:r.remaining+1]
+	}
+
 	n, err := r.ReadCloser.Read(b)
-	if err != nil {
+
+	if int64(n) <= r.remaining {
+		r.remaining -= int64(n)
+		r.stickyErr = err
 		return n, err
 	}
 
-	r.totalRead += int64(n)
-	if r.totalRead > r.limit {
-		return n, ErrRequestEntityTooLarge
-	}
+	n = int(r.remaining)
 
-	return n, nil
+	r.remaining = 0
+	r.stickyErr = ErrRequestEntityTooLarge
+
+	return n, r.stickyErr
 }
 
 // explicit casts to ensure that the main struct methods will be invoked
 // (extra precautions in case of nested interface wrapping erasure)
 // ---
 
-func (r *limitedReader) Reread() {
+func (r *maxBytesReader) Reread() {
 	rereader, ok := r.ReadCloser.(router.Rereader)
 	if ok {
 		rereader.Reread()
+		r.remaining = r.limit
+		r.stickyErr = nil
 	}
 }
 
-func (r *limitedReader) Close() error {
-	closer, ok := r.ReadCloser.(io.Closer)
-	if ok {
-		return closer.Close()
-	}
-	return nil
+func (r *maxBytesReader) Close() error {
+	return r.ReadCloser.Close()
 }

apis/middlewares_body_limit_test.go

@@ -1,9 +1,10 @@
 package apis_test
 
 import (
-	"bytes"
-	"fmt"
+	"io"
+	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/apis"
@@ -19,13 +20,35 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	pbRouter.POST("/a", func(e *core.RequestEvent) error {
-		return e.String(200, "a")
-	}) // default global BodyLimit check
 
-	pbRouter.POST("/b", func(e *core.RequestEvent) error {
-		return e.String(200, "b")
-	}).Bind(apis.BodyLimit(20))
+	testHandler := func(e *core.RequestEvent) error {
+		// read the body multiple times to ensure that the limited
+		// reader guards and rereads are invoked
+		var result any
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		return e.JSON(200, result)
+	}
+
+	const customLimit = 20
+
+	pbRouter.POST("/a", testHandler) // default global BodyLimit check
+	pbRouter.POST("/b", testHandler).Bind(apis.BodyLimit(customLimit))
+	pbRouter.POST("/iof", func(e *core.RequestEvent) error {
+		// ensure that normal io methods still operate correctly
+		b, err := io.ReadAll(e.Request.Body)
+		if err != nil {
+			return err
+		}
+
+		return e.String(http.StatusOK, string(b))
+	}).Bind(apis.BodyLimit(customLimit))
 
 	mux, err := pbRouter.BuildMux()
 	if err != nil {
@@ -33,20 +56,90 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	}
 
 	scenarios := []struct {
-		url            string
-		size           int64
-		expectedStatus int
+		name              string
+		url               string
+		body              string
+		lazyContentLength bool
+		expectedStatus    int
 	}{
-		{"/a", 21, 200},
-		{"/a", apis.DefaultMaxBodySize + 1, 413},
-		{"/b", 20, 200},
-		{"/b", 21, 413},
+		{
+			"(eager content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"(eager content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"io.ReadAll io.EOF exact limit check",
+			"/iof",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
 	}
 
 	for _, s := range scenarios {
-		t.Run(fmt.Sprintf("%s_%d", s.url, s.size), func(t *testing.T) {
+		t.Run(s.name, func(t *testing.T) {
 			rec := httptest.NewRecorder()
-			req := httptest.NewRequest("POST", s.url, bytes.NewReader(make([]byte, s.size)))
+
+			req := httptest.NewRequest("POST", s.url, strings.NewReader(s.body))
+			req.Header.Set("Content-Type", "application/json")
+
+			if s.lazyContentLength {
+				req.ContentLength = -1
+			}
+
 			mux.ServeHTTP(rec, req)
 
 			result := rec.Result()

apis/middlewares_rate_limit.go

@@ -2,6 +2,7 @@ package apis
 
 import (
 	"errors"
+	"net/netip"
 	"sync"
 	"time"
 
@@ -106,6 +107,41 @@ func checkCollectionRateLimit(e *core.RequestEvent, collection *core.Collection,
 	return nil
 }
 
+// isIPInList checks if the specified IP is in a list of other individual IPs or subnets.
+func isIPInList(ipsOrSubnets []string, ip string) bool {
+	if len(ipsOrSubnets) == 0 || ip == "" {
+		return false
+	}
+
+	// normalize
+	searchAddr, err := netip.ParseAddr(ip)
+	if err != nil {
+		return false
+	}
+
+	for _, item := range ipsOrSubnets {
+		// subnet?
+		prefix, err := netip.ParsePrefix(item)
+		if err == nil {
+			if prefix.Contains(searchAddr) {
+				return true
+			}
+			continue
+		}
+
+		// individual ip?
+		addr, err := netip.ParseAddr(item)
+		if err == nil {
+			if addr == searchAddr {
+				return true
+			}
+			continue
+		}
+	}
+
+	return false
+}
+
 // -------------------------------------------------------------------
 
 // @todo consider exporting as helper?
@@ -153,7 +189,9 @@ func checkRateLimit(e *core.RequestEvent, rtId string, rule core.RateLimitRule)
 }
 
 func skipRateLimit(e *core.RequestEvent) bool {
-	return !e.App.Settings().RateLimits.Enabled || e.HasSuperuserAuth()
+	return !e.App.Settings().RateLimits.Enabled ||
+		e.HasSuperuserAuth() ||
+		isIPInList(e.App.Settings().RateLimits.ExcludedIPs, e.RealIP())
 }
 
 var defaultAuthAudience = []string{core.RateLimitRuleAudienceAll, core.RateLimitRuleAudienceAuth}

apis/middlewares_rate_limit_test.go

@@ -1,13 +1,16 @@
 package apis_test
 
 import (
+	"fmt"
 	"net/http/httptest"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/hook"
 )
 
 func TestDefaultRateLimitMiddleware(t *testing.T) {
@@ -85,9 +88,8 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 		{"/norate", 0, false, 200},
 
 		{"/rate/a", 0, false, 200},
-		{"/rate/a", 700, false, 200}, // (fixed window check) wait enough to ensure that it can't fit more than 2 requests in 1s
-		{"/rate/a", 800, false, 200},
-		{"/rate/a", 800, false, 200},
+		{"/rate/a", 900, false, 200}, // (fixed window check) wait enough to ensure that it can't fit more than 2 requests in 1s
+		{"/rate/a", 900, false, 200},
 		{"/rate/a", 0, false, 200},
 		{"/rate/a", 0, false, 429},
 		{"/rate/a", 0, false, 429},
@@ -127,27 +129,29 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 		{"/rate/guest", 0, true, 429},
 	}
 
-	for _, s := range scenarios {
-		t.Run(s.url, func(t *testing.T) {
+	synctest.Test(t, func(t *testing.T) {
+		for i, s := range scenarios {
+			prefix := fmt.Sprintf("[%s:%d] ", s.url, i+1)
+
 			rec := httptest.NewRecorder()
 			req := httptest.NewRequest("GET", s.url, nil)
 
 			if s.authenticated {
 				auth, err := app.FindAuthRecordByEmail("users", "test@example.com")
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				token, err := auth.NewAuthToken()
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				req.Header.Add("Authorization", token)
 			}
 
 			if s.wait > 0 {
-				time.Sleep(time.Duration(s.wait) * time.Millisecond)
+				synctest.Sleep(time.Duration(s.wait) * time.Millisecond)
 			}
 
 			mux.ServeHTTP(rec, req)
@@ -155,8 +159,168 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 			result := rec.Result()
 
 			if result.StatusCode != s.expectedStatus {
-				t.Fatalf("Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
+				t.Fatalf(prefix+"Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
 			}
-		})
+		}
+	})
+}
+
+func TestDefaultRateLimitMiddlewareSkipChecks(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	app.Settings().RateLimits.Enabled = true
+	app.Settings().RateLimits.Rules = []core.RateLimitRule{
+		{
+			Label:       "/rate",
+			MaxRequests: 1,
+			Duration:    5,
+		},
 	}
+
+	pbRouter, err := apis.NewRouter(app)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// just for the exclude tests - load the user IP from a query param
+	pbRouter.Bind(&hook.Handler[*core.RequestEvent]{
+		Priority: apis.DefaultRateLimitMiddlewarePriority - 1,
+		Func: func(e *core.RequestEvent) error {
+			testIp := e.Request.URL.Query().Get("testIP")
+			if testIp != "" {
+				e.Request.Header.Set("x-test-ip", testIp)
+			}
+
+			return e.Next()
+		},
+	})
+
+	pbRouter.GET("/rate", func(e *core.RequestEvent) error {
+		return e.String(200, "test")
+	})
+
+	mux, err := pbRouter.BuildMux()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	checkStatusCodes := func(t *testing.T, got []int, expected []int) {
+		if len(expected) != len(got) {
+			t.Fatalf("Expected status codes %v, got %v", expected, got)
+		}
+
+		for i, item := range expected {
+			if got[i] != item {
+				t.Fatalf("Expected %d status code to be %d, got %d:\n%v", i, item, got[i], got)
+			}
+		}
+	}
+
+	t.Run("base check", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 429, 429})
+	})
+
+	t.Run("disabled rate limiter", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = false
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
+
+	t.Run("authenticated as superuser", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+
+		superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		token, err := superuser.NewAuthToken()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Add("Authorization", token)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
+
+	t.Run("excludedIPs (different)", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+		app.Settings().RateLimits.ExcludedIPs = []string{"10.0.0.0"}
+		app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Set("x-test-ip", "127.0.0.1")
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 429, 429})
+	})
+
+	t.Run("excludedIPs (match)", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+		app.Settings().RateLimits.ExcludedIPs = []string{"127.0.0.1"}
+		app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Set("x-test-ip", "127.0.0.1")
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
 }

apis/middlewares_test.go

@@ -553,3 +553,96 @@ func TestRequireSameCollectionContextAuth(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestSuperuserIPsWhitelist(t *testing.T) {
+	t.Parallel()
+
+	setupWhitelist := func(superuserIPs ...string) func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+		return func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+			// allow loading a mock IP from the test scenario
+			app.Settings().TrustedProxy = core.TrustedProxyConfig{
+				Headers: []string{"x-test-ip"},
+			}
+
+			app.Settings().SuperuserIPs = superuserIPs
+
+			err := app.Save(app.Settings())
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			e.Router.GET("/my/test", func(e *core.RequestEvent) error {
+				return e.String(200, "test123")
+			})
+		}
+	}
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "guest with non-matching IP",
+			Method:          http.MethodGet,
+			URL:             "/my/test",
+			Headers:         map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "regular user with non-matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with non-matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0", "127.0.0.1"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with no whitelisted IPs",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist(),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/realtime.go

@@ -2,7 +2,7 @@ package apis
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"log/slog"
@@ -10,8 +10,8 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/picker"
@@ -28,6 +28,9 @@ const clientsChunkSize = 150
 // RealtimeClientAuthKey is the name of the realtime client store key that holds its auth state.
 const RealtimeClientAuthKey = "auth"
 
+// RealtimeClientIPKey is the name of the realtime client store key that holds the IP of the connected client.
+const RealtimeClientIPKey = "pbRealtimeClientIP"
+
 // bindRealtimeApi registers the realtime api endpoints.
 func bindRealtimeApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
 	sub := rg.Group("/realtime")
@@ -63,8 +66,12 @@ func realtimeConnect(e *core.RequestEvent) error {
 
 	connectEvent := new(core.RealtimeConnectRequestEvent)
 	connectEvent.RequestEvent = e
-	connectEvent.Client = subscriptions.NewDefaultClient()
 	connectEvent.IdleTimeout = 5 * time.Minute
+	connectEvent.MaxTimeout = 30 * time.Minute
+	connectEvent.Client = subscriptions.NewDefaultClient()
+
+	// could be used as an optional cross-reference check in other API endpoints
+	connectEvent.Client.Set(RealtimeClientIPKey, e.RealIP())
 
 	return e.App.OnRealtimeConnectRequest().Trigger(connectEvent, func(ce *core.RealtimeConnectRequestEvent) error {
 		// register new subscription client
@@ -73,7 +80,7 @@ func realtimeConnect(e *core.RequestEvent) error {
 			e.App.SubscriptionsBroker().Unregister(ce.Client.Id())
 		}()
 
-		ce.App.Logger().Debug("Realtime connection established.", slog.String("clientId", ce.Client.Id()))
+		ce.App.Logger().Debug("Realtime connection established", slog.String("clientId", ce.Client.Id()))
 
 		// signalize established connection (aka. fire "connect" message)
 		connectMsgEvent := new(core.RealtimeMessageEvent)
@@ -99,12 +106,19 @@ func realtimeConnect(e *core.RequestEvent) error {
 			return nil
 		}
 
+		// start a max lifetime timer to prevent accumulating too much
+		// connection resources and to allow the GC to run more regularly
+		maxTimer := time.NewTimer(ce.MaxTimeout)
+		defer maxTimer.Stop()
+
 		// start an idle timer to keep track of inactive/forgotten connections
 		idleTimer := time.NewTimer(ce.IdleTimeout)
 		defer idleTimer.Stop()
 
 		for {
 			select {
+			case <-maxTimer.C:
+				cancelRequest()
 			case <-idleTimer.C:
 				cancelRequest()
 			case msg, ok := <-ce.Client.Channel():
@@ -186,6 +200,21 @@ func realtimeSetSubscriptions(e *core.RequestEvent) error {
 		return e.NotFoundError("Missing or invalid client id.", err)
 	}
 
+	// for just in case to prevent someone changing a guest subscription
+	//
+	// note1: this is an extra precaution against clientId bruteforce attempts
+	// for installations allowing longer realtime connections duration
+	//
+	// note2: custom registered clients (aka. those without IP in the store)
+	// are excluded from the check for backward compatibility
+	clientIP, _ := client.Get(RealtimeClientIPKey).(string)
+	if clientIP != "" && clientIP != e.RealIP() {
+		return e.BadRequestError(
+			"Invalid realtime client.",
+			errors.New("the subscription request IP doesn't match with the realtime client IP"),
+		)
+	}
+
 	// for now allow only guest->auth upgrades and any other auth change is forbidden
 	clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 	if clientAuth != nil && !isSameAuth(clientAuth, e.Auth) {
@@ -208,7 +237,7 @@ func realtimeSetSubscriptions(e *core.RequestEvent) error {
 		e.Client.Subscribe(e.Subscriptions...)
 
 		e.App.Logger().Debug(
-			"Realtime subscriptions updated.",
+			"Realtime subscriptions updated",
 			slog.String("clientId", e.Client.Id()),
 			slog.Any("subscriptions", e.Subscriptions),
 		)
@@ -219,38 +248,47 @@ func realtimeSetSubscriptions(e *core.RequestEvent) error {
 	})
 }
 
-// updateClientsAuth updates the existing clients auth record with the new one (matched by ID).
-func realtimeUpdateClientsAuth(app core.App, newAuthRecord *core.Record) error {
+// realtimeUpdateClientsAuth updates the auth state of all clients related to the provided authRecord.
+//
+// Realtime connections has short lifetime by design, but to minimize abuse
+// if the new record has a different tokenKey (e.g. in case of password reset)
+// the auth state of the related realtime connections is also cleared
+// (aka. they remain active but unauthenticated, allowing to reauthenicate with the next subscription).
+func realtimeUpdateClientsAuth(app core.App, authRecord *core.Record) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
-					clientAuth.Id == newAuthRecord.Id &&
-					clientAuth.Collection().Name == newAuthRecord.Collection().Name {
-					client.Set(RealtimeClientAuthKey, newAuthRecord)
+					clientAuth.Id == authRecord.Id &&
+					clientAuth.Collection().Name == authRecord.Collection().Name {
+					if clientAuth.TokenKey() != authRecord.TokenKey() {
+						client.Unset(RealtimeClientAuthKey)
+					} else {
+						client.Set(RealtimeClientAuthKey, authRecord)
+					}
 				}
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
 }
 
-// realtimeUnsetClientsAuthState unsets the auth state of all clients that have the provided auth model.
-func realtimeUnsetClientsAuthState(app core.App, authModel core.Model) error {
+// realtimeUnsetClientsAuthByRecordModelOrProxy unsets the auth state of all clients that have the provided auth model.
+func realtimeUnsetClientsAuthByRecordModelOrProxy(app core.App, authModel core.Model) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
@@ -261,13 +299,82 @@ func realtimeUnsetClientsAuthState(app core.App, authModel core.Model) error {
 			}
 
 			return nil
-		})
+		}))
+	}
+
+	return group.Wait()
+}
+
+// realtimeUnsetClientsAuthByCollection unsets the auth state of all authenticated clients related to the collection.
+func realtimeUnsetClientsAuthByCollection(app core.App, collection *core.Collection) error {
+	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
+
+	group := new(errgroup.Group)
+
+	for _, chunk := range chunks {
+		group.Go(routine.SafeWrap(func() error {
+			for _, client := range chunk {
+				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
+				if clientAuth != nil && clientAuth.Collection().Name == collection.Name {
+					client.Unset(RealtimeClientAuthKey)
+				}
+			}
+
+			return nil
+		}))
 	}
 
 	return group.Wait()
 }
 
 func bindRealtimeEvents(app core.App) {
+	// reset the clients auth on collection secret change
+	// (@todo with the future tracking of old collections data consider replacing with *AfterUpdateSuccess to account for transaction rollback)
+	app.OnCollectionUpdate().Bind(&hook.Handler[*core.CollectionEvent]{
+		Func: func(e *core.CollectionEvent) error {
+			if !e.Collection.IsAuth() {
+				return e.Next()
+			}
+
+			cached, _ := e.App.FindCachedCollectionByNameOrId(e.Collection.Id)
+
+			if err := e.Next(); err != nil {
+				return err
+			}
+
+			if cached != nil && cached.AuthToken.Secret != e.Collection.AuthToken.Secret {
+				if err := realtimeUnsetClientsAuthByCollection(e.App, e.Collection); err != nil {
+					app.Logger().Warn(
+						"Failed to remove client(s) associated to the changed auth collection",
+						slog.String("collectionName", e.Collection.Name),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+
+			return nil
+		},
+		Priority: -99,
+	})
+
+	// unset the clients auth on auth collection delete
+	app.OnCollectionAfterDeleteSuccess().Bind(&hook.Handler[*core.CollectionEvent]{
+		Func: func(e *core.CollectionEvent) error {
+			if e.Collection.IsAuth() {
+				if err := realtimeUnsetClientsAuthByCollection(e.App, e.Collection); err != nil {
+					app.Logger().Warn(
+						"Failed to remove client(s) associated to the deleted auth collection",
+						slog.String("collectionName", e.Collection.Name),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+
+			return e.Next()
+		},
+		Priority: -99,
+	})
+
 	// update the clients that has auth record association
 	app.OnModelAfterUpdateSuccess().Bind(&hook.Handler[*core.ModelEvent]{
 		Func: func(e *core.ModelEvent) error {
@@ -294,7 +401,7 @@ func bindRealtimeEvents(app core.App) {
 		Func: func(e *core.ModelEvent) error {
 			collection := realtimeResolveRecordCollection(e.App, e.Model)
 			if collection != nil && collection.IsAuth() {
-				if err := realtimeUnsetClientsAuthState(e.App, e.Model); err != nil {
+				if err := realtimeUnsetClientsAuthByRecordModelOrProxy(e.App, e.Model); err != nil {
 					app.Logger().Warn(
 						"Failed to remove client(s) associated to the deleted auth model",
 						slog.Any("id", e.Model.PK()),
@@ -516,7 +623,7 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 	}
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			var clientAuth *core.Record
 
 			for _, client := range chunk {
@@ -548,6 +655,20 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 						// which exact fields the client subscription requested or has permissions to access
 						cleanRecord := record.Fresh()
 
+						// -------------------------------------------
+						// @todo consider with the refactoring whether
+						// the default enriching used by the regular APIs
+						// can be reused here too to avoid eventual future
+						// discrepencies in the record event data
+						//
+						// https://github.com/pocketbase/pocketbase/issues/7721
+						// -------------------------------------------
+
+						// enable hidden fields for superuser subscribers
+						if requestInfo.HasSuperuserAuth() {
+							cleanRecord.Unhide(collection.Fields.FieldNames()...)
+						}
+
 						// trigger the enrich hooks
 						enrichErr := triggerRecordEnrichHooks(app, requestInfo, []*core.Record{cleanRecord}, func() error {
 							// apply expand
@@ -645,7 +766,7 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -661,7 +782,7 @@ func realtimeBroadcastDryCacheKey(app core.App, key string) error {
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				messages, ok := client.Get(key).([]subscriptions.Message)
 				if !ok {
@@ -680,7 +801,7 @@ func realtimeBroadcastDryCacheKey(app core.App, key string) error {
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
@@ -696,7 +817,7 @@ func realtimeUnsetDryCacheKey(app core.App, key string) error {
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				if client.Get(key) != nil {
 					client.Unset(key)
@@ -704,7 +825,7 @@ func realtimeUnsetDryCacheKey(app core.App, key string) error {
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()

apis/realtime_test.go

@@ -2,7 +2,7 @@ package apis_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"net/http"
@@ -26,6 +26,7 @@ func TestRealtimeConnect(t *testing.T) {
 			Method:         http.MethodGet,
 			URL:            "/api/realtime",
 			Timeout:        100 * time.Millisecond,
+			Headers:        map[string]string{"x-test-ip": "127.0.0.2"},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`id:`,
@@ -37,6 +38,17 @@ func TestRealtimeConnect(t *testing.T) {
 				"OnRealtimeConnectRequest": 1,
 				"OnRealtimeMessageSend":    1,
 			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				app.OnRealtimeConnectRequest().BindFunc(func(e *core.RealtimeConnectRequestEvent) error {
+					if ip, _ := e.Client.Get(apis.RealtimeClientIPKey).(string); ip != "127.0.0.2" {
+						t.Fatalf("Expected IP %q, got %q", "127.0.0.2", ip)
+					}
+
+					return e.Next()
+				})
+			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				if len(app.SubscriptionsBroker().Clients()) != 0 {
 					t.Errorf("Expected the subscribers to be removed after connection close, found %d", len(app.SubscriptionsBroker().Clients()))
@@ -102,7 +114,8 @@ func TestRealtimeSubscribe(t *testing.T) {
 
 	resetClient := func() {
 		client.Unsubscribe()
-		client.Set(apis.RealtimeClientAuthKey, nil)
+		client.Unset(apis.RealtimeClientAuthKey)
+		client.Unset(apis.RealtimeClientIPKey)
 	}
 
 	validSubscriptionsLimit := make([]string, 1000)
@@ -208,6 +221,26 @@ func TestRealtimeSubscribe(t *testing.T) {
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
+		{
+			Name:            "existing client with different IP",
+			Method:          http.MethodPost,
+			URL:             "/api/realtime",
+			Body:            strings.NewReader(`{"clientId":"` + client.Id() + `","subscriptions":["test"]}`),
+			Headers:         map[string]string{"x-test-ip": "127.0.0.2"},
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				client.Set(apis.RealtimeClientIPKey, "127.0.0.1")
+
+				app.SubscriptionsBroker().Register(client)
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				resetClient()
+			},
+		},
 		{
 			Name:   "existing client with valid topic length",
 			Method: http.MethodPost,
@@ -429,7 +462,10 @@ func TestRealtimeAuthRecordDeleteEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -460,7 +496,10 @@ func TestRealtimeAuthRecordDeleteEvent(t *testing.T) {
 	e.Context = context.Background()
 	e.Model = authRecord1
 
-	testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	err = testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	if total := len(testApp.SubscriptionsBroker().Clients()); total != 3 {
 		t.Fatalf("Expected %d subscription clients, found %d", 3, total)
@@ -484,7 +523,10 @@ func TestRealtimeAuthRecordUpdateEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -495,25 +537,331 @@ func TestRealtimeAuthRecordUpdateEvent(t *testing.T) {
 	client.Set(apis.RealtimeClientAuthKey, authRecord1)
 	testApp.SubscriptionsBroker().Register(client)
 
-	// refetch the authRecord and change its email
+	// refetch the authRecord and change its name
 	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
 		t.Fatal(err)
 	}
-	authRecord2.SetEmail("new@example.com")
 
-	// mock update event
-	e := new(core.ModelEvent)
-	e.App = testApp
-	e.Type = core.ModelEventTypeUpdate
-	e.Context = context.Background()
-	e.Model = authRecord2
+	newName := "test_new_name"
+	authRecord2.Set("name", newName)
 
-	testApp.OnModelAfterUpdateSuccess().Trigger(e)
+	err = testApp.Save(authRecord2)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	clientAuthRecord, _ := client.Get(apis.RealtimeClientAuthKey).(*core.Record)
-	if clientAuthRecord.Email() != authRecord2.Email() {
-		t.Fatalf("Expected authRecord with email %q, got %q", authRecord2.Email(), clientAuthRecord.Email())
+	if clientAuthRecord.Get("name") != newName {
+		t.Fatalf("Expected authRecord with email %q, got %q", newName, clientAuthRecord.Email())
+	}
+}
+
+func TestRealtimeRecordHiddenFields(t *testing.T) {
+	t.Parallel()
+
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// create temp collection with hidden fields
+	testCollection := core.NewBaseCollection("test_realtime")
+	testCollection.ListRule = types.Pointer("@request.auth.id != ''")
+	testCollection.Fields.Add(
+		&core.TextField{Name: "public"},
+		&core.TextField{Name: "hidden", Hidden: true},
+	)
+	if err := testApp.Save(testCollection); err != nil {
+		t.Fatal(err)
+	}
+
+	testSubscription := testCollection.Name + "/*"
+
+	// register guest subscriber
+	guestClient := subscriptions.NewDefaultClient()
+	guestClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(guestClient)
+
+	// register regular user subscriber
+	regular, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	regularClient := subscriptions.NewDefaultClient()
+	regularClient.Set(apis.RealtimeClientAuthKey, regular)
+	regularClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(regularClient)
+
+	// register superuser subscriber
+	superuser, err := testApp.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	superuserClient := subscriptions.NewDefaultClient()
+	superuserClient.Set(apis.RealtimeClientAuthKey, superuser)
+	superuserClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(superuserClient)
+
+	enrichCalls := map[string]int{}
+	testApp.OnRecordEnrich(testCollection.Name).BindFunc(func(e *core.RecordEnrichEvent) error {
+		var id string
+		if e.RequestInfo.Auth != nil {
+			id = e.RequestInfo.Auth.Id
+		}
+		enrichCalls[id]++
+		return e.Next()
+	})
+
+	timeout := time.After(3 * time.Second)
+	done := make(chan struct{})
+
+	// collect first received messages
+	var regularMessageData, superuserMessageData string
+	go func() {
+		regularMessageData = string((<-regularClient.Channel()).Data)
+		superuserMessageData = string((<-superuserClient.Channel()).Data)
+		done <- struct{}{}
+	}()
+
+	// broadcast create message
+	testRecord := core.NewRecord(testCollection)
+	testRecord.Set("public", "test1")
+	testRecord.Set("hidden", "test2")
+	if err := testApp.Save(testRecord); err != nil {
+		t.Fatal(err)
+	}
+
+	// wait for the events
+	select {
+	case <-timeout:
+		t.Fatal("realtime test messages timeout")
+	case <-done:
+		// ready
+	}
+
+	if total := len(enrichCalls); total != 2 {
+		t.Fatalf("Expected %d enrich hook calls, got %d", 2, total)
+	}
+
+	if total := enrichCalls[regular.Id]; total != 1 {
+		t.Fatalf("Expected exactly 1 regular user enrich hook call, got %d", total)
+	}
+
+	if total := enrichCalls[superuser.Id]; total != 1 {
+		t.Fatalf("Expected exactly 1 superuser enrich hook call, got %d", total)
+	}
+
+	// validate messages content
+	scenarios := map[string]bool{
+		"regular message public field should exist":     strings.Contains(regularMessageData, `"public":`),
+		"regular message hidden field should NOT exist": !strings.Contains(regularMessageData, `"hidden":`),
+		"superuser message public field should exist":   strings.Contains(superuserMessageData, `"public":`),
+		"superuser message hidden field should exist":   strings.Contains(superuserMessageData, `"hidden":`),
+	}
+	for name, valid := range scenarios {
+		t.Run(name, func(t *testing.T) {
+			if !valid {
+				t.Fatal("Invalid realtime message expectation")
+			}
+		})
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnTokenKeyRefresh(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client := subscriptions.NewDefaultClient()
+	client.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client)
+
+	// refetch the authRecord and refresh its tokenKey
+	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	authRecord2.RefreshTokenKey()
+
+	err = testApp.Save(authRecord2)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientAuthRecord, _ := client.Get(apis.RealtimeClientAuthKey).(*core.Record)
+	if clientAuthRecord != nil {
+		t.Fatalf("Expected authRecord to be unset, got %q", clientAuthRecord.Email())
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnCollectionSecretChange(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	usersCollection, err := testApp.FindCollectionByNameOrId("users")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientsCollection, err := testApp.FindCollectionByNameOrId("clients")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+
+	authRecord2, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord2)
+
+	authRecord3, err := testApp.FindAuthRecordByEmail(clientsCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord3)
+
+	clientMocks := map[*core.Record]subscriptions.Client{
+		authRecord1: client1,
+		authRecord2: client2,
+		authRecord3: client3,
+	}
+	for _, client := range clientMocks {
+		testApp.SubscriptionsBroker().Register(client)
+	}
+
+	// change the secret of the users collection (should trigger unset)
+	usersCollection.AuthToken.Secret = strings.Repeat("a", 30)
+	err = testApp.Save(usersCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// change something else of the clients collection (shouldn't trigger unset)
+	clientsCollection.ListRule = nil
+	err = testApp.Save(clientsCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expectations := map[*core.Record]bool{
+		// record -> unset
+		authRecord1: true,
+		authRecord2: true,
+		authRecord3: false,
+	}
+	for record, expectedUnset := range expectations {
+		clientAuthRecord, _ := clientMocks[record].Get(apis.RealtimeClientAuthKey).(*core.Record)
+		unset := clientAuthRecord == nil
+		if unset != expectedUnset {
+			t.Fatalf("Expected unset state %v, got %v (%v)", expectedUnset, unset, clientAuthRecord)
+		}
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnCollectionDelete(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	usersCollection, err := testApp.FindCollectionByNameOrId("users")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientsCollection, err := testApp.FindCollectionByNameOrId("clients")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+
+	authRecord2, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord2)
+
+	authRecord3, err := testApp.FindAuthRecordByEmail(clientsCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord3)
+
+	clientMocks := map[*core.Record]subscriptions.Client{
+		authRecord1: client1,
+		authRecord2: client2,
+		authRecord3: client3,
+	}
+	for _, client := range clientMocks {
+		testApp.SubscriptionsBroker().Register(client)
+	}
+
+	// mock users collection delete event to avoid triggering constraints check
+	e := new(core.ModelEvent)
+	e.App = testApp
+	e.Type = core.ModelEventTypeDelete
+	e.Context = context.Background()
+	e.Model = usersCollection
+
+	err = testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expectations := map[*core.Record]bool{
+		// record -> unset
+		authRecord1: true,
+		authRecord2: true,
+		authRecord3: false,
+	}
+	for record, expectedUnset := range expectations {
+		clientAuthRecord, _ := clientMocks[record].Get(apis.RealtimeClientAuthKey).(*core.Record)
+		unset := clientAuthRecord == nil
+		if unset != expectedUnset {
+			t.Fatalf("Expected unset state %v, got %v (%v)", expectedUnset, unset, clientAuthRecord)
+		}
 	}
 }
 
@@ -551,7 +899,10 @@ func TestRealtimeCustomAuthModelDeleteEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -608,7 +959,10 @@ func TestRealtimeCustomAuthModelUpdateEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -817,7 +1171,9 @@ func TestRealtimeRecordResolve(t *testing.T) {
 
 			var mu sync.Mutex
 			notify := func(clientId string, eventData []byte) {
-				data := struct{ Action string }{}
+				data := struct {
+					Action string `json:"action"`
+				}{}
 				_ = json.Unmarshal(eventData, &data)
 
 				mu.Lock()

apis/record_auth_email_change_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
@@ -102,12 +102,6 @@ func (form *EmailChangeConfirmForm) parseToken() (*core.Record, string, error) {
 		return nil, "", validation.NewError("validation_invalid_token_payload", "Invalid token payload - newEmail must be set.")
 	}
 
-	// ensure that there aren't other users with the new email
-	_, err := form.app.FindAuthRecordByEmail(form.collection, newEmail)
-	if err == nil {
-		return nil, "", validation.NewError("validation_existing_token_email", "The new email address is already registered: "+newEmail)
-	}
-
 	// verify that the token is not expired and its signature is valid
 	authRecord, err := form.app.FindAuthRecordByToken(form.Token, core.TokenTypeEmailChange)
 	if err != nil {
@@ -118,5 +112,11 @@ func (form *EmailChangeConfirmForm) parseToken() (*core.Record, string, error) {
 		return nil, "", validation.NewError("validation_token_collection_mismatch", "The provided token is for different auth collection.")
 	}
 
+	// check if there are other users with the new email
+	_, err = form.app.FindAuthRecordByEmail(form.collection, newEmail)
+	if err == nil {
+		return nil, "", validation.NewError("validation_invalid_token_email", "The new email address is invalid.")
+	}
+
 	return authRecord, newEmail, nil
 }

apis/record_auth_email_change_confirm_test.go

@@ -111,12 +111,51 @@ func TestRecordConfirmEmailChange(t *testing.T) {
 				"OnRecordUpdateExecute":             1,
 				"OnRecordAfterUpdateSuccess":        1,
 				"OnRecordValidate":                  1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatalf("Expected the user to be unverified before the confirmation")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
-				_, err := app.FindAuthRecordByEmail("users", "change@example.com")
+				user, err := app.FindAuthRecordByEmail("users", "change@example.com")
 				if err != nil {
 					t.Fatalf("Expected to find user with email %q, got error: %v", "change@example.com", err)
 				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{

apis/record_auth_email_change_request.go

@@ -3,8 +3,8 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 )

apis/record_auth_impersonate.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/record_auth_methods_test.go

@@ -54,7 +54,7 @@ func TestRecordAuthMethodsList(t *testing.T) {
 				`"providers":[{`,
 				`"name":"google"`,
 				`"name":"gitlab"`,
-				`"logo":"\u003csvg`,
+				`"logo":"<svg`,
 				`"logo":""`, // for the legacy fields
 				`"state":`,
 				`"displayName":`,

apis/record_auth_otp_request.go

@@ -6,8 +6,8 @@ import (
 	"fmt"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_password_reset_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/security"

apis/record_auth_password_reset_confirm_test.go

@@ -114,11 +114,18 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				"OnModelUpdate":                       1,
 				"OnModelUpdateExecute":                1,
 				"OnModelAfterUpdateSuccess":           1,
-				"OnModelValidate":                     1,
 				"OnRecordUpdate":                      1,
 				"OnRecordUpdateExecute":               1,
 				"OnRecordAfterUpdateSuccess":          1,
+				"OnModelValidate":                     1,
 				"OnRecordValidate":                    1,
+				// ---
+				"OnModelDelete":              2, // pre-existing OAuth2 links
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
@@ -151,6 +158,15 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				if !user.ValidatePassword("1234567!") {
 					t.Fatal("Password wasn't changed")
 				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -221,6 +237,15 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				if !user.ValidatePassword("1234567!") {
 					t.Fatal("Password wasn't changed")
 				}
+
+				// ensure that all pre-existing OAuth2 were NOT deleted
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) != 2 {
+					t.Fatalf("Expected 2 external auths, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -251,11 +276,20 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 					t.Fatalf("Failed to fetch confirm password user: %v", err)
 				}
 
+				oldTokenKey := user.TokenKey()
+
 				// ensure that the user is already verified
 				user.SetVerified(true)
 				if err := app.Save(user); err != nil {
 					t.Fatalf("Failed to update user verified state")
 				}
+
+				// resave with the old token key since the verified change above
+				// would refresh it and will make the password token invalid
+				user.SetTokenKey(oldTokenKey)
+				if err = app.Save(user); err != nil {
+					t.Fatalf("Failed to restore original user tokenKey: %v", err)
+				}
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				_, err := app.FindAuthRecordByToken(

apis/record_auth_password_reset_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_verification_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"
@@ -45,6 +45,13 @@ func recordConfirmVerification(e *core.RequestEvent) error {
 		if !wasVerified {
 			e.Record.SetVerified(true)
 
+			// similar to the OTP auth, we enforce an extra password reset
+			// guard as this way is less prone to pre-hijacking attacks
+			// in case the password auth is eventually enabled later
+			if !e.Record.Collection().PasswordAuth.Enabled {
+				e.Record.SetRandomPassword()
+			}
+
 			if err := e.App.Save(e.Record); err != nil {
 				return firstApiError(err, e.BadRequestError("An error occurred while saving the verified state.", err))
 			}

apis/record_auth_verification_confirm_test.go

@@ -105,6 +105,130 @@ func TestRecordConfirmVerification(t *testing.T) {
 				"OnRecordValidate":                   1,
 				"OnRecordUpdateExecute":              1,
 				"OnRecordAfterUpdateSuccess":         1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatal("Expected the user to be unverified before the confirmation")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
+			},
+		},
+		{
+			Name:   "valid token (disabled password auth)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/confirm-verification",
+			Body: strings.NewReader(`{
+				"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6InZlcmlmaWNhdGlvbiIsImNvbGxlY3Rpb25JZCI6Il9wYl91c2Vyc19hdXRoXyIsImVtYWlsIjoidGVzdEBleGFtcGxlLmNvbSJ9.SetHpu2H-x-q4TIUz-xiQjwi7MNwLCLvSs4O0hUSp0E"
+			}`),
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{
+				"*":                                  0,
+				"OnRecordConfirmVerificationRequest": 1,
+				"OnModelUpdate":                      1,
+				"OnModelValidate":                    1,
+				"OnModelUpdateExecute":               1,
+				"OnModelAfterUpdateSuccess":          1,
+				"OnRecordUpdate":                     1,
+				"OnRecordValidate":                   1,
+				"OnRecordUpdateExecute":              1,
+				"OnRecordAfterUpdateSuccess":         1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				user.Collection().PasswordAuth.Enabled = false
+				if err = app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatal("Expected the user to be unverified before the confirmation")
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatal("Expected password to be valid")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				if user.ValidatePassword("1234567890") {
+					t.Fatal("Expected the user password to be reset")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{

apis/record_auth_verification_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"

apis/record_auth_with_oauth2.go

@@ -4,7 +4,7 @@ import (
 	"bytes"
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"io"
@@ -17,8 +17,8 @@ import (
 	"syscall"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
@@ -203,7 +203,6 @@ func (form *recordOAuth2LoginForm) validate() error {
 	return validation.ValidateStruct(form,
 		validation.Field(&form.Provider, validation.Required, validation.Length(0, 100), validation.By(form.checkProviderName)),
 		validation.Field(&form.Code, validation.Required),
-		validation.Field(&form.RedirectURL, validation.Required),
 	)
 }
 
@@ -338,26 +337,43 @@ func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *cor
 				e.Auth.Id == e.Record.Id &&
 				e.Auth.Collection().Id == e.Record.Collection().Id
 
-			// set random password for users with unverified email
-			// (this is in case a malicious actor has registered previously with the user email)
-			if !isLoggedAuthRecord && e.Record.Email() != "" && !e.Record.Verified() {
-				e.Record.SetRandomPassword()
+			// prevent pre-hijacking with password auth
+			//
+			// reset the unverified user password in case the record was precreated by a malicious actor
+			if !isLoggedAuthRecord && !e.Record.Verified() {
 				needUpdate = true
+				e.Record.SetRandomPassword()
+			}
+
+			// prevent pre-hijacking with different OAuth2 provider
+			//
+			// delete all other previous OAuth2 record links for the cases
+			// when the user was precreated by malicious OAuth2 auth with custom payload data
+			//
+			// while this would be also done automatically on unverified -> verified upgrade,
+			// doing it manually here ensures that a single unverified record could have
+			// max 1 OAuth2 link to prevent further abuse when mixed with other auth flows
+			if !e.Record.Verified() {
+				err := txApp.DeleteAllExternalAuthsByRecord(e.Record)
+				if err != nil {
+					return err
+				}
+				optExternalAuth = nil // clear to allow recreate below
 			}
 
 			// update the existing auth record empty email if the data.OAuth2User has one
 			// (this is in case previously the auth record was created
 			// with an OAuth2 provider that didn't return an email address)
 			if e.Record.Email() == "" && e.OAuth2User.Email != "" {
-				e.Record.SetEmail(e.OAuth2User.Email)
 				needUpdate = true
+				e.Record.SetEmail(e.OAuth2User.Email)
 			}
 
 			// update the existing auth record verified state
 			// (only if the auth record doesn't have an email or the auth record email match with the one in data.OAuth2User)
 			if !e.Record.Verified() && (e.Record.Email() == "" || e.Record.Email() == e.OAuth2User.Email) {
-				e.Record.SetVerified(true)
 				needUpdate = true
+				e.Record.SetVerified(true)
 			}
 
 			if needUpdate {

apis/record_auth_with_oauth2_redirect.go

@@ -1,7 +1,7 @@
 package apis
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"net/http"
 	"strings"
@@ -55,6 +55,16 @@ func oauth2SubscriptionRedirect(e *core.RequestEvent) error {
 	}
 	defer client.Unsubscribe(oauth2SubscriptionTopic)
 
+	// additional check to minimize the risk of XSRF attack vectors
+	//
+	// note: custom registered clients (aka. those without IP in the store)
+	// are excluded from the check for backward compatibility
+	clientIP, _ := client.Get(RealtimeClientIPKey).(string)
+	if clientIP != "" && clientIP != e.RealIP() {
+		e.App.Logger().Debug("The client IP that completed the authentication is different from the one that initialized the OAuth2 realtime connection")
+		return failureRedirect(e)
+	}
+
 	// temporary store the Apple user's name so that it can be later retrieved with the authWithOAuth2 call
 	// (see https://github.com/pocketbase/pocketbase/issues/7090)
 	if data.AppleUser != "" && data.Error == "" && data.Code != "" {

apis/record_auth_with_oauth2_redirect_test.go

@@ -8,6 +8,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
@@ -16,9 +17,9 @@ import (
 func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 	t.Parallel()
 
-	clientStubs := make([]map[string]subscriptions.Client, 0, 10)
+	clientStubs := make([]map[string]subscriptions.Client, 0, 11)
 
-	for i := 0; i < 10; i++ {
+	for i := 0; i < 11; i++ {
 		c1 := subscriptions.NewDefaultClient()
 
 		c2 := subscriptions.NewDefaultClient()
@@ -335,6 +336,28 @@ func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:    "client with different IP",
+			Method:  http.MethodGet,
+			URL:     "/api/oauth2-redirect?code=123&state=" + clientStubs[10]["c3"].Id(),
+			Headers: map[string]string{"x-test-ip": "127.0.0.2"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				clientStubs[10]["c3"].Set(apis.RealtimeClientIPKey, "127.0.0.1")
+
+				beforeTestFunc(clientStubs[10], map[string][]string{
+					"c3": {`"state":"` + clientStubs[10]["c3"].Id(), `"code":"123"`},
+				})(t, app, e)
+			},
+			ExpectedStatus: http.StatusTemporaryRedirect,
+			ExpectedEvents: map[string]int{"*": 0},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				app.Store().Get("cancelFunc").(context.CancelFunc)()
+
+				checkFailureRedirect(t, app, res)
+			},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/record_auth_with_oauth2_test.go

@@ -90,10 +90,11 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"data":{`,
 				`"provider":`,
 				`"code":`,
-				`"redirectURL":`,
 			},
 			NotExpectedContent: []string{
-				`"codeVerifier":`, // should be optional
+				// should be optional
+				`"codeVerifier":`,
+				`"redirectURL":`,
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
@@ -109,10 +110,11 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"data":{`,
 				`"provider":`,
 				`"code":`,
-				`"redirectURL":`,
 			},
 			NotExpectedContent: []string{
-				`"codeVerifier":`, // should be optional
+				// should be optional
+				`"codeVerifier":`,
+				`"redirectURL":`,
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
@@ -178,6 +180,20 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 					t.Fatal(err)
 				}
 
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that it was deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Id != ea.Id {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
 				// test at least once that the correct request info context is properly loaded
 				app.OnRecordAuthRequest().BindFunc(func(e *core.RecordAuthRequestEvent) error {
 					info, err := e.RequestInfo()
@@ -213,12 +229,12 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordAuthRequest":           1,
 				"OnRecordEnrich":                1,
 				// ---
-				"OnModelCreate":              1,
-				"OnModelCreateExecute":       1,
-				"OnModelAfterCreateSuccess":  1,
-				"OnRecordCreate":             1,
-				"OnRecordCreateExecute":      1,
-				"OnRecordAfterCreateSuccess": 1,
+				"OnModelCreate":              2, // user + recreated external auth
+				"OnModelCreateExecute":       2,
+				"OnModelAfterCreateSuccess":  2,
+				"OnRecordCreate":             2,
+				"OnRecordCreateExecute":      2,
+				"OnRecordAfterCreateSuccess": 2,
 				// ---
 				"OnModelUpdate":              1,
 				"OnModelUpdateExecute":       1,
@@ -227,8 +243,15 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
-				"OnModelValidate":  2, // create + update
-				"OnRecordValidate": 2,
+				"OnModelDelete":              3, // pre-existing external auths
+				"OnModelDeleteExecute":       3,
+				"OnModelAfterDeleteSuccess":  3,
+				"OnRecordDelete":             3,
+				"OnRecordDeleteExecute":      3,
+				"OnRecordAfterDeleteSuccess": 3,
+				// ---
+				"OnModelValidate":  3, // user create/update + recreated external auth
+				"OnRecordValidate": 3,
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
@@ -248,6 +271,24 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				if len(devices) != 1 {
 					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
 				}
+
+				// ensure that other linked external auths have been deleted
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) != 1 {
+					t.Fatalf("Expected only 1 external auth to remain, got %d", len(externalAuths))
+				}
+				if provider := externalAuths[0].Provider(); provider != "test" {
+					t.Fatalf("Expected %q external auth, got %q", "test", provider)
+				}
+				if providerId := externalAuths[0].ProviderId(); providerId != "test_id" {
+					t.Fatalf("Expected %q providerId, got %q", "test_id", providerId)
+				}
+				if recordRef := externalAuths[0].RecordRef(); recordRef != user.Id {
+					t.Fatalf("Expected %q recordRef, got %q", user.Id, recordRef)
+				}
+				if collectionRef := externalAuths[0].CollectionRef(); collectionRef != user.Collection().Id {
+					t.Fatalf("Expected %q collectionRef, got %q", user.Collection().Id, collectionRef)
+				}
 			},
 		},
 		{
@@ -343,7 +384,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected old password %q to be valid", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -353,7 +394,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 		},
 		{
-			Name:   "link by email",
+			Name:   "link by email (unverified user)",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/auth-with-oauth2",
 			Body: strings.NewReader(`{
@@ -376,6 +417,20 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 					t.Fatalf("Expected password %q to be valid", "1234567890")
 				}
 
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that it was deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Provider() != "test" {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
 				// register the test provider
 				auth.Providers["test"] = func() auth.Provider {
 					return &oauth2MockProvider{
@@ -432,6 +487,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -449,6 +511,145 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				if len(devices) != 1 {
 					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
 				}
+
+				// ensure that other linked external auths have been deleted
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) != 1 {
+					t.Fatalf("Expected only 1 external auth to remain, got %d", len(externalAuths))
+				}
+				if provider := externalAuths[0].Provider(); provider != "test" {
+					t.Fatalf("Expected %q external auth, got %q", "test", provider)
+				}
+				if providerId := externalAuths[0].ProviderId(); providerId != "test_id" {
+					t.Fatalf("Expected %q providerId, got %q", "test_id", providerId)
+				}
+				if recordRef := externalAuths[0].RecordRef(); recordRef != user.Id {
+					t.Fatalf("Expected %q recordRef, got %q", user.Id, recordRef)
+				}
+				if collectionRef := externalAuths[0].CollectionRef(); collectionRef != user.Collection().Id {
+					t.Fatalf("Expected %q collectionRef, got %q", user.Collection().Id, collectionRef)
+				}
+			},
+		},
+		{
+			Name:   "link by email (verified user)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"123",
+				"redirectURL": "https://example.com"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test3@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected user %q to be verified", user.Email())
+				}
+
+				// ensure that the old password works
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected password %q to be valid", "1234567890")
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{Id: "test_id", Email: "test3@example.com"},
+						Token:    &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that they are not deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Provider() != "test" {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
+				// add the test provider in the collection
+				user.Collection().MFA.Enabled = false
+				user.Collection().OAuth2.Enabled = true
+				user.Collection().OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"record":{`,
+				`"token":"`,
+				`"meta":{`,
+				`"isNew":false`,
+				`"email":"test3@example.com"`,
+				`"id":"bgs820n361vj1qd"`,
+				`"id":"test_id"`,
+				`"verified":true`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                             0,
+				"OnRecordAuthWithOAuth2Request": 1,
+				"OnRecordAuthRequest":           1,
+				"OnRecordEnrich":                1,
+				// ---
+				"OnModelCreate":              2, // authOrigins + externalAuths
+				"OnModelCreateExecute":       2,
+				"OnModelAfterCreateSuccess":  2,
+				"OnRecordCreate":             2,
+				"OnRecordCreateExecute":      2,
+				"OnRecordAfterCreateSuccess": 2,
+				// ---
+				"OnModelValidate":  2, // authOrigins + externalAuths
+				"OnRecordValidate": 2,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test3@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
+				}
+
+				devices, err := app.FindAllAuthOriginsByRecord(user)
+				if len(devices) != 1 {
+					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
+				}
+
+				var hasTestEA = false
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) <= 1 {
+					t.Fatalf("Expected to have 2+ ExternalAuth records, got %d (%v)", len(externalAuths), err)
+				}
+				for _, rel := range externalAuths {
+					if rel.Provider() == "test" {
+						hasTestEA = true
+						break
+					}
+				}
+				if !hasTestEA {
+					t.Fatal("Expected test external auth to be linked")
+				}
 			},
 		},
 		{
@@ -531,6 +732,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordCreateExecute":      2,
 				"OnRecordAfterCreateSuccess": 2,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  2,
 				"OnRecordValidate": 2,
 			},
@@ -541,7 +749,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -652,6 +860,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -662,7 +877,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -758,6 +973,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -768,7 +990,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)

apis/record_auth_with_otp.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"fmt"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 
@@ -65,28 +65,38 @@ func recordAuthWithOTP(e *core.RequestEvent) error {
 	// ---
 
 	return e.App.OnRecordAuthWithOTPRequest().Trigger(event, func(e *core.RecordAuthWithOTPRequestEvent) error {
+		otpId := e.OTP.Id
+		otpSentTo := e.OTP.SentTo()
+
+		// eagerly delete the OTP to avoid unnecessary double delete model hook calls
+		// triggered by the password change below
+		err := e.App.Delete(e.OTP)
+		if err != nil {
+			e.App.Logger().Error("Failed to delete used OTP", "error", err, "otpId", e.OTP.Id)
+		}
+
 		// update the user email verified state in case the OTP originate from an email address matching the current record one
 		//
 		// note: don't wait for success auth response (it could fail because of MFA) and because we already validated the OTP above
-		otpSentTo := e.OTP.SentTo()
 		if !e.Record.Verified() && otpSentTo != "" && e.Record.Email() == otpSentTo {
 			e.Record.SetVerified(true)
-			err = e.App.Save(e.Record)
-			if err != nil {
+
+			// this is technically not required but we enforce password
+			// reset on verified upgrades in case the OTP is used on its own
+			// since this makes it less error prone to pre-hijacking attacks
+			if !e.Record.Collection().MFA.Enabled {
+				e.Record.SetRandomPassword()
+			}
+
+			if err := e.App.Save(e.Record); err != nil {
 				e.App.Logger().Error("Failed to update record verified state after successful OTP validation",
 					"error", err,
-					"otpId", e.OTP.Id,
+					"otpId", otpId,
 					"recordId", e.Record.Id,
 				)
 			}
 		}
 
-		// try to delete the used otp
-		err = e.App.Delete(e.OTP)
-		if err != nil {
-			e.App.Logger().Error("Failed to delete used OTP", "error", err, "otpId", e.OTP.Id)
-		}
-
 		return RecordAuthResponse(e.RequestEvent, e.Record, core.MFAMethodOTP, nil)
 	})
 }

apis/record_auth_with_otp_test.go

@@ -327,6 +327,15 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if user.Verified() {
 					t.Fatal("Expected the user to remain unverified because sentTo != email")
 				}
+
+				// ensure that all pre-existing OAuth2 were NOT deleted
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) != 2 {
+					t.Fatalf("Expected 2 external auths, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -364,6 +373,15 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if err := app.Save(otp); err != nil {
 					t.Fatal(err)
 				}
+
+				// verify that there are at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auth")
+				}
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
@@ -388,10 +406,10 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				"OnModelCreate":             1,
 				"OnModelCreateExecute":      1,
 				"OnModelAfterCreateSuccess": 1,
-				// OTP delete
-				"OnModelDelete":             1,
-				"OnModelDeleteExecute":      1,
-				"OnModelAfterDeleteSuccess": 1,
+				// record OTP + 2 ExternalAuths delete
+				"OnModelDelete":             3,
+				"OnModelDeleteExecute":      3,
+				"OnModelAfterDeleteSuccess": 3,
 				// user verified update
 				"OnModelUpdate":             1,
 				"OnModelUpdateExecute":      1,
@@ -401,9 +419,9 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				"OnRecordCreate":             1,
 				"OnRecordCreateExecute":      1,
 				"OnRecordAfterCreateSuccess": 1,
-				"OnRecordDelete":             1,
-				"OnRecordDeleteExecute":      1,
-				"OnRecordAfterDeleteSuccess": 1,
+				"OnRecordDelete":             3,
+				"OnRecordDeleteExecute":      3,
+				"OnRecordAfterDeleteSuccess": 3,
 				"OnRecordUpdate":             1,
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
@@ -417,6 +435,24 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if !user.Verified() {
 					t.Fatal("Expected the user to be marked as verified")
 				}
+
+				// ensure that all pre-existing OTPs are cleared
+				otps, err := app.FindAllOTPsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(otps) > 0 {
+					t.Fatalf("Expected all OTPs to be cleared, found %d", len(otps))
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{

apis/record_auth_with_password.go

@@ -6,9 +6,9 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/list"
@@ -85,6 +85,11 @@ func recordAuthWithPassword(e *core.RequestEvent) error {
 
 	return e.App.OnRecordAuthWithPasswordRequest().Trigger(event, func(e *core.RecordAuthWithPasswordRequestEvent) error {
 		if e.Record == nil || !e.Record.ValidatePassword(e.Password) {
+			// dummy password check to minimize enumeration side-channel attacks
+			if e.Record == nil {
+				dummyPasswordCheck(e.App, e.Collection)
+			}
+
 			return e.BadRequestError("Failed to authenticate.", errors.New("invalid login credentials"))
 		}
 
@@ -115,6 +120,21 @@ func (form *authWithPasswordForm) validate(collection *core.Collection) error {
 	)
 }
 
+// dummy password check to minimize side-channel attacks
+// (performed with the collection configured field cost)
+func dummyPasswordCheck(app core.App, collection *core.Collection) {
+	record := &core.Record{}
+
+	// find any random existing record
+	err := app.RecordQuery(collection).Limit(1).One(record)
+	if err != nil {
+		return
+	}
+
+	// the value and result doesn't matter, we just need a constant-time check
+	_ = record.ValidatePassword("")
+}
+
 func findRecordByIdentityField(app core.App, collection *core.Collection, field string, value any) (*core.Record, error) {
 	if !slices.Contains(collection.PasswordAuth.IdentityFields, field) {
 		return nil, errors.New("invalid identity field " + field)

apis/record_helpers.go

@@ -43,6 +43,13 @@ func RecordAuthResponse(e *core.RequestEvent, authRecord *core.Record, authMetho
 }
 
 func recordAuthResponse(e *core.RequestEvent, authRecord *core.Record, token string, authMethod string, meta any) error {
+	if authRecord.IsSuperuser() {
+		allowedIPs := e.App.Settings().SuperuserIPs
+		if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+			return e.ForbiddenError("", errors.New("superuser IP is not whitelisted"))
+		}
+	}
+
 	originalRequestInfo, err := e.RequestInfo()
 	if err != nil {
 		return err

apis/record_helpers_test.go

@@ -1,7 +1,7 @@
 package apis_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"net/http"
 	"net/http/httptest"
@@ -759,3 +759,39 @@ func TestRecordAuthResponseMFACheck(t *testing.T) {
 		}
 	})
 }
+
+func TestRecordAuthResponseSuperuserIPsWhitelistCheck(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+	event := new(core.RequestEvent)
+	event.App = app
+	event.Request = httptest.NewRequest(http.MethodGet, "/", nil)
+	event.Request.Header.Set("x-test-ip", "127.0.0.1")
+	event.Response = httptest.NewRecorder()
+
+	t.Run("non-whitelisted", func(t *testing.T) {
+		app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+		err = apis.RecordAuthResponse(event, superuser, "example", nil)
+		if err == nil {
+			t.Fatal("Expected response error, got nil")
+		}
+	})
+
+	t.Run("whitelisted", func(t *testing.T) {
+		app.Settings().SuperuserIPs = []string{"0.0.0.0", "127.0.0.1"}
+
+		err = apis.RecordAuthResponse(event, superuser, "example", nil)
+		if err != nil {
+			t.Fatal(err)
+		}
+	})
+}

apis/serve.go

@@ -22,7 +22,7 @@ import (
 	"golang.org/x/crypto/acme/autocert"
 )
 
-const defaultCSP = "default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' http://127.0.0.1:* https://tile.openstreetmap.org data: blob:; connect-src 'self' http://127.0.0.1:* https://nominatim.openstreetmap.org; script-src 'self' http://127.0.0.1:*; frame-src 'none'"
+const defaultCSP = "default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' http://127.0.0.1:* https://tile.openstreetmap.org data: blob:; connect-src 'self' http://127.0.0.1:* https://nominatim.openstreetmap.org; script-src 'self' http://127.0.0.1:*; frame-ancestors 'none'"
 
 // ServeConfig defines a configuration struct for apis.Serve().
 type ServeConfig struct {

apis/settings.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/tools/router"

apis/sql.go

@@ -0,0 +1,194 @@
+package apis
+
+import (
+	"context"
+	"errors"
+	"log/slog"
+	"net/http"
+	"strings"
+	"time"
+
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/router"
+)
+
+const (
+	runSQLMaxRows    = 1000
+	runSQLMaxTimeout = 3 * time.Minute
+)
+
+// bindSQLApi registers the SQL api endpoints.
+func bindSQLApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
+	subGroup := rg.Group("/sql").Bind(RequireSuperuserAuth())
+	subGroup.POST("", runSQL)
+}
+
+func runSQL(e *core.RequestEvent) error {
+	// extra precaution in case manually invoked from somewhere else
+	if !e.HasSuperuserAuth() {
+		return e.ForbiddenError("", nil)
+	}
+
+	form := runSQLForm{}
+
+	err := e.BindBody(&form)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while loading the submitted data.", err))
+	}
+
+	err = form.validate()
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
+	}
+
+	result, err := executeQuery(e.App, form.Query, runSQLMaxRows)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("Failed to execute query. Raw error:\n"+err.Error(), nil))
+	}
+
+	return e.JSON(http.StatusOK, result)
+}
+
+type runSQLForm struct {
+	Query string `form:"query" json:"query"`
+}
+
+func (form *runSQLForm) validate() error {
+	return validation.ValidateStruct(form,
+		validation.Field(&form.Query, validation.Required, validation.Length(0, 5000)),
+	)
+}
+
+type runSQLResultColumn struct {
+	Name     string `json:"name"`
+	Type     string `json:"type"`
+	Nullable bool   `json:"nullable"`
+}
+
+type runSQLResult struct {
+	ExecTime     int64                `json:"execTime"`
+	AffectedRows int64                `json:"affectedRows"`
+	Columns      []runSQLResultColumn `json:"columns"`
+	Rows         [][]any              `json:"rows"`
+}
+
+var knownWriteQueryPrefixes = []string{
+	"INSERT", "CREATE", "UPDATE", "DELETE",
+	"DROP", "DETACH", "ALTER", "REPLACE",
+}
+
+func executeQuery(app core.App, query string, maxRows int) (*runSQLResult, error) {
+	query = strings.TrimSpace(query)
+	if query == "" {
+		// see https://github.com/mattn/go-sqlite3/issues/950
+		return nil, errors.New("empty query")
+	}
+
+	var isPossibleWriteQuery bool
+
+	// loosely check the query type
+	ucQuery := strings.ToUpper(query)
+	if !strings.HasPrefix(ucQuery, "SELECT") {
+		for _, prefix := range knownWriteQueryPrefixes {
+			if strings.HasPrefix(ucQuery, prefix) {
+				isPossibleWriteQuery = true
+				break
+			}
+		}
+	}
+
+	// note: don't extend the request context to minimize the risk of
+	// causing integrity issues with custom non-transaction mutations
+	ctx, cancelFunc := context.WithTimeout(context.Background(), runSQLMaxTimeout)
+	defer cancelFunc()
+
+	result := &runSQLResult{
+		// init empty slices to ensure "[]" serialization
+		Columns: []runSQLResultColumn{},
+		Rows:    [][]any{},
+	}
+
+	now := time.Now()
+	defer func() {
+		result.ExecTime = time.Since(now).Milliseconds()
+	}()
+
+	// assume write/mutation query
+	// ---------------------------------------------------------------
+	if isPossibleWriteQuery {
+		// auto wrap in transaction in case there are multiple inline queries
+		txErr := app.RunInTransaction(func(txApp core.App) error {
+			execResult, err := txApp.NonconcurrentDB().NewQuery(query).WithContext(ctx).Execute()
+			if err != nil {
+				return err
+			}
+
+			result.AffectedRows, err = execResult.RowsAffected()
+			if err != nil {
+				// non-critical error (e.g. not supported by the driver)
+				txApp.Logger().Debug("Unable to fetch affected rows", slog.String("error", err.Error()))
+			}
+
+			return nil
+		})
+		if txErr != nil {
+			return nil, txErr
+		}
+
+		return result, nil
+	}
+
+	// assume query returning rows
+	// ---------------------------------------------------------------
+	rows, err := app.ConcurrentDB().NewQuery(query).WithContext(ctx).Rows()
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	// populate columns info
+	// ---
+	colTypes, err := rows.ColumnTypes()
+	if err != nil {
+		return nil, err
+	}
+
+	for _, colType := range colTypes {
+		col := runSQLResultColumn{
+			Name: colType.Name(),
+			Type: colType.DatabaseTypeName(),
+		}
+		col.Nullable, _ = colType.Nullable()
+
+		result.Columns = append(result.Columns, col)
+	}
+
+	// populate rows
+	// ---
+	for rows.Next() {
+		if len(result.Rows) >= maxRows {
+			break
+		}
+
+		rowData := make([]any, len(colTypes))
+		for i := 0; i < len(colTypes); i++ {
+			var v *string
+			rowData[i] = &v
+		}
+
+		err := rows.Scan(rowData...)
+		if err != nil {
+			return nil, err
+		}
+
+		result.Rows = append(result.Rows, rowData)
+	}
+
+	err = rows.Err()
+	if err != nil {
+		return nil, err
+	}
+
+	return result, nil
+}

apis/sql_test.go

@@ -0,0 +1,220 @@
+package apis_test
+
+import (
+	"net/http"
+	"strings"
+	"testing"
+
+	"github.com/pocketbase/pocketbase/tests"
+)
+
+func TestSQLRun(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "guest",
+			Method:          http.MethodPost,
+			URL:             "/api/sql",
+			Body:            strings.NewReader(`{"query":"select 1"}`),
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "regular user",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1"}`),
+			Headers: map[string]string{
+				// users, test2@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6Im9hcDY0MGNvdDR5cnUycyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.GfJo6EHIobgas_AXt-M-tj5IoQendPnrkMSe9ExuSEY",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[{"name":"1","type":"","nullable":true}]`,
+				`"rows":[["1"]]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "empty query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":""}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"query":{`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "invalid query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"invalid"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+				`SQL logic error`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "query with length above the limit",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"` + strings.Repeat("a", 5001) + `"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"query":{`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "query with length equal to the limit",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select '` + strings.Repeat("a", 4985) + `' as id"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[{"name":"id","type":"","nullable":true}]`,
+				`"rows":[["aaa`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "single write query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key)"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if !app.HasTable("test_sql_table") {
+					t.Fatalf("Missing expected new %q table", "test_sql_table")
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[]`,
+				`"rows":[]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple write queries",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key);insert into test_sql_table(id)VALUES(1)"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				var total int
+				err := app.DB().NewQuery("select count(*) from test_sql_table").Row(&total)
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if total != 1 {
+					t.Fatalf("Expected exactly 1 row, found: %d", total)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":1`,
+				`"columns":[]`,
+				`"rows":[]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple write queries (transaction rollback)",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key);insert into test_sql_table(id)VALUES(1);invalid"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if app.HasTable("test_sql_table") {
+					t.Fatalf("Expected table %q to not be created", "test_sql_table")
+				}
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+				`SQL logic error`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple read queries",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1;select 2"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				// only the result of the last query should be returned
+				`"columns":[{"name":"2","type":"","nullable":true}]`,
+				`"rows":[["2"]]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

cmd/superuser.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 
 	"github.com/fatih/color"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cobra"
@@ -24,6 +24,7 @@ func NewSuperuserCommand(app core.App) *cobra.Command {
 	command.AddCommand(superuserUpdateCommand(app))
 	command.AddCommand(superuserDeleteCommand(app))
 	command.AddCommand(superuserOTPCommand(app))
+	command.AddCommand(superuserIPsCommand(app))
 
 	return command
 }
@@ -209,3 +210,38 @@ func superuserOTPCommand(app core.App) *cobra.Command {
 
 	return command
 }
+
+func superuserIPsCommand(app core.App) *cobra.Command {
+	command := &cobra.Command{
+		Use:          "ips",
+		Example:      "superuser ips 127.0.0.1 10.0.0.0/24",
+		Short:        "Updates the superuser IPs whitelist setting (the IPs/subnets arguments must be space separated; leave empty to clear the whitelist restriction)",
+		SilenceUsage: true,
+		RunE: func(command *cobra.Command, args []string) error {
+			settings := app.Settings()
+			settings.SuperuserIPs = args
+
+			if err := app.Save(settings); err != nil {
+				return err
+			}
+
+			if len(args) == 0 {
+				color.Green("Successfully cleared SuperuserIPs setting!")
+			} else {
+				color.New(color.BgGreen, color.FgBlack).Println("Successfully updated SuperuserIPs setting:")
+				superuserIPs := app.Settings().SuperuserIPs
+				for i, ip := range superuserIPs {
+					if i == len(superuserIPs)-1 {
+						color.Green("└─ %s", ip)
+					} else {
+						color.Green("├─ %s", ip)
+					}
+				}
+			}
+
+			return nil
+		},
+	}
+
+	return command
+}

cmd/superuser_test.go

@@ -1,6 +1,7 @@
 package cmd_test
 
 import (
+	"slices"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/cmd"
@@ -401,3 +402,63 @@ func TestSuperuserOTPCommand(t *testing.T) {
 		})
 	}
 }
+
+func TestSuperuserIPsCommand(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	scenarios := []struct {
+		name        string
+		ips         []string
+		expectError bool
+	}{
+		{
+			"no ips",
+			nil,
+			false,
+		},
+		{
+			"invalid ips",
+			[]string{"127.0.0.1", "invalid"},
+			true,
+		},
+		{
+			"valid ips",
+			[]string{"127.0.0.1", "::1", "127.0.0.1/24"},
+			false,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			args := []string{"ips"}
+			args = append(args, s.ips...)
+
+			command := cmd.NewSuperuserCommand(app)
+			command.SetArgs(args)
+
+			err := command.Execute()
+
+			hasErr := err != nil
+			if s.expectError != hasErr {
+				t.Fatalf("Expected hasErr %v, got %v (%v)", s.expectError, hasErr, err)
+			}
+
+			if hasErr {
+				return
+			}
+
+			settingIPs := app.Settings().SuperuserIPs
+
+			if len(settingIPs) != len(s.ips) {
+				t.Fatalf("Expected %d ips, got %d (%v)", len(s.ips), len(settingIPs), settingIPs)
+			}
+
+			for _, ip := range settingIPs {
+				if !slices.Contains(s.ips, ip) {
+					t.Fatalf("Missing expected ip %q (%v)", ip, settingIPs)
+				}
+			}
+		})
+	}
+}

core/app.go

@@ -54,11 +54,16 @@ type App interface {
 	// Bootstrap initializes the application
 	// (aka. create data dir, open db connections, load settings, etc.).
 	//
-	// It will call ResetBootstrapState() if the application was already bootstrapped.
+	// It calls ClearBootstrap() if the application was already bootstrapped.
 	Bootstrap() error
 
-	// ResetBootstrapState releases the initialized core app resources
+	// ClearBootstrap releases the initialized core app resources
 	// (closing db connections, stopping cron ticker, etc.).
+	//
+	// This method is no-op if the application is not bootstrapped yet.
+	ClearBootstrap() error
+
+	// Deprecated: use ClearBootstrap().
 	ResetBootstrapState() error
 
 	// DataDir returns the app data directory path.
@@ -502,6 +507,11 @@ type App interface {
 	// ExternalAuth model that satisfies the non-nil expression.
 	FindFirstExternalAuthByExpr(expr dbx.Expression) (*ExternalAuth, error)
 
+	// DeleteAllExternalAuthsByRecord deletes all ExternalAuth models associated with the provided record.
+	//
+	// Returns a combined error with the failed deletes.
+	DeleteAllExternalAuthsByRecord(authRecord *Record) error
+
 	// ---------------------------------------------------------------
 
 	// FindAllMFAsByRecord returns all MFA models linked to the provided auth record.
@@ -708,6 +718,13 @@ type App interface {
 	// resources (db, app settings, etc).
 	OnBootstrap() *hook.Hook[*BootstrapEvent]
 
+	// OnBootstrapClear hook is triggered when clearing the main application
+	// resources (db connections, cron, logger, etc.)
+	//
+	// It is usually invoked automatically right before app termination
+	// or when manually calling app.ClearBootstrap().
+	OnBootstrapClear() *hook.Hook[*BootstrapEvent]
+
 	// OnServe hook is triggered when the app web server is started
 	// (after starting the TCP listener but before initializing the blocking serve task),
 	// allowing you to adjust its options and attach new routes or middlewares.
@@ -1257,6 +1274,20 @@ type App interface {
 	// triggered and called only if their event data origin matches the tags.
 	OnMailerRecordOTPSend(tags ...string) *hook.TaggedHook[*MailerRecordEvent]
 
+	// ---------------------------------------------------------------
+	// Filesystem event hooks
+	// (not publicly exposed until file_field refactoring)
+	// ---------------------------------------------------------------
+
+	// onFilesystemNewWriter is an internal hook for app.NewFilesystem()
+	// instances that is triggered on every storage filesystem writer initialization
+	// (aka. whenever attempting to create a new file).
+	onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent]
+
+	// onFilesystemDelete is an internal hook for app.NewFilesystem()
+	// instances that is triggered for every storage file delete call.
+	onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent]
+
 	// ---------------------------------------------------------------
 	// Realtime API event hooks
 	// ---------------------------------------------------------------

core/auth_origin_model.go

@@ -5,8 +5,8 @@ import (
 	"errors"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/backup.go

@@ -0,0 +1,137 @@
+package core
+
+import (
+	"context"
+	"fmt"
+	"log/slog"
+	"sort"
+	"time"
+
+	"github.com/pocketbase/pocketbase/tools/inflector"
+)
+
+const (
+	StoreKeyActiveBackup = "@activeBackup"
+)
+
+// generateBackupName generates a new backup name based on the app name and current date.
+func generateBackupName(app App, prefix string) string {
+	appName := inflector.Snakecase(app.Settings().Meta.AppName)
+	if len(appName) > 50 {
+		appName = appName[:50]
+	}
+
+	return fmt.Sprintf(
+		"%s%s_%s.zip",
+		prefix,
+		appName,
+		time.Now().UTC().Format("20060102150405"),
+	)
+}
+
+// registerAutobackupHooks registers the autobackup app serve hooks.
+func (app *BaseApp) registerAutobackupHooks() {
+	const jobId = "__pbAutoBackup__"
+
+	loadJob := func() {
+		rawSchedule := app.Settings().Backups.Cron
+		if rawSchedule == "" {
+			app.Cron().Remove(jobId)
+			return
+		}
+
+		app.Cron().Add(jobId, rawSchedule, func() {
+			const autoPrefix = "@auto_pb_backup_"
+
+			name := generateBackupName(app, autoPrefix)
+
+			if err := app.CreateBackup(context.Background(), name); err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to create backup",
+					slog.String("name", name),
+					slog.String("error", err.Error()),
+				)
+
+				alertError := sendSystemAlertToAllSuperusers(
+					app,
+					"Autobackup failure",
+					"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
+				)
+				if alertError != nil {
+					app.Logger().Warn(
+						"[Backup cron] Failed to send backup error alerts",
+						slog.String("name", name),
+						slog.String("error", alertError.Error()),
+					)
+				}
+			}
+
+			maxKeep := app.Settings().Backups.CronMaxKeep
+
+			if maxKeep == 0 {
+				return // no explicit limit
+			}
+
+			fsys, err := app.NewBackupsFilesystem()
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to initialize the backup filesystem",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+			defer fsys.Close()
+
+			files, err := fsys.List(autoPrefix)
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to list autogenerated backups",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+
+			if maxKeep >= len(files) {
+				return // nothing to remove
+			}
+
+			// sort desc
+			sort.Slice(files, func(i, j int) bool {
+				return files[i].ModTime.After(files[j].ModTime)
+			})
+
+			// keep only the most recent n auto backup files
+			toRemove := files[maxKeep:]
+
+			for _, f := range toRemove {
+				if err := fsys.Delete(f.Key); err != nil {
+					app.Logger().Error(
+						"[Backup cron] Failed to remove old autogenerated backup",
+						slog.String("key", f.Key),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+		})
+	}
+
+	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+
+	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+}

core/backup_create.go

@@ -0,0 +1,437 @@
+package core
+
+import (
+	"archive/zip"
+	"compress/flate"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"log/slog"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
+	"github.com/pocketbase/pocketbase/tools/hook"
+	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/store"
+)
+
+var errIsDir = errors.New("the specified path is a directory and not a regular file")
+
+// CreateBackup creates a new backup of the current app pb_data directory.
+//
+// If name is empty, it will be autogenerated.
+// If backup with the same name exists, the new backup file will replace it.
+//
+// To safely perform the backup, it is recommended to have free disk space
+// for at least 2x the size of the pb_data directory.
+//
+// By default backups are stored in pb_data/backups
+// (the backups directory itself is excluded from the generated backup).
+//
+// Backups can be stored on S3 if it is configured in app.Settings().Backups.
+// When using S3 storage for the uploaded collection files, you have to
+// take care manually to backup those since they are not part of the pb_data.
+//
+// DB write locks are minimal and isolated only for the duration of the
+// VACUUM INTO statement that creates a live copy of the app database.
+//
+// The backup works as follow:
+//
+//  1. Start listening for DELETED storage files.
+//     In case a file is being deleted while the backup is still ongoing,
+//     we directly copy it in the zip before the deletion and mark it as "excluded".
+//
+//  2. Copy the main database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  3. Stop listening for DELETED files.
+//
+//  4. Start listening for NEW storage files and mark all new files from this point as "excluded".
+//
+//  5. Copy the logs database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  6. Copy the rest of the pb_data files in the zip while ignoring the "excluded" list (it should be concurrent safe).
+//
+//  7. Stop listening for NEW storage files.
+//
+// While there is a risk for a race condition between steps 1, 2 and 3, it is an
+// acceptable trade-off between performance and correctness because in
+// the worst case there will be some unused storage files in the backup that don't do any harm.
+func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
+	if app.Store().Has(StoreKeyActiveBackup) {
+		return errors.New("try again later - another backup/restore operation has already been started")
+	}
+
+	app.Store().Set(StoreKeyActiveBackup, name)
+	defer app.Store().Remove(StoreKeyActiveBackup)
+
+	startTime := time.Now()
+
+	event := new(BackupEvent)
+	event.App = app
+	event.Context = ctx
+	event.Name = name
+	// default root dir entries to exclude from the backup generation
+	event.Exclude = []string{
+		LocalBackupsDirName,
+		LocalTempDirName,
+		LocalNotifyDirName,
+		LocalAutocertCacheDirName,
+		lostFoundDirName,
+	}
+
+	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
+		if e.Name == "" {
+			e.Name = generateBackupName(e.App, "pb_backup_")
+		}
+
+		app.Logger().Debug("[" + e.Name + "] zip archive started")
+
+		// create backup zip
+		// (it needs to be inside the current pb_data to avoid "cross-device link" errors)
+		// -----------------------------------------------------------
+		tempZipPath := filepath.Join(app.DataDir(), LocalTempDirName, "pb_backup_"+security.PseudorandomString(6))
+		err := createZip(e, tempZipPath)
+		if err != nil {
+			return err
+		}
+		defer os.Remove(tempZipPath)
+
+		// persist the backup in the backups filesystem
+		// -----------------------------------------------------------
+		fsys, err := e.App.NewBackupsFilesystem()
+		if err != nil {
+			return err
+		}
+		defer fsys.Close()
+
+		fsys.SetContext(e.Context)
+
+		file, err := filesystem.NewFileFromPath(tempZipPath)
+		if err != nil {
+			return err
+		}
+		file.OriginalName = e.Name
+		file.Name = file.OriginalName
+
+		err = fsys.UploadFile(file, file.Name)
+		if err != nil {
+			return err
+		}
+
+		app.Logger().Debug(
+			"["+e.Name+"] zip archive completed",
+			slog.Float64("execTime", float64(time.Since(startTime))/float64(time.Millisecond)),
+		)
+
+		return nil
+	})
+}
+
+func createZip(be *BackupEvent, tempZipPath string) error {
+	logPrefix := "[" + be.Name + "] "
+
+	// make sure that the special temp directory exists
+	localTempDir := filepath.Dir(tempZipPath)
+	if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
+		return fmt.Errorf(logPrefix+"failed to create temp dir: %w", err)
+	}
+
+	const tempFilesHookId = "__pbTempBackupFilesystemWatcher__"
+	defer func() {
+		// unbind again in cacase of an error
+		be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+		be.App.onFilesystemNewWriter().Unbind(tempFilesHookId)
+	}()
+
+	zipper, err := newZipWriter(tempZipPath)
+	if err != nil {
+		return err
+	}
+	defer zipper.close()
+
+	excluded := store.New[string, struct{}](nil)
+	for _, name := range be.Exclude {
+		excluded.Set(normalizePathExclude(name), struct{}{})
+	}
+
+	// init deleted files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemDelete().Bind(&hook.Handler[*FilesystemDeleteEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemDeleteEvent) error {
+			// note: the zip header name allow only forward slashes
+			zipPath := path.Join(LocalStorageDirName, e.FileKey)
+
+			if excluded.Has(normalizePathExclude(zipPath)) || be.App.Settings().S3.Enabled {
+				return e.Next()
+			}
+
+			localPath := filepath.Join(
+				be.App.DataDir(),
+				LocalStorageDirName,
+				e.FileKey,
+			)
+
+			// copy to zip before delete
+			err := zipper.copyFileToZip(localPath, zipPath)
+			if err != nil {
+				// it is ok to ignore directories
+				if !errors.Is(err, errIsDir) {
+					be.App.Logger().Warn(
+						logPrefix+"failed to copy file in backup zip before delete",
+						slog.Any("error", err),
+						slog.String("file", e.FileKey),
+					)
+				}
+			} else {
+				// mark that it was already copied
+				excluded.Set(normalizePathExclude(zipPath), struct{}{})
+			}
+
+			// proceed with the normal deletion
+			return e.Next()
+		},
+	})
+
+	// copy data.db
+	// ---------------------------------------------------------------
+	dataStartTime := time.Now()
+	tempDataDBPath := filepath.Join(localTempDir, dataDBFilename)
+
+	_, err = be.App.ConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempDataDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	// eagerly stop listening for deleted files since we already have what we needed
+	be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+
+	be.App.Logger().Debug(
+		logPrefix+dataDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(dataStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempDataDBPath, dataDBFilename)
+	if err != nil {
+		_ = os.Remove(tempDataDBPath)
+		return err
+	}
+	_ = os.Remove(tempDataDBPath)
+
+	excluded.Set(normalizePathExclude(dataDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-shm"), struct{}{})
+
+	// init to-be-created files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemNewWriter().Bind(&hook.Handler[*FilesystemNewWriterEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemNewWriterEvent) error {
+			if !be.App.Settings().S3.Enabled {
+				// mark for exclude even if the writer eventually fails
+				// (all record files have random name so collisions are unlikely)
+				name := normalizePathExclude(filepath.Join(LocalStorageDirName, e.FileKey))
+				excluded.Set(name, struct{}{})
+			}
+
+			return e.Next()
+		},
+	})
+
+	// copy auxiliary.db
+	// ---------------------------------------------------------------
+	auxStartTime := time.Now()
+	tempAuxDBPath := filepath.Join(localTempDir, auxDBFilename)
+
+	_, err = be.App.AuxConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempAuxDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	be.App.Logger().Debug(
+		logPrefix+auxDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(auxStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempAuxDBPath, auxDBFilename)
+	if err != nil {
+		_ = os.Remove(tempAuxDBPath)
+		return err
+	}
+	_ = os.Remove(tempAuxDBPath)
+
+	excluded.Set(normalizePathExclude(auxDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-shm"), struct{}{})
+
+	// try to run manual checkpoints to ensure that all wal writes during the
+	// previous VACUUM INTO are transferred and don't accumulate
+	// (errors are ignore because some drivers may not support the wal_checkpoint pragma)
+	// ---------------------------------------------------------------
+	_, _ = be.App.NonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+	_, _ = be.App.AuxNonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+
+	// copy the rest of the pb_data
+	// ---------------------------------------------------------------
+	err = zipper.copyDirToZip(os.DirFS(be.App.DataDir()), excluded)
+	if err != nil {
+		return err
+	}
+
+	return zipper.close()
+}
+
+// normalize the provided file path to always end with forward slash
+func normalizePathExclude(filePath string) string {
+	return path.Clean(filePath) + "/"
+}
+
+type zipWriter struct {
+	mu     sync.Mutex
+	w      *zip.Writer
+	f      *os.File
+	closed bool
+}
+
+func newZipWriter(zipFilePath string) (*zipWriter, error) {
+	f, err := os.Create(zipFilePath)
+	if err != nil {
+		return nil, err
+	}
+
+	w := zip.NewWriter(f)
+	w.RegisterCompressor(zip.Deflate, func(out io.Writer) (io.WriteCloser, error) {
+		return flate.NewWriter(out, flate.BestSpeed)
+	})
+
+	return &zipWriter{
+		w: w,
+		f: f,
+	}, nil
+}
+
+func (z *zipWriter) close() error {
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return nil
+	}
+
+	z.closed = true
+
+	return errors.Join(z.w.Close(), z.f.Close())
+}
+
+func (z *zipWriter) copyFileToZip(localPath string, zipPath string) error {
+	info, err := os.Stat(localPath)
+	if err != nil {
+		return err
+	}
+
+	if info.IsDir() {
+		return errIsDir
+	}
+
+	h, err := zip.FileInfoHeader(info)
+	if err != nil {
+		return err
+	}
+
+	h.Name = zipPath
+	h.Method = zip.Deflate
+
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return errors.New("zip writer is already closed")
+	}
+
+	fw, err := z.w.CreateHeader(h)
+	if err != nil {
+		return err
+	}
+
+	f, err := os.Open(localPath)
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+
+	_, err = io.Copy(fw, f)
+
+	return err
+}
+
+func (z *zipWriter) copyDirToZip(fsys fs.FS, excludedPrefixes *store.Store[string, struct{}]) error {
+	return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+
+		// skip excluded prefixes
+		if excludedPrefixes != nil {
+			check := normalizePathExclude(name)
+			prefixes := excludedPrefixes.Keys() // refetch to avoid races
+			for _, prefix := range prefixes {
+				if strings.HasPrefix(check, prefix) {
+					if d.IsDir() {
+						return fs.SkipDir
+					}
+					return nil
+				}
+			}
+		}
+
+		if d.IsDir() {
+			return nil
+		}
+
+		info, err := d.Info()
+		if err != nil {
+			return err
+		}
+
+		h, err := zip.FileInfoHeader(info)
+		if err != nil {
+			return err
+		}
+
+		h.Name = name
+		h.Method = zip.Deflate
+
+		z.mu.Lock()
+		defer z.mu.Unlock()
+
+		if z.closed {
+			// note: fs.WalkDir perform direct comparison with the value
+			return fs.SkipAll
+		}
+
+		fw, err := z.w.CreateHeader(h)
+		if err != nil {
+			return err
+		}
+
+		f, err := fsys.Open(name)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+
+		_, err = io.Copy(fw, f)
+
+		return err
+	})
+}

core/backup_restore.go

@@ -9,111 +9,12 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
-	"sort"
-	"time"
 
 	"github.com/pocketbase/pocketbase/tools/archive"
-	"github.com/pocketbase/pocketbase/tools/filesystem"
-	"github.com/pocketbase/pocketbase/tools/inflector"
 	"github.com/pocketbase/pocketbase/tools/osutils"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
 
-const (
-	StoreKeyActiveBackup = "@activeBackup"
-)
-
-// CreateBackup creates a new backup of the current app pb_data directory.
-//
-// If name is empty, it will be autogenerated.
-// If backup with the same name exists, the new backup file will replace it.
-//
-// The backup is executed within a transaction, meaning that new writes
-// will be temporary "blocked" until the backup file is generated.
-//
-// To safely perform the backup, it is recommended to have free disk space
-// for at least 2x the size of the pb_data directory.
-//
-// By default backups are stored in pb_data/backups
-// (the backups directory itself is excluded from the generated backup).
-//
-// When using S3 storage for the uploaded collection files, you have to
-// take care manually to backup those since they are not part of the pb_data.
-//
-// Backups can be stored on S3 if it is configured in app.Settings().Backups.
-func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
-	if app.Store().Has(StoreKeyActiveBackup) {
-		return errors.New("try again later - another backup/restore operation has already been started")
-	}
-
-	app.Store().Set(StoreKeyActiveBackup, name)
-	defer app.Store().Remove(StoreKeyActiveBackup)
-
-	event := new(BackupEvent)
-	event.App = app
-	event.Context = ctx
-	event.Name = name
-	// default root dir entries to exclude from the backup generation
-	event.Exclude = []string{LocalBackupsDirName, LocalTempDirName, LocalAutocertCacheDirName, lostFoundDirName}
-
-	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
-		// generate a default name if missing
-		if e.Name == "" {
-			e.Name = generateBackupName(e.App, "pb_backup_")
-		}
-
-		// make sure that the special temp directory exists
-		// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
-		localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
-		if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
-			return fmt.Errorf("failed to create a temp dir: %w", err)
-		}
-
-		// archive pb_data in a temp directory, excluding the "backups" and the temp dirs
-		//
-		// run in transaction to temporary block other writes (transactions uses the NonconcurrentDB connection)
-		// ---
-		tempPath := filepath.Join(localTempDir, "pb_backup_"+security.PseudorandomString(6))
-		createErr := e.App.RunInTransaction(func(txApp App) error {
-			return txApp.AuxRunInTransaction(func(txApp App) error {
-				// run manual checkpoint and truncate the WAL files
-				// (errors are ignored because it is not that important and the PRAGMA may not be supported by the used driver)
-				txApp.DB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-				txApp.AuxDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-
-				return archive.Create(txApp.DataDir(), tempPath, e.Exclude...)
-			})
-		})
-		if createErr != nil {
-			return createErr
-		}
-		defer os.Remove(tempPath)
-
-		// persist the backup in the backups filesystem
-		// ---
-		fsys, err := e.App.NewBackupsFilesystem()
-		if err != nil {
-			return err
-		}
-		defer fsys.Close()
-
-		fsys.SetContext(e.Context)
-
-		file, err := filesystem.NewFileFromPath(tempPath)
-		if err != nil {
-			return err
-		}
-		file.OriginalName = e.Name
-		file.Name = file.OriginalName
-
-		if err := fsys.UploadFile(file, file.Name); err != nil {
-			return err
-		}
-
-		return nil
-	})
-}
-
 // RestoreBackup restores the backup with the specified name and restarts
 // the current running application process.
 //
@@ -293,111 +194,3 @@ func (app *BaseApp) RestoreBackup(ctx context.Context, name string) error {
 		return nil
 	})
 }
-
-// registerAutobackupHooks registers the autobackup app serve hooks.
-func (app *BaseApp) registerAutobackupHooks() {
-	const jobId = "__pbAutoBackup__"
-
-	loadJob := func() {
-		rawSchedule := app.Settings().Backups.Cron
-		if rawSchedule == "" {
-			app.Cron().Remove(jobId)
-			return
-		}
-
-		app.Cron().Add(jobId, rawSchedule, func() {
-			const autoPrefix = "@auto_pb_backup_"
-
-			name := generateBackupName(app, autoPrefix)
-
-			if err := app.CreateBackup(context.Background(), name); err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to create backup",
-					slog.String("name", name),
-					slog.String("error", err.Error()),
-				)
-			}
-
-			maxKeep := app.Settings().Backups.CronMaxKeep
-
-			if maxKeep == 0 {
-				return // no explicit limit
-			}
-
-			fsys, err := app.NewBackupsFilesystem()
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to initialize the backup filesystem",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-			defer fsys.Close()
-
-			files, err := fsys.List(autoPrefix)
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to list autogenerated backups",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-
-			if maxKeep >= len(files) {
-				return // nothing to remove
-			}
-
-			// sort desc
-			sort.Slice(files, func(i, j int) bool {
-				return files[i].ModTime.After(files[j].ModTime)
-			})
-
-			// keep only the most recent n auto backup files
-			toRemove := files[maxKeep:]
-
-			for _, f := range toRemove {
-				if err := fsys.Delete(f.Key); err != nil {
-					app.Logger().Error(
-						"[Backup cron] Failed to remove old autogenerated backup",
-						slog.String("key", f.Key),
-						slog.String("error", err.Error()),
-					)
-				}
-			}
-		})
-	}
-
-	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-
-	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-}
-
-func generateBackupName(app App, prefix string) string {
-	appName := inflector.Snakecase(app.Settings().Meta.AppName)
-	if len(appName) > 50 {
-		appName = appName[:50]
-	}
-
-	return fmt.Sprintf(
-		"%s%s_%s.zip",
-		prefix,
-		appName,
-		time.Now().UTC().Format("20060102150405"),
-	)
-}

core/backup_test.go

@@ -126,11 +126,7 @@ func verifyBackupContent(app core.App, path string) error {
 	expectedRootEntries := []string{
 		"storage",
 		"data.db",
-		"data.db-shm",
-		"data.db-wal",
 		"auxiliary.db",
-		"auxiliary.db-shm",
-		"auxiliary.db-wal",
 		".gitignore",
 	}
 

core/base.go

@@ -12,6 +12,7 @@ import (
 	"regexp"
 	"runtime"
 	"strings"
+	"sync/atomic"
 	"time"
 
 	"github.com/fatih/color"
@@ -38,11 +39,15 @@ const (
 
 	LocalStorageDirName       string = "storage"
 	LocalBackupsDirName       string = "backups"
-	LocalTempDirName          string = ".pb_temp_to_delete" // temp pb_data sub directory that will be deleted on each app.Bootstrap()
 	LocalAutocertCacheDirName string = ".autocert_cache"
+	LocalNotifyDirName        string = ".notify"            // optional watched directory that is used as a cross-platform workaround for synchronizing various runtime states between multiple PocketBase instances pointing to the same pb_data
+	LocalTempDirName          string = ".pb_temp_to_delete" // temp pb_data sub directory that will be deleted on each app.Bootstrap()
 
 	// @todo consider removing after backups refactoring
 	lostFoundDirName string = "lost+found"
+
+	dataDBFilename string = "data.db"
+	auxDBFilename  string = "auxiliary.db"
 )
 
 // FilesManager defines an interface with common methods that files manager models should implement.
@@ -85,11 +90,12 @@ type BaseApp struct {
 	auxNonconcurrentDB  dbx.Builder
 
 	// app event hooks
-	onBootstrap     *hook.Hook[*BootstrapEvent]
-	onServe         *hook.Hook[*ServeEvent]
-	onTerminate     *hook.Hook[*TerminateEvent]
-	onBackupCreate  *hook.Hook[*BackupEvent]
-	onBackupRestore *hook.Hook[*BackupEvent]
+	onBootstrap      *hook.Hook[*BootstrapEvent]
+	onBootstrapClear *hook.Hook[*BootstrapEvent]
+	onServe          *hook.Hook[*ServeEvent]
+	onTerminate      *hook.Hook[*TerminateEvent]
+	onBackupCreate   *hook.Hook[*BackupEvent]
+	onBackupRestore  *hook.Hook[*BackupEvent]
 
 	// db model hooks
 	onModelValidate           *hook.Hook[*ModelEvent]
@@ -145,6 +151,17 @@ type BaseApp struct {
 	onMailerRecordOTPSend           *hook.Hook[*MailerRecordEvent]
 	onMailerRecordAuthAlertSend     *hook.Hook[*MailerRecordEvent]
 
+	// filesystem event hooks
+	//
+	// @todo 1:
+	// intentionally not exposed since the events are too "chatty" and
+	// can cause unnecessary userland tests breaking changes;
+	// reevaluate once refactoring the file_field
+	//
+	// @todo 2: if exposed consider registering the same for the backup filesystem
+	_onFilesystemNewWriter *hook.Hook[*FilesystemNewWriterEvent]
+	_onFilesystemDelete    *hook.Hook[*FilesystemDeleteEvent]
+
 	// realtime api event hooks
 	onRealtimeConnectRequest   *hook.Hook[*RealtimeConnectRequestEvent]
 	onRealtimeMessageSend      *hook.Hook[*RealtimeMessageEvent]
@@ -234,6 +251,7 @@ func NewBaseApp(config BaseAppConfig) *BaseApp {
 func (app *BaseApp) initHooks() {
 	// app event hooks
 	app.onBootstrap = &hook.Hook[*BootstrapEvent]{}
+	app.onBootstrapClear = &hook.Hook[*BootstrapEvent]{}
 	app.onServe = &hook.Hook[*ServeEvent]{}
 	app.onTerminate = &hook.Hook[*TerminateEvent]{}
 	app.onBackupCreate = &hook.Hook[*BackupEvent]{}
@@ -293,6 +311,10 @@ func (app *BaseApp) initHooks() {
 	app.onMailerRecordOTPSend = &hook.Hook[*MailerRecordEvent]{}
 	app.onMailerRecordAuthAlertSend = &hook.Hook[*MailerRecordEvent]{}
 
+	// filesystem event hooks
+	app._onFilesystemNewWriter = &hook.Hook[*FilesystemNewWriterEvent]{}
+	app._onFilesystemDelete = &hook.Hook[*FilesystemDeleteEvent]{}
+
 	// realtime API event hooks
 	app.onRealtimeConnectRequest = &hook.Hook[*RealtimeConnectRequestEvent]{}
 	app.onRealtimeMessageSend = &hook.Hook[*RealtimeMessageEvent]{}
@@ -386,14 +408,14 @@ func (app *BaseApp) IsBootstrapped() bool {
 // Bootstrap initializes the application
 // (aka. create data dir, open db connections, load settings, etc.).
 //
-// It will call ResetBootstrapState() if the application was already bootstrapped.
+// It calls ClearBootstrap() if the application was already bootstrapped.
 func (app *BaseApp) Bootstrap() error {
 	event := &BootstrapEvent{}
 	event.App = app
 
 	err := app.OnBootstrap().Trigger(event, func(e *BootstrapEvent) error {
-		// clear resources of previous core state (if any)
-		if err := app.ResetBootstrapState(); err != nil {
+		// clear previous bootstrap state (if any)
+		if err := app.ClearBootstrap(); err != nil {
 			return err
 		}
 
@@ -441,41 +463,55 @@ func (app *BaseApp) Bootstrap() error {
 	return err
 }
 
-type closer interface {
-	Close() error
+// Deprecated: use [ClearBootstrap].
+func (app *BaseApp) ResetBootstrapState() error {
+	return app.ClearBootstrap()
 }
 
-// ResetBootstrapState releases the initialized core app resources
+// ClearBootstrap releases the initialized core app resources
 // (closing db connections, stopping cron ticker, etc.).
-func (app *BaseApp) ResetBootstrapState() error {
-	app.Cron().Stop()
-
-	var errs []error
-
-	dbs := []*dbx.Builder{
-		&app.concurrentDB,
-		&app.nonconcurrentDB,
-		&app.auxConcurrentDB,
-		&app.auxNonconcurrentDB,
+//
+// This method is no-op if the application is not bootstrapped yet.
+func (app *BaseApp) ClearBootstrap() error {
+	if !app.IsBootstrapped() {
+		return nil
 	}
 
-	for _, db := range dbs {
-		if db == nil {
-			continue
+	event := &BootstrapEvent{}
+	event.App = app
+
+	return app.OnBootstrapClear().Trigger(event, func(e *BootstrapEvent) error {
+		type closer interface {
+			Close() error
 		}
-		if v, ok := (*db).(closer); ok {
-			if err := v.Close(); err != nil {
-				errs = append(errs, err)
+
+		var errs []error
+
+		dbs := []*dbx.Builder{
+			&app.concurrentDB,
+			&app.nonconcurrentDB,
+			&app.auxConcurrentDB,
+			&app.auxNonconcurrentDB,
+		}
+
+		for _, db := range dbs {
+			if db == nil {
+				continue
 			}
+			if v, ok := (*db).(closer); ok {
+				if err := v.Close(); err != nil {
+					errs = append(errs, err)
+				}
+			}
+			*db = nil
 		}
-		*db = nil
-	}
 
-	if len(errs) > 0 {
-		return errors.Join(errs...)
-	}
+		if len(errs) > 0 {
+			return errors.Join(errs...)
+		}
 
-	return nil
+		return nil
+	})
 }
 
 // DB returns the default app data.db builder instance.
@@ -711,9 +747,10 @@ func (app *BaseApp) NewMailClient() mailer.Mailer {
 //
 // NB! Make sure to call Close() on the returned result
 // after you are done working with it.
-func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
+func (app *BaseApp) NewFilesystem() (fsys *filesystem.System, err error) {
 	if app.settings != nil && app.settings.S3.Enabled {
-		return filesystem.NewS3(
+		// S3
+		fsys, err = filesystem.NewS3(
 			app.settings.S3.Bucket,
 			app.settings.S3.Region,
 			app.settings.S3.Endpoint,
@@ -721,10 +758,41 @@ func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
 			app.settings.S3.Secret,
 			app.settings.S3.ForcePathStyle,
 		)
+	} else {
+		// local filesystem
+		fsys, err = filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	}
+	if err != nil {
+		return nil, err
 	}
 
-	// fallback to local filesystem
-	return filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	// attach delete hook
+	if app._onFilesystemDelete.Length() > 0 {
+		fsys.OnDelete().BindFunc(func(originalEvent *filesystem.DeleteEvent) error {
+			appEvent := new(FilesystemDeleteEvent)
+			appEvent.DeleteEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemDelete.Trigger(appEvent, func(fde *FilesystemDeleteEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	// attach write hook
+	if app._onFilesystemNewWriter.Length() > 0 {
+		fsys.OnNewWriter().BindFunc(func(originalEvent *filesystem.NewWriterEvent) error {
+			appEvent := new(FilesystemNewWriterEvent)
+			appEvent.NewWriterEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemNewWriter.Trigger(appEvent, func(fwe *FilesystemNewWriterEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	return fsys, nil
 }
 
 // NewBackupsFilesystem creates a new local or S3 filesystem instance
@@ -766,7 +834,7 @@ func (app *BaseApp) Restart() error {
 	event.IsRestart = true
 
 	return app.OnTerminate().Trigger(event, func(e *TerminateEvent) error {
-		_ = e.App.ResetBootstrapState()
+		_ = e.App.ClearBootstrap()
 
 		// attempt to restart the bootstrap process in case execve returns an error for some reason
 		defer func() {
@@ -809,6 +877,10 @@ func (app *BaseApp) OnBootstrap() *hook.Hook[*BootstrapEvent] {
 	return app.onBootstrap
 }
 
+func (app *BaseApp) OnBootstrapClear() *hook.Hook[*BootstrapEvent] {
+	return app.onBootstrapClear
+}
+
 func (app *BaseApp) OnServe() *hook.Hook[*ServeEvent] {
 	return app.onServe
 }
@@ -1015,6 +1087,18 @@ func (app *BaseApp) OnMailerRecordAuthAlertSend(tags ...string) *hook.TaggedHook
 	return hook.NewTaggedHook(app.onMailerRecordAuthAlertSend, tags...)
 }
 
+// -------------------------------------------------------------------
+// Filesystem event hooks
+// -------------------------------------------------------------------
+
+func (app *BaseApp) onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent] {
+	return app._onFilesystemNewWriter
+}
+
+func (app *BaseApp) onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent] {
+	return app._onFilesystemDelete
+}
+
 // -------------------------------------------------------------------
 // Realtime API event hooks
 // -------------------------------------------------------------------
@@ -1172,7 +1256,7 @@ func (app *BaseApp) OnBatchRequest() *hook.Hook[*BatchRequestEvent] {
 // -------------------------------------------------------------------
 
 func (app *BaseApp) initDataDB() error {
-	dbPath := filepath.Join(app.DataDir(), "data.db")
+	dbPath := filepath.Join(app.DataDir(), dataDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1234,7 +1318,7 @@ func normalizeSQLLog(sql string) string {
 func (app *BaseApp) initAuxDB() error {
 	// note: renamed to "auxiliary" because "aux" is a reserved Windows filename
 	// (see https://github.com/pocketbase/pocketbase/issues/5607)
-	dbPath := filepath.Join(app.DataDir(), "auxiliary.db")
+	dbPath := filepath.Join(app.DataDir(), auxDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1350,7 +1434,15 @@ func (app *BaseApp) registerBaseHooks() {
 		Id: "__pbCronStart__",
 		Func: func(e *ServeEvent) error {
 			app.Cron().Start()
+			return e.Next()
+		},
+		Priority: 999,
+	})
 
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbCronStop__",
+		Func: func(e *BootstrapEvent) error {
+			app.Cron().Stop()
 			return e.Next()
 		},
 		Priority: 999,
@@ -1382,6 +1474,7 @@ func (app *BaseApp) registerBaseHooks() {
 	app.registerMFAHooks()
 	app.registerOTPHooks()
 	app.registerAuthOriginHooks()
+	app.registerNotifyWatcherHooks()
 }
 
 // getLoggerMinLevel returns the logger min level based on the
@@ -1406,9 +1499,41 @@ func getLoggerMinLevel(app App) slog.Level {
 }
 
 func (app *BaseApp) initLogger() error {
+	var stopped atomic.Bool
+
 	duration := 3 * time.Second
 	ticker := time.NewTicker(duration)
-	done := make(chan bool, 1)
+
+	done := make(chan struct{}, 1)
+
+	runLogsWrite := func(logs []*logger.Log) {
+		if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
+			return
+		}
+
+		// write the accumulated logs
+		//
+		// note: based on several local tests there is no
+		// significant performance difference between small number
+		// of separate write queries vs 1 big INSERT
+		app.AuxRunInTransaction(func(txApp App) error {
+			model := &Log{}
+			for _, l := range logs {
+				model.MarkAsNew()
+				model.Id = GenerateDefaultRandomId()
+				model.Level = int(l.Level)
+				model.Message = l.Message
+				model.Data = l.Data
+				model.Created, _ = types.ParseDateTime(l.Time)
+
+				if err := txApp.AuxSave(model); err != nil {
+					log.Println("Failed to write log", model, err)
+				}
+			}
+
+			return nil
+		})
+	}
 
 	handler := logger.NewBatchHandler(logger.BatchOptions{
 		Level:     getLoggerMinLevel(app),
@@ -1423,40 +1548,30 @@ func (app *BaseApp) initLogger() error {
 				}
 			}
 
-			ticker.Reset(duration)
+			if !stopped.Load() {
+				ticker.Reset(duration)
+			}
 
 			return app.Settings().Logs.MaxDays > 0
 		},
 		WriteFunc: func(ctx context.Context, logs []*logger.Log) error {
-			if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
-				return nil
+			// don't block and wait for the write transaction to complete
+			// when we can't be sure if the logs write wasn't triggered while
+			// inside another AUX db transaction (ticker or batch threshold reached)
+			// which can block indefinitely and cause deadlock
+			// (https://github.com/pocketbase/pocketbase/issues/7836)
+			shouldBlock, _ := ctx.Value(logger.BlockKey).(bool)
+			if shouldBlock {
+				runLogsWrite(logs)
+			} else {
+				routine.FireAndForget(func() { runLogsWrite(logs) })
 			}
 
-			// write the accumulated logs
-			// (note: based on several local tests there is no significant performance difference between small number of separate write queries vs 1 big INSERT)
-			app.AuxRunInTransaction(func(txApp App) error {
-				model := &Log{}
-				for _, l := range logs {
-					model.MarkAsNew()
-					model.Id = GenerateDefaultRandomId()
-					model.Level = int(l.Level)
-					model.Message = l.Message
-					model.Data = l.Data
-					model.Created, _ = types.ParseDateTime(l.Time)
-
-					if err := txApp.AuxSave(model); err != nil {
-						log.Println("Failed to write log", model, err)
-					}
-				}
-
-				return nil
-			})
-
 			return nil
 		},
 	})
 
-	go func() {
+	routine.FireAndForget(func() {
 		ctx := context.Background()
 
 		for {
@@ -1467,21 +1582,25 @@ func (app *BaseApp) initLogger() error {
 				handler.WriteAll(ctx)
 			}
 		}
-	}()
+	})
 
 	app.logger = slog.New(handler)
 
-	// write all remaining logs before ticker.Stop to avoid races with ResetBootstrap user calls
-	app.OnTerminate().Bind(&hook.Handler[*TerminateEvent]{
-		Id: "__pbAppLoggerOnTerminate__",
-		Func: func(e *TerminateEvent) error {
-			handler.WriteAll(context.Background())
+	// attempt to write all queued logs before clearing the application bootstrap state
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbAppLoggerFlushBeforeStop__",
+		Func: func(e *BootstrapEvent) error {
+			// extra precaution in case the hook was manually triggered while inside aux db transaction
+			_, isTx := e.App.AuxNonconcurrentDB().(*dbx.Tx)
+			ctx := context.WithValue(context.Background(), logger.BlockKey, !isTx)
+			handler.WriteAll(ctx)
 
+			stopped.Store(true)
 			ticker.Stop()
 
-			// don't block in case OnTerminate is triggered more than once
+			// don't block in case the hook is triggered more than once
 			select {
-			case done <- true:
+			case done <- struct{}{}:
 			default:
 			}
 

core/base_test.go

@@ -7,6 +7,7 @@ import (
 	"os"
 	"slices"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	_ "unsafe"
@@ -64,7 +65,7 @@ func TestBaseAppBootstrap(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if app.IsBootstrapped() {
 		t.Fatal("Didn't expect the application to be bootstrapped.")
@@ -114,7 +115,7 @@ func TestBaseAppBootstrap(t *testing.T) {
 	runNilChecks(nilChecksBeforeReset)
 
 	// reset
-	if err := app.ResetBootstrapState(); err != nil {
+	if err := app.ClearBootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
@@ -140,7 +141,7 @@ func TestNewBaseAppTx(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if err := app.Bootstrap(); err != nil {
 		t.Fatal(err)
@@ -184,7 +185,7 @@ func TestBaseAppNewMailClient(t *testing.T) {
 		DataDir:       testDataDir,
 		EncryptionEnv: "pb_test_env",
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	client1 := app.NewMailClient()
 	m1, ok := client1.(*mailer.Sendmail)
@@ -214,7 +215,7 @@ func TestBaseAppNewFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewFilesystem()
@@ -243,7 +244,7 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewBackupsFilesystem()
@@ -265,68 +266,172 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	}
 }
 
+const logsThreshold = 200
+
+func assertLogsCount(t *testing.T, app core.App, expected int) {
+	var total int
+
+	err := app.LogQuery().Select("count(*)").Row(&total)
+	if err != nil {
+		t.Fatalf("Failed to fetch total logs: %v", err)
+	}
+
+	if total != expected {
+		t.Fatalf("Expected %d log(s), got %d", expected, total)
+	}
+}
+
 func TestBaseAppLoggerWrites(t *testing.T) {
 	t.Parallel()
 
+	// note: outside of synctest because the bootstrap tickers could deadlock
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Run("disabled logs retention", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 0
+
+			for i := 0; i < logsThreshold+1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, 0)
+		})
+	})
+
+	t.Run("test batch logs writes", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 2
+
+			for i := 0; i < logsThreshold-1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			// below threshold
+			assertLogsCount(t, app, 0)
+
+			// threshold reached -> should trigger batch write
+			app.Logger().Error("test")
+
+			// should be skipped from this batch and added for the next
+			app.Logger().Error("test")
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, logsThreshold)
+
+			// note: we can't test the flush timer here because the ticker
+			// was started out of the synctest buble to avoid deadlocks
+			// (see TestBaseAppLoggerWritesAwaited for a flaky but real timer test)
+		})
+	})
+}
+
+func TestBaseAppLoggerWritesAwaited(t *testing.T) {
+	t.Parallel()
+
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
 
-	// reset
-	if err := app.DeleteOldLogs(time.Now()); err != nil {
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
 		t.Fatal(err)
 	}
 
-	const logsThreshold = 200
-
-	totalLogs := func(app core.App, t *testing.T) int {
-		var total int
-
-		err := app.LogQuery().Select("count(*)").Row(&total)
-		if err != nil {
-			t.Fatalf("Failed to fetch total logs: %v", err)
-		}
-
-		return total
+	// enable logs persistence
+	app.Settings().Logs.MaxDays = 1
+	err = app.Save(app.Settings())
+	if err != nil {
+		t.Fatal(err)
 	}
 
-	t.Run("disabled logs retention", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 0
+	t.Run("flush on timer tick", func(t *testing.T) {
+		timeout := time.After(5 * time.Second)
+		done := make(chan struct{})
 
-		for i := 0; i < logsThreshold+1; i++ {
-			app.Logger().Error("test")
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			done <- struct{}{}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.Logger().Error("test")
+
+		// short wait to ensure that there is no non-blocking write
+		time.Sleep(500 * time.Millisecond)
+
+		assertLogsCount(t, app, 0)
+
+		// wait for the ticker to write the db record
+		select {
+		case <-timeout:
+			t.Fatal("ticker wait timeout")
+		case <-done:
 		}
 
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
+		assertLogsCount(t, app, 1)
 	})
 
-	t.Run("test batch logs writes", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 1
-
-		for i := 0; i < logsThreshold-1; i++ {
-			app.Logger().Error("test")
-		}
-
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
-
-		// should trigger batch write
+	t.Run("before ClearBootstrap flush", func(t *testing.T) {
 		app.Logger().Error("test")
 
-		// should be added for the next batch write
-		app.Logger().Error("test")
+		app.Bootstrap()
 
-		if total := totalLogs(app, t); total != logsThreshold {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold, total)
+		assertLogsCount(t, app, 2)
+	})
+
+	t.Run("batch flush inside aux transaction shouldn't hang", func(t *testing.T) {
+		timeout := time.After(1 * time.Second)
+		done := make(chan struct{})
+		totalCreated := 0
+
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			totalCreated++
+			if totalCreated == 200 {
+				done <- struct{}{}
+			}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.AuxRunInTransaction(func(txApp core.App) error {
+			for range logsThreshold {
+				txApp.Logger().Error("test")
+			}
+
+			return nil
+		})
+
+		// wait for the non-blocking write
+		select {
+		case <-timeout:
+			t.Fatal("non-blocking write timeout")
+		case <-done:
 		}
 
-		// wait for ~3 secs to check the timer trigger
-		time.Sleep(3200 * time.Millisecond)
-		if total := totalLogs(app, t); total != logsThreshold+1 {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold+1, total)
-		}
+		assertLogsCount(t, app, 202)
+
+		// force clear to ensure that there are no other logs
+		app.Bootstrap()
+
+		assertLogsCount(t, app, 202)
 	})
 }
 
@@ -369,7 +474,7 @@ func TestBaseAppRefreshSettingsLoggerMinLevelEnabled(t *testing.T) {
 				DataDir: testDataDir,
 				IsDev:   s.isDev,
 			})
-			defer app.ResetBootstrapState()
+			defer app.ClearBootstrap()
 
 			if err := app.Bootstrap(); err != nil {
 				t.Fatal(err)

core/collection_import.go

@@ -4,12 +4,13 @@ import (
 	"cmp"
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/jsontext"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/spf13/cast"
 )
 
@@ -186,7 +187,11 @@ func (app *BaseApp) ImportCollections(toImport []map[string]any, deleteMissing b
 			)
 			if err := validator.run(); err != nil {
 				// serialize the validation error(s)
-				serializedErr, _ := json.MarshalIndent(err, "", "  ")
+				serializedErr, _ := json.Marshal(
+					err,
+					jsontext.WithIndentPrefix(""),
+					jsontext.WithIndent("  "),
+				)
 
 				return validation.Errors{"collections": validation.NewError(
 					"validation_collections_import_failure",

core/collection_import_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"strings"
 	"testing"
 
@@ -449,7 +449,7 @@ func TestImportCollectionsCreateRules(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	raw, err := json.Marshal(collection)
+	raw, err := json.Marshal(collection, json.Deterministic(true))
 	if err != nil {
 		t.Fatal(err)
 	}

core/collection_model.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strconv"
 	"strings"
@@ -348,6 +348,7 @@ func (app *BaseApp) registerCollectionHooks() {
 }
 
 // @todo experiment eventually replacing the rules *string with a struct?
+// @todo consider changing the Indexes field to a "getter" for the sqlite_master table?
 type baseCollection struct {
 	BaseModel
 
@@ -521,8 +522,6 @@ func (m *Collection) unmarshalRawOptions() error {
 // For new/"blank" Collection models it replaces the model with a factory
 // instance and then unmarshal the provided data one on top of it.
 func (m *Collection) UnmarshalJSON(b []byte) error {
-	type alias *Collection
-
 	// initialize the default fields
 	// (e.g. in case the collection was NOT created using the designated factories)
 	if m.IsNew() && m.Type == "" {
@@ -539,7 +538,8 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
 		*m = *blank
 	}
 
-	return json.Unmarshal(b, alias(m))
+	type alias Collection
+	return json.Unmarshal(b, (*alias)(m))
 }
 
 // MarshalJSON implements the [json.Marshaler] interface.
@@ -549,10 +549,12 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
 func (m Collection) MarshalJSON() ([]byte, error) {
 	switch m.Type {
 	case CollectionTypeView:
-		return json.Marshal(struct {
+		alias := struct {
 			baseCollection
 			collectionViewOptions
-		}{m.baseCollection, m.collectionViewOptions})
+		}{m.baseCollection, m.collectionViewOptions}
+
+		return json.Marshal(alias, json.Deterministic(true))
 	case CollectionTypeAuth:
 		alias := struct {
 			baseCollection
@@ -581,15 +583,15 @@ func (m Collection) MarshalJSON() ([]byte, error) {
 			alias.OAuth2.Providers = redactedProviders
 		}
 
-		return json.Marshal(alias)
+		return json.Marshal(alias, json.Deterministic(true))
 	default:
-		return json.Marshal(m.baseCollection)
+		return json.Marshal(m.baseCollection, json.Deterministic(true))
 	}
 }
 
 // String returns a string representation of the current collection.
 func (m Collection) String() string {
-	raw, _ := json.Marshal(m)
+	raw, _ := m.MarshalJSON()
 	return string(raw)
 }
 
@@ -820,6 +822,25 @@ func onCollectionSave(e *CollectionEvent) error {
 
 	e.Collection.updateGeneratedIdIfExists(e.App)
 
+	// normalize indexes table name
+	for i, raw := range e.Collection.Indexes {
+		parsed := dbutils.ParseIndex(raw)
+
+		// no need to normalize
+		if parsed.TableName == e.Collection.Name {
+			continue
+		}
+
+		parsed.TableName = e.Collection.Name
+
+		normalized := parsed.Build()
+		if normalized == "" {
+			continue // leave to the model validator to decide whether to return an error
+		}
+
+		e.Collection.Indexes[i] = normalized
+	}
+
 	return e.Next()
 }
 
@@ -905,8 +926,14 @@ func onCollectionSaveExecute(e *CollectionEvent) error {
 	}
 
 	// trigger an update for all views with changed fields as a result of the current collection save
-	// (ignoring view errors to allow users to update the query from the UI)
-	resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	// (only log the error to allow users to adjust the problematic view queries from the UI)
+	depViewsErr := resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	if depViewsErr != nil {
+		e.App.Logger().Warn(
+			"Dependent view collection(s) may need to be updated after "+e.Collection.Name+" collection change",
+			"error", depViewsErr,
+		)
+	}
 
 	return nil
 }

core/collection_model_auth_options.go

@@ -1,12 +1,15 @@
 package core
 
 import (
+	"encoding/json/v2"
+	"errors"
+	"slices"
 	"strconv"
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/security"
@@ -60,7 +63,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		MFA: MFAConfig{
 			Enabled:  false,
-			Duration: 1800, // 30min
+			Duration: 600, // 10min
 		},
 		OTP: OTPConfig{
 			Enabled:       false,
@@ -70,7 +73,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		AuthToken: TokenConfig{
 			Secret:   security.RandomString(50),
-			Duration: 604800, // 7 days
+			Duration: 432000, // 5days
 		},
 		PasswordResetToken: TokenConfig{
 			Secret:   security.RandomString(50),
@@ -82,7 +85,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		VerificationToken: TokenConfig{
 			Secret:   security.RandomString(50),
-			Duration: 259200, // 3days
+			Duration: 86400, // 1day
 		},
 		FileToken: TokenConfig{
 			Secret:   security.RandomString(50),
@@ -409,6 +412,71 @@ type OAuth2Config struct {
 	Enabled bool `form:"enabled" json:"enabled"`
 }
 
+// UnmarshalJSON implements the [json.Unmarshaler] interface.
+//
+// The main difference from the standard unmarshalization is that
+// instead of replacing the entire providers config slice, we ensure
+// that partially submitted provider data (e.g. without clientSecret)
+// is merged on per config level based on the provider name
+// (https://github.com/pocketbase/pocketbase/issues/7815).
+func (c *OAuth2Config) UnmarshalJSON(b []byte) error {
+	originalProviders := slices.Clone(c.Providers)
+
+	type alias OAuth2Config
+	err := json.Unmarshal(b, (*alias)(c))
+	if err != nil {
+		return err
+	}
+
+	if len(c.Providers) == 0 {
+		return nil
+	}
+
+	// unmarshal again but this time into a plain array of objects
+	// so that we have only the submitted fields and no zero defaults
+	plain := struct {
+		Providers []map[string]any `json:"providers"`
+	}{}
+	err = json.Unmarshal(b, &plain)
+	if err != nil {
+		return err
+	}
+
+	// no providers were submitted
+	if len(plain.Providers) == 0 {
+		return nil
+	}
+
+	if len(c.Providers) != len(plain.Providers) {
+		return errors.New("the length of the plain unmarshalized providers and the ones from the config doesn't match")
+	}
+
+ProvidersMergeLoop:
+	for i, plain := range plain.Providers {
+		for _, original := range originalProviders {
+			if original.Name == plain["name"] {
+				raw, err := json.Marshal(plain)
+				if err != nil {
+					return err
+				}
+
+				// unmarshal the new plain data on top of the original one
+				err = json.Unmarshal(raw, &original)
+				if err != nil {
+					return err
+				}
+
+				// reassigne to the updated original
+				c.Providers[i] = original
+
+				continue ProvidersMergeLoop
+			}
+		}
+	}
+
+	return nil
+}
+
 // GetProviderConfig returns the first OAuth2ProviderConfig that matches the specified name.
 //
 // Returns false and zero config if no such provider is available in c.Providers.

core/collection_model_auth_options_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strings"
 	"testing"
@@ -712,6 +712,76 @@ func TestPasswordAuthConfigValidate(t *testing.T) {
 	}
 }
 
+func TestOAuth2ConfigUnmarshalJSON(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []struct {
+		name     string
+		newJSON  string
+		expected string
+	}{
+		{
+			"missing",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"}
+			}`,
+			`{"providers":[{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"b","clientId":"b_clientId","clientSecret":"b_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": []
+			}`,
+			`{"providers":[],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"non-empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": [
+					{"name": "c", "clientId": "c_clientId", "clientSecret": "c_clientSecret"},
+					{"name": "a", "displayName": "a_displayName"}
+				]
+			}`,
+			`{"providers":[{"pkce":null,"name":"c","clientId":"c_clientId","clientSecret":"c_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"a_displayName","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			config := core.OAuth2Config{
+				Enabled: false,
+				MappedFields: core.OAuth2KnownFields{
+					Name: "name_test",
+				},
+				Providers: []core.OAuth2ProviderConfig{
+					{Name: "a", ClientId: "a_clientId", ClientSecret: "a_clientSecret"},
+					{Name: "b", ClientId: "b_clientId", ClientSecret: "b_clientSecret"},
+				},
+			}
+
+			err := json.Unmarshal([]byte(s.newJSON), &config)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			raw, err := json.Marshal(config, json.Deterministic(true))
+			if err != nil {
+				t.Fatal(err)
+			}
+			rawStr := string(raw)
+
+			if rawStr != s.expected {
+				t.Fatalf("Expected OAuth2ProviderConfig\n%s\ngot\n%s", s.expected, rawStr)
+			}
+		})
+	}
+}
+
 func TestOAuth2ConfigGetProviderConfig(t *testing.T) {
 	scenarios := []struct {
 		name           string
@@ -1016,8 +1086,8 @@ func TestOAuth2ProviderConfigInitProvider(t *testing.T) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}
 
-			rawMeta, _ := json.Marshal(provider.Extra())
-			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra)
+			rawMeta, _ := json.Marshal(provider.Extra(), json.Deterministic(true))
+			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra, json.Deterministic(true))
 			if !bytes.Equal(rawMeta, expectedMeta) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}

core/collection_model_auth_templates.go

@@ -18,6 +18,7 @@ var defaultVerificationTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-verification/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Verify</a>
 </p>
+<p><i>If you didn't recently register, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team
@@ -31,7 +32,7 @@ var defaultResetPasswordTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-password-reset/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Reset password</a>
 </p>
-<p><i>If you didn't ask to reset your password, you can ignore this email.</i></p>
+<p><i>If you didn't ask to reset your password, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team
@@ -45,7 +46,7 @@ var defaultConfirmEmailChangeTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-email-change/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Confirm new email</a>
 </p>
-<p><i>If you didn't ask to change your email address, you can ignore this email.</i></p>
+<p><i>If you didn't ask to change your email address, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team

core/collection_model_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -618,7 +618,7 @@ func TestCollectionUnmarshalJSON(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			rawResult, err := json.Marshal(collection)
+			rawResult, err := json.Marshal(collection, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -829,7 +829,7 @@ func TestCollectionDBExport(t *testing.T) {
 		},
 		{
 			core.CollectionTypeAuth,
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":null,"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":null},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":[],"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":[]},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 	}
 
@@ -860,7 +860,7 @@ func TestCollectionDBExport(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			raw, err := json.Marshal(result)
+			raw, err := json.Marshal(result, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -1678,3 +1678,40 @@ func TestCollectionSaveViewWrapping(t *testing.T) {
 		})
 	}
 }
+
+func TestCollectionSaveIndexesTableNameNormalization(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	dummyCollection := core.NewBaseCollection("new_test")
+	dummyCollection.Fields.Add(&core.TextField{Name: "test"})
+	dummyCollection.Indexes = []string{
+		"create index `new_test_idx1` on `` (`test`) where 1=1",
+		"create index `new_test_idx2` on `test` (`test`) where 1=2",
+		"create index `new_test_idx3` on `someting_else` (`test`) where 1=3",
+	}
+
+	err := app.Save(dummyCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// refetch a clean state
+	dummyCollection, err = app.FindCollectionByNameOrId(dummyCollection.Name)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(dummyCollection.Indexes) != 3 {
+		t.Fatalf("Expected 3 indexes, got %v", dummyCollection.Indexes)
+	}
+
+	for _, raw := range dummyCollection.Indexes {
+		parsed := dbutils.ParseIndex(raw)
+		if parsed.TableName != dummyCollection.Name {
+			t.Fatalf("Expected all indexes to have tableName %q, found %q:\n%s", dummyCollection.Name, parsed.TableName, raw)
+		}
+	}
+}

core/collection_model_view_options.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 )
 
 var _ optionsValidator = (*collectionViewOptions)(nil)

core/collection_query.go

@@ -3,7 +3,7 @@ package core
 import (
 	"bytes"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -250,52 +250,6 @@ func (app *BaseApp) TruncateCollection(collection *Collection) error {
 
 // -------------------------------------------------------------------
 
-// saveViewCollection persists the provided View collection changes:
-//   - deletes the old related SQL view (if any)
-//   - creates a new SQL view with the latest newCollection.Options.Query
-//   - generates new feilds list  based on newCollection.Options.Query
-//   - updates newCollection.Fields based on the generated view table info and query
-//   - saves the newCollection
-//
-// This method returns an error if newCollection is not a "view".
-func saveViewCollection(app App, newCollection, oldCollection *Collection) error {
-	if !newCollection.IsView() {
-		return errors.New("not a view collection")
-	}
-
-	return app.RunInTransaction(func(txApp App) error {
-		query := newCollection.ViewQuery
-
-		// generate collection fields from the query
-		viewFields, err := txApp.CreateViewFields(query)
-		if err != nil {
-			return err
-		}
-
-		// delete old renamed view
-		if oldCollection != nil {
-			if err := txApp.DeleteView(oldCollection.Name); err != nil {
-				return err
-			}
-		}
-
-		// wrap view query if necessary
-		query, err = normalizeViewQueryId(txApp, query)
-		if err != nil {
-			return fmt.Errorf("failed to normalize view query id: %w", err)
-		}
-
-		// (re)create the view
-		if err := txApp.SaveView(newCollection.Name, query); err != nil {
-			return err
-		}
-
-		newCollection.Fields = viewFields
-
-		return txApp.Save(newCollection)
-	})
-}
-
 // normalizeViewQueryId wraps (if necessary) the provided view query
 // with a subselect to ensure that the id column is a text since
 // currently we don't support non-string model ids
@@ -342,50 +296,59 @@ func resaveViewsWithChangedFields(app App, excludeIds ...string) error {
 	}
 
 	return app.RunInTransaction(func(txApp App) error {
+		var collectionErrors []error
+
 		for _, collection := range collections {
 			if len(excludeIds) > 0 && list.ExistInSlice(collection.Id, excludeIds) {
 				continue
 			}
 
-			// clone the existing fields for temp modifications
-			oldFields, err := collection.Fields.Clone()
-			if err != nil {
-				return err
+			check := func() error {
+				// clone the existing fields for temp modifications
+				oldFields, err := collection.Fields.Clone()
+				if err != nil {
+					return err
+				}
+
+				// generate new fields from the query
+				newFields, err := txApp.CreateViewFields(collection.ViewQuery)
+				if err != nil {
+					return err
+				}
+
+				// unset the fields' ids to exclude from the comparison
+				for _, f := range oldFields {
+					f.SetId("")
+				}
+				for _, f := range newFields {
+					f.SetId("")
+				}
+
+				encodedNewFields, err := json.Marshal(newFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				encodedOldFields, err := json.Marshal(oldFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				if bytes.EqualFold(encodedNewFields, encodedOldFields) {
+					return nil // no changes
+				}
+
+				return txApp.Save(collection)
 			}
 
-			// generate new fields from the query
-			newFields, err := txApp.CreateViewFields(collection.ViewQuery)
-			if err != nil {
-				return err
-			}
-
-			// unset the fields' ids to exclude from the comparison
-			for _, f := range oldFields {
-				f.SetId("")
-			}
-			for _, f := range newFields {
-				f.SetId("")
-			}
-
-			encodedNewFields, err := json.Marshal(newFields)
-			if err != nil {
-				return err
-			}
-
-			encodedOldFields, err := json.Marshal(oldFields)
-			if err != nil {
-				return err
-			}
-
-			if bytes.EqualFold(encodedNewFields, encodedOldFields) {
-				continue // no changes
-			}
-
-			if err := saveViewCollection(txApp, collection, nil); err != nil {
-				return err
+			if err := check(); err != nil {
+				collectionErrors = append(
+					collectionErrors,
+					fmt.Errorf("[%s] %w", collection.Name, err),
+				)
 			}
 		}
 
-		return nil
+		return errors.Join(collectionErrors...)
 	})
 }

core/collection_record_table_sync.go

@@ -6,8 +6,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
@@ -306,8 +306,10 @@ func dropCollectionIndexes(app App, collection *Collection) error {
 		for _, raw := range collection.Indexes {
 			parsed := dbutils.ParseIndex(raw)
 
-			if !parsed.IsValid() {
-				continue
+			// note: don't check IsValid because the index table name may not be populated
+			// (https://github.com/pocketbase/pocketbase/issues/7689)
+			if parsed.IndexName == "" {
+				return fmt.Errorf("failed to dop index - missing index name: %s", raw)
 			}
 
 			_, err := txApp.DB().NewQuery(fmt.Sprintf("DROP INDEX IF EXISTS [[%s]]", parsed.IndexName)).Execute()

core/collection_record_table_sync_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"testing"
 
 	"github.com/pocketbase/dbx"
@@ -278,12 +278,12 @@ func TestSingleVsMultipleValuesNormalization(t *testing.T) {
 				t.Fatalf("Failed to load record: %v", err)
 			}
 
-			encodedResult, err := json.Marshal(result)
+			encodedResult, err := json.Marshal(result, json.Deterministic(true))
 			if err != nil {
 				t.Fatalf("Failed to encode result: %v", err)
 			}
 
-			encodedExpectation, err := json.Marshal(s.expected)
+			encodedExpectation, err := json.Marshal(s.expected, json.Deterministic(true))
 			if err != nil {
 				t.Fatalf("Failed to encode expectation: %v", err)
 			}
@@ -294,3 +294,49 @@ func TestSingleVsMultipleValuesNormalization(t *testing.T) {
 		})
 	}
 }
+
+func TestDropIndexWithoutTableName(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	properIndex := "CREATE INDEX `new_test_idx2` ON `new_test` (`test`)"
+	indexWithoutTableName := "CREATE INDEX `new_test_idx2` ON `` (`test`)"
+
+	dummyCollection := core.NewBaseCollection("new_test")
+	dummyCollection.Fields.Add(&core.TextField{Name: "test"})
+	dummyCollection.Indexes = []string{properIndex}
+
+	err := app.Save(dummyCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// resave without table name but without hooks to avoid the normalizations
+	dummyCollection.Indexes[0] = indexWithoutTableName
+	err = app.UnsafeWithoutHooks().Save(dummyCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	dummyCollection, err = app.FindCollectionByNameOrId(dummyCollection.Name)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// resave should normalize the index
+	err = app.Save(dummyCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	dummyCollection, err = app.FindCollectionByNameOrId(dummyCollection.Name)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(dummyCollection.Indexes) != 1 || dummyCollection.Indexes[0] != properIndex {
+		t.Fatalf("Expected exactly 1 index\n%s\ngot\n%v", properIndex, dummyCollection.Indexes)
+	}
+}

core/collection_validate.go

@@ -7,8 +7,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/list"

core/db.go

@@ -10,8 +10,8 @@ import (
 	"strconv"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"
 )

core/db_connect.go

@@ -11,7 +11,7 @@ func DefaultDBConnect(dbPath string) (*dbx.DB, error) {
 	// Note: the busy_timeout pragma must be first because
 	// the connection needs to be set to block on busy before WAL mode
 	// is set in case it hasn't been already set by another connection.
-	pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)"
+	pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)&_defensive=1"
 
 	db, err := dbx.Open("sqlite", dbPath+pragmas)
 	if err != nil {

core/db_table_test.go

@@ -3,7 +3,7 @@ package core_test
 import (
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"slices"
 	"testing"
@@ -120,7 +120,11 @@ func TestTableInfo(t *testing.T) {
 		t.Run(fmt.Sprintf("%d_%s", i, s.tableName), func(t *testing.T) {
 			rows, _ := app.TableInfo(s.tableName)
 
-			raw, err := json.Marshal(rows)
+			raw, err := json.Marshal(
+				rows,
+				json.Deterministic(true),
+				json.FormatNilSliceAsNull(true),
+			)
 			if err != nil {
 				t.Fatal(err)
 			}

core/event_request_batch.go

@@ -3,7 +3,7 @@ package core
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/hook"
 )
 

core/event_request_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"net/http"
 	"strings"
 	"testing"
@@ -183,7 +183,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
 			t.Fatalf("Failed to resolve request info: %v", err)
 		}
 
-		raw, err := json.Marshal(info)
+		raw, err := json.Marshal(info, json.Deterministic(true))
 		if err != nil {
 			t.Fatalf("Failed to serialize request info: %v", err)
 		}
@@ -205,7 +205,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
 			t.Fatalf("Failed to resolve request info: %v", err)
 		}
 
-		raw, err := json.Marshal(info)
+		raw, err := json.Marshal(info, json.Deterministic(true))
 		if err != nil {
 			t.Fatalf("Failed to serialize request info: %v", err)
 		}
@@ -308,7 +308,7 @@ func TestRequestInfoClone(t *testing.T) {
 
 	// check the original data
 	// ---
-	originalRaw, err := json.Marshal(info)
+	originalRaw, err := json.Marshal(info, json.Deterministic(true))
 	if err != nil {
 		t.Fatalf("Failed to serialize original request info: %v", err)
 	}
@@ -321,7 +321,7 @@ func TestRequestInfoClone(t *testing.T) {
 
 	// check the clone data
 	// ---
-	cloneRaw, err := json.Marshal(clone)
+	cloneRaw, err := json.Marshal(clone, json.Deterministic(true))
 	if err != nil {
 		t.Fatalf("Failed to serialize clone request info: %v", err)
 	}

core/events.go

@@ -8,6 +8,7 @@ import (
 	"time"
 
 	"github.com/pocketbase/pocketbase/tools/auth"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/mailer"
 	"github.com/pocketbase/pocketbase/tools/router"
@@ -188,6 +189,24 @@ type MailerRecordEvent struct {
 	Meta map[string]any
 }
 
+// -------------------------------------------------------------------
+// Filesystem events data
+// -------------------------------------------------------------------
+
+type FilesystemNewWriterEvent struct {
+	hook.Event
+	*filesystem.NewWriterEvent
+
+	App App
+}
+
+type FilesystemDeleteEvent struct {
+	hook.Event
+	*filesystem.DeleteEvent
+
+	App App
+}
+
 // -------------------------------------------------------------------
 // Model events data
 // -------------------------------------------------------------------
@@ -448,8 +467,24 @@ type RealtimeConnectRequestEvent struct {
 
 	Client subscriptions.Client
 
-	// note: modifying it after the connect has no effect
+	// IdleTimeout specifies the max duration to wait for a new message
+	// before closing the connection.
+	//
+	// Modifying the value after the connection has been established has no effect.
+	//
+	// Defaults to 5 minutes.
 	IdleTimeout time.Duration
+
+	// MaxTimeout specifies the maximum duration a realtime connection
+	// can remain open (including even if there are ongoing messages).
+	//
+	// Once the specified duration expires, the current connection will
+	// be terminated, until a client reconnect is issued (if the client is still active).
+	//
+	// Modifying the value after the connection has been established has no effect.
+	//
+	// Defaults to 30 minutes.
+	MaxTimeout time.Duration
 }
 
 type RealtimeMessageEvent struct {

core/external_auth_model.go

@@ -4,7 +4,7 @@ import (
 	"context"
 	"errors"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"
@@ -137,4 +137,42 @@ func (app *BaseApp) registerExternalAuthHooks() {
 		},
 		Priority: 99,
 	})
+
+	// delete all pre-existing external auths on verified upgrade
+	app.OnRecordUpdateExecute().Bind(&hook.Handler[*RecordEvent]{
+		Func: func(e *RecordEvent) error {
+			if !e.Record.Collection().IsAuth() {
+				return e.Next()
+			}
+
+			hasUpgradedVerified := !e.Record.Original().IsNew() && !e.Record.Original().Verified() && e.Record.Verified()
+
+			if !hasUpgradedVerified {
+				return e.Next()
+			}
+
+			originalApp := e.App
+			return e.App.RunInTransaction(func(txApp App) error {
+				e.App = txApp
+				defer func() { e.App = originalApp }()
+
+				externalAuths, err := txApp.FindAllExternalAuthsByRecord(e.Record)
+				if err != nil {
+					return err
+				}
+				if len(externalAuths) > 0 {
+					// delete all pre-existing external auths
+					if err := txApp.DeleteAllExternalAuthsByRecord(e.Record); err != nil {
+						return err
+					}
+
+					// force refresh tokens reset (if not already)
+					e.Record.RefreshTokenKey()
+				}
+
+				return e.Next()
+			})
+		},
+		Priority: 99,
+	})
 }

core/external_auth_model_test.go

@@ -308,3 +308,104 @@ func TestExternalAuthValidateHook(t *testing.T) {
 		})
 	}
 }
+
+func TestExternalAuthClearOnVerfiedUpgrade(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	t.Run("unverified->no changes", func(t *testing.T) {
+		user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if user.Verified() {
+			t.Fatal("Expected user to be unverified")
+		}
+
+		beforeAuths, err := app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(beforeAuths) == 0 {
+			t.Fatalf("Expected at least one external auth (%v)", err)
+		}
+
+		oldTokenKey := user.TokenKey()
+
+		if err = app.Save(user); err != nil {
+			t.Fatal(err)
+		}
+
+		if oldTokenKey != user.TokenKey() {
+			t.Fatal("Expected tokenKey to remain unchanged")
+		}
+
+		afterAuths, err := app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(afterAuths) != len(beforeAuths) {
+			t.Fatalf("Expected %d external auths, found %d (%v)", len(afterAuths), len(beforeAuths), err)
+		}
+	})
+
+	t.Run("unverified->verified", func(t *testing.T) {
+		user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if user.Verified() {
+			t.Fatal("Expected user to be unverified")
+		}
+
+		externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(externalAuths) == 0 {
+			t.Fatalf("Expected at least one external auth (%v)", err)
+		}
+
+		oldTokenKey := user.TokenKey()
+
+		user.SetVerified(true)
+		if err = app.Save(user); err != nil {
+			t.Fatal(err)
+		}
+
+		if oldTokenKey == user.TokenKey() {
+			t.Fatal("Expected tokenKey to be renewed")
+		}
+
+		externalAuths, err = app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(externalAuths) != 0 {
+			t.Fatalf("Expected all user external auths to be deleted, found %d (%v)", len(externalAuths), err)
+		}
+	})
+
+	t.Run("verified->no changes", func(t *testing.T) {
+		user, err := app.FindAuthRecordByEmail("users", "test3@example.com")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if !user.Verified() {
+			t.Fatal("Expected user to be verified")
+		}
+
+		beforeAuths, err := app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(beforeAuths) == 0 {
+			t.Fatalf("Expected at least one external auth (%v)", err)
+		}
+
+		oldTokenKey := user.TokenKey()
+
+		if err = app.Save(user); err != nil {
+			t.Fatal(err)
+		}
+
+		if oldTokenKey != user.TokenKey() {
+			t.Fatal("Expected tokenKey to remain unchanged")
+		}
+
+		afterAuths, err := app.FindAllExternalAuthsByRecord(user)
+		if err != nil || len(afterAuths) != len(beforeAuths) {
+			t.Fatalf("Expected %d external auths, found %d (%v)", len(afterAuths), len(beforeAuths), err)
+		}
+	})
+}

core/external_auth_query.go

@@ -1,6 +1,8 @@
 package core
 
 import (
+	"errors"
+
 	"github.com/pocketbase/dbx"
 )
 
@@ -59,3 +61,25 @@ func (app *BaseApp) FindFirstExternalAuthByExpr(expr dbx.Expression) (*ExternalA
 
 	return model, nil
 }
+
+// DeleteAllExternalAuthsByRecord deletes all ExternalAuth models associated with the provided record.
+//
+// Returns a combined error with the failed deletes.
+func (app *BaseApp) DeleteAllExternalAuthsByRecord(authRecord *Record) error {
+	models, err := app.FindAllExternalAuthsByRecord(authRecord)
+	if err != nil {
+		return err
+	}
+
+	var errs []error
+	for _, m := range models {
+		if err := app.Delete(m); err != nil {
+			errs = append(errs, err)
+		}
+	}
+	if len(errs) > 0 {
+		return errors.Join(errs...)
+	}
+
+	return nil
+}

core/external_auth_query_test.go

@@ -2,6 +2,7 @@ package core_test
 
 import (
 	"fmt"
+	"slices"
 	"testing"
 
 	"github.com/pocketbase/dbx"
@@ -174,3 +175,68 @@ func TestFindFirstExternalAuthByExpr(t *testing.T) {
 		})
 	}
 }
+
+func TestDeleteAllExternalAuthsByRecord(t *testing.T) {
+	t.Parallel()
+
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	demo1, err := testApp.FindRecordById("demo1", "84nmscqy84lsi1t")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	user1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client1, err := testApp.FindAuthRecordByEmail("clients", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client2, err := testApp.FindAuthRecordByEmail("clients", "test2@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	scenarios := []struct {
+		record     *core.Record
+		deletedIds []string
+	}{
+		{demo1, nil}, // non-auth record
+		{user1, []string{"dlmflokuq1xl342", "clmflokuq1xl341"}},
+		{client1, []string{"f1z5b3843pzc964"}},
+		{client2, nil},
+	}
+
+	for i, s := range scenarios {
+		t.Run(fmt.Sprintf("%d_%s_%s", i, s.record.Collection().Name, s.record.Id), func(t *testing.T) {
+			app, _ := tests.NewTestApp()
+			defer app.Cleanup()
+
+			deletedIds := []string{}
+			app.OnRecordDelete().BindFunc(func(e *core.RecordEvent) error {
+				deletedIds = append(deletedIds, e.Record.Id)
+				return e.Next()
+			})
+
+			err := app.DeleteAllExternalAuthsByRecord(s.record)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			if len(deletedIds) != len(s.deletedIds) {
+				t.Fatalf("Expected deleted ids\n%v\ngot\n%v", s.deletedIds, deletedIds)
+			}
+
+			for _, id := range s.deletedIds {
+				if !slices.Contains(deletedIds, id) {
+					t.Errorf("Expected to find deleted id %q in %v", id, deletedIds)
+				}
+			}
+		})
+	}
+}

core/field.go

@@ -6,7 +6,7 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/list"
 )

core/field_autodate.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_bool.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_date.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/field_editor.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_email.go

@@ -5,8 +5,8 @@ import (
 	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/spf13/cast"
 )

core/field_file.go

@@ -9,7 +9,7 @@ import (
 	"regexp"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
@@ -732,12 +732,16 @@ func (f *FileField) toSliceValue(raw any) []any {
 	case nil:
 		// nothing to cast
 	case *filesystem.File:
-		result = append(result, value)
+		if value != nil {
+			result = append(result, value)
+		}
 	case filesystem.File:
 		result = append(result, &value)
 	case []*filesystem.File:
 		for _, v := range value {
-			result = append(result, v)
+			if v != nil {
+				result = append(result, v)
+			}
 		}
 	case []filesystem.File:
 		for _, v := range value {

core/field_file_test.go

@@ -3,7 +3,7 @@ package core_test
 import (
 	"bytes"
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
@@ -98,11 +98,13 @@ func TestFileFieldPrepareValue(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	f1Raw, err := json.Marshal(f1)
+	f1Raw, err := json.Marshal(f1, json.Deterministic(true))
 	if err != nil {
 		t.Fatal(err)
 	}
 
+	var nilFile *filesystem.File
+
 	scenarios := []struct {
 		raw      any
 		field    *core.FileField
@@ -114,8 +116,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
 		{123, &core.FileField{MaxSelect: 1}, `"123"`},
 		{"a", &core.FileField{MaxSelect: 1}, `"a"`},
 		{`["a"]`, &core.FileField{MaxSelect: 1}, `"a"`},
-		{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
 		{f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
+		{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
+		{nilFile, &core.FileField{MaxSelect: 1}, `""`},
 		{[]string{}, &core.FileField{MaxSelect: 1}, `""`},
 		{[]string{"a", "b"}, &core.FileField{MaxSelect: 1}, `"b"`},
 
@@ -126,8 +129,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
 		{"a", &core.FileField{MaxSelect: 2}, `["a"]`},
 		{`["a"]`, &core.FileField{MaxSelect: 2}, `["a"]`},
 		{[]any{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
-		{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
 		{[]filesystem.File{*f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
+		{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
+		{[]any{nilFile, f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
 		{[]string{}, &core.FileField{MaxSelect: 2}, `[]`},
 		{[]string{"a", "b", "c"}, &core.FileField{MaxSelect: 2}, `["a","b","c"]`},
 	}
@@ -139,7 +143,7 @@ func TestFileFieldPrepareValue(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -208,7 +212,7 @@ func TestFileFieldDriverValue(t *testing.T) {
 				}
 			}
 
-			vRaw, err := json.Marshal(v)
+			vRaw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -695,7 +699,7 @@ func TestFileFieldFindGetter(t *testing.T) {
 
 			v := getter(record)
 
-			raw, err := json.Marshal(v)
+			raw, err := json.Marshal(v, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -812,7 +816,7 @@ func TestFileFieldFindSetter(t *testing.T) {
 
 			setter(record, s.value)
 
-			raw, err := json.Marshal(record.Get(s.field.GetName()))
+			raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -911,8 +915,8 @@ func TestFileFieldIntercept(t *testing.T) {
 
 		tests.TestValidationErrors(t, err, []string{"text"})
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f1.Name, f3})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f1.Name, f3}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -931,8 +935,8 @@ func TestFileFieldIntercept(t *testing.T) {
 			t.Fatalf("Expected save to succeed, got %v", err)
 		}
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -952,8 +956,8 @@ func TestFileFieldIntercept(t *testing.T) {
 			t.Fatalf("Expected save to succeed, got %v", err)
 		}
 
-		raw, _ := json.Marshal(record.GetRaw("file_many"))
-		expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
+		raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+		expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
 		if !bytes.Equal(expectedRaw, raw) {
 			t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 		}
@@ -1063,8 +1067,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
 					t.Fatalf("Expected save to succeed, got %v", err)
 				}
 
-				raw, _ := json.Marshal(record.GetRaw("file_many"))
-				expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
+				raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+				expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
 				if !bytes.Equal(expectedRaw, raw) {
 					t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 				}
@@ -1086,8 +1090,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
 					t.Fatalf("Expected save to succeed, got %v", err)
 				}
 
-				raw, _ := json.Marshal(record.GetRaw("file_many"))
-				expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
+				raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
+				expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
 				if !bytes.Equal(expectedRaw, raw) {
 					t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
 				}

core/field_geo_point.go

@@ -3,7 +3,7 @@ package core
 import (
 	"context"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/types"
 )