bumped app version

Co-authored-by: Hans <hi@hans0805.me>

.github/SECURITY.md

@@ -1,5 +1,128 @@
 # Security
 
-If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
+**Keep in mind that PocketBase is a non-commercial open source project, maintained entirely on volunteer basis (there is no company or dedicated team behind it), and there are no bounties!**
 
-All reports will be promptly addressed and you'll be credited in the fix release notes.
+If you want to responsibly report a security issue you'll have to reach out as a human to **support at pocketbase.io**.
+
+This means:
+- no overconfident and arrogant tone
+- no threatening deadlines
+- no requirement for me to login in your security platform just to read the report
+- no inflated severity (we can discuss the CVSS score after confirming the issue)
+- no LLMs usage as part of your report description or followup communication
+
+Reports that don't follow the above will NOT be reviewed no matter of their validity _(you are of course free to publish whatever you want; see also [#7718](https://github.com/pocketbase/pocketbase/discussions/7718))_.
+
+**Or in other words - a simple _"Hey I think I found a security issue when I do X"_ is enough.**
+
+I try to be as responsive as possible and usually address security issues within couple days but if you didn't receive a reply from me for more than a week it is very likely that your email was flagged and in that case please open a GitHub issue or discussion just mentioning that you found a vulnerability and want to report it so that I can see the notification and will try to contact you for more details.
+
+In case the vulnerability is confirmed:
+
+- I'll start working on a local fix.
+- Once the fix is implemented locally, I'll publish a pre-announcement with a scheduled release date _(and when possible an approximate release time)_.
+- After the release, I'll publish a GitHub security advisory and CVE with remediation steps and **minimal** details regarding the found exploit _(you are free to publish PoC and more details in your own blog, gist, etc. but it is advised to wait at least a week after the release to allow enough time for people to patch their instances before making it more publicly known)_.
+
+### Below is a short list of previous reports that are NOT considered security issues:
+
+<details>
+<summary><strong>Stored XSS</strong></summary>
+
+This was discussed several times, both privately and [publicly](https://github.com/pocketbase/pocketbase/discussions/6694), but I remain on the opinion that it should be handled primarily on the client-side.
+
+Modern browsers recently introduced a basic [`Sanitizer` interface](https://developer.mozilla.org/en-US/docs/Web/API/Sanitizer) that could help filtering HTML strings without external libraries.
+
+Having also a default [Content Security Policy (CSP)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP) either as meta tag or response header is always a good idea to minimize the risk of XSS.
+</details>
+
+<details>
+<summary><strong>SQL injection in low level DB methods like <code>app.DeleteTable(dangerousName)</code></strong></summary>
+
+This is working correctly and it is not an issue but it is a common report most likely found by LLM or some other automated tools that may have stumbled on the [NB! code comments](https://pkg.go.dev/github.com/pocketbase/pocketbase@master/core#BaseApp.DeleteTable).
+
+Raw SQL statements, table and column names are not parameterized and they are vulnerable to SQL injection if used with untrusted input. The documentation as seen above already warns against it. In recent PocketBase releases, many of the arguments of these methods were also prefixed with `dangerous*` to make it even more clear that they should be used with caution.
+</details>
+
+<details>
+<summary><strong>Race conditions</strong></summary>
+
+To avoid DB locks PocketBase deliberately tries to minimize the use of DB transactions.
+This means that operations like record update don't wrap out of the box for example the `SELECT` and `UPDATE` SQL statements in a single transaction, and this can technically lead to a race condition if multiple users edit the same record.
+
+This is an accepted tradeoff and for the majority of cases it has no security implications.
+
+This also apply for the read and delete of MFA and OTP records but for those cases, since they operate in a security sensitive context, they have an extra short-lived duration that is configurable from the collection settings _(there are also system cron jobs that takes care for deleting forgotten/expired entries to prevent accumulation of invalid records)_.
+
+For the cases where transactions are really needed, users can utilize the [Batch Web API](https://pocketbase.io/docs/api-records/#batch-createupdateupsertdelete-records) or [create a transaction programmatically](https://pocketbase.io/docs/go-records/#transaction) _(with PocketBase v0.23+ it is also possible to wrap an entire hook chain in a single transaction)_.
+</details>
+
+<details>
+<summary><strong>List/Search side-channel attacks</strong></summary>
+
+Over the years we've implemented several extra checks to minimize the risk of List/Search side-channel attacks (see especially [v0.32.0](https://github.com/pocketbase/pocketbase/blob/master/CHANGELOG.md#v0320)) but users need to be aware that all client-side filtered fields are technically subject to timing attacks _(whether they are practical or not is a different topic)_.
+
+This is by design and it is accepted tradeoff between performance, security and usability.
+
+If you are concerned about timing attacks and have security sensitive collection data such as `secret`, `code`, `token`, etc. then the general recommendation is to mark their related fields as "Hidden" in order to disallow use in client-side filters.
+</details>
+
+<details>
+<summary><strong>Connecting to a vulnerable OAuth2 provider</strong></summary>
+
+Because PocketBase v0.23+ supports automatically uploading the OAuth2 avatar on user create _(need to be specified from the auth collection OAuth2 fields mapping)_ some security researchers raised a concern regarding a Blind SSRF but this implies that an attacker controls the OAuth2 vendor and this is a very serious assumption in the first place.
+
+The entire OAuth2 flow relies that the application server (PocketBase) trusts the configured OAuth2 vendor.
+If you suspect that an OAuth2 vendor is malicious and cannot be trusted then you MUST NOT use that OAuth2 vendor at all and you should report it.
+
+If someone is able to tamper with the OAuth2 responses then the entire OAuth2 flow can be thrown out of the window because they will be practically able to authenticate as any of your existing users and the eventual avatar URL probing request is the least of your problem.
+
+~Nonetheless, in future PocketBase releases there will be [extra `localhost` domain like checks](https://github.com/orgs/pocketbase/projects/2/views/1?pane=issue&itemId=159545722) when assigning the OAuth2 avatar URL to a `file` field that will further minimize the risk of internal network probing requests in case of a vulnerable OAuth2 provider.~ _Done._
+</details>
+
+<details>
+<summary><strong>Users enumeration</strong></summary>
+
+This is a common and usually valid report but there is no easy solution without confusing and degrading the users experience.
+
+Some endpoints, like the user create/register, can be used for usernames or emails enumeration based on various response heuristics - timing, specific error messages, etc.
+
+In many places where applicable we've tried to minimize the impact by using constant time checks, returning non-descriptive error messages, applying an internal rate limit for some operations, etc. but it is not bulletproof and if somebody wants to find out if a user is registered they will be able to do it one way or another.
+
+If you think that there is a place where we can improve the handling without hurting too much the user experience, feel free to open a regular public issue and it will be considered.
+</details>
+
+<details>
+<summary><strong>Attack-vectors relying on social engineering</strong></summary>
+
+Reports for attacks relying on various social engineering tactics _(e.g. tricking someone to click on a link)_ are valid concerns but usually out of the security scope of the project as there are a lot of cases where the APIs are deliberately designed for minimal friction.
+
+If you have concerns for such attack, feel free to open a regular public issue and we can eventually try to reconsider adding extra guards when feasible _(or at least properly document the existing behavior)_.
+</details>
+
+<details>
+<summary><strong><code>disintegration/imaging</code> CVE-2023-36308</strong></summary>
+
+Just for the past month, due to some corporate security scanners 5 different people raised concerns over [CVE-2023-36308](https://nvd.nist.gov/vuln/detail/CVE-2023-36308) but this is not really a vulnerability, especially not in PocketBase.
+
+[`disintegration/imaging`](https://github.com/disintegration/imaging) is a direct PocketBase dependency responsible for the thumbs generation.
+
+First, a panic (similar to exception in other languages) is NOT a security issue and Go programs usually have to be written defensively with that in mind. In PocketBase specifically all routes have auto panic-recover handling, no matter what the source of the panic is, so the worst case scenario would be an HTTP error response when attempting to access the thumb.
+
+Second, the related issue that the CVE describes is probably caused by a bug in an outdated `golang.org/x/image` dependency listed in the `go.mod` of that package but PocketBase uses a newer patched version of it that is expected to take precedence.
+
+Third, even if that issue is still available, with PocketBase it would have been triggerable ONLY if we supported TIFF thumbs generation but we don't. The supported thumbs formats at the moment are JPG, PNG, GIF (its first frame) and partially WebP (stored as PNG). All other images are served as it is, without any transformation.
+
+In the future I may consider eventually replacing the library because it is no longer actively maintained but as of now it is working correctly and as expected for our use case and you can safely flag the security warning as false-positive.
+</details>
+
+<details>
+<summary><strong>JSVM "sandboxing"</strong></summary>
+
+This is another very common report but **there is no such thing as JSVM "sandboxing" in PocketBase**.
+
+The JS `pb_hooks` (or JSVM for short) are NOT supposed to run untrusted or client provided JavaScript code _(the same way you are not supposed to run untrusted code in your Node.js server)_.
+
+Once interpreted the `pb_hooks` run as part of the same application process together with the rest of the Go code. There are no additional filesystem, network, memory, etc. restrictions. This means that it is OK for developers to be able to access environment variables, perform network calls to any URLs they want, invoke shell commands or even sleep/block the script execution.
+
+So if you are security researcher and not sure if something is a security fault in the JS hooks, ask yourself - "Can I do the same when using PocketBase as Go framework?" and if the answer is "Yes" then it is not a security issue with the JSVM.
+</details>

.github/workflows/release.yaml

@@ -16,19 +16,19 @@ jobs:
         run: echo "flags=--snapshot" >> $GITHUB_ENV
 
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           fetch-depth: 0
 
       - name: Set up Node.js
-        uses: actions/setup-node@v4
+        uses: actions/setup-node@v6
         with:
-          node-version: 20.17.0
+          node-version: '>=25.2.1'
 
       - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@v7
         with:
-          go-version: '>=1.23.0'
+          go-version: '>=1.27.1'
 
       # This step usually is not needed because the /ui/dist is pregenerated locally
       # but its here to ensure that each release embeds the latest admin ui artifacts.

.gitignore

@@ -15,4 +15,6 @@ coverage.out
 # generated markdown previews
 README.html
 CHANGELOG.html
+CHANGELOG_16_22.html
+CHANGELOG_8_15.html
 LICENSE.html

.goreleaser.yaml

@@ -24,13 +24,23 @@ builds:
       - amd64
       - arm64
       - arm
+      - s390x
+      - ppc64le
     goarm:
       - 7
     ignore:
       - goos: windows
         goarch: arm
+      - goos: windows
+        goarch: s390x
+      - goos: windows
+        goarch: ppc64le
       - goos: darwin
         goarch: arm
+      - goos: darwin
+        goarch: s390x
+      - goos: darwin
+        goarch: ppc64le
 
 release:
   draft: true

CHANGELOG.md

@@ -1,439 +1,88 @@
-## v0.23.0-rc14
+## v0.40.4
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
+- Fixed migration deadlock if a logs db write happens to run while the migration is still executing ([#7836](https://github.com/pocketbase/pocketbase/issues/7836)).
 
-- Allow changing collate, sort and partial constraints for indexes on system fields.
+- `app.ResetBootstrapState()` was soft-deprecated in favour of `app.ClearBootstrap()`.
+    _Additionally a new `app.OnClearBootstrap()` hook was added to allow clearing custom allocated `OnBootstrap` resources in case the app uses a non-standard initialization (e.g. doesn't call `Start()` or intentionally skip the `OnTerminate` hook)._
 
-- Added system cron to run once a day `PRAGMA wal_checkpoint(TRUNCATE)` as a fallback to assist in high traffic applications where the autocheckpoint may not be able to take its turn.
-    _Similar to the `PRAGMA optimize`, its execution is optional and even if it is not supported by a custom driver it will result only in a WARN app log._
+- Bumped `golang.org/x/*` dependencies.
 
----
 
-_Unless something else show up, this is expected to be the last release candidate before the final v0.23.0, planned to be released in the next 1-2 weeks.
-Because there were several experiments and changes in the rc versions regarding the migrations, if you had a v0.22.x app that you've migrated already to some of the other rc versions and have a full collections snapshot generated for it, note that you may have to regenerate the snapshot again (hopefully this would be the last time)._
+## v0.40.3
 
+- Write the status header for JSON responses only if the fields picker succeed or has acceptable fallback.
+    _This is to allow custom response status code for failed json writes._
 
-## v0.23.0-rc13
+- Fixed collection index validator to allow expressions with parenthesis in the optional `WHERE` clause.
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
+- Clamped arccosine to [-1,1] in the Harvesine formula for the `geoDistance()` filter function to workaround edge case related to float rounding errors for some coordinates.
 
-- Added WakaTime OAuth2 provider ([#5829](https://github.com/pocketbase/pocketbase/pull/5829); thanks @tigawanna).
+- Prevent unnecessary body chunk read if we already known that we are beyond the allowed limit.
 
-- Added `superuser otp EMAIL` command as fallback for generating superuser OTPs from the command line in case OTP has been enabled for the `_superusers` but the SMTP server has deliverability issues.
+- Updated the `json` field validator to check the `encoding/json/v2` semantics and allow duplicated keys on record marshalize for compliance with old jsonv1 data.
 
-- ⚠️ Changed `OnRecordRequestOTPRequest` hook to be triggered even if there is no record matching the email (aka. `e.Record` could be `nil`), allowing you to manually create a new user with the OTP request and assigning it to `e.Record`.
+- Fixed nested cascade delete of self-referenced relation records.
 
-- Added new `sentTo` system field to the `_otps` collection (it is managed programmatically or by superusers; it could be anything - email, phone, messanger app id, etc.).
-    By default when the OTP is submitted via email it is automatically populated with the user email address (via the `OnMailerRecordOTPSend` hook's finalizer).
-    This allow us on valid `auth-with-otp` request to automatically mark the user email as verified.
+- Minor UI fixes (updated dark primary btn color contrast, force reload the records list if the deleted record has self-referenced cascade relation field, etc.).
 
-- Added `RateLimitRule.Audience` optional field for restricting a rate limit rule for `"@guest"`-only, `"@auth"`-only, `""`-any (default).
+- Changed JSVM `$app` variable definition from TS type to interface ([#7834](https://github.com/pocketbase/pocketbase/issues/7834)).
 
-- Added default max limits for the expressions count and length of the search filter and sort params.
-    _This is just an extra measure mostly for the case when the filter and sort parameters are resolved outside of the request context since the request size limits won't apply._
+- Bumped `golang.org/x/*` dependencies to silence security scanners ([#7829](https://github.com/pocketbase/pocketbase/discussions/7829)).
 
-- Other minor changes (better error in case of duplicated rate limit rule, fixed typos, changed field id and name length validator to max 100, updated Go deps, etc.).
 
+## v0.40.2
 
-## v0.23.0-rc12
+- Return an error when filter params fallback fails to json serialize and optimized params replacement to execute in a single pass.
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
+- Fixed collection index parsing error for indexes with missing name.
 
-- Fixed validation error caused by the id change in v0.23.0-rc10 migration ([#5820](https://github.com/pocketbase/pocketbase/discussions/5820)).
+- Minor UI autocomplete optimizations _(prefix match, autocomplete debounce, etc.)_.
 
+- Fixed linter warnings and comment typos.
 
-## v0.23.0-rc11
+- Bumped goja and its related dependencies _(regex unescaped dash error fix and base64 optimizations)_.
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
+- Bumped the min Go GitHub action version to 1.27.1 as it includes some [minor `database/sql` and `enconding/json/v2` bug fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.27.1).
 
-- Fixed JSVM types errors ([#5797](https://github.com/pocketbase/pocketbase/issues/5797)).
 
-- Skip the default `loadAuthToken` middleware if `e.Auth` is already loaded ([#5800](https://github.com/pocketbase/pocketbase/discussions/5800)).
+## v0.40.1
 
-- Restored the API rules for the default initial "users" collection migration to be the same as in <= v0.22.x.
+- Fixes for some reported regressions related to the `encoding/json/v2` update:
+    - allow mangling invalid UTF8 characters when serializing json data ([#7814](https://github.com/pocketbase/pocketbase/issues/7814))
+    - fixed OAuth2 providers config merge incorrectly replacing the entire slice ([#7815](https://github.com/pocketbase/pocketbase/issues/7815))
 
-- ⚠️ Replaced the old Instagram provider with a new one that is compatible with the new Instagram Login APIs ([#5588](https://github.com/pocketbase/pocketbase/pull/5588); thanks @pnmcosta).
-    _The provider key is `instagram2` to prevent conflicts with existing linked users._
 
-- ⚠️ Changes regarding the first superuser creation based on [#5814](https://github.com/pocketbase/pocketbase/discussions/5814).
+## v0.40.0
 
-- ⚠️ Removed `apis.RequireSuperuserAuthOnlyIfAny()` middleware.
+- Propagate console command errors and recovered panics to `app.Start()` so that the program can exit with non-zero code while still ensuring that `app.OnTerminate` hook was triggered _(responsible for the app graceful shutdown handling)_.
+    _⚠️ Note that this could be a slight breaking change in case you are chaining PocketBase commands and relied on the previous `0` exit status for `Command.RunE` returned errors._
+    _Or in other words, if you have `./pocketbase invalid && someothercommand` and previously relied that `someothercommand` will be always executed then this is no longer the case and you'll have to adjust it or replace `&&` with `;`._
 
-- ⚠️ Removed `RequestEvent.UnsafeRealIP()` to prevent misuse and confusion with `RequestEvent.RealIP()` (_the latter is considered safer because it checks the trusted proxy headers settings_).
+- Added quotes around the default `Content-Disposition` serving filename in case custom name with special characters is provided.
 
+- Added `Cross-Origin-Opener-Policy:same-origin` to the default security response headers.
+    _This is an extra precaution to prevent tab-nabbing in case custom UI plugins use `target="_blank"` without `rel="noopener"`._
 
-## v0.23.0-rc10
+- Added `Record.GetInt64(field)` helper (note that the serializable max safe integer of the `number` field is ~2^53-1).
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
+- Added `Store.Keys()` method that returns a slice with all of the store keys.
 
-- Restore the CRC32 checksum autogeneration for the collection/field ids in order to maintain deterministic default identifier value and minimize conflicts between custom migrations and full collections snapshots.
-  _There is a system migration that will attempt to normalize existing system collections ids, but if you already migrated to v0.23.0-rc and have generated a full collections snapshot migration, you have to delete it and regenerate a new one._
+- Added new `DELETE /api/logs` endpoint and UI control to delete all logs without changing the `maxDays` retention setting.
 
-- Change the behavior of the default generated collections snapshot migration to act as "extend" instead of "replace" to prevent accidental data deletion.
-  _I think this would be rare but if you want the old behaviour you can edit the generated snapshot file and replace the second argument (`deleteMissing`) of `App.ImportCollection/App.ImportCollectionsByMarshaledJSON` from `false` to `true`._
+- Added new log settings option to limit the max `Log.Data` size that will be saved in the database (default to ~16KB).
+    _This is an extra precaution for the cases when logging user supplied data without validating it beforehand._
+    _If the resulting `Log.Data` json is above the limit, it is truncated to the last valid decoded character and an extra `"__pb_truncated__":true` log data entry will be added.`_
+    _Additionally, for just in case the log message is also truncated at max 8k characters._
 
-- Added `app.SubscriptionsBroker().TotalClients()` helper method to return the total registered realtime clients ([#5793](https://github.com/pocketbase/pocketbase/issues/5793)).
+- Added new `filesystem` low-level helper methods:
+    - `filesystem.NewWriter(key, opts)` to allow direct file create from an `io.Reader` value.
+    - `filesystem.OnNewWriter()` hook to allow listening for new/to-be-created files _(it is not exposed in `core.App` instance for now to avoid introducing breaking changes)_.
+    - `filesystem.OnDelete()` hook to allow listening for deleted files _(it is not exposed in `core.App` instance for now to avoid introducing breaking changes)_.
 
+- Optimized backups to no longer transaction lock the database during backup generation ([#7799](https://github.com/pocketbase/pocketbase/discussions/7799#discussioncomment-18108244)).
 
-## v0.23.0-rc9
+- Updated `modernc.org/sqlite` to 1.57.0 and registered by default the new `_defensive=1` DSN query parameter to enable [SQLite's defensive mode](https://sqlite.org/c3ref/c_dbconfig_defensive.html#sqlitedbconfigdefensive).
 
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Fixed auto `www.` redirect due to missing URI schema.
-
-- Fixed collection and field renaming when reusing an old collection/field name ([#5741](https://github.com/pocketbase/pocketbase/issues/5741)).
-
-- Update the "API preview" section to include information about the batch api.
-
-- Exported `core.DefaultDBConnect` function that could be used as a fallback when initializing custom SQLite drivers and builds.
-
-- ⚠️ No longer loads the `mattn/go-sqlite3` driver by default when building with `CGO_ENABLED=1` to avoid `multiple definition ...` linker errors in case different CGO SQLite drivers or builds are used.
-    This means that no matter of the `CGO_ENABLED` value, now out of the box PocketBase will always use only the pure Go driver ([`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite)).
-    This will be documented properly in the new website but if you want to continue using `mattn/go-sqlite3` (e.g. because of the icu or other builtin extension) you could register it as follow:
-    ```go
-    package main
-
-    import (
-        "database/sql"
-        "log"
-
-        "github.com/mattn/go-sqlite3"
-        "github.com/pocketbase/dbx"
-        "github.com/pocketbase/pocketbase"
-    )
-
-    func init() {
-        // initialize default PRAGMAs for each new connection
-        sql.Register("pb_sqlite3",
-            &sqlite3.SQLiteDriver{
-                ConnectHook: func(conn *sqlite3.SQLiteConn) error {
-                    _, err := conn.Exec(`
-                        PRAGMA busy_timeout       = 10000;
-                        PRAGMA journal_mode       = WAL;
-                        PRAGMA journal_size_limit = 200000000;
-                        PRAGMA synchronous        = NORMAL;
-                        PRAGMA foreign_keys       = ON;
-                        PRAGMA temp_store         = MEMORY;
-                        PRAGMA cache_size         = -16000;
-                    `, nil)
-
-                    return err
-                },
-            },
-        )
-
-        dbx.BuilderFuncMap["pb_sqlite3"] = dbx.BuilderFuncMap["sqlite3"]
-    }
-
-    func main() {
-        app := pocketbase.NewWithConfig(pocketbase.Config{
-            DBConnect: func(dbPath string) (*dbx.DB, error) {
-                return dbx.Open("pb_sqlite3", dbPath)
-            },
-        })
-
-        // custom hooks and plugins...
-
-        if err := app.Start(); err != nil {
-            log.Fatal(err)
-        }
-    }
-    ```
-    Also note that if you are not planning to use the `core.DefaultDBConnect` fallback as part of your custom driver registration you can exclude the default pure Go driver from the build with the build tag `-tags no_default_driver` to reduce the binary size a little.
-
-- ⚠️ Removed JSVM `BaseCollection()`, `AuthCollection()`, `ViewCollection()` class aliases for simplicity and to avoid confusion with the accepted constructor arguments (_you can simply use as before `new Collection({ type: "base", ... })`; this will also initialize the default type specific options_).
-
-- Other minor improvements (added validator for duplicated index definitions, updated the impersonate popup styles, added query param support for loading a collection based on its name, etc.).
-
-
-## v0.23.0-rc8
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Lock the `_otps` and `_mfas` system collections Delete API rule for superusers only.
-
-- Reassign in the JSVM executors the global `$app` variable with the hook scoped `e.app` value to minimize the risk of a deadlock when a hook or middleware is wrapped in a transaction.
-
-- Reuse the OAuth2 created user record pointer to ensure that all its following hooks operate on the same record instance.
-
-- Added tags support for the `OnFileTokenRequest` hook.
-
-- Other minor changes (added index for the `_collections` type column, added more detailed godoc for the collection fields and `core.App` methods, fixed flaky record enrich tests, etc.).
-
-
-## v0.23.0-rc7
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Register the default panic-recover middleware after the activity logger so that we can log the error.
-
-- Updated the `RequestEvent.BindBody` FormData type inferring rules to convert numeric strings into float64 only if the resulting minimal number string representation matches the initial FormData string value ([#5687](https://github.com/pocketbase/pocketbase/issues/5687)).
-
-- Fixed the JSVM types to include properly generated function declarations when the related Go functions have shortened/combined return values.
-
-- Reorganized the record table fields<->columns syncing to remove the `PRAGMA writable_schema` usage.
-
-
-## v0.23.0-rc6
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Fixed realtime 403 API error on resubscribe ([#5674](https://github.com/pocketbase/pocketbase/issues/5674)).
-
-- Fixed the auto OAuth2 avatar mapped field assignment when the OAuth2 provider doesn't return an avatar URL ([#5673](https://github.com/pocketbase/pocketbase/pull/5673)).
-  _In case the avatar retrieval fails and the mapped record field "Required" option is not set, the error is silenced and only logged with WARN level._
-
-- Added `Router.SEARCH(path, action)` helper method for registering `SEARCH` endpoints.
-
-- Changed all builtin middlewares to return `*hook.Handler[*core.RequestEvent]` with a default middleware id for consistency and to allow removal.
-  Or in other words, replace `.BindFunc(apis.Gzip())` with `.Bind(apis.Gzip())`.
-
-- Updated the JSVM types to reflect the recent changes.
-
-
-## v0.23.0-rc5
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Added Notion OAuth2 provider ([#4999](https://github.com/pocketbase/pocketbase/pull/4999); thanks @s-li1).
-
-- Added monday.com OAuth2 provider ([#5346](https://github.com/pocketbase/pocketbase/pull/5346); thanks @Jaytpa01).
-
-- Added option to retrieve the OIDC OAuth2 user info from the `id_token` payload for the cases when the provider doesn't have a dedicated user info endpoint.
-
-- Fixed the relation record picker to sort by default by `@rowid` instead of the `created` field as the latter is optional ([#5641](https://github.com/pocketbase/pocketbase/discussions/5641)).
-
-- Fixed the UI "Set Superusers only" button click not properly resetting the input state.
-
-- Fixed the OAuth2 providers logo path shown in the "Authorized providers" UI.
-
-- Fixed the single value UI for the `select`, `file` and `relation` fields ([#5646](https://github.com/pocketbase/pocketbase/discussions/5646))
-
-
-## v0.23.0-rc4
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Fixed the UI settings update form to prevent sending empty string for the mail password or the S3 secret options on resave of the form.
-
-- ⚠️ Added an exception for the `OAuth2` field in the GO->JSVM name mapping rules:
-    ```
-    // old              -> new
-    collection.oAuth2.* -> collection.oauth2.*
-    ```
-
-- Added more user friendly view collection truncate error message.
-
-- Added an extra suffix character to the name of autogenerated template migration file for `*test` suffixed collections to prevent acidentally resulting in `_test.go` migration files.
-
-- Added `FieldsList.AddMarshaledJSON([]byte)` helper method to load a serialized json array of objects or a single json object into an existing collection fields list.
-
-- Fixed the autogenerated Go migration template when updating a collection ([#5631](https://github.com/pocketbase/pocketbase/discussions/5631)).
-
-    ⚠️ If you have already used a previous prerelease and have autogenerated Go migration files, please check the migration files named **`{timestamp}_updated_{collection}.go`** and manually change:
-
-    <table width="100%">
-        <tr>
-            <th width="50%">Old (broken)</th>
-            <th width="50%">New</th>
-        </tr>
-        <tr>
-    <td width="50%">
-
-    ```go
-    // add field / update field
-    if err := json.Unmarshal([]byte(`[{
-        ...
-    }]`), &collection.Fields); err != nil {
-        return err
-    }
-    ```
-
-    </td>
-    <td width="50%">
-
-    ```go
-    // add field / update field
-    if err := collection.Fields.AddMarshaledJSON([]byte(`{
-        ...
-    }`)); err != nil {
-        return err
-    }
-    ```
-
-    </td>
-        </tr>
-    </table>
-    To test that your Go migration files work correctly you can try to start PocketBase with a new temp pb_data, e.g.:
-
-    ```go
-    go run . serve --dir="pb_data_temp"
-    ```
-
-
-## v0.23.0-rc3
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Make `PRAGMA optimize` statement optional in case it is not supported by the driver ([#5611](https://github.com/pocketbase/pocketbase/discussions/5611)).
-
-- Reapply the minimum required `pb_data/auxiliary.db` migrations if the db file was manually deleted ([#5618](https://github.com/pocketbase/pocketbase/discussions/5618)).
-
-- To avoid confusion and unnecessary casting, the `hook.HandlerFunc[T]` type has been removed and instead everywhere we now use directly the underlying function definition, aka.:
-  ```go
-  func(T) error
-  ```
-
-- Fixed the UI input field type of the OTP.length field ([#5617](https://github.com/pocketbase/pocketbase/issues/5617)).
-
-- Other minor fixes (fixed API preview and examples error message typos, better hint for combined/multi-spaced view query columns, fixed the path for the HTTPS green favicon path, etc.).
-
-
-## v0.23.0-rc2
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
-
-- Small update to the earlier v0.23.0-rc that uses `pb_data/auxiliary.db` instead of `pb_data/aux.db` because it seems that on Windows `aux` is disallowed as file name ([#5607](https://github.com/pocketbase/pocketbase/issues/5607)).
-   _If you have already upgraded to v0.23.0-rc please rename manually your `pb_data/aux.db` file to `pb_data/auxiliary.db`._
-
-
-## v0.23.0-rc
-
-> [!CAUTION]
-> **This is a prerelease intended for test and experimental purposes only!**
->
-> It introduces many Go/JSVM breaking changes and requires manual migration steps.
->
-> All new features will be reflected in the new website documentation with the final v0.23.0 release.
-
-> [!NOTE]
->  Please note that you don't have to upgrade to PocketBase v0.23.0 if you are not planning further developing
-> your existing app and/or are satisfied with the v0.22.x features set. There are no identified critical issues
-> with PocketBase v0.22.x yet and in the case of critical bugs and security vulnerabilities, the fixes
-> will be backported for at least until Q1 of 2025 (_if not longer_).
->
-> If you don't plan upgrading just make sure to pin the SDKs version to their latest PocketBase v0.22.x compatible:
-> - JS SDK: `<0.22.0`
-> - Dart SDK: `<0.19.0`
-
-PocketBase v0.23.0-rc is a major refactor of the internals with the overall goal of making PocketBase an easier to use Go framework.
-
-There are many changes but to highlight some of the most notable ones:
-
-- Replaced `echo` with a new router built on top of the Go 1.22 `net/http` mux enhancements.
-- Merged `daos` packages in `core.App` to simplify the DB operations (_the `models` package structs are also migrated in `core`_).
-- Option to specify custom `DBConnect` function as part of the app configuration to allow different `database/sql` SQLite drivers (_turso/libsql, sqlcipher, etc._) and custom builds.
-- New hooks allowing better control over the execution chain and error handling (_including wrapping an entire hook chain in a single DB transaction_).
-- Various `Record` model improvements (_support for get/set modifiers, simplfied file upload by treating the file(s) as regular field value like `record.Set("document", file)`, etc._).
-- Dedicated fields structs with safer defaults to make it easier creating/updating collections programmatically.
-- Option to mark field as Private/Hidden, disallowing regular users to read or modify it (_there is also a dedicated Record hook to hide/unhide Record fields programmatically from a single place_).
-- Option to customize the default system collection fields (`id`, `email`, `password`, etc.).
-- Admins are now system `_superusers` auth records.
-- Builtin rate limiter (_supports tags, wildcards and exact routes matching_).
-- Batch/transactional Web API endpoint.
-- Impersonate Web API endpoint (_it could be also used for generating fixed/non-refreshable superuser tokens, aka. "API keys"_).
-- Support for custom user request activity log attributes.
-- One-Time Password (OTP) auth method (_via email code_).
-- Multi-Factor Authentication (MFA) support (_currently requires any 2 different auth methods to be used_).
-- Support for Record "proxy/projection" in preparation for the planned autogeneration of typed Go record models.
-- Various minor UI improvements (_recursive `Presentable` view, slightly different collection options organization, zoom/pan for the logs chart, etc._)
-- and many more...
-
-In terms of performance, the Go standard router mux is known to be slightly slower compared to Gin, Echo, etc. implementations, but based on my local tests the difference is negliable.
-The [benchmarks repo](https://github.com/pocketbase/benchmarks) will be updated with the final v0.23.0 release (_currently there seems to be ~10% memory consumption increase which I'll have to investigate to see whether it is from the router change or from the new hooks_).
-
-#### Go/JSVM APIs changes
-
-For upgrading to PocketBase v0.23.0, please refer to:
-
-- Go:   https://pocketbase.io/v023upgrade/go/.
-- JSVM: https://pocketbase.io/v023upgrade/jsvm/.
-
-#### SDKs changes
-
-- [JS SDK v0.22.0-rc](https://github.com/pocketbase/js-sdk/blob/develop/CHANGELOG.md)
-- [Dart SDK v0.19.0-rc](https://github.com/pocketbase/dart-sdk/blob/develop/CHANGELOG.md)
-
-#### Web APIs changes
-
-- New `POST /api/batch` endpoint.
-
-- New `GET /api/collections/meta/scaffolds` endpoint.
-
-- New `DELETE /api/collections/{collection}/truncate` endpoint.
-
-- New `POST /api/collections/{collection}/request-otp` endpoint.
-
-- New `POST /api/collections/{collection}/auth-with-otp` endpoint.
-
-- New `POST /api/collections/{collection}/impersonate/{id}` endpoint.
-
-- ⚠️ Removed `/api/admins/*` endpoints because admins are converted to `_superusers` auth collection records.
-
-- ⚠️ Previously when uploading new files to a multiple `file` field, new files were automatically appended to the existing field values.
-     This behaviour has changed with v0.23+ and for consistency with the other multi-valued fields when uploading new files they will replace the old ones. If you want to prepend or append new files to an existing multiple `file` field value you can use the `+` prefix or suffix:
-     ```js
-     "documents": [file1, file2]  // => [file1_name, file2_name]
-     "+documents": [file1, file2] // => [file1_name, file2_name, old1_name, old2_name]
-     "documents+": [file1, file2] // => [old1_name, old2_name, file1_name, file2_name]
-     ```
-
-- ⚠️ Removed `GET /records/{id}/external-auths` and `DELETE /records/{id}/external-auths/{provider}` endpoints because this is now handled by sending list and delete requests to the `_externalAuths` collection.
-
-- ⚠️ Changes to the app settings model fields and response (+new options such as `trustedProxy`, `rateLimits`, `batch`, etc.). The app settings Web APIs are mostly used by the Dashboard UI and rarely by the end users, but if you want to check all settings changes please refer to the [Settings Go struct](https://github.com/pocketbase/pocketbase/blob/develop/core/settings_model.go#L121).
-
-- ⚠️ New flatten Collection model and fields structure. The Collection model Web APIs are mostly used by the Dashboard UI and rarely by the end users, but if you want to check all changes please refer to the [Collection Go struct](https://github.com/pocketbase/pocketbase/blob/develop/core/collection_model.go#L308).
-
-- ⚠️ The top level error response `code` key was renamed to `status` for consistency with the Go APIs.
-    The error field key remains `code`:
-    ```js
-    {
-        "status": 400, // <-- old: "code"
-        "message": "Failed to create record.",
-        "data": {
-            "title": {
-                "code": "validation_required",
-                "message": "Missing required value."
-            }
-        }
-    }
-    ```
-
-- ⚠️ New fields in the `GET /api/collections/{collection}/auth-methods` response.
-    _The old `authProviders`, `usernamePassword`, `emailPassword` fields are still returned in the response but are considered deprecated and will be removed in the future._
-    ```js
-    {
-        "mfa": {
-            "duration": 100,
-            "enabled": true
-        },
-        "otp": {
-            "duration": 0,
-            "enabled": false
-        },
-        "password": {
-            "enabled": true,
-            "identityFields": ["email", "username"]
-        },
-        "oauth2": {
-            "enabled": true,
-            "providers": [{"name": "gitlab", ...}, {"name": "google", ...}]
-        },
-        // old fields...
-    }
-    ```
-
-- ⚠️ Soft-deprecated the OAuth2 auth success `meta.avatarUrl` field in favour of `meta.avatarURL`.
+- Bumped the min Go version to 1.27.0 and migrated to the new `encoding/json/v2` package.
+    _⚠️ Please note that Go 1.27.0 retrofitted `encoding/json` to use the v2 package under the hood but unfortunately is not fully backward compatible._
+    _I recommend to not push blindly an update on production and to test your PocketBase application first locally to see if everything works correctly._

CHANGELOG_08_15.md

@@ -326,7 +326,7 @@
 
 - Added "tags" support for all Record and Model related event hooks.
 
-    The "tags" allow registering event handlers that will be called only on matching table name(s) or colleciton id(s)/name(s).
+    The "tags" allow registering event handlers that will be called only on matching table name(s) or collection id(s)/name(s).
     For example:
     ```go
     app.OnRecordBeforeCreateRequest("articles").Add(func(e *core.RecordCreateEvent) error {
@@ -840,7 +840,7 @@ Please check the individual SDK package changelog and apply the necessary change
     </tr>
   </table>
 
-- All datetime stings are now returned in ISO8601 format - with _Z_ suffix and space as separator between the date and time part:
+- All datetime strings are now returned in ISO8601 format - with _Z_ suffix and space as separator between the date and time part:
   <table class="d-table" width="100%">
     <tr>
       <th>Old</th>

CHANGELOG_16_22.md

@@ -2,6 +2,185 @@
 > For the most recent versions, please refer to [CHANGELOG.md](./CHANGELOG.md)
 ---
 
+## v0.22.55
+
+- (_Backported from v0.40.3_) Fixed collection index validator to allow expressions with parenthesis in the optional `WHERE` clause.
+
+- (_Backported from v0.40.3_) Fixed nested cascade delete of self-referenced relation records.
+
+- (_Backported from v0.40.3_) Bumped `golang.org/x/*` dependencies to silence security scanners ([#7829](https://github.com/pocketbase/pocketbase/discussions/7829)).
+
+
+## v0.22.54
+
+- (_Backported from v0.40.2_) Bumped goja and its related dependencies _(regex unescaped dash error fix and base64 optimizations)_.
+
+- (_Backported from v0.40.2_) Return an error when filter params fallback fails to json serialize and optimized params replacement to execute in a single pass.
+
+- (_Backported from v0.40.2_) Bumped the min Go GitHub action version to 1.27.1 as it includes some [minor `database/sql` and `enconding/json/v2` bug fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.27.1).
+
+
+## v0.22.53
+
+- Workaround `encoding/json` infinite recursion when used with Go 1.27 ([#7808](https://github.com/pocketbase/pocketbase/issues/7808)).
+
+
+## v0.22.52
+
+- (_Backported from v0.39.11_) Bumped `golang.org/x/*` deps and the min Go GitHub action version to 1.26.6 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.6).
+
+
+## v0.22.51
+
+- (_Backported from v0.39.10_) Reverted the auto panic recover handling for the cli commands to preserve the old behavior and allow panic to force exit with non-zero code ([#7781](https://github.com/pocketbase/pocketbase/issues/7781)).
+
+
+## v0.22.50
+
+- (_Backported from v0.39.9_) Bumped goja, fexpr and their related deps.
+
+
+## v0.22.49
+
+- (_Backported from v0.39.8_) Bumped `golang.org/x/*` indirect dependencies as there are some minor security fixes.
+
+- (_Backported from v0.39.8_) Updated `modernc.org/sqlite` to v1.54.0 ([SQLite 3.53.3](https://sqlite.org/src/timeline?from=version-3.53.2&to=version-3.53.3&to2=branch-3.53)).
+
+
+## v0.22.48
+
+- (_Backported from v0.39.7_) Replaced `github.com/go-ozzo/ozzo-validation` with the fork `github.com/pocketbase/ozzo-validation` since the original library has recently changed ownership and the new maintainer cannot be trusted.
+
+- (_Backported from v0.39.7_) Fixed View collection `*` validator and added more friendly error messages ([#7761](https://github.com/pocketbase/pocketbase/issues/7761)).
+
+- (_Backported from v0.39.7_) ⚠️ Security fix for unhandled panic in internal worker goroutines ([#7762](https://github.com/pocketbase/pocketbase/discussions/7762)).
+
+
+## v0.22.47
+
+- (_Backported from v0.39.6_) Bumped the min Go GitHub action version to 1.26.5 as it includes some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.5).
+
+
+## v0.22.46
+
+- (_Backported from v0.39.1_) Bumped the min Go GitHub action version to 1.26.4 as it includes some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.4).
+
+
+## v0.22.45
+
+- (_Backported from v0.38.2_) Updated all `golang.org/x/` packages to cover the recent [security fixes](https://groups.google.com/g/golang-announce/c/PdiGK3xulk4) _(none of them should be a critical issue in PocketBase but nonetheless it is advised to update)_.
+
+
+## v0.22.44
+
+- (_Backported from v0.38.1_) Force unset the auth state of existing realtime connections on user tokenKey change.
+
+
+## v0.22.43
+
+- (_Backported from v0.38.0_) Bumped min Go GitHub action version to 1.26.3 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.3).
+
+
+## v0.22.42
+
+- (_Backported from v0.37.4_) Adjusted Bitbucket, GitHub, GitLab and Gitea/Forgejo OAuth2 providers to better reflect recent API updates and doc references.
+    _In case the userinfo data is not sufficient, some of the providers now send a separate list emails request in order to minimize eventual linking security issues caused by custom onpremise setups (e.g. Gitea/Forgejo allows skipping the email verification if an ENV variable is configured)._
+
+- (_Backported from v0.37.4_) ⚠️ Fixed a pre-hijacking OAuth2 linking vulnerability ([#7662](https://github.com/pocketbase/pocketbase/discussions/7662)).
+
+
+## v0.22.41
+
+- (_Backported from v0.36.9_) Updated the Discord `AuthUser.Name` field to use `global_name`.
+
+- (_Backported from v0.36.9_) Updated `modernc.org/sqlite` to v1.48.2 _(vfs and other error path related fixes)_.
+
+- (_Backported from v0.36.9_) Bumped min Go GitHub action version to 1.26.2 because it comes with several [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.2).
+
+
+## v0.22.40
+
+- (_Backported from v0.36.7_) Updated `modernc.org/sqlite` to v1.46.2 and SQLite 3.51.3.
+    _⚠️ SQLite 3.51.3 fixed a [database corruption bug](https://sqlite.org/wal.html#walresetbug) that is very unlikely to happen (with PocketBase even more so because we queue on app level all writes and explicit transactions through a single db connection), but still it is advised to upgrade._
+
+- (_Backported from v0.36.7_) Updated other minor Go and npm deps.
+    _The min Go version in the go.mod of the package was also bumped to Go 1.25.0 because some of the newer dep versions require it._
+
+
+## v0.22.39
+
+- (_Backported from v0.36.6_) Bumped min Go GitHub action version to 1.26.1 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.1).
+
+
+## v0.22.38
+
+- (_Backported from v0.36.0_) Bumped min Go GitHub action version to 1.25.6 because it comes with some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.25.6).
+
+
+## v0.22.37
+
+- (_Backported from v0.34.1_) - Added missing `:` char to the autocomplete regex ([#7353](https://github.com/pocketbase/pocketbase/pull/7353)).
+
+- (_Backported from v0.34.1_) Bumped min Go GitHub action version to 1.25.5 because it comes with some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.25.5).
+    _The runner action was also updated to `actions/setup-go@v6` since the previous v5 Go source seems [no longer accessible](https://github.com/actions/setup-go/pull/665#issuecomment-3416693714)._
+
+
+## v0.22.36
+
+- (_Backported from v0.30.2_) Bumped min Go GitHub action version to 1.24.8 since it comes with some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.24.8+label%3ACherryPickApproved).
+
+
+## v0.22.35
+
+- (_Backported from v0.29.2_) Bumped min Go GitHub action version to 1.23.12 since it comes with some [minor fixes for the runtime and `database/sql` package](https://github.com/golang/go/issues?q=milestone%3AGo1.23.12+label%3ACherryPickApproved).
+
+
+## v0.22.34
+
+- (_Backported from v0.26.6_) Allow OIDC `email_verified` to be int or boolean string since some OIDC providers like AWS Cognito has non-standard userinfo response ([#6657](https://github.com/pocketbase/pocketbase/pull/6657)).
+
+
+## v0.22.33
+
+- (_Backported from v0.26.3_) Fixed and normalized logs error serialization across common types for more consistent logs error output ([#6631](https://github.com/pocketbase/pocketbase/issues/6631)).
+
+
+## v0.22.32
+
+- (_Backported from v0.26.2_) Updated `golang-jwt/jwt` dependency because it comes with a [minor security fix](https://github.com/golang-jwt/jwt/security/advisories/GHSA-mh63-6h87-95cp).
+
+
+## v0.22.31
+
+- (_Backported from v0.25.5_) Set the current working directory as a default goja script path when executing inline JS strings to allow `require(m)` traversing parent `node_modules` directories.
+
+
+## v0.22.30
+
+- (_Backported from v0.24.4_) Fixed fields extraction for view queries with nested comments ([#6309](https://github.com/pocketbase/pocketbase/discussions/6309)).
+
+- Bumped GitHub action min Go version to 1.23.5 as it comes with some [minor security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.23.5).
+
+
+## v0.22.29
+
+- (_Backported from v0.23.11_) Upgraded `golang.org/x/net` to 0.33.0 to fix [CVE-2024-45338](https://www.cve.org/CVERecord?id=CVE-2024-45338).
+  _PocketBase uses the vulnerable functions primarily for the auto html->text mail generation, but most applications shouldn't be affected unless you are manually embedding unrestricted user provided value in your mail templates._
+
+
+## v0.22.28
+
+- (_Backported from v0.23.10_) Renew the superuser file token cache when clicking on the thumb preview or download link ([#6137](https://github.com/pocketbase/pocketbase/discussions/6137)).
+
+- (_Backported from v0.23.10_) Upgraded `modernc.org/sqlite` to 1.34.3 to fix "disk io" error on arm64 systems.
+  _If you are extending PocketBase with Go and upgrading with `go get -u` make sure to manually set in your go.mod the `modernc.org/libc` indirect dependency to v1.55.3, aka. the exact same version the driver is using._
+
+
+## v0.22.27
+
+- Instead of unregistering the realtime clients, we now just unset their auth state on delete of the related auth record so that the clients can receive the `delete` event ([#5898](https://github.com/pocketbase/pocketbase/issues/5898)).
+
+
 ## v0.22.26
 
 - (_Backported from v0.23.0-rc_) Added manual WAL checkpoints before creating the zip backup to minimize copying unnecessary data.
@@ -11,7 +190,7 @@
 
 - Refresh the old collections state in the Import UI after successful import submission ([#5861](https://github.com/pocketbase/pocketbase/issues/5861)).
 
-- Added randomized throttle on failed filter list requests as a very rudimentary measure since some security researches raised concern regarding the possibity of eventual side-channel attacks.
+- Added randomized throttle on failed filter list requests as a very rudimentary measure since some security researches raised concern regarding the possibility of eventual side-channel attacks.
 
 
 ## v0.22.24
@@ -413,7 +592,7 @@
   A negative or zero value means no tests timeout.
   If a single API test takes more than 3s to complete it will have a log message visible when the test fails or when `go test -v` flag is used.
 
-- Added timestamp at the beginning of the generated JSVM types file to avoid creating it everytime with the app startup.
+- Added timestamp at the beginning of the generated JSVM types file to avoid creating it every time with the app startup.
 
 
 ## v0.20.0
@@ -793,7 +972,7 @@
 
 - ⚠️ Deprecated `RelationOptions.DisplayFields` in favor of the new `SchemaField.Presentable` option to avoid the duplication when a single collection is referenced more than once and/or by multiple other collections.
 
-- ⚠️ Fill the `LastVerificationSentAt` and `LastResetSentAt` fields only after a successfull email send ([#3121](https://github.com/pocketbase/pocketbase/issues/3121)).
+- ⚠️ Fill the `LastVerificationSentAt` and `LastResetSentAt` fields only after a successful email send ([#3121](https://github.com/pocketbase/pocketbase/issues/3121)).
 
 - ⚠️ Skip API `fields` json transformations for non 20x responses ([#3176](https://github.com/pocketbase/pocketbase/issues/3176)).
 
@@ -1138,7 +1317,7 @@
 
 - Updated the schema fields Admin UI for "tidier" fields visualization.
 
-- Updated the logs "real" user IP to check for `Fly-Client-IP` header and changed the `X-Forward-For` header to use the first non-empty leftmost-ish IP as it the closest to the "real IP".
+- Updated the logs "real" user IP to check for `Fly-Client-IP` header and changed the `X-Forwarded-For` header to use the first non-empty leftmost-ish IP as it the closest to the "real IP".
 
 - Added new `tools/archive` helper subpackage for managing archives (_currently works only with zip_).
 

CONTRIBUTING.md

@@ -1,17 +1,20 @@
 # Contributing to PocketBase
 
-Thanks for taking the time to improve PocketBase!
+> [!IMPORTANT]
+> Due to recent LLM spam, PRs are temporary disabled and only existing collaborators can open a PR.
+> If you stumble on a problem that you want to fix, please consider instead opening an issue or discussion with link to your fork _(if not obvious - LLM contributions are not welcome)_.
+> This status may change in the future in case GitHub finally decide to do something about the constant spam, or when I find time to move the project somewhere else.
 
 This document describes how to prepare a PR for a change in the main repository.
 
 - [Prerequisites](#prerequisites)
 - [Making changes in the Go code](#making-changes-in-the-go-code)
-- [Making changes in the Admin UI](#making-changes-in-the-admin-ui)
+- [Making changes in the Superuser UI](#making-changes-in-the-admin-ui)
 
 ## Prerequisites
 
-- Go 1.23+ (for making changes in the Go code)
-- Node 18+ (for making changes in the Admin UI)
+- Go 1.27+ (for making changes in the Go code)
+- Node 24+ (for making changes in the Superuser UI)
 
 If you haven't already, you can fork the main repository and clone your fork so that you can work locally:
 
@@ -34,7 +37,7 @@ So, let's assume that you already done some changes in the PocketBase Go code an
 1. Navigate to `examples/base`
 2. Run `go run main.go serve`
 
-This will start a web server on `http://localhost:8090` with the embedded prebuilt Admin UI from `ui/dist`. And that's it!
+This will start a web server on `http://localhost:8090` with the embedded prebuilt Superuser UI from `ui/dist`. And that's it!
 
 **Before making a PR to the main repository, it is a good idea to:**
 
@@ -57,11 +60,11 @@ This will start a web server on `http://localhost:8090` with the embedded prebui
   make lint
   ```
 
-## Making changes in the Admin UI
+## Making changes in the Superuser UI
 
-PocketBase Admin UI is a single-page application (SPA) built with Svelte and Vite.
+PocketBase Superuser UI is a single-page application (SPA) built with Svelte and Vite.
 
-To start the Admin UI:
+To start the Superuser UI:
 
 1. Navigate to the `ui` project directory
 2. Run `npm install` to install the node dependencies
@@ -70,13 +73,13 @@ To start the Admin UI:
    npm run dev
    ```
 
-You could open the browser and access the running Admin UI at `http://localhost:3000`.
+You could open the browser and access the running Superuser UI at `http://localhost:5173`.
 
-Since the Admin UI is just a client-side application, you need to have the PocketBase backend server also running in the background (either manually running the `examples/base/main.go` or download a prebuilt executable).
+Since the Superuser UI is just a client-side application, you need to have the PocketBase backend server also running in the background (either manually running the `examples/base/main.go` or download a prebuilt executable).
 
 > [!NOTE]
-> By default, the Admin UI is expecting the backend server to be started at `http://localhost:8090`, but you could change that by creating a new `ui/.env.development.local` file with `PB_BACKEND_URL = YOUR_ADDRESS` variable inside it.
+> By default, the Superuser UI is expecting the backend server to be started at `http://localhost:8090`, but you could change that by creating a new `ui/.env.development.local` file with `PB_BACKEND_URL = YOUR_ADDRESS` variable inside it.
 
-Every change you make in the Admin UI should be automatically reflected in the browser at `http://localhost:3000` without reloading the page.
+Every change you make in the Superuser UI should be automatically reflected in the browser at `http://localhost:5173` without reloading the page.
 
-Once you are done with your changes, you have to build the Admin UI with `npm run build`, so that it can be embedded in the go package. And that's it - you can make your PR to the main PocketBase repository.
+Once you are done with your changes, you have to build the Superuser UI with `npm run build`, so that it can be embedded in the go package. And that's it - you can make your PR to the main PocketBase repository.

README.md

@@ -1,6 +1,6 @@
 <p align="center">
     <a href="https://pocketbase.io" target="_blank" rel="noopener">
-        <img src="https://i.imgur.com/5qimnm5.png" alt="PocketBase - open source backend in 1 file" />
+        <img src="https://i.imgur.com/aCBbjKx.png" alt="PocketBase - open source backend in 1 file" />
     </a>
 </p>
 
@@ -25,10 +25,13 @@
 
 ## API SDK clients
 
-The easiest way to interact with the API is to use one of the official SDK clients:
+The easiest way to interact with the PocketBase Web APIs is to use one of the official SDK clients:
+
+- **JavaScript - [pocketbase/js-sdk](https://github.com/pocketbase/js-sdk)** (_Browser, Node.js, React Native_)
+- **Dart - [pocketbase/dart-sdk](https://github.com/pocketbase/dart-sdk)** (_Web, Mobile, Desktop, CLI_)
+
+You could also check the recommendations in https://pocketbase.io/docs/how-to-use/.
 
-- **JavaScript - [pocketbase/js-sdk](https://github.com/pocketbase/js-sdk)** (_browser and node_)
-- **Dart - [pocketbase/dart-sdk](https://github.com/pocketbase/dart-sdk)** (_web, mobile, desktop_)
 
 ## Overview
 
@@ -46,7 +49,7 @@ your own custom app specific business logic and still have a single portable exe
 
 Here is a minimal example:
 
-0. [Install Go 1.23+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 
 1. Create a new project directory with the following `main.go` file inside it:
     ```go
@@ -54,7 +57,6 @@ Here is a minimal example:
 
     import (
         "log"
-        "net/http"
 
         "github.com/pocketbase/pocketbase"
         "github.com/pocketbase/pocketbase/core"
@@ -65,7 +67,7 @@ Here is a minimal example:
 
         app.OnServe().BindFunc(func(se *core.ServeEvent) error {
             // registers new "GET /hello" route
-            se.Router.Get("/hello", func(re *core.RequestEvent) error {
+            se.Router.GET("/hello", func(re *core.RequestEvent) error {
                 return re.String(200, "Hello world!")
             })
 
@@ -84,42 +86,43 @@ Here is a minimal example:
 
 4. To build a statically linked executable, you can run `CGO_ENABLED=0 go build` and then start the created executable with `./myapp serve`.
 
-> [!NOTE]
-> PocketBase embeds SQLite, but doesn't require CGO.
->
-> If CGO is enabled (aka. `CGO_ENABLED=1`), it will use [mattn/go-sqlite3](https://pkg.go.dev/github.com/mattn/go-sqlite3) driver, otherwise - [modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite).
-> Enable CGO only if you really need to squeeze the read/write query performance at the expense of complicating cross compilation.
-
 _For more details please refer to [Extend with Go](https://pocketbase.io/docs/go-overview/)._
 
 ### Building and running the repo main.go example
 
 To build the minimal standalone executable, like the prebuilt ones in the releases page, you can simply run `go build` inside the `examples/base` directory:
 
-0. [Install Go 1.21+](https://go.dev/doc/install) (_if you haven't already_)
+0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
 1. Clone/download the repo
 2. Navigate to `examples/base`
-3. Run `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`
-   (_https://go.dev/doc/install/source#environment_)
+3. Run `CGO_ENABLED=0 go build` to build a binary for your current environment
+   _(or to target other platforms use `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`; see https://go.dev/doc/install/source#environment)_
 4. Start the created executable by running `./base serve`.
 
 Note that the supported build targets by the pure Go SQLite driver at the moment are:
 
-```
-darwin  amd64
-darwin  arm64
-freebsd amd64
-freebsd arm64
-linux   386
-linux   amd64
-linux   arm
-linux   arm64
-linux   ppc64le
-linux   riscv64
-linux   s390x
-windows amd64
-windows arm64
-```
+| GOOS    | GOARCH  |
+|---------|---------|
+| darwin  | amd64   |
+| darwin  | arm64   |
+| freebsd | 386     |
+| freebsd | amd64   |
+| freebsd | arm     |
+| freebsd | arm64   |
+| linux   | 386     |
+| linux   | amd64   |
+| linux   | arm     |
+| linux   | arm64   |
+| linux   | loong64 |
+| linux   | ppc64le |
+| linux   | riscv64 |
+| linux   | s390x   |
+| netbsd  | amd64   |
+| openbsd | amd64   |
+| openbsd | arm64   |
+| windows | 386     |
+| windows | amd64   |
+| windows | arm64   |
 
 ### Testing
 
@@ -136,7 +139,7 @@ Check also the [Testing guide](http://pocketbase.io/docs/testing) to learn how t
 
 If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
 
-All reports will be promptly addressed and you'll be credited in the fix release notes.
+You could find more details in the project [Security policy](https://github.com/pocketbase/pocketbase/security/policy).
 
 ## Contributing
 
@@ -148,10 +151,13 @@ You could help continuing its development by:
 - [Contribute to the source code](CONTRIBUTING.md)
 - [Suggest new features and report issues](https://github.com/pocketbase/pocketbase/issues)
 
-PRs for new OAuth2 providers, bug fixes, code optimizations and documentation improvements are more than welcome.
-
-But please refrain creating PRs for _new features_ without previously discussing the implementation details.
+Please refrain creating PRs for _new features_ without previously discussing the implementation details.
 PocketBase has a [roadmap](https://github.com/orgs/pocketbase/projects/2) and I try to work on issues in specific order and such PRs often come in out of nowhere and skew all initial planning with tedious back-and-forth communication.
 
 Don't get upset if I close your PR, even if it is well executed and tested. This doesn't mean that it will never be merged.
 Later we can always refer to it and/or take pieces of your implementation when the time comes to work on the issue (don't worry you'll be credited in the release notes).
+
+> [!IMPORTANT]
+> Due to recent LLM spam, PRs are temporary disabled and only existing collaborators can open a PR.
+> If you stumble on a problem that you want to fix, please consider instead opening an issue or discussion with link to your fork _(if not obvious - LLM contributions are not welcome)_.
+> This status may change in the future in case GitHub finally decide to do something about the constant spam, or when I find time to move the project somewhere else.

apis/api_error_aliases.go

@@ -6,6 +6,11 @@ import "github.com/pocketbase/pocketbase/tools/router"
 // and for consistency with the JSVM binds.
 // -------------------------------------------------------------------
 
+// ToApiError wraps err into ApiError instance (if not already).
+func ToApiError(err error) *router.ApiError {
+	return router.ToApiError(err)
+}
+
 // NewApiError is an alias for [router.NewApiError].
 func NewApiError(status int, message string, errData any) *router.ApiError {
 	return router.NewApiError(status, message, errData)

apis/backup.go

@@ -18,7 +18,7 @@ func bindBackupApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
 	sub := rg.Group("/backups")
 	sub.GET("", backupsList).Bind(RequireSuperuserAuth())
 	sub.POST("", backupCreate).Bind(RequireSuperuserAuth())
-	sub.POST("/upload", backupUpload).Bind(RequireSuperuserAuth())
+	sub.POST("/upload", backupUpload).Bind(BodyLimit(0), RequireSuperuserAuth())
 	sub.GET("/{key}", backupDownload) // relies on superuser file token
 	sub.DELETE("/{key}", backupDelete).Bind(RequireSuperuserAuth())
 	sub.POST("/{key}/restore", backupRestore).Bind(RequireSuperuserAuth())
@@ -70,8 +70,10 @@ func backupDownload(e *core.RequestEvent) error {
 		return e.ForbiddenError("Insufficient permissions to access the resource.", err)
 	}
 
-	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
-	defer cancel()
+	allowedIPs := e.App.Settings().SuperuserIPs
+	if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+		return e.ForbiddenError("Insufficient permissions to access the resource.", nil)
+	}
 
 	fsys, err := e.App.NewBackupsFilesystem()
 	if err != nil {
@@ -79,6 +81,9 @@ func backupDownload(e *core.RequestEvent) error {
 	}
 	defer fsys.Close()
 
+	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
+	defer cancel()
+
 	fsys.SetContext(ctx)
 
 	key := e.Request.PathValue("key")

apis/backup_create.go

@@ -5,7 +5,7 @@ import (
 	"net/http"
 	"regexp"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 

apis/backup_test.go

@@ -10,9 +10,10 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
-	"gocloud.dev/blob"
+	"github.com/pocketbase/pocketbase/tools/filesystem/blob"
 )
 
 func TestBackupsList(t *testing.T) {
@@ -346,6 +347,19 @@ func TestBackupUpload(t *testing.T) {
 			ExpectedStatus: 204,
 			ExpectedEvents: map[string]int{"*": 0},
 		},
+		{
+			Name:   "ensure that the default body limit is skipped",
+			Method: http.MethodPost,
+			URL:    "/api/backups/upload",
+			Body:   bytes.NewBuffer(make([]byte, apis.DefaultMaxBodySize+100)),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus:     400, // it doesn't matter as long as it is not 413
+			ExpectedContent:    []string{`"data":{`},
+			NotExpectedContent: []string{"entity too large"},
+			ExpectedEvents:     map[string]int{"*": 0},
+		},
 	}
 
 	for _, scenario := range scenarios {
@@ -476,7 +490,7 @@ func TestBackupsDownload(t *testing.T) {
 					t.Fatal(err)
 				}
 			},
-			ExpectedStatus:  400,
+			ExpectedStatus:  404,
 			ExpectedContent: []string{`"data":{}`},
 			ExpectedEvents:  map[string]int{"*": 0},
 		},
@@ -514,6 +528,58 @@ func TestBackupsDownload(t *testing.T) {
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
+		{
+			Name:    "with valid superuser file token AND whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/backups/test1.zip?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := createTestBackups(app); err != nil {
+					t.Fatal(err)
+				}
+
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"127.0.0.1"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				"storage/",
+				"data.db",
+				"auxiliary.db",
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:    "with valid superuser file token BUT non-whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/backups/test1.zip?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := createTestBackups(app); err != nil {
+					t.Fatal(err)
+				}
+
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/backup_upload.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/filesystem"

apis/base.go

@@ -15,7 +15,7 @@ import (
 // StaticWildcardParam is the name of Static handler wildcard parameter.
 const StaticWildcardParam = "path"
 
-// NewRouter returns a new router instance loaded with the default app middlewares and api routes.
+// NewRouter returns a new router instance loaded with the default app middlewares and routes.
 func NewRouter(app core.App) (*router.Router[*core.RequestEvent], error) {
 	pbRouter := router.NewRouter(func(w http.ResponseWriter, r *http.Request) (*core.RequestEvent, router.EventCleanupFunc) {
 		event := new(core.RequestEvent)
@@ -31,9 +31,11 @@ func NewRouter(app core.App) (*router.Router[*core.RequestEvent], error) {
 	pbRouter.Bind(panicRecover())
 	pbRouter.Bind(rateLimit())
 	pbRouter.Bind(loadAuthToken())
+	pbRouter.Bind(superuserIPsWhitelist())
 	pbRouter.Bind(securityHeaders())
 	pbRouter.Bind(BodyLimit(DefaultMaxBodySize))
 
+	// API routes
 	apiGroup := pbRouter.Group("/api")
 	bindSettingsApi(app, apiGroup)
 	bindCollectionApi(app, apiGroup)
@@ -41,10 +43,15 @@ func NewRouter(app core.App) (*router.Router[*core.RequestEvent], error) {
 	bindRecordAuthApi(app, apiGroup)
 	bindLogsApi(app, apiGroup)
 	bindBackupApi(app, apiGroup)
+	bindCronApi(app, apiGroup)
 	bindFileApi(app, apiGroup)
 	bindBatchApi(app, apiGroup)
 	bindRealtimeApi(app, apiGroup)
 	bindHealthApi(app, apiGroup)
+	bindSQLApi(app, apiGroup)
+
+	// UI routes
+	bindUIExtensions(app)
 
 	return pbRouter, nil
 }
@@ -85,7 +92,7 @@ func MustSubFS(fsys fs.FS, dir string) fs.FS {
 
 // Static is a handler function to serve static directory content from fsys.
 //
-// If a file resource is missing and indexFallback is set, the request
+// If a file resource is missing and indexFallback is true, the request
 // will be forwarded to the base index.html (useful for SPA with pretty urls).
 //
 // NB! Expects the route to have a "{path...}" wildcard parameter.
@@ -93,7 +100,7 @@ func MustSubFS(fsys fs.FS, dir string) fs.FS {
 // Special redirects:
 //   - if "path" is a file that ends in index.html, it is redirected to its non-index.html version (eg. /test/index.html -> /test/)
 //   - if "path" is a directory that has index.html, the index.html file is rendered,
-//     otherwise if missing - returns 404 or fallback to the root index.html if indexFallback is set
+//     otherwise if missing - returns 404 or fallback to the root index.html if indexFallback is true
 //
 // Example:
 //

apis/batch.go

@@ -2,7 +2,7 @@ package apis
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"io"
 	"mime/multipart"
@@ -14,10 +14,11 @@ import (
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/types"
 	"github.com/spf13/cast"
 )
@@ -49,7 +50,7 @@ var ValidBatchActions = map[*regexp.Regexp]BatchActionHandlerFunc{
 				params["id"] = id // required for the path value
 				ir.Method = "PATCH"
 				ir.URL = "/api/collections/" + params["collection"] + "/records/" + id + params["query"]
-				return recordUpdate(next)
+				return recordUpdate(false, next)
 			}
 		}
 
@@ -57,16 +58,16 @@ var ValidBatchActions = map[*regexp.Regexp]BatchActionHandlerFunc{
 		// ---
 		ir.Method = "POST"
 		ir.URL = "/api/collections/" + params["collection"] + "/records" + params["query"]
-		return recordCreate(next)
+		return recordCreate(false, next)
 	},
 	regexp.MustCompile(`^POST /api/collections/(?P<collection>[^\/\?]+)/records(\?.*)?$`): func(app core.App, ir *core.InternalRequest, params map[string]string, next func(any) error) HandleFunc {
-		return recordCreate(next)
+		return recordCreate(false, next)
 	},
 	regexp.MustCompile(`^PATCH /api/collections/(?P<collection>[^\/\?]+)/records/(?P<id>[^\/\?]+)(\?.*)?$`): func(app core.App, ir *core.InternalRequest, params map[string]string, next func(any) error) HandleFunc {
-		return recordUpdate(next)
+		return recordUpdate(false, next)
 	},
 	regexp.MustCompile(`^DELETE /api/collections/(?P<collection>[^\/\?]+)/records/(?P<id>[^\/\?]+)(\?.*)?$`): func(app core.App, ir *core.InternalRequest, params map[string]string, next func(any) error) HandleFunc {
-		return recordDelete(next)
+		return recordDelete(false, next)
 	},
 }
 
@@ -88,7 +89,7 @@ func (brs batchRequestsForm) validate() error {
 }
 
 // NB! When the request is submitted as multipart/form-data,
-// the regular fields data is expected to be submitted as serailized
+// the regular fields data is expected to be submitted as serialized
 // json under the @jsonPayload field and file keys need to follow the
 // pattern "requests.N.fileField" or  requests[N].fileField.
 func batchTransaction(e *core.RequestEvent) error {
@@ -195,7 +196,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			p.stopCh <- struct{}{}
 		}()
 
-		go func() {
+		routine.FireAndForget(func() {
 			err := p.process(txApp, batch, 0)
 
 			if err != nil {
@@ -216,7 +217,7 @@ func (p *batchProcessor) Process(batch []*core.InternalRequest, timeout time.Dur
 			}
 
 			p.errCh <- err
-		}()
+		})
 
 		select {
 		case responseErr := <-p.errCh:
@@ -338,6 +339,12 @@ func processInternalRequest(
 	// apply batch request specific headers
 	// ---
 	for k, v := range ir.Headers {
+		// individual Authorization header keys don't have affect
+		// because the auth state is populated from the base event
+		if strings.EqualFold(k, "authorization") {
+			continue
+		}
+
 		r.Header.Set(k, v)
 	}
 	r.Header.Set("Content-Type", mw.FormDataContentType())
@@ -358,6 +365,7 @@ func processInternalRequest(
 	// assign request
 	event.Request = r
 	event.Request.Body = &router.RereadableReadCloser{ReadCloser: r.Body} // enables multiple reads
+	defer event.Request.Body.Close()
 
 	// assign response
 	rec := httptest.NewRecorder()

apis/batch_test.go

@@ -224,7 +224,7 @@ func TestBatchRequest(t *testing.T) {
 			},
 		},
 		{
-			Name:   "mixed create/update/delete (rules failure)",
+			Name:   "mixed create/update/delete (non-superuser rule failure)",
 			Method: http.MethodPost,
 			URL:    "/api/batch",
 			Body: strings.NewReader(`{
@@ -284,6 +284,70 @@ func TestBatchRequest(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "mixed create/update/delete (superuser rule failure)",
+			Method: http.MethodPost,
+			URL:    "/api/batch",
+			Headers: map[string]string{
+				// test@example.com, clients
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
+			Body: strings.NewReader(`{
+				"requests": [
+					{"method":"DELETE", "url":"/api/collections/demo2/records/achvryl401bhse3", "headers": {"Authorization": "ignored"}},
+					{"method":"PATCH", "url":"/api/collections/demo3/records/1tmknxy2868d869", "body": {"title": "batch_update"}, "headers": {"Authorization": "ignored"}},
+					{"method":"POST", "url":"/api/collections/_superusers/records", "body": {"email":"test_batch@example.com","password":"1234567890"}}
+				]
+			}`),
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"requests":{`,
+				`"2":{"code":"batch_request_failed"`,
+				`403`,
+			},
+			NotExpectedContent: []string{
+				`"0":`,
+				`"1":`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                        0,
+				"OnBatchRequest":           1,
+				"OnModelUpdate":            1,
+				"OnModelUpdateExecute":     1,
+				"OnModelAfterUpdateError":  1,
+				"OnModelDelete":            1,
+				"OnModelDeleteExecute":     1,
+				"OnModelAfterDeleteError":  1,
+				"OnModelValidate":          1,
+				"OnRecordUpdateRequest":    1,
+				"OnRecordUpdate":           1,
+				"OnRecordUpdateExecute":    1,
+				"OnRecordAfterUpdateError": 1,
+				"OnRecordDeleteRequest":    1,
+				"OnRecordDelete":           1,
+				"OnRecordDeleteExecute":    1,
+				"OnRecordAfterDeleteError": 1,
+				"OnRecordEnrich":           1,
+				"OnRecordValidate":         1,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				_, err = app.FindRecordById("demo2", "achvryl401bhse3")
+				if err != nil {
+					t.Fatal("Expected record to not be deleted")
+				}
+
+				_, err = app.FindFirstRecordByFilter("demo3", `title="batch_update"`)
+				if err == nil {
+					t.Fatal("Expected record to not be updated")
+				}
+
+				_, err = app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test_batch@example.com")
+				if err == nil {
+					t.Fatal("Expected superuser to not be created")
+				}
+			},
+		},
 		{
 			Name:   "mixed create/update/delete (rules success)",
 			Method: http.MethodPost,
@@ -294,9 +358,9 @@ func TestBatchRequest(t *testing.T) {
 			},
 			Body: strings.NewReader(`{
 				"requests": [
-					{"method":"POST", "url":"/api/collections/demo2/records", "body": {"title": "batch_create"}},
-					{"method":"DELETE", "url":"/api/collections/demo2/records/achvryl401bhse3"},
-					{"method":"PATCH", "url":"/api/collections/demo3/records/1tmknxy2868d869", "body": {"title": "batch_update"}}
+					{"method":"POST", "url":"/api/collections/demo2/records", "body": {"title": "batch_create"}, "headers": {"Authorization": "ignored"}},
+					{"method":"DELETE", "url":"/api/collections/demo2/records/achvryl401bhse3", "headers": {"Authorization": "ignored"}},
+					{"method":"PATCH", "url":"/api/collections/demo3/records/1tmknxy2868d869", "body": {"title": "batch_update"}, "headers": {"Authorization": "ignored"}}
 				]
 			}`),
 			ExpectedStatus: 200,

apis/collection.go

@@ -3,12 +3,15 @@ package apis
 import (
 	"errors"
 	"net/http"
+	"slices"
 	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/router"
 	"github.com/pocketbase/pocketbase/tools/search"
+	"github.com/pocketbase/pocketbase/tools/security"
 )
 
 // bindCollectionApi registers the collection api endpoints and the corresponding handlers.
@@ -22,6 +25,10 @@ func bindCollectionApi(app core.App, rg *router.RouterGroup[*core.RequestEvent])
 	subGroup.DELETE("/{collection}/truncate", collectionTruncate)
 	subGroup.PUT("/import", collectionsImport)
 	subGroup.GET("/meta/scaffolds", collectionScaffolds)
+
+	// @todo experimental
+	subGroup.GET("/meta/oauth2-providers", collectionListOAuth2Providers)
+	subGroup.POST("/meta/dry-run-view", collectionDryRunView)
 }
 
 func collectionsList(e *core.RequestEvent) error {
@@ -45,7 +52,9 @@ func collectionsList(e *core.RequestEvent) error {
 	event.Result = result
 
 	return event.App.OnCollectionsListRequest().Trigger(event, func(e *core.CollectionsListRequestEvent) error {
-		return e.JSON(http.StatusOK, e.Result)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Result)
+		})
 	})
 }
 
@@ -60,7 +69,9 @@ func collectionView(e *core.RequestEvent) error {
 	event.Collection = collection
 
 	return e.App.OnCollectionViewRequest().Trigger(event, func(e *core.CollectionRequestEvent) error {
-		return e.JSON(http.StatusOK, e.Collection)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Collection)
+		})
 	})
 }
 
@@ -98,7 +109,9 @@ func collectionCreate(e *core.RequestEvent) error {
 			return e.BadRequestError("Failed to create collection. Raw error: \n"+err.Error(), nil)
 		}
 
-		return e.JSON(http.StatusOK, e.Collection)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Collection)
+		})
 	})
 }
 
@@ -128,7 +141,9 @@ func collectionUpdate(e *core.RequestEvent) error {
 			return e.BadRequestError("Failed to update collection. Raw error: \n"+err.Error(), nil)
 		}
 
-		return e.JSON(http.StatusOK, e.Collection)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Collection)
+		})
 	})
 }
 
@@ -159,7 +174,9 @@ func collectionDelete(e *core.RequestEvent) error {
 			return e.BadRequestError(msg, err)
 		}
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 
@@ -182,15 +199,98 @@ func collectionTruncate(e *core.RequestEvent) error {
 }
 
 func collectionScaffolds(e *core.RequestEvent) error {
+	randomId := security.RandomStringWithAlphabet(10, core.DefaultIdAlphabet) // could be used as part of the default indexes name
+
 	collections := map[string]*core.Collection{
-		core.CollectionTypeBase: core.NewBaseCollection(""),
-		core.CollectionTypeAuth: core.NewAuthCollection(""),
-		core.CollectionTypeView: core.NewViewCollection(""),
+		core.CollectionTypeBase: core.NewBaseCollection("", randomId),
+		core.CollectionTypeAuth: core.NewAuthCollection("", randomId),
+		core.CollectionTypeView: core.NewViewCollection("", randomId),
 	}
 
 	for _, c := range collections {
-		c.Id = "" // clear autogenerated id
+		c.Id = "" // clear random id
 	}
 
 	return e.JSON(http.StatusOK, collections)
 }
+
+type providerListItem struct {
+	order int
+
+	Name        string `json:"name"`
+	DisplayName string `json:"displayName"`
+	Logo        string `json:"logo"`
+}
+
+func collectionListOAuth2Providers(e *core.RequestEvent) error {
+	providers := make([]*providerListItem, 0, len(auth.Providers))
+
+	for name, factory := range auth.Providers {
+		p := factory()
+
+		providers = append(providers, &providerListItem{
+			order:       p.Order(),
+			Name:        name,
+			DisplayName: p.DisplayName(),
+			Logo:        p.Logo(),
+		})
+	}
+
+	slices.SortStableFunc(providers, func(a, b *providerListItem) int {
+		// sort by order
+		if a.order < b.order {
+			return -1
+		}
+		if a.order > b.order {
+			return 1
+		}
+
+		// fallback sort by name
+		if a.Name < b.Name {
+			return -1
+		}
+		if a.Name > b.Name {
+			return 1
+		}
+
+		return 0
+	})
+
+	return e.JSON(http.StatusOK, providers)
+}
+
+func collectionDryRunView(e *core.RequestEvent) error {
+	// extra precaution in case reused in custom route group
+	if !e.HasSuperuserAuth() {
+		return e.ForbiddenError("", nil)
+	}
+
+	form := dryRunViewForm{}
+
+	err := e.BindBody(&form)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while loading the submitted data.", err))
+	}
+
+	err = form.validate()
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
+	}
+
+	result, err := e.App.DryRunView(form.Query, 10)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("Invalid view query. Raw error: \n"+err.Error(), nil))
+	}
+
+	return e.JSON(http.StatusOK, result)
+}
+
+type dryRunViewForm struct {
+	Query string `form:"query" json:"query"`
+}
+
+func (form *dryRunViewForm) validate() error {
+	return validation.ValidateStruct(form,
+		validation.Field(&form.Query, validation.Required, validation.Length(0, 5000)),
+	)
+}

apis/collection_import.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 
@@ -29,12 +29,14 @@ func collectionsImport(e *core.RequestEvent) error {
 	return event.App.OnCollectionsImportRequest().Trigger(event, func(e *core.CollectionsImportRequestEvent) error {
 		importErr := e.App.ImportCollections(e.CollectionsData, form.DeleteMissing)
 		if importErr == nil {
-			return e.NoContent(http.StatusNoContent)
+			return execAfterSuccessTx(true, e.App, func() error {
+				return e.NoContent(http.StatusNoContent)
+			})
 		}
 
 		// validation failure
 		var validationErrors validation.Errors
-		if errors.As(err, &validationErrors) {
+		if errors.As(importErr, &validationErrors) {
 			return e.BadRequestError("Failed to import collections.", validationErrors)
 		}
 

apis/collection_import_test.go

@@ -1,6 +1,7 @@
 package apis_test
 
 import (
+	"errors"
 	"net/http"
 	"strings"
 	"testing"
@@ -88,6 +89,7 @@ func TestCollectionsImport(t *testing.T) {
 				`import2`,
 				`fields`,
 			},
+			NotExpectedContent: []string{"Raw error:"},
 			ExpectedEvents: map[string]int{
 				"*":                            0,
 				"OnCollectionsImportRequest":   1,
@@ -109,6 +111,71 @@ func TestCollectionsImport(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "authorized as superuser + non-validator failure",
+			Method: http.MethodPut,
+			URL:    "/api/collections/import",
+			Body: strings.NewReader(`{
+				"collections":[
+					{
+						"name": "import1",
+						"fields": [
+							{
+							  "id": "koih1lqx",
+							  "name": "test",
+							  "type": "text"
+							}
+						]
+					},
+					{
+						"name": "import2",
+						"fields": [
+							{
+							  "id": "koih1lqx",
+							  "name": "test",
+							  "type": "text"
+							}
+						],
+						"indexes": [
+							"create index idx_test on import2 (test)"
+						]
+					}
+				]
+			}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"collections":{"code":"validation_collections_import_failure"`,
+				`Raw error:`,
+				`custom_error`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                            0,
+				"OnCollectionsImportRequest":   1,
+				"OnCollectionCreate":           1,
+				"OnCollectionAfterCreateError": 1,
+				"OnModelCreate":                1,
+				"OnModelAfterCreateError":      1,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnCollectionCreate().BindFunc(func(e *core.CollectionEvent) error {
+					return errors.New("custom_error")
+				})
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				collections := []*core.Collection{}
+				if err := app.CollectionQuery().All(&collections); err != nil {
+					t.Fatal(err)
+				}
+				expected := totalCollections
+				if len(collections) != expected {
+					t.Fatalf("Expected %d collections, got %d", expected, len(collections))
+				}
+			},
+		},
 		{
 			Name:   "authorized as superuser + successful collections create",
 			Method: http.MethodPut,
@@ -249,6 +316,51 @@ func TestCollectionsImport(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "OnCollectionsImportRequest tx body write check",
+			Method: http.MethodPut,
+			URL:    "/api/collections/import",
+			Body: strings.NewReader(`{
+				"deleteMissing": true,
+				"collections":[
+					{"name": "test123"},
+					{
+						"id":"wsmn24bux7wo113",
+						"name":"demo1",
+						"fields":[
+							{
+								"id":"_2hlxbmp",
+								"name":"title",
+								"type":"text",
+								"required":true
+							}
+						],
+						"indexes": []
+					}
+				]
+			}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnCollectionsImportRequest().BindFunc(func(e *core.CollectionsImportRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnCollectionsImportRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/collection_test.go

@@ -1,7 +1,6 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"os"
 	"path/filepath"
@@ -130,6 +129,32 @@ func TestCollectionsList(t *testing.T) {
 				"OnCollectionsListRequest": 1,
 			},
 		},
+		{
+			Name:   "OnCollectionsListRequest tx body write check",
+			Method: http.MethodGet,
+			URL:    "/api/collections",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnCollectionsListRequest().BindFunc(func(e *core.CollectionsListRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnCollectionsListRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 	}
 
 	for _, scenario := range scenarios {
@@ -205,6 +230,32 @@ func TestCollectionView(t *testing.T) {
 				"OnCollectionViewRequest": 1,
 			},
 		},
+		{
+			Name:   "OnCollectionViewRequest tx body write check",
+			Method: http.MethodGet,
+			URL:    "/api/collections/wsmn24bux7wo113",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnCollectionViewRequest().BindFunc(func(e *core.CollectionRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnCollectionViewRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 	}
 
 	for _, scenario := range scenarios {
@@ -361,7 +412,7 @@ func TestCollectionDelete(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnCollectionAfterDeleteSuccessRequest error response",
+			Name:   "OnCollectionDeleteRequest tx body write check",
 			Method: http.MethodDelete,
 			URL:    "/api/collections/view2",
 			Headers: map[string]string{
@@ -369,15 +420,22 @@ func TestCollectionDelete(t *testing.T) {
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnCollectionDeleteRequest().BindFunc(func(e *core.CollectionRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                         0,
-				"OnCollectionDeleteRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnCollectionDeleteRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 	}
 
@@ -478,7 +536,7 @@ func TestCollectionCreate(t *testing.T) {
 				`"type":"base"`,
 				`"system":false`,
 				// ensures that id field was prepended
-				`"fields":[{"autogeneratePattern":"[a-z0-9]{15}","hidden":false,"id":"text3208210256","max":15,"min":15,"name":"id","pattern":"^[a-z0-9]+$","presentable":false,"primaryKey":true,"required":true,"system":true,"type":"text"},{"autogeneratePattern":"","hidden":false,"id":"12345789","max":0,"min":0,"name":"test","pattern":"","presentable":false,"primaryKey":false,"required":false,"system":false,"type":"text"}]`,
+				`"fields":[{"autogeneratePattern":"[a-z0-9]{15}","help":"","hidden":false,"id":"text3208210256","max":15,"min":15,"name":"id","pattern":"^[a-z0-9]+$","presentable":false,"primaryKey":true,"required":true,"system":true,"type":"text"},{"autogeneratePattern":"","help":"","hidden":false,"id":"12345789","max":0,"min":0,"name":"test","pattern":"","presentable":false,"primaryKey":false,"required":false,"system":false,"type":"text"}]`,
 			},
 			ExpectedEvents: map[string]int{
 				"*":                              0,
@@ -527,7 +585,7 @@ func TestCollectionCreate(t *testing.T) {
 				`"name":"verified"`,
 				`"duration":123`,
 				// should overwrite the user required option but keep the min value
-				`{"autogeneratePattern":"","hidden":true,"id":"text2504183744","max":0,"min":10,"name":"tokenKey","pattern":"","presentable":false,"primaryKey":false,"required":true,"system":true,"type":"text"}`,
+				`{"autogeneratePattern":"","help":"","hidden":true,"id":"text2504183744","max":0,"min":10,"name":"tokenKey","pattern":"","presentable":false,"primaryKey":false,"required":true,"system":true,"type":"text"}`,
 			},
 			NotExpectedContent: []string{
 				`"secret":"`,
@@ -656,7 +714,7 @@ func TestCollectionCreate(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnCollectionCreateRequest error response",
+			Name:   "OnCollectionCreateRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections",
 			Body:   strings.NewReader(`{"name":"new","type":"base"}`),
@@ -665,15 +723,22 @@ func TestCollectionCreate(t *testing.T) {
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnCollectionCreateRequest().BindFunc(func(e *core.CollectionRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                         0,
-				"OnCollectionCreateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnCollectionCreateRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// view
@@ -686,7 +751,7 @@ func TestCollectionCreate(t *testing.T) {
 				"name":"new",
 				"type":"view",
 				"fields":[{"type":"text","id":"12345789","name":"ignored!@#$"}],
-				"viewQuery":"invalid"
+				"viewQuery":"select '123' as abc"
 			}`),
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
@@ -715,7 +780,7 @@ func TestCollectionCreate(t *testing.T) {
 				"name":"new",
 				"type":"view",
 				"fields":[{"type":"text","id":"12345789","name":"ignored!@#$"}],
-				"viewQuery": "select 1 as id from ` + core.CollectionNameSuperusers + `"
+				"viewQuery": "select 1 as id from ` + core.CollectionNameSuperusers + ` limit 1"
 			}`),
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
@@ -724,7 +789,7 @@ func TestCollectionCreate(t *testing.T) {
 			ExpectedContent: []string{
 				`"name":"new"`,
 				`"type":"view"`,
-				`"fields":[{"autogeneratePattern":"","hidden":false,"id":"text3208210256","max":0,"min":0,"name":"id","pattern":"","presentable":false,"primaryKey":true,"required":true,"system":true,"type":"text"}]`,
+				`"fields":[{"autogeneratePattern":"","help":"","hidden":false,"id":"text3208210256","max":0,"min":0,"name":"id","pattern":"^[a-z0-9]+$","presentable":false,"primaryKey":true,"required":true,"system":true,"type":"text"}]`,
 			},
 			ExpectedEvents: map[string]int{
 				"*":                              0,
@@ -978,7 +1043,7 @@ func TestCollectionUpdate(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnCollectionAfterUpdateSuccessRequest error response",
+			Name:   "OnCollectionUpdateRequest tx body write check",
 			Method: http.MethodPatch,
 			URL:    "/api/collections/demo1",
 			Body:   strings.NewReader(`{}`),
@@ -987,15 +1052,22 @@ func TestCollectionUpdate(t *testing.T) {
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnCollectionUpdateRequest().BindFunc(func(e *core.CollectionRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                         0,
-				"OnCollectionUpdateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnCollectionUpdateRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 		{
 			Name:   "authorized as superuser + invalid data (eg. existing name)",
@@ -1153,6 +1225,71 @@ func TestCollectionUpdate(t *testing.T) {
 				"OnModelValidate":              1,
 			},
 		},
+		{
+			Name:   "add another OAuth2 provider to an auth collection",
+			Method: http.MethodPatch,
+			URL:    "/api/collections/users",
+			Body: strings.NewReader(`{
+				"oauth2": {
+					"providers": [
+						{"name": "apple", "clientId": "a", "clientSecret": "b"},
+						{
+							"pkce": null,
+							"name": "google",
+							"authURL": "",
+							"displayName": "existing",
+							"extra": {}
+						}
+					]
+				}
+			}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				// verify that the collection has google and gitlab as OAuth2 providers
+				users, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if v := len(users.OAuth2.Providers); v != 2 {
+					t.Fatalf("Expected 2 OAuth2 providers, got %d", v)
+				}
+
+				if v := users.OAuth2.Providers[0].Name; v != "gitlab" {
+					t.Fatalf("Expected provider 0 to be %s, got %s", "gitlab", v)
+				}
+
+				if v := users.OAuth2.Providers[1].Name; v != "google" {
+					t.Fatalf("Expected provider 1 to be %s, got %s", "google", v)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"name":"google"`,
+				`"name":"apple"`,
+				`"displayName":"existing"`,
+				`"clientId":"test"`,
+				`"clientId":"a"`,
+			},
+			NotExpectedContent: []string{
+				`"name":"gitlab"`,
+				`clientSecret`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                              0,
+				"OnCollectionUpdateRequest":      1,
+				"OnCollectionUpdate":             1,
+				"OnCollectionUpdateExecute":      1,
+				"OnCollectionAfterUpdateSuccess": 1,
+				"OnCollectionValidate":           1,
+				"OnModelUpdate":                  1,
+				"OnModelUpdateExecute":           1,
+				"OnModelAfterUpdateSuccess":      1,
+				"OnModelValidate":                1,
+			},
+		},
 
 		// view
 		// -----------------------------------------------------------
@@ -1190,7 +1327,7 @@ func TestCollectionUpdate(t *testing.T) {
 			Body: strings.NewReader(`{
 				"name":"view2_update",
 				"fields":[{"type":"text","id":"12345789","name":"ignored!@#$"}],
-				"viewQuery": "select 2 as id, created, updated, email from ` + core.CollectionNameSuperusers + `"
+				"viewQuery": "select 2 as id, created, updated, email from ` + core.CollectionNameSuperusers + ` limit 1"
 			}`),
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
@@ -1512,3 +1649,187 @@ func TestCollectionTruncate(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestCollectionOAuth2Providers(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodGet,
+			URL:             "/api/collections/meta/oauth2-providers",
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodGet,
+			URL:    "/api/collections/meta/oauth2-providers",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodGet,
+			URL:    "/api/collections/meta/oauth2-providers",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`{"name":"oidc3","displayName":"OIDC","logo":"<svg`,
+			},
+			NotExpectedContent: []string{
+				`"order":`,
+				`"pkce":`,
+				`"scopes":`,
+				`"authURL":`,
+				`"tokenURL":`,
+				`"userInfoURL":`,
+			},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}
+
+func TestCollectionTestView(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodPost,
+			URL:             "/api/collections/meta/dry-run-view",
+			Body:            strings.NewReader(`{"query":"select 1 as id"}`),
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"select 1 as id"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"select 1 as id"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"fields":[{`,
+				`"name":"id"`,
+				`"type":"text"`,
+				`"sample":[{`,
+				`"id":"1"`,
+			},
+		},
+		{
+			Name:   "empty query",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":""}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{"query":`,
+			},
+		},
+		{
+			Name:   "query length beyond validator limit",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"` + strings.Repeat("a", 5001) + `"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{"query":`,
+			},
+		},
+		{
+			Name:   "query with length equal to the validator limit",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"select 1 as id` + strings.Repeat(" ", 4986) + `"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"fields":[{`,
+				`"name":"id"`,
+				`"type":"text"`,
+				`"sample":[`,
+				`"id":"1"`,
+			},
+		},
+		{
+			Name:   "missing ids sample",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"(select 1 as id union select '' as id)"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+			},
+		},
+		{
+			Name:   "duplicated ids sample",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"(select 1 as id union all select 1 as id)"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+			},
+		},
+		{
+			Name:   "write query",
+			Method: http.MethodPost,
+			URL:    "/api/collections/meta/dry-run-view",
+			Body:   strings.NewReader(`{"query":"CREATE TABLE t1(x INT)"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+			},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/cron.go

@@ -0,0 +1,59 @@
+package apis
+
+import (
+	"net/http"
+	"slices"
+	"strings"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/cron"
+	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/pocketbase/pocketbase/tools/routine"
+)
+
+// bindCronApi registers the crons api endpoint.
+func bindCronApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
+	subGroup := rg.Group("/crons").Bind(RequireSuperuserAuth())
+	subGroup.GET("", cronsList)
+	subGroup.POST("/{id}", cronRun)
+}
+
+func cronsList(e *core.RequestEvent) error {
+	jobs := e.App.Cron().Jobs()
+
+	slices.SortStableFunc(jobs, func(a, b *cron.Job) int {
+		if strings.HasPrefix(a.Id(), "__pb") {
+			return 1
+		}
+		if strings.HasPrefix(b.Id(), "__pb") {
+			return -1
+		}
+		return strings.Compare(a.Id(), b.Id())
+	})
+
+	return e.JSON(http.StatusOK, jobs)
+}
+
+func cronRun(e *core.RequestEvent) error {
+	cronId := e.Request.PathValue("id")
+
+	var foundJob *cron.Job
+
+	jobs := e.App.Cron().Jobs()
+	for _, j := range jobs {
+		if j.Id() == cronId {
+			foundJob = j
+			break
+		}
+	}
+
+	if foundJob == nil {
+		return e.NotFoundError("Missing or invalid cron job", nil)
+	}
+
+	routine.FireAndForget(func() {
+		foundJob.Run()
+	})
+
+	return e.NoContent(http.StatusNoContent)
+}

apis/cron_test.go

@@ -0,0 +1,148 @@
+package apis_test
+
+import (
+	"net/http"
+	"testing"
+	"time"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tests"
+	"github.com/spf13/cast"
+)
+
+func TestCronsList(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodGet,
+			URL:             "/api/crons",
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodGet,
+			URL:    "/api/crons",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser (empty list)",
+			Method: http.MethodGet,
+			URL:    "/api/crons",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Cron().RemoveAll()
+			},
+			ExpectedStatus:  200,
+			ExpectedContent: []string{`[]`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodGet,
+			URL:    "/api/crons",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`{"id":"__pbLogsCleanup__","expression":"0 */6 * * *"}`,
+				`{"id":"__pbDBOptimize__","expression":"0 0 * * *"}`,
+				`{"id":"__pbMFACleanup__","expression":"0 * * * *"}`,
+				`{"id":"__pbOTPCleanup__","expression":"0 * * * *"}`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}
+
+func TestCronsRun(t *testing.T) {
+	t.Parallel()
+
+	beforeTestFunc := func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+		app.Cron().Add("test", "* * * * *", func() {
+			app.Store().Set("testJobCalls", cast.ToInt(app.Store().Get("testJobCalls"))+1)
+		})
+		app.Cron().Stop()
+	}
+
+	expectedCalls := func(expected int) func(t testing.TB, app *tests.TestApp, res *http.Response) {
+		return func(t testing.TB, app *tests.TestApp, res *http.Response) {
+			time.Sleep(50 * time.Millisecond)
+
+			total := cast.ToInt(app.Store().Get("testJobCalls"))
+			if total != expected {
+				t.Fatalf("Expected total testJobCalls %d, got %d", expected, total)
+			}
+		}
+	}
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodPost,
+			URL:             "/api/crons/test",
+			BeforeTestFunc:  beforeTestFunc,
+			AfterTestFunc:   expectedCalls(0),
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodPost,
+			URL:    "/api/crons/test",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			BeforeTestFunc:  beforeTestFunc,
+			AfterTestFunc:   expectedCalls(0),
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser (missing job)",
+			Method: http.MethodPost,
+			URL:    "/api/crons/missing",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  beforeTestFunc,
+			AfterTestFunc:   expectedCalls(0),
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser (existing job)",
+			Method: http.MethodPost,
+			URL:    "/api/crons/test",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: beforeTestFunc,
+			AfterTestFunc:  expectedCalls(1),
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/extensions.go

@@ -0,0 +1,95 @@
+package apis
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"io"
+	"log/slog"
+	"os"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/hook"
+	"github.com/pocketbase/pocketbase/ui"
+)
+
+// bindUIExtensions binds the superuser UI extensions routes to the ServeEvent.Router.
+//
+// This method does nothing if the superuser UI is not bundled (aka. build with "no_ui" tag),
+func bindUIExtensions(app core.App) {
+	if ui.DistDirFS == nil {
+		return
+	}
+
+	app.OnServe().Bind(&hook.Handler[*core.ServeEvent]{
+		Priority: 9999, // execute as latest as possible
+		Func: func(se *core.ServeEvent) error {
+			uiGroup := se.Router.Group("/_").
+				BindFunc(func(e *core.RequestEvent) error {
+					if !e.App.IsDev() && e.Response.Header().Get("Cache-Control") == "" {
+						e.Response.Header().Set("Cache-Control", "max-age=1209600, stale-while-revalidate=86400")
+					}
+
+					if e.Response.Header().Get("Content-Security-Policy") == "" {
+						e.Response.Header().Set("Content-Security-Policy", defaultCSP)
+					}
+
+					return e.Next()
+				}).
+				Bind(Gzip())
+
+			// register static extension routes
+			for _, ext := range se.UIExtensions {
+				if ext.Name == "" || ext.FS == nil {
+					se.App.Logger().Debug("Invalid UI extension configuration", slog.Any("extension", ext))
+					continue
+				}
+
+				uiGroup.GET("/extensions/"+ext.Name+"/{path...}", Static(ext.FS, false))
+			}
+
+			// combine all extensions main.js in one file
+			//
+			// note: don't cache in memory to allow previewing changes without restart
+			uiGroup.GET("/extensions.js", func(re *core.RequestEvent) error {
+				buf := new(bytes.Buffer)
+
+				for _, ext := range se.UIExtensions {
+					err := copyExtensionMainjs(buf, ext)
+					if err != nil {
+						return re.InternalServerError("An error occurred while generating the main.js extension file", err)
+					}
+				}
+
+				return re.Stream(200, "text/javascript", buf)
+			}).Bind(SkipSuccessActivityLog())
+
+			return se.Next()
+		},
+	})
+}
+
+func copyExtensionMainjs(buf *bytes.Buffer, ext core.UIExtension) error {
+	f, err := ext.FS.Open("main.js")
+	if err != nil {
+		if errors.Is(err, os.ErrNotExist) {
+			return nil // nothing to copy
+		}
+
+		return fmt.Errorf("[UI extension %q] main.js open error: %w", ext.Name, err)
+	}
+	defer f.Close()
+
+	// wrap in a self-executing function to avoid scope and concatenation issues
+	// (the await/async is for top-level await)
+	_, _ = buf.WriteString("await (async function(){")
+
+	_, err = io.Copy(buf, f)
+	if err != nil {
+		return fmt.Errorf("[UI extension %q] main.js copy error: %w", ext.Name, err)
+	}
+
+	_, _ = buf.WriteString("})();")
+
+	return nil
+}

apis/extensions_test.go

@@ -0,0 +1,177 @@
+package apis_test
+
+import (
+	"net/http"
+	"testing"
+	"testing/fstest"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/ui"
+)
+
+// note: don't run in parallel to avoid conflicts with the ui.DistDirFS nil test
+func TestUIExtensions_Mainjs(t *testing.T) {
+	successAfterTestFunc := func(t testing.TB, app *tests.TestApp, res *http.Response) {
+		expected := "text/javascript"
+		if ct := res.Header.Get("content-type"); ct != expected {
+			t.Fatalf("Expected response Content-Type %q, got %q", expected, ct)
+		}
+	}
+
+	oldDistDirFS := ui.DistDirFS
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:   "disabled UI",
+			Method: http.MethodGet,
+			URL:    "/_/extensions.js",
+			TestAppFactory: func(t testing.TB) *tests.TestApp {
+				app, err := tests.NewTestApp()
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// simulate no_ui tag (needs to be cleared before the router is initialized)
+				ui.DistDirFS = nil
+
+				return app
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				ui.DistDirFS = oldDistDirFS
+			},
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:            "no extensions",
+			Method:          http.MethodGet,
+			URL:             "/_/extensions.js",
+			AfterTestFunc:   successAfterTestFunc,
+			ExpectedStatus:  200,
+			ExpectedContent: []string{},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "with extensions",
+			Method: http.MethodGet,
+			URL:    "/_/extensions.js",
+			TestAppFactory: func(t testing.TB) *tests.TestApp {
+				app, err := tests.NewTestApp()
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				app.OnServe().BindFunc(func(e *core.ServeEvent) error {
+					e.UIExtensions = createTestExtensions()
+					return e.Next()
+				})
+
+				return app
+			},
+			AfterTestFunc:   successAfterTestFunc,
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"await (async function(){ext1_main})();await (async function(){ext3_main})();"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}
+
+// note: don't run in parallel to avoid conflicts with the ui.DistDirFS nil test
+func TestUIExtensions_Files(t *testing.T) {
+	testAppFactory := func(t testing.TB) *tests.TestApp {
+		app, err := tests.NewTestApp()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		app.OnServe().BindFunc(func(e *core.ServeEvent) error {
+			e.UIExtensions = createTestExtensions()
+			return e.Next()
+		})
+
+		return app
+	}
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "no extensions",
+			Method:          http.MethodGet,
+			URL:             "/_/extensions/ext1/test.txt",
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:            "with missing extension file",
+			Method:          http.MethodGet,
+			URL:             "/_/extensions/ext1/missing",
+			TestAppFactory:  testAppFactory,
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:            "with existing extension file (ext1)",
+			Method:          http.MethodGet,
+			URL:             "/_/extensions/ext1/test.txt",
+			TestAppFactory:  testAppFactory,
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"ext1_txt"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:            "with existing extension file (extension name escape)",
+			Method:          http.MethodGet,
+			URL:             "/_/extensions/ext3%20with%20spaces/test.txt",
+			TestAppFactory:  testAppFactory,
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"ext3_txt"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}
+
+func createTestExtensions() []core.UIExtension {
+	return []core.UIExtension{
+		{
+			Name: "ext1",
+			FS: fstest.MapFS{
+				"main.js": &fstest.MapFile{
+					Data: []byte("ext1_main"),
+				},
+				"test.txt": &fstest.MapFile{
+					Data: []byte("ext1_txt"),
+				},
+			},
+		},
+		{
+			Name: "ext2",
+			FS: fstest.MapFS{
+				"test.txt": &fstest.MapFile{
+					Data: []byte("ext2_txt"),
+				},
+			},
+		},
+		{
+			Name: "ext3 with spaces",
+			FS: fstest.MapFS{
+				"main.js": &fstest.MapFile{
+					Data: []byte("ext3_main"),
+				},
+				"test.txt": &fstest.MapFile{
+					Data: []byte("ext3_txt"),
+				},
+			},
+		},
+	}
+}

apis/file.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"log/slog"
 	"net/http"
+	"os"
 	"runtime"
 	"time"
 
@@ -13,19 +14,30 @@ import (
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/spf13/cast"
 	"golang.org/x/sync/semaphore"
 	"golang.org/x/sync/singleflight"
 )
 
-var imageContentTypes = []string{"image/png", "image/jpg", "image/jpeg", "image/gif"}
+var imageContentTypes = []string{"image/png", "image/jpg", "image/jpeg", "image/gif", "image/webp"}
 var defaultThumbSizes = []string{"100x100"}
 
 // bindFileApi registers the file api endpoints and the corresponding handlers.
 func bindFileApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
+	maxWorkers := cast.ToInt64(os.Getenv("PB_THUMBS_MAX_WORKERS"))
+	if maxWorkers <= 0 {
+		maxWorkers = int64(runtime.NumCPU() + 2) // the value is arbitrary chosen and may change in the future
+	}
+
+	maxWait := cast.ToInt64(os.Getenv("PB_THUMBS_MAX_WAIT"))
+	if maxWait <= 0 {
+		maxWait = 60
+	}
+
 	api := fileApi{
-		thumbGenSem:     semaphore.NewWeighted(int64(runtime.NumCPU() + 2)), // the value is arbitrary chosen and may change in the future
 		thumbGenPending: new(singleflight.Group),
-		thumbGenMaxWait: 60 * time.Second,
+		thumbGenSem:     semaphore.NewWeighted(maxWorkers),
+		thumbGenMaxWait: time.Duration(maxWait) * time.Second,
 	}
 
 	sub := rg.Group("/files")
@@ -48,6 +60,7 @@ type fileApi struct {
 }
 
 func (api *fileApi) fileToken(e *core.RequestEvent) error {
+	// extra check for just in case the handler is called in a different context
 	if e.Auth == nil {
 		return e.UnauthorizedError("Missing auth context.", nil)
 	}
@@ -63,8 +76,8 @@ func (api *fileApi) fileToken(e *core.RequestEvent) error {
 	event.Record = e.Auth
 
 	return e.App.OnFileTokenRequest().Trigger(event, func(e *core.FileTokenRequestEvent) error {
-		return e.JSON(http.StatusOK, map[string]string{
-			"token": e.Token,
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, map[string]string{"token": e.Token})
 		})
 	})
 }
@@ -102,6 +115,15 @@ func (api *fileApi) download(e *core.RequestEvent) error {
 		token := e.Request.URL.Query().Get("token")
 		authRecord, _ := e.App.FindAuthRecordByToken(token, core.TokenTypeFile)
 
+		// reset the auth state if it is superuser and it is not whitelisted
+		// (not critical because file tokens are short-lived but checked nonetheless as an extra precaution)
+		if authRecord != nil && authRecord.IsSuperuser() {
+			allowedIPs := e.App.Settings().SuperuserIPs
+			if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+				authRecord = nil
+			}
+		}
+
 		// create a shallow copy of the cached request data and adjust it to the current auth record (if any)
 		requestInfo := *originalRequestInfo
 		requestInfo.Context = core.RequestInfoContextProtectedFile
@@ -130,8 +152,14 @@ func (api *fileApi) download(e *core.RequestEvent) error {
 	defer fsys.Close()
 
 	originalPath := baseFilesPath + "/" + filename
-	servedPath := originalPath
-	servedName := filename
+
+	event := new(core.FileDownloadRequestEvent)
+	event.RequestEvent = e
+	event.Collection = collection
+	event.Record = record
+	event.FileField = fileField
+	event.ServedPath = originalPath
+	event.ServedName = filename
 
 	// check for valid thumb size param
 	thumbSize := e.Request.URL.Query().Get("thumb")
@@ -145,34 +173,31 @@ func (api *fileApi) download(e *core.RequestEvent) error {
 		// check if it is an image
 		if list.ExistInSlice(oAttrs.ContentType, imageContentTypes) {
 			// add thumb size as file suffix
-			servedName = thumbSize + "_" + filename
-			servedPath = baseFilesPath + "/thumbs_" + filename + "/" + servedName
+			event.ServedName = thumbSize + "_" + filename
+			event.ServedPath = baseFilesPath + "/thumbs_" + filename + "/" + event.ServedName
 
 			// create a new thumb if it doesn't exist
-			if exists, _ := fsys.Exists(servedPath); !exists {
-				if err := api.createThumb(e, fsys, originalPath, servedPath, thumbSize); err != nil {
+			if exists, _ := fsys.Exists(event.ServedPath); !exists {
+				if err := api.createThumb(e, fsys, originalPath, event.ServedPath, thumbSize); err != nil {
 					e.App.Logger().Warn(
-						"Fallback to original - failed to create thumb "+servedName,
+						"Fallback to original - failed to create thumb "+event.ServedName,
 						slog.Any("error", err),
 						slog.String("original", originalPath),
-						slog.String("thumb", servedPath),
+						slog.String("thumb", event.ServedPath),
 					)
 
 					// fallback to the original
-					servedName = filename
-					servedPath = originalPath
+					event.ThumbError = err
+					event.ServedName = filename
+					event.ServedPath = originalPath
 				}
 			}
 		}
 	}
 
-	event := new(core.FileDownloadRequestEvent)
-	event.RequestEvent = e
-	event.Collection = collection
-	event.Record = record
-	event.FileField = fileField
-	event.ServedPath = servedPath
-	event.ServedName = servedName
+	if thumbSize != "" && event.ThumbError == nil && event.ServedPath == originalPath {
+		event.ThumbError = fmt.Errorf("the thumb size %q or the original file format are not supported", thumbSize)
+	}
 
 	// clickjacking shouldn't be a concern when serving uploaded files,
 	// so it safe to unset the global X-Frame-Options to allow files embedding
@@ -180,7 +205,10 @@ func (api *fileApi) download(e *core.RequestEvent) error {
 	e.Response.Header().Del("X-Frame-Options")
 
 	return e.App.OnFileDownloadRequest().Trigger(event, func(e *core.FileDownloadRequestEvent) error {
-		if err := fsys.Serve(e.Response, e.Request, e.ServedPath, e.ServedName); err != nil {
+		err = execAfterSuccessTx(true, e.App, func() error {
+			return fsys.Serve(e.Response, e.Request, e.ServedPath, e.ServedName)
+		})
+		if err != nil {
 			return e.NotFoundError("", err)
 		}
 

apis/file_test.go

@@ -181,9 +181,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - missing thumb (should fallback to the original)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=999x999",
+			Name:   "existing image - missing thumb (should fallback to the original)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=999x999",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError == nil {
+						t.Fatal("Expected thumb error, got nil")
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testImg)},
 			ExpectedEvents: map[string]int{
@@ -192,9 +200,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (crop center)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50",
+			Name:   "existing image - existing thumb (crop center)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbCropCenter)},
 			ExpectedEvents: map[string]int{
@@ -203,9 +219,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (crop top)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50t",
+			Name:   "existing image - existing thumb (crop top)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50t",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbCropTop)},
 			ExpectedEvents: map[string]int{
@@ -214,9 +238,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (crop bottom)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50b",
+			Name:   "existing image - existing thumb (crop bottom)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50b",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbCropBottom)},
 			ExpectedEvents: map[string]int{
@@ -225,9 +257,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (fit)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50f",
+			Name:   "existing image - existing thumb (fit)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x50f",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbFit)},
 			ExpectedEvents: map[string]int{
@@ -236,9 +276,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (zero width)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=0x50",
+			Name:   "existing image - existing thumb (zero width)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=0x50",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbZeroWidth)},
 			ExpectedEvents: map[string]int{
@@ -247,9 +295,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing image - existing thumb (zero height)",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x0",
+			Name:   "existing image - existing thumb (zero height)",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/4q1xlclmfloku33/300_1SEi6Q6U72.png?thumb=70x0",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError != nil {
+						t.Fatalf("Expected no thumb error, got %v", e.ThumbError)
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testThumbZeroHeight)},
 			ExpectedEvents: map[string]int{
@@ -258,9 +314,17 @@ func TestFileDownload(t *testing.T) {
 			},
 		},
 		{
-			Name:            "existing non image file - thumb parameter should be ignored",
-			Method:          http.MethodGet,
-			URL:             "/api/files/_pb_users_auth_/oap640cot4yru2s/test_kfd2wYLxkz.txt?thumb=100x100",
+			Name:   "existing non image file - thumb parameter should be ignored",
+			Method: http.MethodGet,
+			URL:    "/api/files/_pb_users_auth_/oap640cot4yru2s/test_kfd2wYLxkz.txt?thumb=100x100",
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnFileDownloadRequest().BindFunc(func(e *core.FileDownloadRequestEvent) error {
+					if e.ThumbError == nil {
+						t.Fatal("Expected thumb error, got nil")
+					}
+					return e.Next()
+				})
+			},
 			ExpectedStatus:  200,
 			ExpectedContent: []string{string(testFile)},
 			ExpectedEvents: map[string]int{
@@ -289,6 +353,50 @@ func TestFileDownload(t *testing.T) {
 				"OnFileDownloadRequest": 1,
 			},
 		},
+		{
+			Name:    "protected file - superuser with non-whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/files/demo1/al1h9ijdeojtsjy/300_Jsjq7RdBgA.png?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+				err := app.Save(app.Settings())
+				if err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:    "protected file - superuser with whitelisted IP",
+			Method:  http.MethodGet,
+			URL:     "/api/files/demo1/al1h9ijdeojtsjy/300_Jsjq7RdBgA.png?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6ImZpbGUiLCJjb2xsZWN0aW9uSWQiOiJwYmNfMzE0MjYzNTgyMyJ9.Lupz541xRvrktwkrl55p5pPCF77T69ZRsohsIcb2dxc",
+			Headers: map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy = core.TrustedProxyConfig{
+					Headers: []string{"x-test-ip"},
+				}
+
+				app.Settings().SuperuserIPs = []string{"127.0.0.1"}
+
+				if err := app.Save(app.Settings()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"PNG"},
+			ExpectedEvents: map[string]int{
+				"*":                     0,
+				"OnFileDownloadRequest": 1,
+			},
+		},
 		{
 			Name:            "protected file - guest without view access",
 			Method:          http.MethodGet,

apis/health.go

@@ -25,6 +25,7 @@ func healthCheck(e *core.RequestEvent) error {
 		Message: "API is healthy.",
 	}
 
+	// @todo evaluate whether it is worth removing the extra info from the health endpoint
 	if e.HasSuperuserAuth() {
 		resp.Data = make(map[string]any, 3)
 		resp.Data["canBackup"] = !e.App.Store().Has(core.StoreKeyActiveBackup)
@@ -36,7 +37,7 @@ func healthCheck(e *core.RequestEvent) error {
 		headersToCheck := append(
 			slices.Clone(e.App.Settings().TrustedProxy.Headers),
 			// common proxy headers
-			"CF-Connecting-IP", "Fly-Client-IP", "X‑Forwarded-For",
+			"CF-Connecting-IP", "Fly-Client-IP", "X-Forwarded-For",
 		)
 		for _, header := range headersToCheck {
 			if e.Request.Header.Get(header) != "" {

apis/installer.go

@@ -5,44 +5,61 @@ import (
 	"errors"
 	"fmt"
 	"os"
-	"os/exec"
-	"runtime"
 	"strings"
 	"time"
 
 	"github.com/fatih/color"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
 	"github.com/pocketbase/dbx"
 	"github.com/pocketbase/pocketbase/core"
-	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/osutils"
+	"github.com/pocketbase/pocketbase/ui"
 )
 
-// @todo consider combining with the installer specific hooks after refactoring cmd
-func loadInstaller(app core.App, dashboardURL string) error {
-	if !needInstallerSuperuser(app) {
+// DefaultInstallerFunc is the default PocketBase installer function.
+//
+// It will attempt to open a link in the browser (with a short-lived auth
+// token for the systemSuperuser) to the installer UI so that users can
+// create their own custom superuser record.
+//
+// See https://github.com/pocketbase/pocketbase/discussions/5814.
+func DefaultInstallerFunc(app core.App, systemSuperuser *core.Record, baseURL string) error {
+	if ui.DistDirFS == nil {
+		color.Magenta("You can create your first superuser by running: %s superuser upsert EMAIL PASS", executablePath())
 		return nil
 	}
 
-	installerRecord, err := findOrCreateInstallerSuperuser(app)
-	if err != nil {
-		return err
-	}
-
-	token, err := installerRecord.NewStaticAuthToken(30 * time.Minute)
+	token, err := systemSuperuser.NewStaticAuthToken(30 * time.Minute)
 	if err != nil {
 		return err
 	}
 
 	// launch url (ignore errors and always print a help text as fallback)
-	url := fmt.Sprintf("%s/#/pbinstal/%s", strings.TrimRight(dashboardURL, "/"), token)
-	_ = launchURL(url)
+	url := fmt.Sprintf("%s/_/#/pbinstall/%s", strings.TrimRight(baseURL, "/"), token)
+	_ = osutils.LaunchURL(url)
 	color.Magenta("\n(!) Launch the URL below in the browser if it hasn't been open already to create your first superuser account:")
 	color.New(color.Bold).Add(color.FgCyan).Println(url)
-	color.New(color.FgHiBlack, color.Italic).Printf("(you can also create your first superuser by running: %s superuser upsert EMAIL PASS)\n\n", os.Args[0])
+	color.New(color.FgHiBlack, color.Italic).Printf("(you can also create your first superuser by running: %s superuser upsert EMAIL PASS)\n\n", executablePath())
 
 	return nil
 }
 
+func loadInstaller(
+	app core.App,
+	baseURL string,
+	installerFunc func(app core.App, systemSuperuser *core.Record, baseURL string) error,
+) error {
+	if installerFunc == nil || !needInstallerSuperuser(app) {
+		return nil
+	}
+
+	superuser, err := findOrCreateInstallerSuperuser(app)
+	if err != nil {
+		return err
+	}
+
+	return installerFunc(app, superuser, baseURL)
+}
+
 func needInstallerSuperuser(app core.App) bool {
 	total, err := app.CountRecords(core.CollectionNameSuperusers, dbx.Not(dbx.HashExp{
 		"email": core.DefaultInstallerEmail,
@@ -65,7 +82,7 @@ func findOrCreateInstallerSuperuser(app core.App) (*core.Record, error) {
 
 		record = core.NewRecord(col)
 		record.SetEmail(core.DefaultInstallerEmail)
-		record.SetPassword(security.RandomString(30))
+		record.SetRandomPassword()
 
 		err = app.Save(record)
 		if err != nil {
@@ -76,19 +93,10 @@ func findOrCreateInstallerSuperuser(app core.App) (*core.Record, error) {
 	return record, nil
 }
 
-func launchURL(url string) error {
-	if err := is.URL.Validate(url); err != nil {
-		return err
+func executablePath() string {
+	if osutils.IsProbablyGoRun() {
+		return "go run ."
 	}
 
-	switch runtime.GOOS {
-	case "darwin":
-		return exec.Command("open", url).Start()
-	case "windows":
-		// not sure if this is the best command but seems to be the most reliable based on the comments in
-		// https://stackoverflow.com/questions/3739327/launching-a-website-via-the-windows-commandline#answer-49115945
-		return exec.Command("rundll32", "url.dll,FileProtocolHandler", url).Start()
-	default: // linux, freebsd, etc.
-		return exec.Command("xdg-open", url).Start()
-	}
+	return os.Args[0]
 }

apis/logs.go

@@ -11,8 +11,13 @@ import (
 
 // bindLogsApi registers the request logs api endpoints.
 func bindLogsApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
-	sub := rg.Group("/logs").Bind(RequireSuperuserAuth(), SkipSuccessActivityLog())
+	sub := rg.Group("/logs").Bind(
+		RequireSuperuserAuth(),
+		SkipSuccessActivityLog(),
+	)
+
 	sub.GET("", logsList)
+	sub.DELETE("", logsTruncate)
 	sub.GET("/stats", logsStats)
 	sub.GET("/{id}", logsView)
 }
@@ -71,3 +76,19 @@ func logsView(e *core.RequestEvent) error {
 
 	return e.JSON(http.StatusOK, log)
 }
+
+func logsTruncate(e *core.RequestEvent) error {
+	// delete all rows directly (aka. no model hooks will be fired)
+	_, err := e.App.AuxNonconcurrentDB().Delete((&core.Log{}).TableName(), nil).Execute()
+	if err != nil {
+		return e.InternalServerError("Failed to truncate all logs.", err)
+	}
+
+	// try to free the unused disk space
+	err = e.App.AuxVacuum()
+	if err != nil {
+		e.App.Logger().Warn("Failed to VACUUM aux database", "error", err)
+	}
+
+	return e.NoContent(http.StatusNoContent)
+}

apis/logs_test.go

@@ -210,3 +210,59 @@ func TestLogsStats(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestLogsTruncate(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "unauthorized",
+			Method:          http.MethodDelete,
+			URL:             "/api/logs",
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as regular user",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "authorized as superuser",
+			Method: http.MethodDelete,
+			URL:    "/api/logs",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				if err := tests.StubLogsData(app); err != nil {
+					t.Fatal(err)
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				var found []core.Log
+
+				if err := app.LogQuery().All(&found); err != nil {
+					t.Fatal(err)
+				}
+
+				if len(found) > 0 {
+					t.Fatalf("Expected all logs to be deleted, found: %v", found)
+				}
+			},
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/middlewares.go

@@ -42,6 +42,9 @@ const (
 	DefaultLoadAuthTokenMiddlewarePriority = DefaultRateLimitMiddlewarePriority - 20
 	DefaultLoadAuthTokenMiddlewareId       = "pbLoadAuthToken"
 
+	DefaultSuperuserIPsWhitelistMiddlewarePriority = DefaultLoadAuthTokenMiddlewarePriority + 5
+	DefaultSuperuserIPsWhitelistMiddlewareId       = "pbSuperuserIPsWhitelist"
+
 	DefaultSecurityHeadersMiddlewarePriority = DefaultRateLimitMiddlewarePriority - 10
 	DefaultSecurityHeadersMiddlewareId       = "pbSecurityHeaders"
 
@@ -207,11 +210,13 @@ func loadAuthToken() *hook.Handler[*core.RequestEvent] {
 
 func getAuthTokenFromRequest(e *core.RequestEvent) string {
 	token := e.Request.Header.Get("Authorization")
-	if token != "" {
-		// the schema prefix is not required and it is only for
-		// compatibility with the defaults of some HTTP clients
-		token = strings.TrimPrefix(token, "Bearer ")
+
+	// the "Bearer" schema prefix is not required by PocketBase and it is
+	// supported only for compatibility with the defaults of some HTTP clients
+	if len(token) > 7 && strings.EqualFold(token[:7], "Bearer ") {
+		return token[7:]
 	}
+
 	return token
 }
 
@@ -288,6 +293,7 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
 			e.Response.Header().Set("X-XSS-Protection", "1; mode=block")
 			e.Response.Header().Set("X-Content-Type-Options", "nosniff")
 			e.Response.Header().Set("X-Frame-Options", "SAMEORIGIN")
+			e.Response.Header().Set("Cross-Origin-Opener-Policy", "same-origin")
 
 			// @todo consider a default HSTS?
 			// (see also https://webkit.org/blog/8146/protecting-against-hsts-abuse/)
@@ -297,6 +303,28 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
 	}
 }
 
+// superuserIPsWhitelist middleware checks the current authenticated superuser IP
+// against the configured SuperuserIPs whitelist setting.
+//
+// This middleware is registered by default for all routes.
+func superuserIPsWhitelist() *hook.Handler[*core.RequestEvent] {
+	return &hook.Handler[*core.RequestEvent]{
+		Id:       DefaultSuperuserIPsWhitelistMiddlewareId,
+		Priority: DefaultSuperuserIPsWhitelistMiddlewarePriority,
+		Func: func(e *core.RequestEvent) error {
+			if e.HasSuperuserAuth() {
+				ips := e.App.Settings().SuperuserIPs
+
+				if len(ips) > 0 && !isIPInList(ips, e.RealIP()) {
+					return e.ForbiddenError("", errors.New("superuser IP is not whitelisted"))
+				}
+			}
+
+			return e.Next()
+		},
+	}
+}
+
 // SkipSuccessActivityLog is a helper middleware that instructs the global
 // activity logger to log only requests that have failed/returned an error.
 func SkipSuccessActivityLog() *hook.Handler[*core.RequestEvent] {
@@ -365,11 +393,25 @@ func logRequest(event *core.RequestEvent, err error) {
 
 	// parse the request error
 	if err != nil {
-		if apiErr, ok := err.(*router.ApiError); ok {
-			status = apiErr.Status
+		apiErr, isPlainApiError := err.(*router.ApiError)
+		if isPlainApiError || errors.As(err, &apiErr) {
+			// the status header wasn't written yet
+			if status == 0 {
+				status = apiErr.Status
+			}
+
+			var errMsg string
+			if isPlainApiError {
+				errMsg = apiErr.Message
+			} else {
+				// wrapped ApiError -> add the full serialized version
+				// of the original error since it could contain more information
+				errMsg = err.Error()
+			}
+
 			attrs = append(
 				attrs,
-				slog.String("error", apiErr.Message),
+				slog.String("error", errMsg),
 				slog.Any("details", apiErr.RawData()),
 			)
 		} else {

apis/middlewares_body_limit.go

@@ -11,14 +11,14 @@ import (
 
 var ErrRequestEntityTooLarge = router.NewApiError(http.StatusRequestEntityTooLarge, "Request entity too large", nil)
 
-const DefaultMaxBodySize int64 = 32 << 20
+const DefaultMaxBodySize int64 = 32 << 20 // @todo consider replacing with router.DefaultMaxMemory
 
 const (
 	DefaultBodyLimitMiddlewareId       = "pbBodyLimit"
 	DefaultBodyLimitMiddlewarePriority = DefaultRateLimitMiddlewarePriority + 10
 )
 
-// BodyLimit returns a middleware function that changes the default request body size limit.
+// BodyLimit returns a middleware handler that changes the default request body size limit.
 //
 // If limitBytes <= 0, no limit is applied.
 //
@@ -84,37 +84,73 @@ func applyBodyLimit(e *core.RequestEvent, limitBytes int64) error {
 	}
 
 	// replace the request body
-	//
-	// note: we don't use sync.Pool since the size of the elements could vary too much
-	// and it might not be efficient (see https://github.com/golang/go/issues/23199)
-	e.Request.Body = &limitedReader{ReadCloser: e.Request.Body, limit: limitBytes}
+	e.Request.Body = newMaxBytesReader(e.Request.Body, limitBytes)
 
 	return nil
 }
 
-type limitedReader struct {
-	io.ReadCloser
-	limit     int64
-	totalRead int64
+func newMaxBytesReader(body io.ReadCloser, limitBytes int64) *maxBytesReader {
+	return &maxBytesReader{
+		ReadCloser: body,
+		limit:      limitBytes,
+		remaining:  limitBytes,
+	}
 }
 
-func (r *limitedReader) Read(b []byte) (int, error) {
+// maxBytesReader is very similar to the http.MaxBytesReader but support
+// rereads and doesn't try to prematurely close the related response
+// to allow consequent middlewares to operate correctly.
+type maxBytesReader struct {
+	io.ReadCloser
+	limit     int64
+	remaining int64
+	stickyErr error
+}
+
+func (r *maxBytesReader) Read(b []byte) (int, error) {
+	if r.stickyErr != nil {
+		return 0, r.stickyErr
+	}
+
+	if len(b) == 0 {
+		return 0, nil
+	}
+
+	// if possible no need to read the entire chunk since
+	// remaining+1 is enough to determine whether it exceed the limit
+	if int64(len(b))-1 > r.remaining {
+		b = b[:r.remaining+1]
+	}
+
 	n, err := r.ReadCloser.Read(b)
-	if err != nil {
+
+	if int64(n) <= r.remaining {
+		r.remaining -= int64(n)
+		r.stickyErr = err
 		return n, err
 	}
 
-	r.totalRead += int64(n)
-	if r.totalRead > r.limit {
-		return n, ErrRequestEntityTooLarge
-	}
+	n = int(r.remaining)
 
-	return n, nil
+	r.remaining = 0
+	r.stickyErr = ErrRequestEntityTooLarge
+
+	return n, r.stickyErr
 }
 
-func (r *limitedReader) Reread() {
-	rr, ok := r.ReadCloser.(router.Rereader)
+// explicit casts to ensure that the main struct methods will be invoked
+// (extra precautions in case of nested interface wrapping erasure)
+// ---
+
+func (r *maxBytesReader) Reread() {
+	rereader, ok := r.ReadCloser.(router.Rereader)
 	if ok {
-		rr.Reread()
+		rereader.Reread()
+		r.remaining = r.limit
+		r.stickyErr = nil
 	}
 }
+
+func (r *maxBytesReader) Close() error {
+	return r.ReadCloser.Close()
+}

apis/middlewares_body_limit_test.go

@@ -1,9 +1,10 @@
 package apis_test
 
 import (
-	"bytes"
-	"fmt"
+	"io"
+	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/apis"
@@ -19,13 +20,35 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	pbRouter.POST("/a", func(e *core.RequestEvent) error {
-		return e.String(200, "a")
-	}) // default global BodyLimit check
 
-	pbRouter.POST("/b", func(e *core.RequestEvent) error {
-		return e.String(200, "b")
-	}).Bind(apis.BodyLimit(20))
+	testHandler := func(e *core.RequestEvent) error {
+		// read the body multiple times to ensure that the limited
+		// reader guards and rereads are invoked
+		var result any
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		if err := e.BindBody(&result); err != nil {
+			return err
+		}
+
+		return e.JSON(200, result)
+	}
+
+	const customLimit = 20
+
+	pbRouter.POST("/a", testHandler) // default global BodyLimit check
+	pbRouter.POST("/b", testHandler).Bind(apis.BodyLimit(customLimit))
+	pbRouter.POST("/iof", func(e *core.RequestEvent) error {
+		// ensure that normal io methods still operate correctly
+		b, err := io.ReadAll(e.Request.Body)
+		if err != nil {
+			return err
+		}
+
+		return e.String(http.StatusOK, string(b))
+	}).Bind(apis.BodyLimit(customLimit))
 
 	mux, err := pbRouter.BuildMux()
 	if err != nil {
@@ -33,20 +56,90 @@ func TestBodyLimitMiddleware(t *testing.T) {
 	}
 
 	scenarios := []struct {
-		url            string
-		size           int64
-		expectedStatus int
+		name              string
+		url               string
+		body              string
+		lazyContentLength bool
+		expectedStatus    int
 	}{
-		{"/a", 21, 200},
-		{"/a", apis.DefaultMaxBodySize + 1, 413},
-		{"/b", 20, 200},
-		{"/b", 21, 413},
+		{
+			"(eager content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > default limit",
+			"/a",
+			`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"(eager content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			false,
+			http.StatusOK,
+		},
+		{
+			"(eager content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			false,
+			http.StatusRequestEntityTooLarge,
+		},
+		{
+			"(lazy content-length check) with body = custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
+		{
+			"(lazy content-length check) with body > custom limit",
+			"/b",
+			`"` + strings.Repeat("a", customLimit) + `"`,
+			true,
+			http.StatusRequestEntityTooLarge,
+		},
+		// ---
+		{
+			"io.ReadAll io.EOF exact limit check",
+			"/iof",
+			`"` + strings.Repeat("a", customLimit-2) + `"`,
+			true,
+			http.StatusOK,
+		},
 	}
 
 	for _, s := range scenarios {
-		t.Run(fmt.Sprintf("%s_%d", s.url, s.size), func(t *testing.T) {
+		t.Run(s.name, func(t *testing.T) {
 			rec := httptest.NewRecorder()
-			req := httptest.NewRequest("POST", s.url, bytes.NewReader(make([]byte, s.size)))
+
+			req := httptest.NewRequest("POST", s.url, strings.NewReader(s.body))
+			req.Header.Set("Content-Type", "application/json")
+
+			if s.lazyContentLength {
+				req.ContentLength = -1
+			}
+
 			mux.ServeHTTP(rec, req)
 
 			result := rec.Result()

apis/middlewares_cors.go

@@ -13,6 +13,7 @@ package apis
 // -------------------------------------------------------------------
 
 import (
+	"log"
 	"net/http"
 	"regexp"
 	"strconv"
@@ -124,8 +125,8 @@ var DefaultCORSConfig = CORSConfig{
 	AllowMethods: []string{http.MethodGet, http.MethodHead, http.MethodPut, http.MethodPatch, http.MethodPost, http.MethodDelete},
 }
 
-// CORSWithConfig returns a CORS middleware with config.
-func CORSWithConfig(config CORSConfig) *hook.Handler[*core.RequestEvent] {
+// CORS returns a CORS middleware.
+func CORS(config CORSConfig) *hook.Handler[*core.RequestEvent] {
 	// Defaults
 	if len(config.AllowOrigins) == 0 {
 		config.AllowOrigins = DefaultCORSConfig.AllowOrigins
@@ -134,13 +135,26 @@ func CORSWithConfig(config CORSConfig) *hook.Handler[*core.RequestEvent] {
 		config.AllowMethods = DefaultCORSConfig.AllowMethods
 	}
 
-	allowOriginPatterns := []string{}
+	allowOriginPatterns := make([]*regexp.Regexp, 0, len(config.AllowOrigins))
 	for _, origin := range config.AllowOrigins {
+		if origin == "*" {
+			continue // "*" is handled differently and does not need regexp
+		}
+
 		pattern := regexp.QuoteMeta(origin)
 		pattern = strings.ReplaceAll(pattern, "\\*", ".*")
 		pattern = strings.ReplaceAll(pattern, "\\?", ".")
 		pattern = "^" + pattern + "$"
-		allowOriginPatterns = append(allowOriginPatterns, pattern)
+
+		re, err := regexp.Compile(pattern)
+		if err != nil {
+			// This is to preserve previous behaviour - invalid patterns were just ignored.
+			// If we would turn this to panic, users with invalid patterns
+			// would have applications crashing in production due unrecovered panic.
+			log.Println("invalid AllowOrigins pattern", origin)
+			continue
+		}
+		allowOriginPatterns = append(allowOriginPatterns, re)
 	}
 
 	allowMethods := strings.Join(config.AllowMethods, ",")
@@ -210,7 +224,7 @@ func CORSWithConfig(config CORSConfig) *hook.Handler[*core.RequestEvent] {
 				}
 				if checkPatterns {
 					for _, re := range allowOriginPatterns {
-						if match, _ := regexp.MatchString(re, origin); match {
+						if match := re.MatchString(origin); match {
 							allowOrigin = origin
 							break
 						}
@@ -272,6 +286,7 @@ func matchSubdomain(domain, pattern string) bool {
 	if !matchScheme(domain, pattern) {
 		return false
 	}
+
 	didx := strings.Index(domain, "://")
 	pidx := strings.Index(pattern, "://")
 	if didx == -1 || pidx == -1 {
@@ -307,5 +322,6 @@ func matchSubdomain(domain, pattern string) bool {
 			return false
 		}
 	}
+
 	return false
 }

apis/middlewares_gzip.go

@@ -221,23 +221,26 @@ func (w *gzipResponseWriter) Push(target string, opts *http.PushOptions) error {
 	}
 }
 
-func (w *gzipResponseWriter) ReadFrom(r io.Reader) (n int64, err error) {
-	if w.wroteHeader {
-		w.ResponseWriter.WriteHeader(w.code)
-	}
-
-	rw := w.ResponseWriter
-	for {
-		switch rf := rw.(type) {
-		case io.ReaderFrom:
-			return rf.ReadFrom(r)
-		case router.RWUnwrapper:
-			rw = rf.Unwrap()
-		default:
-			return io.Copy(w.ResponseWriter, r)
-		}
-	}
-}
+// Note: Disable the implementation for now because in case the platform
+// supports the sendfile fast-path it won't run gzipResponseWriter.Write,
+// preventing compression on the fly.
+//
+// func (w *gzipResponseWriter) ReadFrom(r io.Reader) (n int64, err error) {
+// 	if w.wroteHeader {
+// 		w.ResponseWriter.WriteHeader(w.code)
+// 	}
+// 	rw := w.ResponseWriter
+// 	for {
+// 		switch rf := rw.(type) {
+// 		case io.ReaderFrom:
+// 			return rf.ReadFrom(r)
+// 		case router.RWUnwrapper:
+// 			rw = rf.Unwrap()
+// 		default:
+// 			return io.Copy(w.ResponseWriter, r)
+// 		}
+// 	}
+// }
 
 func (w *gzipResponseWriter) Unwrap() http.ResponseWriter {
 	return w.ResponseWriter

apis/middlewares_rate_limit.go

@@ -1,6 +1,8 @@
 package apis
 
 import (
+	"errors"
+	"net/netip"
 	"sync"
 	"time"
 
@@ -32,9 +34,12 @@ func rateLimit() *hook.Handler[*core.RequestEvent] {
 				return e.Next()
 			}
 
-			rule, ok := e.App.Settings().RateLimits.FindRateLimitRule(defaultRateLimitLabels(e))
+			rule, ok := e.App.Settings().RateLimits.FindRateLimitRule(
+				defaultRateLimitLabels(e),
+				defaultRateLimitAudience(e)...,
+			)
 			if ok {
-				err := checkRateLimit(e, e.Request.Pattern, rule)
+				err := checkRateLimit(e, rule.Label+rule.Audience, rule)
 				if err != nil {
 					return err
 				}
@@ -94,38 +99,51 @@ func checkCollectionRateLimit(e *core.RequestEvent, collection *core.Collection,
 	}
 	labels = append(labels, defaultRateLimitLabels(e)...)
 
-	rule, ok := e.App.Settings().RateLimits.FindRateLimitRule(labels)
+	rule, ok := e.App.Settings().RateLimits.FindRateLimitRule(labels, defaultRateLimitAudience(e)...)
 	if ok {
-		return checkRateLimit(e, rtId, rule)
+		return checkRateLimit(e, rtId+rule.Audience, rule)
 	}
 
 	return nil
 }
 
-// -------------------------------------------------------------------
-
-// @todo consider exporting as helper?
-//
-//nolint:unused
-func isClientRateLimited(e *core.RequestEvent, rtId string) bool {
-	rateLimiters, ok := e.App.Store().Get(rateLimitersStoreKey).(*store.Store[*rateLimiter])
-	if !ok || rateLimiters == nil {
+// isIPInList checks if the specified IP is in a list of other individual IPs or subnets.
+func isIPInList(ipsOrSubnets []string, ip string) bool {
+	if len(ipsOrSubnets) == 0 || ip == "" {
 		return false
 	}
 
-	rt, ok := rateLimiters.GetOk(rtId)
-	if !ok || rt == nil {
+	// normalize
+	searchAddr, err := netip.ParseAddr(ip)
+	if err != nil {
 		return false
 	}
 
-	client, ok := rt.getClient(e.RealIP())
-	if !ok || client == nil {
-		return false
+	for _, item := range ipsOrSubnets {
+		// subnet?
+		prefix, err := netip.ParsePrefix(item)
+		if err == nil {
+			if prefix.Contains(searchAddr) {
+				return true
+			}
+			continue
+		}
+
+		// individual ip?
+		addr, err := netip.ParseAddr(item)
+		if err == nil {
+			if addr == searchAddr {
+				return true
+			}
+			continue
+		}
 	}
 
-	return client.available <= 0 && time.Now().Unix()-client.lastConsume < client.interval
+	return false
 }
 
+// -------------------------------------------------------------------
+
 // @todo consider exporting as helper?
 func checkRateLimit(e *core.RequestEvent, rtId string, rule core.RateLimitRule) error {
 	switch rule.Audience {
@@ -143,14 +161,14 @@ func checkRateLimit(e *core.RequestEvent, rtId string, rule core.RateLimitRule)
 
 	rateLimiters := e.App.Store().GetOrSet(rateLimitersStoreKey, func() any {
 		return initRateLimitersStore(e.App)
-	}).(*store.Store[*rateLimiter])
+	}).(*store.Store[string, *rateLimiter])
 	if rateLimiters == nil {
 		e.App.Logger().Warn("Failed to retrieve app rate limiters store")
 		return nil
 	}
 
 	rt := rateLimiters.GetOrSet(rtId, func() *rateLimiter {
-		return newRateLimiter(rule.MaxRequests, rule.Duration, rule.Duration+1800)
+		return newRateLimiter(rule.MaxRequests, rule.Duration, 1800)
 	})
 	if rt == nil {
 		e.App.Logger().Warn("Failed to retrieve app rate limiter", "id", rtId)
@@ -164,14 +182,27 @@ func checkRateLimit(e *core.RequestEvent, rtId string, rule core.RateLimitRule)
 	}
 
 	if !rt.isAllowed(key) {
-		return e.TooManyRequestsError("", nil)
+		return e.TooManyRequestsError("", errors.New("triggered rate limit rule: "+rule.String()))
 	}
 
 	return nil
 }
 
 func skipRateLimit(e *core.RequestEvent) bool {
-	return !e.App.Settings().RateLimits.Enabled || e.HasSuperuserAuth()
+	return !e.App.Settings().RateLimits.Enabled ||
+		e.HasSuperuserAuth() ||
+		isIPInList(e.App.Settings().RateLimits.ExcludedIPs, e.RealIP())
+}
+
+var defaultAuthAudience = []string{core.RateLimitRuleAudienceAll, core.RateLimitRuleAudienceAuth}
+var defaultGuestAudience = []string{core.RateLimitRuleAudienceAll, core.RateLimitRuleAudienceGuest}
+
+func defaultRateLimitAudience(e *core.RequestEvent) []string {
+	if e.Auth != nil {
+		return defaultAuthAudience
+	}
+
+	return defaultGuestAudience
 }
 
 func defaultRateLimitLabels(e *core.RequestEvent) []string {
@@ -184,9 +215,9 @@ func destroyRateLimitersStore(app core.App) {
 	app.Store().Remove(rateLimitersStoreKey)
 }
 
-func initRateLimitersStore(app core.App) *store.Store[*rateLimiter] {
+func initRateLimitersStore(app core.App) *store.Store[string, *rateLimiter] {
 	app.Cron().Add(rateLimitersCronKey, "2 * * * *", func() { // offset a little since too many cleanup tasks execute at 00
-		limitersStore, ok := app.Store().Get(rateLimitersStoreKey).(*store.Store[*rateLimiter])
+		limitersStore, ok := app.Store().Get(rateLimitersStoreKey).(*store.Store[string, *rateLimiter])
 		if !ok {
 			return
 		}
@@ -211,7 +242,7 @@ func initRateLimitersStore(app core.App) *store.Store[*rateLimiter] {
 		},
 	})
 
-	return store.New[*rateLimiter](nil)
+	return store.New[string, *rateLimiter](nil)
 }
 
 func newRateLimiter(maxAllowed int, intervalInSec int64, minDeleteIntervalInSec int64) *rateLimiter {
@@ -219,12 +250,12 @@ func newRateLimiter(maxAllowed int, intervalInSec int64, minDeleteIntervalInSec
 		maxAllowed:        maxAllowed,
 		interval:          intervalInSec,
 		minDeleteInterval: minDeleteIntervalInSec,
-		clients:           map[string]*fixedWindow{},
+		clients:           map[string]*rateClient{},
 	}
 }
 
 type rateLimiter struct {
-	clients map[string]*fixedWindow
+	clients map[string]*rateClient
 
 	maxAllowed        int
 	interval          int64
@@ -235,7 +266,7 @@ type rateLimiter struct {
 }
 
 //nolint:unused
-func (rt *rateLimiter) getClient(key string) (*fixedWindow, bool) {
+func (rt *rateLimiter) getClient(key string) (*rateClient, bool) {
 	rt.RLock()
 	client, ok := rt.clients[key]
 	rt.RUnlock()
@@ -254,7 +285,7 @@ func (rt *rateLimiter) isAllowed(key string) bool {
 		// check again in case the client was added by another request
 		client, ok = rt.clients[key]
 		if !ok {
-			client = newFixedWindow(rt.maxAllowed, rt.interval)
+			client = newRateClient(rt.maxAllowed, rt.interval)
 			rt.clients[key] = client
 		}
 		rt.Unlock()
@@ -280,7 +311,7 @@ func (rt *rateLimiter) clean() {
 	//
 	// @todo remove after https://github.com/golang/go/issues/20135
 	if rt.totalDeleted >= 300 {
-		shrunk := make(map[string]*fixedWindow, len(rt.clients))
+		shrunk := make(map[string]*rateClient, len(rt.clients))
 		for k, v := range rt.clients {
 			shrunk[k] = v
 		}
@@ -289,52 +320,54 @@ func (rt *rateLimiter) clean() {
 	}
 }
 
-func newFixedWindow(maxAllowed int, intervalInSec int64) *fixedWindow {
-	return &fixedWindow{
+func newRateClient(maxAllowed int, intervalInSec int64) *rateClient {
+	return &rateClient{
 		maxAllowed: maxAllowed,
 		interval:   intervalInSec,
 	}
 }
 
-type fixedWindow struct {
+// @todo evaluate swiching to sliding window with approximation counter similar to Cloudflare.
+//
+// rateClient implements fixed window rate limit strategy.
+type rateClient struct {
 	// use plain Mutex instead of RWMutex since the operations are expected
 	// to be mostly writes (e.g. consume()) and it should perform better
 	sync.Mutex
 
-	maxAllowed  int   // the max allowed tokens per interval
-	available   int   // the total available tokens
-	interval    int64 // in seconds
-	lastConsume int64 // the time of the last consume
+	maxAllowed int   // the max allowed tokens per interval
+	available  int   // the total available tokens
+	start      int64 // the start time of the current window
+	interval   int64 // in seconds
 }
 
-// hasExpired checks whether it has been at least minElapsed seconds since the lastConsume time.
+// hasExpired checks whether it has been at least minElapsed seconds after the last active window.
 // (usually used to perform periodic cleanup of staled instances).
-func (l *fixedWindow) hasExpired(relativeNow int64, minElapsed int64) bool {
+func (l *rateClient) hasExpired(relativeNow int64, minElapsed int64) bool {
 	l.Lock()
 	defer l.Unlock()
 
-	return relativeNow-l.lastConsume > minElapsed
+	return relativeNow-(l.start+l.interval) > minElapsed
 }
 
-// consume decrease the current window allowance with 1 (if not exhausted already).
+// consume decreases the current allowance with 1 (if not exhausted already).
 //
 // It returns false if the allowance has been already exhausted and the user
 // has to wait until it resets back to its maxAllowed value.
-func (l *fixedWindow) consume() bool {
+func (l *rateClient) consume() bool {
 	l.Lock()
 	defer l.Unlock()
 
 	nowUnix := time.Now().Unix()
 
-	// reset consumed counter
-	if nowUnix-l.lastConsume >= l.interval {
+	// reset
+	if nowUnix-l.start >= l.interval {
 		l.available = l.maxAllowed
+		l.start = nowUnix
 	}
 
 	if l.available > 0 {
 		l.available--
-		l.lastConsume = nowUnix
-
 		return true
 	}
 

apis/middlewares_rate_limit_test.go

@@ -1,13 +1,16 @@
 package apis_test
 
 import (
+	"fmt"
 	"net/http/httptest"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/hook"
 )
 
 func TestDefaultRateLimitMiddleware(t *testing.T) {
@@ -74,7 +77,7 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 
 	scenarios := []struct {
 		url            string
-		wait           float64
+		wait           float64 // ms
 		authenticated  bool
 		expectedStatus int
 	}{
@@ -85,10 +88,12 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 		{"/norate", 0, false, 200},
 
 		{"/rate/a", 0, false, 200},
+		{"/rate/a", 900, false, 200}, // (fixed window check) wait enough to ensure that it can't fit more than 2 requests in 1s
+		{"/rate/a", 900, false, 200},
 		{"/rate/a", 0, false, 200},
 		{"/rate/a", 0, false, 429},
 		{"/rate/a", 0, false, 429},
-		{"/rate/a", 1.1, false, 200},
+		{"/rate/a", 1000, false, 200},
 		{"/rate/a", 0, false, 200},
 		{"/rate/a", 0, false, 429},
 
@@ -96,39 +101,37 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 		{"/rate/b", 0, false, 200},
 		{"/rate/b", 0, false, 200},
 		{"/rate/b", 0, false, 429},
-		{"/rate/b", 1.1, false, 200},
+		{"/rate/b", 1000, false, 200},
 		{"/rate/b", 0, false, 200},
 		{"/rate/b", 0, false, 200},
 		{"/rate/b", 0, false, 429},
 
-		// "auth" with guest (should be ignored)
-		{"/rate/auth", 0, false, 200},
-		{"/rate/auth", 0, false, 200},
+		// "auth" with guest (should fallback to the /rate/ rule)
 		{"/rate/auth", 0, false, 200},
 		{"/rate/auth", 0, false, 200},
+		{"/rate/auth", 0, false, 429},
+		{"/rate/auth", 0, false, 429},
 
-		// "auth" rule with regular user
+		// "auth" rule with regular user (should match the /rate/auth rule)
 		{"/rate/auth", 0, true, 200},
 		{"/rate/auth", 0, true, 429},
 		{"/rate/auth", 0, true, 429},
 
-		// "guest" with guest
+		// "guest" with guest (should match the /rate/guest rule)
 		{"/rate/guest", 0, false, 200},
 		{"/rate/guest", 0, false, 429},
 		{"/rate/guest", 0, false, 429},
 
-		// "guest" rule with regular user (should be ignored)
-		{"/rate/guest", 0, true, 200},
-		{"/rate/guest", 0, true, 200},
-		{"/rate/guest", 0, true, 200},
+		// "guest" rule with regular user (should fallback to the /rate/ rule)
+		{"/rate/guest", 1000, true, 200},
 		{"/rate/guest", 0, true, 200},
+		{"/rate/guest", 0, true, 429},
+		{"/rate/guest", 0, true, 429},
 	}
 
-	for _, s := range scenarios {
-		t.Run(s.url, func(t *testing.T) {
-			if s.wait > 0 {
-				time.Sleep(time.Duration(s.wait) * time.Second)
-			}
+	synctest.Test(t, func(t *testing.T) {
+		for i, s := range scenarios {
+			prefix := fmt.Sprintf("[%s:%d] ", s.url, i+1)
 
 			rec := httptest.NewRecorder()
 			req := httptest.NewRequest("GET", s.url, nil)
@@ -136,24 +139,188 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
 			if s.authenticated {
 				auth, err := app.FindAuthRecordByEmail("users", "test@example.com")
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				token, err := auth.NewAuthToken()
 				if err != nil {
-					t.Fatal(err)
+					t.Fatalf(prefix+"%v", err)
 				}
 
 				req.Header.Add("Authorization", token)
 			}
 
+			if s.wait > 0 {
+				synctest.Sleep(time.Duration(s.wait) * time.Millisecond)
+			}
+
 			mux.ServeHTTP(rec, req)
 
 			result := rec.Result()
 
 			if result.StatusCode != s.expectedStatus {
-				t.Fatalf("Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
+				t.Fatalf(prefix+"Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
 			}
-		})
+		}
+	})
+}
+
+func TestDefaultRateLimitMiddlewareSkipChecks(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	app.Settings().RateLimits.Enabled = true
+	app.Settings().RateLimits.Rules = []core.RateLimitRule{
+		{
+			Label:       "/rate",
+			MaxRequests: 1,
+			Duration:    5,
+		},
 	}
+
+	pbRouter, err := apis.NewRouter(app)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// just for the exclude tests - load the user IP from a query param
+	pbRouter.Bind(&hook.Handler[*core.RequestEvent]{
+		Priority: apis.DefaultRateLimitMiddlewarePriority - 1,
+		Func: func(e *core.RequestEvent) error {
+			testIp := e.Request.URL.Query().Get("testIP")
+			if testIp != "" {
+				e.Request.Header.Set("x-test-ip", testIp)
+			}
+
+			return e.Next()
+		},
+	})
+
+	pbRouter.GET("/rate", func(e *core.RequestEvent) error {
+		return e.String(200, "test")
+	})
+
+	mux, err := pbRouter.BuildMux()
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	checkStatusCodes := func(t *testing.T, got []int, expected []int) {
+		if len(expected) != len(got) {
+			t.Fatalf("Expected status codes %v, got %v", expected, got)
+		}
+
+		for i, item := range expected {
+			if got[i] != item {
+				t.Fatalf("Expected %d status code to be %d, got %d:\n%v", i, item, got[i], got)
+			}
+		}
+	}
+
+	t.Run("base check", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 429, 429})
+	})
+
+	t.Run("disabled rate limiter", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = false
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
+
+	t.Run("authenticated as superuser", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+
+		superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		token, err := superuser.NewAuthToken()
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Add("Authorization", token)
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
+
+	t.Run("excludedIPs (different)", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+		app.Settings().RateLimits.ExcludedIPs = []string{"10.0.0.0"}
+		app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Set("x-test-ip", "127.0.0.1")
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 429, 429})
+	})
+
+	t.Run("excludedIPs (match)", func(t *testing.T) {
+		app.Settings().RateLimits.Enabled = true
+		app.Settings().RateLimits.ExcludedIPs = []string{"127.0.0.1"}
+		app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+		statusCodes := []int{}
+		for range 3 {
+			rec := httptest.NewRecorder()
+			req := httptest.NewRequest("GET", "/rate", nil)
+			req.Header.Set("x-test-ip", "127.0.0.1")
+
+			mux.ServeHTTP(rec, req)
+
+			result := rec.Result()
+
+			statusCodes = append(statusCodes, result.StatusCode)
+		}
+
+		checkStatusCodes(t, statusCodes, []int{200, 200, 200})
+	})
 }

apis/middlewares_test.go

@@ -224,6 +224,22 @@ func TestRequireAuth(t *testing.T) {
 			ExpectedStatus:  200,
 			ExpectedContent: []string{"test123"},
 		},
+		{
+			Name:   "valid record auth token with Bearer case-insensitive prefix",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				// regular user
+				"Authorization": "BeArEr eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				e.Router.GET("/my/test", func(e *core.RequestEvent) error {
+					return e.String(200, "test123")
+				}).Bind(apis.RequireAuth())
+			},
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+		},
 	}
 
 	for _, scenario := range scenarios {
@@ -537,3 +553,96 @@ func TestRequireSameCollectionContextAuth(t *testing.T) {
 		scenario.Test(t)
 	}
 }
+
+func TestSuperuserIPsWhitelist(t *testing.T) {
+	t.Parallel()
+
+	setupWhitelist := func(superuserIPs ...string) func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+		return func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+			// allow loading a mock IP from the test scenario
+			app.Settings().TrustedProxy = core.TrustedProxyConfig{
+				Headers: []string{"x-test-ip"},
+			}
+
+			app.Settings().SuperuserIPs = superuserIPs
+
+			err := app.Save(app.Settings())
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			e.Router.GET("/my/test", func(e *core.RequestEvent) error {
+				return e.String(200, "test123")
+			})
+		}
+	}
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "guest with non-matching IP",
+			Method:          http.MethodGet,
+			URL:             "/my/test",
+			Headers:         map[string]string{"x-test-ip": "127.0.0.1"},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "regular user with non-matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with non-matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0"),
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with matching IP",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist("0.0.0.0", "127.0.0.1"),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser with no whitelisted IPs",
+			Method: http.MethodGet,
+			URL:    "/my/test",
+			Headers: map[string]string{
+				"x-test-ip":     "127.0.0.1",
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc:  setupWhitelist(),
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"test123"},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/realtime.go

@@ -2,15 +2,16 @@ package apis
 
 import (
 	"context"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
+	"fmt"
 	"log/slog"
 	"net/http"
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/picker"
@@ -27,6 +28,9 @@ const clientsChunkSize = 150
 // RealtimeClientAuthKey is the name of the realtime client store key that holds its auth state.
 const RealtimeClientAuthKey = "auth"
 
+// RealtimeClientIPKey is the name of the realtime client store key that holds the IP of the connected client.
+const RealtimeClientIPKey = "pbRealtimeClientIP"
+
 // bindRealtimeApi registers the realtime api endpoints.
 func bindRealtimeApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
 	sub := rg.Group("/realtime")
@@ -62,8 +66,12 @@ func realtimeConnect(e *core.RequestEvent) error {
 
 	connectEvent := new(core.RealtimeConnectRequestEvent)
 	connectEvent.RequestEvent = e
-	connectEvent.Client = subscriptions.NewDefaultClient()
 	connectEvent.IdleTimeout = 5 * time.Minute
+	connectEvent.MaxTimeout = 30 * time.Minute
+	connectEvent.Client = subscriptions.NewDefaultClient()
+
+	// could be used as an optional cross-reference check in other API endpoints
+	connectEvent.Client.Set(RealtimeClientIPKey, e.RealIP())
 
 	return e.App.OnRealtimeConnectRequest().Trigger(connectEvent, func(ce *core.RealtimeConnectRequestEvent) error {
 		// register new subscription client
@@ -72,7 +80,7 @@ func realtimeConnect(e *core.RequestEvent) error {
 			e.App.SubscriptionsBroker().Unregister(ce.Client.Id())
 		}()
 
-		ce.App.Logger().Debug("Realtime connection established.", slog.String("clientId", ce.Client.Id()))
+		ce.App.Logger().Debug("Realtime connection established", slog.String("clientId", ce.Client.Id()))
 
 		// signalize established connection (aka. fire "connect" message)
 		connectMsgEvent := new(core.RealtimeMessageEvent)
@@ -83,11 +91,10 @@ func realtimeConnect(e *core.RequestEvent) error {
 			Data: []byte(`{"clientId":"` + ce.Client.Id() + `"}`),
 		}
 		connectMsgErr := ce.App.OnRealtimeMessageSend().Trigger(connectMsgEvent, func(me *core.RealtimeMessageEvent) error {
-			me.Response.Write([]byte("id:" + me.Client.Id() + "\n"))
-			me.Response.Write([]byte("event:" + me.Message.Name + "\n"))
-			me.Response.Write([]byte("data:"))
-			me.Response.Write(me.Message.Data)
-			me.Response.Write([]byte("\n\n"))
+			err := me.Message.WriteSSE(me.Response, me.Client.Id())
+			if err != nil {
+				return err
+			}
 			return me.Flush()
 		})
 		if connectMsgErr != nil {
@@ -99,12 +106,19 @@ func realtimeConnect(e *core.RequestEvent) error {
 			return nil
 		}
 
+		// start a max lifetime timer to prevent accumulating too much
+		// connection resources and to allow the GC to run more regularly
+		maxTimer := time.NewTimer(ce.MaxTimeout)
+		defer maxTimer.Stop()
+
 		// start an idle timer to keep track of inactive/forgotten connections
 		idleTimer := time.NewTimer(ce.IdleTimeout)
 		defer idleTimer.Stop()
 
 		for {
 			select {
+			case <-maxTimer.C:
+				cancelRequest()
 			case <-idleTimer.C:
 				cancelRequest()
 			case msg, ok := <-ce.Client.Channel():
@@ -122,11 +136,10 @@ func realtimeConnect(e *core.RequestEvent) error {
 				msgEvent.Client = ce.Client
 				msgEvent.Message = &msg
 				msgErr := ce.App.OnRealtimeMessageSend().Trigger(msgEvent, func(me *core.RealtimeMessageEvent) error {
-					me.Response.Write([]byte("id:" + me.Client.Id() + "\n"))
-					me.Response.Write([]byte("event:" + me.Message.Name + "\n"))
-					me.Response.Write([]byte("data:"))
-					me.Response.Write(me.Message.Data)
-					me.Response.Write([]byte("\n\n"))
+					err := me.Message.WriteSSE(me.Response, me.Client.Id())
+					if err != nil {
+						return err
+					}
 					return me.Flush()
 				})
 				if msgErr != nil {
@@ -187,6 +200,21 @@ func realtimeSetSubscriptions(e *core.RequestEvent) error {
 		return e.NotFoundError("Missing or invalid client id.", err)
 	}
 
+	// for just in case to prevent someone changing a guest subscription
+	//
+	// note1: this is an extra precaution against clientId bruteforce attempts
+	// for installations allowing longer realtime connections duration
+	//
+	// note2: custom registered clients (aka. those without IP in the store)
+	// are excluded from the check for backward compatibility
+	clientIP, _ := client.Get(RealtimeClientIPKey).(string)
+	if clientIP != "" && clientIP != e.RealIP() {
+		return e.BadRequestError(
+			"Invalid realtime client.",
+			errors.New("the subscription request IP doesn't match with the realtime client IP"),
+		)
+	}
+
 	// for now allow only guest->auth upgrades and any other auth change is forbidden
 	clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 	if clientAuth != nil && !isSameAuth(clientAuth, e.Auth) {
@@ -209,64 +237,144 @@ func realtimeSetSubscriptions(e *core.RequestEvent) error {
 		e.Client.Subscribe(e.Subscriptions...)
 
 		e.App.Logger().Debug(
-			"Realtime subscriptions updated.",
+			"Realtime subscriptions updated",
 			slog.String("clientId", e.Client.Id()),
 			slog.Any("subscriptions", e.Subscriptions),
 		)
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 
-// updateClientsAuth updates the existing clients auth record with the new one (matched by ID).
-func realtimeUpdateClientsAuth(app core.App, newAuthRecord *core.Record) error {
+// realtimeUpdateClientsAuth updates the auth state of all clients related to the provided authRecord.
+//
+// Realtime connections has short lifetime by design, but to minimize abuse
+// if the new record has a different tokenKey (e.g. in case of password reset)
+// the auth state of the related realtime connections is also cleared
+// (aka. they remain active but unauthenticated, allowing to reauthenicate with the next subscription).
+func realtimeUpdateClientsAuth(app core.App, authRecord *core.Record) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
-					clientAuth.Id == newAuthRecord.Id &&
-					clientAuth.Collection().Name == newAuthRecord.Collection().Name {
-					client.Set(RealtimeClientAuthKey, newAuthRecord)
+					clientAuth.Id == authRecord.Id &&
+					clientAuth.Collection().Name == authRecord.Collection().Name {
+					if clientAuth.TokenKey() != authRecord.TokenKey() {
+						client.Unset(RealtimeClientAuthKey)
+					} else {
+						client.Set(RealtimeClientAuthKey, authRecord)
+					}
 				}
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
 }
 
-// unregisterClientsByAuthModel unregister all clients that has the provided auth model.
-func realtimeUnregisterClientsByAuth(app core.App, authModel core.Model) error {
+// realtimeUnsetClientsAuthByRecordModelOrProxy unsets the auth state of all clients that have the provided auth model.
+func realtimeUnsetClientsAuthByRecordModelOrProxy(app core.App, authModel core.Model) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
 				if clientAuth != nil &&
 					clientAuth.Id == authModel.PK() &&
 					clientAuth.Collection().Name == authModel.TableName() {
-					app.SubscriptionsBroker().Unregister(client.Id())
+					client.Unset(RealtimeClientAuthKey)
 				}
 			}
 
 			return nil
-		})
+		}))
+	}
+
+	return group.Wait()
+}
+
+// realtimeUnsetClientsAuthByCollection unsets the auth state of all authenticated clients related to the collection.
+func realtimeUnsetClientsAuthByCollection(app core.App, collection *core.Collection) error {
+	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
+
+	group := new(errgroup.Group)
+
+	for _, chunk := range chunks {
+		group.Go(routine.SafeWrap(func() error {
+			for _, client := range chunk {
+				clientAuth, _ := client.Get(RealtimeClientAuthKey).(*core.Record)
+				if clientAuth != nil && clientAuth.Collection().Name == collection.Name {
+					client.Unset(RealtimeClientAuthKey)
+				}
+			}
+
+			return nil
+		}))
 	}
 
 	return group.Wait()
 }
 
 func bindRealtimeEvents(app core.App) {
+	// reset the clients auth on collection secret change
+	// (@todo with the future tracking of old collections data consider replacing with *AfterUpdateSuccess to account for transaction rollback)
+	app.OnCollectionUpdate().Bind(&hook.Handler[*core.CollectionEvent]{
+		Func: func(e *core.CollectionEvent) error {
+			if !e.Collection.IsAuth() {
+				return e.Next()
+			}
+
+			cached, _ := e.App.FindCachedCollectionByNameOrId(e.Collection.Id)
+
+			if err := e.Next(); err != nil {
+				return err
+			}
+
+			if cached != nil && cached.AuthToken.Secret != e.Collection.AuthToken.Secret {
+				if err := realtimeUnsetClientsAuthByCollection(e.App, e.Collection); err != nil {
+					app.Logger().Warn(
+						"Failed to remove client(s) associated to the changed auth collection",
+						slog.String("collectionName", e.Collection.Name),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+
+			return nil
+		},
+		Priority: -99,
+	})
+
+	// unset the clients auth on auth collection delete
+	app.OnCollectionAfterDeleteSuccess().Bind(&hook.Handler[*core.CollectionEvent]{
+		Func: func(e *core.CollectionEvent) error {
+			if e.Collection.IsAuth() {
+				if err := realtimeUnsetClientsAuthByCollection(e.App, e.Collection); err != nil {
+					app.Logger().Warn(
+						"Failed to remove client(s) associated to the deleted auth collection",
+						slog.String("collectionName", e.Collection.Name),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+
+			return e.Next()
+		},
+		Priority: -99,
+	})
+
 	// update the clients that has auth record association
 	app.OnModelAfterUpdateSuccess().Bind(&hook.Handler[*core.ModelEvent]{
 		Func: func(e *core.ModelEvent) error {
@@ -293,7 +401,7 @@ func bindRealtimeEvents(app core.App) {
 		Func: func(e *core.ModelEvent) error {
 			collection := realtimeResolveRecordCollection(e.App, e.Model)
 			if collection != nil && collection.IsAuth() {
-				if err := realtimeUnregisterClientsByAuth(e.App, e.Model); err != nil {
+				if err := realtimeUnsetClientsAuthByRecordModelOrProxy(e.App, e.Model); err != nil {
 					app.Logger().Warn(
 						"Failed to remove client(s) associated to the deleted auth model",
 						slog.Any("id", e.Model.PK()),
@@ -353,7 +461,10 @@ func bindRealtimeEvents(app core.App) {
 		Func: func(e *core.ModelEvent) error {
 			record := realtimeResolveRecord(e.App, e.Model, "")
 			if record != nil {
-				err := realtimeBroadcastRecord(e.App, "delete", record, true)
+				// note: use the outside scoped app instance for the access checks so that the API rules
+				// are performed out of the delete transaction ensuring that they would still work even if
+				// a cascade-deleted record's API rule relies on an already deleted parent record
+				err := realtimeBroadcastRecord(e.App, "delete", record, true, app)
 				if err != nil {
 					app.Logger().Debug(
 						"Failed to dry cache record delete",
@@ -372,14 +483,17 @@ func bindRealtimeEvents(app core.App) {
 	// delete: broadcast
 	app.OnModelAfterDeleteSuccess().Bind(&hook.Handler[*core.ModelEvent]{
 		Func: func(e *core.ModelEvent) error {
-			record := realtimeResolveRecord(e.App, e.Model, "")
-			if record != nil {
-				err := realtimeBroadcastDryCachedRecord(e.App, "delete", record)
+			// note: only ensure that it is a collection record
+			// and don't use realtimeResolveRecord because in case of a
+			// custom model it'll fail to resolve since the record is already deleted
+			collection := realtimeResolveRecordCollection(e.App, e.Model)
+			if collection != nil {
+				err := realtimeBroadcastDryCacheKey(e.App, getDryCacheKey("delete", e.Model))
 				if err != nil {
 					app.Logger().Debug(
 						"Failed to broadcast record delete",
-						slog.String("id", record.Id),
-						slog.String("collectionName", record.Collection().Name),
+						slog.Any("id", e.Model.PK()),
+						slog.String("collectionName", collection.Name),
 						slog.String("error", err.Error()),
 					)
 				}
@@ -395,7 +509,7 @@ func bindRealtimeEvents(app core.App) {
 		Func: func(e *core.ModelErrorEvent) error {
 			record := realtimeResolveRecord(e.App, e.Model, "")
 			if record != nil {
-				err := realtimeUnsetDryCachedRecord(e.App, "delete", record)
+				err := realtimeUnsetDryCacheKey(e.App, getDryCacheKey("delete", record))
 				if err != nil {
 					app.Logger().Debug(
 						"Failed to cleanup after broadcast record delete failure",
@@ -415,7 +529,14 @@ func bindRealtimeEvents(app core.App) {
 // resolveRecord converts *if possible* the provided model interface to a Record.
 // This is usually helpful if the provided model is a custom Record model struct.
 func realtimeResolveRecord(app core.App, model core.Model, optCollectionType string) *core.Record {
-	record, _ := model.(*core.Record)
+	var record *core.Record
+	switch m := model.(type) {
+	case *core.Record:
+		record = m
+	case core.RecordProxy:
+		record = m.ProxyRecord()
+	}
+
 	if record != nil {
 		if optCollectionType == "" || record.Collection().Type == optCollectionType {
 			return record
@@ -444,14 +565,20 @@ func realtimeResolveRecord(app core.App, model core.Model, optCollectionType str
 // realtimeResolveRecordCollection extracts *if possible* the Collection model from the provided model interface.
 // This is usually helpful if the provided model is a custom Record model struct.
 func realtimeResolveRecordCollection(app core.App, model core.Model) (collection *core.Collection) {
-	if record, ok := model.(*core.Record); ok {
-		collection = record.Collection()
-	} else {
-		// check if it is custom Record model struct (ignore "private" tables)
-		collection, _ = app.FindCachedCollectionByNameOrId(model.TableName())
+	switch m := model.(type) {
+	case *core.Record:
+		return m.Collection()
+	case core.RecordProxy:
+		return m.ProxyRecord().Collection()
+	default:
+		// check if it is custom Record model struct
+		collection, err := app.FindCachedCollectionByNameOrId(model.TableName())
+		if err == nil {
+			return collection
+		}
 	}
 
-	return collection
+	return nil
 }
 
 // recordData represents the broadcasted record subscrition message data.
@@ -460,7 +587,11 @@ type recordData struct {
 	Action string `json:"action"`
 }
 
-func realtimeBroadcastRecord(app core.App, action string, record *core.Record, dryCache bool) error {
+// Note: the optAccessCheckApp is there in case you want the access check
+// to be performed against different db app context (e.g. out of a transaction).
+// If set, it is expected that optAccessCheckApp instance is used for read-only operations to avoid deadlocks.
+// If not set, it fallbacks to app.
+func realtimeBroadcastRecord(app core.App, action string, record *core.Record, dryCache bool, optAccessCheckApp ...core.App) error {
 	collection := record.Collection()
 	if collection == nil {
 		return errors.New("[broadcastRecord] Record collection not set")
@@ -482,12 +613,17 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 		(collection.Id + "?"):   collection.ListRule,
 	}
 
-	dryCacheKey := action + "/" + record.Id
+	dryCacheKey := getDryCacheKey(action, record)
 
 	group := new(errgroup.Group)
 
+	accessCheckApp := app
+	if len(optAccessCheckApp) > 0 {
+		accessCheckApp = optAccessCheckApp[0]
+	}
+
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			var clientAuth *core.Record
 
 			for _, client := range chunk {
@@ -502,10 +638,6 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 					clientAuth, _ = client.Get(RealtimeClientAuthKey).(*core.Record)
 
 					for sub, options := range subs {
-						// create a clean record copy without expand and unknown fields
-						// because we don't know yet which exact fields the client subscription has permissions to access
-						cleanRecord := record.Fresh()
-
 						// mock request data
 						requestInfo := &core.RequestInfo{
 							Context: core.RequestInfoContextRealtime,
@@ -515,10 +647,28 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 							Auth:    clientAuth,
 						}
 
-						if !realtimeCanAccessRecord(app, cleanRecord, requestInfo, rule) {
+						if !realtimeCanAccessRecord(accessCheckApp, record, requestInfo, rule) {
 							continue
 						}
 
+						// create a clean record copy without expand and unknown fields because we don't know yet
+						// which exact fields the client subscription requested or has permissions to access
+						cleanRecord := record.Fresh()
+
+						// -------------------------------------------
+						// @todo consider with the refactoring whether
+						// the default enriching used by the regular APIs
+						// can be reused here too to avoid eventual future
+						// discrepencies in the record event data
+						//
+						// https://github.com/pocketbase/pocketbase/issues/7721
+						// -------------------------------------------
+
+						// enable hidden fields for superuser subscribers
+						if requestInfo.HasSuperuserAuth() {
+							cleanRecord.Unhide(collection.Fields.FieldNames()...)
+						}
+
 						// trigger the enrich hooks
 						enrichErr := triggerRecordEnrichHooks(app, requestInfo, []*core.Record{cleanRecord}, func() error {
 							// apply expand
@@ -541,7 +691,7 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 							// for auth owner, superuser or manager
 							if collection.IsAuth() {
 								if isSameAuth(clientAuth, cleanRecord) ||
-									realtimeCanAccessRecord(app, cleanRecord, requestInfo, collection.ManageRule) {
+									realtimeCanAccessRecord(accessCheckApp, cleanRecord, requestInfo, collection.ManageRule) {
 									cleanRecord.IgnoreEmailVisibility(true)
 								}
 							}
@@ -616,25 +766,23 @@ func realtimeBroadcastRecord(app core.App, action string, record *core.Record, d
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
 }
 
-// realtimeBroadcastDryCachedRecord broadcasts all cached record related messages.
-func realtimeBroadcastDryCachedRecord(app core.App, action string, record *core.Record) error {
+// realtimeBroadcastDryCacheKey broadcasts the dry cached key related messages.
+func realtimeBroadcastDryCacheKey(app core.App, key string) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 	if len(chunks) == 0 {
 		return nil // no subscribers
 	}
 
-	key := action + "/" + record.Id
-
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				messages, ok := client.Get(key).([]subscriptions.Message)
 				if !ok {
@@ -653,25 +801,23 @@ func realtimeBroadcastDryCachedRecord(app core.App, action string, record *core.
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
 }
 
-// realtimeUnsetDryCachedRecord removes the dry cached record related messages.
-func realtimeUnsetDryCachedRecord(app core.App, action string, record *core.Record) error {
+// realtimeUnsetDryCacheKey removes the dry cached key related messages.
+func realtimeUnsetDryCacheKey(app core.App, key string) error {
 	chunks := app.SubscriptionsBroker().ChunkedClients(clientsChunkSize)
 	if len(chunks) == 0 {
 		return nil // no subscribers
 	}
 
-	key := action + "/" + record.Id
-
 	group := new(errgroup.Group)
 
 	for _, chunk := range chunks {
-		group.Go(func() error {
+		group.Go(routine.SafeWrap(func() error {
 			for _, client := range chunk {
 				if client.Get(key) != nil {
 					client.Unset(key)
@@ -679,12 +825,21 @@ func realtimeUnsetDryCachedRecord(app core.App, action string, record *core.Reco
 			}
 
 			return nil
-		})
+		}))
 	}
 
 	return group.Wait()
 }
 
+func getDryCacheKey(action string, model core.Model) string {
+	pkStr, ok := model.PK().(string)
+	if !ok {
+		pkStr = fmt.Sprintf("%v", model.PK())
+	}
+
+	return action + "/" + model.TableName() + "/" + pkStr
+}
+
 func isSameAuth(authA, authB *core.Record) bool {
 	if authA == nil {
 		return authB == nil
@@ -722,9 +877,9 @@ func realtimeCanAccessRecord(
 		return false
 	}
 
-	var exists bool
+	var exists int
 
-	q := app.DB().Select("(1)").
+	q := app.ConcurrentDB().Select("(1)").
 		From(record.Collection().Name).
 		AndWhere(dbx.HashExp{record.Collection().Name + ".id": record.Id})
 
@@ -735,9 +890,13 @@ func realtimeCanAccessRecord(
 	}
 
 	q.AndWhere(expr)
-	resolver.UpdateQuery(q)
+
+	err = resolver.UpdateQuery(q)
+	if err != nil {
+		return false
+	}
 
 	err = q.Limit(1).Row(&exists)
 
-	return err == nil && exists
+	return err == nil && exists > 0
 }

apis/realtime_test.go

@@ -2,10 +2,13 @@ package apis_test
 
 import (
 	"context"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"net/http"
+	"slices"
 	"strings"
+	"sync"
 	"testing"
 	"time"
 
@@ -14,6 +17,7 @@ import (
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
+	"github.com/pocketbase/pocketbase/tools/types"
 )
 
 func TestRealtimeConnect(t *testing.T) {
@@ -22,6 +26,7 @@ func TestRealtimeConnect(t *testing.T) {
 			Method:         http.MethodGet,
 			URL:            "/api/realtime",
 			Timeout:        100 * time.Millisecond,
+			Headers:        map[string]string{"x-test-ip": "127.0.0.2"},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`id:`,
@@ -33,6 +38,17 @@ func TestRealtimeConnect(t *testing.T) {
 				"OnRealtimeConnectRequest": 1,
 				"OnRealtimeMessageSend":    1,
 			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				app.OnRealtimeConnectRequest().BindFunc(func(e *core.RealtimeConnectRequestEvent) error {
+					if ip, _ := e.Client.Get(apis.RealtimeClientIPKey).(string); ip != "127.0.0.2" {
+						t.Fatalf("Expected IP %q, got %q", "127.0.0.2", ip)
+					}
+
+					return e.Next()
+				})
+			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				if len(app.SubscriptionsBroker().Clients()) != 0 {
 					t.Errorf("Expected the subscribers to be removed after connection close, found %d", len(app.SubscriptionsBroker().Clients()))
@@ -98,7 +114,8 @@ func TestRealtimeSubscribe(t *testing.T) {
 
 	resetClient := func() {
 		client.Unsubscribe()
-		client.Set(apis.RealtimeClientAuthKey, nil)
+		client.Unset(apis.RealtimeClientAuthKey)
+		client.Unset(apis.RealtimeClientIPKey)
 	}
 
 	validSubscriptionsLimit := make([]string, 1000)
@@ -204,6 +221,26 @@ func TestRealtimeSubscribe(t *testing.T) {
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
+		{
+			Name:            "existing client with different IP",
+			Method:          http.MethodPost,
+			URL:             "/api/realtime",
+			Body:            strings.NewReader(`{"clientId":"` + client.Id() + `","subscriptions":["test"]}`),
+			Headers:         map[string]string{"x-test-ip": "127.0.0.2"},
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				client.Set(apis.RealtimeClientIPKey, "127.0.0.1")
+
+				app.SubscriptionsBroker().Register(client)
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				resetClient()
+			},
+		},
 		{
 			Name:   "existing client with valid topic length",
 			Method: http.MethodPost,
@@ -425,28 +462,59 @@ func TestRealtimeAuthRecordDeleteEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
-
-	authRecord, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	_, err := apis.NewRouter(testApp)
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	client := subscriptions.NewDefaultClient()
-	client.Set(apis.RealtimeClientAuthKey, authRecord)
-	testApp.SubscriptionsBroker().Register(client)
+	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test2@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client1)
+
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client2)
+
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord2)
+	testApp.SubscriptionsBroker().Register(client3)
 
 	// mock delete event
 	e := new(core.ModelEvent)
 	e.App = testApp
 	e.Type = core.ModelEventTypeDelete
 	e.Context = context.Background()
-	e.Model = authRecord
+	e.Model = authRecord1
 
-	testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	err = testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	if err != nil {
+		t.Fatal(err)
+	}
 
-	if total := len(testApp.SubscriptionsBroker().Clients()); total != 0 {
-		t.Fatalf("Expected no subscription clients, found %d", total)
+	if total := len(testApp.SubscriptionsBroker().Clients()); total != 3 {
+		t.Fatalf("Expected %d subscription clients, found %d", 3, total)
+	}
+
+	if auth := client1.Get(apis.RealtimeClientAuthKey); auth != nil {
+		t.Fatalf("[client1] Expected the auth state to be unset, found %#v", auth)
+	}
+
+	if auth := client2.Get(apis.RealtimeClientAuthKey); auth != nil {
+		t.Fatalf("[client2] Expected the auth state to be unset, found %#v", auth)
+	}
+
+	if auth := client3.Get(apis.RealtimeClientAuthKey); auth == nil || auth.(*core.Record).Id != authRecord2.Id {
+		t.Fatalf("[client3] Expected the auth state to be left unchanged, found %#v", auth)
 	}
 }
 
@@ -455,7 +523,10 @@ func TestRealtimeAuthRecordUpdateEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -466,25 +537,331 @@ func TestRealtimeAuthRecordUpdateEvent(t *testing.T) {
 	client.Set(apis.RealtimeClientAuthKey, authRecord1)
 	testApp.SubscriptionsBroker().Register(client)
 
-	// refetch the authRecord and change its email
+	// refetch the authRecord and change its name
 	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
 		t.Fatal(err)
 	}
-	authRecord2.SetEmail("new@example.com")
 
-	// mock update event
-	e := new(core.ModelEvent)
-	e.App = testApp
-	e.Type = core.ModelEventTypeUpdate
-	e.Context = context.Background()
-	e.Model = authRecord2
+	newName := "test_new_name"
+	authRecord2.Set("name", newName)
 
-	testApp.OnModelAfterUpdateSuccess().Trigger(e)
+	err = testApp.Save(authRecord2)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	clientAuthRecord, _ := client.Get(apis.RealtimeClientAuthKey).(*core.Record)
-	if clientAuthRecord.Email() != authRecord2.Email() {
-		t.Fatalf("Expected authRecord with email %q, got %q", authRecord2.Email(), clientAuthRecord.Email())
+	if clientAuthRecord.Get("name") != newName {
+		t.Fatalf("Expected authRecord with email %q, got %q", newName, clientAuthRecord.Email())
+	}
+}
+
+func TestRealtimeRecordHiddenFields(t *testing.T) {
+	t.Parallel()
+
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// create temp collection with hidden fields
+	testCollection := core.NewBaseCollection("test_realtime")
+	testCollection.ListRule = types.Pointer("@request.auth.id != ''")
+	testCollection.Fields.Add(
+		&core.TextField{Name: "public"},
+		&core.TextField{Name: "hidden", Hidden: true},
+	)
+	if err := testApp.Save(testCollection); err != nil {
+		t.Fatal(err)
+	}
+
+	testSubscription := testCollection.Name + "/*"
+
+	// register guest subscriber
+	guestClient := subscriptions.NewDefaultClient()
+	guestClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(guestClient)
+
+	// register regular user subscriber
+	regular, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	regularClient := subscriptions.NewDefaultClient()
+	regularClient.Set(apis.RealtimeClientAuthKey, regular)
+	regularClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(regularClient)
+
+	// register superuser subscriber
+	superuser, err := testApp.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	superuserClient := subscriptions.NewDefaultClient()
+	superuserClient.Set(apis.RealtimeClientAuthKey, superuser)
+	superuserClient.Subscribe(testSubscription)
+	testApp.SubscriptionsBroker().Register(superuserClient)
+
+	enrichCalls := map[string]int{}
+	testApp.OnRecordEnrich(testCollection.Name).BindFunc(func(e *core.RecordEnrichEvent) error {
+		var id string
+		if e.RequestInfo.Auth != nil {
+			id = e.RequestInfo.Auth.Id
+		}
+		enrichCalls[id]++
+		return e.Next()
+	})
+
+	timeout := time.After(3 * time.Second)
+	done := make(chan struct{})
+
+	// collect first received messages
+	var regularMessageData, superuserMessageData string
+	go func() {
+		regularMessageData = string((<-regularClient.Channel()).Data)
+		superuserMessageData = string((<-superuserClient.Channel()).Data)
+		done <- struct{}{}
+	}()
+
+	// broadcast create message
+	testRecord := core.NewRecord(testCollection)
+	testRecord.Set("public", "test1")
+	testRecord.Set("hidden", "test2")
+	if err := testApp.Save(testRecord); err != nil {
+		t.Fatal(err)
+	}
+
+	// wait for the events
+	select {
+	case <-timeout:
+		t.Fatal("realtime test messages timeout")
+	case <-done:
+		// ready
+	}
+
+	if total := len(enrichCalls); total != 2 {
+		t.Fatalf("Expected %d enrich hook calls, got %d", 2, total)
+	}
+
+	if total := enrichCalls[regular.Id]; total != 1 {
+		t.Fatalf("Expected exactly 1 regular user enrich hook call, got %d", total)
+	}
+
+	if total := enrichCalls[superuser.Id]; total != 1 {
+		t.Fatalf("Expected exactly 1 superuser enrich hook call, got %d", total)
+	}
+
+	// validate messages content
+	scenarios := map[string]bool{
+		"regular message public field should exist":     strings.Contains(regularMessageData, `"public":`),
+		"regular message hidden field should NOT exist": !strings.Contains(regularMessageData, `"hidden":`),
+		"superuser message public field should exist":   strings.Contains(superuserMessageData, `"public":`),
+		"superuser message hidden field should exist":   strings.Contains(superuserMessageData, `"hidden":`),
+	}
+	for name, valid := range scenarios {
+		t.Run(name, func(t *testing.T) {
+			if !valid {
+				t.Fatal("Invalid realtime message expectation")
+			}
+		})
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnTokenKeyRefresh(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client := subscriptions.NewDefaultClient()
+	client.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client)
+
+	// refetch the authRecord and refresh its tokenKey
+	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	authRecord2.RefreshTokenKey()
+
+	err = testApp.Save(authRecord2)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientAuthRecord, _ := client.Get(apis.RealtimeClientAuthKey).(*core.Record)
+	if clientAuthRecord != nil {
+		t.Fatalf("Expected authRecord to be unset, got %q", clientAuthRecord.Email())
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnCollectionSecretChange(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	usersCollection, err := testApp.FindCollectionByNameOrId("users")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientsCollection, err := testApp.FindCollectionByNameOrId("clients")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+
+	authRecord2, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord2)
+
+	authRecord3, err := testApp.FindAuthRecordByEmail(clientsCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord3)
+
+	clientMocks := map[*core.Record]subscriptions.Client{
+		authRecord1: client1,
+		authRecord2: client2,
+		authRecord3: client3,
+	}
+	for _, client := range clientMocks {
+		testApp.SubscriptionsBroker().Register(client)
+	}
+
+	// change the secret of the users collection (should trigger unset)
+	usersCollection.AuthToken.Secret = strings.Repeat("a", 30)
+	err = testApp.Save(usersCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// change something else of the clients collection (shouldn't trigger unset)
+	clientsCollection.ListRule = nil
+	err = testApp.Save(clientsCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expectations := map[*core.Record]bool{
+		// record -> unset
+		authRecord1: true,
+		authRecord2: true,
+		authRecord3: false,
+	}
+	for record, expectedUnset := range expectations {
+		clientAuthRecord, _ := clientMocks[record].Get(apis.RealtimeClientAuthKey).(*core.Record)
+		unset := clientAuthRecord == nil
+		if unset != expectedUnset {
+			t.Fatalf("Expected unset state %v, got %v (%v)", expectedUnset, unset, clientAuthRecord)
+		}
+	}
+}
+
+func TestRealtimeAuthRecordUnsetOnCollectionDelete(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	// init realtime handlers
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	usersCollection, err := testApp.FindCollectionByNameOrId("users")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	clientsCollection, err := testApp.FindCollectionByNameOrId("clients")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord1, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+
+	authRecord2, err := testApp.FindAuthRecordByEmail(usersCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord2)
+
+	authRecord3, err := testApp.FindAuthRecordByEmail(clientsCollection, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord3)
+
+	clientMocks := map[*core.Record]subscriptions.Client{
+		authRecord1: client1,
+		authRecord2: client2,
+		authRecord3: client3,
+	}
+	for _, client := range clientMocks {
+		testApp.SubscriptionsBroker().Register(client)
+	}
+
+	// mock users collection delete event to avoid triggering constraints check
+	e := new(core.ModelEvent)
+	e.App = testApp
+	e.Type = core.ModelEventTypeDelete
+	e.Context = context.Background()
+	e.Model = usersCollection
+
+	err = testApp.OnModelAfterDeleteSuccess().Trigger(e)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expectations := map[*core.Record]bool{
+		// record -> unset
+		authRecord1: true,
+		authRecord2: true,
+		authRecord3: false,
+	}
+	for record, expectedUnset := range expectations {
+		clientAuthRecord, _ := clientMocks[record].Get(apis.RealtimeClientAuthKey).(*core.Record)
+		unset := clientAuthRecord == nil
+		if unset != expectedUnset {
+			t.Fatalf("Expected unset state %v, got %v (%v)", expectedUnset, unset, clientAuthRecord)
+		}
 	}
 }
 
@@ -522,19 +899,35 @@ func TestRealtimeCustomAuthModelDeleteEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
-
-	authRecord, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	_, err := apis.NewRouter(testApp)
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	client := subscriptions.NewDefaultClient()
-	client.Set(apis.RealtimeClientAuthKey, authRecord)
-	testApp.SubscriptionsBroker().Register(client)
+	authRecord1, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	authRecord2, err := testApp.FindAuthRecordByEmail("users", "test2@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	client1 := subscriptions.NewDefaultClient()
+	client1.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client1)
+
+	client2 := subscriptions.NewDefaultClient()
+	client2.Set(apis.RealtimeClientAuthKey, authRecord1)
+	testApp.SubscriptionsBroker().Register(client2)
+
+	client3 := subscriptions.NewDefaultClient()
+	client3.Set(apis.RealtimeClientAuthKey, authRecord2)
+	testApp.SubscriptionsBroker().Register(client3)
 
 	// refetch the authRecord as CustomUser
-	customUser, err := findCustomUserByEmail(testApp, "test@example.com")
+	customUser, err := findCustomUserByEmail(testApp, authRecord1.Email())
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -544,8 +937,20 @@ func TestRealtimeCustomAuthModelDeleteEvent(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	if total := len(testApp.SubscriptionsBroker().Clients()); total != 0 {
-		t.Fatalf("Expected no subscription clients, found %d", total)
+	if total := len(testApp.SubscriptionsBroker().Clients()); total != 3 {
+		t.Fatalf("Expected %d subscription clients, found %d", 3, total)
+	}
+
+	if auth := client1.Get(apis.RealtimeClientAuthKey); auth != nil {
+		t.Fatalf("[client1] Expected the auth state to be unset, found %#v", auth)
+	}
+
+	if auth := client2.Get(apis.RealtimeClientAuthKey); auth != nil {
+		t.Fatalf("[client2] Expected the auth state to be unset, found %#v", auth)
+	}
+
+	if auth := client3.Get(apis.RealtimeClientAuthKey); auth == nil || auth.(*core.Record).Id != authRecord2.Id {
+		t.Fatalf("[client3] Expected the auth state to be left unchanged, found %#v", auth)
 	}
 }
 
@@ -554,7 +959,10 @@ func TestRealtimeCustomAuthModelUpdateEvent(t *testing.T) {
 	defer testApp.Cleanup()
 
 	// init realtime handlers
-	apis.NewRouter(testApp)
+	_, err := apis.NewRouter(testApp)
+	if err != nil {
+		t.Fatal(err)
+	}
 
 	authRecord, err := testApp.FindAuthRecordByEmail("users", "test@example.com")
 	if err != nil {
@@ -582,3 +990,252 @@ func TestRealtimeCustomAuthModelUpdateEvent(t *testing.T) {
 		t.Fatalf("Expected authRecord with email %q, got %q", customUser.Email, clientAuthRecord.Email())
 	}
 }
+
+// -------------------------------------------------------------------
+
+var _ core.Model = (*CustomModelResolve)(nil)
+
+type CustomModelResolve struct {
+	core.BaseModel
+	tableName string
+
+	Created string `db:"created"`
+}
+
+func (m *CustomModelResolve) TableName() string {
+	return m.tableName
+}
+
+func TestRealtimeRecordResolve(t *testing.T) {
+	t.Parallel()
+
+	const testCollectionName = "realtime_test_collection"
+
+	testRecordId := core.GenerateDefaultRandomId()
+
+	client0 := subscriptions.NewDefaultClient()
+	client0.Subscribe(testCollectionName + "/*")
+	client0.Discard()
+	// ---
+	client1 := subscriptions.NewDefaultClient()
+	client1.Subscribe(testCollectionName + "/*")
+	// ---
+	client2 := subscriptions.NewDefaultClient()
+	client2.Subscribe(testCollectionName + "/" + testRecordId)
+	// ---
+	client3 := subscriptions.NewDefaultClient()
+	client3.Subscribe("demo1/*")
+
+	scenarios := []struct {
+		name     string
+		op       func(testApp core.App) error
+		expected map[string][]string // clientId -> [events]
+	}{
+		{
+			"core.Record",
+			func(testApp core.App) error {
+				c, err := testApp.FindCollectionByNameOrId(testCollectionName)
+				if err != nil {
+					return err
+				}
+
+				r := core.NewRecord(c)
+				r.Id = testRecordId
+
+				// create
+				err = testApp.Save(r)
+				if err != nil {
+					return err
+				}
+
+				// update
+				err = testApp.Save(r)
+				if err != nil {
+					return err
+				}
+
+				// delete
+				err = testApp.Delete(r)
+				if err != nil {
+					return err
+				}
+
+				return nil
+			},
+			map[string][]string{
+				client1.Id(): {"create", "update", "delete"},
+				client2.Id(): {"create", "update", "delete"},
+			},
+		},
+		{
+			"core.RecordProxy",
+			func(testApp core.App) error {
+				c, err := testApp.FindCollectionByNameOrId(testCollectionName)
+				if err != nil {
+					return err
+				}
+
+				r := core.NewRecord(c)
+
+				proxy := &struct {
+					core.BaseRecordProxy
+				}{}
+				proxy.SetProxyRecord(r)
+				proxy.Id = testRecordId
+
+				// create
+				err = testApp.Save(proxy)
+				if err != nil {
+					return err
+				}
+
+				// update
+				err = testApp.Save(proxy)
+				if err != nil {
+					return err
+				}
+
+				// delete
+				err = testApp.Delete(proxy)
+				if err != nil {
+					return err
+				}
+
+				return nil
+			},
+			map[string][]string{
+				client1.Id(): {"create", "update", "delete"},
+				client2.Id(): {"create", "update", "delete"},
+			},
+		},
+		{
+			"custom model struct",
+			func(testApp core.App) error {
+				m := &CustomModelResolve{tableName: testCollectionName}
+				m.Id = testRecordId
+
+				// create
+				err := testApp.Save(m)
+				if err != nil {
+					return err
+				}
+
+				// update
+				m.Created = "123"
+				err = testApp.Save(m)
+				if err != nil {
+					return err
+				}
+
+				// delete
+				err = testApp.Delete(m)
+				if err != nil {
+					return err
+				}
+
+				return nil
+			},
+			map[string][]string{
+				client1.Id(): {"create", "update", "delete"},
+				client2.Id(): {"create", "update", "delete"},
+			},
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			testApp, _ := tests.NewTestApp()
+			defer testApp.Cleanup()
+
+			// init realtime handlers
+			apis.NewRouter(testApp)
+
+			// create new test collection with public read access
+			testCollection := core.NewBaseCollection(testCollectionName)
+			testCollection.Fields.Add(&core.AutodateField{Name: "created", OnCreate: true, OnUpdate: true})
+			testCollection.ListRule = types.Pointer("")
+			testCollection.ViewRule = types.Pointer("")
+			err := testApp.Save(testCollection)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			testApp.SubscriptionsBroker().Register(client0)
+			testApp.SubscriptionsBroker().Register(client1)
+			testApp.SubscriptionsBroker().Register(client2)
+			testApp.SubscriptionsBroker().Register(client3)
+
+			var wg sync.WaitGroup
+
+			var notifications = map[string][]string{}
+
+			var mu sync.Mutex
+			notify := func(clientId string, eventData []byte) {
+				data := struct {
+					Action string `json:"action"`
+				}{}
+				_ = json.Unmarshal(eventData, &data)
+
+				mu.Lock()
+				defer mu.Unlock()
+
+				if notifications[clientId] == nil {
+					notifications[clientId] = []string{}
+				}
+				notifications[clientId] = append(notifications[clientId], data.Action)
+			}
+
+			wg.Add(1)
+			go func() {
+				defer wg.Done()
+
+				timeout := time.After(250 * time.Millisecond)
+
+				for {
+					select {
+					case e, ok := <-client0.Channel():
+						if ok {
+							notify(client0.Id(), e.Data)
+						}
+					case e, ok := <-client1.Channel():
+						if ok {
+							notify(client1.Id(), e.Data)
+						}
+					case e, ok := <-client2.Channel():
+						if ok {
+							notify(client2.Id(), e.Data)
+						}
+					case e, ok := <-client3.Channel():
+						if ok {
+							notify(client3.Id(), e.Data)
+						}
+					case <-timeout:
+						return
+					}
+				}
+			}()
+
+			err = s.op(testApp)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			wg.Wait()
+
+			if len(s.expected) != len(notifications) {
+				t.Fatalf("Expected %d notified clients, got %d:\n%v", len(s.expected), len(notifications), notifications)
+			}
+
+			for id, events := range s.expected {
+				if len(events) != len(notifications[id]) {
+					t.Fatalf("[%s] Expected %d events, got %d:\n%v\n%v", id, len(events), len(notifications[id]), s.expected, notifications)
+				}
+				for _, event := range events {
+					if !slices.Contains(notifications[id], event) {
+						t.Fatalf("[%s] Missing expected event %q in %v", id, event, notifications[id])
+					}
+				}
+			}
+		})
+	}
+}

apis/record_auth_email_change_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
@@ -38,15 +38,16 @@ func recordConfirmEmailChange(e *core.RequestEvent) error {
 	event.NewEmail = newEmail
 
 	return e.App.OnRecordConfirmEmailChangeRequest().Trigger(event, func(e *core.RecordConfirmEmailChangeRequestEvent) error {
-		authRecord.Set(core.FieldNameEmail, e.NewEmail)
-		authRecord.Set(core.FieldNameVerified, true)
-		authRecord.RefreshTokenKey() // invalidate old tokens
+		e.Record.SetEmail(e.NewEmail)
+		e.Record.SetVerified(true)
 
 		if err := e.App.Save(e.Record); err != nil {
 			return firstApiError(err, e.BadRequestError("Failed to confirm email change.", err))
 		}
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 
@@ -101,12 +102,6 @@ func (form *EmailChangeConfirmForm) parseToken() (*core.Record, string, error) {
 		return nil, "", validation.NewError("validation_invalid_token_payload", "Invalid token payload - newEmail must be set.")
 	}
 
-	// ensure that there aren't other users with the new email
-	_, err := form.app.FindAuthRecordByEmail(form.collection, newEmail)
-	if err == nil {
-		return nil, "", validation.NewError("validation_existing_token_email", "The new email address is already registered: "+newEmail)
-	}
-
 	// verify that the token is not expired and its signature is valid
 	authRecord, err := form.app.FindAuthRecordByToken(form.Token, core.TokenTypeEmailChange)
 	if err != nil {
@@ -117,5 +112,11 @@ func (form *EmailChangeConfirmForm) parseToken() (*core.Record, string, error) {
 		return nil, "", validation.NewError("validation_token_collection_mismatch", "The provided token is for different auth collection.")
 	}
 
+	// check if there are other users with the new email
+	_, err = form.app.FindAuthRecordByEmail(form.collection, newEmail)
+	if err == nil {
+		return nil, "", validation.NewError("validation_invalid_token_email", "The new email address is invalid.")
+	}
+
 	return authRecord, newEmail, nil
 }

apis/record_auth_email_change_confirm_test.go

@@ -1,7 +1,6 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"strings"
 	"testing"
@@ -112,12 +111,51 @@ func TestRecordConfirmEmailChange(t *testing.T) {
 				"OnRecordUpdateExecute":             1,
 				"OnRecordAfterUpdateSuccess":        1,
 				"OnRecordValidate":                  1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatalf("Expected the user to be unverified before the confirmation")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
-				_, err := app.FindAuthRecordByEmail("users", "change@example.com")
+				user, err := app.FindAuthRecordByEmail("users", "change@example.com")
 				if err != nil {
 					t.Fatalf("Expected to find user with email %q, got error: %v", "change@example.com", err)
 				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -136,7 +174,7 @@ func TestRecordConfirmEmailChange(t *testing.T) {
 			ExpectedEvents: map[string]int{"*": 0},
 		},
 		{
-			Name:   "OnRecordAfterConfirmEmailChangeRequest error response",
+			Name:   "OnRecordConfirmEmailChangeRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/confirm-email-change",
 			Body: strings.NewReader(`{
@@ -145,15 +183,22 @@ func TestRecordConfirmEmailChange(t *testing.T) {
 			}`),
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordConfirmEmailChangeRequest().BindFunc(func(e *core.RecordConfirmEmailChangeRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                                 0,
-				"OnRecordConfirmEmailChangeRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordConfirmEmailChangeRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// rate limit checks

apis/record_auth_email_change_request.go

@@ -3,8 +3,8 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 )
@@ -43,7 +43,9 @@ func recordRequestEmailChange(e *core.RequestEvent) error {
 			return firstApiError(err, e.BadRequestError("Failed to request email change.", err))
 		}
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 

apis/record_auth_email_change_request_test.go

@@ -118,6 +118,33 @@ func TestRecordRequestEmailChange(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "OnRecordRequestEmailChangeRequest tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/request-email-change",
+			Body:   strings.NewReader(`{"newEmail":"change@example.com"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordRequestEmailChangeRequest().BindFunc(func(e *core.RecordRequestEmailChangeRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordRequestEmailChangeRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_impersonate.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 
@@ -26,15 +26,15 @@ func recordAuthImpersonate(e *core.RequestEvent) error {
 
 	form := &impersonateForm{}
 	if err = e.BindBody(form); err != nil {
-		return firstApiError(err, e.BadRequestError("An error occurred while loading the submitted data.", err))
+		return e.BadRequestError("An error occurred while loading the submitted data.", err)
 	}
 	if err = form.validate(); err != nil {
-		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
+		return e.BadRequestError("An error occurred while validating the submitted data.", err)
 	}
 
 	token, err := record.NewStaticAuthToken(time.Duration(form.Duration) * time.Second)
 	if err != nil {
-		e.InternalServerError("Failed to generate static auth token", err)
+		return e.InternalServerError("Failed to generate static auth token", err)
 	}
 
 	return recordAuthResponse(e, record, token, "", nil)

apis/record_auth_methods.go

@@ -34,6 +34,7 @@ type oauth2Response struct {
 type providerInfo struct {
 	Name        string `json:"name"`
 	DisplayName string `json:"displayName"`
+	Logo        string `json:"logo"`
 	State       string `json:"state"`
 	AuthURL     string `json:"authURL"`
 
@@ -68,7 +69,14 @@ func (amr *authMethodsResponse) fillLegacyFields() {
 	amr.UsernamePassword = amr.Password.Enabled && slices.Contains(amr.Password.IdentityFields, "username")
 
 	if amr.OAuth2.Enabled {
-		amr.AuthProviders = amr.OAuth2.Providers
+		// clone without the logo
+		legacyProviders := make([]providerInfo, len(amr.OAuth2.Providers))
+		for i, p := range amr.OAuth2.Providers {
+			legacyProviders[i] = p
+			legacyProviders[i].Logo = ""
+		}
+
+		amr.AuthProviders = legacyProviders
 	}
 }
 
@@ -128,6 +136,7 @@ func recordAuthMethods(e *core.RequestEvent) error {
 		info := providerInfo{
 			Name:        config.Name,
 			DisplayName: provider.DisplayName(),
+			Logo:        provider.Logo(),
 			State:       security.RandomString(30),
 		}
 

apis/record_auth_methods_test.go

@@ -54,6 +54,8 @@ func TestRecordAuthMethodsList(t *testing.T) {
 				`"providers":[{`,
 				`"name":"google"`,
 				`"name":"gitlab"`,
+				`"logo":"<svg`,
+				`"logo":""`, // for the legacy fields
 				`"state":`,
 				`"displayName":`,
 				`"codeVerifier":`,

apis/record_auth_otp_request.go

@@ -6,8 +6,8 @@ import (
 	"fmt"
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"
@@ -49,12 +49,12 @@ func recordRequestOTP(e *core.RequestEvent) error {
 
 	return e.App.OnRecordRequestOTPRequest().Trigger(event, func(e *core.RecordCreateOTPRequestEvent) error {
 		if e.Record == nil {
-			// write a dummy 200 response as a very rudimentary user emails enumeration protection
+			// write a dummy 200 response as a very rudimentary emails enumeration "protection"
 			e.JSON(http.StatusOK, map[string]string{
 				"otpId": core.GenerateDefaultRandomId(),
 			})
 
-			return fmt.Errorf("failed to fetch %s record with email %s: %w", collection.Name, form.Email, err)
+			return fmt.Errorf("missing or invalid %s OTP auth record with email %s", collection.Name, form.Email)
 		}
 
 		var otp *core.OTP
@@ -108,8 +108,8 @@ func recordRequestOTP(e *core.RequestEvent) error {
 			})
 		}
 
-		return e.JSON(http.StatusOK, map[string]string{
-			"otpId": otp.Id,
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, map[string]string{"otpId": otp.Id})
 		})
 	})
 }

apis/record_auth_otp_request_test.go

@@ -247,6 +247,31 @@ func TestRecordRequestOTP(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "OnRecordRequestOTPRequest tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/request-otp",
+			Body:   strings.NewReader(`{"email":"test@example.com"}`),
+			Delay:  100 * time.Millisecond,
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordRequestOTPRequest().BindFunc(func(e *core.RecordCreateOTPRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordRequestOTPRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_password_reset_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/core/validators"
 	"github.com/pocketbase/pocketbase/tools/security"
@@ -47,14 +47,16 @@ func recordConfirmPasswordReset(e *core.RequestEvent) error {
 			}
 		}
 
-		err = form.app.Save(authRecord)
+		err = e.App.Save(authRecord)
 		if err != nil {
 			return firstApiError(err, e.BadRequestError("Failed to set new password.", err))
 		}
 
-		form.app.Store().Remove(getPasswordResetResendKey(authRecord))
+		e.App.Store().Remove(getPasswordResetResendKey(authRecord))
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 

apis/record_auth_password_reset_confirm_test.go

@@ -1,7 +1,6 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"strings"
 	"testing"
@@ -115,11 +114,18 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				"OnModelUpdate":                       1,
 				"OnModelUpdateExecute":                1,
 				"OnModelAfterUpdateSuccess":           1,
-				"OnModelValidate":                     1,
 				"OnRecordUpdate":                      1,
 				"OnRecordUpdateExecute":               1,
 				"OnRecordAfterUpdateSuccess":          1,
+				"OnModelValidate":                     1,
 				"OnRecordValidate":                    1,
+				// ---
+				"OnModelDelete":              2, // pre-existing OAuth2 links
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
@@ -152,6 +158,15 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				if !user.ValidatePassword("1234567!") {
 					t.Fatal("Password wasn't changed")
 				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -186,11 +201,20 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 					t.Fatal("Expected the user to be unverified")
 				}
 
+				oldTokenKey := user.TokenKey()
+
 				// manually change the email to check whether the verified state will be updated
 				user.SetEmail("test_update@example.com")
-				if err := app.Save(user); err != nil {
+				if err = app.Save(user); err != nil {
 					t.Fatalf("Failed to update user test email: %v", err)
 				}
+
+				// resave with the old token key since the email change above
+				// would change it and will make the password token invalid
+				user.SetTokenKey(oldTokenKey)
+				if err = app.Save(user); err != nil {
+					t.Fatalf("Failed to restore original user tokenKey: %v", err)
+				}
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				_, err := app.FindAuthRecordByToken(
@@ -213,6 +237,15 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 				if !user.ValidatePassword("1234567!") {
 					t.Fatal("Password wasn't changed")
 				}
+
+				// ensure that all pre-existing OAuth2 were NOT deleted
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) != 2 {
+					t.Fatalf("Expected 2 external auths, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -243,11 +276,20 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 					t.Fatalf("Failed to fetch confirm password user: %v", err)
 				}
 
+				oldTokenKey := user.TokenKey()
+
 				// ensure that the user is already verified
 				user.SetVerified(true)
 				if err := app.Save(user); err != nil {
 					t.Fatalf("Failed to update user verified state")
 				}
+
+				// resave with the old token key since the verified change above
+				// would refresh it and will make the password token invalid
+				user.SetTokenKey(oldTokenKey)
+				if err = app.Save(user); err != nil {
+					t.Fatalf("Failed to restore original user tokenKey: %v", err)
+				}
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				_, err := app.FindAuthRecordByToken(
@@ -273,7 +315,7 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnRecordAfterConfirmPasswordResetRequest error response",
+			Name:   "OnRecordConfirmPasswordResetRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/confirm-password-reset",
 			Body: strings.NewReader(`{
@@ -283,15 +325,22 @@ func TestRecordConfirmPasswordReset(t *testing.T) {
 			}`),
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordConfirmPasswordResetRequest().BindFunc(func(e *core.RecordConfirmPasswordResetRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                                   0,
-				"OnRecordConfirmPasswordResetRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordConfirmPasswordResetRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// rate limit checks

apis/record_auth_password_reset_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"
@@ -65,7 +65,9 @@ func recordRequestPasswordReset(e *core.RequestEvent) error {
 			})
 		})
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 

apis/record_auth_password_reset_request_test.go

@@ -101,6 +101,30 @@ func TestRecordRequestPasswordReset(t *testing.T) {
 				app.Store().Set(resendKey, struct{}{})
 			},
 		},
+		{
+			Name:   "OnRecordRequestPasswordResetRequest tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/request-password-reset",
+			Body:   strings.NewReader(`{"email":"test@example.com"}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordRequestPasswordResetRequest().BindFunc(func(e *core.RecordRequestPasswordResetRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordRequestPasswordResetRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_refresh.go

@@ -12,18 +12,24 @@ func recordAuthRefresh(e *core.RequestEvent) error {
 		return e.NotFoundError("Missing auth record context.", nil)
 	}
 
-	currentToken := getAuthTokenFromRequest(e)
-	claims, _ := security.ParseUnverifiedJWT(currentToken)
-	if v, ok := claims[core.TokenClaimRefreshable]; !ok || !cast.ToBool(v) {
-		return e.ForbiddenError("The current auth token is not refreshable.", nil)
-	}
-
 	event := new(core.RecordAuthRefreshRequestEvent)
 	event.RequestEvent = e
 	event.Collection = record.Collection()
 	event.Record = record
 
 	return e.App.OnRecordAuthRefreshRequest().Trigger(event, func(e *core.RecordAuthRefreshRequestEvent) error {
-		return RecordAuthResponse(e.RequestEvent, e.Record, "", nil)
+		token := getAuthTokenFromRequest(e.RequestEvent)
+
+		// skip token renewal if the token's payload doesn't explicitly allow it (e.g. impersonate tokens)
+		claims, _ := security.ParseUnverifiedJWT(token) //
+		if v, ok := claims[core.TokenClaimRefreshable]; ok && cast.ToBool(v) {
+			var tokenErr error
+			token, tokenErr = e.Record.NewAuthToken()
+			if tokenErr != nil {
+				return e.InternalServerError("Failed to refresh auth token.", tokenErr)
+			}
+		}
+
+		return recordAuthResponse(e.RequestEvent, e.Record, token, "", nil)
 	})
 }

apis/record_auth_refresh_test.go

@@ -1,7 +1,6 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"testing"
 
@@ -74,6 +73,8 @@ func TestRecordAuthRefresh(t *testing.T) {
 			},
 			NotExpectedContent: []string{
 				`"missing":`,
+				// should return a different token
+				"eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
 			},
 			ExpectedEvents: map[string]int{
 				"*":                          0,
@@ -89,9 +90,21 @@ func TestRecordAuthRefresh(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6ZmFsc2V9.4IsO6YMsR19crhwl_YWzvRH8pfq2Ri4Gv2dzGyneLak",
 			},
-			ExpectedStatus:  403,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents:  map[string]int{"*": 0},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				// should return the same token
+				`"token":"eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6ZmFsc2V9.4IsO6YMsR19crhwl_YWzvRH8pfq2Ri4Gv2dzGyneLak"`,
+				`"record":`,
+				`"id":"4q1xlclmfloku33"`,
+				`"emailVisibility":false`,
+				`"email":"test@example.com"`, // the owner can always view their email address
+			},
+			ExpectedEvents: map[string]int{
+				"*":                          0,
+				"OnRecordAuthRefreshRequest": 1,
+				"OnRecordAuthRequest":        1,
+				"OnRecordEnrich":             1,
+			},
 		},
 		{
 			Name:   "unverified auth record in onlyVerified collection",
@@ -130,23 +143,30 @@ func TestRecordAuthRefresh(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnRecordAfterAuthRefreshRequest error response",
+			Name:   "OnRecordAuthRefreshRequest tx body write check",
 			Method: http.MethodPost,
-			URL:    "/api/collections/users/auth-refresh?expand=rel,missing",
+			URL:    "/api/collections/users/auth-refresh",
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordAuthRefreshRequest().BindFunc(func(e *core.RecordAuthRefreshRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                          0,
-				"OnRecordAuthRefreshRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordAuthRefreshRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// rate limit checks

apis/record_auth_verification_confirm.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"
@@ -42,19 +42,26 @@ func recordConfirmVerification(e *core.RequestEvent) error {
 	event.Record = record
 
 	return e.App.OnRecordConfirmVerificationRequest().Trigger(event, func(e *core.RecordConfirmVerificationRequestEvent) error {
-		if wasVerified {
-			return e.NoContent(http.StatusNoContent)
-		}
+		if !wasVerified {
+			e.Record.SetVerified(true)
 
-		e.Record.SetVerified(true)
+			// similar to the OTP auth, we enforce an extra password reset
+			// guard as this way is less prone to pre-hijacking attacks
+			// in case the password auth is eventually enabled later
+			if !e.Record.Collection().PasswordAuth.Enabled {
+				e.Record.SetRandomPassword()
+			}
 
-		if err := e.App.Save(e.Record); err != nil {
-			return firstApiError(err, e.BadRequestError("An error occurred while saving the verified state.", err))
+			if err := e.App.Save(e.Record); err != nil {
+				return firstApiError(err, e.BadRequestError("An error occurred while saving the verified state.", err))
+			}
 		}
 
 		e.App.Store().Remove(getVerificationResendKey(e.Record))
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 

apis/record_auth_verification_confirm_test.go

@@ -1,7 +1,6 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"strings"
 	"testing"
@@ -106,6 +105,130 @@ func TestRecordConfirmVerification(t *testing.T) {
 				"OnRecordValidate":                   1,
 				"OnRecordUpdateExecute":              1,
 				"OnRecordAfterUpdateSuccess":         1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatal("Expected the user to be unverified before the confirmation")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
+			},
+		},
+		{
+			Name:   "valid token (disabled password auth)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/confirm-verification",
+			Body: strings.NewReader(`{
+				"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsImV4cCI6MjUyNDYwNDQ2MSwidHlwZSI6InZlcmlmaWNhdGlvbiIsImNvbGxlY3Rpb25JZCI6Il9wYl91c2Vyc19hdXRoXyIsImVtYWlsIjoidGVzdEBleGFtcGxlLmNvbSJ9.SetHpu2H-x-q4TIUz-xiQjwi7MNwLCLvSs4O0hUSp0E"
+			}`),
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{
+				"*":                                  0,
+				"OnRecordConfirmVerificationRequest": 1,
+				"OnModelUpdate":                      1,
+				"OnModelValidate":                    1,
+				"OnModelUpdateExecute":               1,
+				"OnModelAfterUpdateSuccess":          1,
+				"OnRecordUpdate":                     1,
+				"OnRecordValidate":                   1,
+				"OnRecordUpdateExecute":              1,
+				"OnRecordAfterUpdateSuccess":         1,
+				// unverified->verified external auths removal
+				"OnModelDelete":              2,
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				user.Collection().PasswordAuth.Enabled = false
+				if err = app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+
+				if user.Verified() {
+					t.Fatal("Expected the user to be unverified before the confirmation")
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatal("Expected password to be valid")
+				}
+
+				// ensure that there is at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auths")
+				}
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected the user to be verified after the confirmation")
+				}
+
+				if user.ValidatePassword("1234567890") {
+					t.Fatal("Expected the user password to be reset")
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -144,7 +267,7 @@ func TestRecordConfirmVerification(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnRecordAfterConfirmVerificationRequest error response",
+			Name:   "OnRecordConfirmVerificationRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/confirm-verification",
 			Body: strings.NewReader(`{
@@ -152,15 +275,22 @@ func TestRecordConfirmVerification(t *testing.T) {
 			}`),
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordConfirmVerificationRequest().BindFunc(func(e *core.RecordConfirmVerificationRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                                  0,
-				"OnRecordConfirmVerificationRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordConfirmVerificationRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// rate limit checks

apis/record_auth_verification_request.go

@@ -6,8 +6,8 @@ import (
 	"net/http"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/routine"
@@ -68,7 +68,9 @@ func recordRequestVerification(e *core.RequestEvent) error {
 			})
 		})
 
-		return e.NoContent(http.StatusNoContent)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.NoContent(http.StatusNoContent)
+		})
 	})
 }
 

apis/record_auth_verification_request_test.go

@@ -118,6 +118,30 @@ func TestRecordRequestVerification(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "OnRecordRequestVerificationRequest tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/request-verification",
+			Body:   strings.NewReader(`{"email":"test@example.com"}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordRequestVerificationRequest().BindFunc(func(e *core.RecordRequestVerificationRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordRequestVerificationRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_with_oauth2.go

@@ -1,21 +1,29 @@
 package apis
 
 import (
+	"bytes"
 	"context"
+	"database/sql"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
+	"io"
 	"log/slog"
 	"maps"
+	"net"
 	"net/http"
+	"path"
+	"strings"
+	"syscall"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
-	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/inflector"
 	"golang.org/x/oauth2"
 )
 
@@ -34,6 +42,8 @@ func recordAuthWithOAuth2(e *core.RequestEvent) error {
 		fallbackAuthRecord = e.Auth
 	}
 
+	e.Set(core.RequestEventKeyInfoContext, core.RequestInfoContextOAuth2)
+
 	form := new(recordOAuth2LoginForm)
 	form.collection = collection
 	if err = e.BindBody(form); err != nil {
@@ -87,6 +97,19 @@ func recordAuthWithOAuth2(e *core.RequestEvent) error {
 		return firstApiError(err, e.BadRequestError("Failed to fetch OAuth2 user.", err))
 	}
 
+	// Apple currently returns the user's name only as part of the first redirect data response
+	// so we try to assign the [apis.oauth2SubscriptionRedirect] forwarded name.
+	if form.Provider == auth.NameApple && authUser.Name == "" {
+		nameKey := oauth2RedirectAppleNameStoreKeyPrefix + form.Code
+		name, ok := e.App.Store().Get(nameKey).(string)
+		if ok {
+			e.App.Store().Remove(nameKey)
+			authUser.Name = name
+		} else {
+			e.App.Logger().Debug("Missing or already removed Apple user's name")
+		}
+	}
+
 	var authRecord *core.Record
 
 	// check for existing relation with the auth collection
@@ -95,6 +118,10 @@ func recordAuthWithOAuth2(e *core.RequestEvent) error {
 		"provider":      form.Provider,
 		"providerId":    authUser.Id,
 	})
+	if err != nil && !errors.Is(err, sql.ErrNoRows) {
+		return e.InternalServerError("Failed OAuth2 relation check.", err)
+	}
+
 	switch {
 	case err == nil && externalAuthRel != nil:
 		authRecord, err = e.App.FindRecordById(form.collection, externalAuthRel.RecordRef())
@@ -106,7 +133,10 @@ func recordAuthWithOAuth2(e *core.RequestEvent) error {
 		authRecord = fallbackAuthRecord
 	case authUser.Email != "":
 		// look for an existing auth record by the external auth record's email
-		authRecord, _ = e.App.FindAuthRecordByEmail(form.collection.Id, authUser.Email)
+		authRecord, err = e.App.FindAuthRecordByEmail(form.collection.Id, authUser.Email)
+		if err != nil && !errors.Is(err, sql.ErrNoRows) {
+			return e.InternalServerError("Failed OAuth2 auth record check.", err)
+		}
 	}
 
 	// ---------------------------------------------------------------
@@ -126,13 +156,17 @@ func recordAuthWithOAuth2(e *core.RequestEvent) error {
 			return firstApiError(err, e.BadRequestError("Failed to authenticate.", err))
 		}
 
-		meta := struct {
-			*auth.AuthUser
-			IsNew bool `json:"isNew"`
-		}{
-			AuthUser: e.OAuth2User,
-			IsNew:    e.IsNewRecord,
+		// @todo revert back to struct after removing the custom auth.AuthUser marshalization
+		meta := map[string]any{}
+		rawOAuth2User, err := json.Marshal(e.OAuth2User)
+		if err != nil {
+			return err
 		}
+		err = json.Unmarshal(rawOAuth2User, &meta)
+		if err != nil {
+			return err
+		}
+		meta["isNew"] = e.IsNewRecord
 
 		return RecordAuthResponse(e.RequestEvent, e.Record, core.MFAMethodOAuth2, meta)
 	})
@@ -167,9 +201,8 @@ type recordOAuth2LoginForm struct {
 
 func (form *recordOAuth2LoginForm) validate() error {
 	return validation.ValidateStruct(form,
-		validation.Field(&form.Provider, validation.Required, validation.By(form.checkProviderName)),
+		validation.Field(&form.Provider, validation.Required, validation.Length(0, 100), validation.By(form.checkProviderName)),
 		validation.Field(&form.Code, validation.Required),
-		validation.Field(&form.RedirectURL, validation.Required),
 	)
 }
 
@@ -178,26 +211,48 @@ func (form *recordOAuth2LoginForm) checkProviderName(value any) error {
 
 	_, ok := form.collection.OAuth2.GetProviderConfig(name)
 	if !ok {
-		return validation.NewError("validation_invalid_provider", fmt.Sprintf("Provider with name %q is missing or is not enabled.", name)).
+		return validation.NewError("validation_invalid_provider", "Provider with name {{.name}} is missing or is not enabled.").
 			SetParams(map[string]any{"name": name})
 	}
 
 	return nil
 }
 
+// @todo evaluate if it is still worth keeping as this exists only for backward-compatibility with pre v0.23 versions
 func oldCanAssignUsername(txApp core.App, collection *core.Collection, username string) bool {
-	// ensure that username is unique
-	checkUnique := dbutils.HasSingleColumnUniqueIndex(collection.OAuth2.MappedFields.Username, collection.Indexes)
-	if checkUnique {
-		if _, err := txApp.FindFirstRecordByData(collection, collection.OAuth2.MappedFields.Username, username); err == nil {
-			return false // already exist
-		}
+	field := collection.Fields.GetByName(collection.OAuth2.MappedFields.Username)
+	if field == nil {
+		return false
 	}
 
 	// ensure that the value matches the pattern of the username field (if text)
-	txtField, _ := collection.Fields.GetByName(collection.OAuth2.MappedFields.Username).(*core.TextField)
+	if txtField, ok := field.(*core.TextField); ok && txtField.ValidatePlainValue(username) != nil {
+		return false
+	}
 
-	return txtField != nil && txtField.ValidatePlainValue(username) == nil
+	// ensure that username is unique
+	index, hasUniqueue := dbutils.FindSingleColumnUniqueIndex(collection.Indexes, field.GetName())
+	if hasUniqueue {
+		// it is not required because collection fields are already sanitized
+		// but normalize as an extra precaution in case of a custom validator
+		colName := inflector.Columnify(field.GetName())
+
+		var expr dbx.Expression
+		if strings.EqualFold(index.Columns[0].Collate, "nocase") {
+			// case-insensitive search
+			expr = dbx.NewExp("[["+colName+"]] = {:username} COLLATE NOCASE", dbx.Params{"username": username})
+		} else {
+			expr = dbx.HashExp{colName: username}
+		}
+
+		var exists int
+		_ = txApp.RecordQuery(collection).Select("(1)").AndWhere(expr).Limit(1).Row(&exists)
+		if exists > 0 {
+			return false
+		}
+	}
+
+	return true
 }
 
 func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *core.ExternalAuth) error {
@@ -214,12 +269,10 @@ func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *cor
 				payload = map[string]any{}
 			}
 
-			payload[core.FieldNameEmail] = e.OAuth2User.Email
-
-			// set a random password if none is set
-			if v, _ := payload[core.FieldNamePassword].(string); v == "" {
-				payload[core.FieldNamePassword] = security.RandomString(30)
-				payload[core.FieldNamePassword+"Confirm"] = payload[core.FieldNamePassword]
+			// assign the OAuth2 user email only if the user hasn't submitted one
+			// (ignore empty/invalid values for consistency with the OAuth2->existing user update flow)
+			if v, _ := payload[core.FieldNameEmail].(string); v == "" {
+				payload[core.FieldNameEmail] = e.OAuth2User.Email
 			}
 
 			// map known fields (unless the field was explicitly submitted as part of CreateData)
@@ -237,7 +290,7 @@ func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *cor
 				payload[e.Collection.OAuth2.MappedFields.Username] = e.OAuth2User.Username
 			}
 			if _, ok := payload[e.Collection.OAuth2.MappedFields.AvatarURL]; !ok &&
-				// no existing OAuth2 mapping
+				// no explicit avatar payload value and existing OAuth2 mapping
 				e.Collection.OAuth2.MappedFields.AvatarURL != "" &&
 				// non-empty OAuth2 avatar url
 				e.OAuth2User.AvatarURL != "" {
@@ -245,9 +298,12 @@ func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *cor
 				if mappedField != nil && mappedField.Type() == core.FieldTypeFile {
 					// download the avatar if the mapped field is a file
 					avatarFile, err := func() (*filesystem.File, error) {
-						ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
+						ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
 						defer cancel()
-						return filesystem.NewFileFromURL(ctx, e.OAuth2User.AvatarURL)
+
+						// the extra checks are not required because the OAuth2 APIs are trusted vendor
+						// but are here to minimize the impact in case the provider is vulnerable
+						return safeFileFromURL(ctx, e.OAuth2User.AvatarURL)
 					}()
 					if err != nil {
 						txApp.Logger().Warn("Failed to retrieve OAuth2 avatar", slog.String("error", err.Error()))
@@ -281,26 +337,43 @@ func oauth2Submit(e *core.RecordAuthWithOAuth2RequestEvent, optExternalAuth *cor
 				e.Auth.Id == e.Record.Id &&
 				e.Auth.Collection().Id == e.Record.Collection().Id
 
-			// set random password for users with unverified email
-			// (this is in case a malicious actor has registered previously with the user email)
-			if !isLoggedAuthRecord && e.Record.Email() != "" && !e.Record.Verified() {
-				e.Record.SetPassword(security.RandomString(30))
+			// prevent pre-hijacking with password auth
+			//
+			// reset the unverified user password in case the record was precreated by a malicious actor
+			if !isLoggedAuthRecord && !e.Record.Verified() {
 				needUpdate = true
+				e.Record.SetRandomPassword()
+			}
+
+			// prevent pre-hijacking with different OAuth2 provider
+			//
+			// delete all other previous OAuth2 record links for the cases
+			// when the user was precreated by malicious OAuth2 auth with custom payload data
+			//
+			// while this would be also done automatically on unverified -> verified upgrade,
+			// doing it manually here ensures that a single unverified record could have
+			// max 1 OAuth2 link to prevent further abuse when mixed with other auth flows
+			if !e.Record.Verified() {
+				err := txApp.DeleteAllExternalAuthsByRecord(e.Record)
+				if err != nil {
+					return err
+				}
+				optExternalAuth = nil // clear to allow recreate below
 			}
 
 			// update the existing auth record empty email if the data.OAuth2User has one
 			// (this is in case previously the auth record was created
 			// with an OAuth2 provider that didn't return an email address)
 			if e.Record.Email() == "" && e.OAuth2User.Email != "" {
-				e.Record.SetEmail(e.OAuth2User.Email)
 				needUpdate = true
+				e.Record.SetEmail(e.OAuth2User.Email)
 			}
 
 			// update the existing auth record verified state
 			// (only if the auth record doesn't have an email or the auth record email match with the one in data.OAuth2User)
 			if !e.Record.Verified() && (e.Record.Email() == "" || e.Record.Email() == e.OAuth2User.Email) {
-				e.Record.SetVerified(true)
 				needUpdate = true
+				e.Record.SetVerified(true)
 			}
 
 			if needUpdate {
@@ -350,3 +423,88 @@ func sendOAuth2RecordCreateRequest(txApp core.App, e *core.RecordAuthWithOAuth2R
 
 	return createdRecord, nil
 }
+
+// -------------------------------------------------------------------
+
+// safeHTTPClient initializes a custom http.Client with extra host checks
+// to prevent internal network probing requests
+// (aka. disallow loopback, private, multicast, etc. requests).
+//
+// NB! The host checks are not perfect and there are probably edge cases that are not covered,
+// so if you plan using with untrusted user URL, consider performing additional whitelist checks.
+//
+// @todo Evaluate with the refactoring if worth exporting(+tests) and moving under the security package.
+func safeHTTPClient() *http.Client {
+	dialer := &net.Dialer{
+		// the same options as in http.DefaultTransport.DialContext
+		Timeout:   30 * time.Second,
+		KeepAlive: 30 * time.Second,
+
+		// check the address right after estrablishing the connection to prevent dns rebinding
+		Control: func(network, address string, c syscall.RawConn) error {
+			host, _, err := net.SplitHostPort(address)
+			if err != nil {
+				return err
+			}
+
+			ip := net.ParseIP(host)
+
+			if ip == nil ||
+				ip.IsLoopback() ||
+				ip.IsUnspecified() ||
+				ip.IsPrivate() ||
+				ip.IsLinkLocalUnicast() ||
+				ip.IsLinkLocalMulticast() ||
+				ip.IsMulticast() {
+				return fmt.Errorf("address %q is invalid or resolve to disallowed IP", address)
+			}
+
+			return nil
+		},
+	}
+
+	return &http.Client{
+		Timeout: 180 * time.Second, // can be still cancelled with the request context
+		Transport: &http.Transport{
+			DialContext: dialer.DialContext,
+			// the same options as in http.DefaultTransport
+			ForceAttemptHTTP2:     true,
+			MaxIdleConns:          100,
+			IdleConnTimeout:       90 * time.Second,
+			TLSHandshakeTimeout:   10 * time.Second,
+			ExpectContinueTimeout: 1 * time.Second,
+		},
+	}
+}
+
+// safeFileFromURL downloads the file from the specified url (using safeHTTPClient)
+// and creates a new filesystem.File value from its content (limited to DefaultMaxBodySize).
+//
+// @todo Evaluate with the refactoring if worth exporting/replacing filesystem.NewFileFromURL (or redefine as NewUnsafeFileFromURL).
+func safeFileFromURL(ctx context.Context, url string) (*filesystem.File, error) {
+	req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	client := safeHTTPClient()
+
+	res, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer res.Body.Close()
+
+	if res.StatusCode < 200 || res.StatusCode > 399 {
+		return nil, fmt.Errorf("failed to download url %s (%d)", url, res.StatusCode)
+	}
+
+	body := io.LimitReader(res.Body, DefaultMaxBodySize)
+
+	var buf bytes.Buffer
+	if _, err = io.Copy(&buf, body); err != nil {
+		return nil, err
+	}
+
+	return filesystem.NewFileFromBytes(buf.Bytes(), path.Base(url))
+}

apis/record_auth_with_oauth2_redirect.go

@@ -1,37 +1,40 @@
 package apis
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
+	"errors"
 	"net/http"
+	"strings"
+	"time"
 
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
+	"github.com/pocketbase/pocketbase/ui"
 )
 
 const (
-	oauth2SubscriptionTopic   string = "@oauth2"
-	oauth2RedirectFailurePath string = "../_/#/auth/oauth2-redirect-failure"
-	oauth2RedirectSuccessPath string = "../_/#/auth/oauth2-redirect-success"
+	oauth2SubscriptionTopic               string = "@oauth2"
+	oauth2RedirectFailurePath             string = "../_/#/auth/oauth2-redirect-failure"
+	oauth2RedirectSuccessPath             string = "../_/#/auth/oauth2-redirect-success"
+	oauth2RedirectAppleNameStoreKeyPrefix string = "@redirect_name_"
 )
 
 type oauth2RedirectData struct {
 	State string `form:"state" json:"state"`
 	Code  string `form:"code" json:"code"`
 	Error string `form:"error" json:"error,omitempty"`
+
+	// returned by Apple only
+	AppleUser string `form:"user" json:"-"`
 }
 
 func oauth2SubscriptionRedirect(e *core.RequestEvent) error {
-	redirectStatusCode := http.StatusTemporaryRedirect
-	if e.Request.Method != http.MethodGet {
-		redirectStatusCode = http.StatusSeeOther
-	}
-
 	data := oauth2RedirectData{}
 
 	if e.Request.Method == http.MethodPost {
 		if err := e.BindBody(&data); err != nil {
 			e.App.Logger().Debug("Failed to read OAuth2 redirect data", "error", err)
-			return e.Redirect(redirectStatusCode, oauth2RedirectFailurePath)
+			return failureRedirect(e)
 		}
 	} else {
 		query := e.Request.URL.Query()
@@ -42,20 +45,44 @@ func oauth2SubscriptionRedirect(e *core.RequestEvent) error {
 
 	if data.State == "" {
 		e.App.Logger().Debug("Missing OAuth2 state parameter")
-		return e.Redirect(redirectStatusCode, oauth2RedirectFailurePath)
+		return failureRedirect(e)
 	}
 
 	client, err := e.App.SubscriptionsBroker().ClientById(data.State)
 	if err != nil || client.IsDiscarded() || !client.HasSubscription(oauth2SubscriptionTopic) {
 		e.App.Logger().Debug("Missing or invalid OAuth2 subscription client", "error", err, "clientId", data.State)
-		return e.Redirect(redirectStatusCode, oauth2RedirectFailurePath)
+		return failureRedirect(e)
 	}
 	defer client.Unsubscribe(oauth2SubscriptionTopic)
 
+	// additional check to minimize the risk of XSRF attack vectors
+	//
+	// note: custom registered clients (aka. those without IP in the store)
+	// are excluded from the check for backward compatibility
+	clientIP, _ := client.Get(RealtimeClientIPKey).(string)
+	if clientIP != "" && clientIP != e.RealIP() {
+		e.App.Logger().Debug("The client IP that completed the authentication is different from the one that initialized the OAuth2 realtime connection")
+		return failureRedirect(e)
+	}
+
+	// temporary store the Apple user's name so that it can be later retrieved with the authWithOAuth2 call
+	// (see https://github.com/pocketbase/pocketbase/issues/7090)
+	if data.AppleUser != "" && data.Error == "" && data.Code != "" {
+		nameErr := parseAndStoreAppleRedirectName(
+			e.App,
+			oauth2RedirectAppleNameStoreKeyPrefix+data.Code,
+			data.AppleUser,
+		)
+		if nameErr != nil {
+			// non-critical error
+			e.App.Logger().Debug("Failed to parse and load Apple Redirect name data", "error", nameErr)
+		}
+	}
+
 	encodedData, err := json.Marshal(data)
 	if err != nil {
 		e.App.Logger().Debug("Failed to marshalize OAuth2 redirect data", "error", err)
-		return e.Redirect(redirectStatusCode, oauth2RedirectFailurePath)
+		return failureRedirect(e)
 	}
 
 	msg := subscriptions.Message{
@@ -67,8 +94,87 @@ func oauth2SubscriptionRedirect(e *core.RequestEvent) error {
 
 	if data.Error != "" || data.Code == "" {
 		e.App.Logger().Debug("Failed OAuth2 redirect due to an error or missing code parameter", "error", data.Error, "clientId", data.State)
-		return e.Redirect(redirectStatusCode, oauth2RedirectFailurePath)
+		return failureRedirect(e)
 	}
 
-	return e.Redirect(redirectStatusCode, oauth2RedirectSuccessPath)
+	return successRedirect(e)
+}
+
+func redirectStatusCode(e *core.RequestEvent) int {
+	if e.Request.Method != http.MethodGet {
+		return http.StatusSeeOther
+	}
+
+	return http.StatusTemporaryRedirect
+}
+
+func failureRedirect(e *core.RequestEvent) error {
+	// fallback if UI is not bundled
+	if ui.DistDirFS == nil {
+		return e.String(http.StatusOK, "Failed to authenticate. You can close this window and go back to the app to try again.")
+	}
+
+	return e.Redirect(redirectStatusCode(e), oauth2RedirectFailurePath)
+}
+
+func successRedirect(e *core.RequestEvent) error {
+	// fallback if UI is not bundled
+	if ui.DistDirFS == nil {
+		return e.HTML(http.StatusOK, "Auth completed. You can close this window and go back to the app.")
+	}
+
+	return e.Redirect(redirectStatusCode(e), oauth2RedirectSuccessPath)
+}
+
+// parseAndStoreAppleRedirectName extracts the first and last name
+// from serializedNameData and temporary store them in the app.Store.
+//
+// This is hacky workaround to forward safely and seamlessly the Apple
+// redirect user's name back to the OAuth2 auth handler.
+//
+// Note that currently Apple is the only provider that behaves like this and
+// for now it is unnecessary to check whether the redirect is coming from Apple or not.
+//
+// Ideally this shouldn't be needed and will be removed in the future
+// once Apple adds a dedicated userinfo endpoint.
+func parseAndStoreAppleRedirectName(app core.App, nameKey string, serializedNameData string) error {
+	if serializedNameData == "" {
+		return nil
+	}
+
+	// just in case to prevent storing large strings in memory
+	if len(nameKey) > 1000 {
+		return errors.New("nameKey is too large")
+	}
+
+	// https://developer.apple.com/documentation/signinwithapple/incorporating-sign-in-with-apple-into-other-platforms#Handle-the-response
+	extracted := struct {
+		Name struct {
+			FirstName string `json:"firstName"`
+			LastName  string `json:"lastName"`
+		} `json:"name"`
+	}{}
+	if err := json.Unmarshal([]byte(serializedNameData), &extracted); err != nil {
+		return err
+	}
+
+	fullName := extracted.Name.FirstName + " " + extracted.Name.LastName
+
+	// truncate just in case to prevent storing large strings in memory
+	if len(fullName) > 150 {
+		fullName = fullName[:150]
+	}
+
+	fullName = strings.TrimSpace(fullName)
+	if fullName == "" {
+		return nil
+	}
+
+	// store (and remove)
+	app.Store().Set(nameKey, fullName)
+	time.AfterFunc(1*time.Minute, func() {
+		app.Store().Remove(nameKey)
+	})
+
+	return nil
 }

apis/record_auth_with_oauth2_redirect_test.go

@@ -3,10 +3,12 @@ package apis_test
 import (
 	"context"
 	"net/http"
+	"net/url"
 	"strings"
 	"testing"
 	"time"
 
+	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
@@ -15,9 +17,9 @@ import (
 func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 	t.Parallel()
 
-	clientStubs := make([]map[string]subscriptions.Client, 0, 10)
+	clientStubs := make([]map[string]subscriptions.Client, 0, 11)
 
-	for i := 0; i < 10; i++ {
+	for i := 0; i < 11; i++ {
 		c1 := subscriptions.NewDefaultClient()
 
 		c2 := subscriptions.NewDefaultClient()
@@ -56,18 +58,22 @@ func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 		}
 	}
 
+	// note: don't exit because it is usually called as part of a separate goroutine
 	checkClientMessages := func(t testing.TB, clientId string, msg subscriptions.Message, expectedMessages map[string][]string) {
 		if len(expectedMessages[clientId]) == 0 {
-			t.Fatalf("Unexpected client %q message, got %s:\n%s", clientId, msg.Name, msg.Data)
+			t.Errorf("Unexpected client %q message, got %q:\n%q", clientId, msg.Name, msg.Data)
+			return
 		}
 
 		if msg.Name != "@oauth2" {
-			t.Fatalf("Expected @oauth2 msg.Name, got %q", msg.Name)
+			t.Errorf("Expected @oauth2 msg.Name, got %q", msg.Name)
+			return
 		}
 
 		for _, txt := range expectedMessages[clientId] {
 			if !strings.Contains(string(msg.Data), txt) {
-				t.Fatalf("Failed to find %q in \n%s", txt, msg.Data)
+				t.Errorf("Failed to find %q in \n%s", txt, msg.Data)
+				return
 			}
 		}
 	}
@@ -91,19 +97,37 @@ func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 
 				for {
 					select {
-					case msg := <-clients["c1"].Channel():
-						checkClientMessages(t, "c1", msg, expectedMessages)
-					case msg := <-clients["c2"].Channel():
-						checkClientMessages(t, "c2", msg, expectedMessages)
-					case msg := <-clients["c3"].Channel():
-						checkClientMessages(t, "c3", msg, expectedMessages)
-					case msg := <-clients["c4"].Channel():
-						checkClientMessages(t, "c4", msg, expectedMessages)
-					case msg := <-clients["c5"].Channel():
-						checkClientMessages(t, "c5", msg, expectedMessages)
+					case msg, ok := <-clients["c1"].Channel():
+						if ok {
+							checkClientMessages(t, "c1", msg, expectedMessages)
+						} else {
+							t.Errorf("Unexpected c1 closed channel")
+						}
+					case msg, ok := <-clients["c2"].Channel():
+						if ok {
+							checkClientMessages(t, "c2", msg, expectedMessages)
+						} else {
+							t.Errorf("Unexpected c2 closed channel")
+						}
+					case msg, ok := <-clients["c3"].Channel():
+						if ok {
+							checkClientMessages(t, "c3", msg, expectedMessages)
+						} else {
+							t.Errorf("Unexpected c3 closed channel")
+						}
+					case msg, ok := <-clients["c4"].Channel():
+						if ok {
+							checkClientMessages(t, "c4", msg, expectedMessages)
+						} else {
+							t.Errorf("Unexpected c4 closed channel")
+						}
+					case _, ok := <-clients["c5"].Channel():
+						if ok {
+							t.Errorf("Expected c5 channel to be closed")
+						}
 					case <-ctx.Done():
 						for _, c := range clients {
-							close(c.Channel())
+							c.Discard()
 						}
 						return
 					}
@@ -244,6 +268,96 @@ func TestRecordAuthWithOAuth2Redirect(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:   "(POST) Apple user's name json (nameKey error)",
+			Method: http.MethodPost,
+			URL:    "/api/oauth2-redirect",
+			Body: strings.NewReader(url.Values{
+				"code":  []string{strings.Repeat("a", 986)},
+				"state": []string{clientStubs[8]["c3"].Id()},
+				"user": []string{
+					`{"name":{"firstName":"aaa","lastName":"` + strings.Repeat("b", 200) + `"}}`,
+				},
+			}.Encode()),
+			Headers: map[string]string{
+				"content-type": "application/x-www-form-urlencoded",
+			},
+			BeforeTestFunc: beforeTestFunc(clientStubs[8], map[string][]string{
+				"c3": {`"state":"` + clientStubs[8]["c3"].Id(), `"code":"` + strings.Repeat("a", 986) + `"`},
+			}),
+			ExpectedStatus: http.StatusSeeOther,
+			ExpectedEvents: map[string]int{"*": 0},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				app.Store().Get("cancelFunc").(context.CancelFunc)()
+
+				checkSuccessRedirect(t, app, res)
+
+				if clientStubs[8]["c3"].HasSubscription("@oauth2") {
+					t.Fatalf("Expected oauth2 subscription to be removed")
+				}
+
+				if storedName := app.Store().Get("@redirect_name_" + strings.Repeat("a", 986)); storedName != nil {
+					t.Fatalf("Didn't expect stored name, got %q", storedName)
+				}
+			},
+		},
+		{
+			Name:   "(POST) Apple user's name json",
+			Method: http.MethodPost,
+			URL:    "/api/oauth2-redirect",
+			Body: strings.NewReader(url.Values{
+				"code":  []string{strings.Repeat("a", 985)},
+				"state": []string{clientStubs[9]["c3"].Id()},
+				"user": []string{
+					`{"name":{"firstName":"aaa","lastName":"` + strings.Repeat("b", 200) + `"}}`,
+				},
+			}.Encode()),
+			Headers: map[string]string{
+				"content-type": "application/x-www-form-urlencoded",
+			},
+			BeforeTestFunc: beforeTestFunc(clientStubs[9], map[string][]string{
+				"c3": {`"state":"` + clientStubs[9]["c3"].Id(), `"code":"` + strings.Repeat("a", 985) + `"`},
+			}),
+			ExpectedStatus: http.StatusSeeOther,
+			ExpectedEvents: map[string]int{"*": 0},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				app.Store().Get("cancelFunc").(context.CancelFunc)()
+
+				checkSuccessRedirect(t, app, res)
+
+				if clientStubs[9]["c3"].HasSubscription("@oauth2") {
+					t.Fatalf("Expected oauth2 subscription to be removed")
+				}
+
+				storedName, _ := app.Store().Get("@redirect_name_" + strings.Repeat("a", 985)).(string)
+				expectedName := "aaa " + strings.Repeat("b", 146)
+				if storedName != expectedName {
+					t.Fatalf("Expected stored name\n%q\ngot\n%q", expectedName, storedName)
+				}
+			},
+		},
+		{
+			Name:    "client with different IP",
+			Method:  http.MethodGet,
+			URL:     "/api/oauth2-redirect?code=123&state=" + clientStubs[10]["c3"].Id(),
+			Headers: map[string]string{"x-test-ip": "127.0.0.2"},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+				clientStubs[10]["c3"].Set(apis.RealtimeClientIPKey, "127.0.0.1")
+
+				beforeTestFunc(clientStubs[10], map[string][]string{
+					"c3": {`"state":"` + clientStubs[10]["c3"].Id(), `"code":"123"`},
+				})(t, app, e)
+			},
+			ExpectedStatus: http.StatusTemporaryRedirect,
+			ExpectedEvents: map[string]int{"*": 0},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				app.Store().Get("cancelFunc").(context.CancelFunc)()
+
+				checkFailureRedirect(t, app, res)
+			},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/record_auth_with_oauth2_test.go

@@ -14,6 +14,7 @@ import (
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/auth"
+	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"golang.org/x/oauth2"
 )
 
@@ -44,12 +45,14 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 	t.Parallel()
 
 	// start a test server
-	server := httptest.NewServer(http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) {
+	localServer := httptest.NewServer(http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) {
 		buf := new(bytes.Buffer)
 		png.Encode(buf, image.Rect(0, 0, 1, 1)) // tiny 1x1 png
 		http.ServeContent(res, req, "test_avatar.png", time.Now(), bytes.NewReader(buf.Bytes()))
 	}))
-	defer server.Close()
+	defer localServer.Close()
+
+	externalImageURL := "https://pocketbase.io/images/logo.svg"
 
 	scenarios := []tests.ApiScenario{
 		{
@@ -87,10 +90,11 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"data":{`,
 				`"provider":`,
 				`"code":`,
-				`"redirectURL":`,
 			},
 			NotExpectedContent: []string{
-				`"codeVerifier":`, // should be optional
+				// should be optional
+				`"codeVerifier":`,
+				`"redirectURL":`,
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
@@ -106,10 +110,11 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"data":{`,
 				`"provider":`,
 				`"code":`,
-				`"redirectURL":`,
 			},
 			NotExpectedContent: []string{
-				`"codeVerifier":`, // should be optional
+				// should be optional
+				`"codeVerifier":`,
+				`"redirectURL":`,
 			},
 			ExpectedEvents: map[string]int{"*": 0},
 		},
@@ -174,6 +179,34 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				if err := app.Save(ea); err != nil {
 					t.Fatal(err)
 				}
+
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that it was deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Id != ea.Id {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
+				// test at least once that the correct request info context is properly loaded
+				app.OnRecordAuthRequest().BindFunc(func(e *core.RecordAuthRequestEvent) error {
+					info, err := e.RequestInfo()
+					if err != nil {
+						t.Fatal(err)
+					}
+
+					if info.Context != core.RequestInfoContextOAuth2 {
+						t.Fatalf("Expected request context %q, got %q", core.RequestInfoContextOAuth2, info.Context)
+					}
+
+					return e.Next()
+				})
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
@@ -196,12 +229,12 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordAuthRequest":           1,
 				"OnRecordEnrich":                1,
 				// ---
-				"OnModelCreate":              1,
-				"OnModelCreateExecute":       1,
-				"OnModelAfterCreateSuccess":  1,
-				"OnRecordCreate":             1,
-				"OnRecordCreateExecute":      1,
-				"OnRecordAfterCreateSuccess": 1,
+				"OnModelCreate":              2, // user + recreated external auth
+				"OnModelCreateExecute":       2,
+				"OnModelAfterCreateSuccess":  2,
+				"OnRecordCreate":             2,
+				"OnRecordCreateExecute":      2,
+				"OnRecordAfterCreateSuccess": 2,
 				// ---
 				"OnModelUpdate":              1,
 				"OnModelUpdateExecute":       1,
@@ -210,8 +243,15 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
-				"OnModelValidate":  2, // create + update
-				"OnRecordValidate": 2,
+				"OnModelDelete":              3, // pre-existing external auths
+				"OnModelDeleteExecute":       3,
+				"OnModelAfterDeleteSuccess":  3,
+				"OnRecordDelete":             3,
+				"OnRecordDeleteExecute":      3,
+				"OnRecordAfterDeleteSuccess": 3,
+				// ---
+				"OnModelValidate":  3, // user create/update + recreated external auth
+				"OnRecordValidate": 3,
 			},
 			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
 				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
@@ -231,6 +271,24 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				if len(devices) != 1 {
 					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
 				}
+
+				// ensure that other linked external auths have been deleted
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) != 1 {
+					t.Fatalf("Expected only 1 external auth to remain, got %d", len(externalAuths))
+				}
+				if provider := externalAuths[0].Provider(); provider != "test" {
+					t.Fatalf("Expected %q external auth, got %q", "test", provider)
+				}
+				if providerId := externalAuths[0].ProviderId(); providerId != "test_id" {
+					t.Fatalf("Expected %q providerId, got %q", "test_id", providerId)
+				}
+				if recordRef := externalAuths[0].RecordRef(); recordRef != user.Id {
+					t.Fatalf("Expected %q recordRef, got %q", user.Id, recordRef)
+				}
+				if collectionRef := externalAuths[0].CollectionRef(); collectionRef != user.Collection().Id {
+					t.Fatalf("Expected %q collectionRef, got %q", user.Collection().Id, collectionRef)
+				}
 			},
 		},
 		{
@@ -292,6 +350,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":false`,
 				`"email":"test2@example.com"`,
 				`"id":"oap640cot4yru2s"`,
 				`"id":"test_id"`,
@@ -325,7 +384,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected old password %q to be valid", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -335,7 +394,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 		},
 		{
-			Name:   "link by email",
+			Name:   "link by email (unverified user)",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/auth-with-oauth2",
 			Body: strings.NewReader(`{
@@ -358,6 +417,20 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 					t.Fatalf("Expected password %q to be valid", "1234567890")
 				}
 
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that it was deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Provider() != "test" {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
 				// register the test provider
 				auth.Providers["test"] = func() auth.Provider {
 					return &oauth2MockProvider{
@@ -383,6 +456,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":false`,
 				`"email":"test@example.com"`,
 				`"id":"4q1xlclmfloku33"`,
 				`"id":"test_id"`,
@@ -413,6 +487,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -430,6 +511,145 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				if len(devices) != 1 {
 					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
 				}
+
+				// ensure that other linked external auths have been deleted
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) != 1 {
+					t.Fatalf("Expected only 1 external auth to remain, got %d", len(externalAuths))
+				}
+				if provider := externalAuths[0].Provider(); provider != "test" {
+					t.Fatalf("Expected %q external auth, got %q", "test", provider)
+				}
+				if providerId := externalAuths[0].ProviderId(); providerId != "test_id" {
+					t.Fatalf("Expected %q providerId, got %q", "test_id", providerId)
+				}
+				if recordRef := externalAuths[0].RecordRef(); recordRef != user.Id {
+					t.Fatalf("Expected %q recordRef, got %q", user.Id, recordRef)
+				}
+				if collectionRef := externalAuths[0].CollectionRef(); collectionRef != user.Collection().Id {
+					t.Fatalf("Expected %q collectionRef, got %q", user.Collection().Id, collectionRef)
+				}
+			},
+		},
+		{
+			Name:   "link by email (verified user)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"123",
+				"redirectURL": "https://example.com"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test3@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.Verified() {
+					t.Fatalf("Expected user %q to be verified", user.Email())
+				}
+
+				// ensure that the old password works
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected password %q to be valid", "1234567890")
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{Id: "test_id", Email: "test3@example.com"},
+						Token:    &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				// ensure that there is at least one other external auth different than test
+				// so that later we can verify that they are not deleted
+				var hasAtLeastOneOtherEA = false
+				externalAuths, _ := app.FindAllExternalAuthsByRecord(user)
+				for _, rel := range externalAuths {
+					if rel.Provider() != "test" {
+						hasAtLeastOneOtherEA = true
+						break
+					}
+				}
+				if !hasAtLeastOneOtherEA {
+					t.Fatal("Expected at least one non-test external auth linked")
+				}
+
+				// add the test provider in the collection
+				user.Collection().MFA.Enabled = false
+				user.Collection().OAuth2.Enabled = true
+				user.Collection().OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"record":{`,
+				`"token":"`,
+				`"meta":{`,
+				`"isNew":false`,
+				`"email":"test3@example.com"`,
+				`"id":"bgs820n361vj1qd"`,
+				`"id":"test_id"`,
+				`"verified":true`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                             0,
+				"OnRecordAuthWithOAuth2Request": 1,
+				"OnRecordAuthRequest":           1,
+				"OnRecordEnrich":                1,
+				// ---
+				"OnModelCreate":              2, // authOrigins + externalAuths
+				"OnModelCreateExecute":       2,
+				"OnModelAfterCreateSuccess":  2,
+				"OnRecordCreate":             2,
+				"OnRecordCreateExecute":      2,
+				"OnRecordAfterCreateSuccess": 2,
+				// ---
+				"OnModelValidate":  2, // authOrigins + externalAuths
+				"OnRecordValidate": 2,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindAuthRecordByEmail("users", "test3@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
+				}
+
+				devices, err := app.FindAllAuthOriginsByRecord(user)
+				if len(devices) != 1 {
+					t.Fatalf("Expected only 1 auth origin to be created, got %d (%v)", len(devices), err)
+				}
+
+				var hasTestEA = false
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if len(externalAuths) <= 1 {
+					t.Fatalf("Expected to have 2+ ExternalAuth records, got %d (%v)", len(externalAuths), err)
+				}
+				for _, rel := range externalAuths {
+					if rel.Provider() == "test" {
+						hasTestEA = true
+						break
+					}
+				}
+				if !hasTestEA {
+					t.Fatal("Expected test external auth to be linked")
+				}
 			},
 		},
 		{
@@ -488,6 +708,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":false`,
 				`"email":"test@example.com"`,
 				`"id":"4q1xlclmfloku33"`,
 				`"id":"test_id"`,
@@ -511,6 +732,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordCreateExecute":      2,
 				"OnRecordAfterCreateSuccess": 2,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  2,
 				"OnRecordValidate": 2,
 			},
@@ -521,7 +749,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -558,12 +786,21 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 					t.Fatalf("Expected password %q to be valid", "1234567890")
 				}
 
+				oldTokenKey := user.TokenKey()
+
 				// manually unset the user email
 				user.SetEmail("")
-				if err := app.Save(user); err != nil {
+				if err = app.Save(user); err != nil {
 					t.Fatal(err)
 				}
 
+				// resave with the old token key since the email change above
+				// would change it and will make the password token invalid
+				user.SetTokenKey(oldTokenKey)
+				if err = app.Save(user); err != nil {
+					t.Fatalf("Failed to restore original user tokenKey: %v", err)
+				}
+
 				// register the test provider
 				auth.Providers["test"] = func() auth.Provider {
 					return &oauth2MockProvider{
@@ -592,6 +829,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":false`,
 				`"email":"test_oauth2@example.com"`,
 				`"id":"4q1xlclmfloku33"`,
 				`"id":"test_id"`,
@@ -622,6 +860,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -632,7 +877,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -697,6 +942,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":false`,
 				`"email":"test@example.com"`,
 				`"id":"4q1xlclmfloku33"`,
 				`"id":"test_id"`,
@@ -727,6 +973,13 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
 				// ---
+				"OnModelDelete":              2, // pre-existing external auths
+				"OnModelDeleteExecute":       2,
+				"OnModelAfterDeleteSuccess":  2,
+				"OnRecordDelete":             2,
+				"OnRecordDeleteExecute":      2,
+				"OnRecordAfterDeleteSuccess": 2,
+				// ---
 				"OnModelValidate":  3, // record + authOrigins + externalAuths
 				"OnRecordValidate": 3,
 			},
@@ -737,7 +990,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				}
 
 				if !user.ValidatePassword("1234567890") {
-					t.Fatalf("Expected password %q not to be changed", "1234567890")
+					t.Fatalf("Expected old password %q to remain valid", "1234567890")
 				}
 
 				devices, err := app.FindAllAuthOriginsByRecord(user)
@@ -786,6 +1039,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"record":{`,
 				`"token":"`,
 				`"meta":{`,
+				`"isNew":true`,
 				`"email":""`,
 				`"id":"test_id"`,
 				`"verified":true`,
@@ -867,7 +1121,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				`"verified":{"code":"validation_values_mismatch"`,
 			},
 			NotExpectedContent: []string{
-				`"email":`, // the value is always overwritten with the OAuth2 user email
+				`"email":`, // ignored because the record validator never ran
 				`"rel":`,   // ignored because the record validator never ran
 				`"file":`,  // ignored because the record validator never ran
 			},
@@ -920,12 +1174,10 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			ExpectedStatus: 400,
 			ExpectedContent: []string{
 				`"data":{`,
+				`"email":{"code":"validation_is_email"`,
 				`"rel":{"code":"validation_missing_rel_records"`,
 				`"file":{"code":"validation_invalid_file"`,
 			},
-			NotExpectedContent: []string{
-				`"email":`, // the value is always overwritten with the OAuth2 user email
-			},
 			ExpectedEvents: map[string]int{
 				"*":                             0,
 				"OnRecordAuthWithOAuth2Request": 1,
@@ -939,7 +1191,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 		},
 		{
-			Name:   "creating user (valid create data)",
+			Name:   "creating user (valid create data with empty submitted email)",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/auth-with-oauth2",
 			Body: strings.NewReader(`{
@@ -947,8 +1199,10 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"code":"123",
 				"redirectURL": "https://example.com",
 				"createData": {
-					"email": "invalid",
+					"email": "",
 					"emailVisibility": true,
+					"password": "1234567890",
+					"passwordConfirm": "1234567890",
 					"name": "test_name",
 					"username": "test_username",
 					"rel": "0yxhwia2amd8gec"
@@ -982,6 +1236,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":""`,
 				`"emailVisibility":true`,
 				`"name":"test_name"`,
@@ -1018,6 +1273,105 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnModelValidate":  4,
 				"OnRecordValidate": 4,
 			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindFirstRecordByData("users", "username", "test_username")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected password %q to be valid", "1234567890")
+				}
+			},
+		},
+		{
+			Name:   "creating user (valid create data with non-empty valid submitted email)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"123",
+				"redirectURL": "https://example.com",
+				"createData": {
+					"email": "test_create@example.com",
+					"emailVisibility": true,
+					"password": "1234567890",
+					"passwordConfirm": "1234567890",
+					"name": "test_name",
+					"username": "test_username",
+					"rel": "0yxhwia2amd8gec"
+				}
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				usersCol, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{
+							Id:    "test_id",
+							Email: "oauth2@example.com", // should be ignored because of the explicit submitted email
+						},
+						Token: &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				// add the test provider in the collection
+				usersCol.MFA.Enabled = false
+				usersCol.OAuth2.Enabled = true
+				usersCol.OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(usersCol); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"email":"test_create@example.com"`,
+				`"emailVisibility":true`,
+				`"name":"test_name"`,
+				`"username":"test_username"`,
+				`"verified":false`,
+				`"rel":"0yxhwia2amd8gec"`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                             0,
+				"OnRecordAuthWithOAuth2Request": 1,
+				"OnRecordAuthRequest":           1,
+				"OnRecordCreateRequest":         1,
+				"OnRecordEnrich":                2, // the auth response and from the create request
+				// ---
+				"OnModelCreate":              3, // record + authOrigins + externalAuths
+				"OnModelCreateExecute":       3,
+				"OnModelAfterCreateSuccess":  3,
+				"OnRecordCreate":             3,
+				"OnRecordCreateExecute":      3,
+				"OnRecordAfterCreateSuccess": 3,
+				// ---
+				"OnModelValidate":  3,
+				"OnRecordValidate": 3,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				user, err := app.FindFirstRecordByData("users", "username", "test_username")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if !user.ValidatePassword("1234567890") {
+					t.Fatalf("Expected password %q to be valid", "1234567890")
+				}
+			},
 		},
 		{
 			Name:   "creating user (with mapped OAuth2 fields and avatarURL->file field)",
@@ -1046,7 +1400,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 							Id:        "oauth2_id",
 							Email:     "oauth2@example.com",
 							Username:  "oauth2_username",
-							AvatarURL: server.URL + "/oauth2_avatar.png",
+							AvatarURL: externalImageURL,
 						},
 						Token: &oauth2.Token{AccessToken: "abc"},
 					}
@@ -1071,13 +1425,105 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":"oauth2@example.com"`,
 				`"emailVisibility":true`,
 				`"name":"test_name"`,
 				`"username":"oauth2_username"`,
 				`"verified":true`,
 				`"rel":"0yxhwia2amd8gec"`,
-				`"avatar":"oauth2_avatar_`,
+				`"avatar":"logo_`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                             0,
+				"OnRecordAuthWithOAuth2Request": 1,
+				"OnRecordAuthRequest":           1,
+				"OnRecordCreateRequest":         1,
+				"OnRecordEnrich":                2, // the auth response and from the create request
+				// ---
+				"OnModelCreate":              3, // record + authOrigins + externalAuths
+				"OnModelCreateExecute":       3,
+				"OnModelAfterCreateSuccess":  3,
+				"OnRecordCreate":             3,
+				"OnRecordCreateExecute":      3,
+				"OnRecordAfterCreateSuccess": 3,
+				// ---
+				"OnModelUpdate":              1, // created record verified state change
+				"OnModelUpdateExecute":       1,
+				"OnModelAfterUpdateSuccess":  1,
+				"OnRecordUpdate":             1,
+				"OnRecordUpdateExecute":      1,
+				"OnRecordAfterUpdateSuccess": 1,
+				// ---
+				"OnModelValidate":  4,
+				"OnRecordValidate": 4,
+			},
+		},
+		{
+			Name:   "creating user (with mapped OAuth2 fields and local avatarURL->file field; ensures that safeHTTPClient is being used)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"123",
+				"redirectURL": "https://example.com",
+				"createData": {
+					"name": "test_name",
+					"emailVisibility": true,
+					"rel": "0yxhwia2amd8gec"
+				}
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				usersCol, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{
+							Id:        "oauth2_id",
+							Email:     "oauth2@example.com",
+							Username:  "oauth2_username",
+							AvatarURL: localServer.URL + "/oauth2_avatar.png", // local/private file download is not allowed
+						},
+						Token: &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				// add the test provider in the collection
+				usersCol.MFA.Enabled = false
+				usersCol.OAuth2.Enabled = true
+				usersCol.OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				usersCol.OAuth2.MappedFields = core.OAuth2KnownFields{
+					Username:  "name", // should be ignored because of the explicit submitted value
+					Id:        "username",
+					AvatarURL: "avatar",
+				}
+				if err := app.Save(usersCol); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"isNew":true`,
+				`"email":"oauth2@example.com"`,
+				`"emailVisibility":true`,
+				`"name":"test_name"`,
+				`"username":"oauth2_username"`,
+				`"verified":true`,
+				`"rel":"0yxhwia2amd8gec"`,
+				`"avatar":"`,
 			},
 			NotExpectedContent: []string{
 				// hidden fields
@@ -1153,6 +1599,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":"oauth2@example.com"`,
 				`"emailVisibility":false`,
 				`"verified":true`,
@@ -1189,7 +1636,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 		},
 		{
-			Name:   "creating user (with mapped OAuth2 fields and avatarURL->non-file field)",
+			Name:   "creating user (with mapped OAuth2 fields, case-sensitive username and avatarURL->non-file field)",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/auth-with-oauth2",
 			Body: strings.NewReader(`{
@@ -1209,9 +1656,9 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 						AuthUser: &auth.AuthUser{
 							Id:        "oauth2_id",
 							Email:     "oauth2@example.com",
-							Username:  "oauth2_username",
+							Username:  "tESt2_username", // wouldn't match with existing because the related field index is case-sensitive
 							Name:      "oauth2_name",
-							AvatarURL: server.URL + "/oauth2_avatar.png",
+							AvatarURL: localServer.URL + "/oauth2_avatar.png", // allowed because it is not being downloaded
 						},
 						Token: &oauth2.Token{AccessToken: "abc"},
 					}
@@ -1235,9 +1682,10 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":"oauth2@example.com"`,
 				`"emailVisibility":false`,
-				`"username":"oauth2_username"`,
+				`"username":"tESt2_username"`,
 				`"name":"http://127.`,
 				`"verified":true`,
 				`"avatar":""`,
@@ -1273,7 +1721,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 		},
 		{
-			Name:   "creating user (with mapped OAuth2 fields and duplicated username)",
+			Name:   "creating user (with mapped OAuth2 fields and duplicated case-insensitive username)",
 			Method: http.MethodPost,
 			URL:    "/api/collections/users/auth-with-oauth2",
 			Body: strings.NewReader(`{
@@ -1293,13 +1741,21 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 						AuthUser: &auth.AuthUser{
 							Id:       "oauth2_id",
 							Email:    "oauth2@example.com",
-							Username: "test2_username",
+							Username: "tESt2_username",
 							Name:     "oauth2_name",
 						},
 						Token: &oauth2.Token{AccessToken: "abc"},
 					}
 				}
 
+				// make the username index case-insensitive to ensure that case-insensitive match is used
+				index, ok := dbutils.FindSingleColumnUniqueIndex(usersCol.Indexes, "username")
+				if ok {
+					index.Columns[0].Collate = "nocase"
+					usersCol.RemoveIndex(index.IndexName)
+					usersCol.Indexes = append(usersCol.Indexes, index.Build())
+				}
+
 				// add the test provider in the collection
 				usersCol.MFA.Enabled = false
 				usersCol.OAuth2.Enabled = true
@@ -1317,6 +1773,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":"oauth2@example.com"`,
 				`"emailVisibility":false`,
 				`"verified":true`,
@@ -1398,6 +1855,7 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
+				`"isNew":true`,
 				`"email":"oauth2@example.com"`,
 				`"emailVisibility":false`,
 				`"verified":true`,
@@ -1434,6 +1892,167 @@ func TestRecordAuthWithOAuth2(t *testing.T) {
 				"OnRecordValidate": 4,
 			},
 		},
+		{
+			Name:   "OnRecordAuthWithOAuth2Request tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"123",
+				"redirectURL": "https://example.com"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{Id: "test_id"},
+						Token:    &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				// add the test provider in the collection
+				user.Collection().MFA.Enabled = false
+				user.Collection().OAuth2.Enabled = true
+				user.Collection().OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+
+				// stub linked provider
+				ea := core.NewExternalAuth(app)
+				ea.SetCollectionRef(user.Collection().Id)
+				ea.SetRecordRef(user.Id)
+				ea.SetProvider("test")
+				ea.SetProviderId("test_id")
+				if err := app.Save(ea); err != nil {
+					t.Fatal(err)
+				}
+
+				app.OnRecordAuthWithOAuth2Request().BindFunc(func(e *core.RecordAuthWithOAuth2RequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordAuthWithOAuth2Request": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
+
+		// Apple AuthUser.Name assign checks
+		// -----------------------------------------------------------
+		{
+			Name:   "store name with Apple provider",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "apple",
+				"code":"test_code",
+				"redirectURL": "https://example.com"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				users, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// register the test provider
+				auth.Providers[auth.NameApple] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{Id: "test_id"},
+						Token:    &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				app.Store().Set("@redirect_name_test_code", "test_store_name")
+
+				// add the test provider in the collection
+				users.MFA.Enabled = false
+				users.OAuth2.Enabled = true
+				users.OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         auth.NameApple,
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(users); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"meta":{`,
+				`"name":"test_store_name"`,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if app.Store().Has("@redirect_name_test_code") {
+					t.Fatal("Expected @redirect_name_test_code store key to be removed")
+				}
+			},
+		},
+		{
+			Name:   "store name with non-Apple provider",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-oauth2",
+			Body: strings.NewReader(`{
+				"provider": "test",
+				"code":"test_code",
+				"redirectURL": "https://example.com"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				users, err := app.FindCollectionByNameOrId("users")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// register the test provider
+				auth.Providers["test"] = func() auth.Provider {
+					return &oauth2MockProvider{
+						AuthUser: &auth.AuthUser{Id: "test_id"},
+						Token:    &oauth2.Token{AccessToken: "abc"},
+					}
+				}
+
+				app.Store().Set("@redirect_name_test_code", "test_store_name")
+
+				// add the test provider in the collection
+				users.MFA.Enabled = false
+				users.OAuth2.Enabled = true
+				users.OAuth2.Providers = []core.OAuth2ProviderConfig{{
+					Name:         "test",
+					ClientId:     "123",
+					ClientSecret: "456",
+				}}
+				if err := app.Save(users); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			NotExpectedContent: []string{
+				`"name":"test_store_name"`,
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if !app.Store().Has("@redirect_name_test_code") {
+					t.Fatal("Expected @redirect_name_test_code store key to NOT be deleted")
+				}
+			},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_with_otp.go

@@ -4,7 +4,7 @@ import (
 	"errors"
 	"fmt"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 )
 
@@ -26,6 +26,8 @@ func recordAuthWithOTP(e *core.RequestEvent) error {
 		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
 	}
 
+	e.Set(core.RequestEventKeyInfoContext, core.RequestInfoContextOTP)
+
 	event := new(core.RecordAuthWithOTPRequestEvent)
 	event.RequestEvent = e
 	event.Collection = collection
@@ -51,7 +53,7 @@ func recordAuthWithOTP(e *core.RequestEvent) error {
 		return e.BadRequestError("Invalid or expired OTP", fmt.Errorf("missing auth record: %w", err))
 	}
 
-	// since otps are usually simple digit numbers, enforce an extra rate limit rule as basic enumaration protection
+	// since otps are usually simple digit numbers, enforce an extra rate limit rule as basic enumeration protection
 	err = checkRateLimit(e, "@pb_otp_"+event.Record.Id, core.RateLimitRule{MaxRequests: 5, Duration: 180})
 	if err != nil {
 		return e.TooManyRequestsError("Too many attempts, please try again later with a new OTP.", nil)
@@ -63,34 +65,39 @@ func recordAuthWithOTP(e *core.RequestEvent) error {
 	// ---
 
 	return e.App.OnRecordAuthWithOTPRequest().Trigger(event, func(e *core.RecordAuthWithOTPRequestEvent) error {
+		otpId := e.OTP.Id
+		otpSentTo := e.OTP.SentTo()
+
+		// eagerly delete the OTP to avoid unnecessary double delete model hook calls
+		// triggered by the password change below
+		err := e.App.Delete(e.OTP)
+		if err != nil {
+			e.App.Logger().Error("Failed to delete used OTP", "error", err, "otpId", e.OTP.Id)
+		}
+
 		// update the user email verified state in case the OTP originate from an email address matching the current record one
 		//
 		// note: don't wait for success auth response (it could fail because of MFA) and because we already validated the OTP above
-		otpSentTo := e.OTP.SentTo()
 		if !e.Record.Verified() && otpSentTo != "" && e.Record.Email() == otpSentTo {
 			e.Record.SetVerified(true)
-			err = e.App.Save(e.Record)
-			if err != nil {
+
+			// this is technically not required but we enforce password
+			// reset on verified upgrades in case the OTP is used on its own
+			// since this makes it less error prone to pre-hijacking attacks
+			if !e.Record.Collection().MFA.Enabled {
+				e.Record.SetRandomPassword()
+			}
+
+			if err := e.App.Save(e.Record); err != nil {
 				e.App.Logger().Error("Failed to update record verified state after successful OTP validation",
 					"error", err,
-					"otpId", e.OTP.Id,
+					"otpId", otpId,
 					"recordId", e.Record.Id,
 				)
 			}
 		}
 
-		err = RecordAuthResponse(e.RequestEvent, e.Record, core.MFAMethodOTP, nil)
-		if err != nil {
-			return err
-		}
-
-		// try to delete the used otp
-		err = e.App.Delete(e.OTP)
-		if err != nil {
-			e.App.Logger().Error("Failed to delete used OTP", "error", err, "otpId", e.OTP.Id)
-		}
-
-		return nil
+		return RecordAuthResponse(e.RequestEvent, e.Record, core.MFAMethodOTP, nil)
 	})
 }
 

apis/record_auth_with_otp_test.go

@@ -269,6 +269,20 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if err := app.Save(otp); err != nil {
 					t.Fatal(err)
 				}
+
+				// test at least once that the correct request info context is properly loaded
+				app.OnRecordAuthRequest().BindFunc(func(e *core.RecordAuthRequestEvent) error {
+					info, err := e.RequestInfo()
+					if err != nil {
+						t.Fatal(err)
+					}
+
+					if info.Context != core.RequestInfoContextOTP {
+						t.Fatalf("Expected request context %q, got %q", core.RequestInfoContextOTP, info.Context)
+					}
+
+					return e.Next()
+				})
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
@@ -313,6 +327,15 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if user.Verified() {
 					t.Fatal("Expected the user to remain unverified because sentTo != email")
 				}
+
+				// ensure that all pre-existing OAuth2 were NOT deleted
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) != 2 {
+					t.Fatalf("Expected 2 external auths, found %d", len(externalAuths))
+				}
 			},
 		},
 		{
@@ -350,6 +373,15 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if err := app.Save(otp); err != nil {
 					t.Fatal(err)
 				}
+
+				// verify that there are at least one pre-existing OAuth2 link
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) == 0 {
+					t.Fatal("Expected at least one external auth")
+				}
 			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
@@ -374,10 +406,10 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				"OnModelCreate":             1,
 				"OnModelCreateExecute":      1,
 				"OnModelAfterCreateSuccess": 1,
-				// OTP delete
-				"OnModelDelete":             1,
-				"OnModelDeleteExecute":      1,
-				"OnModelAfterDeleteSuccess": 1,
+				// record OTP + 2 ExternalAuths delete
+				"OnModelDelete":             3,
+				"OnModelDeleteExecute":      3,
+				"OnModelAfterDeleteSuccess": 3,
 				// user verified update
 				"OnModelUpdate":             1,
 				"OnModelUpdateExecute":      1,
@@ -387,9 +419,9 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				"OnRecordCreate":             1,
 				"OnRecordCreateExecute":      1,
 				"OnRecordAfterCreateSuccess": 1,
-				"OnRecordDelete":             1,
-				"OnRecordDeleteExecute":      1,
-				"OnRecordAfterDeleteSuccess": 1,
+				"OnRecordDelete":             3,
+				"OnRecordDeleteExecute":      3,
+				"OnRecordAfterDeleteSuccess": 3,
 				"OnRecordUpdate":             1,
 				"OnRecordUpdateExecute":      1,
 				"OnRecordAfterUpdateSuccess": 1,
@@ -403,8 +435,73 @@ func TestRecordAuthWithOTP(t *testing.T) {
 				if !user.Verified() {
 					t.Fatal("Expected the user to be marked as verified")
 				}
+
+				// ensure that all pre-existing OTPs are cleared
+				otps, err := app.FindAllOTPsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(otps) > 0 {
+					t.Fatalf("Expected all OTPs to be cleared, found %d", len(otps))
+				}
+
+				// ensure that all pre-existing OAuth2 links are cleared
+				externalAuths, err := app.FindAllExternalAuthsByRecord(user)
+				if err != nil {
+					t.Fatal(err)
+				}
+				if len(externalAuths) > 0 {
+					t.Fatalf("Expected all external auths to be cleared, found %d", len(externalAuths))
+				}
 			},
 		},
+		{
+			Name:   "OnRecordAuthWithOTPRequest tx body write check",
+			Method: http.MethodPost,
+			URL:    "/api/collections/users/auth-with-otp",
+			Body: strings.NewReader(`{
+				"otpId":"` + strings.Repeat("a", 15) + `",
+				"password":"123456"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				user, err := app.FindAuthRecordByEmail("users", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// disable MFA
+				user.Collection().MFA.Enabled = false
+				if err = app.Save(user.Collection()); err != nil {
+					t.Fatal(err)
+				}
+
+				otp := core.NewOTP(app)
+				otp.Id = strings.Repeat("a", 15)
+				otp.SetCollectionRef(user.Collection().Id)
+				otp.SetRecordRef(user.Id)
+				otp.SetPassword("123456")
+				if err := app.Save(otp); err != nil {
+					t.Fatal(err)
+				}
+
+				app.OnRecordAuthWithOTPRequest().BindFunc(func(e *core.RecordAuthWithOTPRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordAuthWithOTPRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// rate limit checks
 		// -----------------------------------------------------------

apis/record_auth_with_password.go

@@ -3,10 +3,14 @@ package apis
 import (
 	"database/sql"
 	"errors"
+	"slices"
+	"strings"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/dbutils"
 	"github.com/pocketbase/pocketbase/tools/list"
 )
 
@@ -28,29 +32,40 @@ func recordAuthWithPassword(e *core.RequestEvent) error {
 		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
 	}
 
+	e.Set(core.RequestEventKeyInfoContext, core.RequestInfoContextPasswordAuth)
+
 	var foundRecord *core.Record
 	var foundErr error
 
 	if form.IdentityField != "" {
-		foundRecord, foundErr = e.App.FindFirstRecordByData(collection.Id, form.IdentityField, form.Identity)
+		foundRecord, foundErr = findRecordByIdentityField(e.App, collection, form.IdentityField, form.Identity)
 	} else {
-		// prioritize email lookup
-		isEmail := is.EmailFormat.Validate(form.Identity) == nil
-		if isEmail && list.ExistInSlice(core.FieldNameEmail, collection.PasswordAuth.IdentityFields) {
-			foundRecord, foundErr = e.App.FindAuthRecordByEmail(collection.Id, form.Identity)
+		identityFields := collection.PasswordAuth.IdentityFields
+
+		// @todo consider removing with the stable release or moving it in the collection save
+		//
+		// prioritize email lookup to minimize breaking changes with earlier versions
+		if len(identityFields) > 1 && identityFields[0] != core.FieldNameEmail {
+			identityFields = slices.Clone(identityFields)
+			slices.SortStableFunc(identityFields, func(a, b string) int {
+				if a == "email" {
+					return -1
+				}
+				if b == "email" {
+					return 1
+				}
+				return 0
+			})
 		}
 
-		// search by the other identity fields
-		if !isEmail || foundErr != nil {
-			for _, name := range collection.PasswordAuth.IdentityFields {
-				if !isEmail && name == core.FieldNameEmail {
-					continue // no need to search by the email field if it is not an email
-				}
+		for _, name := range identityFields {
+			if name == core.FieldNameEmail && is.EmailFormat.Validate(form.Identity) != nil {
+				continue // no need to query the database if we know that the submitted value is not an email
+			}
 
-				foundRecord, foundErr = e.App.FindFirstRecordByData(collection.Id, name, form.Identity)
-				if foundErr == nil {
-					break
-				}
+			foundRecord, foundErr = findRecordByIdentityField(e.App, collection, name, form.Identity)
+			if foundErr == nil {
+				break
 			}
 		}
 	}
@@ -70,6 +85,11 @@ func recordAuthWithPassword(e *core.RequestEvent) error {
 
 	return e.App.OnRecordAuthWithPasswordRequest().Trigger(event, func(e *core.RecordAuthWithPasswordRequestEvent) error {
 		if e.Record == nil || !e.Record.ValidatePassword(e.Password) {
+			// dummy password check to minimize enumeration side-channel attacks
+			if e.Record == nil {
+				dummyPasswordCheck(e.App, e.Collection)
+			}
+
 			return e.BadRequestError("Failed to authenticate.", errors.New("invalid login credentials"))
 		}
 
@@ -92,6 +112,53 @@ func (form *authWithPasswordForm) validate(collection *core.Collection) error {
 	return validation.ValidateStruct(form,
 		validation.Field(&form.Identity, validation.Required, validation.Length(1, 255)),
 		validation.Field(&form.Password, validation.Required, validation.Length(1, 255)),
-		validation.Field(&form.IdentityField, validation.In(list.ToInterfaceSlice(collection.PasswordAuth.IdentityFields)...)),
+		validation.Field(
+			&form.IdentityField,
+			validation.Length(1, 255),
+			validation.In(list.ToInterfaceSlice(collection.PasswordAuth.IdentityFields)...),
+		),
 	)
 }
+
+// dummy password check to minimize side-channel attacks
+// (performed with the collection configured field cost)
+func dummyPasswordCheck(app core.App, collection *core.Collection) {
+	record := &core.Record{}
+
+	// find any random existing record
+	err := app.RecordQuery(collection).Limit(1).One(record)
+	if err != nil {
+		return
+	}
+
+	// the value and result doesn't matter, we just need a constant-time check
+	_ = record.ValidatePassword("")
+}
+
+func findRecordByIdentityField(app core.App, collection *core.Collection, field string, value any) (*core.Record, error) {
+	if !slices.Contains(collection.PasswordAuth.IdentityFields, field) {
+		return nil, errors.New("invalid identity field " + field)
+	}
+
+	index, ok := dbutils.FindSingleColumnUniqueIndex(collection.Indexes, field)
+	if !ok {
+		return nil, errors.New("missing " + field + " unique index constraint")
+	}
+
+	var expr dbx.Expression
+	if strings.EqualFold(index.Columns[0].Collate, "nocase") {
+		// case-insensitive search
+		expr = dbx.NewExp("[["+field+"]] = {:identity} COLLATE NOCASE", dbx.Params{"identity": value})
+	} else {
+		expr = dbx.HashExp{field: value}
+	}
+
+	record := &core.Record{}
+
+	err := app.RecordQuery(collection).AndWhere(expr).Limit(1).One(record)
+	if err != nil {
+		return nil, err
+	}
+
+	return record, nil
+}

apis/record_auth_with_password_test.go

@@ -1,18 +1,44 @@
 package apis_test
 
 import (
-	"errors"
 	"net/http"
 	"strings"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/dbutils"
 )
 
 func TestRecordAuthWithPassword(t *testing.T) {
 	t.Parallel()
 
+	updateIdentityIndex := func(collectionIdOrName string, fieldCollateMap map[string]string) func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+		return func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+			collection, err := app.FindCollectionByNameOrId("clients")
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			for column, collate := range fieldCollateMap {
+				index, ok := dbutils.FindSingleColumnUniqueIndex(collection.Indexes, column)
+				if !ok {
+					t.Fatalf("Missing unique identityField index for column %q", column)
+				}
+
+				index.Columns[0].Collate = collate
+
+				collection.RemoveIndex(index.IndexName)
+				collection.Indexes = append(collection.Indexes, index.Build())
+			}
+
+			err = app.Save(collection)
+			if err != nil {
+				t.Fatalf("Failed to update identityField index: %v", err)
+			}
+		}
+	}
+
 	scenarios := []tests.ApiScenario{
 		{
 			Name:            "disabled password auth",
@@ -55,7 +81,7 @@ func TestRecordAuthWithPassword(t *testing.T) {
 			ExpectedEvents: map[string]int{"*": 0},
 		},
 		{
-			Name:   "OnRecordAuthWithPasswordRequest error response",
+			Name:   "OnRecordAuthWithPasswordRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections/clients/auth-with-password",
 			Body: strings.NewReader(`{
@@ -64,15 +90,22 @@ func TestRecordAuthWithPassword(t *testing.T) {
 			}`),
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordAuthWithPasswordRequest().BindFunc(func(e *core.RecordAuthWithPasswordRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                               0,
-				"OnRecordAuthWithPasswordRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordAuthWithPasswordRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 		{
 			Name:   "valid identity field and invalid password",
@@ -99,6 +132,21 @@ func TestRecordAuthWithPassword(t *testing.T) {
 				"identity":"test@example.com",
 				"password":"1234567890"
 			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				// test at least once that the correct request info context is properly loaded
+				app.OnRecordAuthRequest().BindFunc(func(e *core.RecordAuthRequestEvent) error {
+					info, err := e.RequestInfo()
+					if err != nil {
+						t.Fatal(err)
+					}
+
+					if info.Context != core.RequestInfoContextPasswordAuth {
+						t.Fatalf("Expected request context %q, got %q", core.RequestInfoContextPasswordAuth, info.Context)
+					}
+
+					return e.Next()
+				})
+			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`"email":"test@example.com"`,
@@ -164,6 +212,73 @@ func TestRecordAuthWithPassword(t *testing.T) {
 				"OnMailerRecordAuthAlertSend": 1,
 			},
 		},
+		{
+			// https://github.com/pocketbase/pocketbase/issues/7256
+			Name:   "valid non-email identity field with a value that is a properly formatted email",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identity":"username_as_email@example.com",
+				"password":"1234567890"
+			}`),
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				record, err := app.FindAuthRecordByEmail("clients", "test@example.com")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				record.Set("username", "username_as_email@example.com")
+
+				err = app.SaveNoValidate(record)
+				if err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"email":"test@example.com"`,
+				`"username":"username_as_email@example.com"`,
+				`"token":`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                               0,
+				"OnRecordAuthWithPasswordRequest": 1,
+				"OnRecordAuthRequest":             1,
+				"OnRecordEnrich":                  1,
+				// authOrigin track
+				"OnModelCreate":               1,
+				"OnModelCreateExecute":        1,
+				"OnModelAfterCreateSuccess":   1,
+				"OnModelValidate":             1,
+				"OnRecordCreate":              1,
+				"OnRecordCreateExecute":       1,
+				"OnRecordAfterCreateSuccess":  1,
+				"OnRecordValidate":            1,
+				"OnMailerSend":                1,
+				"OnMailerRecordAuthAlertSend": 1,
+			},
+		},
+		{
+			Name:   "unknown explicit identityField",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identityField": "created",
+				"identity":"test@example.com",
+				"password":"1234567890"
+			}`),
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"identityField":{"code":"validation_in_invalid"`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
 		{
 			Name:   "valid identity field and valid password with mismatched explicit identityField",
 			Method: http.MethodPost,
@@ -440,6 +555,141 @@ func TestRecordAuthWithPassword(t *testing.T) {
 			},
 		},
 
+		// case sensitivity checks
+		// -----------------------------------------------------------
+		{
+			Name:   "with explicit identityField (case-sensitive)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identityField": "username",
+				"identity":"Clients57772",
+				"password":"1234567890"
+			}`),
+			BeforeTestFunc:  updateIdentityIndex("clients", map[string]string{"username": ""}),
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents: map[string]int{
+				"*":                               0,
+				"OnRecordAuthWithPasswordRequest": 1,
+			},
+		},
+		{
+			Name:   "with explicit identityField (case-insensitive)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identityField": "username",
+				"identity":"Clients57772",
+				"password":"1234567890"
+			}`),
+			BeforeTestFunc: updateIdentityIndex("clients", map[string]string{"username": "nocase"}),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"email":"test@example.com"`,
+				`"username":"clients57772"`,
+				`"token":`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                               0,
+				"OnRecordAuthWithPasswordRequest": 1,
+				"OnRecordAuthRequest":             1,
+				"OnRecordEnrich":                  1,
+				// authOrigin track
+				"OnModelCreate":               1,
+				"OnModelCreateExecute":        1,
+				"OnModelAfterCreateSuccess":   1,
+				"OnModelValidate":             1,
+				"OnRecordCreate":              1,
+				"OnRecordCreateExecute":       1,
+				"OnRecordAfterCreateSuccess":  1,
+				"OnRecordValidate":            1,
+				"OnMailerSend":                1,
+				"OnMailerRecordAuthAlertSend": 1,
+			},
+		},
+		{
+			Name:   "without explicit identityField and non-email field (case-insensitive)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identity":"Clients57772",
+				"password":"1234567890"
+			}`),
+			BeforeTestFunc: updateIdentityIndex("clients", map[string]string{"username": "nocase"}),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"email":"test@example.com"`,
+				`"username":"clients57772"`,
+				`"token":`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                               0,
+				"OnRecordAuthWithPasswordRequest": 1,
+				"OnRecordAuthRequest":             1,
+				"OnRecordEnrich":                  1,
+				// authOrigin track
+				"OnModelCreate":               1,
+				"OnModelCreateExecute":        1,
+				"OnModelAfterCreateSuccess":   1,
+				"OnModelValidate":             1,
+				"OnRecordCreate":              1,
+				"OnRecordCreateExecute":       1,
+				"OnRecordAfterCreateSuccess":  1,
+				"OnRecordValidate":            1,
+				"OnMailerSend":                1,
+				"OnMailerRecordAuthAlertSend": 1,
+			},
+		},
+		{
+			Name:   "without explicit identityField and email field (case-insensitive)",
+			Method: http.MethodPost,
+			URL:    "/api/collections/clients/auth-with-password",
+			Body: strings.NewReader(`{
+				"identity":"tESt@example.com",
+				"password":"1234567890"
+			}`),
+			BeforeTestFunc: updateIdentityIndex("clients", map[string]string{"email": "nocase"}),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"email":"test@example.com"`,
+				`"username":"clients57772"`,
+				`"token":`,
+			},
+			NotExpectedContent: []string{
+				// hidden fields
+				`"tokenKey"`,
+				`"password"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                               0,
+				"OnRecordAuthWithPasswordRequest": 1,
+				"OnRecordAuthRequest":             1,
+				"OnRecordEnrich":                  1,
+				// authOrigin track
+				"OnModelCreate":               1,
+				"OnModelCreateExecute":        1,
+				"OnModelAfterCreateSuccess":   1,
+				"OnModelValidate":             1,
+				"OnRecordCreate":              1,
+				"OnRecordCreateExecute":       1,
+				"OnRecordAfterCreateSuccess":  1,
+				"OnRecordValidate":            1,
+				"OnMailerSend":                1,
+				"OnMailerRecordAuthAlertSend": 1,
+			},
+		},
+
 		// rate limit checks
 		// -----------------------------------------------------------
 		{

apis/record_crud.go

@@ -13,8 +13,11 @@ import (
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/tools/filesystem"
+	"github.com/pocketbase/pocketbase/tools/inflector"
+	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/router"
 	"github.com/pocketbase/pocketbase/tools/search"
+	"github.com/pocketbase/pocketbase/tools/security"
 )
 
 // bindRecordCrudApi registers the record crud api endpoints and
@@ -25,9 +28,9 @@ func bindRecordCrudApi(app core.App, rg *router.RouterGroup[*core.RequestEvent])
 	subGroup := rg.Group("/collections/{collection}/records").Unbind(DefaultRateLimitMiddlewareId)
 	subGroup.GET("", recordsList)
 	subGroup.GET("/{id}", recordView)
-	subGroup.POST("", recordCreate(nil)).Bind(dynamicCollectionBodyLimit(""))
-	subGroup.PATCH("/{id}", recordUpdate(nil)).Bind(dynamicCollectionBodyLimit(""))
-	subGroup.DELETE("/{id}", recordDelete(nil))
+	subGroup.POST("", recordCreate(true, nil)).Bind(dynamicCollectionBodyLimit(""))
+	subGroup.PATCH("/{id}", recordUpdate(true, nil)).Bind(dynamicCollectionBodyLimit(""))
+	subGroup.DELETE("/{id}", recordDelete(true, nil))
 }
 
 func recordsList(e *core.RequestEvent) error {
@@ -56,23 +59,32 @@ func recordsList(e *core.RequestEvent) error {
 		return err
 	}
 
-	fieldsResolver := core.NewRecordFieldResolver(
-		e.App,
-		collection,
-		requestInfo,
-		// hidden fields are searchable only by superusers
-		requestInfo.HasSuperuserAuth(),
-	)
+	query := e.App.RecordQuery(collection)
 
-	searchProvider := search.NewProvider(fieldsResolver).
-		Query(e.App.RecordQuery(collection))
+	fieldsResolver := core.NewRecordFieldResolver(e.App, collection, requestInfo, true)
 
-	if !requestInfo.HasSuperuserAuth() && collection.ListRule != nil {
-		searchProvider.AddFilter(search.FilterData(*collection.ListRule))
+	if !requestInfo.HasSuperuserAuth() && collection.ListRule != nil && *collection.ListRule != "" {
+		expr, err := search.FilterData(*collection.ListRule).BuildExpr(fieldsResolver)
+		if err != nil {
+			return err
+		}
+		query.AndWhere(expr)
+
+		// will be applied by the search provider right before executing the query
+		// fieldsResolver.UpdateQuery(query)
+	}
+
+	// hidden fields are searchable only by superusers
+	fieldsResolver.SetAllowHiddenFields(requestInfo.HasSuperuserAuth())
+
+	searchProvider := search.NewProvider(fieldsResolver).Query(query)
+
+	// use rowid when available to minimize the need of a covering index with the "id" field
+	if !collection.IsView() {
+		searchProvider.CountCol("_rowid_")
 	}
 
 	records := []*core.Record{}
-
 	result, err := searchProvider.ParseAndExec(e.Request.URL.Query().Encode(), &records)
 	if err != nil {
 		return firstApiError(err, e.BadRequestError("", err))
@@ -91,9 +103,9 @@ func recordsList(e *core.RequestEvent) error {
 
 		// Add a randomized throttle in case of too many empty search filter attempts.
 		//
-		// This is just for extra precaution since security researches raised concern regarding the possibity of eventual
+		// This is just for extra precaution since security researches raised concern regarding the possibility of eventual
 		// timing attacks because the List API rule acts also as filter and executes in a single run with the client-side filters.
-		// This is by design and it is an accepted tradeoff between performance, usability and correctness.
+		// This is by design and it is an accepted trade off between performance, usability and correctness.
 		//
 		// While technically the below doesn't fully guarantee protection against filter timing attacks, in practice combined with the network latency it makes them even less feasible.
 		// A properly configured rate limiter or individual fields Hidden checks are better suited if you are really concerned about eventual information disclosure by side-channel attacks.
@@ -106,10 +118,12 @@ func recordsList(e *core.RequestEvent) error {
 			len(e.Records) == 0 &&
 			checkRateLimit(e.RequestEvent, "@pb_list_timing_check_"+collection.Id, listTimingRateLimitRule) != nil {
 			e.App.Logger().Debug("Randomized throttle because of too many failed searches", "collectionId", collection.Id)
-			randomizedThrottle(100)
+			randomizedThrottle(500)
 		}
 
-		return e.JSON(http.StatusOK, e.Result)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Result)
+		})
 	})
 }
 
@@ -155,12 +169,18 @@ func recordView(e *core.RequestEvent) error {
 	ruleFunc := func(q *dbx.SelectQuery) error {
 		if !requestInfo.HasSuperuserAuth() && collection.ViewRule != nil && *collection.ViewRule != "" {
 			resolver := core.NewRecordFieldResolver(e.App, collection, requestInfo, true)
+
 			expr, err := search.FilterData(*collection.ViewRule).BuildExpr(resolver)
 			if err != nil {
 				return err
 			}
-			resolver.UpdateQuery(q)
+
 			q.AndWhere(expr)
+
+			err = resolver.UpdateQuery(q)
+			if err != nil {
+				return err
+			}
 		}
 		return nil
 	}
@@ -180,11 +200,13 @@ func recordView(e *core.RequestEvent) error {
 			return firstApiError(err, e.InternalServerError("Failed to enrich record", err))
 		}
 
-		return e.JSON(http.StatusOK, e.Record)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Record)
+		})
 	})
 }
 
-func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent) error {
+func recordCreate(responseWriteAfterTx bool, optFinalizer func(data any) error) func(e *core.RequestEvent) error {
 	return func(e *core.RequestEvent) error {
 		collection, err := e.App.FindCachedCollectionByNameOrId(e.Request.PathValue("collection"))
 		if err != nil || collection == nil {
@@ -217,6 +239,16 @@ func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 			return firstApiError(err, e.BadRequestError("Failed to read the submitted data.", err))
 		}
 
+		// set a random password for the OAuth2 ignoring its plain password validators
+		var skipPlainPasswordRecordValidators bool
+		if requestInfo.Context == core.RequestInfoContextOAuth2 {
+			if _, ok := data[core.FieldNamePassword]; !ok {
+				data[core.FieldNamePassword] = security.RandomString(30)
+				data[core.FieldNamePassword+"Confirm"] = data[core.FieldNamePassword]
+				skipPlainPasswordRecordValidators = true
+			}
+		}
+
 		// replace modifiers fields so that the resolved value is always
 		// available when accessing requestInfo.Body
 		requestInfo.Body = data
@@ -227,6 +259,83 @@ func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 		}
 		form.Load(data)
 
+		if skipPlainPasswordRecordValidators {
+			// unset the plain value to skip the plain password field validators
+			if raw, ok := record.GetRaw(core.FieldNamePassword).(*core.PasswordFieldValue); ok {
+				raw.Plain = ""
+			}
+		}
+
+		// check the request and record data against the create and manage rules
+		if !hasSuperuserAuth && collection.CreateRule != nil {
+			dummyRecord := record.Clone()
+
+			dummyRandomPart := "__pb_create__" + security.PseudorandomString(6)
+
+			// set an id if it doesn't have already
+			// (the value doesn't matter; it is used only to minimize the breaking changes with earlier versions)
+			if dummyRecord.Id == "" {
+				dummyRecord.Id = "__temp_id__" + dummyRandomPart
+			}
+
+			// unset the verified field to prevent manage API rule misuse in case the rule relies on it
+			dummyRecord.SetVerified(false)
+
+			// export the dummy record data into db params
+			dummyExport, err := dummyRecord.DBExport(e.App)
+			if err != nil {
+				return e.BadRequestError("Failed to create record", fmt.Errorf("dummy DBExport error: %w", err))
+			}
+
+			dummyParams := make(dbx.Params, len(dummyExport))
+			selects := make([]string, 0, len(dummyExport))
+			var param string
+			for k, v := range dummyExport {
+				k = inflector.Columnify(k) // columnify is just as extra measure in case of custom fields
+				param = "__pb_create__" + k
+				dummyParams[param] = v
+				selects = append(selects, "{:"+param+"} AS [["+k+"]]")
+			}
+
+			// shallow clone the current collection
+			dummyCollection := *collection
+			dummyCollection.Id += dummyRandomPart
+			dummyCollection.Name += inflector.Columnify(dummyRandomPart)
+
+			withFrom := fmt.Sprintf("WITH {{%s}} as (SELECT %s)", dummyCollection.Name, strings.Join(selects, ","))
+
+			// check non-empty create rule
+			if *dummyCollection.CreateRule != "" {
+				ruleQuery := e.App.ConcurrentDB().Select("(1)").PreFragment(withFrom).From(dummyCollection.Name).AndBind(dummyParams)
+
+				resolver := core.NewRecordFieldResolver(e.App, &dummyCollection, requestInfo, true)
+
+				expr, err := search.FilterData(*dummyCollection.CreateRule).BuildExpr(resolver)
+				if err != nil {
+					return e.BadRequestError("Failed to create record", fmt.Errorf("create rule build expression failure: %w", err))
+				}
+				ruleQuery.AndWhere(expr)
+
+				err = resolver.UpdateQuery(ruleQuery)
+				if err != nil {
+					return e.BadRequestError("Failed to create record", fmt.Errorf("create rule update query failure: %w", err))
+				}
+
+				var exists int
+				err = ruleQuery.Limit(1).Row(&exists)
+				if err != nil || exists == 0 {
+					return e.BadRequestError("Failed to create record", fmt.Errorf("create rule failure: %w", err))
+				}
+			}
+
+			// check for manage rule access
+			manageRuleQuery := e.App.ConcurrentDB().Select("(1)").PreFragment(withFrom).From(dummyCollection.Name).AndBind(dummyParams)
+			if !form.HasManageAccess() &&
+				hasAuthManageAccess(e.App, requestInfo, &dummyCollection, manageRuleQuery) {
+				form.GrantManagerAccess()
+			}
+		}
+
 		var isOptFinalizerCalled bool
 
 		event := new(core.RecordRequestEvent)
@@ -238,59 +347,9 @@ func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 			form.SetApp(e.App)
 			form.SetRecord(e.Record)
 
-			// temporary save the record and check it against the create and manage rules
-			if !hasSuperuserAuth && e.Collection.CreateRule != nil {
-				// temporary grant manager access level
-				form.GrantManagerAccess()
-
-				// manually unset the verified field to prevent manage API rule misuse in case the rule relies on it
-				initialVerified := e.Record.Verified()
-				if initialVerified {
-					e.Record.SetVerified(false)
-				}
-
-				createRuleFunc := func(q *dbx.SelectQuery) error {
-					if *e.Collection.CreateRule == "" {
-						return nil // no create rule to resolve
-					}
-
-					resolver := core.NewRecordFieldResolver(e.App, e.Collection, requestInfo, true)
-					expr, err := search.FilterData(*e.Collection.CreateRule).BuildExpr(resolver)
-					if err != nil {
-						return err
-					}
-					resolver.UpdateQuery(q)
-					q.AndWhere(expr)
-
-					return nil
-				}
-
-				testErr := form.DrySubmit(func(txApp core.App, drySavedRecord *core.Record) error {
-					foundRecord, err := txApp.FindRecordById(drySavedRecord.Collection(), drySavedRecord.Id, createRuleFunc)
-					if err != nil {
-						return fmt.Errorf("DrySubmit create rule failure: %w", err)
-					}
-
-					// reset the form access level in case it satisfies the Manage API rule
-					if !hasAuthManageAccess(txApp, requestInfo, foundRecord) {
-						form.ResetAccess()
-					}
-
-					return nil
-				})
-				if testErr != nil {
-					return e.BadRequestError("Failed to create record.", testErr)
-				}
-
-				// restore initial verified state (it will be further validated on submit)
-				if initialVerified != e.Record.Verified() {
-					e.Record.SetVerified(initialVerified)
-				}
-			}
-
 			err := form.Submit()
 			if err != nil {
-				return firstApiError(err, e.BadRequestError("Failed to create record.", err))
+				return firstApiError(err, e.BadRequestError("Failed to create record", err))
 			}
 
 			err = EnrichRecord(e.RequestEvent, e.Record)
@@ -298,7 +357,9 @@ func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 				return firstApiError(err, e.InternalServerError("Failed to enrich record", err))
 			}
 
-			err = e.JSON(http.StatusOK, e.Record)
+			err = execAfterSuccessTx(responseWriteAfterTx, e.App, func() error {
+				return e.JSON(http.StatusOK, e.Record)
+			})
 			if err != nil {
 				return err
 			}
@@ -328,7 +389,7 @@ func recordCreate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 	}
 }
 
-func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent) error {
+func recordUpdate(responseWriteAfterTx bool, optFinalizer func(data any) error) func(e *core.RequestEvent) error {
 	return func(e *core.RequestEvent) error {
 		collection, err := e.App.FindCachedCollectionByNameOrId(e.Request.PathValue("collection"))
 		if err != nil || collection == nil {
@@ -378,12 +439,18 @@ func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 		ruleFunc := func(q *dbx.SelectQuery) error {
 			if !hasSuperuserAuth && collection.UpdateRule != nil && *collection.UpdateRule != "" {
 				resolver := core.NewRecordFieldResolver(e.App, collection, requestInfo, true)
+
 				expr, err := search.FilterData(*collection.UpdateRule).BuildExpr(resolver)
 				if err != nil {
 					return err
 				}
-				resolver.UpdateQuery(q)
+
 				q.AndWhere(expr)
+
+				err = resolver.UpdateQuery(q)
+				if err != nil {
+					return err
+				}
 			}
 			return nil
 		}
@@ -400,6 +467,14 @@ func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 		}
 		form.Load(data)
 
+		manageRuleQuery := e.App.ConcurrentDB().Select("(1)").From(collection.Name).AndWhere(dbx.HashExp{
+			collection.Name + ".id": record.Id,
+		})
+		if !form.HasManageAccess() &&
+			hasAuthManageAccess(e.App, requestInfo, collection, manageRuleQuery) {
+			form.GrantManagerAccess()
+		}
+
 		var isOptFinalizerCalled bool
 
 		event := new(core.RecordRequestEvent)
@@ -410,9 +485,6 @@ func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 		hookErr := e.App.OnRecordUpdateRequest().Trigger(event, func(e *core.RecordRequestEvent) error {
 			form.SetApp(e.App)
 			form.SetRecord(e.Record)
-			if !form.HasManageAccess() && hasAuthManageAccess(e.App, requestInfo, e.Record) {
-				form.GrantManagerAccess()
-			}
 
 			err := form.Submit()
 			if err != nil {
@@ -424,7 +496,9 @@ func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 				return firstApiError(err, e.InternalServerError("Failed to enrich record", err))
 			}
 
-			err = e.JSON(http.StatusOK, e.Record)
+			err = execAfterSuccessTx(responseWriteAfterTx, e.App, func() error {
+				return e.JSON(http.StatusOK, e.Record)
+			})
 			if err != nil {
 				return err
 			}
@@ -454,7 +528,7 @@ func recordUpdate(optFinalizer func(data any) error) func(e *core.RequestEvent)
 	}
 }
 
-func recordDelete(optFinalizer func(data any) error) func(e *core.RequestEvent) error {
+func recordDelete(responseWriteAfterTx bool, optFinalizer func(data any) error) func(e *core.RequestEvent) error {
 	return func(e *core.RequestEvent) error {
 		collection, err := e.App.FindCachedCollectionByNameOrId(e.Request.PathValue("collection"))
 		if err != nil || collection == nil {
@@ -487,12 +561,18 @@ func recordDelete(optFinalizer func(data any) error) func(e *core.RequestEvent)
 		ruleFunc := func(q *dbx.SelectQuery) error {
 			if !requestInfo.HasSuperuserAuth() && collection.DeleteRule != nil && *collection.DeleteRule != "" {
 				resolver := core.NewRecordFieldResolver(e.App, collection, requestInfo, true)
+
 				expr, err := search.FilterData(*collection.DeleteRule).BuildExpr(resolver)
 				if err != nil {
 					return err
 				}
-				resolver.UpdateQuery(q)
+
 				q.AndWhere(expr)
+
+				err = resolver.UpdateQuery(q)
+				if err != nil {
+					return err
+				}
 			}
 			return nil
 		}
@@ -514,7 +594,9 @@ func recordDelete(optFinalizer func(data any) error) func(e *core.RequestEvent)
 				return firstApiError(err, e.BadRequestError("Failed to delete record. Make sure that the record is not part of a required relation reference.", err))
 			}
 
-			err = e.NoContent(http.StatusNoContent)
+			err = execAfterSuccessTx(responseWriteAfterTx, e.App, func() error {
+				return e.NoContent(http.StatusNoContent)
+			})
 			if err != nil {
 				return err
 			}
@@ -561,9 +643,28 @@ func recordDataFromRequest(e *core.RequestEvent, record *core.Record) (map[strin
 		return nil, err
 	}
 	if len(uploadedFiles) > 0 {
-		for k, v := range uploadedFiles {
-			result[k] = v
+		for k, files := range uploadedFiles {
+			uploaded := make([]any, 0, len(files))
+
+			// if not remove/prepend/append -> merge with the submitted
+			// info.Body values to prevent accidental old files deletion
+			if info.Body[k] != nil &&
+				!strings.HasPrefix(k, "+") &&
+				!strings.HasSuffix(k, "+") &&
+				!strings.HasSuffix(k, "-") {
+				existing := list.ToUniqueStringSlice(info.Body[k])
+				for _, name := range existing {
+					uploaded = append(uploaded, name)
+				}
+			}
+
+			for _, file := range files {
+				uploaded = append(uploaded, file)
+			}
+
+			result[k] = uploaded
 		}
+
 		result = record.ReplaceModifiers(result)
 	}
 
@@ -624,3 +725,42 @@ func extractUploadedFiles(re *core.RequestEvent, collection *core.Collection, pr
 
 	return result, nil
 }
+
+// hasAuthManageAccess checks whether the client is allowed to have
+// [forms.RecordUpsert] auth management permissions
+// (e.g. allowing to change system auth fields without oldPassword).
+func hasAuthManageAccess(app core.App, requestInfo *core.RequestInfo, collection *core.Collection, query *dbx.SelectQuery) bool {
+	if !collection.IsAuth() {
+		return false
+	}
+
+	manageRule := collection.ManageRule
+
+	if manageRule == nil || *manageRule == "" {
+		return false // only for superusers (manageRule can't be empty)
+	}
+
+	if requestInfo == nil || requestInfo.Auth == nil {
+		return false // no auth record
+	}
+
+	resolver := core.NewRecordFieldResolver(app, collection, requestInfo, true)
+
+	expr, err := search.FilterData(*manageRule).BuildExpr(resolver)
+	if err != nil {
+		app.Logger().Error("Manage rule build expression error", "error", err, "collectionId", collection.Id)
+		return false
+	}
+	query.AndWhere(expr)
+
+	err = resolver.UpdateQuery(query)
+	if err != nil {
+		return false
+	}
+
+	var exists int
+
+	err = query.Limit(1).Row(&exists)
+
+	return err == nil && exists > 0
+}

apis/record_crud_test.go

@@ -2,7 +2,6 @@ package apis_test
 
 import (
 	"bytes"
-	"errors"
 	"net/http"
 	"net/url"
 	"os"
@@ -259,6 +258,110 @@ func TestRecordCrudList(t *testing.T) {
 				"OnRecordEnrich":       4,
 			},
 		},
+		{
+			Name:   "authenticated regular record that matches the collection list rule with hidden field",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo3/records",
+			Headers: map[string]string{
+				// clients, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				col, err := app.FindCollectionByNameOrId("demo3")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// mock hidden field
+				col.Fields.GetByName("title").SetHidden(true)
+
+				col.ListRule = types.Pointer("title ~ 'test'")
+
+				if err = app.Save(col); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":1`,
+				`"totalItems":4`,
+				`"items":[{`,
+				`"id":"1tmknxy2868d869"`,
+				`"id":"lcl9d87w22ml6jy"`,
+				`"id":"7nwo8tuiatetxdm"`,
+				`"id":"mk5fmymtx4wsprk"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                    0,
+				"OnRecordsListRequest": 1,
+				"OnRecordEnrich":       4,
+			},
+		},
+		{
+			Name:   "authenticated regular record filtering with a hidden field",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo3/records?filter=title~'test'",
+			Headers: map[string]string{
+				// clients, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				col, err := app.FindCollectionByNameOrId("demo3")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// mock hidden field
+				col.Fields.GetByName("title").SetHidden(true)
+
+				if err = app.Save(col); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser filtering with a hidden field",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo3/records?filter=title~'test'",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				col, err := app.FindCollectionByNameOrId("demo3")
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				// mock hidden field
+				col.Fields.GetByName("title").SetHidden(true)
+
+				if err = app.Save(col); err != nil {
+					t.Fatal(err)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":1`,
+				`"totalItems":4`,
+				`"items":[{`,
+				`"id":"1tmknxy2868d869"`,
+				`"id":"lcl9d87w22ml6jy"`,
+				`"id":"7nwo8tuiatetxdm"`,
+				`"id":"mk5fmymtx4wsprk"`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                    0,
+				"OnRecordsListRequest": 1,
+				"OnRecordEnrich":       4,
+			},
+		},
 		{
 			Name:           ":rule modifer",
 			Method:         http.MethodGet,
@@ -279,10 +382,32 @@ func TestRecordCrudList(t *testing.T) {
 			},
 		},
 		{
-			Name:           "multi-match - at least one of",
+			Name:           "multi-match - at least one of (guest - non-satisfied relation filter API rule)",
 			Method:         http.MethodGet,
 			URL:            "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length?=2"),
 			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":0`,
+				`"totalItems":0`,
+				`"items":[]`,
+			},
+			ExpectedEvents: map[string]int{
+				"*":                    0,
+				"OnRecordsListRequest": 1,
+				"OnRecordEnrich":       0,
+			},
+		},
+		{
+			Name:   "multi-match - at least one of (clients)",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length?=2"),
+			Headers: map[string]string{
+				// clients, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
+			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`"page":1`,
 				`"perPage":30`,
@@ -298,9 +423,13 @@ func TestRecordCrudList(t *testing.T) {
 			},
 		},
 		{
-			Name:           "multi-match - all",
-			Method:         http.MethodGet,
-			URL:            "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length=2"),
+			Name:   "multi-match - all (clients)",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length=2"),
+			Headers: map[string]string{
+				// clients, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6ImdrMzkwcWVnczR5NDd3biIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoidjg1MXE0cjc5MHJoa25sIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.0ONnm_BsvPRZyDNT31GN1CKUB6uQRxvVvQ-Wc9AZfG0",
+			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`"page":1`,
@@ -314,6 +443,32 @@ func TestRecordCrudList(t *testing.T) {
 				"OnRecordsListRequest": 1,
 			},
 		},
+		{
+			Name:   "OnRecordsListRequest tx body write check",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo4/records",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordsListRequest().BindFunc(func(e *core.RecordsListRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordsListRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// auth collection
 		// -----------------------------------------------------------
@@ -758,6 +913,32 @@ func TestRecordCrudView(t *testing.T) {
 				"OnRecordEnrich":      7,
 			},
 		},
+		{
+			Name:   "OnRecordViewRequest tx body write check",
+			Method: http.MethodGet,
+			URL:    "/api/collections/demo1/records/al1h9ijdeojtsjy",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnRecordViewRequest().BindFunc(func(e *core.RecordRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnRecordViewRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 
 		// auth collection
 		// -----------------------------------------------------------
@@ -1105,7 +1286,7 @@ func TestRecordCrudDelete(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnRecordAfterDeleteSuccessRequest error response",
+			Name:   "OnRecordDeleteRequest tx body write check",
 			Method: http.MethodDelete,
 			URL:    "/api/collections/clients/records/o1y0dd0spd786md",
 			Headers: map[string]string{
@@ -1113,15 +1294,22 @@ func TestRecordCrudDelete(t *testing.T) {
 			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordDeleteRequest().BindFunc(func(e *core.RecordRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordDeleteRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordDeleteRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 		{
 			Name:   "authenticated record that match the collection delete rule",
@@ -1459,10 +1647,7 @@ func TestRecordCrudCreate(t *testing.T) {
 			Body:            strings.NewReader(`{"title":"test123"}`),
 			ExpectedStatus:  400,
 			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"*": 0},
 		},
 		{
 			Name:   "auth record submit in restricted collection (rule failure check)",
@@ -1475,10 +1660,7 @@ func TestRecordCrudCreate(t *testing.T) {
 			},
 			ExpectedStatus:  400,
 			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"*": 0},
 		},
 		{
 			Name:   "auth record submit in restricted collection (rule pass check) + expand relations",
@@ -1627,10 +1809,7 @@ func TestRecordCrudCreate(t *testing.T) {
 			},
 			ExpectedStatus:  400,
 			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"*": 0},
 		},
 		{
 			Name:   "submit via multipart form data with @jsonPayload key and satisfied @request.body rule",
@@ -1684,29 +1863,44 @@ func TestRecordCrudCreate(t *testing.T) {
 				`"code":"validation_not_unique"`,
 			},
 			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-				// validate events are not fired because the unique check will fail during dry submit
-				// "OnModelValidate":  1,
-				// "OnRecordValidate": 1,
+				"*":                        0,
+				"OnRecordCreateRequest":    1,
+				"OnModelCreate":            1,
+				"OnModelCreateExecute":     1,
+				"OnModelAfterCreateError":  1,
+				"OnModelValidate":          1,
+				"OnRecordCreate":           1,
+				"OnRecordCreateExecute":    1,
+				"OnRecordAfterCreateError": 1,
+				"OnRecordValidate":         1,
 			},
 		},
 		{
-			Name:   "OnRecordAfterCreateSuccessRequest error response",
+			Name:   "OnRecordCreateRequest tx body write check",
 			Method: http.MethodPost,
 			URL:    "/api/collections/demo2/records",
 			Body:   strings.NewReader(`{"title":"new"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordCreateRequest().BindFunc(func(e *core.RecordRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordCreateRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 
 		// ID checks
@@ -1865,14 +2059,9 @@ func TestRecordCrudCreate(t *testing.T) {
 				"total+":4,
 				"total-":2
 			}`),
-			ExpectedStatus: 400,
-			ExpectedContent: []string{
-				`"data":{}`,
-			},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordCreateRequest": 1,
-			},
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
 		},
 		{
 			Name:   "@request.body.field with compute modifers (rule pass check)",
@@ -2345,7 +2534,7 @@ func TestRecordCrudUpdate(t *testing.T) {
 	}
 
 	formData3, mp3, err3 := tests.MockMultipartData(map[string]string{
-		router.JSONPayloadKey: `{"title": "title_test3", "testPayload": 123}`,
+		router.JSONPayloadKey: `{"title": "title_test3", "testPayload": 123, "files":"300_JdfBOieXAW.png"}`,
 	}, "files")
 	if err3 != nil {
 		t.Fatal(err3)
@@ -2686,6 +2875,8 @@ func TestRecordCrudUpdate(t *testing.T) {
 				`"id":"mk5fmymtx4wsprk"`,
 				`"title":"title_test3"`,
 				`"files":["`,
+				`"300_JdfBOieXAW.png"`,
+				`"tmpfile_`,
 			},
 			ExpectedEvents: map[string]int{
 				"*":                          0,
@@ -2702,21 +2893,31 @@ func TestRecordCrudUpdate(t *testing.T) {
 			},
 		},
 		{
-			Name:   "OnRecordAfterUpdateSuccessRequest error response",
+			Name:   "OnRecordUpdateRequest tx body write check",
 			Method: http.MethodPatch,
 			URL:    "/api/collections/demo2/records/0yxhwia2amd8gec",
 			Body:   strings.NewReader(`{"title":"new"}`),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
 			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
 				app.OnRecordUpdateRequest().BindFunc(func(e *core.RecordRequestEvent) error {
-					return errors.New("error")
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
 				})
 			},
 			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-			ExpectedEvents: map[string]int{
-				"*":                     0,
-				"OnRecordUpdateRequest": 1,
-			},
+			ExpectedEvents:  map[string]int{"OnRecordUpdateRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
 		},
 		{
 			Name:   "try to change the id of an existing record",

apis/record_helpers.go

@@ -6,13 +6,16 @@ import (
 	"fmt"
 	"net/http"
 	"strings"
+	"time"
 
 	"github.com/pocketbase/dbx"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/mails"
 	"github.com/pocketbase/pocketbase/tools/router"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/search"
 	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/types"
 )
 
 const (
@@ -20,6 +23,8 @@ const (
 	fieldsQueryParam = "fields"
 )
 
+var ErrMFA = errors.New("mfa required")
+
 // RecordAuthResponse writes standardized json record auth response
 // into the specified request context.
 //
@@ -38,6 +43,13 @@ func RecordAuthResponse(e *core.RequestEvent, authRecord *core.Record, authMetho
 }
 
 func recordAuthResponse(e *core.RequestEvent, authRecord *core.Record, token string, authMethod string, meta any) error {
+	if authRecord.IsSuperuser() {
+		allowedIPs := e.App.Settings().SuperuserIPs
+		if len(allowedIPs) > 0 && !isIPInList(allowedIPs, e.RealIP()) {
+			return e.ForbiddenError("", errors.New("superuser IP is not whitelisted"))
+		}
+	}
+
 	originalRequestInfo, err := e.RequestInfo()
 	if err != nil {
 		return err
@@ -70,9 +82,12 @@ func recordAuthResponse(e *core.RequestEvent, authRecord *core.Record, token str
 
 		// require additional authentication
 		if mfaId != "" {
-			return e.JSON(http.StatusUnauthorized, map[string]string{
+			// eagerly write the mfa response and return an err so that
+			// external middlewars are aware that the auth response requires an extra step
+			e.JSON(http.StatusUnauthorized, map[string]string{
 				"mfaId": mfaId,
 			})
+			return ErrMFA
 		}
 		// ---
 
@@ -109,16 +124,22 @@ func recordAuthResponse(e *core.RequestEvent, authRecord *core.Record, token str
 			}
 		}
 
-		result := map[string]any{
-			"token":  e.Token,
-			"record": e.Record,
+		result := struct {
+			Meta   any          `json:"meta,omitempty"`
+			Record *core.Record `json:"record"`
+			Token  string       `json:"token"`
+		}{
+			Token:  e.Token,
+			Record: e.Record,
 		}
 
 		if e.Meta != nil {
-			result["meta"] = e.Meta
+			result.Meta = e.Meta
 		}
 
-		return e.JSON(http.StatusOK, result)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, result)
+		})
 	})
 }
 
@@ -135,7 +156,7 @@ func wantsMFA(e *core.RequestEvent, record *core.Record) (bool, error) {
 		return true, err
 	}
 
-	var exists bool
+	var exists int
 
 	query := e.App.RecordQuery(record.Collection()).
 		Select("(1)").
@@ -147,14 +168,18 @@ func wantsMFA(e *core.RequestEvent, record *core.Record) (bool, error) {
 	if err != nil {
 		return true, err
 	}
-	resolver.UpdateQuery(query)
+
+	err = resolver.UpdateQuery(query)
+	if err != nil {
+		return true, err
+	}
 
 	err = query.AndWhere(expr).Limit(1).Row(&exists)
 	if err != nil && !errors.Is(err, sql.ErrNoRows) {
 		return true, err
 	}
 
-	return exists, nil
+	return exists > 0, nil
 }
 
 // checkMFA handles any MFA auth checks that needs to be performed for the specified request event.
@@ -366,12 +391,18 @@ func expandFetch(app core.App, originalRequestInfo *core.RequestInfo) core.Expan
 
 			if *relCollection.ViewRule != "" {
 				resolver := core.NewRecordFieldResolver(app, relCollection, requestInfoPtr, true)
+
 				expr, err := search.FilterData(*(relCollection.ViewRule)).BuildExpr(resolver)
 				if err != nil {
 					return err
 				}
-				resolver.UpdateQuery(q)
+
 				q.AndWhere(expr)
+
+				err = resolver.UpdateQuery(q)
+				if err != nil {
+					return err
+				}
 			}
 
 			return nil
@@ -444,18 +475,24 @@ func autoResolveRecordsFlags(app core.App, records []*core.Record, requestInfo *
 	managedIds := []string{}
 
 	query := app.RecordQuery(collection).
-		Select(app.DB().QuoteSimpleColumnName(collection.Name) + ".id").
-		AndWhere(dbx.In(app.DB().QuoteSimpleColumnName(collection.Name)+".id", recordIds...))
+		Select(app.ConcurrentDB().QuoteSimpleColumnName(collection.Name) + ".id").
+		AndWhere(dbx.In(app.ConcurrentDB().QuoteSimpleColumnName(collection.Name)+".id", recordIds...))
 
 	resolver := core.NewRecordFieldResolver(app, collection, requestInfo, true)
 	expr, err := search.FilterData(*collection.ManageRule).BuildExpr(resolver)
 	if err != nil {
 		return err
 	}
-	resolver.UpdateQuery(query)
+
 	query.AndWhere(expr)
 
-	if err := query.Column(&managedIds); err != nil {
+	err = resolver.UpdateQuery(query)
+	if err != nil {
+		return err
+	}
+
+	err = query.Column(&managedIds)
+	if err != nil {
 		return err
 	}
 	// ---
@@ -470,40 +507,6 @@ func autoResolveRecordsFlags(app core.App, records []*core.Record, requestInfo *
 	return nil
 }
 
-// hasAuthManageAccess checks whether the client is allowed to have
-// [forms.RecordUpsert] auth management permissions
-// (e.g. allowing to change system auth fields without oldPassword).
-func hasAuthManageAccess(app core.App, requestInfo *core.RequestInfo, record *core.Record) bool {
-	if !record.Collection().IsAuth() {
-		return false
-	}
-
-	manageRule := record.Collection().ManageRule
-
-	if manageRule == nil || *manageRule == "" {
-		return false // only for superusers (manageRule can't be empty)
-	}
-
-	if requestInfo == nil || requestInfo.Auth == nil {
-		return false // no auth record
-	}
-
-	ruleFunc := func(q *dbx.SelectQuery) error {
-		resolver := core.NewRecordFieldResolver(app, record.Collection(), requestInfo, true)
-		expr, err := search.FilterData(*manageRule).BuildExpr(resolver)
-		if err != nil {
-			return err
-		}
-		resolver.UpdateQuery(q)
-		q.AndWhere(expr)
-		return nil
-	}
-
-	_, findErr := app.FindRecordById(record.Collection().Id, record.Id, ruleFunc)
-
-	return findErr == nil
-}
-
 var ruleQueryParams = []string{search.FilterQueryParam, search.SortQueryParam}
 var superuserOnlyRuleFields = []string{"@collection.", "@request."}
 
@@ -558,18 +561,43 @@ func firstApiError(errs ...error) *router.ApiError {
 	return router.NewInternalServerError("", errors.Join(errs...))
 }
 
+// execAfterSuccessTx ensures that fn is executed only after a successful transaction.
+//
+// If the current app instance is not a transactional or checkTx is false,
+// then fn is directly executed.
+//
+// It could be usually used to allow propagating an error or writing
+// custom response from within the wrapped transaction block.
+func execAfterSuccessTx(checkTx bool, app core.App, fn func() error) error {
+	if txInfo := app.TxInfo(); txInfo != nil && checkTx {
+		txInfo.OnComplete(func(txErr error) error {
+			if txErr == nil {
+				return fn()
+			}
+			return nil
+		})
+		return nil
+	}
+
+	return fn()
+}
+
 // -------------------------------------------------------------------
 
 const maxAuthOrigins = 5
 
 func authAlert(e *core.RequestEvent, authRecord *core.Record) error {
-	// generating fingerprint
+	// generate fingerprint
 	// ---
+	ip := e.RealIP()
+
 	userAgent := e.Request.UserAgent()
-	if len(userAgent) > 300 {
-		userAgent = userAgent[:300]
+	if len(userAgent) > 200 {
+		userAgent = userAgent[:200] + "..."
 	}
-	fingerprint := security.MD5(e.RealIP() + userAgent)
+
+	fingerprint := security.MD5(ip + userAgent)
+	alertInfo := fmt.Sprintf("%s - %s %s", types.NowDateTime().String(), ip, userAgent)
 	// ---
 
 	origins, err := e.App.FindAllAuthOriginsByRecord(authRecord)
@@ -594,8 +622,28 @@ func authAlert(e *core.RequestEvent, authRecord *core.Record) error {
 	}
 
 	// send email alert for the new origin auth (skip first login)
+	//
+	// Note: The "fake" timeout is a temp solution to avoid blocking
+	//       for too long when the SMTP server is not accessible, due
+	//       to the lack of context cancellation support in the underlying
+	//       mailer and net/smtp package.
+	//       The goroutine technically "leaks" but we assume that the OS will
+	//       terminate the connection after some time (usually after 3-4 mins).
 	if !isFirstLogin && currentOrigin.IsNew() && authRecord.Email() != "" {
-		if err := mails.SendRecordAuthAlert(e.App, authRecord); err != nil {
+		mailSent := make(chan error, 1)
+
+		timer := time.AfterFunc(15*time.Second, func() {
+			mailSent <- errors.New("auth alert mail send wait timeout reached")
+		})
+
+		routine.FireAndForget(func() {
+			err := mails.SendRecordAuthAlert(e.App, authRecord, alertInfo)
+			timer.Stop()
+			mailSent <- err
+		})
+
+		err = <-mailSent
+		if err != nil {
 			return err
 		}
 	}

apis/record_helpers_test.go

@@ -1,7 +1,8 @@
 package apis_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
+	"errors"
 	"net/http"
 	"net/http/httptest"
 	"strings"
@@ -584,8 +585,8 @@ func TestRecordAuthResponseMFACheck(t *testing.T) {
 		user.Collection().MFA.Rule = "1=1"
 
 		err = apis.RecordAuthResponse(event, user, "example", nil)
-		if err != nil {
-			t.Fatalf("Expected nil, got error: %v", err)
+		if !errors.Is(err, apis.ErrMFA) {
+			t.Fatalf("Expected ErrMFA, got: %v", err)
 		}
 
 		body := rec.Body.String()
@@ -602,8 +603,8 @@ func TestRecordAuthResponseMFACheck(t *testing.T) {
 		resetMFAs(user)
 
 		err = apis.RecordAuthResponse(event, user, "example", nil)
-		if err != nil {
-			t.Fatalf("Expected nil, got error: %v", err)
+		if !errors.Is(err, apis.ErrMFA) {
+			t.Fatalf("Expected ErrMFA, got: %v", err)
 		}
 
 		body := rec.Body.String()
@@ -758,3 +759,39 @@ func TestRecordAuthResponseMFACheck(t *testing.T) {
 		}
 	})
 }
+
+func TestRecordAuthResponseSuperuserIPsWhitelistCheck(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, "test@example.com")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	app.Settings().TrustedProxy.Headers = []string{"x-test-ip"}
+
+	event := new(core.RequestEvent)
+	event.App = app
+	event.Request = httptest.NewRequest(http.MethodGet, "/", nil)
+	event.Request.Header.Set("x-test-ip", "127.0.0.1")
+	event.Response = httptest.NewRecorder()
+
+	t.Run("non-whitelisted", func(t *testing.T) {
+		app.Settings().SuperuserIPs = []string{"0.0.0.0"}
+
+		err = apis.RecordAuthResponse(event, superuser, "example", nil)
+		if err == nil {
+			t.Fatal("Expected response error, got nil")
+		}
+	})
+
+	t.Run("whitelisted", func(t *testing.T) {
+		app.Settings().SuperuserIPs = []string{"0.0.0.0", "127.0.0.1"}
+
+		err = apis.RecordAuthResponse(event, superuser, "example", nil)
+		if err != nil {
+			t.Fatal(err)
+		}
+	})
+}

apis/serve.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"crypto/tls"
 	"errors"
-	"fmt"
 	"log"
 	"net"
 	"net/http"
@@ -17,11 +16,14 @@ import (
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/list"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/ui"
 	"golang.org/x/crypto/acme"
 	"golang.org/x/crypto/acme/autocert"
 )
 
+const defaultCSP = "default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' http://127.0.0.1:* https://tile.openstreetmap.org data: blob:; connect-src 'self' http://127.0.0.1:* https://nominatim.openstreetmap.org; script-src 'self' http://127.0.0.1:*; frame-ancestors 'none'"
+
 // ServeConfig defines a configuration struct for apis.Serve().
 type ServeConfig struct {
 	// ShowStartBanner indicates whether to show or hide the server start console message.
@@ -72,20 +74,30 @@ func Serve(app core.App, config ServeConfig) error {
 		return err
 	}
 
-	pbRouter.Bind(CORSWithConfig(CORSConfig{
+	pbRouter.Bind(CORS(CORSConfig{
 		AllowOrigins: config.AllowedOrigins,
 		AllowMethods: []string{http.MethodGet, http.MethodHead, http.MethodPut, http.MethodPatch, http.MethodPost, http.MethodDelete},
 	}))
 
-	pbRouter.GET("/_/{path...}", Static(ui.DistDirFS, false)).
-		BindFunc(func(e *core.RequestEvent) error {
-			// ingore root path
-			if e.Request.PathValue(StaticWildcardParam) != "" {
-				e.Response.Header().Set("Cache-Control", "max-age=1209600, stale-while-revalidate=86400")
-			}
-			return e.Next()
-		}).
-		Bind(Gzip())
+	// @todo consider moving in base
+	if ui.DistDirFS != nil {
+		pbRouter.GET("/_/{path...}", Static(ui.DistDirFS, false)).
+			BindFunc(func(e *core.RequestEvent) error {
+				if !e.App.IsDev() &&
+					// exclude root path
+					e.Request.PathValue(StaticWildcardParam) != "" &&
+					e.Response.Header().Get("Cache-Control") == "" {
+					e.Response.Header().Set("Cache-Control", "max-age=1209600, stale-while-revalidate=86400")
+				}
+
+				if e.Response.Header().Get("Content-Security-Policy") == "" {
+					e.Response.Header().Set("Content-Security-Policy", defaultCSP)
+				}
+
+				return e.Next()
+			}).
+			Bind(Gzip())
+	}
 
 	// start http server
 	// ---
@@ -137,9 +149,9 @@ func Serve(app core.App, config ServeConfig) error {
 			NextProtos:     []string{acme.ALPNProto},
 		},
 		// higher defaults to accommodate large file uploads/downloads
-		WriteTimeout:      3 * time.Minute,
-		ReadTimeout:       3 * time.Minute,
-		ReadHeaderTimeout: 30 * time.Second,
+		WriteTimeout:      5 * time.Minute,
+		ReadTimeout:       5 * time.Minute,
+		ReadHeaderTimeout: 1 * time.Minute,
 		Addr:              mainAddr,
 		BaseContext: func(l net.Listener) context.Context {
 			return baseCtx
@@ -147,12 +159,6 @@ func Serve(app core.App, config ServeConfig) error {
 		ErrorLog: log.New(&serverErrorLogWriter{app: app}, "", 0),
 	}
 
-	serveEvent := new(core.ServeEvent)
-	serveEvent.App = app
-	serveEvent.Router = pbRouter
-	serveEvent.Server = server
-	serveEvent.CertManager = certManager
-
 	var listener net.Listener
 
 	// graceful shutdown
@@ -198,6 +204,15 @@ func Serve(app core.App, config ServeConfig) error {
 	}()
 	// ---------------------------------------------------------------
 
+	var baseURL string
+
+	serveEvent := new(core.ServeEvent)
+	serveEvent.App = app
+	serveEvent.Router = pbRouter
+	serveEvent.Server = server
+	serveEvent.CertManager = certManager
+	serveEvent.InstallerFunc = DefaultInstallerFunc
+
 	// trigger the OnServe hook and start the tcp listener
 	serveHookErr := app.OnServe().Trigger(serveEvent, func(e *core.ServeEvent) error {
 		handler, err := e.Router.BuildMux()
@@ -207,10 +222,20 @@ func Serve(app core.App, config ServeConfig) error {
 
 		e.Server.Handler = handler
 
-		addr := e.Server.Addr
+		if config.HttpsAddr == "" {
+			baseURL = "http://" + serverAddrToHost(serveEvent.Server.Addr)
+		} else {
+			baseURL = "https://"
+			if len(config.CertificateDomains) > 0 {
+				baseURL += config.CertificateDomains[0]
+			} else {
+				baseURL += serverAddrToHost(serveEvent.Server.Addr)
+			}
+		}
 
-		// fallback similar to the std Server.ListenAndServe/ListenAndServeTLS
+		addr := e.Server.Addr
 		if addr == "" {
+			// fallback similar to the std Server.ListenAndServe/ListenAndServeTLS
 			if config.HttpsAddr != "" {
 				addr = ":https"
 			} else {
@@ -218,31 +243,36 @@ func Serve(app core.App, config ServeConfig) error {
 			}
 		}
 
-		var lnErr error
+		if e.Listener == nil {
+			listener, err = net.Listen("tcp", addr)
+			if err != nil {
+				return err
+			}
+		} else {
+			listener = e.Listener
+		}
 
-		listener, lnErr = net.Listen("tcp", addr)
+		if e.InstallerFunc != nil {
+			app := e.App
+			installerFunc := e.InstallerFunc
+			routine.FireAndForget(func() {
+				if err := loadInstaller(app, baseURL, installerFunc); err != nil {
+					app.Logger().Warn("Failed to initialize installer", "error", err)
+				}
+			})
+		}
 
-		return lnErr
+		return nil
 	})
 	if serveHookErr != nil {
 		return serveHookErr
 	}
 
 	if listener == nil {
-		return errors.New("The OnServe finalizer wasn't invoked. Did you forget to call the ServeEvent.Next() method?")
+		//nolint:staticcheck
+		return errors.New("The OnServe listener was not initialized. Did you forget to call the ServeEvent.Next() method?")
 	}
 
-	schema := "http"
-	addr := server.Addr
-	if config.HttpsAddr != "" {
-		schema = "https"
-		if len(config.CertificateDomains) > 0 {
-			addr = config.CertificateDomains[0]
-		}
-	}
-	baseURL := fmt.Sprintf("%s://%s", schema, addr)
-	dashboardURL := fmt.Sprintf("%s/_", baseURL)
-
 	if config.ShowStartBanner {
 		date := new(strings.Builder)
 		log.New(date, "", log.LstdFlags).Print()
@@ -255,16 +285,13 @@ func Serve(app core.App, config ServeConfig) error {
 		)
 
 		regular := color.New()
-		regular.Printf("├─ REST API:  %s\n", color.CyanString("%s/api/", baseURL))
-		regular.Printf("└─ Dashboard: %s\n", color.CyanString("%s/", dashboardURL))
-	}
-
-	go func() {
-		installerErr := loadInstaller(app, dashboardURL)
-		if installerErr != nil {
-			app.Logger().Warn("Failed to initialize installer", "error", installerErr)
+		if ui.DistDirFS == nil {
+			regular.Printf("└─ REST API:  %s\n", color.CyanString("%s/api/", baseURL))
+		} else {
+			regular.Printf("├─ REST API:  %s\n", color.CyanString("%s/api/", baseURL))
+			regular.Printf("└─ Dashboard: %s\n", color.CyanString("%s/_/", baseURL))
 		}
-	}()
+	}
 
 	var serveErr error
 	if config.HttpsAddr != "" {
@@ -274,10 +301,10 @@ func Serve(app core.App, config ServeConfig) error {
 		}
 
 		// start HTTPS server
-		serveErr = server.ServeTLS(listener, "", "")
+		serveErr = serveEvent.Server.ServeTLS(listener, "", "")
 	} else {
 		// OR start HTTP server
-		serveErr = server.Serve(listener)
+		serveErr = serveEvent.Server.Serve(listener)
 	}
 	if serveErr != nil && !errors.Is(serveErr, http.ErrServerClosed) {
 		return serveErr
@@ -286,6 +313,14 @@ func Serve(app core.App, config ServeConfig) error {
 	return nil
 }
 
+// serverAddrToHost loosely converts http.Server.Addr string into a host to print.
+func serverAddrToHost(addr string) string {
+	if addr == "" || strings.HasSuffix(addr, ":http") || strings.HasSuffix(addr, ":https") {
+		return "127.0.0.1"
+	}
+	return addr
+}
+
 type serverErrorLogWriter struct {
 	app core.App
 }

apis/settings.go

@@ -3,7 +3,7 @@ package apis
 import (
 	"net/http"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/tools/router"
@@ -16,6 +16,8 @@ func bindSettingsApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
 	subGroup.PATCH("", settingsSet)
 	subGroup.POST("/test/s3", settingsTestS3)
 	subGroup.POST("/test/email", settingsTestEmail)
+
+	// @todo move to collections
 	subGroup.POST("/apple/generate-client-secret", settingsGenerateAppleClientSecret)
 }
 
@@ -30,7 +32,9 @@ func settingsList(e *core.RequestEvent) error {
 	event.Settings = clone
 
 	return e.App.OnSettingsListRequest().Trigger(event, func(e *core.SettingsListRequestEvent) error {
-		return e.JSON(http.StatusOK, e.Settings)
+		return execAfterSuccessTx(true, e.App, func() error {
+			return e.JSON(http.StatusOK, e.Settings)
+		})
 	})
 }
 
@@ -60,12 +64,14 @@ func settingsSet(e *core.RequestEvent) error {
 			return e.BadRequestError("An error occurred while saving the new settings.", err)
 		}
 
-		appSettings, err := e.App.Settings().Clone()
-		if err != nil {
-			return e.InternalServerError("Failed to clone app settings.", err)
-		}
+		return execAfterSuccessTx(true, e.App, func() error {
+			appSettings, err := e.App.Settings().Clone()
+			if err != nil {
+				return e.InternalServerError("Failed to clone app settings.", err)
+			}
 
-		return e.JSON(http.StatusOK, appSettings)
+			return e.JSON(http.StatusOK, appSettings)
+		})
 	})
 }
 

apis/settings_test.go

@@ -11,6 +11,7 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 )
 
@@ -58,6 +59,32 @@ func TestSettingsList(t *testing.T) {
 				"OnSettingsListRequest": 1,
 			},
 		},
+		{
+			Name:   "OnSettingsListRequest tx body write check",
+			Method: http.MethodGet,
+			URL:    "/api/settings",
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnSettingsListRequest().BindFunc(func(e *core.SettingsListRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnSettingsListRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 	}
 
 	for _, scenario := range scenarios {
@@ -70,8 +97,9 @@ func TestSettingsSet(t *testing.T) {
 
 	validData := `{
 		"meta":{"appName":"update_test"},
-		"s3":{"secret": "s3_secret"},
-		"backups":{"s3":{"secret":"backups_s3_secret"}}
+		"smtp":{"password": "new_smtp_password"},
+		"s3":{"secret": "new_s3_secret"},
+		"backups":{"s3":{"secret":"new_backups_s3_secret"}}
 	}`
 
 	scenarios := []tests.ApiScenario{
@@ -152,6 +180,25 @@ func TestSettingsSet(t *testing.T) {
 			Headers: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
 			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				settings := app.Settings()
+
+				// verify that the secret values are persisted
+				secrets := map[string]struct {
+					current  string
+					expected string
+				}{
+					"smtp.password":     {settings.SMTP.Password, "new_smtp_password"},
+					"s3.secret":         {settings.S3.Secret, "new_s3_secret"},
+					"backups.s3.secret": {settings.Backups.S3.Secret, "new_backups_s3_secret"},
+				}
+
+				for name, secret := range secrets {
+					if secret.current != secret.expected {
+						t.Errorf("[%s] expected secret %q, got %q", name, secret.expected, secret.current)
+					}
+				}
+			},
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`"meta":{`,
@@ -176,6 +223,33 @@ func TestSettingsSet(t *testing.T) {
 				"OnSettingsReload":          1,
 			},
 		},
+		{
+			Name:   "OnSettingsUpdateRequest tx body write check",
+			Method: http.MethodPatch,
+			URL:    "/api/settings",
+			Body:   strings.NewReader(validData),
+			Headers: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
+				app.OnSettingsUpdateRequest().BindFunc(func(e *core.SettingsUpdateRequestEvent) error {
+					original := e.App
+					return e.App.RunInTransaction(func(txApp core.App) error {
+						e.App = txApp
+						defer func() { e.App = original }()
+
+						if err := e.Next(); err != nil {
+							return err
+						}
+
+						return e.BadRequestError("TX_ERROR", nil)
+					})
+				})
+			},
+			ExpectedStatus:  400,
+			ExpectedEvents:  map[string]int{"OnSettingsUpdateRequest": 1},
+			ExpectedContent: []string{"TX_ERROR"},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/sql.go

@@ -0,0 +1,194 @@
+package apis
+
+import (
+	"context"
+	"errors"
+	"log/slog"
+	"net/http"
+	"strings"
+	"time"
+
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/router"
+)
+
+const (
+	runSQLMaxRows    = 1000
+	runSQLMaxTimeout = 3 * time.Minute
+)
+
+// bindSQLApi registers the SQL api endpoints.
+func bindSQLApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
+	subGroup := rg.Group("/sql").Bind(RequireSuperuserAuth())
+	subGroup.POST("", runSQL)
+}
+
+func runSQL(e *core.RequestEvent) error {
+	// extra precaution in case manually invoked from somewhere else
+	if !e.HasSuperuserAuth() {
+		return e.ForbiddenError("", nil)
+	}
+
+	form := runSQLForm{}
+
+	err := e.BindBody(&form)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while loading the submitted data.", err))
+	}
+
+	err = form.validate()
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("An error occurred while validating the submitted data.", err))
+	}
+
+	result, err := executeQuery(e.App, form.Query, runSQLMaxRows)
+	if err != nil {
+		return firstApiError(err, e.BadRequestError("Failed to execute query. Raw error:\n"+err.Error(), nil))
+	}
+
+	return e.JSON(http.StatusOK, result)
+}
+
+type runSQLForm struct {
+	Query string `form:"query" json:"query"`
+}
+
+func (form *runSQLForm) validate() error {
+	return validation.ValidateStruct(form,
+		validation.Field(&form.Query, validation.Required, validation.Length(0, 5000)),
+	)
+}
+
+type runSQLResultColumn struct {
+	Name     string `json:"name"`
+	Type     string `json:"type"`
+	Nullable bool   `json:"nullable"`
+}
+
+type runSQLResult struct {
+	ExecTime     int64                `json:"execTime"`
+	AffectedRows int64                `json:"affectedRows"`
+	Columns      []runSQLResultColumn `json:"columns"`
+	Rows         [][]any              `json:"rows"`
+}
+
+var knownWriteQueryPrefixes = []string{
+	"INSERT", "CREATE", "UPDATE", "DELETE",
+	"DROP", "DETACH", "ALTER", "REPLACE",
+}
+
+func executeQuery(app core.App, query string, maxRows int) (*runSQLResult, error) {
+	query = strings.TrimSpace(query)
+	if query == "" {
+		// see https://github.com/mattn/go-sqlite3/issues/950
+		return nil, errors.New("empty query")
+	}
+
+	var isPossibleWriteQuery bool
+
+	// loosely check the query type
+	ucQuery := strings.ToUpper(query)
+	if !strings.HasPrefix(ucQuery, "SELECT") {
+		for _, prefix := range knownWriteQueryPrefixes {
+			if strings.HasPrefix(ucQuery, prefix) {
+				isPossibleWriteQuery = true
+				break
+			}
+		}
+	}
+
+	// note: don't extend the request context to minimize the risk of
+	// causing integrity issues with custom non-transaction mutations
+	ctx, cancelFunc := context.WithTimeout(context.Background(), runSQLMaxTimeout)
+	defer cancelFunc()
+
+	result := &runSQLResult{
+		// init empty slices to ensure "[]" serialization
+		Columns: []runSQLResultColumn{},
+		Rows:    [][]any{},
+	}
+
+	now := time.Now()
+	defer func() {
+		result.ExecTime = time.Since(now).Milliseconds()
+	}()
+
+	// assume write/mutation query
+	// ---------------------------------------------------------------
+	if isPossibleWriteQuery {
+		// auto wrap in transaction in case there are multiple inline queries
+		txErr := app.RunInTransaction(func(txApp core.App) error {
+			execResult, err := txApp.NonconcurrentDB().NewQuery(query).WithContext(ctx).Execute()
+			if err != nil {
+				return err
+			}
+
+			result.AffectedRows, err = execResult.RowsAffected()
+			if err != nil {
+				// non-critical error (e.g. not supported by the driver)
+				txApp.Logger().Debug("Unable to fetch affected rows", slog.String("error", err.Error()))
+			}
+
+			return nil
+		})
+		if txErr != nil {
+			return nil, txErr
+		}
+
+		return result, nil
+	}
+
+	// assume query returning rows
+	// ---------------------------------------------------------------
+	rows, err := app.ConcurrentDB().NewQuery(query).WithContext(ctx).Rows()
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	// populate columns info
+	// ---
+	colTypes, err := rows.ColumnTypes()
+	if err != nil {
+		return nil, err
+	}
+
+	for _, colType := range colTypes {
+		col := runSQLResultColumn{
+			Name: colType.Name(),
+			Type: colType.DatabaseTypeName(),
+		}
+		col.Nullable, _ = colType.Nullable()
+
+		result.Columns = append(result.Columns, col)
+	}
+
+	// populate rows
+	// ---
+	for rows.Next() {
+		if len(result.Rows) >= maxRows {
+			break
+		}
+
+		rowData := make([]any, len(colTypes))
+		for i := 0; i < len(colTypes); i++ {
+			var v *string
+			rowData[i] = &v
+		}
+
+		err := rows.Scan(rowData...)
+		if err != nil {
+			return nil, err
+		}
+
+		result.Rows = append(result.Rows, rowData)
+	}
+
+	err = rows.Err()
+	if err != nil {
+		return nil, err
+	}
+
+	return result, nil
+}

apis/sql_test.go

@@ -0,0 +1,220 @@
+package apis_test
+
+import (
+	"net/http"
+	"strings"
+	"testing"
+
+	"github.com/pocketbase/pocketbase/tests"
+)
+
+func TestSQLRun(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []tests.ApiScenario{
+		{
+			Name:            "guest",
+			Method:          http.MethodPost,
+			URL:             "/api/sql",
+			Body:            strings.NewReader(`{"query":"select 1"}`),
+			ExpectedStatus:  401,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "regular user",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1"}`),
+			Headers: map[string]string{
+				// users, test2@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6Im9hcDY0MGNvdDR5cnUycyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.GfJo6EHIobgas_AXt-M-tj5IoQendPnrkMSe9ExuSEY",
+			},
+			ExpectedStatus:  403,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents:  map[string]int{"*": 0},
+		},
+		{
+			Name:   "superuser",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[{"name":"1","type":"","nullable":true}]`,
+				`"rows":[["1"]]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "empty query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":""}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"query":{`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "invalid query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"invalid"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+				`SQL logic error`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "query with length above the limit",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"` + strings.Repeat("a", 5001) + `"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{`,
+				`"query":{`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "query with length equal to the limit",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select '` + strings.Repeat("a", 4985) + `' as id"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[{"name":"id","type":"","nullable":true}]`,
+				`"rows":[["aaa`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "single write query",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key)"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if !app.HasTable("test_sql_table") {
+					t.Fatalf("Missing expected new %q table", "test_sql_table")
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				`"columns":[]`,
+				`"rows":[]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple write queries",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key);insert into test_sql_table(id)VALUES(1)"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				var total int
+				err := app.DB().NewQuery("select count(*) from test_sql_table").Row(&total)
+				if err != nil {
+					t.Fatal(err)
+				}
+
+				if total != 1 {
+					t.Fatalf("Expected exactly 1 row, found: %d", total)
+				}
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":1`,
+				`"columns":[]`,
+				`"rows":[]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple write queries (transaction rollback)",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"create table test_sql_table(id int primary key);insert into test_sql_table(id)VALUES(1);invalid"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
+				if app.HasTable("test_sql_table") {
+					t.Fatalf("Expected table %q to not be created", "test_sql_table")
+				}
+			},
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+				`Raw error:`,
+				`SQL logic error`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+		{
+			Name:   "multiple read queries",
+			Method: http.MethodPost,
+			URL:    "/api/sql",
+			Body:   strings.NewReader(`{"query":"select 1;select 2"}`),
+			Headers: map[string]string{
+				// superusers, test@example.com
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
+			},
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"execTime":`,
+				`"affectedRows":0`,
+				// only the result of the last query should be returned
+				`"columns":[{"name":"2","type":"","nullable":true}]`,
+				`"rows":[["2"]]`,
+			},
+			ExpectedEvents: map[string]int{"*": 0},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

cmd/superuser.go

@@ -5,7 +5,7 @@ import (
 	"fmt"
 
 	"github.com/fatih/color"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cobra"
@@ -24,6 +24,7 @@ func NewSuperuserCommand(app core.App) *cobra.Command {
 	command.AddCommand(superuserUpdateCommand(app))
 	command.AddCommand(superuserDeleteCommand(app))
 	command.AddCommand(superuserOTPCommand(app))
+	command.AddCommand(superuserIPsCommand(app))
 
 	return command
 }
@@ -36,16 +37,16 @@ func superuserUpsertCommand(app core.App) *cobra.Command {
 		SilenceUsage: true,
 		RunE: func(command *cobra.Command, args []string) error {
 			if len(args) != 2 {
-				return errors.New("Missing email and password arguments.")
+				return errors.New("missing email and password arguments")
 			}
 
 			if args[0] == "" || is.EmailFormat.Validate(args[0]) != nil {
-				return errors.New("Missing or invalid email address.")
+				return errors.New("missing or invalid email address")
 			}
 
 			superusersCol, err := app.FindCachedCollectionByNameOrId(core.CollectionNameSuperusers)
 			if err != nil {
-				return fmt.Errorf("Failed to fetch %q collection: %w.", core.CollectionNameSuperusers, err)
+				return fmt.Errorf("failed to fetch %q collection: %w", core.CollectionNameSuperusers, err)
 			}
 
 			superuser, err := app.FindAuthRecordByEmail(superusersCol, args[0])
@@ -57,7 +58,7 @@ func superuserUpsertCommand(app core.App) *cobra.Command {
 			superuser.SetPassword(args[1])
 
 			if err := app.Save(superuser); err != nil {
-				return fmt.Errorf("Failed to upsert superuser account: %w.", err)
+				return fmt.Errorf("failed to upsert superuser account: %w", err)
 			}
 
 			color.Green("Successfully saved superuser %q!", superuser.Email())
@@ -76,16 +77,16 @@ func superuserCreateCommand(app core.App) *cobra.Command {
 		SilenceUsage: true,
 		RunE: func(command *cobra.Command, args []string) error {
 			if len(args) != 2 {
-				return errors.New("Missing email and password arguments.")
+				return errors.New("missing email and password arguments")
 			}
 
 			if args[0] == "" || is.EmailFormat.Validate(args[0]) != nil {
-				return errors.New("Missing or invalid email address.")
+				return errors.New("missing or invalid email address")
 			}
 
 			superusersCol, err := app.FindCachedCollectionByNameOrId(core.CollectionNameSuperusers)
 			if err != nil {
-				return fmt.Errorf("Failed to fetch %q collection: %w.", core.CollectionNameSuperusers, err)
+				return fmt.Errorf("failed to fetch %q collection: %w", core.CollectionNameSuperusers, err)
 			}
 
 			superuser := core.NewRecord(superusersCol)
@@ -93,7 +94,7 @@ func superuserCreateCommand(app core.App) *cobra.Command {
 			superuser.SetPassword(args[1])
 
 			if err := app.Save(superuser); err != nil {
-				return fmt.Errorf("Failed to create new superuser account: %w.", err)
+				return fmt.Errorf("failed to create new superuser account: %w", err)
 			}
 
 			color.Green("Successfully created new superuser %q!", superuser.Email())
@@ -112,22 +113,22 @@ func superuserUpdateCommand(app core.App) *cobra.Command {
 		SilenceUsage: true,
 		RunE: func(command *cobra.Command, args []string) error {
 			if len(args) != 2 {
-				return errors.New("Missing email and password arguments.")
+				return errors.New("missing email and password arguments")
 			}
 
 			if args[0] == "" || is.EmailFormat.Validate(args[0]) != nil {
-				return errors.New("Missing or invalid email address.")
+				return errors.New("missing or invalid email address")
 			}
 
 			superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, args[0])
 			if err != nil {
-				return fmt.Errorf("Superuser with email %q doesn't exist.", args[0])
+				return fmt.Errorf("superuser with email %q doesn't exist", args[0])
 			}
 
 			superuser.SetPassword(args[1])
 
 			if err := app.Save(superuser); err != nil {
-				return fmt.Errorf("Failed to change superuser %q password: %w.", superuser.Email(), err)
+				return fmt.Errorf("failed to change superuser %q password: %w", superuser.Email(), err)
 			}
 
 			color.Green("Successfully changed superuser %q password!", superuser.Email())
@@ -146,17 +147,17 @@ func superuserDeleteCommand(app core.App) *cobra.Command {
 		SilenceUsage: true,
 		RunE: func(command *cobra.Command, args []string) error {
 			if len(args) == 0 || args[0] == "" || is.EmailFormat.Validate(args[0]) != nil {
-				return errors.New("Invalid or missing email address.")
+				return errors.New("invalid or missing email address")
 			}
 
 			superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, args[0])
 			if err != nil {
-				color.Yellow("Superuser %q is missing or already deleted.", args[0])
+				color.Yellow("superuser %q is missing or already deleted", args[0])
 				return nil
 			}
 
 			if err := app.Delete(superuser); err != nil {
-				return fmt.Errorf("Failed to delete superuser %q: %w.", superuser.Email(), err)
+				return fmt.Errorf("failed to delete superuser %q: %w", superuser.Email(), err)
 			}
 
 			color.Green("Successfully deleted superuser %q!", superuser.Email())
@@ -175,16 +176,16 @@ func superuserOTPCommand(app core.App) *cobra.Command {
 		SilenceUsage: true,
 		RunE: func(command *cobra.Command, args []string) error {
 			if len(args) == 0 || args[0] == "" || is.EmailFormat.Validate(args[0]) != nil {
-				return errors.New("Invalid or missing email address.")
+				return errors.New("invalid or missing email address")
 			}
 
 			superuser, err := app.FindAuthRecordByEmail(core.CollectionNameSuperusers, args[0])
 			if err != nil {
-				return fmt.Errorf("Superuser with email %q doesn't exist.", args[0])
+				return fmt.Errorf("superuser with email %q doesn't exist", args[0])
 			}
 
 			if !superuser.Collection().OTP.Enabled {
-				return errors.New("OTP is not enabled for the _superusers collection.")
+				return errors.New("OTP auth is not enabled for the _superusers collection")
 			}
 
 			pass := security.RandomStringWithAlphabet(superuser.Collection().OTP.Length, "1234567890")
@@ -196,7 +197,7 @@ func superuserOTPCommand(app core.App) *cobra.Command {
 
 			err = app.Save(otp)
 			if err != nil {
-				return fmt.Errorf("Failed to create OTP: %w", err)
+				return fmt.Errorf("failed to create OTP: %w", err)
 			}
 
 			color.New(color.BgGreen, color.FgBlack).Printf("Successfully created OTP for superuser %q:", superuser.Email())
@@ -209,3 +210,38 @@ func superuserOTPCommand(app core.App) *cobra.Command {
 
 	return command
 }
+
+func superuserIPsCommand(app core.App) *cobra.Command {
+	command := &cobra.Command{
+		Use:          "ips",
+		Example:      "superuser ips 127.0.0.1 10.0.0.0/24",
+		Short:        "Updates the superuser IPs whitelist setting (the IPs/subnets arguments must be space separated; leave empty to clear the whitelist restriction)",
+		SilenceUsage: true,
+		RunE: func(command *cobra.Command, args []string) error {
+			settings := app.Settings()
+			settings.SuperuserIPs = args
+
+			if err := app.Save(settings); err != nil {
+				return err
+			}
+
+			if len(args) == 0 {
+				color.Green("Successfully cleared SuperuserIPs setting!")
+			} else {
+				color.New(color.BgGreen, color.FgBlack).Println("Successfully updated SuperuserIPs setting:")
+				superuserIPs := app.Settings().SuperuserIPs
+				for i, ip := range superuserIPs {
+					if i == len(superuserIPs)-1 {
+						color.Green("└─ %s", ip)
+					} else {
+						color.Green("├─ %s", ip)
+					}
+				}
+			}
+
+			return nil
+		},
+	}
+
+	return command
+}

cmd/superuser_test.go

@@ -1,6 +1,7 @@
 package cmd_test
 
 import (
+	"slices"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/cmd"
@@ -401,3 +402,63 @@ func TestSuperuserOTPCommand(t *testing.T) {
 		})
 	}
 }
+
+func TestSuperuserIPsCommand(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	scenarios := []struct {
+		name        string
+		ips         []string
+		expectError bool
+	}{
+		{
+			"no ips",
+			nil,
+			false,
+		},
+		{
+			"invalid ips",
+			[]string{"127.0.0.1", "invalid"},
+			true,
+		},
+		{
+			"valid ips",
+			[]string{"127.0.0.1", "::1", "127.0.0.1/24"},
+			false,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			args := []string{"ips"}
+			args = append(args, s.ips...)
+
+			command := cmd.NewSuperuserCommand(app)
+			command.SetArgs(args)
+
+			err := command.Execute()
+
+			hasErr := err != nil
+			if s.expectError != hasErr {
+				t.Fatalf("Expected hasErr %v, got %v (%v)", s.expectError, hasErr, err)
+			}
+
+			if hasErr {
+				return
+			}
+
+			settingIPs := app.Settings().SuperuserIPs
+
+			if len(settingIPs) != len(s.ips) {
+				t.Fatalf("Expected %d ips, got %d (%v)", len(s.ips), len(settingIPs), settingIPs)
+			}
+
+			for _, ip := range settingIPs {
+				if !slices.Contains(s.ips, ip) {
+					t.Fatalf("Missing expected ip %q (%v)", ip, settingIPs)
+				}
+			}
+		})
+	}
+}

core/app.go

@@ -45,14 +45,25 @@ type App interface {
 	// IsTransactional checks if the current app instance is part of a transaction.
 	IsTransactional() bool
 
+	// TxInfo returns the transaction associated with the current app instance (if any).
+	//
+	// Could be used if you want to execute indirectly a function after
+	// the related app transaction completes using `app.TxInfo().OnAfterFunc(callback)`.
+	TxInfo() *TxAppInfo
+
 	// Bootstrap initializes the application
 	// (aka. create data dir, open db connections, load settings, etc.).
 	//
-	// It will call ResetBootstrapState() if the application was already bootstrapped.
+	// It calls ClearBootstrap() if the application was already bootstrapped.
 	Bootstrap() error
 
-	// ResetBootstrapState releases the initialized core app resources
+	// ClearBootstrap releases the initialized core app resources
 	// (closing db connections, stopping cron ticker, etc.).
+	//
+	// This method is no-op if the application is not bootstrapped yet.
+	ClearBootstrap() error
+
+	// Deprecated: use ClearBootstrap().
 	ResetBootstrapState() error
 
 	// DataDir returns the app data directory path.
@@ -71,7 +82,7 @@ type App interface {
 	Settings() *Settings
 
 	// Store returns the app runtime store.
-	Store() *store.Store[any]
+	Store() *store.Store[string, any]
 
 	// Cron returns the app cron instance.
 	Cron() *cron.Cron
@@ -91,7 +102,7 @@ type App interface {
 	// after you are done working with it.
 	NewFilesystem() (*filesystem.System, error)
 
-	// NewFilesystem creates a new local or S3 filesystem instance
+	// NewBackupsFilesystem creates a new local or S3 filesystem instance
 	// for managing app backups based on the current app settings.
 	//
 	// NB! Make sure to call Close() on the returned result
@@ -140,46 +151,82 @@ type App interface {
 	// DB methods
 	// ---------------------------------------------------------------
 
-	// DB returns the default app data db instance (pb_data/data.db).
+	// DB returns the default app data.db builder instance.
+	//
+	// To minimize SQLITE_BUSY errors, it automatically routes the
+	// SELECT queries to the underlying concurrent db pool and everything else
+	// to the nonconcurrent one.
+	//
+	// For more finer control over the used connections pools you can
+	// call directly ConcurrentDB() or NonconcurrentDB().
 	DB() dbx.Builder
 
-	// NonconcurrentDB returns the nonconcurrent app data db instance (pb_data/data.db).
+	// ConcurrentDB returns the concurrent app data.db builder instance.
+	//
+	// This method is used mainly internally for executing db read
+	// operations in a concurrent/non-blocking manner.
+	//
+	// Most users should use simply DB() as it will automatically
+	// route the query execution to ConcurrentDB() or NonconcurrentDB().
+	//
+	// In a transaction the ConcurrentDB() and NonconcurrentDB() refer to the same *dbx.TX instance.
+	ConcurrentDB() dbx.Builder
+
+	// NonconcurrentDB returns the nonconcurrent app data.db builder instance.
 	//
 	// The returned db instance is limited only to a single open connection,
-	// meaning that it can process only 1 db operation at a time (other operations will be queued up).
+	// meaning that it can process only 1 db operation at a time (other queries queue up).
 	//
 	// This method is used mainly internally and in the tests to execute write
 	// (save/delete) db operations as it helps with minimizing the SQLITE_BUSY errors.
 	//
-	// For the majority of cases you would want to use the regular DB() method
-	// since it allows concurrent db read operations.
+	// Most users should use simply DB() as it will automatically
+	// route the query execution to ConcurrentDB() or NonconcurrentDB().
 	//
 	// In a transaction the ConcurrentDB() and NonconcurrentDB() refer to the same *dbx.TX instance.
 	NonconcurrentDB() dbx.Builder
 
-	// AuxDB returns the default app auxiliary db instance (pb_data/auxiliary.db).
+	// AuxDB returns the app auxiliary.db builder instance.
+	//
+	// To minimize SQLITE_BUSY errors, it automatically routes the
+	// SELECT queries to the underlying concurrent db pool and everything else
+	// to the nonconcurrent one.
+	//
+	// For more finer control over the used connections pools you can
+	// call directly AuxConcurrentDB() or AuxNonconcurrentDB().
 	AuxDB() dbx.Builder
 
-	// AuxNonconcurrentDB returns the nonconcurrent app auxiliary db instance (pb_data/auxiliary.db)..
+	// AuxConcurrentDB returns the concurrent app auxiliary.db builder instance.
+	//
+	// This method is used mainly internally for executing db read
+	// operations in a concurrent/non-blocking manner.
+	//
+	// Most users should use simply AuxDB() as it will automatically
+	// route the query execution to AuxConcurrentDB() or AuxNonconcurrentDB().
+	//
+	// In a transaction the AuxConcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
+	AuxConcurrentDB() dbx.Builder
+
+	// AuxNonconcurrentDB returns the nonconcurrent app auxiliary.db builder instance.
 	//
 	// The returned db instance is limited only to a single open connection,
-	// meaning that it can process only 1 db operation at a time (other operations will be queued up).
+	// meaning that it can process only 1 db operation at a time (other queries queue up).
 	//
 	// This method is used mainly internally and in the tests to execute write
 	// (save/delete) db operations as it helps with minimizing the SQLITE_BUSY errors.
 	//
-	// For the majority of cases you would want to use the regular DB() method
-	// since it allows concurrent db read operations.
+	// Most users should use simply AuxDB() as it will automatically
+	// route the query execution to AuxConcurrentDB() or AuxNonconcurrentDB().
 	//
-	// In a transaction the AuxNonconcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
+	// In a transaction the AuxConcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
 	AuxNonconcurrentDB() dbx.Builder
 
 	// HasTable checks if a table (or view) with the provided name exists (case insensitive).
-	// in the current app.DB() instance.
+	// in the data.db.
 	HasTable(tableName string) bool
 
 	// AuxHasTable checks if a table (or view) with the provided name exists (case insensitive)
-	// in the current app.AuxDB() instance.
+	// in the auxiliary.db.
 	AuxHasTable(tableName string) bool
 
 	// TableColumns returns all column names of a single table by its name.
@@ -198,48 +245,58 @@ type App interface {
 	// This method is a no-op if a table with the provided name doesn't exist.
 	//
 	// NB! Be aware that this method is vulnerable to SQL injection and the
-	// "tableName" argument must come only from trusted input!
-	DeleteTable(tableName string) error
+	// "dangerousTableName" argument must come only from trusted input!
+	DeleteTable(dangerousTableName string) error
 
 	// DeleteView drops the specified view name.
 	//
 	// This method is a no-op if a view with the provided name doesn't exist.
 	//
 	// NB! Be aware that this method is vulnerable to SQL injection and the
-	// "name" argument must come only from trusted input!
-	DeleteView(name string) error
+	// "dangerousViewName" argument must come only from trusted input!
+	DeleteView(dangerousViewName string) error
 
 	// SaveView creates (or updates already existing) persistent SQL view.
 	//
-	// NB! Be aware that this method is vulnerable to SQL injection and the
-	// "selectQuery" argument must come only from trusted input!
-	SaveView(name string, selectQuery string) error
+	// NB! Be aware that this method is vulnerable to SQL injection and
+	// its arguments must come only from trusted input!
+	SaveView(dangerousViewName string, dangerousSelectQuery string) error
 
 	// CreateViewFields creates a new FieldsList from the provided select query.
 	//
 	// There are some caveats:
 	// - The select query must have an "id" column.
 	// - Wildcard ("*") columns are not supported to avoid accidentally leaking sensitive data.
-	CreateViewFields(selectQuery string) (FieldsList, error)
+	//
+	// NB! Be aware that this method is vulnerable to SQL injection and the
+	// "dangerousSelectQuery" argument must come only from trusted input!
+	CreateViewFields(dangerousSelectQuery string) (FieldsList, error)
+
+	// DryRunView executes the provided query by creating a temporary view
+	// collection and returning a sample of the resulting query records (if valid).
+	//
+	// The same caveats from CreateViewFields apply here too.
+	//
+	// NB! Be aware that this method is vulnerable to SQL injection and the
+	// "dangerousSelectQuery" argument must come only from trusted input!
+	DryRunView(dangerousSelectQuery string, sampleSize int) (*DryRunViewResult, error)
 
 	// FindRecordByViewFile returns the original Record of the provided view collection file.
 	FindRecordByViewFile(viewCollectionModelOrIdentifier any, fileFieldName string, filename string) (*Record, error)
 
-	// Vacuum executes VACUUM on the current app.DB() instance
-	// in order to reclaim unused data db disk space.
+	// Vacuum executes VACUUM on the data.db in order to reclaim unused data db disk space.
 	Vacuum() error
 
-	// AuxVacuum executes VACUUM on the current app.AuxDB() instance
-	// in order to reclaim unused auxiliary db disk space.
+	// AuxVacuum executes VACUUM on the auxiliary.db in order to reclaim unused auxiliary db disk space.
 	AuxVacuum() error
 
 	// ---------------------------------------------------------------
 
-	// ModelQuery creates a new preconfigured select app.DB() query with preset
+	// ModelQuery creates a new preconfigured select data.db query with preset
 	// SELECT, FROM and other common fields based on the provided model.
 	ModelQuery(model Model) *dbx.SelectQuery
 
-	// AuxModelQuery creates a new preconfigured select app.AuxDB() query with preset
+	// AuxModelQuery creates a new preconfigured select auxiliary.db query with preset
 	// SELECT, FROM and other common fields based on the provided model.
 	AuxModelQuery(model Model) *dbx.SelectQuery
 
@@ -323,10 +380,10 @@ type App interface {
 	// FindLogById finds a single Log entry by its id.
 	FindLogById(id string) (*Log, error)
 
-	// LogsStatsItem defines the total number of logs for a specific time period.
+	// LogsStatsItem returns hourly grouped logs statistics.
 	LogsStats(expr dbx.Expression) ([]*LogsStatsItem, error)
 
-	// DeleteOldLogs delete all requests that are created before createdBefore.
+	// DeleteOldLogs delete all logs that are created before createdBefore.
 	DeleteOldLogs(createdBefore time.Time) error
 
 	// ---------------------------------------------------------------
@@ -369,16 +426,9 @@ type App interface {
 	//   - If you are updating a Collection in a transaction and then call this method before commit,
 	//     it'll return the cached Collection state and not the one from the uncommitted transaction.
 	//   - The cache is automatically updated on collections db change (create/update/delete).
-	//     To manually reload the cache you can call [App.ReloadCachedCollections()]
+	//     To manually reload the cache you can call [App.ReloadCachedCollections]
 	FindCachedCollectionByNameOrId(nameOrId string) (*Collection, error)
 
-	// IsCollectionNameUnique checks that there is no existing collection
-	// with the provided name (case insensitive!).
-	//
-	// Note: case insensitive check because the name is used also as
-	// table name for the records.
-	IsCollectionNameUnique(name string, excludeIds ...string) bool
-
 	// FindCollectionReferences returns information for all relation
 	// fields referencing the provided collection.
 	//
@@ -387,6 +437,32 @@ type App interface {
 	// as the excludeIds argument.
 	FindCollectionReferences(collection *Collection, excludeIds ...string) (map[*Collection][]Field, error)
 
+	// FindCachedCollectionReferences is similar to [App.FindCollectionReferences]
+	// but retrieves the Collection from the app cache instead of making a db call.
+	//
+	// NB! This method is suitable for read-only Collection operations.
+	//
+	// If you plan making changes to the returned Collection model,
+	// use [App.FindCollectionReferences] instead.
+	//
+	// Caveats:
+	//
+	//   - The returned Collection should be used only for read-only operations.
+	//     Avoid directly modifying the returned cached Collection as it will affect
+	//     the global cached value even if you don't persist the changes in the database!
+	//   - If you are updating a Collection in a transaction and then call this method before commit,
+	//     it'll return the cached Collection state and not the one from the uncommitted transaction.
+	//   - The cache is automatically updated on collections db change (create/update/delete).
+	//     To manually reload the cache you can call [App.ReloadCachedCollections].
+	FindCachedCollectionReferences(collection *Collection, excludeIds ...string) (map[*Collection][]Field, error)
+
+	// IsCollectionNameUnique checks that there is no existing collection
+	// with the provided name (case insensitive!).
+	//
+	// Note: case insensitive check because the name is used also as
+	// table name for the records.
+	IsCollectionNameUnique(name string, excludeIds ...string) bool
+
 	// TruncateCollection deletes all records associated with the provided collection.
 	//
 	// The truncate operation is executed in a single transaction,
@@ -431,6 +507,11 @@ type App interface {
 	// ExternalAuth model that satisfies the non-nil expression.
 	FindFirstExternalAuthByExpr(expr dbx.Expression) (*ExternalAuth, error)
 
+	// DeleteAllExternalAuthsByRecord deletes all ExternalAuth models associated with the provided record.
+	//
+	// Returns a combined error with the failed deletes.
+	DeleteAllExternalAuthsByRecord(authRecord *Record) error
+
 	// ---------------------------------------------------------------
 
 	// FindAllMFAsByRecord returns all MFA models linked to the provided auth record.
@@ -637,6 +718,13 @@ type App interface {
 	// resources (db, app settings, etc).
 	OnBootstrap() *hook.Hook[*BootstrapEvent]
 
+	// OnBootstrapClear hook is triggered when clearing the main application
+	// resources (db connections, cron, logger, etc.)
+	//
+	// It is usually invoked automatically right before app termination
+	// or when manually calling app.ClearBootstrap().
+	OnBootstrapClear() *hook.Hook[*BootstrapEvent]
+
 	// OnServe hook is triggered when the app web server is started
 	// (after starting the TCP listener but before initializing the blocking serve task),
 	// allowing you to adjust its options and attach new routes or middlewares.
@@ -1136,7 +1224,7 @@ type App interface {
 	// ---------------------------------------------------------------
 
 	// OnMailerSend hook is triggered every time when a new email is
-	// being send using the [App.NewMailClient()] instance.
+	// being sent using the [App.NewMailClient()] instance.
 	//
 	// It allows intercepting the email message or to use a custom mailer client.
 	OnMailerSend() *hook.Hook[*MailerEvent]
@@ -1186,6 +1274,20 @@ type App interface {
 	// triggered and called only if their event data origin matches the tags.
 	OnMailerRecordOTPSend(tags ...string) *hook.TaggedHook[*MailerRecordEvent]
 
+	// ---------------------------------------------------------------
+	// Filesystem event hooks
+	// (not publicly exposed until file_field refactoring)
+	// ---------------------------------------------------------------
+
+	// onFilesystemNewWriter is an internal hook for app.NewFilesystem()
+	// instances that is triggered on every storage filesystem writer initialization
+	// (aka. whenever attempting to create a new file).
+	onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent]
+
+	// onFilesystemDelete is an internal hook for app.NewFilesystem()
+	// instances that is triggered for every storage file delete call.
+	onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent]
+
 	// ---------------------------------------------------------------
 	// Realtime API event hooks
 	// ---------------------------------------------------------------

core/auth_origin_model.go

@@ -5,8 +5,8 @@ import (
 	"errors"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/dbx"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"
 )

core/backup.go

@@ -0,0 +1,137 @@
+package core
+
+import (
+	"context"
+	"fmt"
+	"log/slog"
+	"sort"
+	"time"
+
+	"github.com/pocketbase/pocketbase/tools/inflector"
+)
+
+const (
+	StoreKeyActiveBackup = "@activeBackup"
+)
+
+// generateBackupName generates a new backup name based on the app name and current date.
+func generateBackupName(app App, prefix string) string {
+	appName := inflector.Snakecase(app.Settings().Meta.AppName)
+	if len(appName) > 50 {
+		appName = appName[:50]
+	}
+
+	return fmt.Sprintf(
+		"%s%s_%s.zip",
+		prefix,
+		appName,
+		time.Now().UTC().Format("20060102150405"),
+	)
+}
+
+// registerAutobackupHooks registers the autobackup app serve hooks.
+func (app *BaseApp) registerAutobackupHooks() {
+	const jobId = "__pbAutoBackup__"
+
+	loadJob := func() {
+		rawSchedule := app.Settings().Backups.Cron
+		if rawSchedule == "" {
+			app.Cron().Remove(jobId)
+			return
+		}
+
+		app.Cron().Add(jobId, rawSchedule, func() {
+			const autoPrefix = "@auto_pb_backup_"
+
+			name := generateBackupName(app, autoPrefix)
+
+			if err := app.CreateBackup(context.Background(), name); err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to create backup",
+					slog.String("name", name),
+					slog.String("error", err.Error()),
+				)
+
+				alertError := sendSystemAlertToAllSuperusers(
+					app,
+					"Autobackup failure",
+					"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
+				)
+				if alertError != nil {
+					app.Logger().Warn(
+						"[Backup cron] Failed to send backup error alerts",
+						slog.String("name", name),
+						slog.String("error", alertError.Error()),
+					)
+				}
+			}
+
+			maxKeep := app.Settings().Backups.CronMaxKeep
+
+			if maxKeep == 0 {
+				return // no explicit limit
+			}
+
+			fsys, err := app.NewBackupsFilesystem()
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to initialize the backup filesystem",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+			defer fsys.Close()
+
+			files, err := fsys.List(autoPrefix)
+			if err != nil {
+				app.Logger().Error(
+					"[Backup cron] Failed to list autogenerated backups",
+					slog.String("error", err.Error()),
+				)
+				return
+			}
+
+			if maxKeep >= len(files) {
+				return // nothing to remove
+			}
+
+			// sort desc
+			sort.Slice(files, func(i, j int) bool {
+				return files[i].ModTime.After(files[j].ModTime)
+			})
+
+			// keep only the most recent n auto backup files
+			toRemove := files[maxKeep:]
+
+			for _, f := range toRemove {
+				if err := fsys.Delete(f.Key); err != nil {
+					app.Logger().Error(
+						"[Backup cron] Failed to remove old autogenerated backup",
+						slog.String("key", f.Key),
+						slog.String("error", err.Error()),
+					)
+				}
+			}
+		})
+	}
+
+	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+
+	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
+		if err := e.Next(); err != nil {
+			return err
+		}
+
+		loadJob()
+
+		return nil
+	})
+}

core/backup_create.go

@@ -0,0 +1,437 @@
+package core
+
+import (
+	"archive/zip"
+	"compress/flate"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"log/slog"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
+	"github.com/pocketbase/pocketbase/tools/hook"
+	"github.com/pocketbase/pocketbase/tools/security"
+	"github.com/pocketbase/pocketbase/tools/store"
+)
+
+var errIsDir = errors.New("the specified path is a directory and not a regular file")
+
+// CreateBackup creates a new backup of the current app pb_data directory.
+//
+// If name is empty, it will be autogenerated.
+// If backup with the same name exists, the new backup file will replace it.
+//
+// To safely perform the backup, it is recommended to have free disk space
+// for at least 2x the size of the pb_data directory.
+//
+// By default backups are stored in pb_data/backups
+// (the backups directory itself is excluded from the generated backup).
+//
+// Backups can be stored on S3 if it is configured in app.Settings().Backups.
+// When using S3 storage for the uploaded collection files, you have to
+// take care manually to backup those since they are not part of the pb_data.
+//
+// DB write locks are minimal and isolated only for the duration of the
+// VACUUM INTO statement that creates a live copy of the app database.
+//
+// The backup works as follow:
+//
+//  1. Start listening for DELETED storage files.
+//     In case a file is being deleted while the backup is still ongoing,
+//     we directly copy it in the zip before the deletion and mark it as "excluded".
+//
+//  2. Copy the main database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  3. Stop listening for DELETED files.
+//
+//  4. Start listening for NEW storage files and mark all new files from this point as "excluded".
+//
+//  5. Copy the logs database with VACUUM INTO, write it in the zip and mark it as "excluded".
+//
+//  6. Copy the rest of the pb_data files in the zip while ignoring the "excluded" list (it should be concurrent safe).
+//
+//  7. Stop listening for NEW storage files.
+//
+// While there is a risk for a race condition between steps 1, 2 and 3, it is an
+// acceptable trade-off between performance and correctness because in
+// the worst case there will be some unused storage files in the backup that don't do any harm.
+func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
+	if app.Store().Has(StoreKeyActiveBackup) {
+		return errors.New("try again later - another backup/restore operation has already been started")
+	}
+
+	app.Store().Set(StoreKeyActiveBackup, name)
+	defer app.Store().Remove(StoreKeyActiveBackup)
+
+	startTime := time.Now()
+
+	event := new(BackupEvent)
+	event.App = app
+	event.Context = ctx
+	event.Name = name
+	// default root dir entries to exclude from the backup generation
+	event.Exclude = []string{
+		LocalBackupsDirName,
+		LocalTempDirName,
+		LocalNotifyDirName,
+		LocalAutocertCacheDirName,
+		lostFoundDirName,
+	}
+
+	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
+		if e.Name == "" {
+			e.Name = generateBackupName(e.App, "pb_backup_")
+		}
+
+		app.Logger().Debug("[" + e.Name + "] zip archive started")
+
+		// create backup zip
+		// (it needs to be inside the current pb_data to avoid "cross-device link" errors)
+		// -----------------------------------------------------------
+		tempZipPath := filepath.Join(app.DataDir(), LocalTempDirName, "pb_backup_"+security.PseudorandomString(6))
+		err := createZip(e, tempZipPath)
+		if err != nil {
+			return err
+		}
+		defer os.Remove(tempZipPath)
+
+		// persist the backup in the backups filesystem
+		// -----------------------------------------------------------
+		fsys, err := e.App.NewBackupsFilesystem()
+		if err != nil {
+			return err
+		}
+		defer fsys.Close()
+
+		fsys.SetContext(e.Context)
+
+		file, err := filesystem.NewFileFromPath(tempZipPath)
+		if err != nil {
+			return err
+		}
+		file.OriginalName = e.Name
+		file.Name = file.OriginalName
+
+		err = fsys.UploadFile(file, file.Name)
+		if err != nil {
+			return err
+		}
+
+		app.Logger().Debug(
+			"["+e.Name+"] zip archive completed",
+			slog.Float64("execTime", float64(time.Since(startTime))/float64(time.Millisecond)),
+		)
+
+		return nil
+	})
+}
+
+func createZip(be *BackupEvent, tempZipPath string) error {
+	logPrefix := "[" + be.Name + "] "
+
+	// make sure that the special temp directory exists
+	localTempDir := filepath.Dir(tempZipPath)
+	if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
+		return fmt.Errorf(logPrefix+"failed to create temp dir: %w", err)
+	}
+
+	const tempFilesHookId = "__pbTempBackupFilesystemWatcher__"
+	defer func() {
+		// unbind again in cacase of an error
+		be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+		be.App.onFilesystemNewWriter().Unbind(tempFilesHookId)
+	}()
+
+	zipper, err := newZipWriter(tempZipPath)
+	if err != nil {
+		return err
+	}
+	defer zipper.close()
+
+	excluded := store.New[string, struct{}](nil)
+	for _, name := range be.Exclude {
+		excluded.Set(normalizePathExclude(name), struct{}{})
+	}
+
+	// init deleted files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemDelete().Bind(&hook.Handler[*FilesystemDeleteEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemDeleteEvent) error {
+			// note: the zip header name allow only forward slashes
+			zipPath := path.Join(LocalStorageDirName, e.FileKey)
+
+			if excluded.Has(normalizePathExclude(zipPath)) || be.App.Settings().S3.Enabled {
+				return e.Next()
+			}
+
+			localPath := filepath.Join(
+				be.App.DataDir(),
+				LocalStorageDirName,
+				e.FileKey,
+			)
+
+			// copy to zip before delete
+			err := zipper.copyFileToZip(localPath, zipPath)
+			if err != nil {
+				// it is ok to ignore directories
+				if !errors.Is(err, errIsDir) {
+					be.App.Logger().Warn(
+						logPrefix+"failed to copy file in backup zip before delete",
+						slog.Any("error", err),
+						slog.String("file", e.FileKey),
+					)
+				}
+			} else {
+				// mark that it was already copied
+				excluded.Set(normalizePathExclude(zipPath), struct{}{})
+			}
+
+			// proceed with the normal deletion
+			return e.Next()
+		},
+	})
+
+	// copy data.db
+	// ---------------------------------------------------------------
+	dataStartTime := time.Now()
+	tempDataDBPath := filepath.Join(localTempDir, dataDBFilename)
+
+	_, err = be.App.ConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempDataDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	// eagerly stop listening for deleted files since we already have what we needed
+	be.App.onFilesystemDelete().Unbind(tempFilesHookId)
+
+	be.App.Logger().Debug(
+		logPrefix+dataDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(dataStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempDataDBPath, dataDBFilename)
+	if err != nil {
+		_ = os.Remove(tempDataDBPath)
+		return err
+	}
+	_ = os.Remove(tempDataDBPath)
+
+	excluded.Set(normalizePathExclude(dataDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(dataDBFilename+"-shm"), struct{}{})
+
+	// init to-be-created files tracker
+	// ---------------------------------------------------------------
+	be.App.onFilesystemNewWriter().Bind(&hook.Handler[*FilesystemNewWriterEvent]{
+		Id:       tempFilesHookId,
+		Priority: -99,
+		Func: func(e *FilesystemNewWriterEvent) error {
+			if !be.App.Settings().S3.Enabled {
+				// mark for exclude even if the writer eventually fails
+				// (all record files have random name so collisions are unlikely)
+				name := normalizePathExclude(filepath.Join(LocalStorageDirName, e.FileKey))
+				excluded.Set(name, struct{}{})
+			}
+
+			return e.Next()
+		},
+	})
+
+	// copy auxiliary.db
+	// ---------------------------------------------------------------
+	auxStartTime := time.Now()
+	tempAuxDBPath := filepath.Join(localTempDir, auxDBFilename)
+
+	_, err = be.App.AuxConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempAuxDBPath}).Execute()
+	if err != nil {
+		return err
+	}
+
+	be.App.Logger().Debug(
+		logPrefix+auxDBFilename+" copy completed",
+		slog.Float64("execTime", float64(time.Since(auxStartTime))/float64(time.Millisecond)),
+	)
+
+	err = zipper.copyFileToZip(tempAuxDBPath, auxDBFilename)
+	if err != nil {
+		_ = os.Remove(tempAuxDBPath)
+		return err
+	}
+	_ = os.Remove(tempAuxDBPath)
+
+	excluded.Set(normalizePathExclude(auxDBFilename), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-wal"), struct{}{})
+	excluded.Set(normalizePathExclude(auxDBFilename+"-shm"), struct{}{})
+
+	// try to run manual checkpoints to ensure that all wal writes during the
+	// previous VACUUM INTO are transferred and don't accumulate
+	// (errors are ignore because some drivers may not support the wal_checkpoint pragma)
+	// ---------------------------------------------------------------
+	_, _ = be.App.NonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+	_, _ = be.App.AuxNonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
+
+	// copy the rest of the pb_data
+	// ---------------------------------------------------------------
+	err = zipper.copyDirToZip(os.DirFS(be.App.DataDir()), excluded)
+	if err != nil {
+		return err
+	}
+
+	return zipper.close()
+}
+
+// normalize the provided file path to always end with forward slash
+func normalizePathExclude(filePath string) string {
+	return path.Clean(filePath) + "/"
+}
+
+type zipWriter struct {
+	mu     sync.Mutex
+	w      *zip.Writer
+	f      *os.File
+	closed bool
+}
+
+func newZipWriter(zipFilePath string) (*zipWriter, error) {
+	f, err := os.Create(zipFilePath)
+	if err != nil {
+		return nil, err
+	}
+
+	w := zip.NewWriter(f)
+	w.RegisterCompressor(zip.Deflate, func(out io.Writer) (io.WriteCloser, error) {
+		return flate.NewWriter(out, flate.BestSpeed)
+	})
+
+	return &zipWriter{
+		w: w,
+		f: f,
+	}, nil
+}
+
+func (z *zipWriter) close() error {
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return nil
+	}
+
+	z.closed = true
+
+	return errors.Join(z.w.Close(), z.f.Close())
+}
+
+func (z *zipWriter) copyFileToZip(localPath string, zipPath string) error {
+	info, err := os.Stat(localPath)
+	if err != nil {
+		return err
+	}
+
+	if info.IsDir() {
+		return errIsDir
+	}
+
+	h, err := zip.FileInfoHeader(info)
+	if err != nil {
+		return err
+	}
+
+	h.Name = zipPath
+	h.Method = zip.Deflate
+
+	z.mu.Lock()
+	defer z.mu.Unlock()
+
+	if z.closed {
+		return errors.New("zip writer is already closed")
+	}
+
+	fw, err := z.w.CreateHeader(h)
+	if err != nil {
+		return err
+	}
+
+	f, err := os.Open(localPath)
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+
+	_, err = io.Copy(fw, f)
+
+	return err
+}
+
+func (z *zipWriter) copyDirToZip(fsys fs.FS, excludedPrefixes *store.Store[string, struct{}]) error {
+	return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+
+		// skip excluded prefixes
+		if excludedPrefixes != nil {
+			check := normalizePathExclude(name)
+			prefixes := excludedPrefixes.Keys() // refetch to avoid races
+			for _, prefix := range prefixes {
+				if strings.HasPrefix(check, prefix) {
+					if d.IsDir() {
+						return fs.SkipDir
+					}
+					return nil
+				}
+			}
+		}
+
+		if d.IsDir() {
+			return nil
+		}
+
+		info, err := d.Info()
+		if err != nil {
+			return err
+		}
+
+		h, err := zip.FileInfoHeader(info)
+		if err != nil {
+			return err
+		}
+
+		h.Name = name
+		h.Method = zip.Deflate
+
+		z.mu.Lock()
+		defer z.mu.Unlock()
+
+		if z.closed {
+			// note: fs.WalkDir perform direct comparison with the value
+			return fs.SkipAll
+		}
+
+		fw, err := z.w.CreateHeader(h)
+		if err != nil {
+			return err
+		}
+
+		f, err := fsys.Open(name)
+		if err != nil {
+			return err
+		}
+		defer f.Close()
+
+		_, err = io.Copy(fw, f)
+
+		return err
+	})
+}

core/backup_restore.go

@@ -0,0 +1,196 @@
+package core
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"runtime"
+
+	"github.com/pocketbase/pocketbase/tools/archive"
+	"github.com/pocketbase/pocketbase/tools/osutils"
+	"github.com/pocketbase/pocketbase/tools/security"
+)
+
+// RestoreBackup restores the backup with the specified name and restarts
+// the current running application process.
+//
+// NB! This feature is experimental and currently is expected to work only on UNIX based systems.
+//
+// To safely perform the restore it is recommended to have free disk space
+// for at least 2x the size of the restored pb_data backup.
+//
+// The performed steps are:
+//
+//  1. Download the backup with the specified name in a temp location
+//     (this is in case of S3; otherwise it creates a temp copy of the zip)
+//
+//  2. Extract the backup in a temp directory inside the app "pb_data"
+//     (eg. "pb_data/.pb_temp_to_delete/pb_restore").
+//
+//  3. Move the current app "pb_data" content (excluding the local backups and the special temp dir)
+//     under another temp sub dir that will be deleted on the next app start up
+//     (eg. "pb_data/.pb_temp_to_delete/old_pb_data").
+//     This is because on some environments it may not be allowed
+//     to delete the currently open "pb_data" files.
+//
+//  4. Move the extracted dir content to the app "pb_data".
+//
+//  5. Restart the app (on successful app bootstrap it will also remove the old pb_data).
+//
+// If a failure occur during the restore process the dir changes are reverted.
+// If for whatever reason the revert is not possible, it panics.
+//
+// Note that if your pb_data has custom network mounts as subdirectories, then
+// it is possible the restore to fail during the `os.Rename` operations
+// (see https://github.com/pocketbase/pocketbase/issues/4647).
+func (app *BaseApp) RestoreBackup(ctx context.Context, name string) error {
+	if app.Store().Has(StoreKeyActiveBackup) {
+		return errors.New("try again later - another backup/restore operation has already been started")
+	}
+
+	app.Store().Set(StoreKeyActiveBackup, name)
+	defer app.Store().Remove(StoreKeyActiveBackup)
+
+	event := new(BackupEvent)
+	event.App = app
+	event.Context = ctx
+	event.Name = name
+	// default root dir entries to exclude from the backup restore
+	event.Exclude = []string{LocalBackupsDirName, LocalTempDirName, LocalAutocertCacheDirName, lostFoundDirName}
+
+	return app.OnBackupRestore().Trigger(event, func(e *BackupEvent) error {
+		if runtime.GOOS == "windows" {
+			return errors.New("restore is not supported on Windows")
+		}
+
+		// make sure that the special temp directory exists
+		// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
+		localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
+		if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
+			return fmt.Errorf("failed to create a temp dir: %w", err)
+		}
+
+		fsys, err := e.App.NewBackupsFilesystem()
+		if err != nil {
+			return err
+		}
+		defer fsys.Close()
+
+		fsys.SetContext(e.Context)
+
+		if ok, _ := fsys.Exists(name); !ok {
+			return fmt.Errorf("missing or invalid backup file %q to restore", name)
+		}
+
+		extractedDataDir := filepath.Join(localTempDir, "pb_restore_"+security.PseudorandomString(8))
+		defer os.RemoveAll(extractedDataDir)
+
+		// extract the zip
+		if e.App.Settings().Backups.S3.Enabled {
+			br, err := fsys.GetReader(name)
+			if err != nil {
+				return err
+			}
+			defer br.Close()
+
+			// create a temp zip file from the blob.Reader and try to extract it
+			tempZip, err := os.CreateTemp(localTempDir, "pb_restore_zip")
+			if err != nil {
+				return err
+			}
+			defer os.Remove(tempZip.Name())
+			defer tempZip.Close() // note: this technically shouldn't be necessary but it is here to workaround platforms discrepancies
+
+			_, err = io.Copy(tempZip, br)
+			if err != nil {
+				return err
+			}
+
+			err = archive.Extract(tempZip.Name(), extractedDataDir)
+			if err != nil {
+				return err
+			}
+
+			// remove the temp zip file since we no longer need it
+			// (this is in case the app restarts and the defer calls are not called)
+			_ = tempZip.Close()
+			err = os.Remove(tempZip.Name())
+			if err != nil {
+				e.App.Logger().Warn(
+					"[RestoreBackup] Failed to remove the temp zip backup file",
+					slog.String("file", tempZip.Name()),
+					slog.String("error", err.Error()),
+				)
+			}
+		} else {
+			// manually construct the local path to avoid creating a copy of the zip file
+			// since the blob reader currently doesn't implement ReaderAt
+			zipPath := filepath.Join(e.App.DataDir(), LocalBackupsDirName, filepath.Base(name))
+
+			err = archive.Extract(zipPath, extractedDataDir)
+			if err != nil {
+				return err
+			}
+		}
+
+		// ensure that at least a database file exists
+		extractedDB := filepath.Join(extractedDataDir, "data.db")
+		if _, err := os.Stat(extractedDB); err != nil {
+			return fmt.Errorf("data.db file is missing or invalid: %w", err)
+		}
+
+		oldTempDataDir := filepath.Join(localTempDir, "old_pb_data_"+security.PseudorandomString(8))
+
+		replaceErr := e.App.RunInTransaction(func(txApp App) error {
+			return txApp.AuxRunInTransaction(func(txApp App) error {
+				// move the current pb_data content to a special temp location
+				// that will hold the old data between dirs replace
+				// (the temp dir will be automatically removed on the next app start)
+				if err := osutils.MoveDirContent(txApp.DataDir(), oldTempDataDir, e.Exclude...); err != nil {
+					return fmt.Errorf("failed to move the current pb_data content to a temp location: %w", err)
+				}
+
+				// move the extracted archive content to the app's pb_data
+				if err := osutils.MoveDirContent(extractedDataDir, txApp.DataDir(), e.Exclude...); err != nil {
+					return fmt.Errorf("failed to move the extracted archive content to pb_data: %w", err)
+				}
+
+				return nil
+			})
+		})
+		if replaceErr != nil {
+			return replaceErr
+		}
+
+		revertDataDirChanges := func() error {
+			return e.App.RunInTransaction(func(txApp App) error {
+				return txApp.AuxRunInTransaction(func(txApp App) error {
+					if err := osutils.MoveDirContent(txApp.DataDir(), extractedDataDir, e.Exclude...); err != nil {
+						return fmt.Errorf("failed to revert the extracted dir change: %w", err)
+					}
+
+					if err := osutils.MoveDirContent(oldTempDataDir, txApp.DataDir(), e.Exclude...); err != nil {
+						return fmt.Errorf("failed to revert old pb_data dir change: %w", err)
+					}
+
+					return nil
+				})
+			})
+		}
+
+		// restart the app
+		if err := e.App.Restart(); err != nil {
+			if revertErr := revertDataDirChanges(); revertErr != nil {
+				panic(revertErr)
+			}
+
+			return fmt.Errorf("failed to restart the app process: %w", err)
+		}
+
+		return nil
+	})
+}

core/backup_test.go

@@ -126,11 +126,7 @@ func verifyBackupContent(app core.App, path string) error {
 	expectedRootEntries := []string{
 		"storage",
 		"data.db",
-		"data.db-shm",
-		"data.db-wal",
 		"auxiliary.db",
-		"auxiliary.db-shm",
-		"auxiliary.db-wal",
 		".gitignore",
 	}
 

core/base.go

@@ -12,7 +12,7 @@ import (
 	"regexp"
 	"runtime"
 	"strings"
-	"syscall"
+	"sync/atomic"
 	"time"
 
 	"github.com/fatih/color"
@@ -26,6 +26,8 @@ import (
 	"github.com/pocketbase/pocketbase/tools/store"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
 	"github.com/pocketbase/pocketbase/tools/types"
+	"github.com/spf13/cast"
+	"golang.org/x/sync/semaphore"
 )
 
 const (
@@ -37,8 +39,15 @@ const (
 
 	LocalStorageDirName       string = "storage"
 	LocalBackupsDirName       string = "backups"
-	LocalTempDirName          string = ".pb_temp_to_delete" // temp pb_data sub directory that will be deleted on each app.Bootstrap()
 	LocalAutocertCacheDirName string = ".autocert_cache"
+	LocalNotifyDirName        string = ".notify"            // optional watched directory that is used as a cross-platform workaround for synchronizing various runtime states between multiple PocketBase instances pointing to the same pb_data
+	LocalTempDirName          string = ".pb_temp_to_delete" // temp pb_data sub directory that will be deleted on each app.Bootstrap()
+
+	// @todo consider removing after backups refactoring
+	lostFoundDirName string = "lost+found"
+
+	dataDBFilename string = "data.db"
+	auxDBFilename  string = "auxiliary.db"
 )
 
 // FilesManager defines an interface with common methods that files manager models should implement.
@@ -69,8 +78,8 @@ var _ App = (*BaseApp)(nil)
 // BaseApp implements core.App and defines the base PocketBase app structure.
 type BaseApp struct {
 	config              *BaseAppConfig
-	txInfo              *txAppInfo
-	store               *store.Store[any]
+	txInfo              *TxAppInfo
+	store               *store.Store[string, any]
 	cron                *cron.Cron
 	settings            *Settings
 	subscriptionsBroker *subscriptions.Broker
@@ -81,11 +90,12 @@ type BaseApp struct {
 	auxNonconcurrentDB  dbx.Builder
 
 	// app event hooks
-	onBootstrap     *hook.Hook[*BootstrapEvent]
-	onServe         *hook.Hook[*ServeEvent]
-	onTerminate     *hook.Hook[*TerminateEvent]
-	onBackupCreate  *hook.Hook[*BackupEvent]
-	onBackupRestore *hook.Hook[*BackupEvent]
+	onBootstrap      *hook.Hook[*BootstrapEvent]
+	onBootstrapClear *hook.Hook[*BootstrapEvent]
+	onServe          *hook.Hook[*ServeEvent]
+	onTerminate      *hook.Hook[*TerminateEvent]
+	onBackupCreate   *hook.Hook[*BackupEvent]
+	onBackupRestore  *hook.Hook[*BackupEvent]
 
 	// db model hooks
 	onModelValidate           *hook.Hook[*ModelEvent]
@@ -141,6 +151,17 @@ type BaseApp struct {
 	onMailerRecordOTPSend           *hook.Hook[*MailerRecordEvent]
 	onMailerRecordAuthAlertSend     *hook.Hook[*MailerRecordEvent]
 
+	// filesystem event hooks
+	//
+	// @todo 1:
+	// intentionally not exposed since the events are too "chatty" and
+	// can cause unnecessary userland tests breaking changes;
+	// reevaluate once refactoring the file_field
+	//
+	// @todo 2: if exposed consider registering the same for the backup filesystem
+	_onFilesystemNewWriter *hook.Hook[*FilesystemNewWriterEvent]
+	_onFilesystemDelete    *hook.Hook[*FilesystemDeleteEvent]
+
 	// realtime api event hooks
 	onRealtimeConnectRequest   *hook.Hook[*RealtimeConnectRequestEvent]
 	onRealtimeMessageSend      *hook.Hook[*RealtimeMessageEvent]
@@ -194,7 +215,7 @@ type BaseApp struct {
 func NewBaseApp(config BaseAppConfig) *BaseApp {
 	app := &BaseApp{
 		settings:            newDefaultSettings(),
-		store:               store.New[any](nil),
+		store:               store.New[string, any](nil),
 		cron:                cron.New(),
 		subscriptionsBroker: subscriptions.NewBroker(),
 		config:              &config,
@@ -230,6 +251,7 @@ func NewBaseApp(config BaseAppConfig) *BaseApp {
 func (app *BaseApp) initHooks() {
 	// app event hooks
 	app.onBootstrap = &hook.Hook[*BootstrapEvent]{}
+	app.onBootstrapClear = &hook.Hook[*BootstrapEvent]{}
 	app.onServe = &hook.Hook[*ServeEvent]{}
 	app.onTerminate = &hook.Hook[*TerminateEvent]{}
 	app.onBackupCreate = &hook.Hook[*BackupEvent]{}
@@ -289,6 +311,10 @@ func (app *BaseApp) initHooks() {
 	app.onMailerRecordOTPSend = &hook.Hook[*MailerRecordEvent]{}
 	app.onMailerRecordAuthAlertSend = &hook.Hook[*MailerRecordEvent]{}
 
+	// filesystem event hooks
+	app._onFilesystemNewWriter = &hook.Hook[*FilesystemNewWriterEvent]{}
+	app._onFilesystemDelete = &hook.Hook[*FilesystemDeleteEvent]{}
+
 	// realtime API event hooks
 	app.onRealtimeConnectRequest = &hook.Hook[*RealtimeConnectRequestEvent]{}
 	app.onRealtimeMessageSend = &hook.Hook[*RealtimeMessageEvent]{}
@@ -340,8 +366,6 @@ func (app *BaseApp) initHooks() {
 // NB! Note that using the returned app instance may cause data integrity errors
 // since the Record validations and data normalizations (including files uploads)
 // rely on the app hooks to work.
-//
-// @todo consider caching the created instance?
 func (app *BaseApp) UnsafeWithoutHooks() App {
 	clone := *app
 
@@ -362,9 +386,17 @@ func (app *BaseApp) Logger() *slog.Logger {
 	return app.logger
 }
 
+// TxInfo returns the transaction associated with the current app instance (if any).
+//
+// Could be used if you want to execute indirectly a function after
+// the related app transaction completes using `app.TxInfo().OnAfterFunc(callback)`.
+func (app *BaseApp) TxInfo() *TxAppInfo {
+	return app.txInfo
+}
+
 // IsTransactional checks if the current app instance is part of a transaction.
 func (app *BaseApp) IsTransactional() bool {
-	return app.txInfo != nil
+	return app.TxInfo() != nil
 }
 
 // IsBootstrapped checks if the application was initialized
@@ -376,14 +408,14 @@ func (app *BaseApp) IsBootstrapped() bool {
 // Bootstrap initializes the application
 // (aka. create data dir, open db connections, load settings, etc.).
 //
-// It will call ResetBootstrapState() if the application was already bootstrapped.
+// It calls ClearBootstrap() if the application was already bootstrapped.
 func (app *BaseApp) Bootstrap() error {
 	event := &BootstrapEvent{}
 	event.App = app
 
 	err := app.OnBootstrap().Trigger(event, func(e *BootstrapEvent) error {
-		// clear resources of previous core state (if any)
-		if err := app.ResetBootstrapState(); err != nil {
+		// clear previous bootstrap state (if any)
+		if err := app.ClearBootstrap(); err != nil {
 			return err
 		}
 
@@ -431,81 +463,151 @@ func (app *BaseApp) Bootstrap() error {
 	return err
 }
 
-type closer interface {
-	Close() error
-}
-
-// ResetBootstrapState releases the initialized core app resources
-// (closing db connections, stopping cron ticker, etc.).
+// Deprecated: use [ClearBootstrap].
 func (app *BaseApp) ResetBootstrapState() error {
-	app.Cron().Stop()
-
-	var errs []error
-
-	dbs := []*dbx.Builder{
-		&app.concurrentDB,
-		&app.nonconcurrentDB,
-		&app.auxConcurrentDB,
-		&app.auxNonconcurrentDB,
-	}
-
-	for _, db := range dbs {
-		if db == nil {
-			continue
-		}
-		if v, ok := (*db).(closer); ok {
-			if err := v.Close(); err != nil {
-				errs = append(errs, err)
-			}
-		}
-		*db = nil
-	}
-
-	if len(errs) > 0 {
-		return errors.Join(errs...)
-	}
-
-	return nil
+	return app.ClearBootstrap()
 }
 
-// DB returns the default app data db instance (pb_data/data.db).
+// ClearBootstrap releases the initialized core app resources
+// (closing db connections, stopping cron ticker, etc.).
+//
+// This method is no-op if the application is not bootstrapped yet.
+func (app *BaseApp) ClearBootstrap() error {
+	if !app.IsBootstrapped() {
+		return nil
+	}
+
+	event := &BootstrapEvent{}
+	event.App = app
+
+	return app.OnBootstrapClear().Trigger(event, func(e *BootstrapEvent) error {
+		type closer interface {
+			Close() error
+		}
+
+		var errs []error
+
+		dbs := []*dbx.Builder{
+			&app.concurrentDB,
+			&app.nonconcurrentDB,
+			&app.auxConcurrentDB,
+			&app.auxNonconcurrentDB,
+		}
+
+		for _, db := range dbs {
+			if db == nil {
+				continue
+			}
+			if v, ok := (*db).(closer); ok {
+				if err := v.Close(); err != nil {
+					errs = append(errs, err)
+				}
+			}
+			*db = nil
+		}
+
+		if len(errs) > 0 {
+			return errors.Join(errs...)
+		}
+
+		return nil
+	})
+}
+
+// DB returns the default app data.db builder instance.
+//
+// To minimize SQLITE_BUSY errors, it automatically routes the
+// SELECT queries to the underlying concurrent db pool and everything
+// else to the nonconcurrent one.
+//
+// For more finer control over the used connections pools you can
+// call directly ConcurrentDB() or NonconcurrentDB().
 func (app *BaseApp) DB() dbx.Builder {
+	// transactional or both are nil
+	if app.concurrentDB == app.nonconcurrentDB {
+		return app.concurrentDB
+	}
+
+	return &dualDBBuilder{
+		concurrentDB:    app.concurrentDB,
+		nonconcurrentDB: app.nonconcurrentDB,
+	}
+}
+
+// ConcurrentDB returns the concurrent app data.db builder instance.
+//
+// This method is used mainly internally for executing db read
+// operations in a concurrent/non-blocking manner.
+//
+// Most users should use simply DB() as it will automatically
+// route the query execution to ConcurrentDB() or NonconcurrentDB().
+//
+// In a transaction the ConcurrentDB() and NonconcurrentDB() refer to the same *dbx.TX instance.
+func (app *BaseApp) ConcurrentDB() dbx.Builder {
 	return app.concurrentDB
 }
 
-// NonconcurrentDB returns the nonconcurrent app data db instance (pb_data/data.db).
+// NonconcurrentDB returns the nonconcurrent app data.db builder instance.
 //
 // The returned db instance is limited only to a single open connection,
-// meaning that it can process only 1 db operation at a time (other operations will be queued up).
+// meaning that it can process only 1 db operation at a time (other queries queue up).
 //
 // This method is used mainly internally and in the tests to execute write
 // (save/delete) db operations as it helps with minimizing the SQLITE_BUSY errors.
 //
-// For the majority of cases you would want to use the regular DB() method
-// since it allows concurrent db read operations.
+// Most users should use simply DB() as it will automatically
+// route the query execution to ConcurrentDB() or NonconcurrentDB().
 //
 // In a transaction the ConcurrentDB() and NonconcurrentDB() refer to the same *dbx.TX instance.
 func (app *BaseApp) NonconcurrentDB() dbx.Builder {
 	return app.nonconcurrentDB
 }
 
-// AuxDB returns the default app auxiliary db instance (pb_data/auxiliary.db).
+// AuxDB returns the app auxiliary.db builder instance.
+//
+// To minimize SQLITE_BUSY errors, it automatically routes the
+// SELECT queries to the underlying concurrent db pool and everything
+// else to the nonconcurrent one.
+//
+// For more finer control over the used connections pools you can
+// call directly AuxConcurrentDB() or AuxNonconcurrentDB().
 func (app *BaseApp) AuxDB() dbx.Builder {
+	// transactional or both are nil
+	if app.auxConcurrentDB == app.auxNonconcurrentDB {
+		return app.auxConcurrentDB
+	}
+
+	return &dualDBBuilder{
+		concurrentDB:    app.auxConcurrentDB,
+		nonconcurrentDB: app.auxNonconcurrentDB,
+	}
+}
+
+// AuxConcurrentDB returns the concurrent app auxiliary.db builder instance.
+//
+// This method is used mainly internally for executing db read
+// operations in a concurrent/non-blocking manner.
+//
+// Most users should use simply AuxDB() as it will automatically
+// route the query execution to AuxConcurrentDB() or AuxNonconcurrentDB().
+//
+// In a transaction the AuxConcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
+func (app *BaseApp) AuxConcurrentDB() dbx.Builder {
 	return app.auxConcurrentDB
 }
 
-// AuxNonconcurrentDB returns the nonconcurrent app auxiliary db instance (pb_data/auxiliary.db).
+// AuxNonconcurrentDB returns the nonconcurrent app auxiliary.db builder instance.
 //
 // The returned db instance is limited only to a single open connection,
-// meaning that it can process only 1 db operation at a time (other operations will be queued up).
+// meaning that it can process only 1 db operation at a time (other queries queue up).
 //
 // This method is used mainly internally and in the tests to execute write
 // (save/delete) db operations as it helps with minimizing the SQLITE_BUSY errors.
 //
-// For the majority of cases you would want to use the regular DB() method
-// since it allows concurrent db read operations.
+// Most users should use simply AuxDB() as it will automatically
+// route the query execution to AuxConcurrentDB() or AuxNonconcurrentDB().
 //
-// In a transaction the AuxNonconcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
+// In a transaction the AuxConcurrentDB() and AuxNonconcurrentDB() refer to the same *dbx.TX instance.
 func (app *BaseApp) AuxNonconcurrentDB() dbx.Builder {
 	return app.auxNonconcurrentDB
 }
@@ -534,7 +636,7 @@ func (app *BaseApp) Settings() *Settings {
 }
 
 // Store returns the app runtime store.
-func (app *BaseApp) Store() *store.Store[any] {
+func (app *BaseApp) Store() *store.Store[string, any] {
 	return app.store
 }
 
@@ -603,6 +705,14 @@ func (app *BaseApp) NewMailClient() mailer.Mailer {
 							fmt.Fprintf(mailLog, "├─ Attachments: %v\n", attachmentKeys)
 						}
 
+						if len(ae.Message.InlineAttachments) > 0 {
+							attachmentKeys := make([]string, 0, len(ae.Message.InlineAttachments))
+							for k := range ae.Message.InlineAttachments {
+								attachmentKeys = append(attachmentKeys, k)
+							}
+							fmt.Fprintf(mailLog, "├─ InlineAttachments: %v\n", attachmentKeys)
+						}
+
 						const indentation = "        "
 						if ae.Message.Text != "" {
 							textParts := strings.Split(strings.TrimSpace(ae.Message.Text), "\n")
@@ -637,9 +747,10 @@ func (app *BaseApp) NewMailClient() mailer.Mailer {
 //
 // NB! Make sure to call Close() on the returned result
 // after you are done working with it.
-func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
+func (app *BaseApp) NewFilesystem() (fsys *filesystem.System, err error) {
 	if app.settings != nil && app.settings.S3.Enabled {
-		return filesystem.NewS3(
+		// S3
+		fsys, err = filesystem.NewS3(
 			app.settings.S3.Bucket,
 			app.settings.S3.Region,
 			app.settings.S3.Endpoint,
@@ -647,13 +758,44 @@ func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
 			app.settings.S3.Secret,
 			app.settings.S3.ForcePathStyle,
 		)
+	} else {
+		// local filesystem
+		fsys, err = filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	}
+	if err != nil {
+		return nil, err
 	}
 
-	// fallback to local filesystem
-	return filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
+	// attach delete hook
+	if app._onFilesystemDelete.Length() > 0 {
+		fsys.OnDelete().BindFunc(func(originalEvent *filesystem.DeleteEvent) error {
+			appEvent := new(FilesystemDeleteEvent)
+			appEvent.DeleteEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemDelete.Trigger(appEvent, func(fde *FilesystemDeleteEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	// attach write hook
+	if app._onFilesystemNewWriter.Length() > 0 {
+		fsys.OnNewWriter().BindFunc(func(originalEvent *filesystem.NewWriterEvent) error {
+			appEvent := new(FilesystemNewWriterEvent)
+			appEvent.NewWriterEvent = originalEvent
+			appEvent.App = app
+
+			return app._onFilesystemNewWriter.Trigger(appEvent, func(fwe *FilesystemNewWriterEvent) error {
+				return originalEvent.Next()
+			})
+		})
+	}
+
+	return fsys, nil
 }
 
-// NewFilesystem creates a new local or S3 filesystem instance
+// NewBackupsFilesystem creates a new local or S3 filesystem instance
 // for managing app backups based on the current app settings.
 //
 // NB! Make sure to call Close() on the returned result
@@ -692,7 +834,7 @@ func (app *BaseApp) Restart() error {
 	event.IsRestart = true
 
 	return app.OnTerminate().Trigger(event, func(e *TerminateEvent) error {
-		_ = e.App.ResetBootstrapState()
+		_ = e.App.ClearBootstrap()
 
 		// attempt to restart the bootstrap process in case execve returns an error for some reason
 		defer func() {
@@ -701,7 +843,7 @@ func (app *BaseApp) Restart() error {
 			}
 		}()
 
-		return syscall.Exec(execPath, os.Args, os.Environ())
+		return execve(execPath, os.Args, os.Environ())
 	})
 }
 
@@ -735,6 +877,10 @@ func (app *BaseApp) OnBootstrap() *hook.Hook[*BootstrapEvent] {
 	return app.onBootstrap
 }
 
+func (app *BaseApp) OnBootstrapClear() *hook.Hook[*BootstrapEvent] {
+	return app.onBootstrapClear
+}
+
 func (app *BaseApp) OnServe() *hook.Hook[*ServeEvent] {
 	return app.onServe
 }
@@ -941,6 +1087,18 @@ func (app *BaseApp) OnMailerRecordAuthAlertSend(tags ...string) *hook.TaggedHook
 	return hook.NewTaggedHook(app.onMailerRecordAuthAlertSend, tags...)
 }
 
+// -------------------------------------------------------------------
+// Filesystem event hooks
+// -------------------------------------------------------------------
+
+func (app *BaseApp) onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent] {
+	return app._onFilesystemNewWriter
+}
+
+func (app *BaseApp) onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent] {
+	return app._onFilesystemDelete
+}
+
 // -------------------------------------------------------------------
 // Realtime API event hooks
 // -------------------------------------------------------------------
@@ -1098,7 +1256,7 @@ func (app *BaseApp) OnBatchRequest() *hook.Hook[*BatchRequestEvent] {
 // -------------------------------------------------------------------
 
 func (app *BaseApp) initDataDB() error {
-	dbPath := filepath.Join(app.DataDir(), "data.db")
+	dbPath := filepath.Join(app.DataDir(), dataDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1133,26 +1291,25 @@ func (app *BaseApp) initDataDB() error {
 	return nil
 }
 
-var sqlLogReplacements = map[string]string{
-	"{{":    "`",
-	"}}":    "`",
-	"[[":    "`",
-	"]]":    "`",
-	"<nil>": "NULL",
+var sqlLogReplacements = []struct {
+	pattern     *regexp.Regexp
+	replacement string
+}{
+	{regexp.MustCompile(`\[\[([^\[\]\{\}\.]+)\.([^\[\]\{\}\.]+)\]\]`), "`$1`.`$2`"},
+	{regexp.MustCompile(`\{\{([^\[\]\{\}\.]+)\.([^\[\]\{\}\.]+)\}\}`), "`$1`.`$2`"},
+	{regexp.MustCompile(`([^'"])?\{\{`), "$1`"},
+	{regexp.MustCompile(`\}\}([^'"])?`), "`$1"},
+	{regexp.MustCompile(`([^'"])?\[\[`), "$1`"},
+	{regexp.MustCompile(`\]\]([^'"])?`), "`$1"},
+	{regexp.MustCompile(`<nil>`), "NULL"},
 }
-var sqlLogPrefixedTableIdentifierPattern = regexp.MustCompile(`\[\[(.+)\.(.+)\]\]`)
-var sqlLogPrefixedColumnIdentifierPattern = regexp.MustCompile(`\{\{(.+)\.(.+)\}\}`)
 
-// normalizeSQLLog replaces common query builder charactes with their plain SQL version for easier debugging.
+// normalizeSQLLog replaces common query builder characters with their plain SQL version for easier debugging.
 // The query is still not suitable for execution and should be used only for log and debug purposes
 // (the normalization is done here to avoid breaking changes in dbx).
 func normalizeSQLLog(sql string) string {
-	sql = sqlLogPrefixedTableIdentifierPattern.ReplaceAllString(sql, "`$1`.`$2`")
-
-	sql = sqlLogPrefixedColumnIdentifierPattern.ReplaceAllString(sql, "`$1`.`$2`")
-
-	for old, new := range sqlLogReplacements {
-		sql = strings.ReplaceAll(sql, old, new)
+	for _, item := range sqlLogReplacements {
+		sql = item.pattern.ReplaceAllString(sql, item.replacement)
 	}
 
 	return sql
@@ -1161,7 +1318,7 @@ func normalizeSQLLog(sql string) string {
 func (app *BaseApp) initAuxDB() error {
 	// note: renamed to "auxiliary" because "aux" is a reserved Windows filename
 	// (see https://github.com/pocketbase/pocketbase/issues/5607)
-	dbPath := filepath.Join(app.DataDir(), "auxiliary.db")
+	dbPath := filepath.Join(app.DataDir(), auxDBFilename)
 
 	concurrentDB, err := app.config.DBConnect(dbPath)
 	if err != nil {
@@ -1185,6 +1342,33 @@ func (app *BaseApp) initAuxDB() error {
 	return nil
 }
 
+// @todo remove after refactoring the FilesManager interface
+func supportFiles(m Model) bool {
+	var collection *Collection
+	switch v := m.(type) {
+	case *Collection:
+		collection = v
+	case *Record:
+		collection = v.Collection()
+	case RecordProxy:
+		if v.ProxyRecord() != nil {
+			collection = v.ProxyRecord().Collection()
+		}
+	}
+
+	if collection == nil {
+		return true
+	}
+
+	for _, f := range collection.Fields {
+		if f.Type() == FieldTypeFile {
+			return true
+		}
+	}
+
+	return false
+}
+
 func (app *BaseApp) registerBaseHooks() {
 	deletePrefix := func(prefix string) error {
 		fs, err := app.NewFilesystem()
@@ -1201,26 +1385,44 @@ func (app *BaseApp) registerBaseHooks() {
 		return nil
 	}
 
+	maxFilesDeleteWorkers := cast.ToInt64(os.Getenv("PB_FILES_DELETE_MAX_WORKERS"))
+	if maxFilesDeleteWorkers <= 0 {
+		maxFilesDeleteWorkers = 2000 // the value is arbitrary chosen and may change in the future
+	}
+
+	deleteSem := semaphore.NewWeighted(maxFilesDeleteWorkers)
+
 	// try to delete the storage files from deleted Collection, Records, etc. model
 	app.OnModelAfterDeleteSuccess().Bind(&hook.Handler[*ModelEvent]{
 		Id: "__pbFilesManagerDelete__",
 		Func: func(e *ModelEvent) error {
-			if m, ok := e.Model.(FilesManager); ok && m.BaseFilesPath() != "" {
+			if m, ok := e.Model.(FilesManager); ok && m.BaseFilesPath() != "" && supportFiles(e.Model) {
 				// ensure that there is a trailing slash so that the list iterator could start walking from the prefix dir
 				// (https://github.com/pocketbase/pocketbase/discussions/5246#discussioncomment-10128955)
 				prefix := strings.TrimRight(m.BaseFilesPath(), "/") + "/"
 
-				// run in the background for "optimistic" delete to avoid
-				// blocking the delete transaction
-				routine.FireAndForget(func() {
-					if err := deletePrefix(prefix); err != nil {
-						app.Logger().Error(
-							"Failed to delete storage prefix (non critical error; usually could happen because of S3 api limits)",
-							slog.String("prefix", prefix),
-							slog.String("error", err.Error()),
-						)
-					}
-				})
+				// note: for now assume no context cancellation
+				err := deleteSem.Acquire(context.Background(), 1)
+				if err != nil {
+					app.Logger().Error(
+						"Failed to delete storage prefix (couldn't acquire a worker)",
+						slog.String("prefix", prefix),
+						slog.String("error", err.Error()),
+					)
+				} else {
+					// run in the background for "optimistic" delete to avoid blocking the delete transaction
+					routine.FireAndForget(func() {
+						defer deleteSem.Release(1)
+
+						if err := deletePrefix(prefix); err != nil {
+							app.Logger().Error(
+								"Failed to delete storage prefix (non critical error; usually could happen because of S3 api limits)",
+								slog.String("prefix", prefix),
+								slog.String("error", err.Error()),
+							)
+						}
+					})
+				}
 			}
 
 			return e.Next()
@@ -1232,7 +1434,15 @@ func (app *BaseApp) registerBaseHooks() {
 		Id: "__pbCronStart__",
 		Func: func(e *ServeEvent) error {
 			app.Cron().Start()
+			return e.Next()
+		},
+		Priority: 999,
+	})
 
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbCronStop__",
+		Func: func(e *BootstrapEvent) error {
+			app.Cron().Stop()
 			return e.Next()
 		},
 		Priority: 999,
@@ -1249,7 +1459,7 @@ func (app *BaseApp) registerBaseHooks() {
 			app.Logger().Warn("Failed to run periodic PRAGMA wal_checkpoint for the auxiliary DB", slog.String("error", execErr.Error()))
 		}
 
-		_, execErr = app.DB().NewQuery("PRAGMA optimize").Execute()
+		_, execErr = app.NonconcurrentDB().NewQuery("PRAGMA optimize").Execute()
 		if execErr != nil {
 			app.Logger().Warn("Failed to run periodic PRAGMA optimize", slog.String("error", execErr.Error()))
 		}
@@ -1264,6 +1474,7 @@ func (app *BaseApp) registerBaseHooks() {
 	app.registerMFAHooks()
 	app.registerOTPHooks()
 	app.registerAuthOriginHooks()
+	app.registerNotifyWatcherHooks()
 }
 
 // getLoggerMinLevel returns the logger min level based on the
@@ -1288,9 +1499,41 @@ func getLoggerMinLevel(app App) slog.Level {
 }
 
 func (app *BaseApp) initLogger() error {
+	var stopped atomic.Bool
+
 	duration := 3 * time.Second
 	ticker := time.NewTicker(duration)
-	done := make(chan bool)
+
+	done := make(chan struct{}, 1)
+
+	runLogsWrite := func(logs []*logger.Log) {
+		if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
+			return
+		}
+
+		// write the accumulated logs
+		//
+		// note: based on several local tests there is no
+		// significant performance difference between small number
+		// of separate write queries vs 1 big INSERT
+		app.AuxRunInTransaction(func(txApp App) error {
+			model := &Log{}
+			for _, l := range logs {
+				model.MarkAsNew()
+				model.Id = GenerateDefaultRandomId()
+				model.Level = int(l.Level)
+				model.Message = l.Message
+				model.Data = l.Data
+				model.Created, _ = types.ParseDateTime(l.Time)
+
+				if err := txApp.AuxSave(model); err != nil {
+					log.Println("Failed to write log", model, err)
+				}
+			}
+
+			return nil
+		})
+	}
 
 	handler := logger.NewBatchHandler(logger.BatchOptions{
 		Level:     getLoggerMinLevel(app),
@@ -1305,40 +1548,30 @@ func (app *BaseApp) initLogger() error {
 				}
 			}
 
-			ticker.Reset(duration)
+			if !stopped.Load() {
+				ticker.Reset(duration)
+			}
 
 			return app.Settings().Logs.MaxDays > 0
 		},
 		WriteFunc: func(ctx context.Context, logs []*logger.Log) error {
-			if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
-				return nil
+			// don't block and wait for the write transaction to complete
+			// when we can't be sure if the logs write wasn't triggered while
+			// inside another AUX db transaction (ticker or batch threshold reached)
+			// which can block indefinitely and cause deadlock
+			// (https://github.com/pocketbase/pocketbase/issues/7836)
+			shouldBlock, _ := ctx.Value(logger.BlockKey).(bool)
+			if shouldBlock {
+				runLogsWrite(logs)
+			} else {
+				routine.FireAndForget(func() { runLogsWrite(logs) })
 			}
 
-			// write the accumulated logs
-			// (note: based on several local tests there is no significant performance difference between small number of separate write queries vs 1 big INSERT)
-			app.AuxRunInTransaction(func(txApp App) error {
-				model := &Log{}
-				for _, l := range logs {
-					model.MarkAsNew()
-					model.Id = GenerateDefaultRandomId()
-					model.Level = int(l.Level)
-					model.Message = l.Message
-					model.Data = l.Data
-					model.Created, _ = types.ParseDateTime(l.Time)
-
-					if err := txApp.AuxSave(model); err != nil {
-						log.Println("Failed to write log", model, err)
-					}
-				}
-
-				return nil
-			})
-
 			return nil
 		},
 	})
 
-	go func() {
+	routine.FireAndForget(func() {
 		ctx := context.Background()
 
 		for {
@@ -1349,19 +1582,27 @@ func (app *BaseApp) initLogger() error {
 				handler.WriteAll(ctx)
 			}
 		}
-	}()
+	})
 
 	app.logger = slog.New(handler)
 
-	// write all remaining logs before ticker.Stop to avoid races with ResetBootstrap user calls
-	app.OnTerminate().Bind(&hook.Handler[*TerminateEvent]{
-		Id: "__pbAppLoggerOnTerminate__",
-		Func: func(e *TerminateEvent) error {
-			handler.WriteAll(context.Background())
+	// attempt to write all queued logs before clearing the application bootstrap state
+	app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
+		Id: "__pbAppLoggerFlushBeforeStop__",
+		Func: func(e *BootstrapEvent) error {
+			// extra precaution in case the hook was manually triggered while inside aux db transaction
+			_, isTx := e.App.AuxNonconcurrentDB().(*dbx.Tx)
+			ctx := context.WithValue(context.Background(), logger.BlockKey, !isTx)
+			handler.WriteAll(ctx)
 
+			stopped.Store(true)
 			ticker.Stop()
 
-			done <- true
+			// don't block in case the hook is triggered more than once
+			select {
+			case done <- struct{}{}:
+			default:
+			}
 
 			return e.Next()
 		},

core/base_backup.go

@@ -1,364 +0,0 @@
-package core
-
-import (
-	"context"
-	"errors"
-	"fmt"
-	"io"
-	"log/slog"
-	"os"
-	"path/filepath"
-	"runtime"
-	"sort"
-	"time"
-
-	"github.com/pocketbase/pocketbase/tools/archive"
-	"github.com/pocketbase/pocketbase/tools/filesystem"
-	"github.com/pocketbase/pocketbase/tools/inflector"
-	"github.com/pocketbase/pocketbase/tools/osutils"
-	"github.com/pocketbase/pocketbase/tools/security"
-)
-
-const (
-	StoreKeyActiveBackup = "@activeBackup"
-)
-
-// CreateBackup creates a new backup of the current app pb_data directory.
-//
-// If name is empty, it will be autogenerated.
-// If backup with the same name exists, the new backup file will replace it.
-//
-// The backup is executed within a transaction, meaning that new writes
-// will be temporary "blocked" until the backup file is generated.
-//
-// To safely perform the backup, it is recommended to have free disk space
-// for at least 2x the size of the pb_data directory.
-//
-// By default backups are stored in pb_data/backups
-// (the backups directory itself is excluded from the generated backup).
-//
-// When using S3 storage for the uploaded collection files, you have to
-// take care manually to backup those since they are not part of the pb_data.
-//
-// Backups can be stored on S3 if it is configured in app.Settings().Backups.
-func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
-	if app.Store().Has(StoreKeyActiveBackup) {
-		return errors.New("try again later - another backup/restore operation has already been started")
-	}
-
-	app.Store().Set(StoreKeyActiveBackup, name)
-	defer app.Store().Remove(StoreKeyActiveBackup)
-
-	event := new(BackupEvent)
-	event.App = app
-	event.Context = ctx
-	event.Name = name
-	// default root dir entries to exclude from the backup generation
-	event.Exclude = []string{LocalBackupsDirName, LocalTempDirName, LocalAutocertCacheDirName}
-
-	return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
-		// generate a default name if missing
-		if e.Name == "" {
-			e.Name = generateBackupName(e.App, "pb_backup_")
-		}
-
-		// make sure that the special temp directory exists
-		// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
-		localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
-		if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
-			return fmt.Errorf("failed to create a temp dir: %w", err)
-		}
-
-		// archive pb_data in a temp directory, exluding the "backups" and the temp dirs
-		//
-		// Run in transaction to temporary block other writes (transactions uses the NonconcurrentDB connection).
-		// ---
-		tempPath := filepath.Join(localTempDir, "pb_backup_"+security.PseudorandomString(6))
-		createErr := e.App.RunInTransaction(func(txApp App) error {
-			return txApp.AuxRunInTransaction(func(txApp App) error {
-				// run manual checkpoint and truncate the WAL files
-				// (errors are ignored because it is not that important and the PRAGMA may not be supported by the used driver)
-				txApp.DB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-				txApp.AuxDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
-
-				return archive.Create(txApp.DataDir(), tempPath, e.Exclude...)
-			})
-		})
-		if createErr != nil {
-			return createErr
-		}
-		defer os.Remove(tempPath)
-
-		// persist the backup in the backups filesystem
-		// ---
-		fsys, err := e.App.NewBackupsFilesystem()
-		if err != nil {
-			return err
-		}
-		defer fsys.Close()
-
-		fsys.SetContext(e.Context)
-
-		file, err := filesystem.NewFileFromPath(tempPath)
-		if err != nil {
-			return err
-		}
-		file.OriginalName = e.Name
-		file.Name = file.OriginalName
-
-		if err := fsys.UploadFile(file, file.Name); err != nil {
-			return err
-		}
-
-		return nil
-	})
-}
-
-// RestoreBackup restores the backup with the specified name and restarts
-// the current running application process.
-//
-// NB! This feature is experimental and currently is expected to work only on UNIX based systems.
-//
-// To safely perform the restore it is recommended to have free disk space
-// for at least 2x the size of the restored pb_data backup.
-//
-// The performed steps are:
-//
-//  1. Download the backup with the specified name in a temp location
-//     (this is in case of S3; otherwise it creates a temp copy of the zip)
-//
-//  2. Extract the backup in a temp directory inside the app "pb_data"
-//     (eg. "pb_data/.pb_temp_to_delete/pb_restore").
-//
-//  3. Move the current app "pb_data" content (excluding the local backups and the special temp dir)
-//     under another temp sub dir that will be deleted on the next app start up
-//     (eg. "pb_data/.pb_temp_to_delete/old_pb_data").
-//     This is because on some environments it may not be allowed
-//     to delete the currently open "pb_data" files.
-//
-//  4. Move the extracted dir content to the app "pb_data".
-//
-//  5. Restart the app (on successful app bootstap it will also remove the old pb_data).
-//
-// If a failure occure during the restore process the dir changes are reverted.
-// If for whatever reason the revert is not possible, it panics.
-func (app *BaseApp) RestoreBackup(ctx context.Context, name string) error {
-	if app.Store().Has(StoreKeyActiveBackup) {
-		return errors.New("try again later - another backup/restore operation has already been started")
-	}
-
-	app.Store().Set(StoreKeyActiveBackup, name)
-	defer app.Store().Remove(StoreKeyActiveBackup)
-
-	event := new(BackupEvent)
-	event.App = app
-	event.Context = ctx
-	event.Name = name
-	// default root dir entries to exclude from the backup restore
-	event.Exclude = []string{LocalBackupsDirName, LocalTempDirName, LocalAutocertCacheDirName}
-
-	return app.OnBackupRestore().Trigger(event, func(e *BackupEvent) error {
-		if runtime.GOOS == "windows" {
-			return errors.New("restore is not supported on Windows")
-		}
-
-		fsys, err := e.App.NewBackupsFilesystem()
-		if err != nil {
-			return err
-		}
-		defer fsys.Close()
-
-		fsys.SetContext(e.Context)
-
-		// fetch the backup file in a temp location
-		br, err := fsys.GetFile(name)
-		if err != nil {
-			return err
-		}
-		defer br.Close()
-
-		// make sure that the special temp directory exists
-		// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
-		localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
-		if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
-			return fmt.Errorf("failed to create a temp dir: %w", err)
-		}
-
-		// create a temp zip file from the blob.Reader and try to extract it
-		tempZip, err := os.CreateTemp(localTempDir, "pb_restore_zip")
-		if err != nil {
-			return err
-		}
-		defer os.Remove(tempZip.Name())
-
-		if _, err := io.Copy(tempZip, br); err != nil {
-			return err
-		}
-
-		extractedDataDir := filepath.Join(localTempDir, "pb_restore_"+security.PseudorandomString(4))
-		defer os.RemoveAll(extractedDataDir)
-		if err := archive.Extract(tempZip.Name(), extractedDataDir); err != nil {
-			return err
-		}
-
-		// ensure that a database file exists
-		extractedDB := filepath.Join(extractedDataDir, "data.db")
-		if _, err := os.Stat(extractedDB); err != nil {
-			return fmt.Errorf("data.db file is missing or invalid: %w", err)
-		}
-
-		// remove the extracted zip file since we no longer need it
-		// (this is in case the app restarts and the defer calls are not called)
-		if err := os.Remove(tempZip.Name()); err != nil {
-			e.App.Logger().Debug(
-				"[RestoreBackup] Failed to remove the temp zip backup file",
-				slog.String("file", tempZip.Name()),
-				slog.String("error", err.Error()),
-			)
-		}
-
-		// move the current pb_data content to a special temp location
-		// that will hold the old data between dirs replace
-		// (the temp dir will be automatically removed on the next app start)
-		oldTempDataDir := filepath.Join(localTempDir, "old_pb_data_"+security.PseudorandomString(4))
-		if err := osutils.MoveDirContent(e.App.DataDir(), oldTempDataDir, e.Exclude...); err != nil {
-			return fmt.Errorf("failed to move the current pb_data content to a temp location: %w", err)
-		}
-
-		// move the extracted archive content to the app's pb_data
-		if err := osutils.MoveDirContent(extractedDataDir, e.App.DataDir(), e.Exclude...); err != nil {
-			return fmt.Errorf("failed to move the extracted archive content to pb_data: %w", err)
-		}
-
-		revertDataDirChanges := func() error {
-			if err := osutils.MoveDirContent(e.App.DataDir(), extractedDataDir, e.Exclude...); err != nil {
-				return fmt.Errorf("failed to revert the extracted dir change: %w", err)
-			}
-
-			if err := osutils.MoveDirContent(oldTempDataDir, e.App.DataDir(), e.Exclude...); err != nil {
-				return fmt.Errorf("failed to revert old pb_data dir change: %w", err)
-			}
-
-			return nil
-		}
-
-		// restart the app
-		if err := e.App.Restart(); err != nil {
-			if revertErr := revertDataDirChanges(); revertErr != nil {
-				panic(revertErr)
-			}
-
-			return fmt.Errorf("failed to restart the app process: %w", err)
-		}
-
-		return nil
-	})
-}
-
-// registerAutobackupHooks registers the autobackup app serve hooks.
-func (app *BaseApp) registerAutobackupHooks() {
-	const jobId = "__auto_pb_backup__"
-
-	loadJob := func() {
-		rawSchedule := app.Settings().Backups.Cron
-		if rawSchedule == "" {
-			app.Cron().Remove(jobId)
-			return
-		}
-
-		app.Cron().Add(jobId, rawSchedule, func() {
-			const autoPrefix = "@auto_pb_backup_"
-
-			name := generateBackupName(app, autoPrefix)
-
-			if err := app.CreateBackup(context.Background(), name); err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to create backup",
-					slog.String("name", name),
-					slog.String("error", err.Error()),
-				)
-			}
-
-			maxKeep := app.Settings().Backups.CronMaxKeep
-
-			if maxKeep == 0 {
-				return // no explicit limit
-			}
-
-			fsys, err := app.NewBackupsFilesystem()
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to initialize the backup filesystem",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-			defer fsys.Close()
-
-			files, err := fsys.List(autoPrefix)
-			if err != nil {
-				app.Logger().Error(
-					"[Backup cron] Failed to list autogenerated backups",
-					slog.String("error", err.Error()),
-				)
-				return
-			}
-
-			if maxKeep >= len(files) {
-				return // nothing to remove
-			}
-
-			// sort desc
-			sort.Slice(files, func(i, j int) bool {
-				return files[i].ModTime.After(files[j].ModTime)
-			})
-
-			// keep only the most recent n auto backup files
-			toRemove := files[maxKeep:]
-
-			for _, f := range toRemove {
-				if err := fsys.Delete(f.Key); err != nil {
-					app.Logger().Error(
-						"[Backup cron] Failed to remove old autogenerated backup",
-						slog.String("key", f.Key),
-						slog.String("error", err.Error()),
-					)
-				}
-			}
-		})
-	}
-
-	app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-
-	app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
-		if err := e.Next(); err != nil {
-			return err
-		}
-
-		loadJob()
-
-		return nil
-	})
-}
-
-func generateBackupName(app App, prefix string) string {
-	appName := inflector.Snakecase(app.Settings().Meta.AppName)
-	if len(appName) > 50 {
-		appName = appName[:50]
-	}
-
-	return fmt.Sprintf(
-		"%s%s_%s.zip",
-		prefix,
-		appName,
-		time.Now().UTC().Format("20060102150405"),
-	)
-}

core/base_test.go

@@ -5,7 +5,9 @@ import (
 	"database/sql"
 	"log/slog"
 	"os"
+	"slices"
 	"testing"
+	"testing/synctest"
 	"time"
 
 	_ "unsafe"
@@ -63,7 +65,7 @@ func TestBaseAppBootstrap(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if app.IsBootstrapped() {
 		t.Fatal("Didn't expect the application to be bootstrapped.")
@@ -99,9 +101,11 @@ func TestBaseAppBootstrap(t *testing.T) {
 	}
 
 	nilChecksBeforeReset := []nilCheck{
-		{"[before] concurrentDB", app.DB(), false},
+		{"[before] db", app.DB(), false},
+		{"[before] concurrentDB", app.ConcurrentDB(), false},
 		{"[before] nonconcurrentDB", app.NonconcurrentDB(), false},
-		{"[before] auxConcurrentDB", app.AuxDB(), false},
+		{"[before] auxDB", app.AuxDB(), false},
+		{"[before] auxConcurrentDB", app.AuxConcurrentDB(), false},
 		{"[before] auxNonconcurrentDB", app.AuxNonconcurrentDB(), false},
 		{"[before] settings", app.Settings(), false},
 		{"[before] logger", app.Logger(), false},
@@ -111,14 +115,16 @@ func TestBaseAppBootstrap(t *testing.T) {
 	runNilChecks(nilChecksBeforeReset)
 
 	// reset
-	if err := app.ResetBootstrapState(); err != nil {
+	if err := app.ClearBootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
 	nilChecksAfterReset := []nilCheck{
-		{"[after] concurrentDB", app.DB(), true},
+		{"[after] db", app.DB(), true},
+		{"[after] concurrentDB", app.ConcurrentDB(), true},
 		{"[after] nonconcurrentDB", app.NonconcurrentDB(), true},
-		{"[after] auxConcurrentDB", app.AuxDB(), true},
+		{"[after] auxDB", app.AuxDB(), true},
+		{"[after] auxConcurrentDB", app.AuxConcurrentDB(), true},
 		{"[after] auxNonconcurrentDB", app.AuxNonconcurrentDB(), true},
 		{"[after] settings", app.Settings(), false},
 		{"[after] logger", app.Logger(), false},
@@ -128,30 +134,47 @@ func TestBaseAppBootstrap(t *testing.T) {
 	runNilChecks(nilChecksAfterReset)
 }
 
-func TestNewBaseAppIsTransactional(t *testing.T) {
+func TestNewBaseAppTx(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	if err := app.Bootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
-	if app.IsTransactional() {
-		t.Fatalf("Didn't expect the app to be transactional")
+	mustNotHaveTx := func(app core.App) {
+		if app.IsTransactional() {
+			t.Fatalf("Didn't expect the app to be transactional")
+		}
+
+		if app.TxInfo() != nil {
+			t.Fatalf("Didn't expect the app.txInfo to be loaded")
+		}
 	}
 
-	app.RunInTransaction(func(txApp core.App) error {
-		if !txApp.IsTransactional() {
+	mustHaveTx := func(app core.App) {
+		if !app.IsTransactional() {
 			t.Fatalf("Expected the app to be transactional")
 		}
 
+		if app.TxInfo() == nil {
+			t.Fatalf("Expected the app.txInfo to be loaded")
+		}
+	}
+
+	mustNotHaveTx(app)
+
+	app.RunInTransaction(func(txApp core.App) error {
+		mustHaveTx(txApp)
 		return nil
 	})
+
+	mustNotHaveTx(app)
 }
 
 func TestBaseAppNewMailClient(t *testing.T) {
@@ -162,7 +185,7 @@ func TestBaseAppNewMailClient(t *testing.T) {
 		DataDir:       testDataDir,
 		EncryptionEnv: "pb_test_env",
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	client1 := app.NewMailClient()
 	m1, ok := client1.(*mailer.Sendmail)
@@ -192,7 +215,7 @@ func TestBaseAppNewFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewFilesystem()
@@ -221,7 +244,7 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	app := core.NewBaseApp(core.BaseAppConfig{
 		DataDir: testDataDir,
 	})
-	defer app.ResetBootstrapState()
+	defer app.ClearBootstrap()
 
 	// local
 	local, localErr := app.NewBackupsFilesystem()
@@ -243,68 +266,172 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
 	}
 }
 
+const logsThreshold = 200
+
+func assertLogsCount(t *testing.T, app core.App, expected int) {
+	var total int
+
+	err := app.LogQuery().Select("count(*)").Row(&total)
+	if err != nil {
+		t.Fatalf("Failed to fetch total logs: %v", err)
+	}
+
+	if total != expected {
+		t.Fatalf("Expected %d log(s), got %d", expected, total)
+	}
+}
+
 func TestBaseAppLoggerWrites(t *testing.T) {
 	t.Parallel()
 
+	// note: outside of synctest because the bootstrap tickers could deadlock
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	t.Run("disabled logs retention", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 0
+
+			for i := 0; i < logsThreshold+1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, 0)
+		})
+	})
+
+	t.Run("test batch logs writes", func(t *testing.T) {
+		synctest.Test(t, func(t *testing.T) {
+			app.Settings().Logs.MaxDays = 2
+
+			for i := 0; i < logsThreshold-1; i++ {
+				app.Logger().Error("test")
+			}
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			// below threshold
+			assertLogsCount(t, app, 0)
+
+			// threshold reached -> should trigger batch write
+			app.Logger().Error("test")
+
+			// should be skipped from this batch and added for the next
+			app.Logger().Error("test")
+
+			// short delay for the non-blocking write goroutine
+			synctest.Sleep(time.Nanosecond)
+
+			assertLogsCount(t, app, logsThreshold)
+
+			// note: we can't test the flush timer here because the ticker
+			// was started out of the synctest buble to avoid deadlocks
+			// (see TestBaseAppLoggerWritesAwaited for a flaky but real timer test)
+		})
+	})
+}
+
+func TestBaseAppLoggerWritesAwaited(t *testing.T) {
+	t.Parallel()
+
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
 
-	// reset
-	if err := app.DeleteOldLogs(time.Now()); err != nil {
+	// clear old logs
+	err := app.DeleteOldLogs(time.Now())
+	if err != nil {
 		t.Fatal(err)
 	}
 
-	const logsThreshold = 200
-
-	totalLogs := func(app core.App, t *testing.T) int {
-		var total int
-
-		err := app.LogQuery().Select("count(*)").Row(&total)
-		if err != nil {
-			t.Fatalf("Failed to fetch total logs: %v", err)
-		}
-
-		return total
+	// enable logs persistence
+	app.Settings().Logs.MaxDays = 1
+	err = app.Save(app.Settings())
+	if err != nil {
+		t.Fatal(err)
 	}
 
-	t.Run("disabled logs retention", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 0
+	t.Run("flush on timer tick", func(t *testing.T) {
+		timeout := time.After(5 * time.Second)
+		done := make(chan struct{})
 
-		for i := 0; i < logsThreshold+1; i++ {
-			app.Logger().Error("test")
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			done <- struct{}{}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.Logger().Error("test")
+
+		// short wait to ensure that there is no non-blocking write
+		time.Sleep(500 * time.Millisecond)
+
+		assertLogsCount(t, app, 0)
+
+		// wait for the ticker to write the db record
+		select {
+		case <-timeout:
+			t.Fatal("ticker wait timeout")
+		case <-done:
 		}
 
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
+		assertLogsCount(t, app, 1)
 	})
 
-	t.Run("test batch logs writes", func(t *testing.T) {
-		app.Settings().Logs.MaxDays = 1
-
-		for i := 0; i < logsThreshold-1; i++ {
-			app.Logger().Error("test")
-		}
-
-		if total := totalLogs(app, t); total != 0 {
-			t.Fatalf("Expected no logs, got %d", total)
-		}
-
-		// should trigger batch write
+	t.Run("before ClearBootstrap flush", func(t *testing.T) {
 		app.Logger().Error("test")
 
-		// should be added for the next batch write
-		app.Logger().Error("test")
+		app.Bootstrap()
 
-		if total := totalLogs(app, t); total != logsThreshold {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold, total)
+		assertLogsCount(t, app, 2)
+	})
+
+	t.Run("batch flush inside aux transaction shouldn't hang", func(t *testing.T) {
+		timeout := time.After(1 * time.Second)
+		done := make(chan struct{})
+		totalCreated := 0
+
+		logsHook := app.OnModelAfterCreateSuccess("_logs")
+		hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
+			totalCreated++
+			if totalCreated == 200 {
+				done <- struct{}{}
+			}
+			return e.Next()
+		})
+		defer logsHook.Unbind(hookId)
+
+		app.AuxRunInTransaction(func(txApp core.App) error {
+			for range logsThreshold {
+				txApp.Logger().Error("test")
+			}
+
+			return nil
+		})
+
+		// wait for the non-blocking write
+		select {
+		case <-timeout:
+			t.Fatal("non-blocking write timeout")
+		case <-done:
 		}
 
-		// wait for ~3 secs to check the timer trigger
-		time.Sleep(3200 * time.Millisecond)
-		if total := totalLogs(app, t); total != logsThreshold+1 {
-			t.Fatalf("Expected %d logs, got %d", logsThreshold+1, total)
-		}
+		assertLogsCount(t, app, 202)
+
+		// force clear to ensure that there are no other logs
+		app.Bootstrap()
+
+		assertLogsCount(t, app, 202)
 	})
 }
 
@@ -347,15 +474,15 @@ func TestBaseAppRefreshSettingsLoggerMinLevelEnabled(t *testing.T) {
 				DataDir: testDataDir,
 				IsDev:   s.isDev,
 			})
-			defer app.ResetBootstrapState()
+			defer app.ClearBootstrap()
 
 			if err := app.Bootstrap(); err != nil {
 				t.Fatal(err)
 			}
 
 			// silence query logs
-			app.DB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {}
-			app.DB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {}
+			app.ConcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {}
+			app.ConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {}
 			app.NonconcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {}
 			app.NonconcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {}
 
@@ -378,3 +505,171 @@ func TestBaseAppRefreshSettingsLoggerMinLevelEnabled(t *testing.T) {
 		})
 	}
 }
+
+func TestBaseAppDBDualBuilder(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	concurrentQueries := []string{}
+	nonconcurrentQueries := []string{}
+	app.ConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		concurrentQueries = append(concurrentQueries, sql)
+	}
+	app.ConcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		concurrentQueries = append(concurrentQueries, sql)
+	}
+	app.NonconcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		nonconcurrentQueries = append(nonconcurrentQueries, sql)
+	}
+	app.NonconcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		nonconcurrentQueries = append(nonconcurrentQueries, sql)
+	}
+
+	type testQuery struct {
+		query        string
+		isConcurrent bool
+	}
+
+	regularTests := []testQuery{
+		{"  \n  sEleCt 1", true},
+		{"With abc(x) AS (select 2) SELECT x FROM abc", true},
+		{"create table t1(x int)", false},
+		{"insert into t1(x) values(1)", false},
+		{"update t1 set x = 2", false},
+		{"delete from t1", false},
+	}
+
+	txTests := []testQuery{
+		{"select 3", false},
+		{" \n WITH abc(x) AS (select 4) SELECT x FROM abc", false},
+		{"create table t2(x int)", false},
+		{"insert into t2(x) values(1)", false},
+		{"update t2 set x = 2", false},
+		{"delete from t2", false},
+	}
+
+	for _, item := range regularTests {
+		_, err := app.DB().NewQuery(item.query).Execute()
+		if err != nil {
+			t.Fatalf("Failed to execute query %q error: %v", item.query, err)
+		}
+	}
+
+	app.RunInTransaction(func(txApp core.App) error {
+		for _, item := range txTests {
+			_, err := txApp.DB().NewQuery(item.query).Execute()
+			if err != nil {
+				t.Fatalf("Failed to execute query %q error: %v", item.query, err)
+			}
+		}
+
+		return nil
+	})
+
+	allTests := append(regularTests, txTests...)
+	for _, item := range allTests {
+		if item.isConcurrent {
+			if !slices.Contains(concurrentQueries, item.query) {
+				t.Fatalf("Expected concurrent query\n%q\ngot\nconcurrent:%v\nnonconcurrent:%v", item.query, concurrentQueries, nonconcurrentQueries)
+			}
+		} else {
+			if !slices.Contains(nonconcurrentQueries, item.query) {
+				t.Fatalf("Expected nonconcurrent query\n%q\ngot\nconcurrent:%v\nnonconcurrent:%v", item.query, concurrentQueries, nonconcurrentQueries)
+			}
+		}
+	}
+}
+
+func TestBaseAppAuxDBDualBuilder(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	concurrentQueries := []string{}
+	nonconcurrentQueries := []string{}
+	app.AuxConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		concurrentQueries = append(concurrentQueries, sql)
+	}
+	app.AuxConcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		concurrentQueries = append(concurrentQueries, sql)
+	}
+	app.AuxNonconcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		nonconcurrentQueries = append(nonconcurrentQueries, sql)
+	}
+	app.AuxNonconcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		nonconcurrentQueries = append(nonconcurrentQueries, sql)
+	}
+
+	type testQuery struct {
+		query        string
+		isConcurrent bool
+	}
+
+	regularTests := []testQuery{
+		{"  \n  sEleCt 1", true},
+		{"With abc(x) AS (select 2) SELECT x FROM abc", true},
+		{"create table t1(x int)", false},
+		{"insert into t1(x) values(1)", false},
+		{"update t1 set x = 2", false},
+		{"delete from t1", false},
+	}
+
+	txTests := []testQuery{
+		{"select 3", false},
+		{" \n WITH abc(x) AS (select 4) SELECT x FROM abc", false},
+		{"create table t2(x int)", false},
+		{"insert into t2(x) values(1)", false},
+		{"update t2 set x = 2", false},
+		{"delete from t2", false},
+	}
+
+	for _, item := range regularTests {
+		_, err := app.AuxDB().NewQuery(item.query).Execute()
+		if err != nil {
+			t.Fatalf("Failed to execute query %q error: %v", item.query, err)
+		}
+	}
+
+	app.AuxRunInTransaction(func(txApp core.App) error {
+		for _, item := range txTests {
+			_, err := txApp.AuxDB().NewQuery(item.query).Execute()
+			if err != nil {
+				t.Fatalf("Failed to execute query %q error: %v", item.query, err)
+			}
+		}
+
+		return nil
+	})
+
+	allTests := append(regularTests, txTests...)
+	for _, item := range allTests {
+		if item.isConcurrent {
+			if !slices.Contains(concurrentQueries, item.query) {
+				t.Fatalf("Expected concurrent query\n%q\ngot\nconcurrent:%v\nnonconcurrent:%v", item.query, concurrentQueries, nonconcurrentQueries)
+			}
+		} else {
+			if !slices.Contains(nonconcurrentQueries, item.query) {
+				t.Fatalf("Expected nonconcurrent query\n%q\ngot\nconcurrent:%v\nnonconcurrent:%v", item.query, concurrentQueries, nonconcurrentQueries)
+			}
+		}
+	}
+}
+
+func TestBaseAppTriggerOnTerminate(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	event := new(core.TerminateEvent)
+	event.App = app
+
+	// trigger OnTerminate multiple times to ensure that it doesn't deadlock
+	// https://github.com/pocketbase/pocketbase/pull/7305
+	app.OnTerminate().Trigger(event)
+	app.OnTerminate().Trigger(event)
+	app.OnTerminate().Trigger(event)
+}

core/collection_import.go

@@ -4,12 +4,13 @@ import (
 	"cmp"
 	"context"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/jsontext"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
 	"slices"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 	"github.com/spf13/cast"
 )
 
@@ -186,7 +187,11 @@ func (app *BaseApp) ImportCollections(toImport []map[string]any, deleteMissing b
 			)
 			if err := validator.run(); err != nil {
 				// serialize the validation error(s)
-				serializedErr, _ := json.MarshalIndent(err, "", "  ")
+				serializedErr, _ := json.Marshal(
+					err,
+					jsontext.WithIndentPrefix(""),
+					jsontext.WithIndent("  "),
+				)
 
 				return validation.Errors{"collections": validation.NewError(
 					"validation_collections_import_failure",

core/collection_import_test.go

@@ -1,7 +1,7 @@
 package core_test
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"strings"
 	"testing"
 
@@ -170,7 +170,7 @@ func TestImportCollections(t *testing.T) {
 				expectedCollectionFields := map[string]int{
 					core.CollectionNameAuthOrigins: 6,
 					"nologin":                      10,
-					"demo1":                        18,
+					"demo1":                        19,
 					"demo2":                        5,
 					"demo3":                        5,
 					"demo4":                        16,
@@ -449,7 +449,7 @@ func TestImportCollectionsCreateRules(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	raw, err := json.Marshal(collection)
+	raw, err := json.Marshal(collection, json.Deterministic(true))
 	if err != nil {
 		t.Fatal(err)
 	}

core/collection_model.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strconv"
 	"strings"
@@ -28,15 +28,20 @@ const (
 
 const systemHookIdCollection = "__pbCollectionSystemHook__"
 
+const defaultLowercaseRecordIdPattern = "^[a-z0-9]+$"
+
 func (app *BaseApp) registerCollectionHooks() {
 	app.OnModelValidate().Bind(&hook.Handler[*ModelEvent]{
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionValidate().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionValidate().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -48,10 +53,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionCreate().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionCreate().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -63,10 +71,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionCreateExecute().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionCreateExecute().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -78,10 +89,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionAfterCreateSuccess().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionAfterCreateSuccess().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -93,10 +107,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelErrorEvent) error {
 			if ce, ok := newCollectionErrorEventFromModelErrorEvent(me); ok {
-				return me.App.OnCollectionAfterCreateError().Trigger(ce, func(ce *CollectionErrorEvent) error {
+				err := me.App.OnCollectionAfterCreateError().Trigger(ce, func(ce *CollectionErrorEvent) error {
 					syncModelErrorEventWithCollectionErrorEvent(me, ce)
+					defer syncCollectionErrorEventWithModelErrorEvent(ce, me)
 					return me.Next()
 				})
+				syncModelErrorEventWithCollectionErrorEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -108,10 +125,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionUpdate().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionUpdate().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -123,10 +143,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionUpdateExecute().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionUpdateExecute().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -138,10 +161,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionAfterUpdateSuccess().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionAfterUpdateSuccess().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -153,10 +179,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelErrorEvent) error {
 			if ce, ok := newCollectionErrorEventFromModelErrorEvent(me); ok {
-				return me.App.OnCollectionAfterUpdateError().Trigger(ce, func(ce *CollectionErrorEvent) error {
+				err := me.App.OnCollectionAfterUpdateError().Trigger(ce, func(ce *CollectionErrorEvent) error {
 					syncModelErrorEventWithCollectionErrorEvent(me, ce)
+					defer syncCollectionErrorEventWithModelErrorEvent(ce, me)
 					return me.Next()
 				})
+				syncModelErrorEventWithCollectionErrorEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -168,10 +197,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionDelete().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionDelete().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -183,10 +215,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionDeleteExecute().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionDeleteExecute().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -198,10 +233,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelEvent) error {
 			if ce, ok := newCollectionEventFromModelEvent(me); ok {
-				return me.App.OnCollectionAfterDeleteSuccess().Trigger(ce, func(ce *CollectionEvent) error {
+				err := me.App.OnCollectionAfterDeleteSuccess().Trigger(ce, func(ce *CollectionEvent) error {
 					syncModelEventWithCollectionEvent(me, ce)
+					defer syncCollectionEventWithModelEvent(ce, me)
 					return me.Next()
 				})
+				syncModelEventWithCollectionEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -213,10 +251,13 @@ func (app *BaseApp) registerCollectionHooks() {
 		Id: systemHookIdCollection,
 		Func: func(me *ModelErrorEvent) error {
 			if ce, ok := newCollectionErrorEventFromModelErrorEvent(me); ok {
-				return me.App.OnCollectionAfterDeleteError().Trigger(ce, func(ce *CollectionErrorEvent) error {
+				err := me.App.OnCollectionAfterDeleteError().Trigger(ce, func(ce *CollectionErrorEvent) error {
 					syncModelErrorEventWithCollectionErrorEvent(me, ce)
+					defer syncCollectionErrorEventWithModelErrorEvent(ce, me)
 					return me.Next()
 				})
+				syncModelErrorEventWithCollectionErrorEvent(me, ce)
+				return err
 			}
 
 			return me.Next()
@@ -307,6 +348,7 @@ func (app *BaseApp) registerCollectionHooks() {
 }
 
 // @todo experiment eventually replacing the rules *string with a struct?
+// @todo consider changing the Indexes field to a "getter" for the sqlite_master table?
 type baseCollection struct {
 	BaseModel
 
@@ -480,8 +522,6 @@ func (m *Collection) unmarshalRawOptions() error {
 // For new/"blank" Collection models it replaces the model with a factory
 // instance and then unmarshal the provided data one on top of it.
 func (m *Collection) UnmarshalJSON(b []byte) error {
-	type alias *Collection
-
 	// initialize the default fields
 	// (e.g. in case the collection was NOT created using the designated factories)
 	if m.IsNew() && m.Type == "" {
@@ -498,50 +538,60 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
 		*m = *blank
 	}
 
-	return json.Unmarshal(b, alias(m))
+	type alias Collection
+	return json.Unmarshal(b, (*alias)(m))
 }
 
 // MarshalJSON implements the [json.Marshaler] interface.
 //
 // Note that non-type related fields are ignored from the serialization
-// (ex. for "view" colections the "auth" fields are skipped).
+// (ex. for "view" collections the "auth" fields are skipped).
 func (m Collection) MarshalJSON() ([]byte, error) {
 	switch m.Type {
 	case CollectionTypeView:
-		return json.Marshal(struct {
+		alias := struct {
 			baseCollection
 			collectionViewOptions
-		}{m.baseCollection, m.collectionViewOptions})
+		}{m.baseCollection, m.collectionViewOptions}
+
+		return json.Marshal(alias, json.Deterministic(true))
 	case CollectionTypeAuth:
 		alias := struct {
 			baseCollection
 			collectionAuthOptions
 		}{m.baseCollection, m.collectionAuthOptions}
 
-		// ensure that it is always returned as array
-		if alias.OAuth2.Providers == nil {
-			alias.OAuth2.Providers = []OAuth2ProviderConfig{}
-		}
-
+		// @todo to avoid the below changes consider omitting the field values from the individual structs json tags
+		//
 		// hide secret keys from the serialization
 		alias.AuthToken.Secret = ""
 		alias.FileToken.Secret = ""
 		alias.PasswordResetToken.Secret = ""
 		alias.EmailChangeToken.Secret = ""
 		alias.VerificationToken.Secret = ""
-		for i := range alias.OAuth2.Providers {
-			alias.OAuth2.Providers[i].ClientSecret = ""
+
+		if alias.OAuth2.Providers == nil {
+			// ensure that it is always returned as array
+			alias.OAuth2.Providers = []OAuth2ProviderConfig{}
+		} else {
+			// create a deep copy of the slice to avoid modifying the cached model state
+			redactedProviders := make([]OAuth2ProviderConfig, len(alias.OAuth2.Providers))
+			copy(redactedProviders, alias.OAuth2.Providers)
+			for i := range redactedProviders {
+				redactedProviders[i].ClientSecret = ""
+			}
+			alias.OAuth2.Providers = redactedProviders
 		}
 
-		return json.Marshal(alias)
+		return json.Marshal(alias, json.Deterministic(true))
 	default:
-		return json.Marshal(m.baseCollection)
+		return json.Marshal(m.baseCollection, json.Deterministic(true))
 	}
 }
 
 // String returns a string representation of the current collection.
 func (m Collection) String() string {
-	raw, _ := json.Marshal(m)
+	raw, _ := m.MarshalJSON()
 	return string(raw)
 }
 
@@ -721,9 +771,9 @@ func (c *Collection) updateGeneratedIdIfExists(app App) {
 
 	// add a number to the current id (if already exists)
 	for i := 2; i < 1000; i++ {
-		var exists bool
+		var exists int
 		_ = app.CollectionQuery().Select("(1)").AndWhere(dbx.HashExp{"id": newId}).Limit(1).Row(&exists)
-		if !exists {
+		if exists == 0 {
 			break
 		}
 		newId = c.idChecksum() + strconv.Itoa(i)
@@ -772,6 +822,25 @@ func onCollectionSave(e *CollectionEvent) error {
 
 	e.Collection.updateGeneratedIdIfExists(e.App)
 
+	// normalize indexes table name
+	for i, raw := range e.Collection.Indexes {
+		parsed := dbutils.ParseIndex(raw)
+
+		// no need to normalize
+		if parsed.TableName == e.Collection.Name {
+			continue
+		}
+
+		parsed.TableName = e.Collection.Name
+
+		normalized := parsed.Build()
+		if normalized == "" {
+			continue // leave to the model validator to decide whether to return an error
+		}
+
+		e.Collection.Indexes[i] = normalized
+	}
+
 	return e.Next()
 }
 
@@ -857,8 +926,14 @@ func onCollectionSaveExecute(e *CollectionEvent) error {
 	}
 
 	// trigger an update for all views with changed fields as a result of the current collection save
-	// (ignoring view errors to allow users to update the query from the UI)
-	resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	// (only log the error to allow users to adjust the problematic view queries from the UI)
+	depViewsErr := resaveViewsWithChangedFields(e.App, e.Collection.Id)
+	if depViewsErr != nil {
+		e.App.Logger().Warn(
+			"Dependent view collection(s) may need to be updated after "+e.Collection.Name+" collection change",
+			"error", depViewsErr,
+		)
+	}
 
 	return nil
 }
@@ -890,7 +965,7 @@ func (c *Collection) initIdField() {
 			Required:            true,
 			Min:                 15,
 			Max:                 15,
-			Pattern:             `^[a-z0-9]+$`,
+			Pattern:             defaultLowercaseRecordIdPattern,
 			AutogeneratePattern: `[a-z0-9]{15}`,
 		}
 
@@ -902,6 +977,9 @@ func (c *Collection) initIdField() {
 		field.Required = true
 		field.PrimaryKey = true
 		field.Hidden = false
+		if field.Pattern == "" {
+			field.Pattern = defaultLowercaseRecordIdPattern
+		}
 	}
 }
 
@@ -945,7 +1023,7 @@ func (c *Collection) initTokenKeyField() {
 	}
 
 	// ensure that there is a unique index for the field
-	if !dbutils.HasSingleColumnUniqueIndex(FieldNameTokenKey, c.Indexes) {
+	if _, ok := dbutils.FindSingleColumnUniqueIndex(c.Indexes, FieldNameTokenKey); !ok {
 		c.Indexes = append(c.Indexes, fmt.Sprintf(
 			"CREATE UNIQUE INDEX `%s` ON `%s` (`%s`)",
 			c.fieldIndexName(FieldNameTokenKey),
@@ -971,7 +1049,7 @@ func (c *Collection) initEmailField() {
 	}
 
 	// ensure that there is a unique index for the email field
-	if !dbutils.HasSingleColumnUniqueIndex(FieldNameEmail, c.Indexes) {
+	if _, ok := dbutils.FindSingleColumnUniqueIndex(c.Indexes, FieldNameEmail); !ok {
 		c.Indexes = append(c.Indexes, fmt.Sprintf(
 			"CREATE UNIQUE INDEX `%s` ON `%s` (`%s`) WHERE `%s` != ''",
 			c.fieldIndexName(FieldNameEmail),
@@ -1018,7 +1096,7 @@ func (c *Collection) fieldIndexName(field string) string {
 	} else if c.Name != "" {
 		name += c.Name
 	} else {
-		name += security.PseudorandomString(10)
+		name += security.PseudorandomStringWithAlphabet(10, DefaultIdAlphabet)
 	}
 
 	if len(name) > 64 {

core/collection_model_auth_options.go

@@ -1,12 +1,15 @@
 package core
 
 import (
+	"encoding/json/v2"
+	"errors"
+	"slices"
 	"strconv"
 	"strings"
 	"time"
 
-	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/go-ozzo/ozzo-validation/v4/is"
+	validation "github.com/pocketbase/ozzo-validation/v4"
+	"github.com/pocketbase/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/tools/auth"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/security"
@@ -60,7 +63,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		MFA: MFAConfig{
 			Enabled:  false,
-			Duration: 1800, // 30min
+			Duration: 600, // 10min
 		},
 		OTP: OTPConfig{
 			Enabled:       false,
@@ -70,7 +73,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		AuthToken: TokenConfig{
 			Secret:   security.RandomString(50),
-			Duration: 604800, // 7 days
+			Duration: 432000, // 5days
 		},
 		PasswordResetToken: TokenConfig{
 			Secret:   security.RandomString(50),
@@ -82,7 +85,7 @@ func (m *Collection) setDefaultAuthOptions() {
 		},
 		VerificationToken: TokenConfig{
 			Secret:   security.RandomString(50),
-			Duration: 259200, // 3days
+			Duration: 86400, // 1day
 		},
 		FileToken: TokenConfig{
 			Secret:   security.RandomString(50),
@@ -409,6 +412,71 @@ type OAuth2Config struct {
 	Enabled bool `form:"enabled" json:"enabled"`
 }
 
+// UnmarshalJSON implements the [json.Unmarshaler] interface.
+//
+// The main difference from the standard unmarshalization is that
+// instead of replacing the entire providers config slice, we ensure
+// that partially submitted provider data (e.g. without clientSecret)
+// is merged on per config level based on the provider name
+// (https://github.com/pocketbase/pocketbase/issues/7815).
+func (c *OAuth2Config) UnmarshalJSON(b []byte) error {
+	originalProviders := slices.Clone(c.Providers)
+
+	type alias OAuth2Config
+	err := json.Unmarshal(b, (*alias)(c))
+	if err != nil {
+		return err
+	}
+
+	if len(c.Providers) == 0 {
+		return nil
+	}
+
+	// unmarshal again but this time into a plain array of objects
+	// so that we have only the submitted fields and no zero defaults
+	plain := struct {
+		Providers []map[string]any `json:"providers"`
+	}{}
+	err = json.Unmarshal(b, &plain)
+	if err != nil {
+		return err
+	}
+
+	// no providers were submitted
+	if len(plain.Providers) == 0 {
+		return nil
+	}
+
+	if len(c.Providers) != len(plain.Providers) {
+		return errors.New("the length of the plain unmarshalized providers and the ones from the config doesn't match")
+	}
+
+ProvidersMergeLoop:
+	for i, plain := range plain.Providers {
+		for _, original := range originalProviders {
+			if original.Name == plain["name"] {
+				raw, err := json.Marshal(plain)
+				if err != nil {
+					return err
+				}
+
+				// unmarshal the new plain data on top of the original one
+				err = json.Unmarshal(raw, &original)
+				if err != nil {
+					return err
+				}
+
+				// reassigne to the updated original
+				c.Providers[i] = original
+
+				continue ProvidersMergeLoop
+			}
+		}
+	}
+
+	return nil
+}
+
 // GetProviderConfig returns the first OAuth2ProviderConfig that matches the specified name.
 //
 // Returns false and zero config if no such provider is available in c.Providers.
@@ -445,7 +513,7 @@ func checkForDuplicatedProviders(value any) error {
 		if _, ok := existing[c.Name]; ok {
 			return validation.Errors{
 				strconv.Itoa(i): validation.Errors{
-					"name": validation.NewError("validation_duplicated_provider", "The provider "+c.Name+" is already registered.").
+					"name": validation.NewError("validation_duplicated_provider", "The provider {{.name}} is already registered.").
 						SetParams(map[string]any{"name": c.Name}),
 				},
 			}
@@ -493,7 +561,7 @@ func checkProviderName(value any) error {
 	}
 
 	if _, err := auth.NewProviderByName(name); err != nil {
-		return validation.NewError("validation_missing_provider", "Invalid or missing provider with name "+name+".").
+		return validation.NewError("validation_missing_provider", "Invalid or missing provider with name {{.name}}.").
 			SetParams(map[string]any{"name": name})
 	}
 

core/collection_model_auth_options_test.go

@@ -2,7 +2,7 @@ package core_test
 
 import (
 	"bytes"
-	"encoding/json"
+	"encoding/json/v2"
 	"fmt"
 	"strings"
 	"testing"
@@ -712,6 +712,76 @@ func TestPasswordAuthConfigValidate(t *testing.T) {
 	}
 }
 
+func TestOAuth2ConfigUnmarshalJSON(t *testing.T) {
+	t.Parallel()
+
+	scenarios := []struct {
+		name     string
+		newJSON  string
+		expected string
+	}{
+		{
+			"missing",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"}
+			}`,
+			`{"providers":[{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"b","clientId":"b_clientId","clientSecret":"b_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": []
+			}`,
+			`{"providers":[],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+		{
+			"non-empty",
+			`{
+				"enabled": true,
+				"mappedFields": {"username": "username_test"},
+				"providers": [
+					{"name": "c", "clientId": "c_clientId", "clientSecret": "c_clientSecret"},
+					{"name": "a", "displayName": "a_displayName"}
+				]
+			}`,
+			`{"providers":[{"pkce":null,"name":"c","clientId":"c_clientId","clientSecret":"c_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"a_displayName","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
+		},
+	}
+
+	for _, s := range scenarios {
+		t.Run(s.name, func(t *testing.T) {
+			config := core.OAuth2Config{
+				Enabled: false,
+				MappedFields: core.OAuth2KnownFields{
+					Name: "name_test",
+				},
+				Providers: []core.OAuth2ProviderConfig{
+					{Name: "a", ClientId: "a_clientId", ClientSecret: "a_clientSecret"},
+					{Name: "b", ClientId: "b_clientId", ClientSecret: "b_clientSecret"},
+				},
+			}
+
+			err := json.Unmarshal([]byte(s.newJSON), &config)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			raw, err := json.Marshal(config, json.Deterministic(true))
+			if err != nil {
+				t.Fatal(err)
+			}
+			rawStr := string(raw)
+
+			if rawStr != s.expected {
+				t.Fatalf("Expected OAuth2ProviderConfig\n%s\ngot\n%s", s.expected, rawStr)
+			}
+		})
+	}
+}
+
 func TestOAuth2ConfigGetProviderConfig(t *testing.T) {
 	scenarios := []struct {
 		name           string
@@ -1016,8 +1086,8 @@ func TestOAuth2ProviderConfigInitProvider(t *testing.T) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}
 
-			rawMeta, _ := json.Marshal(provider.Extra())
-			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra)
+			rawMeta, _ := json.Marshal(provider.Extra(), json.Deterministic(true))
+			expectedMeta, _ := json.Marshal(s.expectedConfig.Extra, json.Deterministic(true))
 			if !bytes.Equal(rawMeta, expectedMeta) {
 				t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
 			}

core/collection_model_auth_templates.go

@@ -2,11 +2,12 @@ package core
 
 // Common settings placeholder tokens
 const (
-	EmailPlaceholderAppName string = "{APP_NAME}"
-	EmailPlaceholderAppURL  string = "{APP_URL}"
-	EmailPlaceholderToken   string = "{TOKEN}"
-	EmailPlaceholderOTP     string = "{OTP}"
-	EmailPlaceholderOTPId   string = "{OTP_ID}"
+	EmailPlaceholderAppName   string = "{APP_NAME}"
+	EmailPlaceholderAppURL    string = "{APP_URL}"
+	EmailPlaceholderToken     string = "{TOKEN}"
+	EmailPlaceholderOTP       string = "{OTP}"
+	EmailPlaceholderOTPId     string = "{OTP_ID}"
+	EmailPlaceholderAlertInfo string = "{ALERT_INFO}"
 )
 
 var defaultVerificationTemplate = EmailTemplate{
@@ -17,6 +18,7 @@ var defaultVerificationTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-verification/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Verify</a>
 </p>
+<p><i>If you didn't recently register, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team
@@ -30,7 +32,7 @@ var defaultResetPasswordTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-password-reset/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Reset password</a>
 </p>
-<p><i>If you didn't ask to reset your password, you can ignore this email.</i></p>
+<p><i>If you didn't ask to reset your password, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team
@@ -44,7 +46,7 @@ var defaultConfirmEmailChangeTemplate = EmailTemplate{
 <p>
   <a class="btn" href="` + EmailPlaceholderAppURL + "/_/#/auth/confirm-email-change/" + EmailPlaceholderToken + `" target="_blank" rel="noopener">Confirm new email</a>
 </p>
-<p><i>If you didn't ask to change your email address, you can ignore this email.</i></p>
+<p><i>If you didn't ask to change your email address, please ignore this email.</i></p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team
@@ -65,9 +67,10 @@ var defaultOTPTemplate = EmailTemplate{
 var defaultAuthAlertTemplate = EmailTemplate{
 	Subject: "Login from a new location",
 	Body: `<p>Hello,</p>
-<p>We noticed a login to your ` + EmailPlaceholderAppName + ` account from a new location.</p>
-<p>If this was you, you may disregard this email.</p>
+<p>We noticed a login to your ` + EmailPlaceholderAppName + ` account from a new location:</p>
+<p><em>` + EmailPlaceholderAlertInfo + `</em></p>
 <p><strong>If this wasn't you, you should immediately change your ` + EmailPlaceholderAppName + ` account password to revoke access from all other locations.</strong></p>
+<p>If this was you, you may disregard this email.</p>
 <p>
   Thanks,<br/>
   ` + EmailPlaceholderAppName + ` team

core/collection_model_test.go

@@ -1,9 +1,12 @@
 package core_test
 
 import (
-	"encoding/json"
+	"context"
+	"encoding/json/v2"
+	"errors"
 	"fmt"
 	"slices"
+	"strconv"
 	"strings"
 	"testing"
 
@@ -11,6 +14,7 @@ import (
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/dbutils"
+	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/types"
 )
 
@@ -614,7 +618,7 @@ func TestCollectionUnmarshalJSON(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			rawResult, err := json.Marshal(collection)
+			rawResult, err := json.Marshal(collection, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -756,6 +760,46 @@ func TestCollectionSerialize(t *testing.T) {
 	}
 }
 
+func TestCollectionSerializeNotModifyingCache(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	c, err := app.FindCachedCollectionByNameOrId("users")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	_, err = json.Marshal(c)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	redactedFields := map[string]string{
+		"AuthToken.Secret":          c.AuthToken.Secret,
+		"FileToken.Secret":          c.FileToken.Secret,
+		"PasswordResetToken.Secret": c.PasswordResetToken.Secret,
+		"EmailChangeToken.Secret":   c.EmailChangeToken.Secret,
+		"VerificationToken.Secret":  c.VerificationToken.Secret,
+	}
+
+	if len(c.OAuth2.Providers) == 0 {
+		t.Fatal("Expected at least one users OAuth2 provider, got 0")
+	}
+	for _, p := range c.OAuth2.Providers {
+		redactedFields[p.Name+".ClientSecret"] = p.ClientSecret
+	}
+
+	for k, v := range redactedFields {
+		t.Run(k, func(t *testing.T) {
+			if v == "" {
+				t.Fatalf("Expected the redacted field %q to remain unmodified after serialization, got empty value", k)
+			}
+		})
+	}
+}
+
 func TestCollectionDBExport(t *testing.T) {
 	t.Parallel()
 
@@ -773,19 +817,19 @@ func TestCollectionDBExport(t *testing.T) {
 	}{
 		{
 			"unknown",
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":"{}","system":true,"type":"unknown","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":"{}","system":true,"type":"unknown","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 		{
 			core.CollectionTypeBase,
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":"{}","system":true,"type":"base","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":"{}","system":true,"type":"base","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 		{
 			core.CollectionTypeView,
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"viewQuery":"select 1"},"system":true,"type":"view","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"viewQuery":"select 1"},"system":true,"type":"view","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 		{
 			core.CollectionTypeAuth,
-			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":null,"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":null},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
+			`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":[],"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":[]},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
 		},
 	}
 
@@ -816,7 +860,7 @@ func TestCollectionDBExport(t *testing.T) {
 				t.Fatal(err)
 			}
 
-			raw, err := json.Marshal(result)
+			raw, err := json.Marshal(result, json.Deterministic(true))
 			if err != nil {
 				t.Fatal(err)
 			}
@@ -976,6 +1020,280 @@ func TestCollectionDelete(t *testing.T) {
 	}
 }
 
+func TestCollectionModelEventSync(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	testCollections := make([]*core.Collection, 4)
+	for i := 0; i < 4; i++ {
+		testCollections[i] = core.NewBaseCollection("sync_test_" + strconv.Itoa(i))
+		if err := app.Save(testCollections[i]); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	createModelEvent := func() *core.ModelEvent {
+		event := new(core.ModelEvent)
+		event.App = app
+		event.Context = context.Background()
+		event.Type = "test_a"
+		event.Model = testCollections[0]
+		return event
+	}
+
+	createModelErrorEvent := func() *core.ModelErrorEvent {
+		event := new(core.ModelErrorEvent)
+		event.ModelEvent = *createModelEvent()
+		event.Error = errors.New("error_a")
+		return event
+	}
+
+	changeCollectionEventBefore := func(e *core.CollectionEvent) {
+		e.Type = "test_b"
+		//nolint:staticcheck
+		e.Context = context.WithValue(context.Background(), "test", 123)
+		e.Collection = testCollections[1]
+	}
+
+	modelEventFinalizerChange := func(e *core.ModelEvent) {
+		e.Type = "test_c"
+		//nolint:staticcheck
+		e.Context = context.WithValue(context.Background(), "test", 456)
+		e.Model = testCollections[2]
+	}
+
+	changeCollectionEventAfter := func(e *core.CollectionEvent) {
+		e.Type = "test_d"
+		//nolint:staticcheck
+		e.Context = context.WithValue(context.Background(), "test", 789)
+		e.Collection = testCollections[3]
+	}
+
+	expectedBeforeModelEventHandlerChecks := func(t *testing.T, e *core.ModelEvent) {
+		if e.Type != "test_a" {
+			t.Fatalf("Expected type %q, got %q", "test_a", e.Type)
+		}
+
+		if v := e.Context.Value("test"); v != nil {
+			t.Fatalf("Expected context value %v, got %v", nil, v)
+		}
+
+		if e.Model.PK() != testCollections[0].Id {
+			t.Fatalf("Expected collection with id %q, got %q (%d)", testCollections[0].Id, e.Model.PK(), 0)
+		}
+	}
+
+	expectedAfterModelEventHandlerChecks := func(t *testing.T, e *core.ModelEvent) {
+		if e.Type != "test_d" {
+			t.Fatalf("Expected type %q, got %q", "test_d", e.Type)
+		}
+
+		if v := e.Context.Value("test"); v != 789 {
+			t.Fatalf("Expected context value %v, got %v", 789, v)
+		}
+
+		if e.Model.PK() != testCollections[3].Id {
+			t.Fatalf("Expected collection with id %q, got %q (%d)", testCollections[3].Id, e.Model.PK(), 3)
+		}
+	}
+
+	expectedBeforeCollectionEventHandlerChecks := func(t *testing.T, e *core.CollectionEvent) {
+		if e.Type != "test_a" {
+			t.Fatalf("Expected type %q, got %q", "test_a", e.Type)
+		}
+
+		if v := e.Context.Value("test"); v != nil {
+			t.Fatalf("Expected context value %v, got %v", nil, v)
+		}
+
+		if e.Collection.Id != testCollections[0].Id {
+			t.Fatalf("Expected collection with id %q, got %q (%d)", testCollections[0].Id, e.Collection.Id, 0)
+		}
+	}
+
+	expectedAfterCollectionEventHandlerChecks := func(t *testing.T, e *core.CollectionEvent) {
+		if e.Type != "test_c" {
+			t.Fatalf("Expected type %q, got %q", "test_c", e.Type)
+		}
+
+		if v := e.Context.Value("test"); v != 456 {
+			t.Fatalf("Expected context value %v, got %v", 456, v)
+		}
+
+		if e.Collection.Id != testCollections[2].Id {
+			t.Fatalf("Expected collection with id %q, got %q (%d)", testCollections[2].Id, e.Collection.Id, 2)
+		}
+	}
+
+	modelEventFinalizer := func(e *core.ModelEvent) error {
+		modelEventFinalizerChange(e)
+		return nil
+	}
+
+	modelErrorEventFinalizer := func(e *core.ModelErrorEvent) error {
+		modelEventFinalizerChange(&e.ModelEvent)
+		e.Error = errors.New("error_c")
+		return nil
+	}
+
+	modelEventHandler := &hook.Handler[*core.ModelEvent]{
+		Priority: -999,
+		Func: func(e *core.ModelEvent) error {
+			t.Run("before model", func(t *testing.T) {
+				expectedBeforeModelEventHandlerChecks(t, e)
+			})
+
+			_ = e.Next()
+
+			t.Run("after model", func(t *testing.T) {
+				expectedAfterModelEventHandlerChecks(t, e)
+			})
+
+			return nil
+		},
+	}
+
+	modelErrorEventHandler := &hook.Handler[*core.ModelErrorEvent]{
+		Priority: -999,
+		Func: func(e *core.ModelErrorEvent) error {
+			t.Run("before model error", func(t *testing.T) {
+				expectedBeforeModelEventHandlerChecks(t, &e.ModelEvent)
+				if v := e.Error.Error(); v != "error_a" {
+					t.Fatalf("Expected error %q, got %q", "error_a", v)
+				}
+			})
+
+			_ = e.Next()
+
+			t.Run("after model error", func(t *testing.T) {
+				expectedAfterModelEventHandlerChecks(t, &e.ModelEvent)
+				if v := e.Error.Error(); v != "error_d" {
+					t.Fatalf("Expected error %q, got %q", "error_d", v)
+				}
+			})
+
+			return nil
+		},
+	}
+
+	recordEventHandler := &hook.Handler[*core.CollectionEvent]{
+		Priority: -999,
+		Func: func(e *core.CollectionEvent) error {
+			t.Run("before collection", func(t *testing.T) {
+				expectedBeforeCollectionEventHandlerChecks(t, e)
+			})
+
+			changeCollectionEventBefore(e)
+
+			_ = e.Next()
+
+			t.Run("after collection", func(t *testing.T) {
+				expectedAfterCollectionEventHandlerChecks(t, e)
+			})
+
+			changeCollectionEventAfter(e)
+
+			return nil
+		},
+	}
+
+	collectionErrorEventHandler := &hook.Handler[*core.CollectionErrorEvent]{
+		Priority: -999,
+		Func: func(e *core.CollectionErrorEvent) error {
+			t.Run("before collection error", func(t *testing.T) {
+				expectedBeforeCollectionEventHandlerChecks(t, &e.CollectionEvent)
+				if v := e.Error.Error(); v != "error_a" {
+					t.Fatalf("Expected error %q, got %q", "error_c", v)
+				}
+			})
+
+			changeCollectionEventBefore(&e.CollectionEvent)
+			e.Error = errors.New("error_b")
+
+			_ = e.Next()
+
+			t.Run("after collection error", func(t *testing.T) {
+				expectedAfterCollectionEventHandlerChecks(t, &e.CollectionEvent)
+				if v := e.Error.Error(); v != "error_c" {
+					t.Fatalf("Expected error %q, got %q", "error_c", v)
+				}
+			})
+
+			changeCollectionEventAfter(&e.CollectionEvent)
+			e.Error = errors.New("error_d")
+
+			return nil
+		},
+	}
+
+	// OnModelValidate
+	app.OnCollectionValidate().Bind(recordEventHandler)
+	app.OnModelValidate().Bind(modelEventHandler)
+	app.OnModelValidate().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelCreate
+	app.OnCollectionCreate().Bind(recordEventHandler)
+	app.OnModelCreate().Bind(modelEventHandler)
+	app.OnModelCreate().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelCreateExecute
+	app.OnCollectionCreateExecute().Bind(recordEventHandler)
+	app.OnModelCreateExecute().Bind(modelEventHandler)
+	app.OnModelCreateExecute().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterCreateSuccess
+	app.OnCollectionAfterCreateSuccess().Bind(recordEventHandler)
+	app.OnModelAfterCreateSuccess().Bind(modelEventHandler)
+	app.OnModelAfterCreateSuccess().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterCreateError
+	app.OnCollectionAfterCreateError().Bind(collectionErrorEventHandler)
+	app.OnModelAfterCreateError().Bind(modelErrorEventHandler)
+	app.OnModelAfterCreateError().Trigger(createModelErrorEvent(), modelErrorEventFinalizer)
+
+	// OnModelUpdate
+	app.OnCollectionUpdate().Bind(recordEventHandler)
+	app.OnModelUpdate().Bind(modelEventHandler)
+	app.OnModelUpdate().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelUpdateExecute
+	app.OnCollectionUpdateExecute().Bind(recordEventHandler)
+	app.OnModelUpdateExecute().Bind(modelEventHandler)
+	app.OnModelUpdateExecute().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterUpdateSuccess
+	app.OnCollectionAfterUpdateSuccess().Bind(recordEventHandler)
+	app.OnModelAfterUpdateSuccess().Bind(modelEventHandler)
+	app.OnModelAfterUpdateSuccess().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterUpdateError
+	app.OnCollectionAfterUpdateError().Bind(collectionErrorEventHandler)
+	app.OnModelAfterUpdateError().Bind(modelErrorEventHandler)
+	app.OnModelAfterUpdateError().Trigger(createModelErrorEvent(), modelErrorEventFinalizer)
+
+	// OnModelDelete
+	app.OnCollectionDelete().Bind(recordEventHandler)
+	app.OnModelDelete().Bind(modelEventHandler)
+	app.OnModelDelete().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelDeleteExecute
+	app.OnCollectionDeleteExecute().Bind(recordEventHandler)
+	app.OnModelDeleteExecute().Bind(modelEventHandler)
+	app.OnModelDeleteExecute().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterDeleteSuccess
+	app.OnCollectionAfterDeleteSuccess().Bind(recordEventHandler)
+	app.OnModelAfterDeleteSuccess().Bind(modelEventHandler)
+	app.OnModelAfterDeleteSuccess().Trigger(createModelEvent(), modelEventFinalizer)
+
+	// OnModelAfterDeleteError
+	app.OnCollectionAfterDeleteError().Bind(collectionErrorEventHandler)
+	app.OnModelAfterDeleteError().Bind(modelErrorEventHandler)
+	app.OnModelAfterDeleteError().Trigger(createModelErrorEvent(), modelErrorEventFinalizer)
+}
+
 func TestCollectionSaveModel(t *testing.T) {
 	t.Parallel()
 
@@ -1258,60 +1576,62 @@ func TestCollectionSaveViewWrapping(t *testing.T) {
 
 	viewName := "test_wrapping"
 
+	// note: some of the queries use "limit 0" because the tested field value could be empty
+	// which will trigger the extra sample records validation that are not important for this test
 	scenarios := []struct {
 		name     string
 		query    string
 		expected string
 	}{
 		{
-			"no wrapping - text field",
-			"select text as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select text as id, bool from demo1)",
+			"no wrapping - id field",
+			"select id, bool from demo1",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select id, bool from demo1)",
 		},
 		{
-			"no wrapping - id field",
-			"select text as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select text as id, bool from demo1)",
+			"no wrapping - text field",
+			"select text as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select text as id, bool from demo1 limit 0)",
 		},
 		{
 			"no wrapping - relation field",
-			"select rel_one as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select rel_one as id, bool from demo1)",
+			"select rel_one as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select rel_one as id, bool from demo1 limit 0)",
 		},
 		{
 			"no wrapping - select field",
-			"select select_many as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select select_many as id, bool from demo1)",
+			"select select_many as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select select_many as id, bool from demo1 limit 0)",
 		},
 		{
 			"no wrapping - email field",
-			"select email as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select email as id, bool from demo1)",
+			"select email as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select email as id, bool from demo1 limit 0)",
 		},
 		{
 			"no wrapping - datetime field",
-			"select datetime as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select datetime as id, bool from demo1)",
+			"select datetime as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select datetime as id, bool from demo1 limit 0)",
 		},
 		{
 			"no wrapping - url field",
-			"select url as id, bool from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select url as id, bool from demo1)",
+			"select url as id, bool from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (select url as id, bool from demo1 limit 0)",
 		},
 		{
 			"wrapping - bool field",
-			"select bool as id, text as txt, url from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT CAST(`id` as TEXT) `id`,`txt`,`url` FROM (select bool as id, text as txt, url from demo1))",
+			"select bool as id, text as txt, url from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT CAST(`id` as TEXT) `id`,`txt`,`url` FROM (select bool as id, text as txt, url from demo1 limit 0))",
 		},
 		{
 			"wrapping - bool field (different order)",
-			"select text as txt, url, bool as id from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT `txt`,`url`,CAST(`id` as TEXT) `id` FROM (select text as txt, url, bool as id from demo1))",
+			"select text as txt, url, bool as id from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT `txt`,`url`,CAST(`id` as TEXT) `id` FROM (select text as txt, url, bool as id from demo1 limit 0))",
 		},
 		{
 			"wrapping - json field",
-			"select json as id, text, url from demo1",
-			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT CAST(`id` as TEXT) `id`,`text`,`url` FROM (select json as id, text, url from demo1))",
+			"select json as id, text, url from demo1 limit 0",
+			"CREATE VIEW `test_wrapping` AS SELECT * FROM (SELECT CAST(`id` as TEXT) `id`,`text`,`url` FROM (select json as id, text, url from demo1 limit 0))",
 		},
 		{
 			"wrapping - numeric id",
@@ -1345,7 +1665,7 @@ func TestCollectionSaveViewWrapping(t *testing.T) {
 
 			var sql string
 
-			rowErr := app.DB().NewQuery("SELECT sql FROM sqlite_master WHERE type='view' AND name={:name}").
+			rowErr := app.ConcurrentDB().NewQuery("SELECT sql FROM sqlite_master WHERE type='view' AND name={:name}").
 				Bind(dbx.Params{"name": viewName}).
 				Row(&sql)
 			if rowErr != nil {
@@ -1358,3 +1678,40 @@ func TestCollectionSaveViewWrapping(t *testing.T) {
 		})
 	}
 }
+
+func TestCollectionSaveIndexesTableNameNormalization(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	dummyCollection := core.NewBaseCollection("new_test")
+	dummyCollection.Fields.Add(&core.TextField{Name: "test"})
+	dummyCollection.Indexes = []string{
+		"create index `new_test_idx1` on `` (`test`) where 1=1",
+		"create index `new_test_idx2` on `test` (`test`) where 1=2",
+		"create index `new_test_idx3` on `someting_else` (`test`) where 1=3",
+	}
+
+	err := app.Save(dummyCollection)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// refetch a clean state
+	dummyCollection, err = app.FindCollectionByNameOrId(dummyCollection.Name)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if len(dummyCollection.Indexes) != 3 {
+		t.Fatalf("Expected 3 indexes, got %v", dummyCollection.Indexes)
+	}
+
+	for _, raw := range dummyCollection.Indexes {
+		parsed := dbutils.ParseIndex(raw)
+		if parsed.TableName != dummyCollection.Name {
+			t.Fatalf("Expected all indexes to have tableName %q, found %q:\n%s", dummyCollection.Name, parsed.TableName, raw)
+		}
+	}
+}

core/collection_model_view_options.go

@@ -1,7 +1,7 @@
 package core
 
 import (
-	validation "github.com/go-ozzo/ozzo-validation/v4"
+	validation "github.com/pocketbase/ozzo-validation/v4"
 )
 
 var _ optionsValidator = (*collectionViewOptions)(nil)

core/collection_model_view_options_test.go

@@ -41,6 +41,24 @@ func TestCollectionViewOptionsValidate(t *testing.T) {
 			},
 			expectedErrors: []string{"fields", "viewQuery"},
 		},
+		{
+			name: "view with valid query but empty sample id",
+			collection: func(app core.App) (*core.Collection, error) {
+				c := core.NewViewCollection("new_auth")
+				c.ViewQuery = "select '' as id"
+				return c, nil
+			},
+			expectedErrors: []string{"viewQuery"},
+		},
+		{
+			name: "view with valid query but duplicated sample id",
+			collection: func(app core.App) (*core.Collection, error) {
+				c := core.NewViewCollection("new_auth")
+				c.ViewQuery = "(select 'a' as id union all select 'a' as id union all select 'c' as id)"
+				return c, nil
+			},
+			expectedErrors: []string{"viewQuery"},
+		},
 		{
 			name: "view with valid query",
 			collection: func(app core.App) (*core.Collection, error) {

core/collection_query.go

@@ -3,9 +3,10 @@ package core
 import (
 	"bytes"
 	"database/sql"
-	"encoding/json"
+	"encoding/json/v2"
 	"errors"
 	"fmt"
+	"slices"
 	"strings"
 
 	"github.com/pocketbase/dbx"
@@ -37,7 +38,7 @@ func (app *BaseApp) FindAllCollections(collectionTypes ...string) ([]*Collection
 		q.AndWhere(dbx.In("type", list.ToInterfaceSlice(types)...))
 	}
 
-	err := q.OrderBy("created ASC").All(&collections)
+	err := q.OrderBy("rowid ASC").All(&collections)
 	if err != nil {
 		return nil, err
 	}
@@ -75,16 +76,16 @@ func (app *BaseApp) FindCollectionByNameOrId(nameOrId string) (*Collection, erro
 	return m, nil
 }
 
-// FindCachedCollectionByNameOrId is similar to [App.FindCollectionByNameOrId]
+// FindCachedCollectionByNameOrId is similar to [BaseApp.FindCollectionByNameOrId]
 // but retrieves the Collection from the app cache instead of making a db call.
 //
 // NB! This method is suitable for read-only Collection operations.
 //
 // Returns [sql.ErrNoRows] if no Collection is found for consistency
-// with the [App.FindCollectionByNameOrId] method.
+// with the [BaseApp.FindCollectionByNameOrId] method.
 //
 // If you plan making changes to the returned Collection model,
-// use [App.FindCollectionByNameOrId] instead.
+// use [BaseApp.FindCollectionByNameOrId] instead.
 //
 // Caveats:
 //
@@ -94,7 +95,7 @@ func (app *BaseApp) FindCollectionByNameOrId(nameOrId string) (*Collection, erro
 //   - If you are updating a Collection in a transaction and then call this method before commit,
 //     it'll return the cached Collection state and not the one from the uncommitted transaction.
 //   - The cache is automatically updated on collections db change (create/update/delete).
-//     To manually reload the cache you can call [App.ReloadCachedCollections()]
+//     To manually reload the cache you can call [BaseApp.ReloadCachedCollections].
 func (app *BaseApp) FindCachedCollectionByNameOrId(nameOrId string) (*Collection, error) {
 	collections, _ := app.Store().Get(StoreKeyCachedCollections).([]*Collection)
 	if collections == nil {
@@ -111,30 +112,6 @@ func (app *BaseApp) FindCachedCollectionByNameOrId(nameOrId string) (*Collection
 	return nil, sql.ErrNoRows
 }
 
-// IsCollectionNameUnique checks that there is no existing collection
-// with the provided name (case insensitive!).
-//
-// Note: case insensitive check because the name is used also as
-// table name for the records.
-func (app *BaseApp) IsCollectionNameUnique(name string, excludeIds ...string) bool {
-	if name == "" {
-		return false
-	}
-
-	query := app.CollectionQuery().
-		Select("count(*)").
-		AndWhere(dbx.NewExp("LOWER([[name]])={:name}", dbx.Params{"name": strings.ToLower(name)})).
-		Limit(1)
-
-	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
-		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(uniqueExcludeIds)...))
-	}
-
-	var exists bool
-
-	return query.Row(&exists) == nil && !exists
-}
-
 // FindCollectionReferences returns information for all relation fields
 // referencing the provided collection.
 //
@@ -168,6 +145,72 @@ func (app *BaseApp) FindCollectionReferences(collection *Collection, excludeIds
 	return result, nil
 }
 
+// FindCachedCollectionReferences is similar to [BaseApp.FindCollectionReferences]
+// but retrieves the Collection from the app cache instead of making a db call.
+//
+// NB! This method is suitable for read-only Collection operations.
+//
+// If you plan making changes to the returned Collection model,
+// use [BaseApp.FindCollectionReferences] instead.
+//
+// Caveats:
+//
+//   - The returned Collection should be used only for read-only operations.
+//     Avoid directly modifying the returned cached Collection as it will affect
+//     the global cached value even if you don't persist the changes in the database!
+//   - If you are updating a Collection in a transaction and then call this method before commit,
+//     it'll return the cached Collection state and not the one from the uncommitted transaction.
+//   - The cache is automatically updated on collections db change (create/update/delete).
+//     To manually reload the cache you can call [BaseApp.ReloadCachedCollections].
+func (app *BaseApp) FindCachedCollectionReferences(collection *Collection, excludeIds ...string) (map[*Collection][]Field, error) {
+	collections, _ := app.Store().Get(StoreKeyCachedCollections).([]*Collection)
+	if collections == nil {
+		// cache is not initialized yet (eg. run in a system migration)
+		return app.FindCollectionReferences(collection, excludeIds...)
+	}
+
+	result := map[*Collection][]Field{}
+
+	for _, c := range collections {
+		if slices.Contains(excludeIds, c.Id) {
+			continue
+		}
+
+		for _, rawField := range c.Fields {
+			f, ok := rawField.(*RelationField)
+			if ok && f.CollectionId == collection.Id {
+				result[c] = append(result[c], f)
+			}
+		}
+	}
+
+	return result, nil
+}
+
+// IsCollectionNameUnique checks that there is no existing collection
+// with the provided name (case insensitive!).
+//
+// Note: case insensitive check because the name is used also as
+// table name for the records.
+func (app *BaseApp) IsCollectionNameUnique(name string, excludeIds ...string) bool {
+	if name == "" {
+		return false
+	}
+
+	query := app.CollectionQuery().
+		Select("count(*)").
+		AndWhere(dbx.NewExp("LOWER([[name]])={:name}", dbx.Params{"name": strings.ToLower(name)})).
+		Limit(1)
+
+	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
+		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(uniqueExcludeIds)...))
+	}
+
+	var total int
+
+	return query.Row(&total) == nil && total == 0
+}
+
 // TruncateCollection deletes all records associated with the provided collection.
 //
 // The truncate operation is executed in a single transaction,
@@ -177,7 +220,7 @@ func (app *BaseApp) FindCollectionReferences(collection *Collection, excludeIds
 // cascade and file delete actions.
 func (app *BaseApp) TruncateCollection(collection *Collection) error {
 	if collection.IsView() {
-		return errors.New("view collections cannot be truncated since they don't store their own records.")
+		return errors.New("view collections cannot be truncated since they don't store their own records")
 	}
 
 	return app.RunInTransaction(func(txApp App) error {
@@ -207,52 +250,6 @@ func (app *BaseApp) TruncateCollection(collection *Collection) error {
 
 // -------------------------------------------------------------------
 
-// saveViewCollection persists the provided View collection changes:
-//   - deletes the old related SQL view (if any)
-//   - creates a new SQL view with the latest newCollection.Options.Query
-//   - generates new feilds list  based on newCollection.Options.Query
-//   - updates newCollection.Fields based on the generated view table info and query
-//   - saves the newCollection
-//
-// This method returns an error if newCollection is not a "view".
-func saveViewCollection(app App, newCollection, oldCollection *Collection) error {
-	if !newCollection.IsView() {
-		return errors.New("not a view collection")
-	}
-
-	return app.RunInTransaction(func(txApp App) error {
-		query := newCollection.ViewQuery
-
-		// generate collection fields from the query
-		viewFields, err := txApp.CreateViewFields(query)
-		if err != nil {
-			return err
-		}
-
-		// delete old renamed view
-		if oldCollection != nil {
-			if err := txApp.DeleteView(oldCollection.Name); err != nil {
-				return err
-			}
-		}
-
-		// wrap view query if necessary
-		query, err = normalizeViewQueryId(txApp, query)
-		if err != nil {
-			return fmt.Errorf("failed to normalize view query id: %w", err)
-		}
-
-		// (re)create the view
-		if err := txApp.SaveView(newCollection.Name, query); err != nil {
-			return err
-		}
-
-		newCollection.Fields = viewFields
-
-		return txApp.Save(newCollection)
-	})
-}
-
 // normalizeViewQueryId wraps (if necessary) the provided view query
 // with a subselect to ensure that the id column is a text since
 // currently we don't support non-string model ids
@@ -299,50 +296,59 @@ func resaveViewsWithChangedFields(app App, excludeIds ...string) error {
 	}
 
 	return app.RunInTransaction(func(txApp App) error {
+		var collectionErrors []error
+
 		for _, collection := range collections {
 			if len(excludeIds) > 0 && list.ExistInSlice(collection.Id, excludeIds) {
 				continue
 			}
 
-			// clone the existing fields for temp modifications
-			oldFields, err := collection.Fields.Clone()
-			if err != nil {
-				return err
+			check := func() error {
+				// clone the existing fields for temp modifications
+				oldFields, err := collection.Fields.Clone()
+				if err != nil {
+					return err
+				}
+
+				// generate new fields from the query
+				newFields, err := txApp.CreateViewFields(collection.ViewQuery)
+				if err != nil {
+					return err
+				}
+
+				// unset the fields' ids to exclude from the comparison
+				for _, f := range oldFields {
+					f.SetId("")
+				}
+				for _, f := range newFields {
+					f.SetId("")
+				}
+
+				encodedNewFields, err := json.Marshal(newFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				encodedOldFields, err := json.Marshal(oldFields, json.Deterministic(true))
+				if err != nil {
+					return err
+				}
+
+				if bytes.EqualFold(encodedNewFields, encodedOldFields) {
+					return nil // no changes
+				}
+
+				return txApp.Save(collection)
 			}
 
-			// generate new fields from the query
-			newFields, err := txApp.CreateViewFields(collection.ViewQuery)
-			if err != nil {
-				return err
-			}
-
-			// unset the fields' ids to exclude from the comparison
-			for _, f := range oldFields {
-				f.SetId("")
-			}
-			for _, f := range newFields {
-				f.SetId("")
-			}
-
-			encodedNewFields, err := json.Marshal(newFields)
-			if err != nil {
-				return err
-			}
-
-			encodedOldFields, err := json.Marshal(oldFields)
-			if err != nil {
-				return err
-			}
-
-			if bytes.EqualFold(encodedNewFields, encodedOldFields) {
-				continue // no changes
-			}
-
-			if err := saveViewCollection(txApp, collection, nil); err != nil {
-				return err
+			if err := check(); err != nil {
+				collectionErrors = append(
+					collectionErrors,
+					fmt.Errorf("[%s] %w", collection.Name, err),
+				)
 			}
 		}
 
-		return nil
+		return errors.Join(collectionErrors...)
 	})
 }

core/collection_query_test.go

@@ -153,7 +153,7 @@ func TestFindCachedCollectionByNameOrId(t *testing.T) {
 	defer app.Cleanup()
 
 	totalQueries := 0
-	app.DB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+	app.ConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
 		totalQueries++
 	}
 
@@ -208,34 +208,6 @@ func TestFindCachedCollectionByNameOrId(t *testing.T) {
 	run(false)
 }
 
-func TestIsCollectionNameUnique(t *testing.T) {
-	t.Parallel()
-
-	app, _ := tests.NewTestApp()
-	defer app.Cleanup()
-
-	scenarios := []struct {
-		name      string
-		excludeId string
-		expected  bool
-	}{
-		{"", "", false},
-		{"demo1", "", false},
-		{"Demo1", "", false},
-		{"new", "", true},
-		{"demo1", "wsmn24bux7wo113", true},
-	}
-
-	for i, s := range scenarios {
-		t.Run(fmt.Sprintf("%d_%s", i, s.name), func(t *testing.T) {
-			result := app.IsCollectionNameUnique(s.name, s.excludeId)
-			if result != s.expected {
-				t.Errorf("Expected %v, got %v", s.expected, result)
-			}
-		})
-	}
-}
-
 func TestFindCollectionReferences(t *testing.T) {
 	t.Parallel()
 
@@ -288,6 +260,115 @@ func TestFindCollectionReferences(t *testing.T) {
 	}
 }
 
+func TestFindCachedCollectionReferences(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	collection, err := app.FindCollectionByNameOrId("demo3")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	totalQueries := 0
+	app.ConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		totalQueries++
+	}
+
+	run := func(withCache bool) {
+		var expectedTotalQueries int
+
+		if withCache {
+			err := app.ReloadCachedCollections()
+			if err != nil {
+				t.Fatal(err)
+			}
+		} else {
+			app.Store().Reset(nil)
+			expectedTotalQueries = 1
+		}
+
+		totalQueries = 0
+
+		result, err := app.FindCachedCollectionReferences(
+			collection,
+			collection.Id,
+			// test whether "nonempty" exclude ids condition will be skipped
+			"",
+			"",
+		)
+		if err != nil {
+			t.Fatal(err)
+		}
+
+		if len(result) != 1 {
+			t.Fatalf("Expected 1 collection, got %d: %v", len(result), result)
+		}
+
+		expectedFields := []string{
+			"rel_one_no_cascade",
+			"rel_one_no_cascade_required",
+			"rel_one_cascade",
+			"rel_one_unique",
+			"rel_many_no_cascade",
+			"rel_many_no_cascade_required",
+			"rel_many_cascade",
+			"rel_many_unique",
+		}
+
+		for col, fields := range result {
+			if col.Name != "demo4" {
+				t.Fatalf("Expected collection demo4, got %s", col.Name)
+			}
+			if len(fields) != len(expectedFields) {
+				t.Fatalf("Expected fields %v, got %v", expectedFields, fields)
+			}
+			for i, f := range fields {
+				if !slices.Contains(expectedFields, f.GetName()) {
+					t.Fatalf("[%d] Didn't expect field %v", i, f)
+				}
+			}
+		}
+
+		if totalQueries != expectedTotalQueries {
+			t.Fatalf("Expected %d totalQueries, got %d", expectedTotalQueries, totalQueries)
+		}
+	}
+
+	run(true)
+
+	run(false)
+}
+
+func TestIsCollectionNameUnique(t *testing.T) {
+	t.Parallel()
+
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	scenarios := []struct {
+		name      string
+		excludeId string
+		expected  bool
+	}{
+		{"", "", false},
+		{"demo1", "", false},
+		{"Demo1", "", false},
+		{"new", "", true},
+		{"demo1", "wsmn24bux7wo113", true},
+	}
+
+	for i, s := range scenarios {
+		t.Run(fmt.Sprintf("%d_%s", i, s.name), func(t *testing.T) {
+			result := app.IsCollectionNameUnique(s.name, s.excludeId)
+			if result != s.expected {
+				t.Errorf("Expected %v, got %v", s.expected, result)
+			}
+		})
+	}
+}
+
 func TestFindCollectionTruncate(t *testing.T) {
 	t.Parallel()