updated ui/dist

Signed-off-by: Yuxiang Gao <yuxiang-gao@outlook.com>

.github/SECURITY.md

@@ -0,0 +1,5 @@
+# Security
+
+If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
+
+All reports will be promptly addressed, and you'll be credited accordingly.

.github/workflows/release.yaml

@@ -21,7 +21,7 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v3
         with:
-          go-version: '>=1.19.4'
+          go-version: '>=1.19.5'
 
       # This step usually is not needed because the /ui/dist is pregenerated locally
       # but its here to ensure that each release embeds the latest admin ui artifacts.

CHANGELOG.md

@@ -1,3 +1,196 @@
+## v0.11.3
+
+- Fix realtime API panic on concurrent clients iteration ([#1628](https://github.com/pocketbase/pocketbase/issues/1628))
+
+  - `app.SubscriptionsBroker().Clients()` now returns a shallow copy of the underlying map.
+
+  - Added `Discard()` and `IsDiscarded()` helper methods to the `subscriptions.Client` interface.
+
+  - Slow clients should no longer "block" the main action completion.
+
+
+## v0.11.2
+
+- Fixed `fs.DeleteByPrefix()` hang on invalid S3 settings ([#1575](https://github.com/pocketbase/pocketbase/discussions/1575#discussioncomment-4661089)).
+
+- Updated file(s) delete to run in the background on record/collection delete to avoid blocking the delete model transaction.
+  _Currently the cascade files delete operation is treated as "non-critical" and in case of an error it is just logged during debug._
+  _This will be improved in the near future with the planned async job queue implementation._
+
+
+## v0.11.1
+
+- Unescaped path parameter values ([#1552](https://github.com/pocketbase/pocketbase/issues/1552)).
+
+
+## v0.11.0
+
+- Added `+` and `-` body field modifiers for `number`, `files`, `select` and `relation` fields.
+  ```js
+  {
+    // oldValue + 2
+    "someNumber+": 2,
+
+    // oldValue + ["id1", "id2"] - ["id3"]
+    "someRelation+": ["id1", "id2"],
+    "someRelation-": ["id3"],
+
+    // delete single file by its name (file fields supports only the "-" modifier!)
+    "someFile-": "filename.png",
+  }
+  ```
+  _Note1: `@request.data.someField` will contain the final resolved value._
+
+  _Note2: The old index (`"field.0":null`) and filename (`"field.filename.png":null`) based suffixed syntax for deleting files is still supported._
+
+- ! Added support for multi-match/match-all request data and collection multi-valued fields (`select`, `relation`) conditions.
+  If you want a "at least one of" type of condition, you can prefix the operator with `?`.
+  ```js
+  // for each someRelA.someRelB record require the "status" field to be "active"
+  someRelA.someRelB.status = "active"
+
+  // OR for "at least one of" condition
+  someRelA.someRelB.status ?= "active"
+  ```
+  _**Note: Previously the behavior for multi-valued fields was as the "at least one of" type.
+  The release comes with system db migration that will update your existing API rules (if needed) to preserve the compatibility.
+  If you have multi-select or multi-relation filter checks in your client-side code and want to preserve the old behavior, you'll have to prefix with `?` your operators.**_
+
+- Added support for querying `@request.data.someRelField.*` relation fields.
+  ```js
+  // example submitted data: {"someRel": "REL_RECORD_ID"}
+  @request.data.someRel.status = "active"
+  ```
+
+- Added `:isset` modifier for the static request data fields.
+  ```js
+  // prevent changing the "role" field
+  @request.data.role:isset = false
+  ```
+
+- Added `:length` modifier for the arrayable request data and collection fields (`select`, `file`, `relation`).
+  ```js
+  // example submitted data: {"someSelectField": ["val1", "val2"]}
+  @request.data.someSelectField:length = 2
+
+  // check existing record field length
+  someSelectField:length = 2
+  ```
+
+- Added `:each` modifier support for the multi-`select` request data and collection field.
+  ```js
+  // check if all selected rows has "pb_" prefix
+  roles:each ~ 'pb_%'
+  ```
+
+- Improved the Admin UI filters autocomplete.
+
+- Added `@random` sort key for `RANDOM()` sorted list results.
+
+- Added Strava OAuth2 provider ([#1443](https://github.com/pocketbase/pocketbase/pull/1443); thanks @szsascha).
+
+- Added Gitee OAuth2 provider ([#1448](https://github.com/pocketbase/pocketbase/pull/1448); thanks @yuxiang-gao).
+
+- Added IME status check to the textarea keydown handler ([#1370](https://github.com/pocketbase/pocketbase/pull/1370); thanks @tenthree).
+
+- Added `filesystem.NewFileFromBytes()` helper ([#1420](https://github.com/pocketbase/pocketbase/pull/1420); thanks @dschissler).
+
+- Added support for reordering uploaded multiple files.
+
+- Added `webp` to the default images mime type presets list ([#1469](https://github.com/pocketbase/pocketbase/pull/1469); thanks @khairulhaaziq).
+
+- Added the OAuth2 refresh token to the auth meta response ([#1487](https://github.com/pocketbase/pocketbase/issues/1487)).
+
+- Fixed the text wrapping in the Admin UI listing searchbar ([#1416](https://github.com/pocketbase/pocketbase/issues/1416)).
+
+- Fixed number field value output in the records listing ([#1447](https://github.com/pocketbase/pocketbase/issues/1447)).
+
+- Fixed duplicated settings view pages caused by uncompleted transitions ([#1498](https://github.com/pocketbase/pocketbase/issues/1498)).
+
+- Allowed sending `Authorization` header with the `/auth-with-password` record and admin login requests ([#1494](https://github.com/pocketbase/pocketbase/discussions/1494)).
+
+- `migrate down` now reverts migrations in the applied order.
+
+- Added additional list-bucket check in the S3 config test API.
+
+- Other minor improvements.
+
+
+## v0.10.4
+
+- Fixed `Record.MergeExpand` panic when the main model expand map is not initialized ([#1365](https://github.com/pocketbase/pocketbase/issues/1365)).
+
+
+## v0.10.3
+
+- ! Renamed the metadata key `original_filename` to `original-filename` due to an S3 file upload error caused by the underscore character ([#1343](https://github.com/pocketbase/pocketbase/pull/1343); thanks @yuxiang-gao).
+
+- Fixed request verification docs api url ([#1332](https://github.com/pocketbase/pocketbase/pull/1332); thanks @JoyMajumdar2001)
+
+- Excluded `collectionId` and `collectionName` from the displayable relation props list ([1322](https://github.com/pocketbase/pocketbase/issues/1322); thanks @dhall2).
+
+
+## v0.10.2
+
+- Fixed nested multiple expands with shared path ([#586](https://github.com/pocketbase/pocketbase/issues/586#issuecomment-1357784227)).
+  A new helper method `models.Record.MergeExpand(map[string]any)` was also added to simplify the expand handling and unit testing.
+
+
+## v0.10.1
+
+- Fixed nested transactions deadlock when authenticating with OAuth2 ([#1291](https://github.com/pocketbase/pocketbase/issues/1291)).
+
+
+## v0.10.0
+
+- Added `/api/health` endpoint (thanks @MarvinJWendt).
+
+- Added support for SMTP `LOGIN` auth for Microsoft/Outlook and other providers that don't support the `PLAIN` auth method ([#1217](https://github.com/pocketbase/pocketbase/discussions/1217#discussioncomment-4387970)).
+
+- Reduced memory consumption (you can expect ~20% less allocated memory).
+
+- Added support for split (concurrent and nonconcurrent) DB connections pool increasing even further the concurrent throughput without blocking reads on heavy write load.
+
+- Improved record references delete performance.
+
+- Removed the unnecessary parenthesis in the generated filter SQL query, reducing the "_parse stack overflow_" errors.
+
+- Fixed `~` expressions backslash literal escaping ([#1231](https://github.com/pocketbase/pocketbase/discussions/1231)).
+
+- Refactored the `core.app.Bootstrap()` to be called before starting the cobra commands ([#1267](https://github.com/pocketbase/pocketbase/discussions/1267)).
+
+- ! Changed `pocketbase.NewWithConfig(config Config)` to `pocketbase.NewWithConfig(config *Config)` and added 4 new config settings:
+  ```go
+  DataMaxOpenConns int // default to core.DefaultDataMaxOpenConns
+  DataMaxIdleConns int // default to core.DefaultDataMaxIdleConns
+  LogsMaxOpenConns int // default to core.DefaultLogsMaxOpenConns
+  LogsMaxIdleConns int // default to core.DefaultLogsMaxIdleConns
+  ```
+
+- Added new helper method `core.App.IsBootstrapped()` to check the current app bootstrap state.
+
+- ! Changed `core.NewBaseApp(dir, encryptionEnv, isDebug)` to `NewBaseApp(config *BaseAppConfig)`.
+
+- ! Removed `rest.UploadedFile` struct (see below `filesystem.File`).
+
+- Added generic file resource struct that allows loading and uploading file content from
+  different sources (at the moment multipart/form-data requests and from the local filesystem).
+  ```
+  filesystem.File{}
+  filesystem.NewFileFromPath(path)
+  filesystem.NewFileFromMultipart(multipartHeader)
+  filesystem/System.UploadFile(file)
+  ```
+
+- Refactored `forms.RecordUpsert` to allow more easily loading and removing files programmatically.
+  ```
+  forms.RecordUpsert.AddFiles(key, filesystem.File...) // add new filesystem.File to the form for upload
+  forms.RecordUpsert.RemoveFiles(key, filenames...)     // marks the filenames for deletion
+  ```
+
+- Trigger the `password` validators if any of the others password change fields is set.
+
+
 ## v0.9.2
 
 - Fixed field column name conflict on record deletion ([#1220](https://github.com/pocketbase/pocketbase/discussions/1220)).
@@ -13,7 +206,7 @@
 
 - Replaced `c.QueryString()` with `c.QueryParams().Encode()` to allow loading middleware modified query parameters in the default crud actions ([#1210](https://github.com/pocketbase/pocketbase/discussions/1210)).
 
-- Fixed the datetime field not triggerering the `onChange` event on manual field edit and added a "Clear" button ([#1219](https://github.com/pocketbase/pocketbase/issues/1219)).
+- Fixed the datetime field not triggering the `onChange` event on manual field edit and added a "Clear" button ([#1219](https://github.com/pocketbase/pocketbase/issues/1219)).
 
 - Updated the GitHub goreleaser action to use go 1.19.4 since it comes with [some security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.19.4+label%3ACherryPickApproved).
 
@@ -73,10 +266,10 @@
   ```
 
 - Refactored the `migrate` command to support **external JavaScript migration files** using an embedded JS interpreter ([goja](https://github.com/dop251/goja)).
-  This allow writting custom migration scripts such as programmatically creating collections,
+  This allow writing custom migration scripts such as programmatically creating collections,
   initializing default settings, running data imports, etc., with a JavaScript API very similar to the Go one (_more documentation will be available soon_).
 
-  The `migrate` command is available by default for the prebult executable,
+  The `migrate` command is available by default for the prebuilt executable,
   but if you use PocketBase as framework you need register it manually:
   ```go
   migrationsDir := "" // default to "pb_migrations" (for js) and "migrations" (for go)
@@ -176,7 +369,7 @@ In addition to the `Users` and `profiles` merge, this release comes with several
 
 - The `json` field type now supports filtering and sorting [#423](https://github.com/pocketbase/pocketbase/issues/423#issuecomment-1258302125).
 
-- The `relation` field now allows unlimitted `maxSelect` (aka. without upper limit).
+- The `relation` field now allows unlimited `maxSelect` (aka. without upper limit).
 
 - Added support for combined email/username + password authentication (see below `authWithPassword()`).
 

LICENSE.md

@@ -1,5 +1,5 @@
 The MIT License (MIT)
-Copyright (c) 2022, Gani Georgiev
+Copyright (c) 2022 - present, Gani Georgiev
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software
 and associated documentation files (the "Software"), to deal in the Software without restriction,

README.md

@@ -110,7 +110,7 @@ Check also the [Testing guide](http://pocketbase.io/docs/testing) to learn how t
 
 If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
 
-All reports will be promptly addressed and you'll be credited accordingly.
+All reports will be promptly addressed, and you'll be credited accordingly.
 
 
 ## Contributing

apis/admin.go

@@ -18,7 +18,7 @@ func bindAdminApi(app core.App, rg *echo.Group) {
 	api := adminApi{app: app}
 
 	subGroup := rg.Group("/admins", ActivityLogger(app))
-	subGroup.POST("/auth-with-password", api.authWithPassword, RequireGuestOnly())
+	subGroup.POST("/auth-with-password", api.authWithPassword)
 	subGroup.POST("/request-password-reset", api.requestPasswordReset)
 	subGroup.POST("/confirm-password-reset", api.confirmPasswordReset)
 	subGroup.POST("/auth-refresh", api.authRefresh, RequireAdminAuth())

apis/admin_test.go

@@ -48,6 +48,20 @@ func TestAdminAuthWithEmail(t *testing.T) {
 			ExpectedStatus:  400,
 			ExpectedContent: []string{`"data":{}`},
 		},
+		{
+			Name:           "valid email/password (guest)",
+			Method:         http.MethodPost,
+			Url:            "/api/admins/auth-with-password",
+			Body:           strings.NewReader(`{"identity":"test@example.com","password":"1234567890"}`),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"admin":{"id":"sywbhecnh46rhm0"`,
+				`"token":`,
+			},
+			ExpectedEvents: map[string]int{
+				"OnAdminAuthRequest": 1,
+			},
+		},
 		{
 			Name:   "valid email/password (already authorized)",
 			Method: http.MethodPost,
@@ -56,14 +70,6 @@ func TestAdminAuthWithEmail(t *testing.T) {
 			RequestHeaders: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4MTYwMH0.han3_sG65zLddpcX2ic78qgy7FKecuPfOpFa8Dvi5Bg",
 			},
-			ExpectedStatus:  400,
-			ExpectedContent: []string{`"message":"The request can be accessed only by guests.","data":{}`},
-		},
-		{
-			Name:           "valid email/password (guest)",
-			Method:         http.MethodPost,
-			Url:            "/api/admins/auth-with-password",
-			Body:           strings.NewReader(`{"identity":"test@example.com","password":"1234567890"}`),
 			ExpectedStatus: 200,
 			ExpectedContent: []string{
 				`"admin":{"id":"sywbhecnh46rhm0"`,

apis/base.go

@@ -26,6 +26,13 @@ func InitApi(app core.App) (*echo.Echo, error) {
 	e := echo.New()
 	e.Debug = app.IsDebug()
 
+	// configure a custom router
+	e.ResetRouterCreator(func(ec *echo.Echo) echo.Router {
+		return echo.NewRouter(echo.RouterConfig{
+			UnescapePathParamValues: true,
+		})
+	})
+
 	// default middlewares
 	e.Pre(middleware.RemoveTrailingSlashWithConfig(middleware.RemoveTrailingSlashConfig{
 		Skipper: func(c echo.Context) bool {
@@ -100,6 +107,7 @@ func InitApi(app core.App) (*echo.Echo, error) {
 	bindFileApi(app, api)
 	bindRealtimeApi(app, api)
 	bindLogsApi(app, api)
+	bindHealthApi(app, api)
 
 	// trigger the custom BeforeServe hook for the created api router
 	// allowing users to further adjust its options or register new routes

apis/base_test.go

@@ -66,6 +66,22 @@ func TestCustomRoutesAndErrorsHandling(t *testing.T) {
 			ExpectedStatus:  200,
 			ExpectedContent: []string{"test123"},
 		},
+		{
+			Name:   "custom route with url encoded parameter",
+			Method: http.MethodGet,
+			Url:    "/a%2Bb%2Bc",
+			BeforeTestFunc: func(t *testing.T, app *tests.TestApp, e *echo.Echo) {
+				e.AddRoute(echo.Route{
+					Method: http.MethodGet,
+					Path:   "/:param",
+					Handler: func(c echo.Context) error {
+						return c.String(200, c.PathParam("param"))
+					},
+				})
+			},
+			ExpectedStatus:  200,
+			ExpectedContent: []string{"a+b+c"},
+		},
 		{
 			Name:   "route with HTTPError",
 			Method: http.MethodGet,

apis/collection_test.go

@@ -6,6 +6,7 @@ import (
 	"path/filepath"
 	"strings"
 	"testing"
+	"time"
 
 	"github.com/labstack/echo/v5"
 	"github.com/pocketbase/pocketbase/models"
@@ -42,7 +43,7 @@ func TestCollectionsList(t *testing.T) {
 			ExpectedContent: []string{
 				`"page":1`,
 				`"perPage":30`,
-				`"totalItems":7`,
+				`"totalItems":8`,
 				`"items":[{`,
 				`"id":"_pb_users_auth_"`,
 				`"id":"v851q4r790rhknl"`,
@@ -51,6 +52,7 @@ func TestCollectionsList(t *testing.T) {
 				`"id":"sz5l5z67tg7gku0"`,
 				`"id":"wzlqyes4orhoygb"`,
 				`"id":"4d1blo5cuycfaca"`,
+				`"id":"9n89pl5vkct6330"`,
 				`"type":"auth"`,
 				`"type":"base"`,
 			},
@@ -69,10 +71,10 @@ func TestCollectionsList(t *testing.T) {
 			ExpectedContent: []string{
 				`"page":2`,
 				`"perPage":2`,
-				`"totalItems":7`,
+				`"totalItems":8`,
 				`"items":[{`,
+				`"id":"v851q4r790rhknl"`,
 				`"id":"4d1blo5cuycfaca"`,
-				`"id":"wzlqyes4orhoygb"`,
 			},
 			ExpectedEvents: map[string]int{
 				"OnCollectionsListRequest": 1,
@@ -99,12 +101,13 @@ func TestCollectionsList(t *testing.T) {
 			ExpectedContent: []string{
 				`"page":1`,
 				`"perPage":30`,
-				`"totalItems":4`,
+				`"totalItems":5`,
 				`"items":[{`,
 				`"id":"wsmn24bux7wo113"`,
 				`"id":"sz5l5z67tg7gku0"`,
 				`"id":"wzlqyes4orhoygb"`,
 				`"id":"4d1blo5cuycfaca"`,
+				`"id":"9n89pl5vkct6330"`,
 			},
 			ExpectedEvents: map[string]int{
 				"OnCollectionsListRequest": 1,
@@ -230,6 +233,7 @@ func TestCollectionDelete(t *testing.T) {
 			RequestHeaders: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
 			},
+			Delay:          100 * time.Millisecond,
 			ExpectedStatus: 204,
 			ExpectedEvents: map[string]int{
 				"OnModelBeforeDelete":             1,
@@ -248,6 +252,7 @@ func TestCollectionDelete(t *testing.T) {
 			RequestHeaders: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
 			},
+			Delay:          100 * time.Millisecond,
 			ExpectedStatus: 204,
 			ExpectedEvents: map[string]int{
 				"OnModelBeforeDelete":             1,
@@ -786,7 +791,7 @@ func TestCollectionImport(t *testing.T) {
 				if err := app.Dao().CollectionQuery().All(&collections); err != nil {
 					t.Fatal(err)
 				}
-				expected := 7
+				expected := 8
 				if len(collections) != expected {
 					t.Fatalf("Expected %d collections, got %d", expected, len(collections))
 				}
@@ -814,7 +819,7 @@ func TestCollectionImport(t *testing.T) {
 				if err := app.Dao().CollectionQuery().All(&collections); err != nil {
 					t.Fatal(err)
 				}
-				expected := 7
+				expected := 8
 				if len(collections) != expected {
 					t.Fatalf("Expected %d collections, got %d", expected, len(collections))
 				}
@@ -856,7 +861,7 @@ func TestCollectionImport(t *testing.T) {
 				if err := app.Dao().CollectionQuery().All(&collections); err != nil {
 					t.Fatal(err)
 				}
-				expected := 7
+				expected := 8
 				if len(collections) != expected {
 					t.Fatalf("Expected %d collections, got %d", expected, len(collections))
 				}
@@ -909,7 +914,7 @@ func TestCollectionImport(t *testing.T) {
 				if err := app.Dao().CollectionQuery().All(&collections); err != nil {
 					t.Fatal(err)
 				}
-				expected := 10
+				expected := 11
 				if len(collections) != expected {
 					t.Fatalf("Expected %d collections, got %d", expected, len(collections))
 				}
@@ -996,8 +1001,8 @@ func TestCollectionImport(t *testing.T) {
 			ExpectedEvents: map[string]int{
 				"OnCollectionsAfterImportRequest":  1,
 				"OnCollectionsBeforeImportRequest": 1,
-				"OnModelBeforeDelete":              5,
-				"OnModelAfterDelete":               5,
+				"OnModelBeforeDelete":              6,
+				"OnModelAfterDelete":               6,
 				"OnModelBeforeUpdate":              2,
 				"OnModelAfterUpdate":               2,
 				"OnModelBeforeCreate":              1,

apis/health.go

@@ -0,0 +1,30 @@
+package apis
+
+import (
+	"net/http"
+
+	"github.com/labstack/echo/v5"
+	"github.com/pocketbase/pocketbase/core"
+)
+
+// bindHealthApi registers the health api endpoint.
+func bindHealthApi(app core.App, rg *echo.Group) {
+	api := healthApi{app: app}
+
+	subGroup := rg.Group("/health")
+	subGroup.GET("", api.healthCheck)
+}
+
+type healthApi struct {
+	app core.App
+}
+
+// healthCheck returns a 200 OK response if the server is healthy.
+func (api *healthApi) healthCheck(c echo.Context) error {
+	payload := map[string]any{
+		"code":    http.StatusOK,
+		"message": "API is healthy.",
+	}
+
+	return c.JSON(http.StatusOK, payload)
+}

apis/health_test.go

@@ -0,0 +1,26 @@
+package apis_test
+
+import (
+	"net/http"
+	"testing"
+
+	"github.com/pocketbase/pocketbase/tests"
+)
+
+func TestHealthAPI(t *testing.T) {
+	scenarios := []tests.ApiScenario{
+		{
+			Name:           "health status returns 200",
+			Method:         http.MethodGet,
+			Url:            "/api/health",
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"code":200`,
+			},
+		},
+	}
+
+	for _, scenario := range scenarios {
+		scenario.Test(t)
+	}
+}

apis/middlewares.go

@@ -333,19 +333,8 @@ func ActivityLogger(app core.App) echo.MiddlewareFunc {
 			model.RefreshUpdated()
 
 			routine.FireAndForget(func() {
-				attempts := 1
-
-			BeginSave:
-				logErr := app.LogsDao().SaveRequest(model)
-				if logErr != nil {
-					// try one more time after 10s in case of SQLITE_BUSY or "database is locked" error
-					if attempts <= 2 {
-						attempts++
-						time.Sleep(10 * time.Second)
-						goto BeginSave
-					} else if app.IsDebug() {
-						log.Println("Log save failed:", logErr)
-					}
+				if err := app.LogsDao().SaveRequest(model); err != nil && app.IsDebug() {
+					log.Println("Log save failed:", err)
 				}
 
 				// Delete old request logs

apis/realtime.go

@@ -15,6 +15,7 @@ import (
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/resolvers"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/search"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
 )
@@ -43,10 +44,14 @@ func (api *realtimeApi) connect(c echo.Context) error {
 	client := subscriptions.NewDefaultClient()
 	api.app.SubscriptionsBroker().Register(client)
 	defer func() {
-		api.app.OnRealtimeDisconnectRequest().Trigger(&core.RealtimeDisconnectEvent{
+		disconnectEvent := &core.RealtimeDisconnectEvent{
 			HttpContext: c,
 			Client:      client,
-		})
+		}
+
+		if err := api.app.OnRealtimeDisconnectRequest().Trigger(disconnectEvent); err != nil && api.app.IsDebug() {
+			log.Println(err)
+		}
 
 		api.app.SubscriptionsBroker().Unregister(client.Id())
 	}()
@@ -259,21 +264,27 @@ func (api *realtimeApi) bindEvents() {
 
 	api.app.OnModelAfterCreate().PreAdd(func(e *core.ModelEvent) error {
 		if record, ok := e.Model.(*models.Record); ok {
-			api.broadcastRecord("create", record)
+			if err := api.broadcastRecord("create", record); err != nil && api.app.IsDebug() {
+				log.Println(err)
+			}
 		}
 		return nil
 	})
 
 	api.app.OnModelAfterUpdate().PreAdd(func(e *core.ModelEvent) error {
 		if record, ok := e.Model.(*models.Record); ok {
-			api.broadcastRecord("update", record)
+			if err := api.broadcastRecord("update", record); err != nil && api.app.IsDebug() {
+				log.Println(err)
+			}
 		}
 		return nil
 	})
 
 	api.app.OnModelBeforeDelete().Add(func(e *core.ModelEvent) error {
 		if record, ok := e.Model.(*models.Record); ok {
-			api.broadcastRecord("delete", record)
+			if err := api.broadcastRecord("delete", record); err != nil && api.app.IsDebug() {
+				log.Println(err)
+			}
 		}
 		return nil
 	})
@@ -370,6 +381,8 @@ func (api *realtimeApi) broadcastRecord(action string, record *models.Record) er
 	encodedData := string(dataBytes)
 
 	for _, client := range clients {
+		client := client
+
 		for subscription, rule := range subscriptionRuleMap {
 			if !client.HasSubscription(subscription) {
 				continue
@@ -398,7 +411,11 @@ func (api *realtimeApi) broadcastRecord(action string, record *models.Record) er
 				}
 			}
 
-			client.Channel() <- msg
+			routine.FireAndForget(func() {
+				if !client.IsDiscarded() {
+					client.Channel() <- msg
+				}
+			})
 		}
 	}
 

apis/record_auth.go

@@ -35,8 +35,8 @@ func bindRecordAuthApi(app core.App, rg *echo.Group) {
 
 	subGroup.GET("/auth-methods", api.authMethods)
 	subGroup.POST("/auth-refresh", api.authRefresh, RequireSameContextRecordAuth())
-	subGroup.POST("/auth-with-oauth2", api.authWithOAuth2) // allow anyone so that we can link the OAuth2 profile with the authenticated record
-	subGroup.POST("/auth-with-password", api.authWithPassword, RequireGuestOnly())
+	subGroup.POST("/auth-with-oauth2", api.authWithOAuth2)
+	subGroup.POST("/auth-with-password", api.authWithPassword)
 	subGroup.POST("/request-password-reset", api.requestPasswordReset)
 	subGroup.POST("/confirm-password-reset", api.confirmPasswordReset)
 	subGroup.POST("/request-verification", api.requestVerification)

apis/record_auth_test.go

@@ -66,26 +66,6 @@ func TestRecordAuthMethodsList(t *testing.T) {
 
 func TestRecordAuthWithPassword(t *testing.T) {
 	scenarios := []tests.ApiScenario{
-		{
-			Name:   "authenticated record",
-			Method: http.MethodPost,
-			Url:    "/api/collections/users/auth-with-password",
-			RequestHeaders: map[string]string{
-				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoUmVjb3JkIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyMjA4OTg1MjYxfQ.UwD8JvkbQtXpymT09d7J6fdA0aP9g4FJ1GPh_ggEkzc",
-			},
-			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-		},
-		{
-			Name:   "authenticated admin",
-			Method: http.MethodPost,
-			Url:    "/api/collections/users/auth-with-password",
-			RequestHeaders: map[string]string{
-				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
-			},
-			ExpectedStatus:  400,
-			ExpectedContent: []string{`"data":{}`},
-		},
 		{
 			Name:            "invalid body format",
 			Method:          http.MethodPost,
@@ -226,6 +206,52 @@ func TestRecordAuthWithPassword(t *testing.T) {
 				"OnRecordAuthRequest": 1,
 			},
 		},
+
+		// with already authenticated record or admin
+		{
+			Name:   "authenticated record",
+			Method: http.MethodPost,
+			Url:    "/api/collections/users/auth-with-password",
+			RequestHeaders: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoUmVjb3JkIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyMjA4OTg1MjYxfQ.UwD8JvkbQtXpymT09d7J6fdA0aP9g4FJ1GPh_ggEkzc",
+			},
+			Body: strings.NewReader(`{
+				"identity":"test@example.com",
+				"password":"1234567890"
+			}`),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"record":{`,
+				`"token":"`,
+				`"id":"4q1xlclmfloku33"`,
+				`"email":"test@example.com"`,
+			},
+			ExpectedEvents: map[string]int{
+				"OnRecordAuthRequest": 1,
+			},
+		},
+		{
+			Name:   "authenticated admin",
+			Method: http.MethodPost,
+			Url:    "/api/collections/users/auth-with-password",
+			RequestHeaders: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
+			},
+			Body: strings.NewReader(`{
+				"identity":"test@example.com",
+				"password":"1234567890"
+			}`),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"record":{`,
+				`"token":"`,
+				`"id":"4q1xlclmfloku33"`,
+				`"email":"test@example.com"`,
+			},
+			ExpectedEvents: map[string]int{
+				"OnRecordAuthRequest": 1,
+			},
+		},
 	}
 
 	for _, scenario := range scenarios {

apis/record_crud.go

@@ -166,6 +166,20 @@ func (api *recordApi) create(c echo.Context) error {
 
 	// temporary save the record and check it against the create rule
 	if requestData.Admin == nil && collection.CreateRule != nil {
+		testRecord := models.NewRecord(collection)
+
+		// replace modifiers fields so that the resolved value is always
+		// available when accessing requestData.Data using just the field name
+		if requestData.HasModifierDataKeys() {
+			requestData.Data = testRecord.ReplaceModifers(requestData.Data)
+		}
+
+		testForm := forms.NewRecordUpsert(api.app, testRecord)
+		testForm.SetFullManageAccess(true)
+		if err := testForm.LoadRequest(c.Request(), ""); err != nil {
+			return NewBadRequestError("Failed to load the submitted data due to invalid formatting.", err)
+		}
+
 		createRuleFunc := func(q *dbx.SelectQuery) error {
 			if *collection.CreateRule == "" {
 				return nil // no create rule to resolve
@@ -181,13 +195,6 @@ func (api *recordApi) create(c echo.Context) error {
 			return nil
 		}
 
-		testRecord := models.NewRecord(collection)
-		testForm := forms.NewRecordUpsert(api.app, testRecord)
-		testForm.SetFullManageAccess(true)
-		if err := testForm.LoadRequest(c.Request(), ""); err != nil {
-			return NewBadRequestError("Failed to load the submitted data due to invalid formatting.", err)
-		}
-
 		testErr := testForm.DrySubmit(func(txDao *daos.Dao) error {
 			foundRecord, err := txDao.FindRecordById(collection.Id, testRecord.Id, createRuleFunc)
 			if err != nil {
@@ -258,6 +265,16 @@ func (api *recordApi) update(c echo.Context) error {
 		return NewForbiddenError("Only admins can perform this action.", nil)
 	}
 
+	// eager fetch the record so that the modifier field values are replaced
+	// and available when accessing requestData.Data using just the field name
+	if requestData.HasModifierDataKeys() {
+		record, err := api.app.Dao().FindRecordById(collection.Id, recordId)
+		if err != nil || record == nil {
+			return NewNotFoundError("", err)
+		}
+		requestData.Data = record.ReplaceModifers(requestData.Data)
+	}
+
 	ruleFunc := func(q *dbx.SelectQuery) error {
 		if requestData.Admin == nil && collection.UpdateRule != nil && *collection.UpdateRule != "" {
 			resolver := resolvers.NewRecordFieldResolver(api.app.Dao(), collection, requestData, true)

apis/record_crud_test.go

@@ -2,10 +2,12 @@ package apis_test
 
 import (
 	"net/http"
+	"net/url"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
+	"time"
 
 	"github.com/labstack/echo/v5"
 	"github.com/pocketbase/pocketbase/models"
@@ -209,6 +211,50 @@ func TestRecordCrudList(t *testing.T) {
 			},
 			ExpectedEvents: map[string]int{"OnRecordsListRequest": 1},
 		},
+		{
+			Name:           ":rule modifer",
+			Method:         http.MethodGet,
+			Url:            "/api/collections/demo5/records",
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":1`,
+				`"totalItems":1`,
+				`"items":[{`,
+				`"id":"qjeql998mtp1azp"`,
+			},
+			ExpectedEvents: map[string]int{"OnRecordsListRequest": 1},
+		},
+		{
+			Name:           "multi-match - at least one of",
+			Method:         http.MethodGet,
+			Url:            "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length?=2"),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":1`,
+				`"totalItems":1`,
+				`"items":[{`,
+				`"id":"qzaqccwrmva4o1n"`,
+			},
+			ExpectedEvents: map[string]int{"OnRecordsListRequest": 1},
+		},
+		{
+			Name:           "multi-match - all",
+			Method:         http.MethodGet,
+			Url:            "/api/collections/demo4/records?filter=" + url.QueryEscape("rel_many_no_cascade_required.files:length=2"),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"page":1`,
+				`"perPage":30`,
+				`"totalPages":0`,
+				`"totalItems":0`,
+				`"items":[]`,
+			},
+			ExpectedEvents: map[string]int{"OnRecordsListRequest": 1},
+		},
 
 		// auth collection checks
 		// -----------------------------------------------------------
@@ -684,18 +730,19 @@ func TestRecordCrudDelete(t *testing.T) {
 			},
 		},
 		{
-			Name:   "authenticated record that does match the collection delete rule",
+			Name:   "authenticated record that match the collection delete rule",
 			Method: http.MethodDelete,
 			Url:    "/api/collections/users/records/4q1xlclmfloku33",
 			RequestHeaders: map[string]string{
 				// users, test@example.com
 				"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoUmVjb3JkIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyMjA4OTg1MjYxfQ.UwD8JvkbQtXpymT09d7J6fdA0aP9g4FJ1GPh_ggEkzc",
 			},
+			Delay:          100 * time.Millisecond,
 			ExpectedStatus: 204,
 			ExpectedEvents: map[string]int{
-				"OnModelAfterDelete":          1,
+				"OnModelAfterDelete":          3, // +2 because of the external auths
+				"OnModelBeforeDelete":         3, // +2 because of the external auths
 				"OnModelAfterUpdate":          1,
-				"OnModelBeforeDelete":         1,
 				"OnModelBeforeUpdate":         1,
 				"OnRecordAfterDeleteRequest":  1,
 				"OnRecordBeforeDeleteRequest": 1,
@@ -716,6 +763,25 @@ func TestRecordCrudDelete(t *testing.T) {
 				}
 			},
 		},
+		{
+			Name:            "@request :isset (rule failure check)",
+			Method:          http.MethodDelete,
+			Url:             "/api/collections/demo5/records/la4y2w4o98acwuj",
+			ExpectedStatus:  404,
+			ExpectedContent: []string{`"data":{}`},
+		},
+		{
+			Name:           "@request :isset (rule pass check)",
+			Method:         http.MethodDelete,
+			Url:            "/api/collections/demo5/records/la4y2w4o98acwuj?test=1",
+			ExpectedStatus: 204,
+			ExpectedEvents: map[string]int{
+				"OnModelAfterDelete":          1,
+				"OnModelBeforeDelete":         1,
+				"OnRecordAfterDeleteRequest":  1,
+				"OnRecordBeforeDeleteRequest": 1,
+			},
+		},
 
 		// cascade delete checks
 		// -----------------------------------------------------------
@@ -730,7 +796,7 @@ func TestRecordCrudDelete(t *testing.T) {
 			ExpectedContent: []string{`"data":{}`},
 			ExpectedEvents: map[string]int{
 				"OnRecordBeforeDeleteRequest": 1,
-				"OnModelBeforeUpdate":         1, // self_rel_many update of test1 record
+				"OnModelBeforeUpdate":         2, // self_rel_many update of test1 record + rel_one_cascade demo4 cascaded in demo5
 				"OnModelBeforeDelete":         2, // the record itself + rel_one_cascade of test1 record
 			},
 		},
@@ -758,6 +824,7 @@ func TestRecordCrudDelete(t *testing.T) {
 			RequestHeaders: map[string]string{
 				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
 			},
+			Delay:          100 * time.Millisecond,
 			ExpectedStatus: 204,
 			ExpectedEvents: map[string]int{
 				"OnModelBeforeDelete":         2,
@@ -1092,6 +1159,63 @@ func TestRecordCrudCreate(t *testing.T) {
 			},
 		},
 
+		// fields modifier checks
+		// -----------------------------------------------------------
+		{
+			Name:   "trying to delete a record while being part of a non-cascade required relation",
+			Method: http.MethodDelete,
+			Url:    "/api/collections/demo3/records/7nwo8tuiatetxdm",
+			RequestHeaders: map[string]string{
+				"Authorization": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhZG1pbiIsImV4cCI6MjIwODk4NTI2MX0.M1m--VOqGyv0d23eeUc0r9xE8ZzHaYVmVFw1VZW6gT8",
+			},
+			ExpectedStatus:  400,
+			ExpectedContent: []string{`"data":{}`},
+			ExpectedEvents: map[string]int{
+				"OnRecordBeforeDeleteRequest": 1,
+				"OnModelBeforeUpdate":         2, // self_rel_many update of test1 record + rel_one_cascade demo4 cascaded in demo5
+				"OnModelBeforeDelete":         2, // the record itself + rel_one_cascade of test1 record
+			},
+		},
+
+		// check whether if @request.data modifer fields are properly resolved
+		// -----------------------------------------------------------
+		{
+			Name:   "@request.data.field with compute modifers (rule failure check)",
+			Method: http.MethodPost,
+			Url:    "/api/collections/demo5/records",
+			Body: strings.NewReader(`{
+				"total":1,
+				"total+":4,
+				"total-":1
+			}`),
+			ExpectedStatus: 400,
+			ExpectedContent: []string{
+				`"data":{}`,
+			},
+		},
+		{
+			Name:   "@request.data.field with compute modifers (rule pass check)",
+			Method: http.MethodPost,
+			Url:    "/api/collections/demo5/records",
+			Body: strings.NewReader(`{
+				"total":1,
+				"total+":3,
+				"total-":1
+			}`),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"id":"`,
+				`"collectionName":"demo5"`,
+				`"total":3`,
+			},
+			ExpectedEvents: map[string]int{
+				"OnModelAfterCreate":          1,
+				"OnModelBeforeCreate":         1,
+				"OnRecordAfterCreateRequest":  1,
+				"OnRecordBeforeCreateRequest": 1,
+			},
+		},
+
 		// auth records
 		// -----------------------------------------------------------
 		{
@@ -1501,6 +1625,43 @@ func TestRecordCrudUpdate(t *testing.T) {
 			},
 		},
 
+		// check whether if @request.data modifer fields are properly resolved
+		// -----------------------------------------------------------
+		{
+			Name:   "@request.data.field with compute modifers (rule failure check)",
+			Method: http.MethodPatch,
+			Url:    "/api/collections/demo5/records/la4y2w4o98acwuj",
+			Body: strings.NewReader(`{
+				"total+":3,
+				"total-":1
+			}`),
+			ExpectedStatus: 404,
+			ExpectedContent: []string{
+				`"data":{}`,
+			},
+		},
+		{
+			Name:   "@request.data.field with compute modifers (rule pass check)",
+			Method: http.MethodPatch,
+			Url:    "/api/collections/demo5/records/la4y2w4o98acwuj",
+			Body: strings.NewReader(`{
+				"total+":2,
+				"total-":1
+			}`),
+			ExpectedStatus: 200,
+			ExpectedContent: []string{
+				`"id":"la4y2w4o98acwuj"`,
+				`"collectionName":"demo5"`,
+				`"total":3`,
+			},
+			ExpectedEvents: map[string]int{
+				"OnModelAfterUpdate":          1,
+				"OnModelBeforeUpdate":         1,
+				"OnRecordAfterUpdateRequest":  1,
+				"OnRecordBeforeUpdateRequest": 1,
+			},
+		},
+
 		// auth records
 		// -----------------------------------------------------------
 		{

apis/record_helpers.go

@@ -15,11 +15,6 @@ import (
 
 const ContextRequestDataKey = "requestData"
 
-// Deprecated: Will be removed after v0.9. Use apis.RequestData(c) instead.
-func GetRequestData(c echo.Context) *models.RequestData {
-	return RequestData(c)
-}
-
 // RequestData exports cached common request data fields
 // (query, body, logged auth state, etc.) from the provided context.
 func RequestData(c echo.Context) *models.RequestData {

apis/settings.go

@@ -1,6 +1,7 @@
 package apis
 
 import (
+	"fmt"
 	"net/http"
 
 	validation "github.com/go-ozzo/ozzo-validation/v4"
@@ -91,14 +92,17 @@ func (api *settingsApi) testS3(c echo.Context) error {
 	}
 	defer fs.Close()
 
-	testFileKey := "pb_test_" + security.PseudorandomString(5) + "/test.txt"
+	testPrefix := "pb_settings_test_" + security.PseudorandomString(5)
+	testFileKey := testPrefix + "/test.txt"
 
+	// try to upload a test file
 	if err := fs.Upload([]byte("test"), testFileKey); err != nil {
 		return NewBadRequestError("Failed to upload a test file. Raw error: \n"+err.Error(), nil)
 	}
 
-	if err := fs.Delete(testFileKey); err != nil {
-		return NewBadRequestError("Failed to delete a test file. Raw error: \n"+err.Error(), nil)
+	// test prefix deletion (ensures that both bucket list and delete works)
+	if errs := fs.DeletePrefix(testPrefix); len(errs) > 0 {
+		return NewBadRequestError(fmt.Sprintf("Failed to delete a test file. Raw error: %v", errs), nil)
 	}
 
 	return c.NoContent(http.StatusNoContent)

apis/settings_test.go

@@ -58,6 +58,8 @@ func TestSettingsList(t *testing.T) {
 				`"spotifyAuth":{`,
 				`"kakaoAuth":{`,
 				`"twitchAuth":{`,
+				`"stravaAuth":{`,
+				`"giteeAuth":{`,
 				`"secret":"******"`,
 				`"clientSecret":"******"`,
 			},
@@ -125,6 +127,8 @@ func TestSettingsSet(t *testing.T) {
 				`"spotifyAuth":{`,
 				`"kakaoAuth":{`,
 				`"twitchAuth":{`,
+				`"stravaAuth":{`,
+				`"giteeAuth":{`,
 				`"secret":"******"`,
 				`"clientSecret":"******"`,
 				`"appName":"acme_test"`,
@@ -181,6 +185,8 @@ func TestSettingsSet(t *testing.T) {
 				`"spotifyAuth":{`,
 				`"kakaoAuth":{`,
 				`"twitchAuth":{`,
+				`"stravaAuth":{`,
+				`"giteeAuth":{`,
 				`"secret":"******"`,
 				`"clientSecret":"******"`,
 				`"appName":"update_test"`,

cmd/serve.go

@@ -78,7 +78,8 @@ func NewServeCommand(app core.App, showStartBanner bool) *cobra.Command {
 					GetCertificate: certManager.GetCertificate,
 					NextProtos:     []string{acme.ALPNProto},
 				},
-				ReadTimeout: 60 * time.Second,
+				ReadTimeout:       5 * time.Minute,
+				ReadHeaderTimeout: 30 * time.Second,
 				// WriteTimeout: 60 * time.Second, // breaks sse!
 				Handler: router,
 				Addr:    mainAddr,

core/app.go

@@ -16,6 +16,11 @@ import (
 
 // App defines the main PocketBase app interface.
 type App interface {
+	// Deprecated:
+	// This method may get removed in the near future.
+	// It is recommended to access the app db instance from app.Dao().DB() or
+	// if you want more flexibility - app.Dao().ConcurrentDB() and app.Dao().NonconcurrentDB().
+	//
 	// DB returns the default app database instance.
 	DB() *dbx.DB
 
@@ -26,6 +31,11 @@ type App interface {
 	// trying to access the request logs table will result in error.
 	Dao() *daos.Dao
 
+	// Deprecated:
+	// This method may get removed in the near future.
+	// It is recommended to access the logs db instance from app.LogsDao().DB() or
+	// if you want more flexibility - app.LogsDao().ConcurrentDB() and app.LogsDao().NonconcurrentDB().
+	//
 	// LogsDB returns the app logs database instance.
 	LogsDB() *dbx.DB
 
@@ -68,8 +78,14 @@ type App interface {
 	// RefreshSettings reinitializes and reloads the stored application settings.
 	RefreshSettings() error
 
+	// IsBootstrapped checks if the application was initialized
+	// (aka. whether Bootstrap() was called).
+	IsBootstrapped() bool
+
 	// Bootstrap takes care for initializing the application
-	// (open db connections, load settings, etc.)
+	// (open db connections, load settings, etc.).
+	//
+	// It will call ResetBootstrapState() if the application was already bootstrapped.
 	Bootstrap() error
 
 	// ResetBootstrapState takes care for releasing initialized app resources

core/base.go

@@ -17,25 +17,35 @@ import (
 	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/hook"
 	"github.com/pocketbase/pocketbase/tools/mailer"
+	"github.com/pocketbase/pocketbase/tools/routine"
 	"github.com/pocketbase/pocketbase/tools/store"
 	"github.com/pocketbase/pocketbase/tools/subscriptions"
 )
 
+const (
+	DefaultDataMaxOpenConns int = 100
+	DefaultDataMaxIdleConns int = 20
+	DefaultLogsMaxOpenConns int = 10
+	DefaultLogsMaxIdleConns int = 2
+)
+
 var _ App = (*BaseApp)(nil)
 
 // BaseApp implements core.App and defines the base PocketBase app structure.
 type BaseApp struct {
 	// configurable parameters
-	isDebug       bool
-	dataDir       string
-	encryptionEnv string
+	isDebug          bool
+	dataDir          string
+	encryptionEnv    string
+	dataMaxOpenConns int
+	dataMaxIdleConns int
+	logsMaxOpenConns int
+	logsMaxIdleConns int
 
 	// internals
 	cache               *store.Store[any]
 	settings            *settings.Settings
-	db                  *dbx.DB
 	dao                 *daos.Dao
-	logsDB              *dbx.DB
 	logsDao             *daos.Dao
 	subscriptionsBroker *subscriptions.Broker
 
@@ -132,15 +142,30 @@ type BaseApp struct {
 	onCollectionsAfterImportRequest  *hook.Hook[*CollectionsImportEvent]
 }
 
+// BaseAppConfig defines a BaseApp configuration option
+type BaseAppConfig struct {
+	DataDir          string
+	EncryptionEnv    string
+	IsDebug          bool
+	DataMaxOpenConns int // default to 500
+	DataMaxIdleConns int // default 20
+	LogsMaxOpenConns int // default to 100
+	LogsMaxIdleConns int // default to 5
+}
+
 // NewBaseApp creates and returns a new BaseApp instance
 // configured with the provided arguments.
 //
 // To initialize the app, you need to call `app.Bootstrap()`.
-func NewBaseApp(dataDir string, encryptionEnv string, isDebug bool) *BaseApp {
+func NewBaseApp(config *BaseAppConfig) *BaseApp {
 	app := &BaseApp{
-		dataDir:             dataDir,
-		isDebug:             isDebug,
-		encryptionEnv:       encryptionEnv,
+		dataDir:             config.DataDir,
+		isDebug:             config.IsDebug,
+		encryptionEnv:       config.EncryptionEnv,
+		dataMaxOpenConns:    config.DataMaxOpenConns,
+		dataMaxIdleConns:    config.DataMaxIdleConns,
+		logsMaxOpenConns:    config.LogsMaxOpenConns,
+		logsMaxIdleConns:    config.LogsMaxIdleConns,
 		cache:               store.New[any](nil),
 		settings:            settings.New(),
 		subscriptionsBroker: subscriptions.NewBroker(),
@@ -243,8 +268,16 @@ func NewBaseApp(dataDir string, encryptionEnv string, isDebug bool) *BaseApp {
 	return app
 }
 
+// IsBootstrapped checks if the application was initialized
+// (aka. whether Bootstrap() was called).
+func (app *BaseApp) IsBootstrapped() bool {
+	return app.dao != nil && app.logsDao != nil && app.settings != nil
+}
+
 // Bootstrap initializes the application
-// (aka. create data dir, open db connections, load settings, etc.)
+// (aka. create data dir, open db connections, load settings, etc.).
+//
+// It will call ResetBootstrapState() if the application was already bootstrapped.
 func (app *BaseApp) Bootstrap() error {
 	event := &BootstrapEvent{app}
 
@@ -283,14 +316,20 @@ func (app *BaseApp) Bootstrap() error {
 // ResetBootstrapState takes care for releasing initialized app resources
 // (eg. closing db connections).
 func (app *BaseApp) ResetBootstrapState() error {
-	if app.db != nil {
-		if err := app.db.Close(); err != nil {
+	if app.Dao() != nil {
+		if err := app.Dao().ConcurrentDB().(*dbx.DB).Close(); err != nil {
+			return err
+		}
+		if err := app.Dao().NonconcurrentDB().(*dbx.DB).Close(); err != nil {
 			return err
 		}
 	}
 
-	if app.logsDB != nil {
-		if err := app.logsDB.Close(); err != nil {
+	if app.LogsDao() != nil {
+		if err := app.LogsDao().ConcurrentDB().(*dbx.DB).Close(); err != nil {
+			return err
+		}
+		if err := app.LogsDao().NonconcurrentDB().(*dbx.DB).Close(); err != nil {
 			return err
 		}
 	}
@@ -302,9 +341,23 @@ func (app *BaseApp) ResetBootstrapState() error {
 	return nil
 }
 
+// Deprecated:
+// This method may get removed in the near future.
+// It is recommended to access the db instance from app.Dao().DB() or
+// if you want more flexibility - app.Dao().ConcurrentDB() and app.Dao().NonconcurrentDB().
+//
 // DB returns the default app database instance.
 func (app *BaseApp) DB() *dbx.DB {
-	return app.db
+	if app.Dao() == nil {
+		return nil
+	}
+
+	db, ok := app.Dao().DB().(*dbx.DB)
+	if !ok {
+		return nil
+	}
+
+	return db
 }
 
 // Dao returns the default app Dao instance.
@@ -312,9 +365,23 @@ func (app *BaseApp) Dao() *daos.Dao {
 	return app.dao
 }
 
+// Deprecated:
+// This method may get removed in the near future.
+// It is recommended to access the logs db instance from app.LogsDao().DB() or
+// if you want more flexibility - app.LogsDao().ConcurrentDB() and app.LogsDao().NonconcurrentDB().
+//
 // LogsDB returns the app logs database instance.
 func (app *BaseApp) LogsDB() *dbx.DB {
-	return app.logsDB
+	if app.LogsDao() == nil {
+		return nil
+	}
+
+	db, ok := app.LogsDao().DB().(*dbx.DB)
+	if !ok {
+		return nil
+	}
+
+	return db
 }
 
 // LogsDao returns the app logs Dao instance.
@@ -358,13 +425,14 @@ func (app *BaseApp) SubscriptionsBroker() *subscriptions.Broker {
 // based on the current app settings.
 func (app *BaseApp) NewMailClient() mailer.Mailer {
 	if app.Settings().Smtp.Enabled {
-		return mailer.NewSmtpClient(
-			app.Settings().Smtp.Host,
-			app.Settings().Smtp.Port,
-			app.Settings().Smtp.Username,
-			app.Settings().Smtp.Password,
-			app.Settings().Smtp.Tls,
-		)
+		return &mailer.SmtpClient{
+			Host:       app.Settings().Smtp.Host,
+			Port:       app.Settings().Smtp.Port,
+			Username:   app.Settings().Smtp.Username,
+			Password:   app.Settings().Smtp.Password,
+			Tls:        app.Settings().Smtp.Tls,
+			AuthMethod: app.Settings().Smtp.AuthMethod,
+		}
 	}
 
 	return &mailer.Sendmail{}
@@ -750,41 +818,81 @@ func (app *BaseApp) OnCollectionsAfterImportRequest() *hook.Hook[*CollectionsImp
 // -------------------------------------------------------------------
 
 func (app *BaseApp) initLogsDB() error {
-	var connectErr error
-	app.logsDB, connectErr = connectDB(filepath.Join(app.DataDir(), "logs.db"))
-	if connectErr != nil {
-		return connectErr
+	maxOpenConns := DefaultLogsMaxOpenConns
+	maxIdleConns := DefaultLogsMaxIdleConns
+	if app.logsMaxOpenConns > 0 {
+		maxOpenConns = app.logsMaxOpenConns
+	}
+	if app.logsMaxIdleConns > 0 {
+		maxIdleConns = app.logsMaxIdleConns
 	}
 
-	app.logsDao = daos.New(app.logsDB)
+	concurrentDB, err := connectDB(filepath.Join(app.DataDir(), "logs.db"))
+	if err != nil {
+		return err
+	}
+	concurrentDB.DB().SetMaxOpenConns(maxOpenConns)
+	concurrentDB.DB().SetMaxIdleConns(maxIdleConns)
+	concurrentDB.DB().SetConnMaxIdleTime(5 * time.Minute)
+
+	nonconcurrentDB, err := connectDB(filepath.Join(app.DataDir(), "logs.db"))
+	if err != nil {
+		return err
+	}
+	nonconcurrentDB.DB().SetMaxOpenConns(1)
+	nonconcurrentDB.DB().SetMaxIdleConns(1)
+	nonconcurrentDB.DB().SetConnMaxIdleTime(5 * time.Minute)
+
+	app.logsDao = daos.NewMultiDB(concurrentDB, nonconcurrentDB)
 
 	return nil
 }
 
 func (app *BaseApp) initDataDB() error {
-	var connectErr error
-	app.db, connectErr = connectDB(filepath.Join(app.DataDir(), "data.db"))
-	if connectErr != nil {
-		return connectErr
+	maxOpenConns := DefaultDataMaxOpenConns
+	maxIdleConns := DefaultDataMaxIdleConns
+	if app.dataMaxOpenConns > 0 {
+		maxOpenConns = app.dataMaxOpenConns
 	}
+	if app.dataMaxIdleConns > 0 {
+		maxIdleConns = app.dataMaxIdleConns
+	}
+
+	concurrentDB, err := connectDB(filepath.Join(app.DataDir(), "data.db"))
+	if err != nil {
+		return err
+	}
+	concurrentDB.DB().SetMaxOpenConns(maxOpenConns)
+	concurrentDB.DB().SetMaxIdleConns(maxIdleConns)
+	concurrentDB.DB().SetConnMaxIdleTime(5 * time.Minute)
+
+	nonconcurrentDB, err := connectDB(filepath.Join(app.DataDir(), "data.db"))
+	if err != nil {
+		return err
+	}
+	nonconcurrentDB.DB().SetMaxOpenConns(1)
+	nonconcurrentDB.DB().SetMaxIdleConns(1)
+	nonconcurrentDB.DB().SetConnMaxIdleTime(5 * time.Minute)
 
 	if app.IsDebug() {
-		app.db.QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		nonconcurrentDB.QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
 			color.HiBlack("[%.2fms] %v\n", float64(t.Milliseconds()), sql)
 		}
+		concurrentDB.QueryLogFunc = nonconcurrentDB.QueryLogFunc
 
-		app.db.ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		nonconcurrentDB.ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
 			color.HiBlack("[%.2fms] %v\n", float64(t.Milliseconds()), sql)
 		}
+		concurrentDB.ExecLogFunc = nonconcurrentDB.ExecLogFunc
 	}
 
-	app.dao = app.createDaoWithHooks(app.db)
+	app.dao = app.createDaoWithHooks(concurrentDB, nonconcurrentDB)
 
 	return nil
 }
 
-func (app *BaseApp) createDaoWithHooks(db dbx.Builder) *daos.Dao {
-	dao := daos.New(db)
+func (app *BaseApp) createDaoWithHooks(concurrentDB, nonconcurrentDB dbx.Builder) *daos.Dao {
+	dao := daos.NewMultiDB(concurrentDB, nonconcurrentDB)
 
 	dao.BeforeCreateFunc = func(eventDao *daos.Dao, m models.Model) error {
 		return app.OnModelBeforeCreate().Trigger(&ModelEvent{eventDao, m})
@@ -832,20 +940,29 @@ func (app *BaseApp) registerDefaultHooks() {
 
 		failed := fs.DeletePrefix(prefix)
 		if len(failed) > 0 {
-			return errors.New("Failed to delete the files at " + prefix)
+			return errors.New("failed to delete the files at " + prefix)
 		}
 
 		return nil
 	}
 
-	// delete storage files from deleted Collection, Records, etc.
+	// try to delete the storage files from deleted Collection, Records, etc. model
 	app.OnModelAfterDelete().Add(func(e *ModelEvent) error {
 		if m, ok := e.Model.(models.FilesManager); ok && m.BaseFilesPath() != "" {
-			if err := deletePrefix(m.BaseFilesPath()); err != nil && app.IsDebug() {
-				// non critical error - only log for debug
-				// (usually could happen because of S3 api limits)
-				log.Println(err)
-			}
+			prefix := m.BaseFilesPath()
+
+			// run in the background for "optimistic" delete to avoid
+			// blocking the delete transaction
+			//
+			// @todo consider creating a bg process queue so that the
+			// call could be "retried" in case of a failure.
+			routine.FireAndForget(func() {
+				if err := deletePrefix(prefix); err != nil && app.IsDebug() {
+					// non critical error - only log for debug
+					// (usually could happen because of S3 api limits)
+					log.Println(err)
+				}
+			})
 		}
 
 		return nil

core/base_test.go

@@ -11,7 +11,11 @@ func TestNewBaseApp(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
-	app := NewBaseApp(testDataDir, "test_env", true)
+	app := NewBaseApp(&BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "test_env",
+		IsDebug:       true,
+	})
 
 	if app.dataDir != testDataDir {
 		t.Fatalf("expected dataDir %q, got %q", testDataDir, app.dataDir)
@@ -42,14 +46,26 @@ func TestBaseAppBootstrap(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
-	app := NewBaseApp(testDataDir, "pb_test_env", false)
+	app := NewBaseApp(&BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "pb_test_env",
+		IsDebug:       false,
+	})
 	defer app.ResetBootstrapState()
 
+	if app.IsBootstrapped() {
+		t.Fatal("Didn't expect the application to be bootstrapped.")
+	}
+
 	// bootstrap
 	if err := app.Bootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
+	if !app.IsBootstrapped() {
+		t.Fatal("Expected the application to be bootstrapped.")
+	}
+
 	if stat, err := os.Stat(testDataDir); err != nil || !stat.IsDir() {
 		t.Fatal("Expected test data directory to be created.")
 	}
@@ -112,29 +128,33 @@ func TestBaseAppGetters(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
-	app := NewBaseApp(testDataDir, "pb_test_env", false)
+	app := NewBaseApp(&BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "pb_test_env",
+		IsDebug:       false,
+	})
 	defer app.ResetBootstrapState()
 
 	if err := app.Bootstrap(); err != nil {
 		t.Fatal(err)
 	}
 
-	if app.db != app.DB() {
-		t.Fatalf("Expected app.DB %v, got %v", app.DB(), app.db)
-	}
-
 	if app.dao != app.Dao() {
 		t.Fatalf("Expected app.Dao %v, got %v", app.Dao(), app.dao)
 	}
 
-	if app.logsDB != app.LogsDB() {
-		t.Fatalf("Expected app.LogsDB %v, got %v", app.LogsDB(), app.logsDB)
+	if app.dao.ConcurrentDB() != app.DB() {
+		t.Fatalf("Expected app.DB %v, got %v", app.DB(), app.dao.ConcurrentDB())
 	}
 
 	if app.logsDao != app.LogsDao() {
 		t.Fatalf("Expected app.LogsDao %v, got %v", app.LogsDao(), app.logsDao)
 	}
 
+	if app.logsDao.ConcurrentDB() != app.LogsDB() {
+		t.Fatalf("Expected app.LogsDB %v, got %v", app.LogsDB(), app.logsDao.ConcurrentDB())
+	}
+
 	if app.dataDir != app.DataDir() {
 		t.Fatalf("Expected app.DataDir %v, got %v", app.DataDir(), app.dataDir)
 	}
@@ -400,7 +420,11 @@ func TestBaseAppNewMailClient(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
-	app := NewBaseApp(testDataDir, "pb_test_env", false)
+	app := NewBaseApp(&BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "pb_test_env",
+		IsDebug:       false,
+	})
 
 	client1 := app.NewMailClient()
 	if val, ok := client1.(*mailer.Sendmail); !ok {
@@ -419,7 +443,11 @@ func TestBaseAppNewFilesystem(t *testing.T) {
 	const testDataDir = "./pb_base_app_test_data_dir/"
 	defer os.RemoveAll(testDataDir)
 
-	app := NewBaseApp(testDataDir, "pb_test_env", false)
+	app := NewBaseApp(&BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "pb_test_env",
+		IsDebug:       false,
+	})
 
 	// local
 	local, localErr := app.NewFilesystem()

core/db.go

@@ -0,0 +1,20 @@
+package core
+
+import (
+	"github.com/pocketbase/dbx"
+)
+
+func initPragmas(db *dbx.DB) error {
+	// note: the busy_timeout pragma must be first because
+	// the connection needs to be set to block on busy before WAL mode
+	// is set in case it hasn't been already set by another connection
+	_, err := db.NewQuery(`
+		PRAGMA busy_timeout       = 10000;
+		PRAGMA journal_mode       = WAL;
+		PRAGMA journal_size_limit = 100000000;
+		PRAGMA synchronous        = NORMAL;
+		PRAGMA foreign_keys       = TRUE;
+	`).Execute()
+
+	return err
+}

core/db_cgo.go

@@ -3,35 +3,20 @@
 package core
 
 import (
-	"fmt"
-	"time"
-
 	"github.com/pocketbase/dbx"
 	_ "github.com/mattn/go-sqlite3"
 )
 
 func connectDB(dbPath string) (*dbx.DB, error) {
-	// note: the busy_timeout pragma must be first because
-	// the connection needs to be set to block on busy before WAL mode
-	// is set in case it hasn't been already set by another connection
-	pragmas := "_busy_timeout=10000&_journal_mode=WAL&_foreign_keys=1&_synchronous=NORMAL"
-
-	db, openErr := dbx.MustOpen("sqlite3", fmt.Sprintf("%s?%s", dbPath, pragmas))
-	if openErr != nil {
-		return nil, openErr
+	db, err := dbx.Open("sqlite3", dbPath)
+	if err != nil {
+		return nil, err
 	}
 
-	// use a fixed connection pool to limit the SQLITE_BUSY errors
-	// and reduce the open file descriptors
-	// (the limits are arbitrary and may change in the future)
-	db.DB().SetMaxOpenConns(800)
-	db.DB().SetMaxIdleConns(30)
-	db.DB().SetConnMaxIdleTime(5 * time.Minute)
+	if err := initPragmas(db); err != nil {
+		db.Close()
+		return nil, err
+	}
 
-	// additional pragmas not supported through the dsn string
-	_, err := db.NewQuery(`
-		pragma journal_size_limit = 100000000;
-	`).Execute()
-
-	return db, err
+	return db, nil
 }

core/db_nocgo.go

@@ -3,32 +3,20 @@
 package core
 
 import (
-	"fmt"
-	"time"
-
 	"github.com/pocketbase/dbx"
 	_ "modernc.org/sqlite"
 )
 
 func connectDB(dbPath string) (*dbx.DB, error) {
-	// note: the busy_timeout pragma must be first because
-	// the connection needs to be set to block on busy before WAL mode
-	// is set in case it hasn't been already set by another connection
-	pragmas := "_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=foreign_keys(1)&_pragma=synchronous(NORMAL)&_pragma=journal_size_limit(100000000)"
-
-	db, err := dbx.MustOpen("sqlite", fmt.Sprintf("%s?%s", dbPath, pragmas))
+	db, err := dbx.Open("sqlite", dbPath)
 	if err != nil {
 		return nil, err
 	}
 
-	// use a fixed connection pool to limit the SQLITE_BUSY errors and
-	// reduce the open file descriptors
-	// (the limits are arbitrary and may change in the future)
-	//
-	// @see https://gitlab.com/cznic/sqlite/-/issues/115
-	db.DB().SetMaxOpenConns(800)
-	db.DB().SetMaxIdleConns(30)
-	db.DB().SetConnMaxIdleTime(5 * time.Minute)
+	if err := initPragmas(db); err != nil {
+		db.Close()
+		return nil, err
+	}
 
 	return db, nil
 }

daos/admin.go

@@ -97,8 +97,8 @@ func (dao *Dao) IsAdminEmailUnique(email string, excludeIds ...string) bool {
 		AndWhere(dbx.HashExp{"email": email}).
 		Limit(1)
 
-	if len(excludeIds) > 0 {
-		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(excludeIds)...))
+	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
+		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(uniqueExcludeIds)...))
 	}
 
 	var exists bool

daos/base.go

@@ -17,17 +17,29 @@ import (
 
 const DefaultMaxFailRetries = 5
 
-// New creates a new Dao instance with the provided db builder.
+// New creates a new Dao instance with the provided db builder
+// (for both async and sync db operations).
 func New(db dbx.Builder) *Dao {
+	return NewMultiDB(db, db)
+}
+
+// New creates a new Dao instance with the provided dedicated
+// async and sync db builders.
+func NewMultiDB(concurrentDB, nonconcurrentDB dbx.Builder) *Dao {
 	return &Dao{
-		db: db,
+		concurrentDB:    concurrentDB,
+		nonconcurrentDB: nonconcurrentDB,
 	}
 }
 
 // Dao handles various db operations.
 // Think of Dao as a repository and service layer in one.
 type Dao struct {
-	db  dbx.Builder
+	// in a transaction both refer to the same *dbx.TX instance
+	concurrentDB    dbx.Builder
+	nonconcurrentDB dbx.Builder
+
+	// @todo delete after removing Block and Continue
 	sem *semaphore.Weighted
 	mux sync.RWMutex
 
@@ -39,11 +51,31 @@ type Dao struct {
 	AfterDeleteFunc  func(eventDao *Dao, m models.Model)
 }
 
-// DB returns the internal db builder (*dbx.DB or *dbx.TX).
+// DB returns the default dao db builder (*dbx.DB or *dbx.TX).
+//
+// Currently the default db builder is dao.concurrentDB but that may change in the future.
 func (dao *Dao) DB() dbx.Builder {
-	return dao.db
+	return dao.ConcurrentDB()
 }
 
+// ConcurrentDB returns the dao concurrent (aka. multiple open connections)
+// db builder (*dbx.DB or *dbx.TX).
+//
+// In a transaction the concurrentDB and nonconcurrentDB refer to the same *dbx.TX instance.
+func (dao *Dao) ConcurrentDB() dbx.Builder {
+	return dao.concurrentDB
+}
+
+// NonconcurrentDB returns the dao nonconcurrent (aka. single open connection)
+// db builder (*dbx.DB or *dbx.TX).
+//
+// In a transaction the concurrentDB and nonconcurrentDB refer to the same *dbx.TX instance.
+func (dao *Dao) NonconcurrentDB() dbx.Builder {
+	return dao.nonconcurrentDB
+}
+
+// Deprecated: Will be removed in the next releases. Use [Dao.NonconcurrentDB()] instead.
+//
 // Block acquires a lock and blocks all other go routines that uses
 // the Dao instance until dao.Continue() is called, effectively making
 // the concurrent requests to perform synchronous db operations.
@@ -61,9 +93,9 @@ func (dao *Dao) DB() dbx.Builder {
 // 		}
 // 		defer app.Dao().Continue()
 //
-//  	return app.Dao().RunInTransaction(func (txDao *daos.Dao) error {
-//  	    // some long running read&write transaction...
-//  	})
+// 		return app.Dao().RunInTransaction(func (txDao *daos.Dao) error {
+// 			// some long running read&write transaction...
+// 		})
 // 	}
 func (dao *Dao) Block(ctx context.Context) error {
 	if dao.sem == nil {
@@ -75,6 +107,8 @@ func (dao *Dao) Block(ctx context.Context) error {
 	return dao.sem.Acquire(ctx, 1)
 }
 
+// Deprecated: Will be removed in the next releases. Use [Dao.NonconcurrentDB()] instead.
+//
 // Continue releases the previously acquired Block() lock.
 func (dao *Dao) Continue() {
 	if dao.sem == nil {
@@ -88,7 +122,7 @@ func (dao *Dao) Continue() {
 // based on the provided model argument.
 func (dao *Dao) ModelQuery(m models.Model) *dbx.SelectQuery {
 	tableName := m.TableName()
-	return dao.db.Select("{{" + tableName + "}}.*").From(tableName)
+	return dao.DB().Select("{{" + tableName + "}}.*").From(tableName)
 }
 
 // FindById finds a single db record with the specified id and
@@ -105,13 +139,13 @@ type afterCallGroup struct {
 
 // RunInTransaction wraps fn into a transaction.
 //
-// It is safe to nest RunInTransaction calls.
+// It is safe to nest RunInTransaction calls as long as you use the txDao.
 func (dao *Dao) RunInTransaction(fn func(txDao *Dao) error) error {
-	switch txOrDB := dao.db.(type) {
+	switch txOrDB := dao.NonconcurrentDB().(type) {
 	case *dbx.Tx:
 		// nested transactions are not supported by default
 		// so execute the function within the current transaction
-
+		// ---
 		// create a new dao with the same hooks to avoid semaphore deadlock when nesting
 		txDao := New(txOrDB)
 		txDao.BeforeCreateFunc = dao.BeforeCreateFunc
@@ -165,14 +199,15 @@ func (dao *Dao) RunInTransaction(fn func(txDao *Dao) error) error {
 
 		if txError == nil {
 			// execute after event calls on successful transaction
+			// (note: using the non-transaction dao to allow following queries in the after hooks)
 			for _, call := range afterCalls {
 				switch call.Action {
 				case "create":
-					dao.AfterCreateFunc(call.EventDao, call.Model)
+					dao.AfterCreateFunc(dao, call.Model)
 				case "update":
-					dao.AfterUpdateFunc(call.EventDao, call.Model)
+					dao.AfterUpdateFunc(dao, call.Model)
 				case "delete":
-					dao.AfterDeleteFunc(call.EventDao, call.Model)
+					dao.AfterDeleteFunc(dao, call.Model)
 				}
 			}
 		}
@@ -180,7 +215,7 @@ func (dao *Dao) RunInTransaction(fn func(txDao *Dao) error) error {
 		return txError
 	}
 
-	return errors.New("Failed to start transaction (unknown dao.db)")
+	return errors.New("failed to start transaction (unknown dao.NonconcurrentDB() instance)")
 }
 
 // Delete deletes the provided model.
@@ -196,7 +231,7 @@ func (dao *Dao) Delete(m models.Model) error {
 			}
 		}
 
-		if err := retryDao.db.Model(m).Delete(); err != nil {
+		if err := retryDao.NonconcurrentDB().Model(m).Delete(); err != nil {
 			return err
 		}
 
@@ -241,7 +276,7 @@ func (dao *Dao) update(m models.Model) error {
 	if v, ok := any(m).(models.ColumnValueMapper); ok {
 		dataMap := v.ColumnValueMap()
 
-		_, err := dao.db.Update(
+		_, err := dao.NonconcurrentDB().Update(
 			m.TableName(),
 			dataMap,
 			dbx.HashExp{"id": m.GetId()},
@@ -251,7 +286,7 @@ func (dao *Dao) update(m models.Model) error {
 			return err
 		}
 	} else {
-		if err := dao.db.Model(m).Update(); err != nil {
+		if err := dao.NonconcurrentDB().Model(m).Update(); err != nil {
 			return err
 		}
 	}
@@ -292,12 +327,12 @@ func (dao *Dao) create(m models.Model) error {
 			dataMap["id"] = m.GetId()
 		}
 
-		_, err := dao.db.Insert(m.TableName(), dataMap).Execute()
+		_, err := dao.NonconcurrentDB().Insert(m.TableName(), dataMap).Execute()
 		if err != nil {
 			return err
 		}
 	} else {
-		if err := dao.db.Model(m).Insert(); err != nil {
+		if err := dao.NonconcurrentDB().Model(m).Insert(); err != nil {
 			return err
 		}
 	}
@@ -320,12 +355,10 @@ Retry:
 	if attempts == 2 {
 		// assign new Dao without the before hooks to avoid triggering
 		// the already fired before callbacks multiple times
-		retryDao = &Dao{
-			db:              dao.db,
-			AfterCreateFunc: dao.AfterCreateFunc,
-			AfterUpdateFunc: dao.AfterUpdateFunc,
-			AfterDeleteFunc: dao.AfterDeleteFunc,
-		}
+		retryDao = NewMultiDB(dao.concurrentDB, dao.nonconcurrentDB)
+		retryDao.AfterCreateFunc = dao.AfterCreateFunc
+		retryDao.AfterUpdateFunc = dao.AfterUpdateFunc
+		retryDao.AfterDeleteFunc = dao.AfterDeleteFunc
 	}
 
 	// execute

daos/base_test.go

@@ -20,6 +20,25 @@ func TestNew(t *testing.T) {
 	}
 }
 
+func TestNewMultiDB(t *testing.T) {
+	testApp, _ := tests.NewTestApp()
+	defer testApp.Cleanup()
+
+	dao := daos.NewMultiDB(testApp.Dao().ConcurrentDB(), testApp.Dao().NonconcurrentDB())
+
+	if dao.DB() != testApp.Dao().ConcurrentDB() {
+		t.Fatal("[db-concurrentDB] The 2 db instances are different")
+	}
+
+	if dao.ConcurrentDB() != testApp.Dao().ConcurrentDB() {
+		t.Fatal("[concurrentDB-concurrentDB] The 2 db instances are different")
+	}
+
+	if dao.NonconcurrentDB() != testApp.Dao().NonconcurrentDB() {
+		t.Fatal("[nonconcurrentDB-nonconcurrentDB] The 2 db instances are different")
+	}
+}
+
 func TestDaoModelQuery(t *testing.T) {
 	testApp, _ := tests.NewTestApp()
 	defer testApp.Cleanup()

daos/collection.go

@@ -65,8 +65,7 @@ func (dao *Dao) IsCollectionNameUnique(name string, excludeIds ...string) bool {
 		AndWhere(dbx.NewExp("LOWER([[name]])={:name}", dbx.Params{"name": strings.ToLower(name)})).
 		Limit(1)
 
-	if len(excludeIds) > 0 {
-		uniqueExcludeIds := list.NonzeroUniques(excludeIds)
+	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
 		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(uniqueExcludeIds)...))
 	}
 
@@ -85,15 +84,17 @@ func (dao *Dao) FindCollectionReferences(collection *models.Collection, excludeI
 	collections := []*models.Collection{}
 
 	query := dao.CollectionQuery()
-	if len(excludeIds) > 0 {
-		uniqueExcludeIds := list.NonzeroUniques(excludeIds)
+
+	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
 		query.AndWhere(dbx.NotIn("id", list.ToInterfaceSlice(uniqueExcludeIds)...))
 	}
+
 	if err := query.All(&collections); err != nil {
 		return nil, err
 	}
 
 	result := map[*models.Collection][]*schema.SchemaField{}
+
 	for _, c := range collections {
 		for _, f := range c.Schema.Fields() {
 			if f.Type != schema.FieldTypeRelation {

daos/collection_test.go

@@ -37,7 +37,7 @@ func TestFindCollectionsByType(t *testing.T) {
 		{"", false, 0},
 		{"unknown", false, 0},
 		{models.CollectionTypeAuth, false, 3},
-		{models.CollectionTypeBase, false, 4},
+		{models.CollectionTypeBase, false, 5},
 	}
 
 	for i, scenario := range scenarios {
@@ -122,7 +122,13 @@ func TestFindCollectionReferences(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	result, err := app.Dao().FindCollectionReferences(collection, collection.Id)
+	result, err := app.Dao().FindCollectionReferences(
+		collection,
+		collection.Id,
+		// test whether "nonempty" exclude ids condition will be skipped
+		"",
+		"",
+	)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -296,7 +302,7 @@ func TestImportCollections(t *testing.T) {
 			name:                   "empty collections",
 			jsonData:               `[]`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 		},
 		{
 			name: "minimal collection import",
@@ -306,7 +312,7 @@ func TestImportCollections(t *testing.T) {
 			]`,
 			deleteMissing:          false,
 			expectError:            false,
-			expectCollectionsCount: 9,
+			expectCollectionsCount: 10,
 		},
 		{
 			name: "minimal collection import + failed beforeRecordsSync",
@@ -318,7 +324,7 @@ func TestImportCollections(t *testing.T) {
 			},
 			deleteMissing:          false,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 		},
 		{
 			name: "minimal collection import + successful beforeRecordsSync",
@@ -330,7 +336,7 @@ func TestImportCollections(t *testing.T) {
 			},
 			deleteMissing:          false,
 			expectError:            false,
-			expectCollectionsCount: 8,
+			expectCollectionsCount: 9,
 		},
 		{
 			name: "new + update + delete system collection",
@@ -366,7 +372,7 @@ func TestImportCollections(t *testing.T) {
 			]`,
 			deleteMissing:          true,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 		},
 		{
 			name: "new + update + delete non-system collection",
@@ -495,7 +501,7 @@ func TestImportCollections(t *testing.T) {
 			]`,
 			deleteMissing:          false,
 			expectError:            false,
-			expectCollectionsCount: 8,
+			expectCollectionsCount: 9,
 			afterTestFunc: func(testApp *tests.TestApp, resultCollections []*models.Collection) {
 				expectedCollectionFields := map[string]int{
 					"nologin":    1,
@@ -503,6 +509,7 @@ func TestImportCollections(t *testing.T) {
 					"demo2":      2,
 					"demo3":      2,
 					"demo4":      11,
+					"demo5":      5,
 					"new_import": 1,
 				}
 				for name, expectedCount := range expectedCollectionFields {

daos/record.go

@@ -1,11 +1,9 @@
 package daos
 
 import (
-	"context"
 	"errors"
 	"fmt"
 	"strings"
-	"time"
 
 	"github.com/pocketbase/dbx"
 	"github.com/pocketbase/pocketbase/models"
@@ -85,7 +83,7 @@ func (dao *Dao) FindRecordsByIds(
 		}
 	}
 
-	rows := []dbx.NullStringMap{}
+	rows := make([]dbx.NullStringMap, 0, len(recordIds))
 	if err := query.All(&rows); err != nil {
 		return nil, err
 	}
@@ -193,8 +191,7 @@ func (dao *Dao) IsRecordValueUnique(
 		AndWhere(expr).
 		Limit(1)
 
-	if len(excludeIds) > 0 {
-		uniqueExcludeIds := list.NonzeroUniques(excludeIds)
+	if uniqueExcludeIds := list.NonzeroUniques(excludeIds); len(uniqueExcludeIds) > 0 {
 		query.AndWhere(dbx.NotIn(collection.Name+".id", list.ToInterfaceSlice(uniqueExcludeIds)...))
 	}
 
@@ -216,7 +213,7 @@ func (dao *Dao) FindAuthRecordByToken(token string, baseTokenKey string) (*model
 	id, _ := unverifiedClaims["id"].(string)
 	collectionId, _ := unverifiedClaims["collectionId"].(string)
 	if id == "" || collectionId == "" {
-		return nil, errors.New("Missing or invalid token claims.")
+		return nil, errors.New("missing or invalid token claims")
 	}
 
 	record, err := dao.FindRecordById(collectionId, id)
@@ -243,8 +240,11 @@ func (dao *Dao) FindAuthRecordByToken(token string, baseTokenKey string) (*model
 // Returns an error if it is not an auth collection or the record is not found.
 func (dao *Dao) FindAuthRecordByEmail(collectionNameOrId string, email string) (*models.Record, error) {
 	collection, err := dao.FindCollectionByNameOrId(collectionNameOrId)
-	if err != nil || !collection.IsAuth() {
-		return nil, errors.New("Missing or not an auth collection.")
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch auth collection %q (%w)", collectionNameOrId, err)
+	}
+	if !collection.IsAuth() {
+		return nil, fmt.Errorf("%q is not an auth collection", collectionNameOrId)
 	}
 
 	row := dbx.NullStringMap{}
@@ -266,8 +266,11 @@ func (dao *Dao) FindAuthRecordByEmail(collectionNameOrId string, email string) (
 // Returns an error if it is not an auth collection or the record is not found.
 func (dao *Dao) FindAuthRecordByUsername(collectionNameOrId string, username string) (*models.Record, error) {
 	collection, err := dao.FindCollectionByNameOrId(collectionNameOrId)
-	if err != nil || !collection.IsAuth() {
-		return nil, errors.New("Missing or not an auth collection.")
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch auth collection %q (%w)", collectionNameOrId, err)
+	}
+	if !collection.IsAuth() {
+		return nil, fmt.Errorf("%q is not an auth collection", collectionNameOrId)
 	}
 
 	row := dbx.NullStringMap{}
@@ -344,10 +347,10 @@ func (dao *Dao) SaveRecord(record *models.Record) error {
 // DeleteRecord deletes the provided Record model.
 //
 // This method will also cascade the delete operation to all linked
-// relational records (delete or set to NULL, depending on the rel settings).
+// relational records (delete or unset, depending on the rel settings).
 //
 // The delete operation may fail if the record is part of a required
-// reference in another record (aka. cannot be deleted or set to NULL).
+// reference in another record (aka. cannot be deleted or unset).
 func (dao *Dao) DeleteRecord(record *models.Record) error {
 	// fetch rel references (if any)
 	//
@@ -358,234 +361,128 @@ func (dao *Dao) DeleteRecord(record *models.Record) error {
 		return err
 	}
 
-	// run all consequent DeleteRecord requests synchroniously
-	// to minimize SQLITE_BUSY errors
-	if len(refs) > 0 {
-		ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
-		defer cancel()
-		if err := dao.Block(ctx); err != nil {
-			return err
-		}
-		defer dao.Continue()
-	}
-
 	return dao.RunInTransaction(func(txDao *Dao) error {
-		// always delete the record first to ensure that there will be no "A<->B"
-		// relations to prevent deadlock when calling DeleteRecord recursively
+		// manually trigger delete on any linked external auth to ensure
+		// that the `OnModel*` hooks are triggered
+		if record.Collection().IsAuth() {
+			// note: the select is outside of the transaction to minimize
+			// SQLITE_BUSY errors when mixing read&write in a single transaction
+			externalAuths, err := dao.FindAllExternalAuthsByRecord(record)
+			if err != nil {
+				return err
+			}
+			for _, auth := range externalAuths {
+				if err := txDao.DeleteExternalAuth(auth); err != nil {
+					return err
+				}
+			}
+		}
+
+		// delete the record before the relation references to ensure that there
+		// will be no "A<->B" relations to prevent deadlock when calling DeleteRecord recursively
 		if err := txDao.Delete(record); err != nil {
 			return err
 		}
 
-		// check if related records has to be deleted (if `CascadeDelete` is set)
-		// OR
-		// just unset the record id from any relation field values (if they are not required)
-		uniqueJsonEachAlias := "__je__" + security.PseudorandomString(5)
-		for refCollection, fields := range refs {
-			for _, field := range fields {
-				options, _ := field.Options.(*schema.RelationOptions)
+		return txDao.cascadeRecordDelete(record, refs)
+	})
+}
 
-				rows := []dbx.NullStringMap{}
+// cascadeRecordDelete triggers cascade deletion for the provided references.
+//
+// NB! This method is expected to be called inside a transaction.
+func (dao *Dao) cascadeRecordDelete(mainRecord *models.Record, refs map[*models.Collection][]*schema.SchemaField) error {
+	uniqueJsonEachAlias := "__je__" + security.PseudorandomString(4)
 
-				// fetch all referenced records
-				recordTableName := inflector.Columnify(refCollection.Name)
-				prefixedFieldName := recordTableName + "." + inflector.Columnify(field.Name)
-				err := txDao.RecordQuery(refCollection).
-					Distinct(true).
-					LeftJoin(fmt.Sprintf(
-						// note: the case is used to normalize value access for single and multiple relations.
-						`json_each(CASE WHEN json_valid([[%s]]) THEN [[%s]] ELSE json_array([[%s]]) END) as {{%s}}`,
-						prefixedFieldName, prefixedFieldName, prefixedFieldName, uniqueJsonEachAlias,
-					), nil).
-					AndWhere(dbx.Not(dbx.HashExp{recordTableName + ".id": record.Id})).
-					AndWhere(dbx.HashExp{uniqueJsonEachAlias + ".value": record.Id}).
-					All(&rows)
-				if err != nil {
+	for refCollection, fields := range refs {
+		for _, field := range fields {
+			recordTableName := inflector.Columnify(refCollection.Name)
+			prefixedFieldName := recordTableName + "." + inflector.Columnify(field.Name)
+
+			// @todo optimize single relation lookup in v0.12+
+			query := dao.RecordQuery(refCollection).
+				Distinct(true).
+				AndWhere(dbx.Not(dbx.HashExp{recordTableName + ".id": mainRecord.Id})).
+				InnerJoin(fmt.Sprintf(
+					// note: the case is used to normalize the value access
+					`json_each(CASE WHEN json_valid([[%s]]) THEN [[%s]] ELSE json_array([[%s]]) END) as {{%s}}`,
+					prefixedFieldName, prefixedFieldName, prefixedFieldName, uniqueJsonEachAlias,
+				), dbx.HashExp{uniqueJsonEachAlias + ".value": mainRecord.Id})
+
+			// trigger cascade for each batchSize rel items until there is none
+			batchSize := 4000
+			rows := make([]dbx.NullStringMap, 0, batchSize)
+			for {
+				if err := query.Limit(int64(batchSize)).All(&rows); err != nil {
 					return err
 				}
 
+				total := len(rows)
+				if total == 0 {
+					break
+				}
+
 				refRecords := models.NewRecordsFromNullStringMaps(refCollection, rows)
-				for _, refRecord := range refRecords {
-					ids := refRecord.GetStringSlice(field.Name)
 
-					// unset the record id
-					for i := len(ids) - 1; i >= 0; i-- {
-						if ids[i] == record.Id {
-							ids = append(ids[:i], ids[i+1:]...)
-							break
-						}
-					}
-
-					// cascade delete the reference
-					// (only if there are no other active references in case of multiple select)
-					if options.CascadeDelete && len(ids) == 0 {
-						if err := txDao.DeleteRecord(refRecord); err != nil {
-							return err
-						}
-						// no further actions are needed (the reference is deleted)
-						continue
-					}
-
-					if field.Required && len(ids) == 0 {
-						return fmt.Errorf("The record cannot be deleted because it is part of a required reference in record %s (%s collection).", refRecord.Id, refCollection.Name)
-					}
-
-					// save the reference changes
-					refRecord.Set(field.Name, field.PrepareValue(ids))
-					if err := txDao.SaveRecord(refRecord); err != nil {
-						return err
-					}
+				err := dao.deleteRefRecords(mainRecord, refRecords, field)
+				if err != nil {
+					return err
 				}
+
+				if total < batchSize {
+					break // no more items
+				}
+
+				rows = rows[:0] // keep allocated memory
 			}
 		}
-
-		// delete linked external auths
-		if record.Collection().IsAuth() {
-			_, err = txDao.DB().Delete((&models.ExternalAuth{}).TableName(), dbx.HashExp{
-				"collectionId": record.Collection().Id,
-				"recordId":     record.Id,
-			}).Execute()
-			if err != nil {
-				return err
-			}
-		}
-
-		return nil
-	})
-}
-
-// SyncRecordTableSchema compares the two provided collections
-// and applies the necessary related record table changes.
-//
-// If `oldCollection` is null, then only `newCollection` is used to create the record table.
-func (dao *Dao) SyncRecordTableSchema(newCollection *models.Collection, oldCollection *models.Collection) error {
-	// create
-	if oldCollection == nil {
-		cols := map[string]string{
-			schema.FieldNameId:      "TEXT PRIMARY KEY",
-			schema.FieldNameCreated: "TEXT DEFAULT '' NOT NULL",
-			schema.FieldNameUpdated: "TEXT DEFAULT '' NOT NULL",
-		}
-
-		if newCollection.IsAuth() {
-			cols[schema.FieldNameUsername] = "TEXT NOT NULL"
-			cols[schema.FieldNameEmail] = "TEXT DEFAULT '' NOT NULL"
-			cols[schema.FieldNameEmailVisibility] = "BOOLEAN DEFAULT FALSE NOT NULL"
-			cols[schema.FieldNameVerified] = "BOOLEAN DEFAULT FALSE NOT NULL"
-			cols[schema.FieldNameTokenKey] = "TEXT NOT NULL"
-			cols[schema.FieldNamePasswordHash] = "TEXT NOT NULL"
-			cols[schema.FieldNameLastResetSentAt] = "TEXT DEFAULT '' NOT NULL"
-			cols[schema.FieldNameLastVerificationSentAt] = "TEXT DEFAULT '' NOT NULL"
-		}
-
-		// ensure that the new collection has an id
-		if !newCollection.HasId() {
-			newCollection.RefreshId()
-			newCollection.MarkAsNew()
-		}
-
-		tableName := newCollection.Name
-
-		// add schema field definitions
-		for _, field := range newCollection.Schema.Fields() {
-			cols[field.Name] = field.ColDefinition()
-		}
-
-		// create table
-		if _, err := dao.DB().CreateTable(tableName, cols).Execute(); err != nil {
-			return err
-		}
-
-		// add named index on the base `created` column
-		if _, err := dao.DB().CreateIndex(tableName, "_"+newCollection.Id+"_created_idx", "created").Execute(); err != nil {
-			return err
-		}
-
-		// add named unique index on the email and tokenKey columns
-		if newCollection.IsAuth() {
-			_, err := dao.DB().NewQuery(fmt.Sprintf(
-				`
-				CREATE UNIQUE INDEX _%s_username_idx ON {{%s}} ([[username]]);
-				CREATE UNIQUE INDEX _%s_email_idx ON {{%s}} ([[email]]) WHERE [[email]] != '';
-				CREATE UNIQUE INDEX _%s_tokenKey_idx ON {{%s}} ([[tokenKey]]);
-				`,
-				newCollection.Id, tableName,
-				newCollection.Id, tableName,
-				newCollection.Id, tableName,
-			)).Execute()
-			if err != nil {
-				return err
-			}
-		}
-
-		return nil
 	}
 
-	// update
-	return dao.RunInTransaction(func(txDao *Dao) error {
-		oldTableName := oldCollection.Name
-		newTableName := newCollection.Name
-		oldSchema := oldCollection.Schema
-		newSchema := newCollection.Schema
-
-		// check for renamed table
-		if !strings.EqualFold(oldTableName, newTableName) {
-			_, err := txDao.DB().RenameTable(oldTableName, newTableName).Execute()
-			if err != nil {
-				return err
+	return nil
+}
+
+// deleteRefRecords checks if related records has to be deleted (if `CascadeDelete` is set)
+// OR
+// just unset the record id from any relation field values (if they are not required).
+//
+// NB! This method is expected to be called inside a transaction.
+func (dao *Dao) deleteRefRecords(mainRecord *models.Record, refRecords []*models.Record, field *schema.SchemaField) error {
+	options, _ := field.Options.(*schema.RelationOptions)
+	if options == nil {
+		return errors.New("relation field options are not initialized")
+	}
+
+	for _, refRecord := range refRecords {
+		ids := refRecord.GetStringSlice(field.Name)
+
+		// unset the record id
+		for i := len(ids) - 1; i >= 0; i-- {
+			if ids[i] == mainRecord.Id {
+				ids = append(ids[:i], ids[i+1:]...)
+				break
 			}
 		}
 
-		// check for deleted columns
-		for _, oldField := range oldSchema.Fields() {
-			if f := newSchema.GetFieldById(oldField.Id); f != nil {
-				continue // exist
-			}
-
-			_, err := txDao.DB().DropColumn(newTableName, oldField.Name).Execute()
-			if err != nil {
+		// cascade delete the reference
+		// (only if there are no other active references in case of multiple select)
+		if options.CascadeDelete && len(ids) == 0 {
+			if err := dao.DeleteRecord(refRecord); err != nil {
 				return err
 			}
+			// no further actions are needed (the reference is deleted)
+			continue
 		}
-
-		// check for new or renamed columns
-		toRename := map[string]string{}
-		for _, field := range newSchema.Fields() {
-			oldField := oldSchema.GetFieldById(field.Id)
-			// Note:
-			// We are using a temporary column name when adding or renaming columns
-			// to ensure that there are no name collisions in case there is
-			// names switch/reuse of existing columns (eg. name, title -> title, name).
-			// This way we are always doing 1 more rename operation but it provides better dev experience.
 
-			if oldField == nil {
-				tempName := field.Name + security.PseudorandomString(5)
-				toRename[tempName] = field.Name
-
-				// add
-				_, err := txDao.DB().AddColumn(newTableName, tempName, field.ColDefinition()).Execute()
-				if err != nil {
-					return err
-				}
-			} else if oldField.Name != field.Name {
-				tempName := field.Name + security.PseudorandomString(5)
-				toRename[tempName] = field.Name
-
-				// rename
-				_, err := txDao.DB().RenameColumn(newTableName, oldField.Name, tempName).Execute()
-				if err != nil {
-					return err
-				}
-			}
+		if field.Required && len(ids) == 0 {
+			return fmt.Errorf("the record cannot be deleted because it is part of a required reference in record %s (%s collection)", refRecord.Id, refRecord.Collection().Name)
 		}
 
-		// set the actual columns name
-		for tempName, actualName := range toRename {
-			_, err := txDao.DB().RenameColumn(newTableName, tempName, actualName).Execute()
-			if err != nil {
-				return err
-			}
+		// save the reference changes
+		refRecord.Set(field.Name, field.PrepareValue(ids))
+		if err := dao.SaveRecord(refRecord); err != nil {
+			return err
 		}
+	}
 
-		return nil
-	})
+	return nil
 }

daos/record_expand.go

@@ -176,7 +176,7 @@ func (dao *Dao) expandRecords(records []*models.Record, expandPath string, fetch
 	}
 
 	// reindex with the rel id
-	indexedRels := map[string]*models.Record{}
+	indexedRels := make(map[string]*models.Record, len(rels))
 	for _, rel := range rels {
 		indexedRels[rel.GetId()] = rel
 	}
@@ -216,12 +216,7 @@ func (dao *Dao) expandRecords(records []*models.Record, expandPath string, fetch
 					continue
 				}
 
-				oldRelExpand := oldExpandedRel.Expand()
-				newRelExpand := rel.Expand()
-				for k, v := range oldRelExpand {
-					newRelExpand[k] = v
-				}
-				rel.SetExpand(newRelExpand)
+				rel.MergeExpand(oldExpandedRel.Expand())
 			}
 		}
 

daos/record_expand_test.go

@@ -167,6 +167,22 @@ func TestExpandRecords(t *testing.T) {
 			5,
 			0,
 		},
+		{
+			"expand multiple relations sharing a common path",
+			"demo4",
+			[]string{"qzaqccwrmva4o1n"},
+			[]string{
+				"rel_one_no_cascade",
+				"rel_many_no_cascade",
+				"self_rel_many.self_rel_one.rel_many_cascade",
+				"self_rel_many.self_rel_one.rel_many_no_cascade_required",
+			},
+			func(c *models.Collection, ids []string) ([]*models.Record, error) {
+				return app.Dao().FindRecordsByIds(c.Id, ids, nil)
+			},
+			5,
+			0,
+		},
 	}
 
 	for _, s := range scenarios {

daos/record_table_sync.go

@@ -0,0 +1,147 @@
+package daos
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/pocketbase/pocketbase/models"
+	"github.com/pocketbase/pocketbase/models/schema"
+	"github.com/pocketbase/pocketbase/tools/security"
+)
+
+// SyncRecordTableSchema compares the two provided collections
+// and applies the necessary related record table changes.
+//
+// If `oldCollection` is null, then only `newCollection` is used to create the record table.
+func (dao *Dao) SyncRecordTableSchema(newCollection *models.Collection, oldCollection *models.Collection) error {
+	// create
+	if oldCollection == nil {
+		cols := map[string]string{
+			schema.FieldNameId:      "TEXT PRIMARY KEY NOT NULL",
+			schema.FieldNameCreated: "TEXT DEFAULT '' NOT NULL",
+			schema.FieldNameUpdated: "TEXT DEFAULT '' NOT NULL",
+		}
+
+		if newCollection.IsAuth() {
+			cols[schema.FieldNameUsername] = "TEXT NOT NULL"
+			cols[schema.FieldNameEmail] = "TEXT DEFAULT '' NOT NULL"
+			cols[schema.FieldNameEmailVisibility] = "BOOLEAN DEFAULT FALSE NOT NULL"
+			cols[schema.FieldNameVerified] = "BOOLEAN DEFAULT FALSE NOT NULL"
+			cols[schema.FieldNameTokenKey] = "TEXT NOT NULL"
+			cols[schema.FieldNamePasswordHash] = "TEXT NOT NULL"
+			cols[schema.FieldNameLastResetSentAt] = "TEXT DEFAULT '' NOT NULL"
+			cols[schema.FieldNameLastVerificationSentAt] = "TEXT DEFAULT '' NOT NULL"
+		}
+
+		// ensure that the new collection has an id
+		if !newCollection.HasId() {
+			newCollection.RefreshId()
+			newCollection.MarkAsNew()
+		}
+
+		tableName := newCollection.Name
+
+		// add schema field definitions
+		for _, field := range newCollection.Schema.Fields() {
+			cols[field.Name] = field.ColDefinition()
+		}
+
+		// create table
+		if _, err := dao.DB().CreateTable(tableName, cols).Execute(); err != nil {
+			return err
+		}
+
+		// add named index on the base `created` column
+		if _, err := dao.DB().CreateIndex(tableName, "_"+newCollection.Id+"_created_idx", "created").Execute(); err != nil {
+			return err
+		}
+
+		// add named unique index on the email and tokenKey columns
+		if newCollection.IsAuth() {
+			_, err := dao.DB().NewQuery(fmt.Sprintf(
+				`
+				CREATE UNIQUE INDEX _%s_username_idx ON {{%s}} ([[username]]);
+				CREATE UNIQUE INDEX _%s_email_idx ON {{%s}} ([[email]]) WHERE [[email]] != '';
+				CREATE UNIQUE INDEX _%s_tokenKey_idx ON {{%s}} ([[tokenKey]]);
+				`,
+				newCollection.Id, tableName,
+				newCollection.Id, tableName,
+				newCollection.Id, tableName,
+			)).Execute()
+			if err != nil {
+				return err
+			}
+		}
+
+		return nil
+	}
+
+	// update
+	return dao.RunInTransaction(func(txDao *Dao) error {
+		oldTableName := oldCollection.Name
+		newTableName := newCollection.Name
+		oldSchema := oldCollection.Schema
+		newSchema := newCollection.Schema
+
+		// check for renamed table
+		if !strings.EqualFold(oldTableName, newTableName) {
+			_, err := txDao.DB().RenameTable(oldTableName, newTableName).Execute()
+			if err != nil {
+				return err
+			}
+		}
+
+		// check for deleted columns
+		for _, oldField := range oldSchema.Fields() {
+			if f := newSchema.GetFieldById(oldField.Id); f != nil {
+				continue // exist
+			}
+
+			_, err := txDao.DB().DropColumn(newTableName, oldField.Name).Execute()
+			if err != nil {
+				return err
+			}
+		}
+
+		// check for new or renamed columns
+		toRename := map[string]string{}
+		for _, field := range newSchema.Fields() {
+			oldField := oldSchema.GetFieldById(field.Id)
+			// Note:
+			// We are using a temporary column name when adding or renaming columns
+			// to ensure that there are no name collisions in case there is
+			// names switch/reuse of existing columns (eg. name, title -> title, name).
+			// This way we are always doing 1 more rename operation but it provides better dev experience.
+
+			if oldField == nil {
+				tempName := field.Name + security.PseudorandomString(5)
+				toRename[tempName] = field.Name
+
+				// add
+				_, err := txDao.DB().AddColumn(newTableName, tempName, field.ColDefinition()).Execute()
+				if err != nil {
+					return err
+				}
+			} else if oldField.Name != field.Name {
+				tempName := field.Name + security.PseudorandomString(5)
+				toRename[tempName] = field.Name
+
+				// rename
+				_, err := txDao.DB().RenameColumn(newTableName, oldField.Name, tempName).Execute()
+				if err != nil {
+					return err
+				}
+			}
+		}
+
+		// set the actual columns name
+		for tempName, actualName := range toRename {
+			_, err := txDao.DB().RenameColumn(newTableName, tempName, actualName).Execute()
+			if err != nil {
+				return err
+			}
+		}
+
+		return nil
+	})
+}

daos/record_table_sync_test.go

@@ -0,0 +1,119 @@
+package daos_test
+
+import (
+	"testing"
+
+	"github.com/pocketbase/pocketbase/models"
+	"github.com/pocketbase/pocketbase/models/schema"
+	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/list"
+)
+
+func TestSyncRecordTableSchema(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	oldCollection, err := app.Dao().FindCollectionByNameOrId("demo2")
+	if err != nil {
+		t.Fatal(err)
+	}
+	updatedCollection, err := app.Dao().FindCollectionByNameOrId("demo2")
+	if err != nil {
+		t.Fatal(err)
+	}
+	updatedCollection.Name = "demo_renamed"
+	updatedCollection.Schema.RemoveField(updatedCollection.Schema.GetFieldByName("active").Id)
+	updatedCollection.Schema.AddField(
+		&schema.SchemaField{
+			Name: "new_field",
+			Type: schema.FieldTypeEmail,
+		},
+	)
+	updatedCollection.Schema.AddField(
+		&schema.SchemaField{
+			Id:   updatedCollection.Schema.GetFieldByName("title").Id,
+			Name: "title_renamed",
+			Type: schema.FieldTypeEmail,
+		},
+	)
+
+	scenarios := []struct {
+		newCollection     *models.Collection
+		oldCollection     *models.Collection
+		expectedTableName string
+		expectedColumns   []string
+	}{
+		// new base collection
+		{
+			&models.Collection{
+				Name: "new_table",
+				Schema: schema.NewSchema(
+					&schema.SchemaField{
+						Name: "test",
+						Type: schema.FieldTypeText,
+					},
+				),
+			},
+			nil,
+			"new_table",
+			[]string{"id", "created", "updated", "test"},
+		},
+		// new auth collection
+		{
+			&models.Collection{
+				Name: "new_table_auth",
+				Type: models.CollectionTypeAuth,
+				Schema: schema.NewSchema(
+					&schema.SchemaField{
+						Name: "test",
+						Type: schema.FieldTypeText,
+					},
+				),
+			},
+			nil,
+			"new_table_auth",
+			[]string{
+				"id", "created", "updated", "test",
+				"username", "email", "verified", "emailVisibility",
+				"tokenKey", "passwordHash", "lastResetSentAt", "lastVerificationSentAt",
+			},
+		},
+		// no changes
+		{
+			oldCollection,
+			oldCollection,
+			"demo3",
+			[]string{"id", "created", "updated", "title", "active"},
+		},
+		// renamed table, deleted column, renamed columnd and new column
+		{
+			updatedCollection,
+			oldCollection,
+			"demo_renamed",
+			[]string{"id", "created", "updated", "title_renamed", "new_field"},
+		},
+	}
+
+	for i, scenario := range scenarios {
+		err := app.Dao().SyncRecordTableSchema(scenario.newCollection, scenario.oldCollection)
+		if err != nil {
+			t.Errorf("(%d) %v", i, err)
+			continue
+		}
+
+		if !app.Dao().HasTable(scenario.newCollection.Name) {
+			t.Errorf("(%d) Expected table %s to exist", i, scenario.newCollection.Name)
+		}
+
+		cols, _ := app.Dao().GetTableColumns(scenario.newCollection.Name)
+		if len(cols) != len(scenario.expectedColumns) {
+			t.Errorf("(%d) Expected columns %v, got %v", i, scenario.expectedColumns, cols)
+		}
+
+		for _, c := range cols {
+			if !list.ExistInSlice(c, scenario.expectedColumns) {
+				t.Errorf("(%d) Couldn't find column %s in %v", i, c, scenario.expectedColumns)
+			}
+		}
+	}
+}

daos/record_test.go

@@ -11,10 +11,12 @@ import (
 	"time"
 
 	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/daos"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/models/schema"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/list"
+	"github.com/pocketbase/pocketbase/tools/types"
 )
 
 func TestRecordQuery(t *testing.T) {
@@ -632,10 +634,16 @@ func TestDeleteRecord(t *testing.T) {
 	// delete existing record + cascade
 	// ---
 	calledQueries := []string{}
-	app.DB().QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+	app.Dao().NonconcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
 		calledQueries = append(calledQueries, sql)
 	}
-	app.DB().ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+	app.Dao().ConcurrentDB().(*dbx.DB).QueryLogFunc = func(ctx context.Context, t time.Duration, sql string, rows *sql.Rows, err error) {
+		calledQueries = append(calledQueries, sql)
+	}
+	app.Dao().NonconcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
+		calledQueries = append(calledQueries, sql)
+	}
+	app.Dao().ConcurrentDB().(*dbx.DB).ExecLogFunc = func(ctx context.Context, t time.Duration, sql string, result sql.Result, err error) {
 		calledQueries = append(calledQueries, sql)
 	}
 	rec3, _ := app.Dao().FindRecordById("users", "oap640cot4yru2s")
@@ -655,121 +663,135 @@ func TestDeleteRecord(t *testing.T) {
 	}
 	// ensure that the json rel fields were prefixed
 	joinedQueries := strings.Join(calledQueries, " ")
-	expectedRelManyJoin := "`demo1` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.rel_many]]) THEN [[demo1.rel_many]] ELSE json_array([[demo1.rel_many]]) END)"
+	expectedRelManyJoin := "`demo1` INNER JOIN json_each(CASE WHEN json_valid([[demo1.rel_many]]) THEN [[demo1.rel_many]] ELSE json_array([[demo1.rel_many]]) END)"
 	if !strings.Contains(joinedQueries, expectedRelManyJoin) {
 		t.Fatalf("(rec3) Expected the cascade delete to call the query \n%v, got \n%v", expectedRelManyJoin, calledQueries)
 	}
-	expectedRelOneJoin := "`demo1` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.rel_one]]) THEN [[demo1.rel_one]] ELSE json_array([[demo1.rel_one]]) END)"
+	expectedRelOneJoin := "`demo1` INNER JOIN json_each(CASE WHEN json_valid([[demo1.rel_one]]) THEN [[demo1.rel_one]] ELSE json_array([[demo1.rel_one]]) END)"
 	if !strings.Contains(joinedQueries, expectedRelOneJoin) {
 		t.Fatalf("(rec3) Expected the cascade delete to call the query \n%v, got \n%v", expectedRelOneJoin, calledQueries)
 	}
 }
 
-func TestSyncRecordTableSchema(t *testing.T) {
+func TestDeleteRecordBatchProcessing(t *testing.T) {
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
 
-	oldCollection, err := app.Dao().FindCollectionByNameOrId("demo2")
-	if err != nil {
+	if err := createMockBatchProcessingData(app.Dao()); err != nil {
 		t.Fatal(err)
 	}
-	updatedCollection, err := app.Dao().FindCollectionByNameOrId("demo2")
-	if err != nil {
+
+	// find and delete the first c1 record to trigger cascade
+	mainRecord, _ := app.Dao().FindRecordById("c1", "a")
+	if err := app.Dao().DeleteRecord(mainRecord); err != nil {
 		t.Fatal(err)
 	}
-	updatedCollection.Name = "demo_renamed"
-	updatedCollection.Schema.RemoveField(updatedCollection.Schema.GetFieldByName("active").Id)
-	updatedCollection.Schema.AddField(
-		&schema.SchemaField{
-			Name: "new_field",
-			Type: schema.FieldTypeEmail,
-		},
-	)
-	updatedCollection.Schema.AddField(
-		&schema.SchemaField{
-			Id:   updatedCollection.Schema.GetFieldByName("title").Id,
-			Name: "title_renamed",
-			Type: schema.FieldTypeEmail,
-		},
-	)
 
-	scenarios := []struct {
-		newCollection     *models.Collection
-		oldCollection     *models.Collection
-		expectedTableName string
-		expectedColumns   []string
-	}{
-		// new base collection
-		{
-			&models.Collection{
-				Name: "new_table",
-				Schema: schema.NewSchema(
-					&schema.SchemaField{
-						Name: "test",
-						Type: schema.FieldTypeText,
-					},
-				),
-			},
-			nil,
-			"new_table",
-			[]string{"id", "created", "updated", "test"},
-		},
-		// new auth collection
-		{
-			&models.Collection{
-				Name: "new_table_auth",
-				Type: models.CollectionTypeAuth,
-				Schema: schema.NewSchema(
-					&schema.SchemaField{
-						Name: "test",
-						Type: schema.FieldTypeText,
-					},
-				),
-			},
-			nil,
-			"new_table_auth",
-			[]string{
-				"id", "created", "updated", "test",
-				"username", "email", "verified", "emailVisibility",
-				"tokenKey", "passwordHash", "lastResetSentAt", "lastVerificationSentAt",
-			},
-		},
-		// no changes
-		{
-			oldCollection,
-			oldCollection,
-			"demo3",
-			[]string{"id", "created", "updated", "title", "active"},
-		},
-		// renamed table, deleted column, renamed columnd and new column
-		{
-			updatedCollection,
-			oldCollection,
-			"demo_renamed",
-			[]string{"id", "created", "updated", "title_renamed", "new_field"},
-		},
+	// check if the main record was deleted
+	_, err := app.Dao().FindRecordById(mainRecord.Collection().Id, mainRecord.Id)
+	if err == nil {
+		t.Fatal("The main record wasn't deleted")
 	}
 
-	for i, scenario := range scenarios {
-		err := app.Dao().SyncRecordTableSchema(scenario.newCollection, scenario.oldCollection)
-		if err != nil {
-			t.Errorf("(%d) %v", i, err)
-			continue
+	// check if the c2 rel fields were updated
+	c2Records, err := app.Dao().FindRecordsByExpr("c2", nil)
+	if err != nil || len(c2Records) == 0 {
+		t.Fatalf("Failed to fetch c2 records: %v", err)
+	}
+	for _, r := range c2Records {
+		ids := r.GetStringSlice("rel")
+		if len(ids) != 1 || ids[0] != "b" {
+			t.Fatalf("Expected only 'b' rel id, got %v", ids)
 		}
+	}
 
-		if !app.Dao().HasTable(scenario.newCollection.Name) {
-			t.Errorf("(%d) Expected table %s to exist", i, scenario.newCollection.Name)
-		}
-
-		cols, _ := app.Dao().GetTableColumns(scenario.newCollection.Name)
-		if len(cols) != len(scenario.expectedColumns) {
-			t.Errorf("(%d) Expected columns %v, got %v", i, scenario.expectedColumns, cols)
-		}
-
-		for _, c := range cols {
-			if !list.ExistInSlice(c, scenario.expectedColumns) {
-				t.Errorf("(%d) Couldn't find column %s in %v", i, c, scenario.expectedColumns)
-			}
-		}
+	// check if all c3 relations were deleted
+	c3Records, err := app.Dao().FindRecordsByExpr("c3", nil)
+	if err != nil {
+		t.Fatalf("Failed to fetch c3 records: %v", err)
+	}
+	if total := len(c3Records); total != 0 {
+		t.Fatalf("Expected c3 records to be deleted, found %d", total)
 	}
 }
+
+func createMockBatchProcessingData(dao *daos.Dao) error {
+	// create mock collection without relation
+	c1 := &models.Collection{}
+	c1.Id = "c1"
+	c1.Name = c1.Id
+	c1.Schema = schema.NewSchema(
+		&schema.SchemaField{
+			Name: "text",
+			Type: schema.FieldTypeText,
+		},
+	)
+	if err := dao.SaveCollection(c1); err != nil {
+		return err
+	}
+
+	// create mock collection with a multi-rel field
+	c2 := &models.Collection{}
+	c2.Id = "c2"
+	c2.Name = c2.Id
+	c2.Schema = schema.NewSchema(
+		&schema.SchemaField{
+			Name: "rel",
+			Type: schema.FieldTypeRelation,
+			Options: &schema.RelationOptions{
+				MaxSelect:     types.Pointer(10),
+				CollectionId:  "c1",
+				CascadeDelete: false, // should unset all rel fields
+			},
+		},
+	)
+	if err := dao.SaveCollection(c2); err != nil {
+		return err
+	}
+
+	// create mock collection with a single-rel field
+	c3 := &models.Collection{}
+	c3.Id = "c3"
+	c3.Name = c3.Id
+	c3.Schema = schema.NewSchema(
+		&schema.SchemaField{
+			Name: "rel",
+			Type: schema.FieldTypeRelation,
+			Options: &schema.RelationOptions{
+				MaxSelect:     types.Pointer(1),
+				CollectionId:  "c1",
+				CascadeDelete: true, // should delete all c3 records
+			},
+		},
+	)
+	if err := dao.SaveCollection(c3); err != nil {
+		return err
+	}
+
+	// insert mock records
+	c1RecordA := models.NewRecord(c1)
+	c1RecordA.Id = "a"
+	if err := dao.Save(c1RecordA); err != nil {
+		return err
+	}
+	c1RecordB := models.NewRecord(c1)
+	c1RecordB.Id = "b"
+	if err := dao.Save(c1RecordB); err != nil {
+		return err
+	}
+	for i := 0; i < 2400; i++ {
+		c2Record := models.NewRecord(c2)
+		c2Record.Set("rel", []string{c1RecordA.Id, c1RecordB.Id})
+		if err := dao.Save(c2Record); err != nil {
+			return err
+		}
+
+		c3Record := models.NewRecord(c3)
+		c3Record.Set("rel", c1RecordA.Id)
+		if err := dao.Save(c3Record); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}

forms/collection_upsert.go

@@ -215,7 +215,7 @@ func (form *CollectionUpsert) ensureExistingRelationCollectionId(value any) erro
 			continue
 		}
 
-		if _, err := form.dao.FindCollectionByNameOrId(options.CollectionId); err != nil {
+		if err := form.dao.FindById(&models.Collection{}, options.CollectionId); err != nil {
 			return validation.Errors{fmt.Sprint(i): validation.NewError(
 				"validation_field_invalid_relation",
 				"The relation collection doesn't exist.",

forms/collections_import_test.go

@@ -52,7 +52,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				"collections": []
 			}`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 			expectEvents:           nil,
 		},
 		{
@@ -82,7 +82,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 			expectEvents: map[string]int{
 				"OnModelBeforeCreate": 2,
 			},
@@ -101,7 +101,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 			expectEvents: map[string]int{
 				"OnModelBeforeCreate": 2,
 			},
@@ -137,7 +137,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            false,
-			expectCollectionsCount: 10,
+			expectCollectionsCount: 11,
 			expectEvents: map[string]int{
 				"OnModelBeforeCreate": 3,
 				"OnModelAfterCreate":  3,
@@ -160,7 +160,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 			expectEvents: map[string]int{
 				"OnModelBeforeCreate": 1,
 			},
@@ -202,7 +202,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            true,
-			expectCollectionsCount: 7,
+			expectCollectionsCount: 8,
 			expectEvents: map[string]int{
 				"OnModelBeforeDelete": 5,
 			},
@@ -253,7 +253,7 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				]
 			}`,
 			expectError:            false,
-			expectCollectionsCount: 9,
+			expectCollectionsCount: 10,
 			expectEvents: map[string]int{
 				"OnModelBeforeUpdate": 1,
 				"OnModelAfterUpdate":  1,
@@ -341,8 +341,8 @@ func TestCollectionsImportSubmit(t *testing.T) {
 				"OnModelAfterUpdate":  2,
 				"OnModelBeforeCreate": 1,
 				"OnModelAfterCreate":  1,
-				"OnModelBeforeDelete": 5,
-				"OnModelAfterDelete":  5,
+				"OnModelBeforeDelete": 6,
+				"OnModelAfterDelete":  6,
 			},
 		},
 	}

forms/record_upsert.go

@@ -7,7 +7,6 @@ import (
 	"log"
 	"net/http"
 	"regexp"
-	"strconv"
 	"strings"
 
 	validation "github.com/go-ozzo/ozzo-validation/v4"
@@ -18,6 +17,7 @@ import (
 	"github.com/pocketbase/pocketbase/forms/validators"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/models/schema"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/rest"
 	"github.com/pocketbase/pocketbase/tools/security"
@@ -34,7 +34,7 @@ type RecordUpsert struct {
 	manageAccess bool
 	record       *models.Record
 
-	filesToUpload map[string][]*rest.UploadedFile
+	filesToUpload map[string][]*filesystem.File
 	filesToDelete []string // names list
 
 	// base model fields
@@ -51,7 +51,7 @@ type RecordUpsert struct {
 	OldPassword     string `json:"oldPassword"`
 	// ---
 
-	Data map[string]any `json:"data"`
+	data map[string]any
 }
 
 // NewRecordUpsert creates a new [RecordUpsert] form with initializer
@@ -66,7 +66,7 @@ func NewRecordUpsert(app core.App, record *models.Record) *RecordUpsert {
 		dao:           app.Dao(),
 		record:        record,
 		filesToDelete: []string{},
-		filesToUpload: map[string][]*rest.UploadedFile{},
+		filesToUpload: map[string][]*filesystem.File{},
 	}
 
 	form.loadFormDefaults()
@@ -74,6 +74,11 @@ func NewRecordUpsert(app core.App, record *models.Record) *RecordUpsert {
 	return form
 }
 
+// Data returns the loaded form's data.
+func (form *RecordUpsert) Data() map[string]any {
+	return form.data
+}
+
 // SetFullManageAccess sets the manageAccess bool flag of the current
 // form to enable/disable directly changing some system record fields
 // (often used with auth collection records).
@@ -96,9 +101,9 @@ func (form *RecordUpsert) loadFormDefaults() {
 		form.Verified = form.record.Verified()
 	}
 
-	form.Data = map[string]any{}
+	form.data = map[string]any{}
 	for _, field := range form.record.Collection().Schema.Fields() {
-		form.Data[field.Name] = form.record.Get(field.Name)
+		form.data[field.Name] = form.record.Get(field.Name)
 	}
 }
 
@@ -112,49 +117,56 @@ func (form *RecordUpsert) getContentType(r *http.Request) string {
 	return t
 }
 
-func (form *RecordUpsert) extractRequestData(r *http.Request, keyPrefix string) (map[string]any, error) {
+func (form *RecordUpsert) extractRequestData(
+	r *http.Request,
+	keyPrefix string,
+) (map[string]any, map[string][]*filesystem.File, error) {
 	switch form.getContentType(r) {
 	case "application/json":
 		return form.extractJsonData(r, keyPrefix)
 	case "multipart/form-data":
 		return form.extractMultipartFormData(r, keyPrefix)
 	default:
-		return nil, errors.New("Unsupported request Content-Type.")
+		return nil, nil, errors.New("unsupported request content-type")
 	}
 }
 
-func (form *RecordUpsert) extractJsonData(r *http.Request, keyPrefix string) (map[string]any, error) {
-	result := map[string]any{}
+func (form *RecordUpsert) extractJsonData(
+	r *http.Request,
+	keyPrefix string,
+) (map[string]any, map[string][]*filesystem.File, error) {
+	data := map[string]any{}
 
-	err := rest.CopyJsonBody(r, &result)
+	err := rest.CopyJsonBody(r, &data)
 
 	if keyPrefix != "" {
 		parts := strings.Split(keyPrefix, ".")
 		for _, part := range parts {
-			if result[part] == nil {
+			if data[part] == nil {
 				break
 			}
-			if v, ok := result[part].(map[string]any); ok {
-				result = v
+			if v, ok := data[part].(map[string]any); ok {
+				data = v
 			}
 		}
 	}
 
-	return result, err
+	return data, nil, err
 }
 
-func (form *RecordUpsert) extractMultipartFormData(r *http.Request, keyPrefix string) (map[string]any, error) {
-	result := map[string]any{}
-
+func (form *RecordUpsert) extractMultipartFormData(
+	r *http.Request,
+	keyPrefix string,
+) (map[string]any, map[string][]*filesystem.File, error) {
 	// parse form data (if not already)
 	if err := r.ParseMultipartForm(rest.DefaultMaxMemory); err != nil {
-		return result, err
+		return nil, nil, err
 	}
 
+	data := map[string]any{}
+	filesToUpload := map[string][]*filesystem.File{}
 	arrayValueSupportTypes := schema.ArraybleFieldTypes()
 
-	form.filesToUpload = map[string][]*rest.UploadedFile{}
-
 	for fullKey, values := range r.PostForm {
 		key := fullKey
 		if keyPrefix != "" {
@@ -162,15 +174,15 @@ func (form *RecordUpsert) extractMultipartFormData(r *http.Request, keyPrefix st
 		}
 
 		if len(values) == 0 {
-			result[key] = nil
+			data[key] = nil
 			continue
 		}
 
 		field := form.record.Collection().Schema.GetFieldByName(key)
 		if field != nil && list.ExistInSlice(field.Type, arrayValueSupportTypes) {
-			result[key] = values
+			data[key] = values
 		} else {
-			result[key] = values[0]
+			data[key] = values[0]
 		}
 	}
 
@@ -196,76 +208,155 @@ func (form *RecordUpsert) extractMultipartFormData(r *http.Request, keyPrefix st
 			continue
 		}
 
-		options, ok := field.Options.(*schema.FileOptions)
-		if !ok {
-			continue
-		}
-
-		if form.filesToUpload[key] == nil {
-			form.filesToUpload[key] = []*rest.UploadedFile{}
-		}
-
-		if options.MaxSelect == 1 {
-			form.filesToUpload[key] = append(form.filesToUpload[key], files[0])
-		} else if options.MaxSelect > 1 {
-			form.filesToUpload[key] = append(form.filesToUpload[key], files...)
-		}
+		filesToUpload[key] = append(filesToUpload[key], files...)
 	}
 
-	return result, nil
-}
-
-func (form *RecordUpsert) normalizeData() error {
-	for _, field := range form.record.Collection().Schema.Fields() {
-		if v, ok := form.Data[field.Name]; ok {
-			form.Data[field.Name] = field.PrepareValue(v)
-		}
-	}
-	return nil
+	return data, filesToUpload, nil
 }
 
 // LoadRequest extracts the json or multipart/form-data request data
 // and lods it into the form.
 //
 // File upload is supported only via multipart/form-data.
-//
-// To DELETE previously uploaded file(s) you can suffix the field name
-// with the file index or filename (eg. `myfile.0`) and set it to null or empty string.
-// For single file upload fields, you can skip the index and directly
-// reset the field using its field name (eg. `myfile = null`).
 func (form *RecordUpsert) LoadRequest(r *http.Request, keyPrefix string) error {
-	requestData, err := form.extractRequestData(r, keyPrefix)
+	requestData, uploadedFiles, err := form.extractRequestData(r, keyPrefix)
 	if err != nil {
 		return err
 	}
 
-	return form.LoadData(requestData)
+	if err := form.LoadData(requestData); err != nil {
+		return err
+	}
+
+	for key, files := range uploadedFiles {
+		form.AddFiles(key, files...)
+	}
+
+	return nil
 }
 
-// LoadData loads and normalizes the provided data into the form.
+// AddFiles adds the provided file(s) to the specified file field.
 //
-// To DELETE previously uploaded file(s) you can suffix the field name
-// with the file index or filename (eg. `myfile.0`) and set it to null or empty string.
-// For single file upload fields, you can skip the index and directly
-// reset the field using its field name (eg. `myfile = null`).
+// If the file field is a SINGLE-value file field (aka. "Max Select = 1"),
+// then the newly added file will REPLACE the existing one.
+// In this case if you pass more than 1 files only the first one will be assigned.
+//
+// If the file field is a MULTI-value file field (aka. "Max Select > 1"),
+// then the newly added file(s) will be APPENDED to the existing one(s).
+//
+// Example
+//
+// 	f1, _ := filesystem.NewFileFromPath("/path/to/file1.txt")
+// 	f2, _ := filesystem.NewFileFromPath("/path/to/file2.txt")
+// 	form.AddFiles("documents", f1, f2)
+func (form *RecordUpsert) AddFiles(key string, files ...*filesystem.File) error {
+	field := form.record.Collection().Schema.GetFieldByName(key)
+	if field == nil || field.Type != schema.FieldTypeFile {
+		return errors.New("invalid field key")
+	}
+
+	options, ok := field.Options.(*schema.FileOptions)
+	if !ok {
+		return errors.New("failed to initilize field options")
+	}
+
+	if len(files) == 0 {
+		return nil // nothing to upload
+	}
+
+	if form.filesToUpload == nil {
+		form.filesToUpload = map[string][]*filesystem.File{}
+	}
+
+	oldNames := list.ToUniqueStringSlice(form.data[key])
+
+	if options.MaxSelect == 1 {
+		// mark previous file(s) for deletion before replacing
+		if len(oldNames) > 0 {
+			form.filesToDelete = list.ToUniqueStringSlice(append(form.filesToDelete, oldNames...))
+		}
+
+		// replace
+		form.filesToUpload[key] = []*filesystem.File{files[0]}
+		form.data[key] = field.PrepareValue(files[0].Name)
+	} else {
+		// append
+		form.filesToUpload[key] = append(form.filesToUpload[key], files...)
+		for _, f := range files {
+			oldNames = append(oldNames, f.Name)
+		}
+		form.data[key] = field.PrepareValue(oldNames)
+	}
+
+	return nil
+}
+
+// RemoveFiles removes a single or multiple file from the specified file field.
+//
+// NB! If filesToDelete is not set it will remove all existing files
+// assigned to the file field (including those assigned with AddFiles)!
+//
+// Example
+//
+//  // mark only only 2 files for removal
+// 	form.AddFiles("documents", "file1_aw4bdrvws6.txt", "file2_xwbs36bafv.txt")
+//
+// 	// mark all "documents" files for removal
+// 	form.AddFiles("documents")
+func (form *RecordUpsert) RemoveFiles(key string, toDelete ...string) error {
+	field := form.record.Collection().Schema.GetFieldByName(key)
+	if field == nil || field.Type != schema.FieldTypeFile {
+		return errors.New("invalid field key")
+	}
+
+	existing := list.ToUniqueStringSlice(form.data[key])
+
+	// mark all files for deletion
+	if len(toDelete) == 0 {
+		toDelete = make([]string, len(existing))
+		copy(toDelete, existing)
+	}
+
+	// check for existing files
+	for i := len(existing) - 1; i >= 0; i-- {
+		if list.ExistInSlice(existing[i], toDelete) {
+			form.filesToDelete = append(form.filesToDelete, existing[i])
+			existing = append(existing[:i], existing[i+1:]...)
+		}
+	}
+
+	// check for newly uploaded files
+	for i := len(form.filesToUpload[key]) - 1; i >= 0; i-- {
+		f := form.filesToUpload[key][i]
+		if list.ExistInSlice(f.Name, toDelete) {
+			form.filesToUpload[key] = append(form.filesToUpload[key][:i], form.filesToUpload[key][i+1:]...)
+		}
+	}
+
+	form.data[key] = field.PrepareValue(existing)
+
+	return nil
+}
+
+// LoadData loads and normalizes the provided regular record data fields into the form.
 func (form *RecordUpsert) LoadData(requestData map[string]any) error {
 	// load base system fields
-	if v, ok := requestData["id"]; ok {
+	if v, ok := requestData[schema.FieldNameId]; ok {
 		form.Id = cast.ToString(v)
 	}
 
 	// load auth system fields
 	if form.record.Collection().IsAuth() {
-		if v, ok := requestData["username"]; ok {
+		if v, ok := requestData[schema.FieldNameUsername]; ok {
 			form.Username = cast.ToString(v)
 		}
-		if v, ok := requestData["email"]; ok {
+		if v, ok := requestData[schema.FieldNameEmail]; ok {
 			form.Email = cast.ToString(v)
 		}
-		if v, ok := requestData["emailVisibility"]; ok {
+		if v, ok := requestData[schema.FieldNameEmailVisibility]; ok {
 			form.EmailVisibility = cast.ToBool(v)
 		}
-		if v, ok := requestData["verified"]; ok {
+		if v, ok := requestData[schema.FieldNameVerified]; ok {
 			form.Verified = cast.ToBool(v)
 		}
 		if v, ok := requestData["password"]; ok {
@@ -279,8 +370,16 @@ func (form *RecordUpsert) LoadData(requestData map[string]any) error {
 		}
 	}
 
-	// extend the record schema data with the request data
-	extendedData := form.record.SchemaData()
+	// replace modifiers (if any)
+	requestData = form.record.ReplaceModifers(requestData)
+
+	// create a shallow copy of form.data
+	var extendedData = make(map[string]any, len(form.data))
+	for k, v := range form.data {
+		extendedData[k] = v
+	}
+
+	// extend form.data with the request data
 	rawData, err := json.Marshal(requestData)
 	if err != nil {
 		return err
@@ -291,89 +390,59 @@ func (form *RecordUpsert) LoadData(requestData map[string]any) error {
 
 	for _, field := range form.record.Collection().Schema.Fields() {
 		key := field.Name
-		value := extendedData[key]
-		value = field.PrepareValue(value)
+		value := field.PrepareValue(extendedData[key])
 
 		if field.Type != schema.FieldTypeFile {
-			form.Data[key] = value
+			form.data[key] = value
 			continue
 		}
 
-		options, _ := field.Options.(*schema.FileOptions)
-		oldNames := list.ToUniqueStringSlice(form.Data[key])
-
 		// -----------------------------------------------------------
 		// Delete previously uploaded file(s)
 		// -----------------------------------------------------------
 
+		oldNames := form.record.GetStringSlice(key)
+		submittedNames := list.ToUniqueStringSlice(value)
+
+		// ensure that all submitted names are existing to prevent accidental files deletions
+		if len(submittedNames) > len(oldNames) || len(list.SubtractSlice(submittedNames, oldNames)) != 0 {
+			return validation.Errors{
+				key: validation.NewError(
+					"validation_unknown_filenames",
+					"The field contains unknown filenames.",
+				),
+			}
+		}
+
 		// if empty value was set, mark all previously uploaded files for deletion
-		if len(list.ToUniqueStringSlice(value)) == 0 && len(oldNames) > 0 {
-			form.filesToDelete = append(form.filesToDelete, oldNames...)
-			form.Data[key] = []string{}
+		// otherwise check for "deleted" (aka. unsubmitted) file names
+		if len(submittedNames) == 0 && len(oldNames) > 0 {
+			form.RemoveFiles(key)
 		} else if len(oldNames) > 0 {
-			indexesToDelete := make([]int, 0, len(extendedData))
 
-			// search for individual file name to delete (eg. "file.test.png = null")
-			for i, name := range oldNames {
-				if v, ok := extendedData[key+"."+name]; ok && cast.ToString(v) == "" {
-					indexesToDelete = append(indexesToDelete, i)
-				}
-			}
+			toDelete := []string{}
 
-			// search for individual file index to delete (eg. "file.0 = null")
-			keyExp, _ := regexp.Compile(`^` + regexp.QuoteMeta(key) + `\.\d+$`)
-			for indexedKey := range extendedData {
-				if keyExp.MatchString(indexedKey) && cast.ToString(extendedData[indexedKey]) == "" {
-					index, indexErr := strconv.Atoi(indexedKey[len(key)+1:])
-					if indexErr != nil || index >= len(oldNames) {
-						continue
-					}
-					indexesToDelete = append(indexesToDelete, index)
-				}
-			}
-
-			// slice to fill only with the non-deleted indexes
-			nonDeleted := make([]string, 0, len(oldNames))
-			for i, name := range oldNames {
-				// not marked for deletion
-				if !list.ExistInSlice(i, indexesToDelete) {
-					nonDeleted = append(nonDeleted, name)
+			for _, name := range oldNames {
+				// submitted as a modifier or a new array
+				if !list.ExistInSlice(name, submittedNames) {
+					toDelete = append(toDelete, name)
 					continue
 				}
-
-				// store the id to actually delete the file later
-				form.filesToDelete = append(form.filesToDelete, name)
 			}
-			form.Data[key] = nonDeleted
+
+			if len(toDelete) > 0 {
+				form.RemoveFiles(key, toDelete...)
+			}
 		}
 
-		// -----------------------------------------------------------
-		// Check for new uploaded file
-		// -----------------------------------------------------------
-
-		if len(form.filesToUpload[key]) == 0 {
-			continue
-		}
-
-		// refresh oldNames list
-		oldNames = list.ToUniqueStringSlice(form.Data[key])
-
-		if options.MaxSelect == 1 {
-			// delete previous file(s) before replacing
-			if len(oldNames) > 0 {
-				form.filesToDelete = list.ToUniqueStringSlice(append(form.filesToDelete, oldNames...))
-			}
-			form.Data[key] = form.filesToUpload[key][0].Name()
-		} else if options.MaxSelect > 1 {
-			// append the id of each uploaded file instance
-			for _, file := range form.filesToUpload[key] {
-				oldNames = append(oldNames, file.Name())
-			}
-			form.Data[key] = oldNames
+		// allow file key reasignments for file names sorting
+		// (only if all submitted values already exists)
+		if len(submittedNames) > 0 && len(list.SubtractSlice(submittedNames, oldNames)) == 0 {
+			form.data[key] = submittedNames
 		}
 	}
 
-	return form.normalizeData()
+	return nil
 }
 
 // Validate makes the form validatable by implementing [validation.Validatable] interface.
@@ -429,13 +498,16 @@ func (form *RecordUpsert) Validate() error {
 			),
 			validation.Field(
 				&form.Password,
-				validation.When(form.record.IsNew(), validation.Required),
+				validation.When(
+					(form.record.IsNew() || form.PasswordConfirm != "" || form.OldPassword != ""),
+					validation.Required,
+				),
 				validation.Length(form.record.Collection().AuthOptions().MinPasswordLength, 72),
 			),
 			validation.Field(
 				&form.PasswordConfirm,
 				validation.When(
-					(form.record.IsNew() || form.Password != ""),
+					(form.record.IsNew() || form.Password != "" || form.OldPassword != ""),
 					validation.Required,
 				),
 				validation.By(validators.Compare(form.Password)),
@@ -446,7 +518,7 @@ func (form *RecordUpsert) Validate() error {
 				// - form.manageAccess is not set
 				// - changing the existing password
 				validation.When(
-					!form.record.IsNew() && !form.manageAccess && form.Password != "",
+					!form.record.IsNew() && !form.manageAccess && (form.Password != "" || form.PasswordConfirm != ""),
 					validation.Required,
 					validation.By(form.checkOldPassword),
 				),
@@ -463,7 +535,7 @@ func (form *RecordUpsert) Validate() error {
 		form.dao,
 		form.record,
 		form.filesToUpload,
-	).Validate(form.Data)
+	).Validate(form.data)
 }
 
 func (form *RecordUpsert) checkUniqueUsername(value any) error {
@@ -583,7 +655,7 @@ func (form *RecordUpsert) ValidateAndFill() error {
 			}
 		}
 
-		if form.Password != "" {
+		if form.Password != "" && form.Password == form.PasswordConfirm {
 			if err := form.record.SetPassword(form.Password); err != nil {
 				return err
 			}
@@ -591,7 +663,7 @@ func (form *RecordUpsert) ValidateAndFill() error {
 	}
 
 	// bulk load the remaining form data
-	form.record.Load(form.Data)
+	form.record.Load(form.data)
 
 	return nil
 }
@@ -607,20 +679,23 @@ func (form *RecordUpsert) DrySubmit(callback func(txDao *daos.Dao) error) error
 		return err
 	}
 
-	// use the default app.Dao to prevent changing the transaction form.Dao
-	// and causing "transaction has already been committed or rolled back" error
-	return form.app.Dao().RunInTransaction(func(txDao *daos.Dao) error {
+	var dryDao *daos.Dao
+	if form.dao.ConcurrentDB() == form.dao.NonconcurrentDB() {
+		// it is already in a transaction and therefore use the app concurrent db pool
+		// to prevent "transaction has already been committed or rolled back" error
+		dryDao = daos.New(form.app.Dao().ConcurrentDB())
+	} else {
+		// otherwise use the form noncurrent dao db pool
+		dryDao = daos.New(form.dao.NonconcurrentDB())
+	}
+
+	return dryDao.RunInTransaction(func(txDao *daos.Dao) error {
 		tx, ok := txDao.DB().(*dbx.Tx)
 		if !ok {
 			return errors.New("failed to get transaction db")
 		}
 		defer tx.Rollback()
 
-		txDao.BeforeCreateFunc = nil
-		txDao.AfterCreateFunc = nil
-		txDao.BeforeUpdateFunc = nil
-		txDao.AfterUpdateFunc = nil
-
 		if err := txDao.SaveRecord(form.record); err != nil {
 			return err
 		}
@@ -685,7 +760,7 @@ func (form *RecordUpsert) getFilesToUploadNames() []string {
 
 	for fieldKey := range form.filesToUpload {
 		for _, file := range form.filesToUpload[fieldKey] {
-			names = append(names, file.Name())
+			names = append(names, file.Name)
 		}
 	}
 
@@ -712,8 +787,8 @@ func (form *RecordUpsert) processFilesToUpload() error {
 
 	for fieldKey := range form.filesToUpload {
 		for i, file := range form.filesToUpload[fieldKey] {
-			path := form.record.BaseFilesPath() + "/" + file.Name()
-			if err := fs.UploadMultipart(file.Header(), path); err == nil {
+			path := form.record.BaseFilesPath() + "/" + file.Name
+			if err := fs.UploadFile(file, path); err == nil {
 				// keep track of the already uploaded file
 				uploaded = append(uploaded, path)
 			} else {

forms/record_upsert_test.go

@@ -6,6 +6,7 @@ import (
 	"errors"
 	"net/http"
 	"net/http/httptest"
+	"os"
 	"path/filepath"
 	"strings"
 	"testing"
@@ -16,6 +17,7 @@ import (
 	"github.com/pocketbase/pocketbase/forms"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
 )
 
@@ -44,9 +46,9 @@ func TestNewRecordUpsert(t *testing.T) {
 
 	form := forms.NewRecordUpsert(app, record)
 
-	val := form.Data["title"]
+	val := form.Data()["title"]
 	if val != "test_value" {
-		t.Errorf("Expected record data to be loaded, got %v", form.Data)
+		t.Errorf("Expected record data to be loaded, got %v", form.Data())
 	}
 }
 
@@ -87,9 +89,10 @@ func TestRecordUpsertLoadRequestJson(t *testing.T) {
 				"unknown": "test456",
 				// file fields unset/delete
 				"file_one":                     nil,
-				"file_many.0":                  "",         // delete by index
-				"file_many.1":                  "test.png", // should be ignored
-				"file_many.300_WlbFWSGmW9.png": nil,        // delete by filename
+				"file_many.0":                  "",                                                     // delete by index
+				"file_many-":                   []string{"test_MaWC6mWyrP.txt", "test_tC1Yc87DfC.txt"}, // multiple delete with modifier
+				"file_many.300_WlbFWSGmW9.png": nil,                                                    // delete by filename
+				"file_many.2":                  "test.png",                                             // should be ignored
 			},
 		},
 	}
@@ -107,15 +110,15 @@ func TestRecordUpsertLoadRequestJson(t *testing.T) {
 		t.Fatalf("Expect id field to be %q, got %q", "test_id", form.Id)
 	}
 
-	if v, ok := form.Data["text"]; !ok || v != "test123" {
+	if v, ok := form.Data()["text"]; !ok || v != "test123" {
 		t.Fatalf("Expect title field to be %q, got %q", "test123", v)
 	}
 
-	if v, ok := form.Data["unknown"]; ok {
+	if v, ok := form.Data()["unknown"]; ok {
 		t.Fatalf("Didn't expect unknown field to be set, got %v", v)
 	}
 
-	fileOne, ok := form.Data["file_one"]
+	fileOne, ok := form.Data()["file_one"]
 	if !ok {
 		t.Fatal("Expect file_one field to be set")
 	}
@@ -123,7 +126,7 @@ func TestRecordUpsertLoadRequestJson(t *testing.T) {
 		t.Fatalf("Expect file_one field to be empty string, got %v", fileOne)
 	}
 
-	fileMany, ok := form.Data["file_many"]
+	fileMany, ok := form.Data()["file_many"]
 	if !ok || fileMany == nil {
 		t.Fatal("Expect file_many field to be set")
 	}
@@ -147,11 +150,12 @@ func TestRecordUpsertLoadRequestMultipart(t *testing.T) {
 		"a.b.text":    "test123",
 		"a.b.unknown": "test456",
 		// file fields unset/delete
-		"a.b.file_one":                     "",
-		"a.b.file_many.0":                  "",
-		"a.b.file_many.300_WlbFWSGmW9.png": "test.png", // delete by name
-		"a.b.file_many.1":                  "test.png", // should be ignored
-	}, "file_many")
+		"a.b.file_one-":                    "test_d61b33QdDU.txt", // delete with modifier
+		"a.b.file_many.0":                  "",                    // delete by index
+		"a.b.file_many-":                   "test_tC1Yc87DfC.txt", // delete with modifier
+		"a.b.file_many.300_WlbFWSGmW9.png": "",                    // delete by filename
+		"a.b.file_many.2":                  "test.png",            // should be ignored
+	}, "a.b.file_many")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -168,15 +172,15 @@ func TestRecordUpsertLoadRequestMultipart(t *testing.T) {
 		t.Fatalf("Expect id field to be %q, got %q", "test_id", form.Id)
 	}
 
-	if v, ok := form.Data["text"]; !ok || v != "test123" {
+	if v, ok := form.Data()["text"]; !ok || v != "test123" {
 		t.Fatalf("Expect text field to be %q, got %q", "test123", v)
 	}
 
-	if v, ok := form.Data["unknown"]; ok {
+	if v, ok := form.Data()["unknown"]; ok {
 		t.Fatalf("Didn't expect unknown field to be set, got %v", v)
 	}
 
-	fileOne, ok := form.Data["file_one"]
+	fileOne, ok := form.Data()["file_one"]
 	if !ok {
 		t.Fatal("Expect file_one field to be set")
 	}
@@ -184,12 +188,12 @@ func TestRecordUpsertLoadRequestMultipart(t *testing.T) {
 		t.Fatalf("Expect file_one field to be empty string, got %v", fileOne)
 	}
 
-	fileMany, ok := form.Data["file_many"]
+	fileMany, ok := form.Data()["file_many"]
 	if !ok || fileMany == nil {
 		t.Fatal("Expect file_many field to be set")
 	}
 	manyfilesRemains := len(list.ToUniqueStringSlice(fileMany))
-	expectedRemains := 2 // -2 from 3 removed + 1 new upload
+	expectedRemains := 3 // 5 old; 3 deleted and 1 new uploaded
 	if manyfilesRemains != expectedRemains {
 		t.Fatalf("Expect file_many to be %d, got %d (%v)", expectedRemains, manyfilesRemains, fileMany)
 	}
@@ -214,11 +218,11 @@ func TestRecordUpsertLoadData(t *testing.T) {
 		t.Fatal(loadErr)
 	}
 
-	if v, ok := form.Data["title"]; !ok || v != "test_new" {
+	if v, ok := form.Data()["title"]; !ok || v != "test_new" {
 		t.Fatalf("Expect title field to be %v, got %v", "test_new", v)
 	}
 
-	if v, ok := form.Data["active"]; !ok || v != true {
+	if v, ok := form.Data()["active"]; !ok || v != true {
 		t.Fatalf("Expect active field to be %v, got %v", true, v)
 	}
 }
@@ -316,12 +320,10 @@ func TestRecordUpsertDrySubmitSuccess(t *testing.T) {
 	}
 
 	// ensure that the record changes weren't persisted
-	// ---
 	recordAfter, err := app.Dao().FindRecordById(collection.Id, recordBefore.Id)
 	if err != nil {
 		t.Fatal(err)
 	}
-
 	if recordAfter.GetString("title") == "dry_test" {
 		t.Fatalf("Expected record.title to be %v, got %v", recordAfter.GetString("title"), "dry_test")
 	}
@@ -335,6 +337,66 @@ func TestRecordUpsertDrySubmitSuccess(t *testing.T) {
 	}
 }
 
+func TestRecordUpsertDrySubmitWithNestedTx(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	collection, _ := app.Dao().FindCollectionByNameOrId("demo1")
+	recordBefore, err := app.Dao().FindRecordById(collection.Id, "84nmscqy84lsi1t")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	formData, mp, err := tests.MockMultipartData(map[string]string{
+		"title": "dry_test",
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	txErr := app.Dao().RunInTransaction(func(txDao *daos.Dao) error {
+		form := forms.NewRecordUpsert(app, recordBefore)
+		form.SetDao(txDao)
+		req := httptest.NewRequest(http.MethodGet, "/", formData)
+		req.Header.Set(echo.HeaderContentType, mp.FormDataContentType())
+		form.LoadRequest(req, "")
+
+		callbackCalls := 0
+
+		result := form.DrySubmit(func(innerTxDao *daos.Dao) error {
+			callbackCalls++
+			return nil
+		})
+		if result != nil {
+			t.Fatalf("Expected nil, got error %v", result)
+		}
+
+		// ensure callback was called
+		if callbackCalls != 1 {
+			t.Fatalf("Expected callbackCalls to be 1, got %d", callbackCalls)
+		}
+
+		// ensure that the original txDao can still be used after the DrySubmit rollback
+		if _, err := txDao.FindRecordById(collection.Id, recordBefore.Id); err != nil {
+			t.Fatalf("Expected the dry submit rollback to not affect the outer tx context, got %v", err)
+		}
+
+		// ensure that the record changes weren't persisted
+		recordAfter, err := app.Dao().FindRecordById(collection.Id, recordBefore.Id)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if recordAfter.GetString("title") == "dry_test" {
+			t.Fatalf("Expected record.title to be %v, got %v", recordBefore.GetString("title"), "dry_test")
+		}
+
+		return nil
+	})
+	if txErr != nil {
+		t.Fatalf("Nested transactions failure: %v", txErr)
+	}
+}
+
 func TestRecordUpsertSubmitFailure(t *testing.T) {
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
@@ -405,7 +467,7 @@ func TestRecordUpsertSubmitFailure(t *testing.T) {
 	if v := recordAfter.Get("email"); v == "invalid" {
 		t.Fatalf("Expected record.email not to change, got %v", v)
 	}
-	if v := recordAfter.GetStringSlice("file_many"); len(v) != 3 {
+	if v := recordAfter.GetStringSlice("file_many"); len(v) != 5 {
 		t.Fatalf("Expected record.file_many not to change, got %v", v)
 	}
 
@@ -477,8 +539,8 @@ func TestRecordUpsertSubmitSuccess(t *testing.T) {
 	}
 
 	fileMany := (recordAfter.GetStringSlice("file_many"))
-	if len(fileMany) != 4 { // 1 replace + 1 new
-		t.Fatalf("Expected 4 record.file_many, got %d (%v)", len(fileMany), fileMany)
+	if len(fileMany) != 6 { // 1 replace + 1 new
+		t.Fatalf("Expected 6 record.file_many, got %d (%v)", len(fileMany), fileMany)
 	}
 	for _, f := range fileMany {
 		if !hasRecordFile(app, recordAfter, f) {
@@ -498,7 +560,7 @@ func TestRecordUpsertSubmitInterceptors(t *testing.T) {
 	}
 
 	form := forms.NewRecordUpsert(app, record)
-	form.Data["title"] = "test_new"
+	form.Data()["title"] = "test_new"
 
 	testErr := errors.New("test_error")
 	interceptorRecordTitle := ""
@@ -533,7 +595,7 @@ func TestRecordUpsertSubmitInterceptors(t *testing.T) {
 		t.Fatalf("Expected interceptor2 to be called")
 	}
 
-	if interceptorRecordTitle != form.Data["title"].(string) {
+	if interceptorRecordTitle != form.Data()["title"].(string) {
 		t.Fatalf("Expected the form model to be filled before calling the interceptors")
 	}
 }
@@ -792,6 +854,24 @@ func TestRecordUpsertAuthRecord(t *testing.T) {
 		},
 
 		// password
+		{
+			"trigger the password validations if only oldPassword is set",
+			"4q1xlclmfloku33",
+			map[string]any{
+				"oldPassword": "1234567890",
+			},
+			false,
+			true,
+		},
+		{
+			"trigger the password validations if only passwordConfirm is set",
+			"4q1xlclmfloku33",
+			map[string]any{
+				"passwordConfirm": "1234567890",
+			},
+			false,
+			true,
+		},
 		{
 			"try to update password without managed access",
 			"4q1xlclmfloku33",
@@ -863,3 +943,75 @@ func TestRecordUpsertAuthRecord(t *testing.T) {
 		}
 	}
 }
+
+func TestRecordUpsertAddAndRemoveFiles(t *testing.T) {
+	app, _ := tests.NewTestApp()
+	defer app.Cleanup()
+
+	recordBefore, err := app.Dao().FindRecordById("demo1", "84nmscqy84lsi1t")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// create test temp files
+	tempDir := filepath.Join(app.DataDir(), "temp")
+	if err := os.MkdirAll(app.DataDir(), os.ModePerm); err != nil {
+		t.Fatal(err)
+	}
+	defer os.RemoveAll(tempDir)
+	tmpFile, _ := os.CreateTemp(os.TempDir(), "tmpfile1-*.txt")
+	tmpFile.Close()
+
+	form := forms.NewRecordUpsert(app, recordBefore)
+
+	f1, err := filesystem.NewFileFromPath(tmpFile.Name())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	f2, err := filesystem.NewFileFromPath(tmpFile.Name())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	f3, err := filesystem.NewFileFromPath(tmpFile.Name())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	form.AddFiles("file_one", f1) // should replace the existin file
+
+	form.AddFiles("file_many", f2, f3) // should append
+
+	form.RemoveFiles("file_many", "300_WlbFWSGmW9.png", "logo_vcfJJG5TAh.svg") // should remove
+
+	if err := form.Submit(); err != nil {
+		t.Fatalf("Failed to submit the RecordUpsert form, got %v", err)
+	}
+
+	recordAfter, err := app.Dao().FindRecordById("demo1", "84nmscqy84lsi1t")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// ensure files deletion
+	if hasRecordFile(app, recordAfter, "test_d61b33QdDU.txt") {
+		t.Fatalf("Expected the old file_one file to be deleted")
+	}
+	if hasRecordFile(app, recordAfter, "300_WlbFWSGmW9.png") {
+		t.Fatalf("Expected 300_WlbFWSGmW9.png to be deleted")
+	}
+	if hasRecordFile(app, recordAfter, "logo_vcfJJG5TAh.svg") {
+		t.Fatalf("Expected logo_vcfJJG5TAh.svg to be deleted")
+	}
+
+	fileOne := recordAfter.GetStringSlice("file_one")
+	if len(fileOne) == 0 {
+		t.Fatalf("Expected new file_one file to be uploaded")
+	}
+
+	fileMany := recordAfter.GetStringSlice("file_many")
+	if len(fileMany) != 5 {
+		t.Fatalf("Expected file_many to be 5, got %v", fileMany)
+	}
+}

forms/validators/file.go

@@ -6,7 +6,7 @@ import (
 
 	"github.com/gabriel-vasile/mimetype"
 	validation "github.com/go-ozzo/ozzo-validation/v4"
-	"github.com/pocketbase/pocketbase/tools/rest"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 )
 
 // UploadedFileSize checks whether the validated `rest.UploadedFile`
@@ -16,12 +16,12 @@ import (
 //	validation.Field(&form.File, validation.By(validators.UploadedFileSize(1000)))
 func UploadedFileSize(maxBytes int) validation.RuleFunc {
 	return func(value any) error {
-		v, _ := value.(*rest.UploadedFile)
+		v, _ := value.(*filesystem.File)
 		if v == nil {
 			return nil // nothing to validate
 		}
 
-		if int(v.Header().Size) > maxBytes {
+		if int(v.Size) > maxBytes {
 			return validation.NewError("validation_file_size_limit", fmt.Sprintf("Maximum allowed file size is %v bytes.", maxBytes))
 		}
 
@@ -37,7 +37,7 @@ func UploadedFileSize(maxBytes int) validation.RuleFunc {
 //	validation.Field(&form.File, validation.By(validators.UploadedFileMimeType(validMimeTypes)))
 func UploadedFileMimeType(validTypes []string) validation.RuleFunc {
 	return func(value any) error {
-		v, _ := value.(*rest.UploadedFile)
+		v, _ := value.(*filesystem.File)
 		if v == nil {
 			return nil // nothing to validate
 		}
@@ -46,7 +46,7 @@ func UploadedFileMimeType(validTypes []string) validation.RuleFunc {
 			return validation.NewError("validation_invalid_mime_type", "Unsupported file type.")
 		}
 
-		f, err := v.Header().Open()
+		f, err := v.Reader.Open()
 		if err != nil {
 			return validation.NewError("validation_invalid_mime_type", "Unsupported file type.")
 		}

forms/validators/file_test.go

@@ -7,6 +7,7 @@ import (
 
 	"github.com/pocketbase/pocketbase/forms/validators"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/rest"
 )
 
@@ -30,7 +31,7 @@ func TestUploadedFileSize(t *testing.T) {
 
 	scenarios := []struct {
 		maxBytes    int
-		file        *rest.UploadedFile
+		file        *filesystem.File
 		expectError bool
 	}{
 		{0, nil, false},
@@ -70,7 +71,7 @@ func TestUploadedFileMimeType(t *testing.T) {
 
 	scenarios := []struct {
 		types       []string
-		file        *rest.UploadedFile
+		file        *filesystem.File
 		expectError bool
 	}{
 		{nil, nil, false},

forms/validators/record_data.go

@@ -12,8 +12,8 @@ import (
 	"github.com/pocketbase/pocketbase/daos"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/models/schema"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/list"
-	"github.com/pocketbase/pocketbase/tools/rest"
 	"github.com/pocketbase/pocketbase/tools/types"
 )
 
@@ -28,7 +28,7 @@ var requiredErr = validation.NewError("validation_required", "Missing required v
 func NewRecordDataValidator(
 	dao *daos.Dao,
 	record *models.Record,
-	uploadedFiles map[string][]*rest.UploadedFile,
+	uploadedFiles map[string][]*filesystem.File,
 ) *RecordDataValidator {
 	return &RecordDataValidator{
 		dao:           dao,
@@ -42,7 +42,7 @@ func NewRecordDataValidator(
 type RecordDataValidator struct {
 	dao           *daos.Dao
 	record        *models.Record
-	uploadedFiles map[string][]*rest.UploadedFile
+	uploadedFiles map[string][]*filesystem.File
 }
 
 // Validate validates the provided `data` by checking it against
@@ -314,9 +314,9 @@ func (validator *RecordDataValidator) checkFileValue(field *schema.SchemaField,
 	}
 
 	// extract the uploaded files
-	files := make([]*rest.UploadedFile, 0, len(validator.uploadedFiles[field.Name]))
+	files := make([]*filesystem.File, 0, len(validator.uploadedFiles[field.Name]))
 	for _, file := range validator.uploadedFiles[field.Name] {
-		if list.ExistInSlice(file.Name(), names) {
+		if list.ExistInSlice(file.Name, names) {
 			files = append(files, file)
 		}
 	}

forms/validators/record_data_test.go

@@ -13,6 +13,7 @@ import (
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/models/schema"
 	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
 	"github.com/pocketbase/pocketbase/tools/rest"
 	"github.com/pocketbase/pocketbase/tools/types"
 )
@@ -20,7 +21,7 @@ import (
 type testDataFieldScenario struct {
 	name           string
 	data           map[string]any
-	files          map[string][]*rest.UploadedFile
+	files          map[string][]*filesystem.File
 	expectedErrors []string
 }
 
@@ -1086,10 +1087,10 @@ func TestRecordDataValidatorValidateFile(t *testing.T) {
 			"check MaxSelect constraint",
 			map[string]any{
 				"field1": "test1",
-				"field2": []string{"test1", testFiles[0].Name(), testFiles[3].Name()},
+				"field2": []string{"test1", testFiles[0].Name, testFiles[3].Name},
 				"field3": []string{"test1", "test2", "test3", "test4"},
 			},
-			map[string][]*rest.UploadedFile{
+			map[string][]*filesystem.File{
 				"field2": {testFiles[0], testFiles[3]},
 			},
 			[]string{"field2", "field3"},
@@ -1097,11 +1098,11 @@ func TestRecordDataValidatorValidateFile(t *testing.T) {
 		{
 			"check MaxSize constraint",
 			map[string]any{
-				"field1": testFiles[0].Name(),
-				"field2": []string{"test1", testFiles[0].Name()},
+				"field1": testFiles[0].Name,
+				"field2": []string{"test1", testFiles[0].Name},
 				"field3": []string{"test1", "test2", "test3"},
 			},
-			map[string][]*rest.UploadedFile{
+			map[string][]*filesystem.File{
 				"field1": {testFiles[0]},
 				"field2": {testFiles[0]},
 			},
@@ -1111,10 +1112,10 @@ func TestRecordDataValidatorValidateFile(t *testing.T) {
 			"check MimeTypes constraint",
 			map[string]any{
 				"field1": "test1",
-				"field2": []string{"test1", testFiles[0].Name()},
-				"field3": []string{testFiles[1].Name(), testFiles[2].Name()},
+				"field2": []string{"test1", testFiles[0].Name},
+				"field3": []string{testFiles[1].Name, testFiles[2].Name},
 			},
-			map[string][]*rest.UploadedFile{
+			map[string][]*filesystem.File{
 				"field2": {testFiles[0], testFiles[1], testFiles[2]},
 				"field3": {testFiles[1], testFiles[2]},
 			},
@@ -1134,10 +1135,10 @@ func TestRecordDataValidatorValidateFile(t *testing.T) {
 			"valid data - just new files",
 			map[string]any{
 				"field1": nil,
-				"field2": []string{testFiles[0].Name(), testFiles[1].Name()},
+				"field2": []string{testFiles[0].Name, testFiles[1].Name},
 				"field3": nil,
 			},
-			map[string][]*rest.UploadedFile{
+			map[string][]*filesystem.File{
 				"field2": {testFiles[0], testFiles[1]},
 			},
 			[]string{},
@@ -1146,10 +1147,10 @@ func TestRecordDataValidatorValidateFile(t *testing.T) {
 			"valid data - mixed existing and new files",
 			map[string]any{
 				"field1": "test1",
-				"field2": []string{"test1", testFiles[0].Name()},
+				"field2": []string{"test1", testFiles[0].Name},
 				"field3": "test1", // will be casted
 			},
-			map[string][]*rest.UploadedFile{
+			map[string][]*filesystem.File{
 				"field2": {testFiles[0], testFiles[1], testFiles[2]},
 			},
 			[]string{},

go.mod

@@ -4,14 +4,14 @@ go 1.18
 
 require (
 	github.com/AlecAivazis/survey/v2 v2.3.6
-	github.com/aws/aws-sdk-go v1.44.156
+	github.com/aws/aws-sdk-go v1.44.178
 	github.com/disintegration/imaging v1.6.2
 	github.com/domodwyer/mailyak/v3 v3.3.4
 	github.com/dop251/goja v0.0.0-20221118162653-d4bf6fde1b86
 	github.com/dop251/goja_nodejs v0.0.0-20221009164102-3aa5028e57f6
 	github.com/fatih/color v1.13.0
 	github.com/gabriel-vasile/mimetype v1.4.1
-	github.com/ganigeorgiev/fexpr v0.1.1
+	github.com/ganigeorgiev/fexpr v0.3.0
 	github.com/go-ozzo/ozzo-validation/v4 v4.3.0
 	github.com/golang-jwt/jwt/v4 v4.4.3
 	github.com/labstack/echo/v5 v5.0.0-20220201181537-ed2888cfa198
@@ -19,36 +19,37 @@ require (
 	github.com/pocketbase/dbx v1.8.0
 	github.com/spf13/cast v1.5.0
 	github.com/spf13/cobra v1.6.1
-	gocloud.dev v0.27.0
-	golang.org/x/crypto v0.4.0
-	golang.org/x/net v0.4.0
-	golang.org/x/oauth2 v0.3.0
+	gocloud.dev v0.28.0
+	golang.org/x/crypto v0.5.0
+	golang.org/x/net v0.5.0
+	golang.org/x/oauth2 v0.4.0
 	golang.org/x/sync v0.1.0
-	modernc.org/sqlite v1.20.0
+	modernc.org/sqlite v1.20.2
 )
 
 require (
 	github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
-	github.com/aws/aws-sdk-go-v2 v1.17.2 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.17.3 // indirect
 	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.18.4 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.13.4 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.20 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.43 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.26 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.20 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.3.27 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.17 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.18.8 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.13.8 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.21 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.47 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.27 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.21 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/ini v1.3.28 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.18 // indirect
 	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.11 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.21 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.20 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.20 // indirect
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.29.5 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.11.26 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.13.9 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.17.6 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.22 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.21 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.21 // indirect
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.30.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.12.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.18.0 // indirect
 	github.com/aws/smithy-go v1.13.5 // indirect
 	github.com/dlclark/regexp2 v1.7.0 // indirect
+	github.com/dustin/go-humanize v1.0.0 // indirect
 	github.com/go-sourcemap/sourcemap v2.1.3+incompatible // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
@@ -59,30 +60,30 @@ require (
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
 	github.com/mattn/go-colorable v0.1.13 // indirect
-	github.com/mattn/go-isatty v0.0.16 // indirect
+	github.com/mattn/go-isatty v0.0.17 // indirect
 	github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20220927061507-ef77025ab5aa // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/valyala/bytebufferpool v1.0.0 // indirect
 	github.com/valyala/fasttemplate v1.2.2 // indirect
 	go.opencensus.io v0.24.0 // indirect
-	golang.org/x/image v0.2.0 // indirect
+	golang.org/x/image v0.3.0 // indirect
 	golang.org/x/mod v0.7.0 // indirect
-	golang.org/x/sys v0.3.0 // indirect
-	golang.org/x/term v0.3.0 // indirect
-	golang.org/x/text v0.5.0 // indirect
+	golang.org/x/sys v0.4.0 // indirect
+	golang.org/x/term v0.4.0 // indirect
+	golang.org/x/text v0.6.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
-	golang.org/x/tools v0.4.0 // indirect
+	golang.org/x/tools v0.5.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
-	google.golang.org/api v0.104.0 // indirect
+	google.golang.org/api v0.106.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20221207170731-23e4bf6bdc37 // indirect
-	google.golang.org/grpc v1.51.0 // indirect
+	google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f // indirect
+	google.golang.org/grpc v1.52.0 // indirect
 	google.golang.org/protobuf v1.28.1 // indirect
 	lukechampine.com/uint128 v1.2.0 // indirect
 	modernc.org/cc/v3 v3.40.0 // indirect
 	modernc.org/ccgo/v3 v3.16.13 // indirect
-	modernc.org/libc v1.21.5 // indirect
+	modernc.org/libc v1.22.2 // indirect
 	modernc.org/mathutil v1.5.0 // indirect
 	modernc.org/memory v1.5.0 // indirect
 	modernc.org/opt v0.1.3 // indirect

golangci.yml

@@ -7,10 +7,8 @@ linters:
   disable-all: true
   enable:
     - asciicheck
-    - deadcode
     - depguard
     - exportloopref
-    - gocritic
     - gofmt
     - goimports
     - gomodguard
@@ -26,5 +24,4 @@ linters:
     - typecheck
     - unconvert
     - unused
-    - varcheck
     - whitespace

mails/admin.go

@@ -2,6 +2,7 @@ package mails
 
 import (
 	"fmt"
+	"log"
 	"net/mail"
 
 	"github.com/pocketbase/pocketbase/core"
@@ -72,7 +73,9 @@ func SendAdminPasswordReset(app core.App, admin *models.Admin) error {
 	})
 
 	if sendErr == nil {
-		app.OnMailerAfterAdminResetPasswordSend().Trigger(event)
+		if err := app.OnMailerAfterAdminResetPasswordSend().Trigger(event); err != nil && app.IsDebug() {
+			log.Println(err)
+		}
 	}
 
 	return sendErr

mails/record.go

@@ -2,6 +2,7 @@ package mails
 
 import (
 	"html/template"
+	"log"
 	"net/mail"
 
 	"github.com/pocketbase/pocketbase/core"
@@ -48,7 +49,9 @@ func SendRecordPasswordReset(app core.App, authRecord *models.Record) error {
 	})
 
 	if sendErr == nil {
-		app.OnMailerAfterRecordResetPasswordSend().Trigger(event)
+		if err := app.OnMailerAfterRecordResetPasswordSend().Trigger(event); err != nil && app.IsDebug() {
+			log.Println(err)
+		}
 	}
 
 	return sendErr
@@ -90,7 +93,9 @@ func SendRecordVerification(app core.App, authRecord *models.Record) error {
 	})
 
 	if sendErr == nil {
-		app.OnMailerAfterRecordVerificationSend().Trigger(event)
+		if err := app.OnMailerAfterRecordVerificationSend().Trigger(event); err != nil && app.IsDebug() {
+			log.Println(err)
+		}
 	}
 
 	return sendErr
@@ -135,7 +140,9 @@ func SendRecordChangeEmail(app core.App, record *models.Record, newEmail string)
 	})
 
 	if sendErr == nil {
-		app.OnMailerAfterRecordChangeEmailSend().Trigger(event)
+		if err := app.OnMailerAfterRecordChangeEmailSend().Trigger(event); err != nil && app.IsDebug() {
+			log.Println(err)
+		}
 	}
 
 	return sendErr

migrations/1640988000_init.go

@@ -36,7 +36,7 @@ func init() {
 	AppMigrations.Register(func(db dbx.Builder) error {
 		_, tablesErr := db.NewQuery(`
 			CREATE TABLE {{_admins}} (
-				[[id]]              TEXT PRIMARY KEY,
+				[[id]]              TEXT PRIMARY KEY NOT NULL,
 				[[avatar]]          INTEGER DEFAULT 0 NOT NULL,
 				[[email]]           TEXT UNIQUE NOT NULL,
 				[[tokenKey]]        TEXT UNIQUE NOT NULL,
@@ -47,7 +47,7 @@ func init() {
 			);
 
 			CREATE TABLE {{_collections}} (
-				[[id]]         TEXT PRIMARY KEY,
+				[[id]]         TEXT PRIMARY KEY NOT NULL,
 				[[system]]     BOOLEAN DEFAULT FALSE NOT NULL,
 				[[type]]       TEXT DEFAULT "base" NOT NULL,
 				[[name]]       TEXT UNIQUE NOT NULL,
@@ -63,7 +63,7 @@ func init() {
 			);
 
 			CREATE TABLE {{_params}} (
-				[[id]]      TEXT PRIMARY KEY,
+				[[id]]      TEXT PRIMARY KEY NOT NULL,
 				[[key]]     TEXT UNIQUE NOT NULL,
 				[[value]]   JSON DEFAULT NULL,
 				[[created]] TEXT DEFAULT "" NOT NULL,
@@ -71,7 +71,7 @@ func init() {
 			);
 
 			CREATE TABLE {{_externalAuths}} (
-				[[id]]           TEXT PRIMARY KEY,
+				[[id]]           TEXT PRIMARY KEY NOT NULL,
 				[[collectionId]] TEXT NOT NULL,
 				[[recordId]]     TEXT NOT NULL,
 				[[provider]]     TEXT NOT NULL,
@@ -133,6 +133,7 @@ func init() {
 						"image/png",
 						"image/svg+xml",
 						"image/gif",
+						"image/webp",
 					},
 				},
 			},

migrations/1673167670_multi_match_migrate.go

@@ -0,0 +1,215 @@
+package migrations
+
+import (
+	"regexp"
+	"strings"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/daos"
+	"github.com/pocketbase/pocketbase/models"
+	"github.com/pocketbase/pocketbase/models/schema"
+)
+
+// This migration replaces for backward compatibility the default operators
+// (=, !=, >, etc.) with their any/opt equivalent (?=, ?=, ?>, etc.)
+// in any muli-rel expression collection rule.
+func init() {
+	AppMigrations.Register(func(db dbx.Builder) error {
+		dao := daos.New(db)
+
+		exprRegex := regexp.MustCompile(`([\@\'\"\w\.]+)\s*(=|!=|~|!~|>|>=|<|<=)\s*([\@\'\"\w\.]+)`)
+
+		collections := []*models.Collection{}
+		if err := dao.CollectionQuery().All(&collections); err != nil {
+			return err
+		}
+
+		findCollection := func(nameOrId string) *models.Collection {
+			for _, c := range collections {
+				if c.Id == nameOrId || c.Name == nameOrId {
+					return c
+				}
+			}
+
+			return nil
+		}
+
+		var isMultiRelLiteral func(mainCollection *models.Collection, literal string) bool
+		isMultiRelLiteral = func(mainCollection *models.Collection, literal string) bool {
+			if strings.HasPrefix(literal, "@collection.") {
+				return true
+			}
+
+			if strings.HasPrefix(literal, `"`) ||
+				strings.HasPrefix(literal, `'`) ||
+				strings.HasPrefix(literal, "@request.method") ||
+				strings.HasPrefix(literal, "@request.data") ||
+				strings.HasPrefix(literal, "@request.query") {
+				return false
+			}
+
+			parts := strings.Split(literal, ".")
+			if len(parts) <= 1 {
+				return false
+			}
+
+			if strings.HasPrefix(literal, "@request.auth") && len(parts) >= 4 {
+				// check each auth collection
+				for _, c := range collections {
+					if c.IsAuth() && isMultiRelLiteral(c, strings.Join(parts[2:], ".")) {
+						return true
+					}
+				}
+
+				return false
+			}
+
+			activeCollection := mainCollection
+
+			for i, p := range parts {
+				f := activeCollection.Schema.GetFieldByName(p)
+				if f == nil || f.Type != schema.FieldTypeRelation {
+					return false // not a  relation field
+				}
+
+				// is multi-relation and not the last prop
+				opt, ok := f.Options.(*schema.RelationOptions)
+				if ok && (opt.MaxSelect == nil || *opt.MaxSelect != 1) && i != len(parts)-1 {
+					return true
+				}
+
+				activeCollection = findCollection(opt.CollectionId)
+				if activeCollection == nil {
+					return false
+				}
+			}
+
+			return false
+		}
+
+		// replace all multi-match operators to their any/opt equivalent, eg. "=" => "?="
+		migrateRule := func(collection *models.Collection, rule *string) (*string, error) {
+			if rule == nil || *rule == "" {
+				return rule, nil
+			}
+
+			newRule := *rule
+			parts := exprRegex.FindAllStringSubmatch(newRule, -1)
+
+			for _, p := range parts {
+				if isMultiRelLiteral(collection, p[1]) || isMultiRelLiteral(collection, p[3]) {
+					newRule = strings.ReplaceAll(newRule, p[0], p[1]+" ?"+p[2]+" "+p[3])
+				}
+			}
+
+			return &newRule, nil
+		}
+
+		var ruleErr error
+		for _, c := range collections {
+			c.ListRule, ruleErr = migrateRule(c, c.ListRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.ViewRule, ruleErr = migrateRule(c, c.ViewRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.CreateRule, ruleErr = migrateRule(c, c.CreateRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.UpdateRule, ruleErr = migrateRule(c, c.UpdateRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.DeleteRule, ruleErr = migrateRule(c, c.DeleteRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			if c.IsAuth() {
+				opt := c.AuthOptions()
+				opt.ManageRule, ruleErr = migrateRule(c, opt.ManageRule)
+				if ruleErr != nil {
+					return ruleErr
+				}
+				c.SetOptions(opt)
+			}
+
+			if err := dao.Save(c); err != nil {
+				return err
+			}
+		}
+
+		return nil
+	}, func(db dbx.Builder) error {
+		dao := daos.New(db)
+
+		collections := []*models.Collection{}
+		if err := dao.CollectionQuery().All(&collections); err != nil {
+			return err
+		}
+
+		anyOpRegex := regexp.MustCompile(`\?(=|!=|~|!~|>|>=|<|<=)`)
+
+		// replace any/opt operators to their old versions, eg. "?=" => "="
+		revertRule := func(rule *string) (*string, error) {
+			if rule == nil || *rule == "" {
+				return rule, nil
+			}
+
+			newRule := *rule
+			newRule = anyOpRegex.ReplaceAllString(newRule, "${1}")
+
+			return &newRule, nil
+		}
+
+		var ruleErr error
+		for _, c := range collections {
+			c.ListRule, ruleErr = revertRule(c.ListRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.ViewRule, ruleErr = revertRule(c.ViewRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.CreateRule, ruleErr = revertRule(c.CreateRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.UpdateRule, ruleErr = revertRule(c.UpdateRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			c.DeleteRule, ruleErr = revertRule(c.DeleteRule)
+			if ruleErr != nil {
+				return ruleErr
+			}
+
+			if c.IsAuth() {
+				opt := c.AuthOptions()
+				opt.ManageRule, ruleErr = revertRule(opt.ManageRule)
+				if ruleErr != nil {
+					return ruleErr
+				}
+				c.SetOptions(opt)
+			}
+
+			if err := dao.Save(c); err != nil {
+				return err
+			}
+		}
+
+		return nil
+	})
+}

migrations/logs/1640988000_init.go

@@ -11,7 +11,7 @@ func init() {
 	LogsMigrations.Register(func(db dbx.Builder) (err error) {
 		_, err = db.NewQuery(`
 			CREATE TABLE {{_requests}} (
-				[[id]]        TEXT PRIMARY KEY,
+				[[id]]        TEXT PRIMARY KEY NOT NULL,
 				[[url]]       TEXT DEFAULT "" NOT NULL,
 				[[method]]    TEXT DEFAULT "get" NOT NULL,
 				[[status]]    INTEGER DEFAULT 200 NOT NULL,

models/record.go

@@ -4,6 +4,8 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"regexp"
+	"strconv"
 	"time"
 
 	"github.com/pocketbase/dbx"
@@ -57,7 +59,7 @@ func nullStringMapValue(data dbx.NullStringMap, key string) any {
 // NewRecordFromNullStringMap initializes a single new Record model
 // with data loaded from the provided NullStringMap.
 func NewRecordFromNullStringMap(collection *Collection, data dbx.NullStringMap) *Record {
-	resultMap := map[string]any{}
+	resultMap := make(map[string]any, len(data))
 
 	// load schema fields
 	for _, field := range collection.Schema.Fields() {
@@ -132,9 +134,79 @@ func (m *Record) SetExpand(expand map[string]any) {
 	m.expand = shallowCopy(expand)
 }
 
+// MergeExpand merges recursively the provided expand data into
+// the current model's expand (if any).
+//
+// Note that if an expanded prop with the same key is a slice (old or new expand)
+// then both old and new records will be merged into a new slice (aka. a :merge: [b,c] => [a,b,c]).
+// Otherwise the "old" expanded record will be replace with the "new" one (aka. a :merge: aNew => aNew).
+func (m *Record) MergeExpand(expand map[string]any) {
+	if m.expand == nil && len(expand) > 0 {
+		m.expand = make(map[string]any, len(expand))
+	}
+
+	for key, new := range expand {
+		old, ok := m.expand[key]
+		if !ok {
+			m.expand[key] = new
+			continue
+		}
+
+		var wasOldSlice bool
+		var oldSlice []*Record
+		switch v := old.(type) {
+		case *Record:
+			oldSlice = []*Record{v}
+		case []*Record:
+			wasOldSlice = true
+			oldSlice = v
+		default:
+			// invalid old expand data -> assign directly the new
+			// (no matter whether new is valid or not)
+			m.expand[key] = new
+			continue
+		}
+
+		var wasNewSlice bool
+		var newSlice []*Record
+		switch v := new.(type) {
+		case *Record:
+			newSlice = []*Record{v}
+		case []*Record:
+			wasNewSlice = true
+			newSlice = v
+		default:
+			// invalid new expand data -> skip
+			continue
+		}
+
+		oldIndexed := make(map[string]*Record, len(oldSlice))
+		for _, oldRecord := range oldSlice {
+			oldIndexed[oldRecord.Id] = oldRecord
+		}
+
+		for _, newRecord := range newSlice {
+			oldRecord := oldIndexed[newRecord.Id]
+			if oldRecord != nil {
+				// note: there is no need to update oldSlice since oldRecord is a reference
+				oldRecord.MergeExpand(newRecord.Expand())
+			} else {
+				// missing new entry
+				oldSlice = append(oldSlice, newRecord)
+			}
+		}
+
+		if wasOldSlice || wasNewSlice || len(oldSlice) == 0 {
+			m.expand[key] = oldSlice
+		} else {
+			m.expand[key] = oldSlice[0]
+		}
+	}
+}
+
 // SchemaData returns a shallow copy ONLY of the defined record schema fields data.
 func (m *Record) SchemaData() map[string]any {
-	result := map[string]any{}
+	result := make(map[string]any, len(m.collection.Schema.Fields()))
 
 	for _, field := range m.collection.Schema.Fields() {
 		if v, ok := m.data[field.Name]; ok {
@@ -300,7 +372,7 @@ func (m *Record) Load(data map[string]any) {
 
 // ColumnValueMap implements [ColumnValueMapper] interface.
 func (m *Record) ColumnValueMap() map[string]any {
-	result := map[string]any{}
+	result := make(map[string]any, len(m.collection.Schema.Fields())+3)
 
 	// export schema field values
 	for _, field := range m.collection.Schema.Fields() {
@@ -326,7 +398,7 @@ func (m *Record) ColumnValueMap() map[string]any {
 //
 // Fields marked as hidden will be exported only if `m.IgnoreEmailVisibility(true)` is set.
 func (m *Record) PublicExport() map[string]any {
-	result := map[string]any{}
+	result := make(map[string]any, len(m.collection.Schema.Fields())+5)
 
 	// export unknown data fields if allowed
 	if m.exportUnknown {
@@ -387,6 +459,101 @@ func (m *Record) UnmarshalJSON(data []byte) error {
 	return nil
 }
 
+// ReplaceModifers returns a new map with applied modifier
+// values based on the current record and the specified data.
+//
+// The resolved modifier keys will be removed.
+//
+// Multiple modifiers will be applied one after another,
+// while reusing the previous base key value result (eg. 1; -5; +2 => -2).
+//
+// Example usage:
+//
+//  newData := record.ReplaceModifers(data)
+//	// record:  {"field": 10}
+// 	// data:    {"field+": 5}
+// 	// newData: {"field": 15}
+func (m *Record) ReplaceModifers(data map[string]any) map[string]any {
+	var clone = shallowCopy(data)
+	if len(clone) == 0 {
+		return clone
+	}
+
+	var recordDataCache map[string]any
+
+	// export recordData lazily
+	recordData := func() map[string]any {
+		if recordDataCache == nil {
+			recordDataCache = m.SchemaData()
+		}
+		return recordDataCache
+	}
+
+	modifiers := schema.FieldValueModifiers()
+
+	for _, field := range m.Collection().Schema.Fields() {
+		key := field.Name
+
+		for _, m := range modifiers {
+			if mv, mOk := clone[key+m]; mOk {
+				if _, ok := clone[key]; !ok {
+					// get base value from the merged data
+					clone[key] = recordData()[key]
+				}
+
+				clone[key] = field.PrepareValueWithModifier(clone[key], m, mv)
+				delete(clone, key+m)
+			}
+		}
+
+		if field.Type != schema.FieldTypeFile {
+			continue
+		}
+
+		// -----------------------------------------------------------
+		// legacy file field modifiers (kept for backward compatability)
+		// -----------------------------------------------------------
+
+		var oldNames []string
+		var toDelete []string
+		if _, ok := clone[key]; ok {
+			oldNames = list.ToUniqueStringSlice(clone[key])
+		} else {
+			// get oldNames from the model
+			oldNames = list.ToUniqueStringSlice(recordData()[key])
+		}
+
+		// search for individual file name to delete (eg. "file.test.png = null")
+		for _, name := range oldNames {
+			suffixedKey := key + "." + name
+			if v, ok := clone[suffixedKey]; ok && cast.ToString(v) == "" {
+				toDelete = append(toDelete, name)
+				delete(clone, suffixedKey)
+				continue
+			}
+		}
+
+		// search for individual file index to delete (eg. "file.0 = null")
+		keyExp, _ := regexp.Compile(`^` + regexp.QuoteMeta(key) + `\.\d+$`)
+		for indexedKey := range clone {
+			if keyExp.MatchString(indexedKey) && cast.ToString(clone[indexedKey]) == "" {
+				index, indexErr := strconv.Atoi(indexedKey[len(key)+1:])
+				if indexErr != nil || index < 0 || index >= len(oldNames) {
+					continue
+				}
+				toDelete = append(toDelete, oldNames[index])
+				delete(clone, indexedKey)
+			}
+		}
+
+		if toDelete != nil {
+			clone[key] = field.PrepareValue(list.SubtractSlice(oldNames, toDelete))
+		}
+	}
+
+	return clone
+}
+
 // getNormalizeDataValueForDB returns the "key" data value formatted for db storage.
 func (m *Record) getNormalizeDataValueForDB(key string) any {
 	var val any
@@ -426,7 +593,7 @@ func (m *Record) getNormalizeDataValueForDB(key string) any {
 
 // shallowCopy shallow copy data into a new map.
 func shallowCopy(data map[string]any) map[string]any {
-	result := map[string]any{}
+	result := make(map[string]any, len(data))
 
 	for k, v := range data {
 		result[k] = v

models/record_test.go

@@ -138,7 +138,7 @@ func TestNewRecordFromNullStringMap(t *testing.T) {
 			Valid:  true,
 		},
 		"field5": sql.NullString{
-			String: `["test1","test2"]`, // will select only the first elem
+			String: `["test1","test2"]`, // will select only the last elem
 			Valid:  true,
 		},
 		"field6": sql.NullString{
@@ -157,11 +157,11 @@ func TestNewRecordFromNullStringMap(t *testing.T) {
 	}{
 		{
 			models.CollectionTypeBase,
-			`{"collectionId":"","collectionName":"test","created":"2022-01-01 10:00:00.123Z","field1":"test","field2":"","field3":true,"field4":123.123,"field5":"test1","field6":["test"],"id":"test_id","updated":"2022-01-01 10:00:00.456Z"}`,
+			`{"collectionId":"","collectionName":"test","created":"2022-01-01 10:00:00.123Z","field1":"test","field2":"","field3":true,"field4":123.123,"field5":"test2","field6":["test"],"id":"test_id","updated":"2022-01-01 10:00:00.456Z"}`,
 		},
 		{
 			models.CollectionTypeAuth,
-			`{"collectionId":"","collectionName":"test","created":"2022-01-01 10:00:00.123Z","email":"test_email","emailVisibility":true,"field1":"test","field2":"","field3":true,"field4":123.123,"field5":"test1","field6":["test"],"id":"test_id","updated":"2022-01-01 10:00:00.456Z","username":"test_username","verified":false}`,
+			`{"collectionId":"","collectionName":"test","created":"2022-01-01 10:00:00.123Z","email":"test_email","emailVisibility":true,"field1":"test","field2":"","field3":true,"field4":123.123,"field5":"test2","field6":["test"],"id":"test_id","updated":"2022-01-01 10:00:00.456Z","username":"test_username","verified":false}`,
 		},
 	}
 
@@ -358,7 +358,7 @@ func TestRecordOriginalCopy(t *testing.T) {
 	}
 }
 
-func TestRecordExpand(t *testing.T) {
+func TestRecordSetAndGetExpand(t *testing.T) {
 	collection := &models.Collection{}
 	m := models.NewRecord(collection)
 
@@ -375,6 +375,139 @@ func TestRecordExpand(t *testing.T) {
 	}
 }
 
+func TestRecordMergeExpand(t *testing.T) {
+	collection := &models.Collection{}
+	m := models.NewRecord(collection)
+	m.Id = "m"
+
+	// a
+	a := models.NewRecord(collection)
+	a.Id = "a"
+	a1 := models.NewRecord(collection)
+	a1.Id = "a1"
+	a2 := models.NewRecord(collection)
+	a2.Id = "a2"
+	a3 := models.NewRecord(collection)
+	a3.Id = "a3"
+	a31 := models.NewRecord(collection)
+	a31.Id = "a31"
+	a32 := models.NewRecord(collection)
+	a32.Id = "a32"
+	a.SetExpand(map[string]any{
+		"a1":  a1,
+		"a23": []*models.Record{a2, a3},
+	})
+	a3.SetExpand(map[string]any{
+		"a31": a31,
+		"a32": []*models.Record{a32},
+	})
+
+	// b
+	b := models.NewRecord(collection)
+	b.Id = "b"
+	b1 := models.NewRecord(collection)
+	b1.Id = "b1"
+	b.SetExpand(map[string]any{
+		"b1": b1,
+	})
+
+	// c
+	c := models.NewRecord(collection)
+	c.Id = "c"
+
+	// load initial expand
+	m.SetExpand(map[string]any{
+		"a": a,
+		"b": b,
+		"c": []*models.Record{c},
+	})
+
+	// a (new)
+	aNew := models.NewRecord(collection)
+	aNew.Id = a.Id
+	a3New := models.NewRecord(collection)
+	a3New.Id = a3.Id
+	a32New := models.NewRecord(collection)
+	a32New.Id = "a32New"
+	a33New := models.NewRecord(collection)
+	a33New.Id = "a33New"
+	a3New.SetExpand(map[string]any{
+		"a32":    []*models.Record{a32New},
+		"a33New": a33New,
+	})
+	aNew.SetExpand(map[string]any{
+		"a23": []*models.Record{a2, a3New},
+	})
+
+	// b (new)
+	bNew := models.NewRecord(collection)
+	bNew.Id = "bNew"
+	dNew := models.NewRecord(collection)
+	dNew.Id = "dNew"
+
+	// merge expands
+	m.MergeExpand(map[string]any{
+		"a":    aNew,
+		"b":    []*models.Record{bNew},
+		"dNew": dNew,
+	})
+
+	result := m.Expand()
+
+	raw, err := json.Marshal(result)
+	if err != nil {
+		t.Fatal(err)
+	}
+	rawStr := string(raw)
+
+	expected := `{"a":{"collectionId":"","collectionName":"","created":"","expand":{"a1":{"collectionId":"","collectionName":"","created":"","id":"a1","updated":""},"a23":[{"collectionId":"","collectionName":"","created":"","id":"a2","updated":""},{"collectionId":"","collectionName":"","created":"","expand":{"a31":{"collectionId":"","collectionName":"","created":"","id":"a31","updated":""},"a32":[{"collectionId":"","collectionName":"","created":"","id":"a32","updated":""},{"collectionId":"","collectionName":"","created":"","id":"a32New","updated":""}],"a33New":{"collectionId":"","collectionName":"","created":"","id":"a33New","updated":""}},"id":"a3","updated":""}]},"id":"a","updated":""},"b":[{"collectionId":"","collectionName":"","created":"","expand":{"b1":{"collectionId":"","collectionName":"","created":"","id":"b1","updated":""}},"id":"b","updated":""},{"collectionId":"","collectionName":"","created":"","id":"bNew","updated":""}],"c":[{"collectionId":"","collectionName":"","created":"","id":"c","updated":""}],"dNew":{"collectionId":"","collectionName":"","created":"","id":"dNew","updated":""}}`
+
+	if expected != rawStr {
+		t.Fatalf("Expected \n%v, \ngot \n%v", expected, rawStr)
+	}
+}
+
+func TestRecordMergeExpandNilCheck(t *testing.T) {
+	collection := &models.Collection{}
+
+	scenarios := []struct {
+		name     string
+		expand   map[string]any
+		expected string
+	}{
+		{
+			"nil expand",
+			nil,
+			`{"collectionId":"","collectionName":"","created":"","id":"","updated":""}`,
+		},
+		{
+			"empty expand",
+			map[string]any{},
+			`{"collectionId":"","collectionName":"","created":"","id":"","updated":""}`,
+		},
+		{
+			"non-empty expand",
+			map[string]any{"test": models.NewRecord(collection)},
+			`{"collectionId":"","collectionName":"","created":"","expand":{"test":{"collectionId":"","collectionName":"","created":"","id":"","updated":""}},"id":"","updated":""}`,
+		},
+	}
+
+	for _, s := range scenarios {
+		m := models.NewRecord(collection)
+		m.MergeExpand(s.expand)
+
+		raw, err := json.Marshal(m)
+		if err != nil {
+			t.Fatal(err)
+		}
+		rawStr := string(raw)
+
+		if rawStr != s.expected {
+			t.Fatalf("[%s] Expected \n%v, \ngot \n%v", s.name, s.expected, rawStr)
+		}
+	}
+}
+
 func TestRecordSchemaData(t *testing.T) {
 	collection := &models.Collection{
 		Type: models.CollectionTypeAuth,
@@ -1292,6 +1425,109 @@ func TestRecordUnmarshalJSON(t *testing.T) {
 	}
 }
 
+func TestRecordReplaceModifers(t *testing.T) {
+	collection := &models.Collection{
+		Schema: schema.NewSchema(
+			&schema.SchemaField{
+				Name: "text",
+				Type: schema.FieldTypeText,
+			},
+			&schema.SchemaField{
+				Name: "number",
+				Type: schema.FieldTypeNumber,
+			},
+			&schema.SchemaField{
+				Name:    "rel_one",
+				Type:    schema.FieldTypeRelation,
+				Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)},
+			},
+			&schema.SchemaField{
+				Name: "rel_many",
+				Type: schema.FieldTypeRelation,
+			},
+			&schema.SchemaField{
+				Name:    "select_one",
+				Type:    schema.FieldTypeSelect,
+				Options: &schema.SelectOptions{MaxSelect: 1},
+			},
+			&schema.SchemaField{
+				Name:    "select_many",
+				Type:    schema.FieldTypeSelect,
+				Options: &schema.SelectOptions{MaxSelect: 10},
+			},
+			&schema.SchemaField{
+				Name:    "file_one",
+				Type:    schema.FieldTypeFile,
+				Options: &schema.FileOptions{MaxSelect: 1},
+			},
+			&schema.SchemaField{
+				Name:    "file_one_index",
+				Type:    schema.FieldTypeFile,
+				Options: &schema.FileOptions{MaxSelect: 1},
+			},
+			&schema.SchemaField{
+				Name:    "file_one_name",
+				Type:    schema.FieldTypeFile,
+				Options: &schema.FileOptions{MaxSelect: 1},
+			},
+			&schema.SchemaField{
+				Name:    "file_many",
+				Type:    schema.FieldTypeFile,
+				Options: &schema.FileOptions{MaxSelect: 10},
+			},
+		),
+	}
+
+	record := models.NewRecord(collection)
+
+	record.Load(map[string]any{
+		"text":           "test",
+		"number":         10,
+		"rel_one":        "a",
+		"rel_many":       []string{"a", "b"},
+		"select_one":     "a",
+		"select_many":    []string{"a", "b", "c"},
+		"file_one":       "a",
+		"file_one_index": "b",
+		"file_one_name":  "c",
+		"file_many":      []string{"a", "b", "c", "d", "e", "f"},
+	})
+
+	result := record.ReplaceModifers(map[string]any{
+		"text-":            "m-",
+		"text+":            "m+",
+		"number-":          3,
+		"number+":          5,
+		"rel_one-":         "a",
+		"rel_one+":         "b",
+		"rel_many-":        []string{"a"},
+		"rel_many+":        []string{"c", "d", "e"},
+		"select_one-":      "a",
+		"select_one+":      "c",
+		"select_many-":     []string{"b", "c"},
+		"select_many+":     []string{"d", "e"},
+		"file_one+":        "skip", // should be ignored
+		"file_one-":        "a",
+		"file_one_index.0": "",
+		"file_one_name.c":  "",
+		"file_many+":       []string{"e", "f"}, // should be ignored
+		"file_many-":       []string{"c", "d"},
+		"file_many.f":      nil,
+		"file_many.0":      nil,
+	})
+
+	raw, err := json.Marshal(result)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	expected := `{"file_many":["b","e"],"file_one":"","file_one_index":"","file_one_name":"","number":12,"rel_many":["b","c","d","e"],"rel_one":"b","select_many":["a","d","e"],"select_one":"c","text":"test"}`
+
+	if v := string(raw); v != expected {
+		t.Fatalf("Expected \n%s, \ngot \n%s", expected, v)
+	}
+}
+
 // -------------------------------------------------------------------
 // Auth helpers:
 // -------------------------------------------------------------------

models/request_data.go

@@ -1,5 +1,11 @@
 package models
 
+import (
+	"strings"
+
+	"github.com/pocketbase/pocketbase/models/schema"
+)
+
 // RequestData defines a HTTP request data struct, usually used
 // as part of the `@request.*` filter resolver.
 type RequestData struct {
@@ -9,3 +15,18 @@ type RequestData struct {
 	AuthRecord *Record        `json:"authRecord"`
 	Admin      *Admin         `json:"admin"`
 }
+
+// HasModifierDataKeys loosely checks if the current struct has any modifier Data keys.
+func (r *RequestData) HasModifierDataKeys() bool {
+	allModifiers := schema.FieldValueModifiers()
+
+	for key := range r.Data {
+		for _, m := range allModifiers {
+			if strings.HasSuffix(key, m) {
+				return true
+			}
+		}
+	}
+
+	return false
+}

models/request_data_test.go

@@ -0,0 +1,58 @@
+package models_test
+
+import (
+	"testing"
+
+	"github.com/pocketbase/pocketbase/models"
+)
+
+func TestRequestDataHasModifierDataKeys(t *testing.T) {
+	scenarios := []struct {
+		name        string
+		requestData *models.RequestData
+		expected    bool
+	}{
+		{
+			"empty",
+			&models.RequestData{},
+			false,
+		},
+		{
+			"Data with regular fields",
+			&models.RequestData{
+				Query: map[string]any{"data+": "demo"}, // should be ignored
+				Data:  map[string]any{"a": 123, "b": "test", "c.d": false},
+			},
+			false,
+		},
+		{
+			"Data with +modifier fields",
+			&models.RequestData{
+				Data: map[string]any{"a+": 123, "b": "test", "c.d": false},
+			},
+			true,
+		},
+		{
+			"Data with -modifier fields",
+			&models.RequestData{
+				Data: map[string]any{"a": 123, "b-": "test", "c.d": false},
+			},
+			true,
+		},
+		{
+			"Data with mixed modifier fields",
+			&models.RequestData{
+				Data: map[string]any{"a": 123, "b-": "test", "c.d+": false},
+			},
+			true,
+		},
+	}
+
+	for _, s := range scenarios {
+		result := s.requestData.HasModifierDataKeys()
+
+		if result != s.expected {
+			t.Fatalf("[%s] Expected %v, got %v", s.name, s.expected, result)
+		}
+	}
+}

models/schema/schema_field.go

@@ -15,22 +15,36 @@ import (
 
 var schemaFieldNameRegex = regexp.MustCompile(`^\w+$`)
 
+// field value modifiers
+const (
+	FieldValueModifierAdd      string = "+"
+	FieldValueModifierSubtract string = "-"
+)
+
+// FieldValueModifiers returns a list with all available field modifier tokens.
+func FieldValueModifiers() []string {
+	return []string{
+		FieldValueModifierAdd,
+		FieldValueModifierSubtract,
+	}
+}
+
 // commonly used field names
 const (
-	FieldNameId                     = "id"
-	FieldNameCreated                = "created"
-	FieldNameUpdated                = "updated"
-	FieldNameCollectionId           = "collectionId"
-	FieldNameCollectionName         = "collectionName"
-	FieldNameExpand                 = "expand"
-	FieldNameUsername               = "username"
-	FieldNameEmail                  = "email"
-	FieldNameEmailVisibility        = "emailVisibility"
-	FieldNameVerified               = "verified"
-	FieldNameTokenKey               = "tokenKey"
-	FieldNamePasswordHash           = "passwordHash"
-	FieldNameLastResetSentAt        = "lastResetSentAt"
-	FieldNameLastVerificationSentAt = "lastVerificationSentAt"
+	FieldNameId                     string = "id"
+	FieldNameCreated                string = "created"
+	FieldNameUpdated                string = "updated"
+	FieldNameCollectionId           string = "collectionId"
+	FieldNameCollectionName         string = "collectionName"
+	FieldNameExpand                 string = "expand"
+	FieldNameUsername               string = "username"
+	FieldNameEmail                  string = "email"
+	FieldNameEmailVisibility        string = "emailVisibility"
+	FieldNameVerified               string = "verified"
+	FieldNameTokenKey               string = "tokenKey"
+	FieldNamePasswordHash           string = "passwordHash"
+	FieldNameLastResetSentAt        string = "lastResetSentAt"
+	FieldNameLastVerificationSentAt string = "lastVerificationSentAt"
 )
 
 // BaseModelFieldNames returns the field names that all models have (id, created, updated).
@@ -168,8 +182,8 @@ func (f SchemaField) Validate() error {
 	f.InitOptions()
 
 	excludeNames := BaseModelFieldNames()
-	// exclude filter literals
-	excludeNames = append(excludeNames, "null", "true", "false")
+	// exclude special filter literals
+	excludeNames = append(excludeNames, "null", "true", "false", "isset")
 	// exclude system literals
 	excludeNames = append(excludeNames, SystemFieldNames()...)
 
@@ -276,7 +290,7 @@ func (f *SchemaField) PrepareValue(value any) any {
 		options, _ := f.Options.(*SelectOptions)
 		if options.MaxSelect <= 1 {
 			if len(val) > 0 {
-				return val[0]
+				return val[len(val)-1] // the last selected
 			}
 			return ""
 		}
@@ -288,7 +302,7 @@ func (f *SchemaField) PrepareValue(value any) any {
 		options, _ := f.Options.(*FileOptions)
 		if options.MaxSelect <= 1 {
 			if len(val) > 0 {
-				return val[0]
+				return val[len(val)-1] // the last selected
 			}
 			return ""
 		}
@@ -300,7 +314,7 @@ func (f *SchemaField) PrepareValue(value any) any {
 		options, _ := f.Options.(*RelationOptions)
 		if options.MaxSelect != nil && *options.MaxSelect <= 1 {
 			if len(ids) > 0 {
-				return ids[0]
+				return ids[len(ids)-1] // the last selected
 			}
 			return ""
 		}
@@ -311,6 +325,46 @@ func (f *SchemaField) PrepareValue(value any) any {
 	}
 }
 
+// PrepareValueWithModifier returns normalized and properly formatted field value
+// by "merging" baseValue with the modifierValue based on the specified modifier (+ or -).
+func (f *SchemaField) PrepareValueWithModifier(baseValue any, modifier string, modifierValue any) any {
+	resolvedValue := baseValue
+
+	switch f.Type {
+	case FieldTypeNumber:
+		switch modifier {
+		case FieldValueModifierAdd:
+			resolvedValue = cast.ToFloat64(baseValue) + cast.ToFloat64(modifierValue)
+		case FieldValueModifierSubtract:
+			resolvedValue = cast.ToFloat64(baseValue) - cast.ToFloat64(modifierValue)
+		}
+	case FieldTypeSelect, FieldTypeRelation:
+		switch modifier {
+		case FieldValueModifierAdd:
+			resolvedValue = append(
+				list.ToUniqueStringSlice(baseValue),
+				list.ToUniqueStringSlice(modifierValue)...,
+			)
+		case FieldValueModifierSubtract:
+			resolvedValue = list.SubtractSlice(
+				list.ToUniqueStringSlice(baseValue),
+				list.ToUniqueStringSlice(modifierValue),
+			)
+		}
+	case FieldTypeFile:
+		// note: file for now supports only the subtract modifier
+		switch modifier {
+		case FieldValueModifierSubtract:
+			resolvedValue = list.SubtractSlice(
+				list.ToUniqueStringSlice(baseValue),
+				list.ToUniqueStringSlice(modifierValue),
+			)
+		}
+	}
+
+	return f.PrepareValue(resolvedValue)
+}
+
 // -------------------------------------------------------------------
 
 // FieldOptions interfaces that defines common methods that every field options struct has.

models/schema/schema_field_test.go

@@ -603,7 +603,7 @@ func TestSchemaFieldPrepareValue(t *testing.T) {
 		{schema.SchemaField{Type: schema.FieldTypeSelect}, "", `""`},
 		{schema.SchemaField{Type: schema.FieldTypeSelect}, 123, `"123"`},
 		{schema.SchemaField{Type: schema.FieldTypeSelect}, "test", `"test"`},
-		{schema.SchemaField{Type: schema.FieldTypeSelect}, []string{"test1", "test2"}, `"test1"`},
+		{schema.SchemaField{Type: schema.FieldTypeSelect}, []string{"test1", "test2"}, `"test2"`},
 		{
 			// no values validation/filtering
 			schema.SchemaField{
@@ -680,7 +680,7 @@ func TestSchemaFieldPrepareValue(t *testing.T) {
 		{schema.SchemaField{Type: schema.FieldTypeFile}, "", `""`},
 		{schema.SchemaField{Type: schema.FieldTypeFile}, 123, `"123"`},
 		{schema.SchemaField{Type: schema.FieldTypeFile}, "test", `"test"`},
-		{schema.SchemaField{Type: schema.FieldTypeFile}, []string{"test1", "test2"}, `"test1"`},
+		{schema.SchemaField{Type: schema.FieldTypeFile}, []string{"test1", "test2"}, `"test2"`},
 		// file (multiple)
 		{
 			schema.SchemaField{
@@ -785,7 +785,7 @@ func TestSchemaFieldPrepareValue(t *testing.T) {
 		{
 			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
 			[]string{"1ba88b4f-e9da-42f0-9764-9a55c953e724", "2ba88b4f-e9da-42f0-9764-9a55c953e724"},
-			`"1ba88b4f-e9da-42f0-9764-9a55c953e724"`,
+			`"2ba88b4f-e9da-42f0-9764-9a55c953e724"`,
 		},
 		// relation (multiple)
 		{
@@ -863,6 +863,696 @@ func TestSchemaFieldPrepareValue(t *testing.T) {
 	}
 }
 
+func TestSchemaFieldPrepareValueWithModifier(t *testing.T) {
+	scenarios := []struct {
+		name          string
+		field         schema.SchemaField
+		baseValue     any
+		modifier      string
+		modifierValue any
+		expectJson    string
+	}{
+		// text
+		{
+			"text with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeText},
+			"base",
+			"+",
+			"new",
+			`"base"`,
+		},
+		{
+			"text with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeText},
+			"base",
+			"-",
+			"new",
+			`"base"`,
+		},
+		{
+			"text with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeText},
+			"base",
+			"?",
+			"new",
+			`"base"`,
+		},
+		{
+			"text cast check",
+			schema.SchemaField{Type: schema.FieldTypeText},
+			123,
+			"?",
+			"new",
+			`"123"`,
+		},
+
+		// number
+		{
+			"number with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeNumber},
+			1,
+			"+",
+			4,
+			`5`,
+		},
+		{
+			"number with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeNumber},
+			1,
+			"-",
+			4,
+			`-3`,
+		},
+		{
+			"number with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeNumber},
+			"1",
+			"?",
+			4,
+			`1`,
+		},
+		{
+			"number cast check",
+			schema.SchemaField{Type: schema.FieldTypeNumber},
+			"test",
+			"+",
+			"4",
+			`4`,
+		},
+
+		// bool
+		{
+			"bool with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeBool},
+			true,
+			"+",
+			false,
+			`true`,
+		},
+		{
+			"bool with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeBool},
+			true,
+			"-",
+			false,
+			`true`,
+		},
+		{
+			"bool with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeBool},
+			true,
+			"?",
+			false,
+			`true`,
+		},
+		{
+			"bool cast check",
+			schema.SchemaField{Type: schema.FieldTypeBool},
+			"true",
+			"?",
+			false,
+			`true`,
+		},
+
+		// email
+		{
+			"email with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeEmail},
+			"base",
+			"+",
+			"new",
+			`"base"`,
+		},
+		{
+			"email with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeEmail},
+			"base",
+			"-",
+			"new",
+			`"base"`,
+		},
+		{
+			"email with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeEmail},
+			"base",
+			"?",
+			"new",
+			`"base"`,
+		},
+		{
+			"email cast check",
+			schema.SchemaField{Type: schema.FieldTypeEmail},
+			123,
+			"?",
+			"new",
+			`"123"`,
+		},
+
+		// url
+		{
+			"url with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeUrl},
+			"base",
+			"+",
+			"new",
+			`"base"`,
+		},
+		{
+			"url with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeUrl},
+			"base",
+			"-",
+			"new",
+			`"base"`,
+		},
+		{
+			"url with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeUrl},
+			"base",
+			"?",
+			"new",
+			`"base"`,
+		},
+		{
+			"url cast check",
+			schema.SchemaField{Type: schema.FieldTypeUrl},
+			123,
+			"-",
+			"new",
+			`"123"`,
+		},
+
+		// date
+		{
+			"date with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeDate},
+			"2023-01-01 00:00:00.123",
+			"+",
+			"2023-02-01 00:00:00.456",
+			`"2023-01-01 00:00:00.123Z"`,
+		},
+		{
+			"date with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeDate},
+			"2023-01-01 00:00:00.123Z",
+			"-",
+			"2023-02-01 00:00:00.456Z",
+			`"2023-01-01 00:00:00.123Z"`,
+		},
+		{
+			"date with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeDate},
+			"2023-01-01 00:00:00.123",
+			"?",
+			"2023-01-01 00:00:00.456",
+			`"2023-01-01 00:00:00.123Z"`,
+		},
+		{
+			"date cast check",
+			schema.SchemaField{Type: schema.FieldTypeDate},
+			1672524000, // 2022-12-31 22:00:00.000Z
+			"+",
+			100,
+			`"2022-12-31 22:00:00.000Z"`,
+		},
+
+		// json
+		{
+			"json with '+' modifier",
+			schema.SchemaField{Type: schema.FieldTypeJson},
+			10,
+			"+",
+			5,
+			`10`,
+		},
+		{
+			"json with '+' modifier (slice)",
+			schema.SchemaField{Type: schema.FieldTypeJson},
+			[]string{"a", "b"},
+			"+",
+			"c",
+			`["a","b"]`,
+		},
+		{
+			"json with '-' modifier",
+			schema.SchemaField{Type: schema.FieldTypeJson},
+			10,
+			"-",
+			5,
+			`10`,
+		},
+		{
+			"json with '-' modifier (slice)",
+			schema.SchemaField{Type: schema.FieldTypeJson},
+			`["a","b"]`,
+			"-",
+			"c",
+			`["a","b"]`,
+		},
+		{
+			"json with unknown modifier",
+			schema.SchemaField{Type: schema.FieldTypeJson},
+			`"base"`,
+			"?",
+			`"new"`,
+			`"base"`,
+		},
+
+		// single select
+		{
+			"single select with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"",
+			"+",
+			"b",
+			`"b"`,
+		},
+		{
+			"single select with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"a",
+			"+",
+			"b",
+			`"b"`,
+		},
+		{
+			"single select with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single select with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"",
+			`"a"`,
+		},
+		{
+			"single select with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"b",
+			`"a"`,
+		},
+		{
+			"single select with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single select with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			[]string{"b", "a", "c", "123"},
+			`""`,
+		},
+		{
+			"single select with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 1}},
+			"",
+			"?",
+			"a",
+			`""`,
+		},
+
+		// multi select
+		{
+			"multi select with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			nil,
+			"+",
+			"b",
+			`["b"]`,
+		},
+		{
+			"multi select with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			[]string{"a"},
+			"+",
+			[]string{"b", "c"},
+			`["a","b","c"]`,
+		},
+		{
+			"multi select with '+' modifier (nonempty base; already existing value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			[]string{"a", "b"},
+			"+",
+			"b",
+			`["a","b"]`,
+		},
+		{
+			"multi select with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			nil,
+			"-",
+			[]string{"a"},
+			`[]`,
+		},
+		{
+			"multi select with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			"a",
+			"-",
+			"",
+			`["a"]`,
+		},
+		{
+			"multi select with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			"a",
+			"-",
+			"b",
+			`["a"]`,
+		},
+		{
+			"multi select with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			"c",
+			`["a","b","d"]`,
+		},
+		{
+			"multi select with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			[]string{"b", "a", "123"},
+			`["c","d"]`,
+		},
+		{
+			"multi select with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeSelect, Options: &schema.SelectOptions{MaxSelect: 10}},
+			[]string{"a", "b"},
+			"?",
+			"a",
+			`["a","b"]`,
+		},
+
+		// single relation
+		{
+			"single relation with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"",
+			"+",
+			"b",
+			`"b"`,
+		},
+		{
+			"single relation with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"a",
+			"+",
+			"b",
+			`"b"`,
+		},
+		{
+			"single relation with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single relation with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"a",
+			"-",
+			"",
+			`"a"`,
+		},
+		{
+			"single relation with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"a",
+			"-",
+			"b",
+			`"a"`,
+		},
+		{
+			"single relation with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"a",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single relation with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"a",
+			"-",
+			[]string{"b", "a", "c", "123"},
+			`""`,
+		},
+		{
+			"single relation with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeRelation, Options: &schema.RelationOptions{MaxSelect: types.Pointer(1)}},
+			"",
+			"?",
+			"a",
+			`""`,
+		},
+
+		// multi relation
+		{
+			"multi relation with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			nil,
+			"+",
+			"b",
+			`["b"]`,
+		},
+		{
+			"multi relation with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			[]string{"a"},
+			"+",
+			[]string{"b", "c"},
+			`["a","b","c"]`,
+		},
+		{
+			"multi relation with '+' modifier (nonempty base; already existing value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			[]string{"a", "b"},
+			"+",
+			"b",
+			`["a","b"]`,
+		},
+		{
+			"multi relation with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			nil,
+			"-",
+			[]string{"a"},
+			`[]`,
+		},
+		{
+			"multi relation with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			"a",
+			"-",
+			"",
+			`["a"]`,
+		},
+		{
+			"multi relation with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			"a",
+			"-",
+			"b",
+			`["a"]`,
+		},
+		{
+			"multi relation with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			"c",
+			`["a","b","d"]`,
+		},
+		{
+			"multi relation with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			[]string{"b", "a", "123"},
+			`["c","d"]`,
+		},
+		{
+			"multi relation with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeRelation},
+			[]string{"a", "b"},
+			"?",
+			"a",
+			`["a","b"]`,
+		},
+
+		// single file
+		{
+			"single file with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"",
+			"+",
+			"b",
+			`""`,
+		},
+		{
+			"single file with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"a",
+			"+",
+			"b",
+			`"a"`,
+		},
+		{
+			"single file with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single file with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"",
+			`"a"`,
+		},
+		{
+			"single file with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"b",
+			`"a"`,
+		},
+		{
+			"single file with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			"a",
+			`""`,
+		},
+		{
+			"single file with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"a",
+			"-",
+			[]string{"b", "a", "c", "123"},
+			`""`,
+		},
+		{
+			"single file with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 1}},
+			"",
+			"?",
+			"a",
+			`""`,
+		},
+
+		// multi file
+		{
+			"multi file with '+' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			nil,
+			"+",
+			"b",
+			`[]`,
+		},
+		{
+			"multi file with '+' modifier (nonempty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			[]string{"a"},
+			"+",
+			[]string{"b", "c"},
+			`["a"]`,
+		},
+		{
+			"multi file with '+' modifier (nonempty base; already existing value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			[]string{"a", "b"},
+			"+",
+			"b",
+			`["a","b"]`,
+		},
+		{
+			"multi file with '-' modifier (empty base)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			nil,
+			"-",
+			[]string{"a"},
+			`[]`,
+		},
+		{
+			"multi file with '-' modifier (nonempty base and empty modifier value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			"a",
+			"-",
+			"",
+			`["a"]`,
+		},
+		{
+			"multi file with '-' modifier (nonempty base and different value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			"a",
+			"-",
+			"b",
+			`["a"]`,
+		},
+		{
+			"multi file with '-' modifier (nonempty base and matching value)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			"c",
+			`["a","b","d"]`,
+		},
+		{
+			"multi file with '-' modifier (nonempty base and matching value in a slice)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			[]string{"a", "b", "c", "d"},
+			"-",
+			[]string{"b", "a", "123"},
+			`["c","d"]`,
+		},
+		{
+			"multi file with unknown modifier (nonempty)",
+			schema.SchemaField{Type: schema.FieldTypeFile, Options: &schema.FileOptions{MaxSelect: 10}},
+			[]string{"a", "b"},
+			"?",
+			"a",
+			`["a","b"]`,
+		},
+	}
+
+	for _, s := range scenarios {
+		result := s.field.PrepareValueWithModifier(s.baseValue, s.modifier, s.modifierValue)
+
+		encoded, err := json.Marshal(result)
+		if err != nil {
+			t.Fatalf("[%s] %v", s.name, err)
+		}
+
+		if string(encoded) != s.expectJson {
+			t.Fatalf("[%s], Expected %v, got %v", s.name, s.expectJson, string(encoded))
+		}
+	}
+}
+
 // -------------------------------------------------------------------
 
 type fieldOptionsScenario struct {
@@ -1306,7 +1996,7 @@ func TestRelationOptionsValidate(t *testing.T) {
 			[]string{"maxSelect"},
 		},
 		{
-			"MaxSelect > 0 && non-empty CollectionId",
+			"MaxSelect > 0 && nonempty CollectionId",
 			schema.RelationOptions{
 				CollectionId: "abc",
 				MaxSelect:    types.Pointer(1),

models/settings/settings.go

@@ -10,6 +10,7 @@ import (
 	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/go-ozzo/ozzo-validation/v4/is"
 	"github.com/pocketbase/pocketbase/tools/auth"
+	"github.com/pocketbase/pocketbase/tools/mailer"
 	"github.com/pocketbase/pocketbase/tools/rest"
 	"github.com/pocketbase/pocketbase/tools/security"
 )
@@ -43,6 +44,8 @@ type Settings struct {
 	SpotifyAuth   AuthProviderConfig `form:"spotifyAuth" json:"spotifyAuth"`
 	KakaoAuth     AuthProviderConfig `form:"kakaoAuth" json:"kakaoAuth"`
 	TwitchAuth    AuthProviderConfig `form:"twitchAuth" json:"twitchAuth"`
+	StravaAuth    AuthProviderConfig `form:"stravaAuth" json:"stravaAuth"`
+	GiteeAuth     AuthProviderConfig `form:"giteeAuth" json:"giteeAuth"`
 }
 
 // New creates and returns a new default Settings instance.
@@ -123,6 +126,12 @@ func New() *Settings {
 		TwitchAuth: AuthProviderConfig{
 			Enabled: false,
 		},
+		StravaAuth: AuthProviderConfig{
+			Enabled: false,
+		},
+		GiteeAuth: AuthProviderConfig{
+			Enabled: false,
+		},
 	}
 }
 
@@ -152,6 +161,8 @@ func (s *Settings) Validate() error {
 		validation.Field(&s.SpotifyAuth),
 		validation.Field(&s.KakaoAuth),
 		validation.Field(&s.TwitchAuth),
+		validation.Field(&s.StravaAuth),
+		validation.Field(&s.GiteeAuth),
 	)
 }
 
@@ -206,6 +217,8 @@ func (s *Settings) RedactClone() (*Settings, error) {
 		&clone.SpotifyAuth.ClientSecret,
 		&clone.KakaoAuth.ClientSecret,
 		&clone.TwitchAuth.ClientSecret,
+		&clone.StravaAuth.ClientSecret,
+		&clone.GiteeAuth.ClientSecret,
 	}
 
 	// mask all sensitive fields
@@ -235,6 +248,8 @@ func (s *Settings) NamedAuthProviderConfigs() map[string]AuthProviderConfig {
 		auth.NameSpotify:   s.SpotifyAuth,
 		auth.NameKakao:     s.KakaoAuth,
 		auth.NameTwitch:    s.TwitchAuth,
+		auth.NameStrava:    s.StravaAuth,
+		auth.NameGitee:     s.GiteeAuth,
 	}
 }
 
@@ -262,6 +277,9 @@ type SmtpConfig struct {
 	Username string `form:"username" json:"username"`
 	Password string `form:"password" json:"password"`
 
+	// SMTP AUTH - PLAIN (default) or LOGIN
+	AuthMethod string `form:"authMethod" json:"authMethod"`
+
 	// Whether to enforce TLS encryption for the mail server connection.
 	//
 	// When set to false StartTLS command is send, leaving the server
@@ -272,8 +290,23 @@ type SmtpConfig struct {
 // Validate makes SmtpConfig validatable by implementing [validation.Validatable] interface.
 func (c SmtpConfig) Validate() error {
 	return validation.ValidateStruct(&c,
-		validation.Field(&c.Host, is.Host, validation.When(c.Enabled, validation.Required)),
-		validation.Field(&c.Port, validation.When(c.Enabled, validation.Required), validation.Min(0)),
+		validation.Field(
+			&c.Host,
+			validation.When(c.Enabled, validation.Required),
+			is.Host,
+		),
+		validation.Field(
+			&c.Port,
+			validation.When(c.Enabled, validation.Required),
+			validation.Min(0),
+		),
+		validation.Field(
+			&c.AuthMethod,
+			// don't require it for backward compatibility
+			// (fallback internally to PLAIN)
+			// validation.When(c.Enabled, validation.Required),
+			validation.In(mailer.SmtpAuthLogin, mailer.SmtpAuthPlain),
+		),
 	)
 }
 

models/settings/settings_test.go

@@ -9,6 +9,7 @@ import (
 	validation "github.com/go-ozzo/ozzo-validation/v4"
 	"github.com/pocketbase/pocketbase/models/settings"
 	"github.com/pocketbase/pocketbase/tools/auth"
+	"github.com/pocketbase/pocketbase/tools/mailer"
 )
 
 func TestSettingsValidate(t *testing.T) {
@@ -47,6 +48,10 @@ func TestSettingsValidate(t *testing.T) {
 	s.KakaoAuth.ClientId = ""
 	s.TwitchAuth.Enabled = true
 	s.TwitchAuth.ClientId = ""
+	s.StravaAuth.Enabled = true
+	s.StravaAuth.ClientId = ""
+	s.GiteeAuth.Enabled = true
+	s.GiteeAuth.ClientId = ""
 
 	// check if Validate() is triggering the members validate methods.
 	err := s.Validate()
@@ -75,6 +80,8 @@ func TestSettingsValidate(t *testing.T) {
 		`"spotifyAuth":{`,
 		`"kakaoAuth":{`,
 		`"twitchAuth":{`,
+		`"stravaAuth":{`,
+		`"giteeAuth":{`,
 	}
 
 	errBytes, _ := json.Marshal(err)
@@ -123,6 +130,10 @@ func TestSettingsMerge(t *testing.T) {
 	s2.KakaoAuth.ClientId = "kakao_test"
 	s2.TwitchAuth.Enabled = true
 	s2.TwitchAuth.ClientId = "twitch_test"
+	s2.StravaAuth.Enabled = true
+	s2.StravaAuth.ClientId = "strava_test"
+	s2.GiteeAuth.Enabled = true
+	s2.GiteeAuth.ClientId = "gitee_test"
 
 	if err := s1.Merge(s2); err != nil {
 		t.Fatal(err)
@@ -194,6 +205,8 @@ func TestSettingsRedactClone(t *testing.T) {
 	s1.SpotifyAuth.ClientSecret = "test123"
 	s1.KakaoAuth.ClientSecret = "test123"
 	s1.TwitchAuth.ClientSecret = "test123"
+	s1.StravaAuth.ClientSecret = "test123"
+	s1.GiteeAuth.ClientSecret = "test123"
 
 	s2, err := s1.RedactClone()
 	if err != nil {
@@ -205,7 +218,7 @@ func TestSettingsRedactClone(t *testing.T) {
 		t.Fatal(err)
 	}
 
-	expected := `{"meta":{"appName":"test123","appUrl":"http://localhost:8090","hideControls":false,"senderName":"Support","senderAddress":"support@example.com","verificationTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eThank you for joining us at {APP_NAME}.\u003c/p\u003e\n\u003cp\u003eClick on the button below to verify your email address.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eVerify\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Verify your {APP_NAME} email","actionUrl":"{APP_URL}/_/#/auth/confirm-verification/{TOKEN}"},"resetPasswordTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eClick on the button below to reset your password.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eReset password\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003ci\u003eIf you didn't ask to reset your password, you can ignore this email.\u003c/i\u003e\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Reset your {APP_NAME} password","actionUrl":"{APP_URL}/_/#/auth/confirm-password-reset/{TOKEN}"},"confirmEmailChangeTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eClick on the button below to confirm your new email address.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eConfirm new email\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003ci\u003eIf you didn't ask to change your email address, you can ignore this email.\u003c/i\u003e\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Confirm your {APP_NAME} new email address","actionUrl":"{APP_URL}/_/#/auth/confirm-email-change/{TOKEN}"}},"logs":{"maxDays":5},"smtp":{"enabled":false,"host":"smtp.example.com","port":587,"username":"","password":"******","tls":true},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","secret":"******","forcePathStyle":false},"adminAuthToken":{"secret":"******","duration":1209600},"adminPasswordResetToken":{"secret":"******","duration":1800},"recordAuthToken":{"secret":"******","duration":1209600},"recordPasswordResetToken":{"secret":"******","duration":1800},"recordEmailChangeToken":{"secret":"******","duration":1800},"recordVerificationToken":{"secret":"******","duration":604800},"emailAuth":{"enabled":false,"exceptDomains":null,"onlyDomains":null,"minPasswordLength":0},"googleAuth":{"enabled":false,"clientSecret":"******"},"facebookAuth":{"enabled":false,"clientSecret":"******"},"githubAuth":{"enabled":false,"clientSecret":"******"},"gitlabAuth":{"enabled":false,"clientSecret":"******"},"discordAuth":{"enabled":false,"clientSecret":"******"},"twitterAuth":{"enabled":false,"clientSecret":"******"},"microsoftAuth":{"enabled":false,"clientSecret":"******"},"spotifyAuth":{"enabled":false,"clientSecret":"******"},"kakaoAuth":{"enabled":false,"clientSecret":"******"},"twitchAuth":{"enabled":false,"clientSecret":"******"}}`
+	expected := `{"meta":{"appName":"test123","appUrl":"http://localhost:8090","hideControls":false,"senderName":"Support","senderAddress":"support@example.com","verificationTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eThank you for joining us at {APP_NAME}.\u003c/p\u003e\n\u003cp\u003eClick on the button below to verify your email address.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eVerify\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Verify your {APP_NAME} email","actionUrl":"{APP_URL}/_/#/auth/confirm-verification/{TOKEN}"},"resetPasswordTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eClick on the button below to reset your password.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eReset password\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003ci\u003eIf you didn't ask to reset your password, you can ignore this email.\u003c/i\u003e\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Reset your {APP_NAME} password","actionUrl":"{APP_URL}/_/#/auth/confirm-password-reset/{TOKEN}"},"confirmEmailChangeTemplate":{"body":"\u003cp\u003eHello,\u003c/p\u003e\n\u003cp\u003eClick on the button below to confirm your new email address.\u003c/p\u003e\n\u003cp\u003e\n  \u003ca class=\"btn\" href=\"{ACTION_URL}\" target=\"_blank\" rel=\"noopener\"\u003eConfirm new email\u003c/a\u003e\n\u003c/p\u003e\n\u003cp\u003e\u003ci\u003eIf you didn't ask to change your email address, you can ignore this email.\u003c/i\u003e\u003c/p\u003e\n\u003cp\u003e\n  Thanks,\u003cbr/\u003e\n  {APP_NAME} team\n\u003c/p\u003e","subject":"Confirm your {APP_NAME} new email address","actionUrl":"{APP_URL}/_/#/auth/confirm-email-change/{TOKEN}"}},"logs":{"maxDays":5},"smtp":{"enabled":false,"host":"smtp.example.com","port":587,"username":"","password":"******","authMethod":"","tls":true},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","secret":"******","forcePathStyle":false},"adminAuthToken":{"secret":"******","duration":1209600},"adminPasswordResetToken":{"secret":"******","duration":1800},"recordAuthToken":{"secret":"******","duration":1209600},"recordPasswordResetToken":{"secret":"******","duration":1800},"recordEmailChangeToken":{"secret":"******","duration":1800},"recordVerificationToken":{"secret":"******","duration":604800},"emailAuth":{"enabled":false,"exceptDomains":null,"onlyDomains":null,"minPasswordLength":0},"googleAuth":{"enabled":false,"clientSecret":"******"},"facebookAuth":{"enabled":false,"clientSecret":"******"},"githubAuth":{"enabled":false,"clientSecret":"******"},"gitlabAuth":{"enabled":false,"clientSecret":"******"},"discordAuth":{"enabled":false,"clientSecret":"******"},"twitterAuth":{"enabled":false,"clientSecret":"******"},"microsoftAuth":{"enabled":false,"clientSecret":"******"},"spotifyAuth":{"enabled":false,"clientSecret":"******"},"kakaoAuth":{"enabled":false,"clientSecret":"******"},"twitchAuth":{"enabled":false,"clientSecret":"******"},"stravaAuth":{"enabled":false,"clientSecret":"******"},"giteeAuth":{"enabled":false,"clientSecret":"******"}}`
 
 	if encodedStr := string(encoded); encodedStr != expected {
 		t.Fatalf("Expected\n%v\ngot\n%v", expected, encodedStr)
@@ -226,6 +239,8 @@ func TestNamedAuthProviderConfigs(t *testing.T) {
 	s.SpotifyAuth.ClientId = "spotify_test"
 	s.KakaoAuth.ClientId = "kakao_test"
 	s.TwitchAuth.ClientId = "twitch_test"
+	s.StravaAuth.ClientId = "strava_test"
+	s.GiteeAuth.ClientId = "gitee_test"
 
 	result := s.NamedAuthProviderConfigs()
 
@@ -246,6 +261,8 @@ func TestNamedAuthProviderConfigs(t *testing.T) {
 		`"twitter":{"enabled":false,"clientId":"twitter_test"}`,
 		`"kakao":{"enabled":false,"clientId":"kakao_test"}`,
 		`"twitch":{"enabled":false,"clientId":"twitch_test"}`,
+		`"strava":{"enabled":false,"clientId":"strava_test"}`,
+		`"gitee":{"enabled":false,"clientId":"gitee_test"}`,
 	}
 	for _, p := range expectedParts {
 		if !strings.Contains(encodedStr, p) {
@@ -327,7 +344,17 @@ func TestSmtpConfigValidate(t *testing.T) {
 			},
 			true,
 		},
-		// valid data
+		// invalid auth method
+		{
+			settings.SmtpConfig{
+				Enabled:    true,
+				Host:       "example.com",
+				Port:       100,
+				AuthMethod: "example",
+			},
+			true,
+		},
+		// valid data (no explicit auth method)
 		{
 			settings.SmtpConfig{
 				Enabled: true,
@@ -337,6 +364,16 @@ func TestSmtpConfigValidate(t *testing.T) {
 			},
 			false,
 		},
+		// valid data (explicit auth method - login)
+		{
+			settings.SmtpConfig{
+				Enabled:    true,
+				Host:       "example.com",
+				Port:       100,
+				AuthMethod: mailer.SmtpAuthLogin,
+			},
+			false,
+		},
 	}
 
 	for i, scenario := range scenarios {

plugins/jsvm/migrations.go

@@ -1,8 +1,10 @@
 package jsvm
 
 import (
+	"fmt"
 	"os"
 	"path/filepath"
+	"strings"
 
 	"github.com/dop251/goja_nodejs/console"
 	"github.com/dop251/goja_nodejs/require"
@@ -75,7 +77,7 @@ func RegisterMigrations(app core.App, options *MigrationsOptions) error {
 
 		_, err := vm.RunString(string(content))
 		if err != nil {
-			return err
+			return fmt.Errorf("failed to run migration %s: %w", file, err)
 		}
 	}
 
@@ -97,8 +99,8 @@ func readDirFiles(dirPath string) (map[string][]byte, error) {
 	result := map[string][]byte{}
 
 	for _, f := range files {
-		if f.IsDir() {
-			continue
+		if f.IsDir() || !strings.HasSuffix(f.Name(), ".js") {
+			continue // not a .js file
 		}
 		raw, err := os.ReadFile(filepath.Join(dirPath, f.Name()))
 		if err != nil {

pocketbase.go

@@ -11,6 +11,7 @@ import (
 
 	"github.com/pocketbase/pocketbase/cmd"
 	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/spf13/cobra"
 )
 
@@ -49,6 +50,12 @@ type Config struct {
 
 	// hide the default console server info on app startup
 	HideStartBanner bool
+
+	// optional DB configurations
+	DataMaxOpenConns int // default to core.DefaultDataMaxOpenConns
+	DataMaxIdleConns int // default to core.DefaultDataMaxIdleConns
+	LogsMaxOpenConns int // default to core.DefaultLogsMaxOpenConns
+	LogsMaxIdleConns int // default to core.DefaultLogsMaxIdleConns
 }
 
 // New creates a new PocketBase instance with the default configuration.
@@ -62,7 +69,7 @@ type Config struct {
 func New() *PocketBase {
 	_, isUsingGoRun := inspectRuntime()
 
-	return NewWithConfig(Config{
+	return NewWithConfig(&Config{
 		DefaultDebug: isUsingGoRun,
 	})
 }
@@ -74,7 +81,11 @@ func New() *PocketBase {
 // Everything will be initialized when [Start()] is executed.
 // If you want to initialize the application before calling [Start()],
 // then you'll have to manually call [Bootstrap()].
-func NewWithConfig(config Config) *PocketBase {
+func NewWithConfig(config *Config) *PocketBase {
+	if config == nil {
+		panic("missing config")
+	}
+
 	// initialize a default data directory based on the executable baseDir
 	if config.DefaultDataDir == "" {
 		baseDir, _ := inspectRuntime()
@@ -105,20 +116,19 @@ func NewWithConfig(config Config) *PocketBase {
 	pb.eagerParseFlags(config)
 
 	// initialize the app instance
-	pb.appWrapper = &appWrapper{core.NewBaseApp(
-		pb.dataDirFlag,
-		pb.encryptionEnvFlag,
-		pb.debugFlag,
-	)}
+	pb.appWrapper = &appWrapper{core.NewBaseApp(&core.BaseAppConfig{
+		DataDir:          pb.dataDirFlag,
+		EncryptionEnv:    pb.encryptionEnvFlag,
+		IsDebug:          pb.debugFlag,
+		DataMaxOpenConns: config.DataMaxOpenConns,
+		DataMaxIdleConns: config.DataMaxIdleConns,
+		LogsMaxOpenConns: config.LogsMaxOpenConns,
+		LogsMaxIdleConns: config.LogsMaxIdleConns,
+	})}
 
 	// hide the default help command (allow only `--help` flag)
 	pb.RootCmd.SetHelpCommand(&cobra.Command{Hidden: true})
 
-	// hook the bootstrap process
-	pb.RootCmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
-		return pb.Bootstrap()
-	}
-
 	return pb
 }
 
@@ -138,6 +148,12 @@ func (pb *PocketBase) Start() error {
 // This method differs from pb.Start() by not registering the default
 // system commands!
 func (pb *PocketBase) Execute() error {
+	if !pb.skipBootstrap() {
+		if err := pb.Bootstrap(); err != nil {
+			return err
+		}
+	}
+
 	var wg sync.WaitGroup
 
 	wg.Add(1)
@@ -171,7 +187,7 @@ func (pb *PocketBase) onTerminate() error {
 
 // eagerParseFlags parses the global app flags before calling pb.RootCmd.Execute().
 // so we can have all PocketBase flags ready for use on initialization.
-func (pb *PocketBase) eagerParseFlags(config Config) error {
+func (pb *PocketBase) eagerParseFlags(config *Config) error {
 	pb.RootCmd.PersistentFlags().StringVar(
 		&pb.dataDirFlag,
 		"dir",
@@ -196,7 +212,50 @@ func (pb *PocketBase) eagerParseFlags(config Config) error {
 	return pb.RootCmd.ParseFlags(os.Args[1:])
 }
 
-// tries to find the base executable directory and how it was run
+// skipBootstrap eagerly checks if the app should skip the bootstap process:
+// - already bootstrapped
+// - is unknown command
+// - is the default help command
+// - is the default version command
+//
+// https://github.com/pocketbase/pocketbase/issues/404
+// https://github.com/pocketbase/pocketbase/discussions/1267
+func (pb *PocketBase) skipBootstrap() bool {
+	flags := []string{
+		"-h",
+		"--help",
+		"-v",
+		"--version",
+	}
+
+	if pb.IsBootstrapped() {
+		return true // already bootstrapped
+	}
+
+	cmd, _, err := pb.RootCmd.Find(os.Args[1:])
+	if err != nil {
+		return true // unknown command
+	}
+
+	for _, arg := range os.Args {
+		if !list.ExistInSlice(arg, flags) {
+			continue
+		}
+
+		// ensure that there is no user defined flag with the same name/shorthand
+		trimmed := strings.TrimLeft(arg, "-")
+		if len(trimmed) > 1 && cmd.Flags().Lookup(trimmed) == nil {
+			return true
+		}
+		if len(trimmed) == 1 && cmd.Flags().ShorthandLookup(trimmed) == nil {
+			return true
+		}
+	}
+
+	return false
+}
+
+// inspectRuntime tries to find the base executable directory and how it was run.
 func inspectRuntime() (baseDir string, withGoRun bool) {
 	if strings.HasPrefix(os.Args[0], os.TempDir()) {
 		// probably ran with go run

pocketbase_test.go

@@ -2,20 +2,23 @@ package pocketbase
 
 import (
 	"os"
+	"path/filepath"
 	"testing"
+
+	"github.com/spf13/cobra"
 )
 
 func TestNew(t *testing.T) {
 	// copy os.Args
-	originalArgs := []string{}
+	originalArgs := make([]string, len(os.Args))
 	copy(originalArgs, os.Args)
 	defer func() {
 		// restore os.Args
-		copy(os.Args, originalArgs)
+		os.Args = originalArgs
 	}()
 
 	// change os.Args
-	os.Args = os.Args[0:1]
+	os.Args = os.Args[:1]
 	os.Args = append(
 		os.Args,
 		"--dir=test_dir",
@@ -51,7 +54,7 @@ func TestNew(t *testing.T) {
 }
 
 func TestNewWithConfig(t *testing.T) {
-	app := NewWithConfig(Config{
+	app := NewWithConfig(&Config{
 		DefaultDebug:         true,
 		DefaultDataDir:       "test_dir",
 		DefaultEncryptionEnv: "test_encryption_env",
@@ -89,15 +92,15 @@ func TestNewWithConfig(t *testing.T) {
 
 func TestNewWithConfigAndFlags(t *testing.T) {
 	// copy os.Args
-	originalArgs := []string{}
+	originalArgs := make([]string, len(os.Args))
 	copy(originalArgs, os.Args)
 	defer func() {
 		// restore os.Args
-		copy(os.Args, originalArgs)
+		os.Args = originalArgs
 	}()
 
 	// change os.Args
-	os.Args = os.Args[0:1]
+	os.Args = os.Args[:1]
 	os.Args = append(
 		os.Args,
 		"--dir=test_dir_flag",
@@ -105,7 +108,7 @@ func TestNewWithConfigAndFlags(t *testing.T) {
 		"--debug=false",
 	)
 
-	app := NewWithConfig(Config{
+	app := NewWithConfig(&Config{
 		DefaultDebug:         true,
 		DefaultDataDir:       "test_dir",
 		DefaultEncryptionEnv: "test_encryption_env",
@@ -140,3 +143,82 @@ func TestNewWithConfigAndFlags(t *testing.T) {
 		t.Fatal("Expected app.IsDebug() false, got true")
 	}
 }
+
+func TestSkipBootstrap(t *testing.T) {
+	// copy os.Args
+	originalArgs := make([]string, len(os.Args))
+	copy(originalArgs, os.Args)
+	defer func() {
+		// restore os.Args
+		os.Args = originalArgs
+	}()
+
+	tempDir := filepath.Join(os.TempDir(), "temp_pb_data")
+	defer os.RemoveAll(tempDir)
+
+	// already bootstrapped
+	app0 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+	app0.Bootstrap()
+	if v := app0.skipBootstrap(); !v {
+		t.Fatal("[bootstrapped] Expected true, got false")
+	}
+
+	// unknown command
+	os.Args = os.Args[:1]
+	os.Args = append(os.Args, "demo")
+	app1 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+	app1.RootCmd.AddCommand(&cobra.Command{Use: "test"})
+	if v := app1.skipBootstrap(); !v {
+		t.Fatal("[unknown] Expected true, got false")
+	}
+
+	// default flags
+	flagScenarios := []struct {
+		name  string
+		short string
+	}{
+		{"help", "h"},
+		{"version", "v"},
+	}
+
+	for _, s := range flagScenarios {
+		// base flag
+		os.Args = os.Args[:1]
+		os.Args = append(os.Args, "--"+s.name)
+		app1 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+		if v := app1.skipBootstrap(); !v {
+			t.Fatalf("[--%s] Expected true, got false", s.name)
+		}
+
+		// short flag
+		os.Args = os.Args[:1]
+		os.Args = append(os.Args, "-"+s.short)
+		app2 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+		if v := app2.skipBootstrap(); !v {
+			t.Fatalf("[-%s] Expected true, got false", s.short)
+		}
+
+		customCmd := &cobra.Command{Use: "custom"}
+		customCmd.PersistentFlags().BoolP(s.name, s.short, false, "")
+
+		// base flag in custom command
+		os.Args = os.Args[:1]
+		os.Args = append(os.Args, "custom")
+		os.Args = append(os.Args, "--"+s.name)
+		app3 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+		app3.RootCmd.AddCommand(customCmd)
+		if v := app3.skipBootstrap(); v {
+			t.Fatalf("[--%s custom] Expected false, got true", s.name)
+		}
+
+		// short flag in custom command
+		os.Args = os.Args[:1]
+		os.Args = append(os.Args, "custom")
+		os.Args = append(os.Args, "-"+s.short)
+		app4 := NewWithConfig(&Config{DefaultDataDir: tempDir})
+		app4.RootCmd.AddCommand(customCmd)
+		if v := app4.skipBootstrap(); v {
+			t.Fatalf("[-%s custom] Expected false, got true", s.short)
+		}
+	}
+}

resolvers/multi_match_subquery.go

@@ -0,0 +1,70 @@
+package resolvers
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/pocketbase/dbx"
+)
+
+var _ dbx.Expression = (*multiMatchSubquery)(nil)
+
+// join defines the specification for a single SQL JOIN clause.
+type join struct {
+	tableName  string
+	tableAlias string
+	on         dbx.Expression
+}
+
+// multiMatchSubquery defines a record multi-match subquery expression.
+type multiMatchSubquery struct {
+	baseTableAlias  string
+	fromTableName   string
+	fromTableAlias  string
+	valueIdentifier string
+	joins           []*join
+	params          dbx.Params
+}
+
+// Build converts the expression into a SQL fragment.
+//
+// Implements [dbx.Expression] interface.
+func (m *multiMatchSubquery) Build(db *dbx.DB, params dbx.Params) string {
+	if m.baseTableAlias == "" || m.fromTableName == "" || m.fromTableAlias == "" {
+		return "0=1"
+	}
+
+	if params == nil {
+		params = m.params
+	} else {
+		// merge by updating the parent params
+		for k, v := range m.params {
+			params[k] = v
+		}
+	}
+
+	var mergedJoins strings.Builder
+	for i, j := range m.joins {
+		if i > 0 {
+			mergedJoins.WriteString(" ")
+		}
+		mergedJoins.WriteString("LEFT JOIN ")
+		mergedJoins.WriteString(db.QuoteTableName(j.tableName))
+		mergedJoins.WriteString(" ")
+		mergedJoins.WriteString(db.QuoteTableName(j.tableAlias))
+		if j.on != nil {
+			mergedJoins.WriteString(" ON ")
+			mergedJoins.WriteString(j.on.Build(db, params))
+		}
+	}
+
+	return fmt.Sprintf(
+		`SELECT %s as [[multiMatchValue]] FROM %s %s %s WHERE %s = %s`,
+		db.QuoteColumnName(m.valueIdentifier),
+		db.QuoteTableName(m.fromTableName),
+		db.QuoteTableName(m.fromTableAlias),
+		mergedJoins.String(),
+		db.QuoteColumnName(m.fromTableAlias+".id"),
+		db.QuoteColumnName(m.baseTableAlias+".id"),
+	)
+}

resolvers/record_field_resolve_runner.go

@@ -0,0 +1,589 @@
+package resolvers
+
+import (
+	"encoding/json"
+	"fmt"
+	"strconv"
+	"strings"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/models"
+	"github.com/pocketbase/pocketbase/models/schema"
+	"github.com/pocketbase/pocketbase/tools/inflector"
+	"github.com/pocketbase/pocketbase/tools/list"
+	"github.com/pocketbase/pocketbase/tools/search"
+	"github.com/pocketbase/pocketbase/tools/security"
+)
+
+// parseAndRun starts a new one-off RecordFieldResolver.Resolve execution.
+func parseAndRun(fieldName string, resolver *RecordFieldResolver) (*search.ResolverResult, error) {
+	r := &runner{
+		fieldName: fieldName,
+		resolver:  resolver,
+	}
+
+	return r.run()
+}
+
+type runner struct {
+	used      bool                 // indicates whether the runner was already executed
+	resolver  *RecordFieldResolver // resolver is the shared expression fields resolver
+	fieldName string               // the name of the single field expression the runner is responsible for
+
+	// shared processing state
+	// ---------------------------------------------------------------
+	activeProps                []string            // holds the active props that remains to be processed
+	activeCollectionName       string              // the last used collection name
+	activeTableAlias           string              // the last used table alias
+	allowHiddenFields          bool                // indicates whether hidden fields (eg. email) should be allowed without extra conditions
+	nullifyMisingField         bool                // indicating whether to return null on missing field or return an error
+	withMultiMatch             bool                // indicates whether to attach a multiMatchSubquery condition to the ResolverResult
+	multiMatchActiveTableAlias string              // the last used multi-match table alias
+	multiMatch                 *multiMatchSubquery // the multi-match subquery expression generated from the fieldName
+}
+
+func (r *runner) run() (*search.ResolverResult, error) {
+	if r.used {
+		return nil, fmt.Errorf("the runner was already used")
+	}
+
+	if len(r.resolver.allowedFields) > 0 && !list.ExistInSliceWithRegex(r.fieldName, r.resolver.allowedFields) {
+		return nil, fmt.Errorf("failed to resolve field %q", r.fieldName)
+	}
+
+	defer func() {
+		r.used = true
+	}()
+
+	r.prepare()
+
+	// check for @collection field (aka. non-relational join)
+	// must be in the format "@collection.COLLECTION_NAME.FIELD[.FIELD2....]"
+	if r.activeProps[0] == "@collection" {
+		return r.processCollectionField()
+	}
+
+	if r.activeProps[0] == "@request" {
+		if r.resolver.requestData == nil {
+			return &search.ResolverResult{Identifier: "NULL"}, nil
+		}
+
+		if strings.HasPrefix(r.fieldName, "@request.auth.") {
+			return r.processRequestAuthField()
+		}
+
+		if strings.HasPrefix(r.fieldName, "@request.data.") && len(r.activeProps) > 2 {
+			name, modifier, err := splitModifier(r.activeProps[2])
+			if err != nil {
+				return nil, err
+			}
+
+			dataField := r.resolver.baseCollection.Schema.GetFieldByName(name)
+			if dataField == nil {
+				return r.resolver.resolveStaticRequestField(r.activeProps[1:]...)
+			}
+
+			dataField.InitOptions()
+
+			// check for data relation field
+			if dataField.Type == schema.FieldTypeRelation && len(r.activeProps) > 3 {
+				return r.processRequestDataRelationField(dataField)
+			}
+
+			// check for select:each field
+			if modifier == eachModifier && dataField.Type == schema.FieldTypeSelect && len(r.activeProps) == 3 {
+				return r.processRequestDataSelectEachModifier(dataField)
+			}
+
+			// check for data arrayble fields ":length" modifier
+			if modifier == lengthModifier && list.ExistInSlice(dataField.Type, schema.ArraybleFieldTypes()) && len(r.activeProps) == 3 {
+				return r.processRequestDataLengthModifier(dataField)
+			}
+		}
+
+		// some other @request.* static field
+		return r.resolver.resolveStaticRequestField(r.activeProps[1:]...)
+	}
+
+	// regular field
+	return r.processActiveProps()
+}
+
+func (r *runner) prepare() {
+	r.activeProps = strings.Split(r.fieldName, ".")
+
+	r.activeCollectionName = r.resolver.baseCollection.Name
+	r.activeTableAlias = inflector.Columnify(r.activeCollectionName)
+
+	r.allowHiddenFields = r.resolver.allowHiddenFields
+	// always allow hidden fields since the @.* filter is a system one
+	if r.activeProps[0] == "@collection" || r.activeProps[0] == "@request" {
+		r.allowHiddenFields = true
+	}
+
+	// enable the ignore flag for missing @request.* fields for backward
+	// compatibility and consistency with all @request.* filter fields and types
+	r.nullifyMisingField = r.activeProps[0] == "@request"
+
+	// prepare a multi-match subquery
+	r.multiMatch = &multiMatchSubquery{
+		baseTableAlias: r.activeTableAlias,
+		params:         dbx.Params{},
+	}
+	r.multiMatch.fromTableName = inflector.Columnify(r.activeCollectionName)
+	r.multiMatch.fromTableAlias = "__mm_" + r.activeTableAlias
+	r.multiMatchActiveTableAlias = r.multiMatch.fromTableAlias
+	r.withMultiMatch = false
+}
+
+func (r *runner) processCollectionField() (*search.ResolverResult, error) {
+	if len(r.activeProps) < 3 {
+		return nil, fmt.Errorf("invalid @collection field path in %q", r.fieldName)
+	}
+
+	collection, err := r.resolver.loadCollection(r.activeProps[1])
+	if err != nil {
+		return nil, fmt.Errorf("failed to load collection %q from field path %q", r.activeProps[1], r.fieldName)
+	}
+
+	r.activeCollectionName = collection.Name
+	r.activeTableAlias = inflector.Columnify("__collection_" + r.activeCollectionName)
+
+	r.withMultiMatch = true
+
+	// join the collection to the main query
+	r.resolver.registerJoin(inflector.Columnify(collection.Name), r.activeTableAlias, nil)
+
+	// join the collection to the multi-match subquery
+	r.multiMatchActiveTableAlias = "__mm" + r.activeTableAlias
+	r.multiMatch.joins = append(r.multiMatch.joins, &join{
+		tableName:  inflector.Columnify(collection.Name),
+		tableAlias: r.multiMatchActiveTableAlias,
+	})
+
+	// leave only the collection fields
+	// aka. @collection.someCollection.fieldA.fieldB -> fieldA.fieldB
+	r.activeProps = r.activeProps[2:]
+
+	return r.processActiveProps()
+}
+
+func (r *runner) processRequestAuthField() (*search.ResolverResult, error) {
+	// plain auth field
+	// ---
+	if list.ExistInSlice(r.fieldName, plainRequestAuthFields) {
+		return r.resolver.resolveStaticRequestField(r.activeProps[1:]...)
+	}
+
+	// resolve the auth collection field
+	// ---
+	if r.resolver.requestData == nil || r.resolver.requestData.AuthRecord == nil || r.resolver.requestData.AuthRecord.Collection() == nil {
+		return &search.ResolverResult{Identifier: "NULL"}, nil
+	}
+
+	collection := r.resolver.requestData.AuthRecord.Collection()
+	r.resolver.loadedCollections = append(r.resolver.loadedCollections, collection)
+
+	r.activeCollectionName = collection.Name
+	r.activeTableAlias = "__auth_" + inflector.Columnify(r.activeCollectionName)
+
+	// join the auth collection to the main query
+	r.resolver.registerJoin(
+		inflector.Columnify(r.activeCollectionName),
+		r.activeTableAlias,
+		dbx.HashExp{
+			// aka. __auth_users.id = :userId
+			(r.activeTableAlias + ".id"): r.resolver.requestData.AuthRecord.Id,
+		},
+	)
+
+	// join the auth collection to the multi-match subquery
+	r.multiMatchActiveTableAlias = "__mm_" + r.activeTableAlias
+	r.multiMatch.joins = append(
+		r.multiMatch.joins,
+		&join{
+			tableName:  inflector.Columnify(r.activeCollectionName),
+			tableAlias: r.multiMatchActiveTableAlias,
+			on: dbx.HashExp{
+				(r.multiMatchActiveTableAlias + ".id"): r.resolver.requestData.AuthRecord.Id,
+			},
+		},
+	)
+
+	// leave only the auth relation fields
+	// aka. @request.auth.fieldA.fieldB -> fieldA.fieldB
+	r.activeProps = r.activeProps[2:]
+
+	return r.processActiveProps()
+}
+
+func (r *runner) processRequestDataLengthModifier(dataField *schema.SchemaField) (*search.ResolverResult, error) {
+	dataItems := list.ToUniqueStringSlice(r.resolver.requestData.Data[dataField.Name])
+
+	result := &search.ResolverResult{
+		Identifier: fmt.Sprintf("%d", len(dataItems)),
+	}
+
+	return result, nil
+}
+
+func (r *runner) processRequestDataSelectEachModifier(dataField *schema.SchemaField) (*search.ResolverResult, error) {
+	options, ok := dataField.Options.(*schema.SelectOptions)
+	if !ok {
+		return nil, fmt.Errorf("failed to initialize field %q options", dataField.Name)
+	}
+
+	dataItems := list.ToUniqueStringSlice(r.resolver.requestData.Data[dataField.Name])
+	rawJson, err := json.Marshal(dataItems)
+	if err != nil {
+		return nil, fmt.Errorf("cannot marshalize the data select item for field %q", r.activeProps[2])
+	}
+
+	placeholder := "dataSelect" + security.PseudorandomString(4)
+	cleanFieldName := inflector.Columnify(dataField.Name)
+	jeTable := fmt.Sprintf("json_each({:%s})", placeholder)
+	jeAlias := "__dataSelect_" + cleanFieldName + "_je"
+	r.resolver.registerJoin(jeTable, jeAlias, nil)
+
+	result := &search.ResolverResult{
+		Identifier: fmt.Sprintf("[[%s.value]]", jeAlias),
+		Params:     dbx.Params{placeholder: rawJson},
+	}
+
+	if options.MaxSelect != 1 {
+		r.withMultiMatch = true
+	}
+
+	if r.withMultiMatch {
+		placeholder2 := "mm" + placeholder
+		jeTable2 := fmt.Sprintf("json_each({:%s})", placeholder2)
+		jeAlias2 := "__mm" + jeAlias
+
+		r.multiMatch.joins = append(r.multiMatch.joins, &join{
+			tableName:  jeTable2,
+			tableAlias: jeAlias2,
+		})
+		r.multiMatch.params[placeholder2] = rawJson
+		r.multiMatch.valueIdentifier = fmt.Sprintf("[[%s.value]]", jeAlias2)
+
+		result.MultiMatchSubQuery = r.multiMatch
+	}
+
+	return result, nil
+}
+
+func (r *runner) processRequestDataRelationField(dataField *schema.SchemaField) (*search.ResolverResult, error) {
+	options, ok := dataField.Options.(*schema.RelationOptions)
+	if !ok {
+		return nil, fmt.Errorf("failed to initialize data field %q options", dataField.Name)
+	}
+
+	dataRelCollection, err := r.resolver.loadCollection(options.CollectionId)
+	if err != nil {
+		return nil, fmt.Errorf("failed to load collection %q from data field %q", options.CollectionId, dataField.Name)
+	}
+
+	var dataRelIds []string
+	if r.resolver.requestData != nil && len(r.resolver.requestData.Data) != 0 {
+		dataRelIds = list.ToUniqueStringSlice(r.resolver.requestData.Data[dataField.Name])
+	}
+	if len(dataRelIds) == 0 {
+		return &search.ResolverResult{Identifier: "NULL"}, nil
+	}
+
+	r.activeCollectionName = dataRelCollection.Name
+	r.activeTableAlias = inflector.Columnify("__data_" + dataRelCollection.Name)
+
+	// join the data rel collection to the main collection
+	r.resolver.registerJoin(
+		inflector.Columnify(r.activeCollectionName),
+		r.activeTableAlias,
+		dbx.In(
+			fmt.Sprintf("[[%s.id]]", inflector.Columnify(r.activeTableAlias)),
+			list.ToInterfaceSlice(dataRelIds)...,
+		),
+	)
+
+	if options.MaxSelect == nil || *options.MaxSelect != 1 {
+		r.withMultiMatch = true
+	}
+
+	// join the data rel collection to the multi-match subquery
+	r.multiMatchActiveTableAlias = inflector.Columnify("__data_mm_" + dataRelCollection.Name)
+	r.multiMatch.joins = append(
+		r.multiMatch.joins,
+		&join{
+			tableName:  inflector.Columnify(r.activeCollectionName),
+			tableAlias: r.multiMatchActiveTableAlias,
+			on:         dbx.In(r.multiMatchActiveTableAlias+".id", list.ToInterfaceSlice(dataRelIds)...),
+		},
+	)
+
+	// leave only the data relation fields
+	// aka. @request.data.someRel.fieldA.fieldB -> fieldA.fieldB
+	r.activeProps = r.activeProps[3:]
+
+	return r.processActiveProps()
+}
+
+func (r *runner) processActiveProps() (*search.ResolverResult, error) {
+	totalProps := len(r.activeProps)
+
+	for i, prop := range r.activeProps {
+		collection, err := r.resolver.loadCollection(r.activeCollectionName)
+		if err != nil {
+			return nil, fmt.Errorf("failed to resolve field %q", prop)
+		}
+
+		// last prop
+		if i == totalProps-1 {
+			// system field, aka. internal model prop
+			// (always available but not part of the collection schema)
+			// -------------------------------------------------------
+			if list.ExistInSlice(prop, resolvableSystemFieldNames(collection)) {
+				result := &search.ResolverResult{
+					Identifier: fmt.Sprintf("[[%s.%s]]", r.activeTableAlias, inflector.Columnify(prop)),
+				}
+
+				// allow querying only auth records with emails marked as public
+				if prop == schema.FieldNameEmail && !r.allowHiddenFields {
+					result.AfterBuild = func(expr dbx.Expression) dbx.Expression {
+						return dbx.And(expr, dbx.NewExp(fmt.Sprintf(
+							"[[%s.%s]] = TRUE",
+							r.activeTableAlias,
+							schema.FieldNameEmailVisibility,
+						)))
+					}
+				}
+
+				if r.withMultiMatch {
+					r.multiMatch.valueIdentifier = fmt.Sprintf("[[%s.%s]]", r.multiMatchActiveTableAlias, inflector.Columnify(prop))
+					result.MultiMatchSubQuery = r.multiMatch
+				}
+
+				return result, nil
+			}
+
+			name, modifier, err := splitModifier(prop)
+			if err != nil {
+				return nil, err
+			}
+
+			field := collection.Schema.GetFieldByName(name)
+			if field == nil {
+				if r.nullifyMisingField {
+					return &search.ResolverResult{Identifier: "NULL"}, nil
+				}
+				return nil, fmt.Errorf("unknown field %q", name)
+			}
+
+			cleanFieldName := inflector.Columnify(field.Name)
+
+			// arrayble fields ":length" modifier
+			// -------------------------------------------------------
+			if modifier == lengthModifier && list.ExistInSlice(field.Type, schema.ArraybleFieldTypes()) {
+				jePair := r.activeTableAlias + "." + cleanFieldName
+
+				result := &search.ResolverResult{
+					Identifier: jsonArrayLength(jePair),
+				}
+
+				if r.withMultiMatch {
+					jePair2 := r.multiMatchActiveTableAlias + "." + cleanFieldName
+					r.multiMatch.valueIdentifier = jsonArrayLength(jePair2)
+					result.MultiMatchSubQuery = r.multiMatch
+				}
+
+				return result, nil
+			}
+
+			// select field with ":each" modifier
+			// -------------------------------------------------------
+			if field.Type == schema.FieldTypeSelect && modifier == eachModifier {
+				jePair := r.activeTableAlias + "." + cleanFieldName
+				jeAlias := r.activeTableAlias + "_" + cleanFieldName + "_je"
+				r.resolver.registerJoin(jsonEach(jePair), jeAlias, nil)
+
+				result := &search.ResolverResult{
+					Identifier: fmt.Sprintf("[[%s.value]]", jeAlias),
+				}
+
+				field.InitOptions()
+				options, ok := field.Options.(*schema.SelectOptions)
+				if !ok {
+					return nil, fmt.Errorf("failed to initialize field %q options", prop)
+				}
+
+				if options.MaxSelect != 1 {
+					r.withMultiMatch = true
+				}
+
+				if r.withMultiMatch {
+					jePair2 := r.multiMatchActiveTableAlias + "." + cleanFieldName
+					jeAlias2 := r.multiMatchActiveTableAlias + "_" + cleanFieldName + "_je"
+
+					r.multiMatch.joins = append(r.multiMatch.joins, &join{
+						tableName:  jsonEach(jePair2),
+						tableAlias: jeAlias2,
+					})
+					r.multiMatch.valueIdentifier = fmt.Sprintf("[[%s.value]]", jeAlias2)
+
+					result.MultiMatchSubQuery = r.multiMatch
+				}
+
+				return result, nil
+			}
+
+			// default
+			// -------------------------------------------------------
+			result := &search.ResolverResult{
+				Identifier: fmt.Sprintf("[[%s.%s]]", r.activeTableAlias, cleanFieldName),
+			}
+
+			if r.withMultiMatch {
+				r.multiMatch.valueIdentifier = fmt.Sprintf("[[%s.%s]]", r.multiMatchActiveTableAlias, cleanFieldName)
+				result.MultiMatchSubQuery = r.multiMatch
+			}
+
+			return result, nil
+		}
+
+		field := collection.Schema.GetFieldByName(prop)
+		if field == nil {
+			if r.nullifyMisingField {
+				return &search.ResolverResult{Identifier: "NULL"}, nil
+			}
+			return nil, fmt.Errorf("unknown field %q", prop)
+		}
+
+		// check if it is a json field
+		if field.Type == schema.FieldTypeJson {
+			var jsonPath strings.Builder
+			jsonPath.WriteString("$")
+			for _, p := range r.activeProps[i+1:] {
+				if _, err := strconv.Atoi(p); err == nil {
+					jsonPath.WriteString("[")
+					jsonPath.WriteString(inflector.Columnify(p))
+					jsonPath.WriteString("]")
+				} else {
+					jsonPath.WriteString(".")
+					jsonPath.WriteString(inflector.Columnify(p))
+				}
+			}
+
+			result := &search.ResolverResult{
+				Identifier: fmt.Sprintf(
+					"JSON_EXTRACT([[%s.%s]], '%s')",
+					r.activeTableAlias,
+					inflector.Columnify(prop),
+					jsonPath.String(),
+				),
+			}
+
+			if r.withMultiMatch {
+				r.multiMatch.valueIdentifier = fmt.Sprintf(
+					"JSON_EXTRACT([[%s.%s]], '%s')",
+					r.multiMatchActiveTableAlias,
+					inflector.Columnify(prop),
+					jsonPath.String(),
+				)
+				result.MultiMatchSubQuery = r.multiMatch
+			}
+
+			return result, nil
+		}
+
+		// check if it is a relation field
+		if field.Type != schema.FieldTypeRelation {
+			return nil, fmt.Errorf("field %q is not a valid relation", prop)
+		}
+
+		// join the relation to the main query
+		// ---
+		field.InitOptions()
+		options, ok := field.Options.(*schema.RelationOptions)
+		if !ok {
+			return nil, fmt.Errorf("failed to initialize field %q options", prop)
+		}
+
+		relCollection, relErr := r.resolver.loadCollection(options.CollectionId)
+		if relErr != nil {
+			return nil, fmt.Errorf("failed to find field %q collection", prop)
+		}
+
+		cleanFieldName := inflector.Columnify(field.Name)
+		newCollectionName := relCollection.Name
+		newTableAlias := r.activeTableAlias + "_" + cleanFieldName
+
+		jeAlias := r.activeTableAlias + "_" + cleanFieldName + "_je"
+		jePair := r.activeTableAlias + "." + cleanFieldName
+		r.resolver.registerJoin(jsonEach(jePair), jeAlias, nil)
+		r.resolver.registerJoin(
+			inflector.Columnify(newCollectionName),
+			newTableAlias,
+			dbx.NewExp(fmt.Sprintf("[[%s.id]] = [[%s.value]]", newTableAlias, jeAlias)),
+		)
+		r.activeCollectionName = newCollectionName
+		r.activeTableAlias = newTableAlias
+		// ---
+
+		// join the relation to the multi-match subquery
+		// ---
+		if options.MaxSelect == nil || *options.MaxSelect != 1 {
+			r.withMultiMatch = true
+		}
+
+		newTableAlias2 := r.multiMatchActiveTableAlias + "_" + cleanFieldName
+		jeAlias2 := r.multiMatchActiveTableAlias + "_" + cleanFieldName + "_je"
+		jePair2 := r.multiMatchActiveTableAlias + "." + cleanFieldName
+		r.multiMatchActiveTableAlias = newTableAlias2
+
+		r.multiMatch.joins = append(
+			r.multiMatch.joins,
+			&join{
+				tableName:  jsonEach(jePair2),
+				tableAlias: jeAlias2,
+			},
+			&join{
+				tableName:  inflector.Columnify(newCollectionName),
+				tableAlias: newTableAlias2,
+				on:         dbx.NewExp(fmt.Sprintf("[[%s.id]] = [[%s.value]]", newTableAlias2, jeAlias2)),
+			},
+		)
+		// ---
+	}
+
+	return nil, fmt.Errorf("failed to resolve field %q", r.fieldName)
+}
+
+func jsonArrayLength(tableColumnPair string) string {
+	return fmt.Sprintf(
+		// note: the case is used to normalize value access for single and multiple relations.
+		`json_array_length(CASE WHEN json_valid([[%s]]) THEN [[%s]] ELSE json_array([[%s]]) END)`,
+		tableColumnPair, tableColumnPair, tableColumnPair,
+	)
+}
+
+func jsonEach(tableColumnPair string) string {
+	return fmt.Sprintf(
+		// note: the case is used to normalize value access for single and multiple relations.
+		`json_each(CASE WHEN json_valid([[%s]]) THEN [[%s]] ELSE json_array([[%s]]) END)`,
+		tableColumnPair, tableColumnPair, tableColumnPair,
+	)
+}
+
+func resolvableSystemFieldNames(collection *models.Collection) []string {
+	result := schema.BaseModelFieldNames()
+
+	if collection.IsAuth() {
+		result = append(
+			result,
+			schema.FieldNameUsername,
+			schema.FieldNameVerified,
+			schema.FieldNameEmailVisibility,
+			schema.FieldNameEmail,
+		)
+	}
+
+	return result
+}

resolvers/record_field_resolver.go

@@ -10,18 +10,20 @@ import (
 	"github.com/pocketbase/pocketbase/daos"
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/models/schema"
-	"github.com/pocketbase/pocketbase/tools/inflector"
-	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/search"
 	"github.com/pocketbase/pocketbase/tools/security"
 	"github.com/spf13/cast"
 )
 
-// ensure that `search.FieldResolver` interface is implemented
-var _ search.FieldResolver = (*RecordFieldResolver)(nil)
+// filter modifiers
+const (
+	eachModifier   string = "each"
+	issetModifier  string = "isset"
+	lengthModifier string = "length"
+)
 
 // list of auth filter fields that don't require join with the auth
-// collection or any other extra checks to be resolved
+// collection or any other extra checks to be resolved.
 var plainRequestAuthFields = []string{
 	"@request.auth." + schema.FieldNameId,
 	"@request.auth." + schema.FieldNameCollectionId,
@@ -34,32 +36,28 @@ var plainRequestAuthFields = []string{
 	"@request.auth." + schema.FieldNameUpdated,
 }
 
-type join struct {
-	id    string
-	table string
-	on    dbx.Expression
-}
+// ensure that `search.FieldResolver` interface is implemented
+var _ search.FieldResolver = (*RecordFieldResolver)(nil)
 
 // RecordFieldResolver defines a custom search resolver struct for
 // managing Record model search fields.
 //
 // Usually used together with `search.Provider`. Example:
-//	resolver := resolvers.NewRecordFieldResolver(
-//		app.Dao(),
-//		myCollection,
-//		&models.RequestData{...},
-//		true,
-//	)
-//	provider := search.NewProvider(resolver)
-//	...
+//  resolver := resolvers.NewRecordFieldResolver(
+//      app.Dao(),
+//      myCollection,
+//      &models.RequestData{...},
+//      true,
+//  )
+//  provider := search.NewProvider(resolver)
+//  ...
 type RecordFieldResolver struct {
 	dao               *daos.Dao
 	baseCollection    *models.Collection
 	allowHiddenFields bool
 	allowedFields     []string
 	loadedCollections []*models.Collection
-	joins             []join // we cannot use a map because the insertion order is not preserved
-	exprs             []dbx.Expression
+	joins             []*join // we cannot use a map because the insertion order is not preserved
 	requestData       *models.RequestData
 	staticRequestData map[string]any
 }
@@ -76,20 +74,18 @@ func NewRecordFieldResolver(
 		baseCollection:    baseCollection,
 		requestData:       requestData,
 		allowHiddenFields: allowHiddenFields,
-		joins:             []join{},
-		exprs:             []dbx.Expression{},
+		joins:             []*join{},
 		loadedCollections: []*models.Collection{baseCollection},
 		allowedFields: []string{
-			`^\w+[\w\.]*$`,
+			`^\w+[\w\.\:]*$`,
 			`^\@request\.method$`,
-			`^\@request\.auth\.\w+[\w\.]*$`,
-			`^\@request\.data\.\w+[\w\.]*$`,
-			`^\@request\.query\.\w+[\w\.]*$`,
-			`^\@collection\.\w+\.\w+[\w\.]*$`,
+			`^\@request\.auth\.[\w\.\:]*\w+$`,
+			`^\@request\.data\.[\w\.\:]*\w+$`,
+			`^\@request\.query\.[\w\.\:]*\w+$`,
+			`^\@collection\.\w+\.[\w\.\:]*\w+$`,
 		},
 	}
 
-	// @todo remove after IN operator and multi-match filter enhancements
 	r.staticRequestData = map[string]any{}
 	if r.requestData != nil {
 		r.staticRequestData["method"] = r.requestData.Method
@@ -115,13 +111,10 @@ func (r *RecordFieldResolver) UpdateQuery(query *dbx.SelectQuery) error {
 		query.Distinct(true)
 
 		for _, join := range r.joins {
-			query.LeftJoin(join.table, join.on)
-		}
-	}
-
-	for _, expr := range r.exprs {
-		if expr != nil {
-			query.AndWhere(expr)
+			query.LeftJoin(
+				(join.tableName + " " + join.tableAlias),
+				join.on,
+			)
 		}
 	}
 
@@ -130,225 +123,63 @@ func (r *RecordFieldResolver) UpdateQuery(query *dbx.SelectQuery) error {
 
 // Resolve implements `search.FieldResolver` interface.
 //
-// Example of resolvable field formats:
-//	id
-//	project.screen.status
-//	@request.status
-//	@request.auth.someRelation.name
-//	@collection.product.name
-func (r *RecordFieldResolver) Resolve(fieldName string) (resultName string, placeholderParams dbx.Params, err error) {
-	if len(r.allowedFields) > 0 && !list.ExistInSliceWithRegex(fieldName, r.allowedFields) {
-		return "", nil, fmt.Errorf("Failed to resolve field %q", fieldName)
-	}
-
-	props := strings.Split(fieldName, ".")
-
-	currentCollectionName := r.baseCollection.Name
-	currentTableAlias := inflector.Columnify(currentCollectionName)
-
-	// flag indicating whether to return null on missing field or return on an error
-	nullifyMisingField := false
-
-	allowHiddenFields := r.allowHiddenFields
-
-	// check for @collection field (aka. non-relational join)
-	// must be in the format "@collection.COLLECTION_NAME.FIELD[.FIELD2....]"
-	if props[0] == "@collection" {
-		if len(props) < 3 {
-			return "", nil, fmt.Errorf("Invalid @collection field path in %q.", fieldName)
-		}
-
-		currentCollectionName = props[1]
-		currentTableAlias = inflector.Columnify("__collection_" + currentCollectionName)
-
-		collection, err := r.loadCollection(currentCollectionName)
-		if err != nil {
-			return "", nil, fmt.Errorf("Failed to load collection %q from field path %q.", currentCollectionName, fieldName)
-		}
-
-		// always allow hidden fields since the @collection.* filter is a system one
-		allowHiddenFields = true
-
-		r.registerJoin(inflector.Columnify(collection.Name), currentTableAlias, nil)
-
-		props = props[2:] // leave only the collection fields
-	} else if props[0] == "@request" {
-		if len(props) == 1 {
-			return "", nil, fmt.Errorf("Invalid @request data field path in %q.", fieldName)
-		}
-
-		if r.requestData == nil {
-			return "NULL", nil, nil
-		}
-
-		// plain @request.* field
-		if !strings.HasPrefix(fieldName, "@request.auth.") || list.ExistInSlice(fieldName, plainRequestAuthFields) {
-			return r.resolveStaticRequestField(props[1:]...)
-		}
-
-		// always allow hidden fields since the @request.* filter is a system one
-		allowHiddenFields = true
-
-		// enable the ignore flag for missing @request.auth.* fields
-		// for consistency with @request.data.* and @request.query.*
-		nullifyMisingField = true
-
-		// resolve the auth collection fields
-		// ---
-		if r.requestData == nil || r.requestData.AuthRecord == nil || r.requestData.AuthRecord.Collection() == nil {
-			return "NULL", nil, nil
-		}
-
-		collection := r.requestData.AuthRecord.Collection()
-		r.loadedCollections = append(r.loadedCollections, collection)
-
-		currentCollectionName = collection.Name
-		currentTableAlias = "__auth_" + inflector.Columnify(currentCollectionName)
-
-		authIdParamKey := "auth" + security.PseudorandomString(5)
-		authIdParams := dbx.Params{authIdParamKey: r.requestData.AuthRecord.Id}
-		// ---
-
-		// join the auth collection
-		r.registerJoin(
-			inflector.Columnify(collection.Name),
-			currentTableAlias,
-			dbx.NewExp(fmt.Sprintf(
-				// aka. __auth_users.id = :userId
-				"[[%s.id]] = {:%s}",
-				inflector.Columnify(currentTableAlias),
-				authIdParamKey,
-			), authIdParams),
-		)
-
-		props = props[2:] // leave only the auth relation fields
-	}
-
-	totalProps := len(props)
-
-	for i, prop := range props {
-		collection, err := r.loadCollection(currentCollectionName)
-		if err != nil {
-			return "", nil, fmt.Errorf("Failed to resolve field %q.", prop)
-		}
-
-		systemFieldNames := schema.BaseModelFieldNames()
-		if collection.IsAuth() {
-			systemFieldNames = append(
-				systemFieldNames,
-				schema.FieldNameUsername,
-				schema.FieldNameVerified,
-				schema.FieldNameEmailVisibility,
-				schema.FieldNameEmail,
-			)
-		}
-
-		// internal model prop (always available but not part of the collection schema)
-		if list.ExistInSlice(prop, systemFieldNames) {
-			// allow querying only auth records with emails marked as public
-			if prop == schema.FieldNameEmail && !allowHiddenFields {
-				r.registerExpr(dbx.NewExp(fmt.Sprintf(
-					"[[%s.%s]] = TRUE",
-					currentTableAlias,
-					inflector.Columnify(schema.FieldNameEmailVisibility),
-				)))
-			}
-
-			return fmt.Sprintf("[[%s.%s]]", currentTableAlias, inflector.Columnify(prop)), nil, nil
-		}
-
-		field := collection.Schema.GetFieldByName(prop)
-		if field == nil {
-			if nullifyMisingField {
-				return "NULL", nil, nil
-			}
-
-			return "", nil, fmt.Errorf("Unrecognized field %q.", prop)
-		}
-
-		// last prop
-		if i == totalProps-1 {
-			return fmt.Sprintf("[[%s.%s]]", currentTableAlias, inflector.Columnify(prop)), nil, nil
-		}
-
-		// check if it is a json field
-		if field.Type == schema.FieldTypeJson {
-			var jsonPath strings.Builder
-			jsonPath.WriteString("$")
-			for _, p := range props[i+1:] {
-				if _, err := strconv.Atoi(p); err == nil {
-					jsonPath.WriteString("[")
-					jsonPath.WriteString(inflector.Columnify(p))
-					jsonPath.WriteString("]")
-				} else {
-					jsonPath.WriteString(".")
-					jsonPath.WriteString(inflector.Columnify(p))
-				}
-			}
-			return fmt.Sprintf(
-				"JSON_EXTRACT([[%s.%s]], '%s')",
-				currentTableAlias,
-				inflector.Columnify(prop),
-				jsonPath.String(),
-			), nil, nil
-		}
-
-		// check if it is a relation field
-		if field.Type != schema.FieldTypeRelation {
-			return "", nil, fmt.Errorf("Field %q is not a valid relation.", prop)
-		}
-
-		// auto join the relation
-		// ---
-		field.InitOptions()
-		options, ok := field.Options.(*schema.RelationOptions)
-		if !ok {
-			return "", nil, fmt.Errorf("Failed to initialize field %q options.", prop)
-		}
-
-		relCollection, relErr := r.loadCollection(options.CollectionId)
-		if relErr != nil {
-			return "", nil, fmt.Errorf("Failed to find field %q collection.", prop)
-		}
-
-		cleanFieldName := inflector.Columnify(field.Name)
-		newCollectionName := relCollection.Name
-		newTableAlias := currentTableAlias + "_" + cleanFieldName
-
-		jeTable := currentTableAlias + "_" + cleanFieldName + "_je"
-		jePair := currentTableAlias + "." + cleanFieldName
-
-		r.registerJoin(
-			fmt.Sprintf(
-				// note: the case is used to normalize value access for single and multiple relations.
-				`json_each(CASE WHEN json_valid([[%s]]) THEN [[%s]] ELSE json_array([[%s]]) END)`,
-				jePair, jePair, jePair,
-			),
-			jeTable,
-			nil,
-		)
-		r.registerJoin(
-			inflector.Columnify(newCollectionName),
-			newTableAlias,
-			dbx.NewExp(fmt.Sprintf("[[%s.id]] = [[%s.value]]", newTableAlias, jeTable)),
-		)
-
-		currentCollectionName = newCollectionName
-		currentTableAlias = newTableAlias
-	}
-
-	return "", nil, fmt.Errorf("Failed to resolve field %q.", fieldName)
+// Example of some resolvable fieldName formats:
+//
+//  id
+//  someSelect.each
+//  project.screen.status
+//  @request.status
+//  @request.query.filter
+//  @request.auth.someRelation.name
+//  @request.data.someRelation.name
+//  @request.data.someField
+//  @request.data.someSelect:each
+//  @request.data.someField:isset
+//  @collection.product.name
+func (r *RecordFieldResolver) Resolve(fieldName string) (*search.ResolverResult, error) {
+	return parseAndRun(fieldName, r)
 }
 
-func (r *RecordFieldResolver) resolveStaticRequestField(path ...string) (resultName string, placeholderParams dbx.Params, err error) {
-	// ignore error because requestData is dynamic and some of the
-	// lookup keys may not be defined for the request
-	resultVal, _ := extractNestedMapVal(r.staticRequestData, path...)
+func (r *RecordFieldResolver) resolveStaticRequestField(path ...string) (*search.ResolverResult, error) {
+	if len(path) == 0 {
+		return nil, fmt.Errorf("at least one path key should be provided")
+	}
+
+	lastProp, modifier, err := splitModifier(path[len(path)-1])
+	if err != nil {
+		return nil, err
+	}
+
+	path[len(path)-1] = lastProp
+
+	// extract value
+	resultVal, err := extractNestedMapVal(r.staticRequestData, path...)
+
+	if modifier == issetModifier {
+		if err != nil {
+			return &search.ResolverResult{Identifier: "FALSE"}, nil
+		}
+		return &search.ResolverResult{Identifier: "TRUE"}, nil
+	}
+
+	// note: we are ignoring the error because requestData is dynamic
+	// and some of the lookup keys may not be defined for the request
 
 	switch v := resultVal.(type) {
 	case nil:
-		return "NULL", nil, nil
-	case string, bool, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float32, float64:
+		return &search.ResolverResult{Identifier: "NULL"}, nil
+	case string:
+		// check if it is a number field and explicitly try to cast to
+		// float in case of a numeric string value was used
+		// (this usually the case when the data is from a multipart/form-data request)
+		field := r.baseCollection.Schema.GetFieldByName(path[len(path)-1])
+		if field != nil && field.Type == schema.FieldTypeNumber {
+			if nv, err := strconv.ParseFloat(v, 64); err == nil {
+				resultVal = nv
+			}
+		}
+		// otherwise - no further processing is needed...
+	case bool, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float32, float64:
 		// no further processing is needed...
 	default:
 		// non-plain value
@@ -367,33 +198,11 @@ func (r *RecordFieldResolver) resolveStaticRequestField(path ...string) (resultN
 	}
 
 	placeholder := "f" + security.PseudorandomString(5)
-	name := fmt.Sprintf("{:%s}", placeholder)
-	params := dbx.Params{placeholder: resultVal}
 
-	return name, params, nil
-}
-
-func extractNestedMapVal(m map[string]any, keys ...string) (result any, err error) {
-	var ok bool
-
-	if len(keys) == 0 {
-		return nil, fmt.Errorf("At least one key should be provided.")
-	}
-
-	if result, ok = m[keys[0]]; !ok {
-		return nil, fmt.Errorf("Invalid key path - missing key %q.", keys[0])
-	}
-
-	// end key reached
-	if len(keys) == 1 {
-		return result, nil
-	}
-
-	if m, ok = result.(map[string]any); !ok {
-		return nil, fmt.Errorf("Expected map structure, got %#v.", result)
-	}
-
-	return extractNestedMapVal(m, keys[1:]...)
+	return &search.ResolverResult{
+		Identifier: "{:" + placeholder + "}",
+		Params:     dbx.Params{placeholder: resultVal},
+	}, nil
 }
 
 func (r *RecordFieldResolver) loadCollection(collectionNameOrId string) (*models.Collection, error) {
@@ -415,17 +224,15 @@ func (r *RecordFieldResolver) loadCollection(collectionNameOrId string) (*models
 }
 
 func (r *RecordFieldResolver) registerJoin(tableName string, tableAlias string, on dbx.Expression) {
-	tableExpr := (tableName + " " + tableAlias)
-
-	join := join{
-		id:    tableAlias,
-		table: tableExpr,
-		on:    on,
+	join := &join{
+		tableName:  tableName,
+		tableAlias: tableAlias,
+		on:         on,
 	}
 
 	// replace existing join
 	for i, j := range r.joins {
-		if j.id == join.id {
+		if j.tableAlias == join.tableAlias {
 			r.joins[i] = join
 			return
 		}
@@ -435,6 +242,44 @@ func (r *RecordFieldResolver) registerJoin(tableName string, tableAlias string,
 	r.joins = append(r.joins, join)
 }
 
-func (r *RecordFieldResolver) registerExpr(expr dbx.Expression) {
-	r.exprs = append(r.exprs, expr)
+func extractNestedMapVal(m map[string]any, keys ...string) (any, error) {
+	if len(keys) == 0 {
+		return nil, fmt.Errorf("at least one key should be provided")
+	}
+
+	var result any
+	var ok bool
+
+	if result, ok = m[keys[0]]; !ok {
+		return nil, fmt.Errorf("invalid key path - missing key %q", keys[0])
+	}
+
+	// end key reached
+	if len(keys) == 1 {
+		return result, nil
+	}
+
+	if m, ok = result.(map[string]any); !ok {
+		return nil, fmt.Errorf("expected map, got %#v", result)
+	}
+
+	return extractNestedMapVal(m, keys[1:]...)
+}
+
+func splitModifier(combined string) (string, string, error) {
+	parts := strings.Split(combined, ":")
+
+	if len(parts) != 2 {
+		return combined, "", nil
+	}
+
+	// validate modifier
+	switch parts[1] {
+	case issetModifier,
+		eachModifier,
+		lengthModifier:
+		return parts[0], parts[1], nil
+	}
+
+	return "", "", fmt.Errorf("unknown modifier in %q", combined)
 }

resolvers/record_field_resolver_test.go

@@ -3,12 +3,14 @@ package resolvers_test
 import (
 	"encoding/json"
 	"regexp"
+	"strings"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/models"
 	"github.com/pocketbase/pocketbase/resolvers"
 	"github.com/pocketbase/pocketbase/tests"
 	"github.com/pocketbase/pocketbase/tools/list"
+	"github.com/pocketbase/pocketbase/tools/search"
 )
 
 func TestRecordFieldResolverUpdateQuery(t *testing.T) {
@@ -21,149 +23,307 @@ func TestRecordFieldResolverUpdateQuery(t *testing.T) {
 	}
 
 	requestData := &models.RequestData{
+		Query: map[string]any{
+			"a": nil,
+			"b": 123,
+		},
+		Data: map[string]any{
+			"a":                nil,
+			"b":                123,
+			"number":           10,
+			"select_many":      []string{"optionA", "optionC"},
+			"rel_one":          "test",
+			"rel_many":         []string{"test1", "test2"},
+			"file_one":         "test",
+			"file_many":        []string{"test1", "test2", "test3"},
+			"self_rel_one":     "test",
+			"self_rel_many":    []string{"test1"},
+			"rel_many_cascade": []string{"test1", "test2"},
+		},
 		AuthRecord: authRecord,
 	}
 
 	scenarios := []struct {
 		name               string
 		collectionIdOrName string
-		fields             []string
+		rule               string
 		allowHiddenFields  bool
 		expectQuery        string
 	}{
 		{
-			"missing field",
+			"non relation field (with all default operators)",
 			"demo4",
-			[]string{""},
+			"title = true || title != 'test' || title ~ 'test1' || title !~ '%test2' || title > 1 || title >= 2 || title < 3 || title <= 4",
 			false,
-			"SELECT `demo4`.* FROM `demo4`",
+			"SELECT `demo4`.* FROM `demo4` WHERE (COALESCE([[demo4.title]], '') = COALESCE(1, '') OR COALESCE([[demo4.title]], '') != COALESCE({:TEST}, '') OR [[demo4.title]] LIKE {:TEST} ESCAPE '\\' OR [[demo4.title]] NOT LIKE {:TEST} ESCAPE '\\' OR [[demo4.title]] > {:TEST} OR [[demo4.title]] >= {:TEST} OR [[demo4.title]] < {:TEST} OR [[demo4.title]] <= {:TEST})",
 		},
 		{
-			"non relation field",
+			"non relation field (with all opt/any operators)",
 			"demo4",
-			[]string{"title"},
+			"title ?= true || title ?!= 'test' || title ?~ 'test1' || title ?!~ '%test2' || title ?> 1 || title ?>= 2 || title ?< 3 || title ?<= 4",
 			false,
-			"SELECT `demo4`.* FROM `demo4`",
+			"SELECT `demo4`.* FROM `demo4` WHERE (COALESCE([[demo4.title]], '') = COALESCE(1, '') OR COALESCE([[demo4.title]], '') != COALESCE({:TEST}, '') OR [[demo4.title]] LIKE {:TEST} ESCAPE '\\' OR [[demo4.title]] NOT LIKE {:TEST} ESCAPE '\\' OR [[demo4.title]] > {:TEST} OR [[demo4.title]] >= {:TEST} OR [[demo4.title]] < {:TEST} OR [[demo4.title]] <= {:TEST})",
 		},
 		{
 			"incomplete rel",
 			"demo4",
-			[]string{"self_rel_one"},
+			"self_rel_one > true",
 			false,
-			"SELECT `demo4`.* FROM `demo4`",
+			"SELECT `demo4`.* FROM `demo4` WHERE [[demo4.self_rel_one]] > 1",
 		},
 		{
 			"single rel (self rel)",
 			"demo4",
-			[]string{"self_rel_one.title"},
+			"self_rel_one.title > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]] WHERE [[demo4_self_rel_one.title]] > 1",
 		},
 		{
 			"single rel (other collection)",
 			"demo4",
-			[]string{"rel_one_cascade.title"},
+			"rel_one_cascade.title > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.rel_one_cascade]]) THEN [[demo4.rel_one_cascade]] ELSE json_array([[demo4.rel_one_cascade]]) END) `demo4_rel_one_cascade_je` LEFT JOIN `demo3` `demo4_rel_one_cascade` ON [[demo4_rel_one_cascade.id]] = [[demo4_rel_one_cascade_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.rel_one_cascade]]) THEN [[demo4.rel_one_cascade]] ELSE json_array([[demo4.rel_one_cascade]]) END) `demo4_rel_one_cascade_je` LEFT JOIN `demo3` `demo4_rel_one_cascade` ON [[demo4_rel_one_cascade.id]] = [[demo4_rel_one_cascade_je.value]] WHERE [[demo4_rel_one_cascade.title]] > 1",
 		},
 		{
 			"non-relation field + single rel",
 			"demo4",
-			[]string{"title", "self_rel_one.title"},
+			"title > true || self_rel_one.title > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]] WHERE ([[demo4.title]] > 1 OR [[demo4_self_rel_one.title]] > 1)",
 		},
 		{
-			"nested incomplete rels",
+			"nested incomplete rels (opt/any operator)",
 			"demo4",
-			[]string{"self_rel_many.self_rel_one"},
+			"self_rel_many.self_rel_one ?> true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] WHERE [[demo4_self_rel_many.self_rel_one]] > 1",
 		},
 		{
-			"nested complete rels",
+			"nested incomplete rels (multi-match operator)",
 			"demo4",
-			[]string{"self_rel_many.self_rel_one.title"},
+			"self_rel_many.self_rel_one > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] WHERE (([[demo4_self_rel_many.self_rel_one]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo4_self_rel_many.self_rel_one]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4.self_rel_many]]) THEN [[__mm_demo4.self_rel_many]] ELSE json_array([[__mm_demo4.self_rel_many]]) END) `__mm_demo4_self_rel_many_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many` ON [[__mm_demo4_self_rel_many.id]] = [[__mm_demo4_self_rel_many_je.value]] WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{TEST}} WHERE (NOT ([[TEST.multiMatchValue]] > 1)) OR ([[TEST.multiMatchValue]] IS NULL))))",
 		},
 		{
-			"repeated nested rels",
+			"nested complete rels (opt/any operator)",
 			"demo4",
-			[]string{"self_rel_many.self_rel_one.self_rel_many.self_rel_one.title"},
+			"self_rel_many.self_rel_one.title ?> true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one.self_rel_many]]) THEN [[demo4_self_rel_many_self_rel_one.self_rel_many]] ELSE json_array([[demo4_self_rel_many_self_rel_one.self_rel_many]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many` ON [[demo4_self_rel_many_self_rel_one_self_rel_many.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]] WHERE [[demo4_self_rel_many_self_rel_one.title]] > 1",
 		},
 		{
-			"multiple rels",
+			"nested complete rels (multi-match operator)",
 			"demo4",
-			[]string{"self_rel_many.title", "self_rel_one.onefile"},
+			"self_rel_many.self_rel_one.title > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]]",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]] WHERE (([[demo4_self_rel_many_self_rel_one.title]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo4_self_rel_many_self_rel_one.title]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4.self_rel_many]]) THEN [[__mm_demo4.self_rel_many]] ELSE json_array([[__mm_demo4.self_rel_many]]) END) `__mm_demo4_self_rel_many_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many` ON [[__mm_demo4_self_rel_many.id]] = [[__mm_demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4_self_rel_many.self_rel_one]]) THEN [[__mm_demo4_self_rel_many.self_rel_one]] ELSE json_array([[__mm_demo4_self_rel_many.self_rel_one]]) END) `__mm_demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many_self_rel_one` ON [[__mm_demo4_self_rel_many_self_rel_one.id]] = [[__mm_demo4_self_rel_many_self_rel_one_je.value]] WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{TEST}} WHERE (NOT ([[TEST.multiMatchValue]] > 1)) OR ([[TEST.multiMatchValue]] IS NULL))))",
 		},
 		{
-			"@collection join",
+			"repeated nested rels (opt/any operator)",
 			"demo4",
-			[]string{"@collection.demo1.text", "@collection.demo2.active", "@collection.demo1.file_one"},
+			"self_rel_many.self_rel_one.self_rel_many.self_rel_one.title ?> true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `demo1` `__collection_demo1` LEFT JOIN `demo2` `__collection_demo2`",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one.self_rel_many]]) THEN [[demo4_self_rel_many_self_rel_one.self_rel_many]] ELSE json_array([[demo4_self_rel_many_self_rel_one.self_rel_many]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many` ON [[demo4_self_rel_many_self_rel_one_self_rel_many.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je.value]] WHERE [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.title]] > 1",
+		},
+		{
+			"repeated nested rels (multi-match operator)",
+			"demo4",
+			"self_rel_many.self_rel_one.self_rel_many.self_rel_one.title > true",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one.self_rel_many]]) THEN [[demo4_self_rel_many_self_rel_one.self_rel_many]] ELSE json_array([[demo4_self_rel_many_self_rel_one.self_rel_many]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many` ON [[demo4_self_rel_many_self_rel_one_self_rel_many.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) THEN [[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]] ELSE json_array([[demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) END) `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one` ON [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.id]] = [[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je.value]] WHERE (([[demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.title]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.title]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4.self_rel_many]]) THEN [[__mm_demo4.self_rel_many]] ELSE json_array([[__mm_demo4.self_rel_many]]) END) `__mm_demo4_self_rel_many_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many` ON [[__mm_demo4_self_rel_many.id]] = [[__mm_demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4_self_rel_many.self_rel_one]]) THEN [[__mm_demo4_self_rel_many.self_rel_one]] ELSE json_array([[__mm_demo4_self_rel_many.self_rel_one]]) END) `__mm_demo4_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many_self_rel_one` ON [[__mm_demo4_self_rel_many_self_rel_one.id]] = [[__mm_demo4_self_rel_many_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4_self_rel_many_self_rel_one.self_rel_many]]) THEN [[__mm_demo4_self_rel_many_self_rel_one.self_rel_many]] ELSE json_array([[__mm_demo4_self_rel_many_self_rel_one.self_rel_many]]) END) `__mm_demo4_self_rel_many_self_rel_one_self_rel_many_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many_self_rel_one_self_rel_many` ON [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many.id]] = [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) THEN [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]] ELSE json_array([[__mm_demo4_self_rel_many_self_rel_one_self_rel_many.self_rel_one]]) END) `__mm_demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one` ON [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one.id]] = [[__mm_demo4_self_rel_many_self_rel_one_self_rel_many_self_rel_one_je.value]] WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{TEST}} WHERE (NOT ([[TEST.multiMatchValue]] > 1)) OR ([[TEST.multiMatchValue]] IS NULL))))",
+		},
+		{
+			"multiple rels (opt/any operators)",
+			"demo4",
+			"self_rel_many.title ?= 'test' || self_rel_one.json_object.a ?> true",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]] WHERE (COALESCE([[demo4_self_rel_many.title]], '') = COALESCE({:TEST}, '') OR JSON_EXTRACT([[demo4_self_rel_one.json_object]], '$.a') > 1)",
+		},
+		{
+			"multiple rels (multi-match operators)",
+			"demo4",
+			"self_rel_many.title = 'test' || self_rel_one.json_object.a > true",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_many]]) THEN [[demo4.self_rel_many]] ELSE json_array([[demo4.self_rel_many]]) END) `demo4_self_rel_many_je` LEFT JOIN `demo4` `demo4_self_rel_many` ON [[demo4_self_rel_many.id]] = [[demo4_self_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]] WHERE ((COALESCE([[demo4_self_rel_many.title]], '') = COALESCE({:TEST}, '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo4_self_rel_many.title]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4.self_rel_many]]) THEN [[__mm_demo4.self_rel_many]] ELSE json_array([[__mm_demo4.self_rel_many]]) END) `__mm_demo4_self_rel_many_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_many` ON [[__mm_demo4_self_rel_many.id]] = [[__mm_demo4_self_rel_many_je.value]] WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{__TEST}} WHERE NOT (COALESCE([[__TEST.multiMatchValue]], '') = COALESCE({:TEST}, '')))) OR JSON_EXTRACT([[demo4_self_rel_one.json_object]], '$.a') > 1)",
+		},
+		{
+			"@collection join (opt/any operators)",
+			"demo4",
+			"@collection.demo1.text ?> true || @collection.demo2.active ?> true || @collection.demo1.file_one ?> true",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `demo1` `__collection_demo1` LEFT JOIN `demo2` `__collection_demo2` WHERE ([[__collection_demo1.text]] > 1 OR [[__collection_demo2.active]] > 1 OR [[__collection_demo1.file_one]] > 1)",
+		},
+		{
+			"@collection join (multi-match operators)",
+			"demo4",
+			"@collection.demo1.text > true || @collection.demo2.active > true || @collection.demo1.file_one > true",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `demo1` `__collection_demo1` LEFT JOIN `demo2` `__collection_demo2` WHERE (([[__collection_demo1.text]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__collection_demo1.text]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN `demo1` `__mm__collection_demo1` WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{TEST}} WHERE (NOT ([[TEST.multiMatchValue]] > 1)) OR ([[TEST.multiMatchValue]] IS NULL))) OR ([[__collection_demo2.active]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__collection_demo2.active]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN `demo2` `__mm__collection_demo2` WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{__TEST}} WHERE (NOT ([[__TEST.multiMatchValue]] > 1)) OR ([[__TEST.multiMatchValue]] IS NULL))) OR ([[__collection_demo1.file_one]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__collection_demo1.file_one]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN `demo1` `__mm__collection_demo1` WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{TEST}} WHERE (NOT ([[TEST.multiMatchValue]] > 1)) OR ([[TEST.multiMatchValue]] IS NULL))))",
 		},
 		{
 			"@request.auth fields",
 			"demo4",
-			[]string{"@request.auth.id", "@request.auth.username", "@request.auth.rel.title", "@request.data.demo"},
+			"@request.auth.id > true || @request.auth.username > true || @request.auth.rel.title > true || @request.data.demo > true",
 			false,
-			"^" +
-				regexp.QuoteMeta("SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `users` `__auth_users` ON [[__auth_users.id]] =") +
-				" {:.*} " +
-				regexp.QuoteMeta("LEFT JOIN json_each(CASE WHEN json_valid([[__auth_users.rel]]) THEN [[__auth_users.rel]] ELSE json_array([[__auth_users.rel]]) END) `__auth_users_rel_je` LEFT JOIN `demo2` `__auth_users_rel` ON [[__auth_users_rel.id]] = [[__auth_users_rel_je.value]]") +
-				"$",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `users` `__auth_users` ON `__auth_users`.`id`={:TEST} LEFT JOIN json_each(CASE WHEN json_valid([[__auth_users.rel]]) THEN [[__auth_users.rel]] ELSE json_array([[__auth_users.rel]]) END) `__auth_users_rel_je` LEFT JOIN `demo2` `__auth_users_rel` ON [[__auth_users_rel.id]] = [[__auth_users_rel_je.value]] WHERE ({:TEST} > 1 OR {:TEST} > 1 OR [[__auth_users_rel.title]] > 1 OR NULL > 1)",
 		},
 		{
-			"hidden field with system filters (ignore emailVisibility)",
+			"hidden field with system filters (multi-match and ignore emailVisibility)",
 			"demo4",
-			[]string{"@collection.users.email", "@request.auth.email"},
+			"@collection.users.email > true || @request.auth.email > true",
 			false,
-			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `users` `__collection_users`",
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `users` `__collection_users` WHERE (([[__collection_users.email]] > 1) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__collection_users.email]] as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN `users` `__mm__collection_users` WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{__smTEST}} WHERE (NOT ([[__smTEST.multiMatchValue]] > 1)) OR ([[__smTEST.multiMatchValue]] IS NULL))) OR {:TEST} > 1)",
 		},
 		{
 			"hidden field (add emailVisibility)",
 			"users",
-			[]string{"email"},
+			"email > true",
 			false,
-			"SELECT `users`.* FROM `users` WHERE [[users.emailVisibility]] = TRUE",
+			"SELECT `users`.* FROM `users` WHERE (([[users.email]] > 1) AND ([[users.emailVisibility]] = TRUE))",
 		},
 		{
 			"hidden field (force ignore emailVisibility)",
 			"users",
-			[]string{"email"},
+			"email > true",
 			true,
-			"SELECT `users`.* FROM `users`",
+			"SELECT `users`.* FROM `users` WHERE [[users.email]] > 1",
+		},
+		{
+			"isset key",
+			"demo1",
+			"@request.data.a:isset > true ||" +
+				"@request.data.b:isset > true ||" +
+				"@request.data.c:isset > true ||" +
+				"@request.query.a:isset > true ||" +
+				"@request.query.b:isset > true ||" +
+				"@request.query.c:isset > true",
+			false,
+			"SELECT `demo1`.* FROM `demo1` WHERE (TRUE > 1 OR TRUE > 1 OR FALSE > 1 OR TRUE > 1 OR TRUE > 1 OR FALSE > 1)",
+		},
+		{
+			"@request.data.rel.* fields",
+			"demo1",
+			"@request.data.rel_one > true &&" +
+				"@request.data.rel_one.text > true &&" +
+				"@request.data.rel_many > true &&" +
+				"@request.data.rel_many.email != 'test' &&" +
+				"@request.data.rel_many.url ?= 'test' &&" +
+				"@request.data.rel_many.avatar ~ 'test'",
+			false,
+			"SELECT DISTINCT `demo1`.* FROM `demo1` LEFT JOIN `demo1` `__data_demo1` ON [[__data_demo1.id]]={:TEST} LEFT JOIN `users` `__data_users` ON [[__data_users.id]] IN ({:TEST}, {:TEST}) WHERE ({:TEST} > 1 AND [[__data_demo1.text]] > 1 AND {:TEST} > 1 AND (COALESCE([[__data_users.email]], '') != COALESCE({:TEST}, '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__data_mm_users.email]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN `users` `__data_mm_users` ON `__data_mm_users`.`id` IN ({:TEST}, {:TEST}) WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE (NOT (COALESCE([[__smTEST.multiMatchValue]], '') != COALESCE({:TEST}, ''))) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND COALESCE(NULL, '') = COALESCE({:TEST}, '') AND ([[__data_users.avatar]] LIKE {:TEST} ESCAPE '\\') AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__data_mm_users.avatar]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN `users` `__data_mm_users` ON `__data_mm_users`.`id` IN ({:TEST}, {:TEST}) WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE (NOT ([[__smTEST.multiMatchValue]] LIKE {:TEST} ESCAPE '\\')) OR ([[__smTEST.multiMatchValue]] IS NULL))))",
+		},
+		{
+			"@request.data.select:each fields",
+			"demo1",
+			"@request.data.select_one = 'test' &&" +
+				"@request.data.select_one:each != 'test' &&" +
+				"@request.data.select_one:each ?= 'test' &&" +
+				"@request.data.select_many ~ 'test' &&" +
+				"@request.data.select_many:each = 'test' &&" +
+				"@request.data.select_many:each ?< true",
+			false,
+			"SELECT DISTINCT `demo1`.* FROM `demo1` LEFT JOIN json_each({:TEST}) `__dataSelect_select_one_je` LEFT JOIN json_each({:TEST}) `__dataSelect_select_many_je` WHERE (COALESCE(NULL, '') = COALESCE({:TEST}, '') AND COALESCE([[__dataSelect_select_one_je.value]], '') != COALESCE({:TEST}, '') AND COALESCE([[__dataSelect_select_one_je.value]], '') = COALESCE({:TEST}, '') AND {:TEST} LIKE {:TEST} ESCAPE '\\' AND (COALESCE([[__dataSelect_select_many_je.value]], '') = COALESCE({:TEST}, '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__dataSelect_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each({:TEST}) `__mm__dataSelect_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE NOT (COALESCE([[__smTEST.multiMatchValue]], '') = COALESCE({:TEST}, '')))) AND [[__dataSelect_select_many_je.value]] < 1)",
+		},
+		{
+			"regular select:each fields",
+			"demo1",
+			"select_one = 'test' &&" +
+				"select_one:each != 'test' &&" +
+				"select_one:each ?> true &&" +
+				"select_many ~ 'test' &&" +
+				"select_many:each = 'test' &&" +
+				"select_many:each ?> true",
+			false,
+			"SELECT DISTINCT `demo1`.* FROM `demo1` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.select_one]]) THEN [[demo1.select_one]] ELSE json_array([[demo1.select_one]]) END) `demo1_select_one_je` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.select_many]]) THEN [[demo1.select_many]] ELSE json_array([[demo1.select_many]]) END) `demo1_select_many_je` WHERE (COALESCE([[demo1.select_one]], '') = COALESCE({:TEST}, '') AND COALESCE([[demo1_select_one_je.value]], '') != COALESCE({:TEST}, '') AND [[demo1_select_one_je.value]] > 1 AND [[demo1.select_many]] LIKE {:TEST} ESCAPE '\\' AND (COALESCE([[demo1_select_many_je.value]], '') = COALESCE({:TEST}, '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.select_many]]) THEN [[__mm_demo1.select_many]] ELSE json_array([[__mm_demo1.select_many]]) END) `__mm_demo1_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE NOT (COALESCE([[__smTEST.multiMatchValue]], '') = COALESCE({:TEST}, '')))) AND [[demo1_select_many_je.value]] > 1)",
+		},
+		{
+			"select:each vs select:each",
+			"demo1",
+			"select_one:each != select_many:each &&" +
+				"select_many:each > select_one:each &&" +
+				"select_many:each ?< select_one:each &&" +
+				"select_many:each = @request.data.select_many:each",
+			false,
+			"SELECT DISTINCT `demo1`.* FROM `demo1` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.select_one]]) THEN [[demo1.select_one]] ELSE json_array([[demo1.select_one]]) END) `demo1_select_one_je` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.select_many]]) THEN [[demo1.select_many]] ELSE json_array([[demo1.select_many]]) END) `demo1_select_many_je` LEFT JOIN json_each({:TEST}) `__dataSelect_select_many_je` WHERE ((COALESCE([[demo1_select_one_je.value]], '') != COALESCE([[demo1_select_many_je.value]], '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.select_many]]) THEN [[__mm_demo1.select_many]] ELSE json_array([[__mm_demo1.select_many]]) END) `__mm_demo1_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE (NOT (COALESCE([[demo1_select_one_je.value]], '') != COALESCE([[__smTEST.multiMatchValue]], ''))) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND ([[demo1_select_many_je.value]] > [[demo1_select_one_je.value]]) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.select_many]]) THEN [[__mm_demo1.select_many]] ELSE json_array([[__mm_demo1.select_many]]) END) `__mm_demo1_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE (NOT ([[__smTEST.multiMatchValue]] > [[demo1_select_one_je.value]])) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND [[demo1_select_many_je.value]] < [[demo1_select_one_je.value]] AND (COALESCE([[demo1_select_many_je.value]], '') = COALESCE([[__dataSelect_select_many_je.value]], '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.select_many]]) THEN [[__mm_demo1.select_many]] ELSE json_array([[__mm_demo1.select_many]]) END) `__mm_demo1_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mlTEST}} LEFT JOIN (SELECT [[__mm__dataSelect_select_many_je.value]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each({:TEST}) `__mm__dataSelect_select_many_je` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mrTEST}} WHERE NOT (COALESCE([[__mlTEST.multiMatchValue]], '') = COALESCE([[__mrTEST.multiMatchValue]], '')))))",
+		},
+		{
+			"mixed multi-match vs multi-match",
+			"demo1",
+			"rel_many.rel.active != rel_many.name &&" +
+				"rel_many.rel.active ?= rel_many.name &&" +
+				"rel_many.rel.title ~ rel_one.email &&" +
+				"@collection.demo2.active = rel_many.rel.active &&" +
+				"@collection.demo2.active ?= rel_many.rel.active &&" +
+				"rel_many.email > @request.data.rel_many.email",
+			false,
+			"SELECT DISTINCT `demo1`.* FROM `demo1` LEFT JOIN json_each(CASE WHEN json_valid([[demo1.rel_many]]) THEN [[demo1.rel_many]] ELSE json_array([[demo1.rel_many]]) END) `demo1_rel_many_je` LEFT JOIN `users` `demo1_rel_many` ON [[demo1_rel_many.id]] = [[demo1_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo1_rel_many.rel]]) THEN [[demo1_rel_many.rel]] ELSE json_array([[demo1_rel_many.rel]]) END) `demo1_rel_many_rel_je` LEFT JOIN `demo2` `demo1_rel_many_rel` ON [[demo1_rel_many_rel.id]] = [[demo1_rel_many_rel_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo1.rel_one]]) THEN [[demo1.rel_one]] ELSE json_array([[demo1.rel_one]]) END) `demo1_rel_one_je` LEFT JOIN `demo1` `demo1_rel_one` ON [[demo1_rel_one.id]] = [[demo1_rel_one_je.value]] LEFT JOIN `demo2` `__collection_demo2` LEFT JOIN `users` `__data_users` ON [[__data_users.id]] IN ({:TEST}, {:TEST}) WHERE ((COALESCE([[demo1_rel_many_rel.active]], '') != COALESCE([[demo1_rel_many.name]], '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_rel_many_rel.active]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.rel_many]]) THEN [[__mm_demo1.rel_many]] ELSE json_array([[__mm_demo1.rel_many]]) END) `__mm_demo1_rel_many_je` LEFT JOIN `users` `__mm_demo1_rel_many` ON [[__mm_demo1_rel_many.id]] = [[__mm_demo1_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1_rel_many.rel]]) THEN [[__mm_demo1_rel_many.rel]] ELSE json_array([[__mm_demo1_rel_many.rel]]) END) `__mm_demo1_rel_many_rel_je` LEFT JOIN `demo2` `__mm_demo1_rel_many_rel` ON [[__mm_demo1_rel_many_rel.id]] = [[__mm_demo1_rel_many_rel_je.value]] WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mlTEST}} LEFT JOIN (SELECT [[__mm_demo1_rel_many.name]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.rel_many]]) THEN [[__mm_demo1.rel_many]] ELSE json_array([[__mm_demo1.rel_many]]) END) `__mm_demo1_rel_many_je` LEFT JOIN `users` `__mm_demo1_rel_many` ON [[__mm_demo1_rel_many.id]] = [[__mm_demo1_rel_many_je.value]] WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mrTEST}} WHERE (NOT (COALESCE([[__mlTEST.multiMatchValue]], '') != COALESCE([[__mrTEST.multiMatchValue]], ''))) OR ([[__mlTEST.multiMatchValue]] IS NULL) OR ([[__mrTEST.multiMatchValue]] IS NULL))) AND COALESCE([[demo1_rel_many_rel.active]], '') = COALESCE([[demo1_rel_many.name]], '') AND ([[demo1_rel_many_rel.title]] LIKE ('%' || [[demo1_rel_one.email]] || '%') ESCAPE '\\') AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_rel_many_rel.title]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.rel_many]]) THEN [[__mm_demo1.rel_many]] ELSE json_array([[__mm_demo1.rel_many]]) END) `__mm_demo1_rel_many_je` LEFT JOIN `users` `__mm_demo1_rel_many` ON [[__mm_demo1_rel_many.id]] = [[__mm_demo1_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1_rel_many.rel]]) THEN [[__mm_demo1_rel_many.rel]] ELSE json_array([[__mm_demo1_rel_many.rel]]) END) `__mm_demo1_rel_many_rel_je` LEFT JOIN `demo2` `__mm_demo1_rel_many_rel` ON [[__mm_demo1_rel_many_rel.id]] = [[__mm_demo1_rel_many_rel_je.value]] WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__smTEST}} WHERE (NOT ([[__smTEST.multiMatchValue]] LIKE ('%' || [[demo1_rel_one.email]] || '%') ESCAPE '\\')) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND (COALESCE([[__collection_demo2.active]], '') = COALESCE([[demo1_rel_many_rel.active]], '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm__collection_demo2.active]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN `demo2` `__mm__collection_demo2` WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mlTEST}} LEFT JOIN (SELECT [[__mm_demo1_rel_many_rel.active]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.rel_many]]) THEN [[__mm_demo1.rel_many]] ELSE json_array([[__mm_demo1.rel_many]]) END) `__mm_demo1_rel_many_je` LEFT JOIN `users` `__mm_demo1_rel_many` ON [[__mm_demo1_rel_many.id]] = [[__mm_demo1_rel_many_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1_rel_many.rel]]) THEN [[__mm_demo1_rel_many.rel]] ELSE json_array([[__mm_demo1_rel_many.rel]]) END) `__mm_demo1_rel_many_rel_je` LEFT JOIN `demo2` `__mm_demo1_rel_many_rel` ON [[__mm_demo1_rel_many_rel.id]] = [[__mm_demo1_rel_many_rel_je.value]] WHERE `__mm_demo1`.`id` = `demo1`.`id`) {{__mrTEST}} WHERE NOT (COALESCE([[__mlTEST.multiMatchValue]], '') = COALESCE([[__mrTEST.multiMatchValue]], '')))) AND COALESCE([[__collection_demo2.active]], '') = COALESCE([[demo1_rel_many_rel.active]], '') AND (([[demo1_rel_many.email]] > [[__data_users.email]]) AND (NOT EXISTS (SELECT 1 FROM (SELECT [[__mm_demo1_rel_many.email]] as [[multiMatchValue]] FROM `demo1` `__mm_demo1` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo1.rel_many]]) THEN [[__mm_demo1.rel_many]] ELSE json_array([[__mm_demo1.rel_many]]) END) `__mm_demo1_rel_many_je` LEFT JOIN `usersLine truncated
+		},
+		{
+			"@request.data.arrayable:length fields",
+			"demo1",
+			"@request.data.select_one:length > 1 &&" +
+				"@request.data.select_one:length ?> 2 &&" +
+				"@request.data.select_many:length < 3 &&" +
+				"@request.data.select_many:length ?> 4 &&" +
+				"@request.data.rel_one:length = 5 &&" +
+				"@request.data.rel_one:length ?= 6 &&" +
+				"@request.data.rel_many:length != 7 &&" +
+				"@request.data.rel_many:length ?!= 8 &&" +
+				"@request.data.file_one:length = 9 &&" +
+				"@request.data.file_one:length ?= 0 &&" +
+				"@request.data.file_many:length != 1 &&" +
+				"@request.data.file_many:length ?!= 2",
+			false,
+			"SELECT `demo1`.* FROM `demo1` WHERE (0 > {:TEST} AND 0 > {:TEST} AND 2 < {:TEST} AND 2 > {:TEST} AND COALESCE(1, '') = COALESCE({:TEST}, '') AND COALESCE(1, '') = COALESCE({:TEST}, '') AND COALESCE(2, '') != COALESCE({:TEST}, '') AND COALESCE(2, '') != COALESCE({:TEST}, '') AND COALESCE(1, '') = COALESCE({:TEST}, '') AND COALESCE(1, '') = COALESCE({:TEST}, '') AND COALESCE(3, '') != COALESCE({:TEST}, '') AND COALESCE(3, '') != COALESCE({:TEST}, ''))",
+		},
+		{
+			"regular arrayable:length fields",
+			"demo4",
+			"@request.data.self_rel_one.self_rel_many:length > 1 &&" +
+				"@request.data.self_rel_one.self_rel_many:length ?> 2 &&" +
+				"@request.data.rel_many_cascade.files:length ?< 3 &&" +
+				"@request.data.rel_many_cascade.files:length < 4 &&" +
+				"self_rel_one.self_rel_many:length = 5 &&" +
+				"self_rel_one.self_rel_many:length ?= 6 &&" +
+				"self_rel_one.rel_many_cascade.files:length != 7 &&" +
+				"self_rel_one.rel_many_cascade.files:length ?!= 8",
+			false,
+			"SELECT DISTINCT `demo4`.* FROM `demo4` LEFT JOIN `demo4` `__data_demo4` ON [[__data_demo4.id]]={:TEST} LEFT JOIN `demo3` `__data_demo3` ON [[__data_demo3.id]] IN ({:TEST}, {:TEST}) LEFT JOIN json_each(CASE WHEN json_valid([[demo4.self_rel_one]]) THEN [[demo4.self_rel_one]] ELSE json_array([[demo4.self_rel_one]]) END) `demo4_self_rel_one_je` LEFT JOIN `demo4` `demo4_self_rel_one` ON [[demo4_self_rel_one.id]] = [[demo4_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[demo4_self_rel_one.rel_many_cascade]]) THEN [[demo4_self_rel_one.rel_many_cascade]] ELSE json_array([[demo4_self_rel_one.rel_many_cascade]]) END) `demo4_self_rel_one_rel_many_cascade_je` LEFT JOIN `demo3` `demo4_self_rel_one_rel_many_cascade` ON [[demo4_self_rel_one_rel_many_cascade.id]] = [[demo4_self_rel_one_rel_many_cascade_je.value]] WHERE (json_array_length(CASE WHEN json_valid([[__data_demo4.self_rel_many]]) THEN [[__data_demo4.self_rel_many]] ELSE json_array([[__data_demo4.self_rel_many]]) END) > {:TEST} AND json_array_length(CASE WHEN json_valid([[__data_demo4.self_rel_many]]) THEN [[__data_demo4.self_rel_many]] ELSE json_array([[__data_demo4.self_rel_many]]) END) > {:TEST} AND json_array_length(CASE WHEN json_valid([[__data_demo3.files]]) THEN [[__data_demo3.files]] ELSE json_array([[__data_demo3.files]]) END) < {:TEST} AND (json_array_length(CASE WHEN json_valid([[__data_demo3.files]]) THEN [[__data_demo3.files]] ELSE json_array([[__data_demo3.files]]) END) < {:TEST}) AND (NOT EXISTS (SELECT 1 FROM (SELECT json_array_length(CASE WHEN json_valid([[__data_mm_demo3.files]]) THEN [[__data_mm_demo3.files]] ELSE json_array([[__data_mm_demo3.files]]) END) as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN `demo3` `__data_mm_demo3` ON `__data_mm_demo3`.`id` IN ({:TEST}, {:TEST}) WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{__smTEST}} WHERE (NOT ([[__smTEST.multiMatchValue]] < {:TEST})) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND COALESCE(json_array_length(CASE WHEN json_valid([[demo4_self_rel_one.self_rel_many]]) THEN [[demo4_self_rel_one.self_rel_many]] ELSE json_array([[demo4_self_rel_one.self_rel_many]]) END), '') = COALESCE({:TEST}, '') AND COALESCE(json_array_length(CASE WHEN json_valid([[demo4_self_rel_one.self_rel_many]]) THEN [[demo4_self_rel_one.self_rel_many]] ELSE json_array([[demo4_self_rel_one.self_rel_many]]) END), '') = COALESCE({:TEST}, '') AND (COALESCE(json_array_length(CASE WHEN json_valid([[demo4_self_rel_one_rel_many_cascade.files]]) THEN [[demo4_self_rel_one_rel_many_cascade.files]] ELSE json_array([[demo4_self_rel_one_rel_many_cascade.files]]) END), '') != COALESCE({:TEST}, '')) AND (NOT EXISTS (SELECT 1 FROM (SELECT json_array_length(CASE WHEN json_valid([[__mm_demo4_self_rel_one_rel_many_cascade.files]]) THEN [[__mm_demo4_self_rel_one_rel_many_cascade.files]] ELSE json_array([[__mm_demo4_self_rel_one_rel_many_cascade.files]]) END) as [[multiMatchValue]] FROM `demo4` `__mm_demo4` LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4.self_rel_one]]) THEN [[__mm_demo4.self_rel_one]] ELSE json_array([[__mm_demo4.self_rel_one]]) END) `__mm_demo4_self_rel_one_je` LEFT JOIN `demo4` `__mm_demo4_self_rel_one` ON [[__mm_demo4_self_rel_one.id]] = [[__mm_demo4_self_rel_one_je.value]] LEFT JOIN json_each(CASE WHEN json_valid([[__mm_demo4_self_rel_one.rel_many_cascade]]) THEN [[__mm_demo4_self_rel_one.rel_many_cascade]] ELSE json_array([[__mm_demo4_self_rel_one.rel_many_cascade]]) END) `__mm_demo4_self_rel_one_rel_many_cascade_je` LEFT JOIN `demo3` `__mm_demo4_self_rel_one_rel_many_cascade` ON [[__mm_demo4_self_rel_one_rel_many_cascade.id]] = [[__mm_demo4_self_rel_one_rel_many_cascade_je.value]] WHERE `__mm_demo4`.`id` = `demo4`.`id`) {{__smTEST}} WHERE (NOT (COALESCE([[__smTEST.multiMatchValue]], '') != COALESCE({:TEST}, ''))) OR ([[__smTEST.multiMatchValue]] IS NULL))) AND COALESCE(json_array_length(CASE WHEN json_valid([[demo4_self_rel_one_rel_many_cascade.files]]) THEN [[demo4_self_rel_one_rel_many_cascade.files]] ELSE json_array([[demo4_self_rel_one_rel_many_cascade.files]]) END), '') != COALESCE({:TEST}, ''))",
 		},
 	}
 
 	for _, s := range scenarios {
 		collection, err := app.Dao().FindCollectionByNameOrId(s.collectionIdOrName)
 		if err != nil {
-			t.Errorf("[%s] Failed to load collection %s: %v", s.name, s.collectionIdOrName, err)
+			t.Fatalf("[%s] Failed to load collection %s: %v", s.name, s.collectionIdOrName, err)
 		}
 
 		query := app.Dao().RecordQuery(collection)
 
 		r := resolvers.NewRecordFieldResolver(app.Dao(), collection, requestData, s.allowHiddenFields)
-		for _, field := range s.fields {
-			r.Resolve(field)
+
+		expr, err := search.FilterData(s.rule).BuildExpr(r)
+		if err != nil {
+			t.Fatalf("[%s] BuildExpr failed with error %v", s.name, err)
 		}
 
 		if err := r.UpdateQuery(query); err != nil {
-			t.Errorf("[%s] UpdateQuery failed with error %v", s.name, err)
-			continue
+			t.Fatalf("[%s] UpdateQuery failed with error %v", s.name, err)
 		}
 
-		rawQuery := query.Build().SQL()
+		rawQuery := query.AndWhere(expr).Build().SQL()
 
-		if !list.ExistInSliceWithRegex(rawQuery, []string{s.expectQuery}) {
-			t.Errorf("[%s] Expected query\n %v \ngot:\n %v", s.name, s.expectQuery, rawQuery)
+		// replace TEST placeholder with .+ regex pattern
+		expectQuery := strings.ReplaceAll(
+			"^"+regexp.QuoteMeta(s.expectQuery)+"$",
+			"TEST",
+			`\w+`,
+		)
+
+		if !list.ExistInSliceWithRegex(rawQuery, []string{expectQuery}) {
+			t.Fatalf("[%s] Expected query\n %v \ngot:\n %v", s.name, expectQuery, rawQuery)
 		}
 	}
 }
@@ -230,7 +390,7 @@ func TestRecordFieldResolverResolveSchemaFields(t *testing.T) {
 	}
 
 	for _, s := range scenarios {
-		name, params, err := r.Resolve(s.fieldName)
+		r, err := r.Resolve(s.fieldName)
 
 		hasErr := err != nil
 		if hasErr != s.expectError {
@@ -238,13 +398,17 @@ func TestRecordFieldResolverResolveSchemaFields(t *testing.T) {
 			continue
 		}
 
-		if name != s.expectName {
-			t.Errorf("(%q) Expected name %q, got %q", s.fieldName, s.expectName, name)
+		if hasErr {
+			continue
+		}
+
+		if r.Identifier != s.expectName {
+			t.Errorf("(%q) Expected r.Identifier %q, got %q", s.fieldName, s.expectName, r.Identifier)
 		}
 
 		// params should be empty for non @request fields
-		if len(params) != 0 {
-			t.Errorf("(%q) Expected 0 params, got %v", s.fieldName, params)
+		if len(r.Params) != 0 {
+			t.Errorf("(%q) Expected 0 r.Params, got %v", s.fieldName, r.Params)
 		}
 	}
 }
@@ -253,7 +417,7 @@ func TestRecordFieldResolverResolveStaticRequestDataFields(t *testing.T) {
 	app, _ := tests.NewTestApp()
 	defer app.Cleanup()
 
-	collection, err := app.Dao().FindCollectionByNameOrId("demo4")
+	collection, err := app.Dao().FindCollectionByNameOrId("demo1")
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -269,8 +433,10 @@ func TestRecordFieldResolverResolveStaticRequestDataFields(t *testing.T) {
 			"a": 123,
 		},
 		Data: map[string]any{
-			"b": 456,
-			"c": map[string]int{"sub": 1},
+			"number":         "10",
+			"number_unknown": "20",
+			"b":              456,
+			"c":              map[string]int{"sub": 1},
 		},
 		AuthRecord: authRecord,
 	}
@@ -292,6 +458,8 @@ func TestRecordFieldResolverResolveStaticRequestDataFields(t *testing.T) {
 		{"@request.query.a.missing", false, ``},
 		{"@request.data", true, ``},
 		{"@request.data.b", false, `456`},
+		{"@request.data.number", false, `10`},           // number field normalization
+		{"@request.data.number_unknown", false, `"20"`}, // no numeric normalizations for unknown fields
 		{"@request.data.b.missing", false, ``},
 		{"@request.data.c", false, `"{\"sub\":1}"`},
 		{"@request.auth", true, ""},
@@ -304,7 +472,7 @@ func TestRecordFieldResolverResolveStaticRequestDataFields(t *testing.T) {
 	}
 
 	for i, s := range scenarios {
-		name, params, err := r.Resolve(s.fieldName)
+		r, err := r.Resolve(s.fieldName)
 
 		hasErr := err != nil
 		if hasErr != s.expectError {
@@ -318,34 +486,34 @@ func TestRecordFieldResolverResolveStaticRequestDataFields(t *testing.T) {
 
 		// missing key
 		// ---
-		if len(params) == 0 {
-			if name != "NULL" {
-				t.Errorf("(%d) Expected 0 placeholder parameters for %v, got %v", i, name, params)
+		if len(r.Params) == 0 {
+			if r.Identifier != "NULL" {
+				t.Errorf("(%d) Expected 0 placeholder parameters for %v, got %v", i, r.Identifier, r.Params)
 			}
 			continue
 		}
 
 		// existing key
 		// ---
-		if len(params) != 1 {
-			t.Errorf("(%d) Expected 1 placeholder parameter for %v, got %v", i, name, params)
+		if len(r.Params) != 1 {
+			t.Errorf("(%d) Expected 1 placeholder parameter for %v, got %v", i, r.Identifier, r.Params)
 			continue
 		}
 
 		var paramName string
 		var paramValue any
-		for k, v := range params {
+		for k, v := range r.Params {
 			paramName = k
 			paramValue = v
 		}
 
-		if name != ("{:" + paramName + "}") {
-			t.Errorf("(%d) Expected parameter name %q, got %q", i, paramName, name)
+		if r.Identifier != ("{:" + paramName + "}") {
+			t.Errorf("(%d) Expected parameter r.Identifier %q, got %q", i, paramName, r.Identifier)
 		}
 
 		encodedParamValue, _ := json.Marshal(paramValue)
 		if string(encodedParamValue) != s.expectParamValue {
-			t.Errorf("(%d) Expected params %v for %v, got %v", i, s.expectParamValue, name, string(encodedParamValue))
+			t.Errorf("(%d) Expected r.Params %v for %v, got %v", i, s.expectParamValue, r.Identifier, string(encodedParamValue))
 		}
 	}
 }

tests/app.go

@@ -7,6 +7,7 @@ import (
 	"path"
 	"path/filepath"
 	"runtime"
+	"sync"
 
 	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/mailer"
@@ -16,6 +17,8 @@ import (
 type TestApp struct {
 	*core.BaseApp
 
+	mux sync.Mutex
+
 	// EventCalls defines a map to inspect which app events
 	// (and how many times) were triggered.
 	//
@@ -42,15 +45,35 @@ func (t *TestApp) Cleanup() {
 }
 
 func (t *TestApp) NewMailClient() mailer.Mailer {
+	t.mux.Lock()
+	defer t.mux.Unlock()
+
 	t.TestMailer.Reset()
+
 	return t.TestMailer
 }
 
 // ResetEventCalls resets the EventCalls counter.
 func (t *TestApp) ResetEventCalls() {
+	t.mux.Lock()
+	defer t.mux.Unlock()
+
 	t.EventCalls = make(map[string]int)
 }
 
+func (t *TestApp) registerEventCall(name string) error {
+	t.mux.Lock()
+	defer t.mux.Unlock()
+
+	if t.EventCalls == nil {
+		t.EventCalls = make(map[string]int)
+	}
+
+	t.EventCalls[name]++
+
+	return nil
+}
+
 // NewTestApp creates and initializes a test application instance.
 //
 // It is the caller's responsibility to call `app.Cleanup()`
@@ -70,7 +93,11 @@ func NewTestApp(optTestDataDir ...string) (*TestApp, error) {
 		return nil, err
 	}
 
-	app := core.NewBaseApp(tempDir, "pb_test_env", false)
+	app := core.NewBaseApp(&core.BaseAppConfig{
+		DataDir:       tempDir,
+		EncryptionEnv: "pb_test_env",
+		IsDebug:       false,
+	})
 
 	// load data dir and db connections
 	if err := app.Bootstrap(); err != nil {
@@ -88,348 +115,279 @@ func NewTestApp(optTestDataDir ...string) (*TestApp, error) {
 	}
 
 	t.OnBeforeApiError().Add(func(e *core.ApiErrorEvent) error {
-		t.EventCalls["OnBeforeApiError"]++
-		return nil
+		return t.registerEventCall("OnBeforeApiError")
 	})
 
 	t.OnAfterApiError().Add(func(e *core.ApiErrorEvent) error {
-		t.EventCalls["OnAfterApiError"]++
-		return nil
+		return t.registerEventCall("OnAfterApiError")
 	})
 
 	t.OnModelBeforeCreate().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelBeforeCreate"]++
-		return nil
+		return t.registerEventCall("OnModelBeforeCreate")
 	})
 
 	t.OnModelAfterCreate().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelAfterCreate"]++
-		return nil
+		return t.registerEventCall("OnModelAfterCreate")
 	})
 
 	t.OnModelBeforeUpdate().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelBeforeUpdate"]++
-		return nil
+		return t.registerEventCall("OnModelBeforeUpdate")
 	})
 
 	t.OnModelAfterUpdate().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelAfterUpdate"]++
-		return nil
+		return t.registerEventCall("OnModelAfterUpdate")
 	})
 
 	t.OnModelBeforeDelete().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelBeforeDelete"]++
-		return nil
+		return t.registerEventCall("OnModelBeforeDelete")
 	})
 
 	t.OnModelAfterDelete().Add(func(e *core.ModelEvent) error {
-		t.EventCalls["OnModelAfterDelete"]++
-		return nil
+		return t.registerEventCall("OnModelAfterDelete")
 	})
 
 	t.OnRecordsListRequest().Add(func(e *core.RecordsListEvent) error {
-		t.EventCalls["OnRecordsListRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordsListRequest")
 	})
 
 	t.OnRecordViewRequest().Add(func(e *core.RecordViewEvent) error {
-		t.EventCalls["OnRecordViewRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordViewRequest")
 	})
 
 	t.OnRecordBeforeCreateRequest().Add(func(e *core.RecordCreateEvent) error {
-		t.EventCalls["OnRecordBeforeCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeCreateRequest")
 	})
 
 	t.OnRecordAfterCreateRequest().Add(func(e *core.RecordCreateEvent) error {
-		t.EventCalls["OnRecordAfterCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterCreateRequest")
 	})
 
 	t.OnRecordBeforeUpdateRequest().Add(func(e *core.RecordUpdateEvent) error {
-		t.EventCalls["OnRecordBeforeUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeUpdateRequest")
 	})
 
 	t.OnRecordAfterUpdateRequest().Add(func(e *core.RecordUpdateEvent) error {
-		t.EventCalls["OnRecordAfterUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterUpdateRequest")
 	})
 
 	t.OnRecordBeforeDeleteRequest().Add(func(e *core.RecordDeleteEvent) error {
-		t.EventCalls["OnRecordBeforeDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeDeleteRequest")
 	})
 
 	t.OnRecordAfterDeleteRequest().Add(func(e *core.RecordDeleteEvent) error {
-		t.EventCalls["OnRecordAfterDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterDeleteRequest")
 	})
 
 	t.OnRecordAuthRequest().Add(func(e *core.RecordAuthEvent) error {
-		t.EventCalls["OnRecordAuthRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAuthRequest")
 	})
 
 	t.OnRecordBeforeRequestPasswordResetRequest().Add(func(e *core.RecordRequestPasswordResetEvent) error {
-		t.EventCalls["OnRecordBeforeRequestPasswordResetRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeRequestPasswordResetRequest")
 	})
 
 	t.OnRecordAfterRequestPasswordResetRequest().Add(func(e *core.RecordRequestPasswordResetEvent) error {
-		t.EventCalls["OnRecordAfterRequestPasswordResetRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterRequestPasswordResetRequest")
 	})
 
 	t.OnRecordBeforeConfirmPasswordResetRequest().Add(func(e *core.RecordConfirmPasswordResetEvent) error {
-		t.EventCalls["OnRecordBeforeConfirmPasswordResetRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeConfirmPasswordResetRequest")
 	})
 
 	t.OnRecordAfterConfirmPasswordResetRequest().Add(func(e *core.RecordConfirmPasswordResetEvent) error {
-		t.EventCalls["OnRecordAfterConfirmPasswordResetRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterConfirmPasswordResetRequest")
 	})
 
 	t.OnRecordBeforeRequestVerificationRequest().Add(func(e *core.RecordRequestVerificationEvent) error {
-		t.EventCalls["OnRecordBeforeRequestVerificationRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeRequestVerificationRequest")
 	})
 
 	t.OnRecordAfterRequestVerificationRequest().Add(func(e *core.RecordRequestVerificationEvent) error {
-		t.EventCalls["OnRecordAfterRequestVerificationRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterRequestVerificationRequest")
 	})
 
 	t.OnRecordBeforeConfirmVerificationRequest().Add(func(e *core.RecordConfirmVerificationEvent) error {
-		t.EventCalls["OnRecordBeforeConfirmVerificationRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeConfirmVerificationRequest")
 	})
 
 	t.OnRecordAfterConfirmVerificationRequest().Add(func(e *core.RecordConfirmVerificationEvent) error {
-		t.EventCalls["OnRecordAfterConfirmVerificationRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterConfirmVerificationRequest")
 	})
 
 	t.OnRecordBeforeRequestEmailChangeRequest().Add(func(e *core.RecordRequestEmailChangeEvent) error {
-		t.EventCalls["OnRecordBeforeRequestEmailChangeRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeRequestEmailChangeRequest")
 	})
 
 	t.OnRecordAfterRequestEmailChangeRequest().Add(func(e *core.RecordRequestEmailChangeEvent) error {
-		t.EventCalls["OnRecordAfterRequestEmailChangeRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterRequestEmailChangeRequest")
 	})
 
 	t.OnRecordBeforeConfirmEmailChangeRequest().Add(func(e *core.RecordConfirmEmailChangeEvent) error {
-		t.EventCalls["OnRecordBeforeConfirmEmailChangeRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeConfirmEmailChangeRequest")
 	})
 
 	t.OnRecordAfterConfirmEmailChangeRequest().Add(func(e *core.RecordConfirmEmailChangeEvent) error {
-		t.EventCalls["OnRecordAfterConfirmEmailChangeRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterConfirmEmailChangeRequest")
 	})
 
 	t.OnRecordListExternalAuthsRequest().Add(func(e *core.RecordListExternalAuthsEvent) error {
-		t.EventCalls["OnRecordListExternalAuthsRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordListExternalAuthsRequest")
 	})
 
 	t.OnRecordBeforeUnlinkExternalAuthRequest().Add(func(e *core.RecordUnlinkExternalAuthEvent) error {
-		t.EventCalls["OnRecordBeforeUnlinkExternalAuthRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordBeforeUnlinkExternalAuthRequest")
 	})
 
 	t.OnRecordAfterUnlinkExternalAuthRequest().Add(func(e *core.RecordUnlinkExternalAuthEvent) error {
-		t.EventCalls["OnRecordAfterUnlinkExternalAuthRequest"]++
-		return nil
+		return t.registerEventCall("OnRecordAfterUnlinkExternalAuthRequest")
 	})
 
 	t.OnMailerBeforeAdminResetPasswordSend().Add(func(e *core.MailerAdminEvent) error {
-		t.EventCalls["OnMailerBeforeAdminResetPasswordSend"]++
-		return nil
+		return t.registerEventCall("OnMailerBeforeAdminResetPasswordSend")
 	})
 
 	t.OnMailerAfterAdminResetPasswordSend().Add(func(e *core.MailerAdminEvent) error {
-		t.EventCalls["OnMailerAfterAdminResetPasswordSend"]++
-		return nil
+		return t.registerEventCall("OnMailerAfterAdminResetPasswordSend")
 	})
 
 	t.OnMailerBeforeRecordResetPasswordSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerBeforeRecordResetPasswordSend"]++
-		return nil
+		return t.registerEventCall("OnMailerBeforeRecordResetPasswordSend")
 	})
 
 	t.OnMailerAfterRecordResetPasswordSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerAfterRecordResetPasswordSend"]++
-		return nil
+		return t.registerEventCall("OnMailerAfterRecordResetPasswordSend")
 	})
 
 	t.OnMailerBeforeRecordVerificationSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerBeforeRecordVerificationSend"]++
-		return nil
+		return t.registerEventCall("OnMailerBeforeRecordVerificationSend")
 	})
 
 	t.OnMailerAfterRecordVerificationSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerAfterRecordVerificationSend"]++
-		return nil
+		return t.registerEventCall("OnMailerAfterRecordVerificationSend")
 	})
 
 	t.OnMailerBeforeRecordChangeEmailSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerBeforeRecordChangeEmailSend"]++
-		return nil
+		return t.registerEventCall("OnMailerBeforeRecordChangeEmailSend")
 	})
 
 	t.OnMailerAfterRecordChangeEmailSend().Add(func(e *core.MailerRecordEvent) error {
-		t.EventCalls["OnMailerAfterRecordChangeEmailSend"]++
-		return nil
+		return t.registerEventCall("OnMailerAfterRecordChangeEmailSend")
 	})
 
 	t.OnRealtimeConnectRequest().Add(func(e *core.RealtimeConnectEvent) error {
-		t.EventCalls["OnRealtimeConnectRequest"]++
-		return nil
+		return t.registerEventCall("OnRealtimeConnectRequest")
 	})
 
 	t.OnRealtimeDisconnectRequest().Add(func(e *core.RealtimeDisconnectEvent) error {
-		t.EventCalls["OnRealtimeDisconnectRequest"]++
-		return nil
+		return t.registerEventCall("OnRealtimeDisconnectRequest")
 	})
 
 	t.OnRealtimeBeforeMessageSend().Add(func(e *core.RealtimeMessageEvent) error {
-		t.EventCalls["OnRealtimeBeforeMessageSend"]++
-		return nil
+		return t.registerEventCall("OnRealtimeBeforeMessageSend")
 	})
 
 	t.OnRealtimeAfterMessageSend().Add(func(e *core.RealtimeMessageEvent) error {
-		t.EventCalls["OnRealtimeAfterMessageSend"]++
-		return nil
+		return t.registerEventCall("OnRealtimeAfterMessageSend")
 	})
 
 	t.OnRealtimeBeforeSubscribeRequest().Add(func(e *core.RealtimeSubscribeEvent) error {
-		t.EventCalls["OnRealtimeBeforeSubscribeRequest"]++
-		return nil
+		return t.registerEventCall("OnRealtimeBeforeSubscribeRequest")
 	})
 
 	t.OnRealtimeAfterSubscribeRequest().Add(func(e *core.RealtimeSubscribeEvent) error {
-		t.EventCalls["OnRealtimeAfterSubscribeRequest"]++
-		return nil
+		return t.registerEventCall("OnRealtimeAfterSubscribeRequest")
 	})
 
 	t.OnSettingsListRequest().Add(func(e *core.SettingsListEvent) error {
-		t.EventCalls["OnSettingsListRequest"]++
-		return nil
+		return t.registerEventCall("OnSettingsListRequest")
 	})
 
 	t.OnSettingsBeforeUpdateRequest().Add(func(e *core.SettingsUpdateEvent) error {
-		t.EventCalls["OnSettingsBeforeUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnSettingsBeforeUpdateRequest")
 	})
 
 	t.OnSettingsAfterUpdateRequest().Add(func(e *core.SettingsUpdateEvent) error {
-		t.EventCalls["OnSettingsAfterUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnSettingsAfterUpdateRequest")
 	})
 
 	t.OnCollectionsListRequest().Add(func(e *core.CollectionsListEvent) error {
-		t.EventCalls["OnCollectionsListRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionsListRequest")
 	})
 
 	t.OnCollectionViewRequest().Add(func(e *core.CollectionViewEvent) error {
-		t.EventCalls["OnCollectionViewRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionViewRequest")
 	})
 
 	t.OnCollectionBeforeCreateRequest().Add(func(e *core.CollectionCreateEvent) error {
-		t.EventCalls["OnCollectionBeforeCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionBeforeCreateRequest")
 	})
 
 	t.OnCollectionAfterCreateRequest().Add(func(e *core.CollectionCreateEvent) error {
-		t.EventCalls["OnCollectionAfterCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionAfterCreateRequest")
 	})
 
 	t.OnCollectionBeforeUpdateRequest().Add(func(e *core.CollectionUpdateEvent) error {
-		t.EventCalls["OnCollectionBeforeUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionBeforeUpdateRequest")
 	})
 
 	t.OnCollectionAfterUpdateRequest().Add(func(e *core.CollectionUpdateEvent) error {
-		t.EventCalls["OnCollectionAfterUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionAfterUpdateRequest")
 	})
 
 	t.OnCollectionBeforeDeleteRequest().Add(func(e *core.CollectionDeleteEvent) error {
-		t.EventCalls["OnCollectionBeforeDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionBeforeDeleteRequest")
 	})
 
 	t.OnCollectionAfterDeleteRequest().Add(func(e *core.CollectionDeleteEvent) error {
-		t.EventCalls["OnCollectionAfterDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionAfterDeleteRequest")
 	})
 
 	t.OnCollectionsBeforeImportRequest().Add(func(e *core.CollectionsImportEvent) error {
-		t.EventCalls["OnCollectionsBeforeImportRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionsBeforeImportRequest")
 	})
 
 	t.OnCollectionsAfterImportRequest().Add(func(e *core.CollectionsImportEvent) error {
-		t.EventCalls["OnCollectionsAfterImportRequest"]++
-		return nil
+		return t.registerEventCall("OnCollectionsAfterImportRequest")
 	})
 
 	t.OnAdminsListRequest().Add(func(e *core.AdminsListEvent) error {
-		t.EventCalls["OnAdminsListRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminsListRequest")
 	})
 
 	t.OnAdminViewRequest().Add(func(e *core.AdminViewEvent) error {
-		t.EventCalls["OnAdminViewRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminViewRequest")
 	})
 
 	t.OnAdminBeforeCreateRequest().Add(func(e *core.AdminCreateEvent) error {
-		t.EventCalls["OnAdminBeforeCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminBeforeCreateRequest")
 	})
 
 	t.OnAdminAfterCreateRequest().Add(func(e *core.AdminCreateEvent) error {
-		t.EventCalls["OnAdminAfterCreateRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminAfterCreateRequest")
 	})
 
 	t.OnAdminBeforeUpdateRequest().Add(func(e *core.AdminUpdateEvent) error {
-		t.EventCalls["OnAdminBeforeUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminBeforeUpdateRequest")
 	})
 
 	t.OnAdminAfterUpdateRequest().Add(func(e *core.AdminUpdateEvent) error {
-		t.EventCalls["OnAdminAfterUpdateRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminAfterUpdateRequest")
 	})
 
 	t.OnAdminBeforeDeleteRequest().Add(func(e *core.AdminDeleteEvent) error {
-		t.EventCalls["OnAdminBeforeDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminBeforeDeleteRequest")
 	})
 
 	t.OnAdminAfterDeleteRequest().Add(func(e *core.AdminDeleteEvent) error {
-		t.EventCalls["OnAdminAfterDeleteRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminAfterDeleteRequest")
 	})
 
 	t.OnAdminAuthRequest().Add(func(e *core.AdminAuthEvent) error {
-		t.EventCalls["OnAdminAuthRequest"]++
-		return nil
+		return t.registerEventCall("OnAdminAuthRequest")
 	})
 
 	t.OnFileDownloadRequest().Add(func(e *core.FileDownloadEvent) error {
-		t.EventCalls["OnFileDownloadRequest"]++
-		return nil
+		return t.registerEventCall("OnFileDownloadRequest")
 	})
 
 	return t, nil

tests/data/storage/wsmn24bux7wo113/84nmscqy84lsi1t/test_MaWC6mWyrP.txt

@@ -0,0 +1 @@
+test

tests/data/storage/wsmn24bux7wo113/84nmscqy84lsi1t/test_MaWC6mWyrP.txt.attrs

@@ -0,0 +1 @@
+{"user.cache_control":"","user.content_disposition":"","user.content_encoding":"","user.content_language":"","user.content_type":"text/plain; charset=utf-8","user.metadata":{"original-filename":"test.txt"},"md5":"2Oj8otwPiW/Xy0ywAxuiSQ=="}

tests/data/storage/wsmn24bux7wo113/84nmscqy84lsi1t/test_tC1Yc87DfC.txt

@@ -0,0 +1 @@
+test

tests/data/storage/wsmn24bux7wo113/84nmscqy84lsi1t/test_tC1Yc87DfC.txt.attrs

@@ -0,0 +1 @@
+{"user.cache_control":"","user.content_disposition":"","user.content_encoding":"","user.content_language":"","user.content_type":"text/plain; charset=utf-8","user.metadata":{"original-filename":"test.txt"},"md5":"2Oj8otwPiW/Xy0ywAxuiSQ=="}

tests/request.go

@@ -29,24 +29,32 @@ func MockMultipartData(data map[string]string, fileFields ...string) (*bytes.Buf
 	// write file fields
 	for _, fileField := range fileFields {
 		// create a test temporary file
-		tmpFile, err := os.CreateTemp(os.TempDir(), "tmpfile-*.txt")
-		if err != nil {
-			return nil, nil, err
-		}
-		if _, err := tmpFile.Write([]byte("test")); err != nil {
-			return nil, nil, err
-		}
-		tmpFile.Seek(0, 0)
-		defer tmpFile.Close()
-		defer os.Remove(tmpFile.Name())
+		err := func() error {
+			tmpFile, err := os.CreateTemp(os.TempDir(), "tmpfile-*.txt")
+			if err != nil {
+				return err
+			}
 
-		// stub uploaded file
-		w, err := mp.CreateFormFile(fileField, tmpFile.Name())
+			if _, err := tmpFile.Write([]byte("test")); err != nil {
+				return err
+			}
+			tmpFile.Seek(0, 0)
+			defer tmpFile.Close()
+			defer os.Remove(tmpFile.Name())
+
+			// stub uploaded file
+			w, err := mp.CreateFormFile(fileField, tmpFile.Name())
+			if err != nil {
+				return err
+			}
+			if _, err := io.Copy(w, tmpFile); err != nil {
+				return err
+			}
+
+			return nil
+		}()
 		if err != nil {
-			return nil, mp, err
-		}
-		if _, err := io.Copy(w, tmpFile); err != nil {
-			return nil, mp, err
+			return nil, nil, err
 		}
 	}
 

tools/auth/auth.go

@@ -9,13 +9,14 @@ import (
 
 // AuthUser defines a standardized oauth2 user data structure.
 type AuthUser struct {
-	Id          string         `json:"id"`
-	Name        string         `json:"name"`
-	Username    string         `json:"username"`
-	Email       string         `json:"email"`
-	AvatarUrl   string         `json:"avatarUrl"`
-	RawUser     map[string]any `json:"rawUser"`
-	AccessToken string         `json:"accessToken"`
+	Id           string         `json:"id"`
+	Name         string         `json:"name"`
+	Username     string         `json:"username"`
+	Email        string         `json:"email"`
+	AvatarUrl    string         `json:"avatarUrl"`
+	RawUser      map[string]any `json:"rawUser"`
+	AccessToken  string         `json:"accessToken"`
+	RefreshToken string         `json:"refreshToken"`
 }
 
 // Provider defines a common interface for an OAuth2 client.
@@ -105,6 +106,10 @@ func NewProviderByName(name string) (Provider, error) {
 		return NewKakaoProvider(), nil
 	case NameTwitch:
 		return NewTwitchProvider(), nil
+	case NameStrava:
+		return NewStravaProvider(), nil
+	case NameGitee:
+		return NewGiteeProvider(), nil
 	default:
 		return nil, errors.New("Missing provider " + name)
 	}

tools/auth/auth_test.go

@@ -108,4 +108,22 @@ func TestNewProviderByName(t *testing.T) {
 	if _, ok := p.(*auth.Twitch); !ok {
 		t.Error("Expected to be instance of *auth.Twitch")
 	}
+
+	// strava
+	p, err = auth.NewProviderByName(auth.NameStrava)
+	if err != nil {
+		t.Errorf("Expected nil, got error %v", err)
+	}
+	if _, ok := p.(*auth.Strava); !ok {
+		t.Error("Expected to be instance of *auth.Strava")
+	}
+
+	// gitee
+	p, err = auth.NewProviderByName(auth.NameGitee)
+	if err != nil {
+		t.Errorf("Expected nil, got error %v", err)
+	}
+	if _, ok := p.(*auth.Gitee); !ok {
+		t.Error("Expected to be instance of *auth.Gitee")
+	}
 }

tools/auth/base_provider.go

@@ -3,7 +3,7 @@ package auth
 import (
 	"context"
 	"fmt"
-	"io/ioutil"
+	"io"
 	"net/http"
 
 	"golang.org/x/oauth2"
@@ -125,7 +125,7 @@ func (p *baseProvider) sendRawUserDataRequest(req *http.Request, token *oauth2.T
 	}
 	defer response.Body.Close()
 
-	result, err := ioutil.ReadAll(response.Body)
+	result, err := io.ReadAll(response.Body)
 	if err != nil {
 		return nil, err
 	}

tools/auth/discord.go

@@ -63,12 +63,13 @@ func (p *Discord) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	username := fmt.Sprintf("%s#%s", extracted.Username, extracted.Discriminator)
 
 	user := &AuthUser{
-		Id:          extracted.Id,
-		Name:        username,
-		Username:    extracted.Username,
-		AvatarUrl:   avatarUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Id,
+		Name:         username,
+		Username:     extracted.Username,
+		AvatarUrl:    avatarUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 	if extracted.Verified {
 		user.Email = extracted.Email

tools/auth/facebook.go

@@ -54,12 +54,13 @@ func (p *Facebook) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Id,
-		Name:        extracted.Name,
-		Email:       extracted.Email,
-		AvatarUrl:   extracted.Picture.Data.Url,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Id,
+		Name:         extracted.Name,
+		Email:        extracted.Email,
+		AvatarUrl:    extracted.Picture.Data.Url,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/auth/gitee.go

@@ -0,0 +1,131 @@
+package auth
+
+import (
+	"encoding/json"
+	"io"
+	"strconv"
+
+	"github.com/go-ozzo/ozzo-validation/v4/is"
+	"golang.org/x/oauth2"
+)
+
+var _ Provider = (*Gitee)(nil)
+
+// NameGitee is the unique name of the Gitee provider.
+const NameGitee string = "gitee"
+
+// Gitee allows authentication via Gitee OAuth2.
+type Gitee struct {
+	*baseProvider
+}
+
+// NewGiteeProvider creates new Gitee provider instance with some defaults.
+func NewGiteeProvider() *Gitee {
+	return &Gitee{&baseProvider{
+		scopes:     []string{"user_info", "emails"},
+		authUrl:    "https://gitee.com/oauth/authorize",
+		tokenUrl:   "https://gitee.com/oauth/token",
+		userApiUrl: "https://gitee.com/api/v5/user",
+	}}
+}
+
+// FetchAuthUser returns an AuthUser instance based the Gitee's user api.
+//
+// API reference: https://gitee.com/api/v5/swagger#/getV5User
+func (p *Gitee) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
+	data, err := p.FetchRawUserData(token)
+	if err != nil {
+		return nil, err
+	}
+
+	rawUser := map[string]any{}
+	if err := json.Unmarshal(data, &rawUser); err != nil {
+		return nil, err
+	}
+
+	extracted := struct {
+		Login     string `json:"login"`
+		Id        int    `json:"id"`
+		Name      string `json:"name"`
+		Email     string `json:"email"`
+		AvatarUrl string `json:"avatar_url"`
+	}{}
+	if err := json.Unmarshal(data, &extracted); err != nil {
+		return nil, err
+	}
+
+	user := &AuthUser{
+		Id:           strconv.Itoa(extracted.Id),
+		Name:         extracted.Name,
+		Username:     extracted.Login,
+		AvatarUrl:    extracted.AvatarUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
+	}
+
+	if extracted.Email != "" && is.EmailFormat.Validate(extracted.Email) == nil {
+		// valid public primary email
+		user.Email = extracted.Email
+	} else {
+		// send an additional optional request to retrieve the email
+		email, err := p.fetchPrimaryEmail(token)
+		if err != nil {
+			return nil, err
+		}
+		user.Email = email
+	}
+
+	return user, nil
+}
+
+// fetchPrimaryEmail sends an API request to retrieve the verified primary email,
+// in case the user hasn't set "Public email address" or has unchecked
+// the "Access your emails data" permission during authentication.
+//
+// NB! This method can succeed and still return an empty email.
+// Error responses that are result of insufficient scopes permissions are ignored.
+//
+// API reference: https://gitee.com/api/v5/swagger#/getV5Emails
+func (p *Gitee) fetchPrimaryEmail(token *oauth2.Token) (string, error) {
+	client := p.Client(token)
+
+	response, err := client.Get("https://gitee.com/api/v5/emails")
+	if err != nil {
+		return "", err
+	}
+	defer response.Body.Close()
+
+	// ignore common http errors caused by insufficient scope permissions
+	if response.StatusCode == 401 || response.StatusCode == 403 || response.StatusCode == 404 {
+		return "", nil
+	}
+
+	content, err := io.ReadAll(response.Body)
+	if err != nil {
+		return "", err
+	}
+
+	emails := []struct {
+		Email string
+		State string
+		Scope []string
+	}{}
+	if err := json.Unmarshal(content, &emails); err != nil {
+		// ignore unmarshal error in case "Keep my email address private"
+		// was set because response.Body will be something like:
+		// {"email":"12285415+test@user.noreply.gitee.com"}
+		return "", nil
+	}
+
+	// extract the first verified primary email
+	for _, email := range emails {
+		for _, scope := range email.Scope {
+			if email.State == "confirmed" && scope == "primary" && is.EmailFormat.Validate(email.Email) == nil {
+				return email.Email, nil
+			}
+		}
+	}
+
+	return "", nil
+}

tools/auth/github.go

@@ -2,7 +2,7 @@ package auth
 
 import (
 	"encoding/json"
-	"io/ioutil"
+	"io"
 	"strconv"
 
 	"golang.org/x/oauth2"
@@ -55,48 +55,71 @@ func (p *Github) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          strconv.Itoa(extracted.Id),
-		Name:        extracted.Name,
-		Username:    extracted.Login,
-		Email:       extracted.Email,
-		AvatarUrl:   extracted.AvatarUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           strconv.Itoa(extracted.Id),
+		Name:         extracted.Name,
+		Username:     extracted.Login,
+		Email:        extracted.Email,
+		AvatarUrl:    extracted.AvatarUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
-	// in case user set "Keep my email address private",
-	// email should be retrieved via extra API request
+	// in case user has set "Keep my email address private", send an
+	// **optional** API request to retrieve the verified primary email
 	if user.Email == "" {
-		client := p.Client(token)
-
-		response, err := client.Get(p.userApiUrl + "/emails")
+		email, err := p.fetchPrimaryEmail(token)
 		if err != nil {
-			return user, err
-		}
-		defer response.Body.Close()
-
-		content, err := ioutil.ReadAll(response.Body)
-		if err != nil {
-			return user, err
-		}
-
-		emails := []struct {
-			Email    string
-			Verified bool
-			Primary  bool
-		}{}
-		if err := json.Unmarshal(content, &emails); err != nil {
-			return user, err
-		}
-
-		// extract the verified primary email
-		for _, email := range emails {
-			if email.Verified && email.Primary {
-				user.Email = email.Email
-				break
-			}
+			return nil, err
 		}
+		user.Email = email
 	}
 
 	return user, nil
 }
+
+// fetchPrimaryEmail sends an API request to retrieve the verified
+// primary email, in case "Keep my email address private" was set.
+//
+// NB! This method can succeed and still return an empty email.
+// Error responses that are result of insufficient scopes permissions are ignored.
+//
+// API reference: https://docs.github.com/en/rest/users/emails?apiVersion=2022-11-28
+func (p *Github) fetchPrimaryEmail(token *oauth2.Token) (string, error) {
+	client := p.Client(token)
+
+	response, err := client.Get(p.userApiUrl + "/emails")
+	if err != nil {
+		return "", err
+	}
+	defer response.Body.Close()
+
+	// ignore common http errors caused by insufficient scope permissions
+	// (the email field is optional, aka. return the auth user without it)
+	if response.StatusCode == 401 || response.StatusCode == 403 || response.StatusCode == 404 {
+		return "", nil
+	}
+
+	content, err := io.ReadAll(response.Body)
+	if err != nil {
+		return "", err
+	}
+
+	emails := []struct {
+		Email    string
+		Verified bool
+		Primary  bool
+	}{}
+	if err := json.Unmarshal(content, &emails); err != nil {
+		return "", err
+	}
+
+	// extract the verified primary email
+	for _, email := range emails {
+		if email.Verified && email.Primary {
+			return email.Email, nil
+		}
+	}
+
+	return "", nil
+}

tools/auth/gitlab.go

@@ -53,13 +53,14 @@ func (p *Gitlab) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          strconv.Itoa(extracted.Id),
-		Name:        extracted.Name,
-		Username:    extracted.Username,
-		Email:       extracted.Email,
-		AvatarUrl:   extracted.AvatarUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           strconv.Itoa(extracted.Id),
+		Name:         extracted.Name,
+		Username:     extracted.Username,
+		Email:        extracted.Email,
+		AvatarUrl:    extracted.AvatarUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/auth/google.go

@@ -52,12 +52,13 @@ func (p *Google) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Id,
-		Name:        extracted.Name,
-		Email:       extracted.Email,
-		AvatarUrl:   extracted.Picture,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Id,
+		Name:         extracted.Name,
+		Email:        extracted.Email,
+		AvatarUrl:    extracted.Picture,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/auth/kakao.go

@@ -59,11 +59,12 @@ func (p *Kakao) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          strconv.Itoa(extracted.Id),
-		Username:    extracted.Profile.Nickname,
-		AvatarUrl:   extracted.Profile.ImageUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           strconv.Itoa(extracted.Id),
+		Username:     extracted.Profile.Nickname,
+		AvatarUrl:    extracted.Profile.ImageUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 	if extracted.KakaoAccount.IsEmailValid && extracted.KakaoAccount.IsEmailVerified {
 		user.Email = extracted.KakaoAccount.Email

tools/auth/microsoft.go

@@ -53,11 +53,12 @@ func (p *Microsoft) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Id,
-		Name:        extracted.Name,
-		Email:       extracted.Email,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Id,
+		Name:         extracted.Name,
+		Email:        extracted.Email,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/auth/spotify.go

@@ -61,10 +61,11 @@ func (p *Spotify) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Id,
-		Name:        extracted.Name,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Id,
+		Name:         extracted.Name,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 	if len(extracted.Images) > 0 {
 		user.AvatarUrl = extracted.Images[0].Url

tools/auth/strava.go

@@ -0,0 +1,71 @@
+package auth
+
+import (
+	"encoding/json"
+	"strconv"
+
+	"golang.org/x/oauth2"
+)
+
+var _ Provider = (*Strava)(nil)
+
+// NameStrava is the unique name of the Strava provider.
+const NameStrava string = "strava"
+
+// Strava allows authentication via Strava OAuth2.
+type Strava struct {
+	*baseProvider
+}
+
+// NewStravaProvider creates new Strava provider instance with some defaults.
+func NewStravaProvider() *Strava {
+	return &Strava{&baseProvider{
+		scopes: []string{
+			"profile:read_all",
+		},
+		authUrl:    "https://www.strava.com/oauth/authorize",
+		tokenUrl:   "https://www.strava.com/api/v3/oauth/token",
+		userApiUrl: "https://www.strava.com/api/v3/athlete",
+	}}
+}
+
+// FetchAuthUser returns an AuthUser instance based on the Strava's user api.
+//
+// API reference: https://developers.strava.com/docs/authentication/
+func (p *Strava) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
+	data, err := p.FetchRawUserData(token)
+	if err != nil {
+		return nil, err
+	}
+
+	rawUser := map[string]any{}
+	if err := json.Unmarshal(data, &rawUser); err != nil {
+		return nil, err
+	}
+
+	extracted := struct {
+		Id              int    `json:"id"`
+		FirstName       string `json:"firstname"`
+		LastName        string `json:"lastname"`
+		Username        string `json:"username"`
+		ProfileImageUrl string `json:"profile"`
+
+		// At the time of writing, Strava OAuth2 doesn't support returning the user email address
+		// Email string `json:"email"`
+	}{}
+	if err := json.Unmarshal(data, &extracted); err != nil {
+		return nil, err
+	}
+
+	user := &AuthUser{
+		Id:           strconv.Itoa(extracted.Id),
+		Name:         extracted.FirstName + " " + extracted.LastName,
+		Username:     extracted.Username,
+		AvatarUrl:    extracted.ProfileImageUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
+	}
+
+	return user, nil
+}

tools/auth/twitch.go

@@ -61,13 +61,14 @@ func (p *Twitch) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Data[0].Id,
-		Name:        extracted.Data[0].DisplayName,
-		Username:    extracted.Data[0].Login,
-		Email:       extracted.Data[0].Email,
-		AvatarUrl:   extracted.Data[0].ProfileImageUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Data[0].Id,
+		Name:         extracted.Data[0].DisplayName,
+		Username:     extracted.Data[0].Login,
+		Email:        extracted.Data[0].Email,
+		AvatarUrl:    extracted.Data[0].ProfileImageUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/auth/twitter.go

@@ -63,12 +63,13 @@ func (p *Twitter) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
 	}
 
 	user := &AuthUser{
-		Id:          extracted.Data.Id,
-		Name:        extracted.Data.Name,
-		Username:    extracted.Data.Username,
-		AvatarUrl:   extracted.Data.ProfileImageUrl,
-		RawUser:     rawUser,
-		AccessToken: token.AccessToken,
+		Id:           extracted.Data.Id,
+		Name:         extracted.Data.Name,
+		Username:     extracted.Data.Username,
+		AvatarUrl:    extracted.Data.ProfileImageUrl,
+		RawUser:      rawUser,
+		AccessToken:  token.AccessToken,
+		RefreshToken: token.RefreshToken,
 	}
 
 	return user, nil

tools/filesystem/file.go

@@ -0,0 +1,178 @@
+package filesystem
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"io"
+	"mime/multipart"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+
+	"github.com/gabriel-vasile/mimetype"
+	"github.com/pocketbase/pocketbase/tools/inflector"
+	"github.com/pocketbase/pocketbase/tools/security"
+)
+
+// FileReader defines an interface for a file resource reader.
+type FileReader interface {
+	Open() (io.ReadSeekCloser, error)
+}
+
+// File defines a single file [io.ReadSeekCloser] resource.
+//
+// The file could be from a local path, multipipart/formdata header, etc.
+type File struct {
+	Name         string
+	OriginalName string
+	Size         int64
+	Reader       FileReader
+}
+
+// NewFileFromPath creates a new File instance from the provided local file path.
+func NewFileFromPath(path string) (*File, error) {
+	f := &File{}
+
+	info, err := os.Stat(path)
+	if err != nil {
+		return nil, err
+	}
+
+	f.Reader = &PathReader{Path: path}
+	f.Size = info.Size()
+	f.OriginalName = info.Name()
+	f.Name = normalizeName(f.Reader, f.OriginalName)
+
+	return f, nil
+}
+
+// NewFileFromBytes creates a new File instance from the provided byte slice.
+func NewFileFromBytes(b []byte, name string) (*File, error) {
+	size := len(b)
+	if size == 0 {
+		return nil, errors.New("cannot create an empty file")
+	}
+
+	f := &File{}
+
+	f.Reader = &BytesReader{b}
+	f.Size = int64(size)
+	f.OriginalName = name
+	f.Name = normalizeName(f.Reader, f.OriginalName)
+
+	return f, nil
+}
+
+// NewFileFromMultipart creates a new File instace from the provided multipart header.
+func NewFileFromMultipart(mh *multipart.FileHeader) (*File, error) {
+	f := &File{}
+
+	f.Reader = &MultipartReader{Header: mh}
+	f.Size = mh.Size
+	f.OriginalName = mh.Filename
+	f.Name = normalizeName(f.Reader, f.OriginalName)
+
+	return f, nil
+}
+
+// -------------------------------------------------------------------
+
+var _ FileReader = (*MultipartReader)(nil)
+
+// MultipartReader defines a FileReader from [multipart.FileHeader].
+type MultipartReader struct {
+	Header *multipart.FileHeader
+}
+
+// Open implements the [filesystem.FileReader] interface.
+func (r *MultipartReader) Open() (io.ReadSeekCloser, error) {
+	return r.Header.Open()
+}
+
+// -------------------------------------------------------------------
+
+var _ FileReader = (*PathReader)(nil)
+
+// PathReader defines a FileReader from a local file path.
+type PathReader struct {
+	Path string
+}
+
+// Open implements the [filesystem.FileReader] interface.
+func (r *PathReader) Open() (io.ReadSeekCloser, error) {
+	return os.Open(r.Path)
+}
+
+// -------------------------------------------------------------------
+
+var _ FileReader = (*BytesReader)(nil)
+
+// BytesReader defines a FileReader from bytes content.
+type BytesReader struct {
+	Bytes []byte
+}
+
+// Open implements the [filesystem.FileReader] interface.
+func (r *BytesReader) Open() (io.ReadSeekCloser, error) {
+	return &bytesReadSeekCloser{bytes.NewReader(r.Bytes)}, nil
+}
+
+type bytesReadSeekCloser struct {
+	*bytes.Reader
+}
+
+// Close implements the [io.ReadSeekCloser] interface.
+func (r *bytesReadSeekCloser) Close() error {
+	return nil
+}
+
+// -------------------------------------------------------------------
+
+var extInvalidCharsRegex = regexp.MustCompile(`[^\w\.\*\-\+\=\#]+`)
+
+func normalizeName(fr FileReader, name string) string {
+	// extension
+	// ---
+	originalExt := filepath.Ext(name)
+	cleanExt := extInvalidCharsRegex.ReplaceAllString(originalExt, "")
+	if cleanExt == "" {
+		// try to detect the extension from the file content
+		cleanExt, _ = detectExtension(fr)
+	}
+
+	// name
+	// ---
+	cleanName := inflector.Snakecase(strings.TrimSuffix(name, originalExt))
+	if length := len(cleanName); length < 3 {
+		// the name is too short so we concatenate an additional random part
+		cleanName += security.RandomString(10)
+	} else if length > 100 {
+		// keep only the first 100 characters (it is multibyte safe after Snakecase)
+		cleanName = cleanName[:100]
+	}
+
+	return fmt.Sprintf(
+		"%s_%s%s",
+		cleanName,
+		security.RandomString(10), // ensure that there is always a random part
+		cleanExt,
+	)
+}
+
+func detectExtension(fr FileReader) (string, error) {
+	// try to detect the extension from the mime type
+	r, err := fr.Open()
+	if err != nil {
+		return "", err
+	}
+	defer r.Close()
+
+	mt, _ := mimetype.DetectReader(r)
+	if err != nil {
+		return "", err
+	}
+
+	return mt.Extension(), nil
+}

tools/filesystem/file_test.go

@@ -0,0 +1,111 @@
+package filesystem_test
+
+import (
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"regexp"
+	"testing"
+
+	"github.com/labstack/echo/v5"
+	"github.com/pocketbase/pocketbase/tests"
+	"github.com/pocketbase/pocketbase/tools/filesystem"
+)
+
+func TestNewFileFromPath(t *testing.T) {
+	testDir := createTestDir(t)
+	defer os.RemoveAll(testDir)
+
+	// missing file
+	_, err := filesystem.NewFileFromPath("missing")
+	if err == nil {
+		t.Fatal("Expected error, got nil")
+	}
+
+	// existing file
+	originalName := "image_! noext"
+	normalizedNamePattern := regexp.QuoteMeta("image_noext_") + `\w{10}` + regexp.QuoteMeta(".png")
+	f, err := filesystem.NewFileFromPath(filepath.Join(testDir, originalName))
+	if err != nil {
+		t.Fatalf("Expected nil error, got %v", err)
+	}
+	if f.OriginalName != originalName {
+		t.Fatalf("Expected OriginalName %q, got %q", originalName, f.OriginalName)
+	}
+	if match, _ := regexp.Match(normalizedNamePattern, []byte(f.Name)); !match {
+		t.Fatalf("Expected Name to match %v, got %q (%v)", normalizedNamePattern, f.Name, err)
+	}
+	if f.Size != 73 {
+		t.Fatalf("Expected Size %v, got %v", 73, f.Size)
+	}
+	if _, ok := f.Reader.(*filesystem.PathReader); !ok {
+		t.Fatalf("Expected Reader to be PathReader, got %v", f.Reader)
+	}
+}
+
+func TestNewFileFromBytes(t *testing.T) {
+	// nil bytes
+	if _, err := filesystem.NewFileFromBytes(nil, "photo.jpg"); err == nil {
+		t.Fatal("Expected error, got nil")
+	}
+
+	// zero bytes
+	if _, err := filesystem.NewFileFromBytes([]byte{}, "photo.jpg"); err == nil {
+		t.Fatal("Expected error, got nil")
+	}
+
+	originalName := "image_! noext"
+	normalizedNamePattern := regexp.QuoteMeta("image_noext_") + `\w{10}` + regexp.QuoteMeta(".txt")
+	f, err := filesystem.NewFileFromBytes([]byte("text\n"), originalName)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if f.Size != 5 {
+		t.Fatalf("Expected Size %v, got %v", 5, f.Size)
+	}
+	if f.OriginalName != originalName {
+		t.Fatalf("Expected OriginalName %q, got %q", originalName, f.OriginalName)
+	}
+	if match, _ := regexp.Match(normalizedNamePattern, []byte(f.Name)); !match {
+		t.Fatalf("Expected Name to match %v, got %q (%v)", normalizedNamePattern, f.Name, err)
+	}
+}
+
+func TestNewFileFromMultipart(t *testing.T) {
+	formData, mp, err := tests.MockMultipartData(nil, "test")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	req := httptest.NewRequest("", "/", formData)
+	req.Header.Set(echo.HeaderContentType, mp.FormDataContentType())
+	req.ParseMultipartForm(32 << 20)
+
+	_, mh, err := req.FormFile("test")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	f, err := filesystem.NewFileFromMultipart(mh)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	originalNamePattern := regexp.QuoteMeta("tmpfile-") + `\w+` + regexp.QuoteMeta(".txt")
+	if match, _ := regexp.Match(originalNamePattern, []byte(f.OriginalName)); !match {
+		t.Fatalf("Expected OriginalName to match %v, got %q (%v)", originalNamePattern, f.OriginalName, err)
+	}
+
+	normalizedNamePattern := regexp.QuoteMeta("tmpfile_") + `\w+\_\w{10}` + regexp.QuoteMeta(".txt")
+	if match, _ := regexp.Match(normalizedNamePattern, []byte(f.Name)); !match {
+		t.Fatalf("Expected Name to match %v, got %q (%v)", normalizedNamePattern, f.Name, err)
+	}
+
+	if f.Size != 4 {
+		t.Fatalf("Expected Size %v, got %v", 4, f.Size)
+	}
+
+	if _, ok := f.Reader.(*filesystem.MultipartReader); !ok {
+		t.Fatalf("Expected Reader to be MultipartReader, got %v", f.Reader)
+	}
+}

tools/filesystem/filesystem.go

@@ -117,7 +117,49 @@ func (s *System) Upload(content []byte, fileKey string) error {
 	return w.Close()
 }
 
-// UploadMultipart upload the provided multipart file to the fileKey location.
+// UploadFile uploads the provided multipart file to the fileKey location.
+func (s *System) UploadFile(file *File, fileKey string) error {
+	f, err := file.Reader.Open()
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+
+	mt, err := mimetype.DetectReader(f)
+	if err != nil {
+		return err
+	}
+
+	// rewind
+	f.Seek(0, io.SeekStart)
+
+	originalName := file.OriginalName
+	if len(originalName) > 255 {
+		// keep only the first 255 chars as a very rudimentary measure
+		// to prevent the metadata to grow too big in size
+		originalName = originalName[:255]
+	}
+	opts := &blob.WriterOptions{
+		ContentType: mt.String(),
+		Metadata: map[string]string{
+			"original-filename": originalName,
+		},
+	}
+
+	w, err := s.bucket.NewWriter(s.ctx, fileKey, opts)
+	if err != nil {
+		return err
+	}
+
+	if _, err := w.ReadFrom(f); err != nil {
+		w.Close()
+		return err
+	}
+
+	return w.Close()
+}
+
+// UploadMultipart uploads the provided multipart file to the fileKey location.
 func (s *System) UploadMultipart(fh *multipart.FileHeader, fileKey string) error {
 	f, err := fh.Open()
 	if err != nil {
@@ -142,7 +184,7 @@ func (s *System) UploadMultipart(fh *multipart.FileHeader, fileKey string) error
 	opts := &blob.WriterOptions{
 		ContentType: mt.String(),
 		Metadata: map[string]string{
-			"original_filename": originalName,
+			"original-filename": originalName,
 		},
 	}
 
@@ -183,13 +225,11 @@ func (s *System) DeletePrefix(prefix string) []error {
 	})
 	for {
 		obj, err := iter.Next(s.ctx)
-		if err == io.EOF {
-			break
-		}
-
 		if err != nil {
-			failed = append(failed, err)
-			continue
+			if err != io.EOF {
+				failed = append(failed, err)
+			}
+			break
 		}
 
 		if err := s.Delete(obj.Key); err != nil {

tools/filesystem/filesystem_test.go

@@ -170,15 +170,52 @@ func TestFileSystemUploadMultipart(t *testing.T) {
 	}
 
 	if exists, _ := fs.Exists(fileKey); !exists {
-		t.Fatalf("Expected newdir/newkey.txt to exist")
+		t.Fatalf("Expected %q to exist", fileKey)
 	}
 
 	attrs, err := fs.Attributes(fileKey)
 	if err != nil {
 		t.Fatalf("Failed to fetch file attributes: %v", err)
 	}
-	if name, ok := attrs.Metadata["original_filename"]; !ok || name != "test" {
-		t.Fatalf("Expected original_filename to be %q, got %q", "test", name)
+	if name, ok := attrs.Metadata["original-filename"]; !ok || name != "test" {
+		t.Fatalf("Expected original-filename to be %q, got %q", "test", name)
+	}
+}
+
+func TestFileSystemUploadFile(t *testing.T) {
+	dir := createTestDir(t)
+	defer os.RemoveAll(dir)
+
+	fs, err := filesystem.NewLocal(dir)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer fs.Close()
+
+	fileKey := "newdir/newkey.txt"
+
+	file, err := filesystem.NewFileFromPath(filepath.Join(dir, "image.svg"))
+	if err != nil {
+		t.Fatalf("Failed to load test file: %v", err)
+	}
+
+	file.OriginalName = "test.txt"
+
+	uploadErr := fs.UploadFile(file, fileKey)
+	if uploadErr != nil {
+		t.Fatal(uploadErr)
+	}
+
+	if exists, _ := fs.Exists(fileKey); !exists {
+		t.Fatalf("Expected %q to exist", fileKey)
+	}
+
+	attrs, err := fs.Attributes(fileKey)
+	if err != nil {
+		t.Fatalf("Failed to fetch file attributes: %v", err)
+	}
+	if name, ok := attrs.Metadata["original-filename"]; !ok || name != file.OriginalName {
+		t.Fatalf("Expected original-filename to be %q, got %q", file.OriginalName, name)
 	}
 }
 
@@ -192,13 +229,15 @@ func TestFileSystemUpload(t *testing.T) {
 	}
 	defer fs.Close()
 
-	uploadErr := fs.Upload([]byte("demo"), "newdir/newkey.txt")
+	fileKey := "newdir/newkey.txt"
+
+	uploadErr := fs.Upload([]byte("demo"), fileKey)
 	if uploadErr != nil {
 		t.Fatal(uploadErr)
 	}
 
-	if exists, _ := fs.Exists("newdir/newkey.txt"); !exists {
-		t.Fatalf("Expected newdir/newkey.txt to exist")
+	if exists, _ := fs.Exists(fileKey); !exists {
+		t.Fatalf("Expected %s to exist", fileKey)
 	}
 }
 
@@ -448,8 +487,7 @@ func createTestDir(t *testing.T) string {
 	if err != nil {
 		t.Fatal(err)
 	}
-	// tiny 1x1 png
-	imgRect := image.Rect(0, 0, 1, 1)
+	imgRect := image.Rect(0, 0, 1, 1) // tiny 1x1 png
 	png.Encode(file3, imgRect)
 	file3.Close()
 	err2 := os.WriteFile(filepath.Join(dir, "image.png.attrs"), []byte(`{"user.cache_control":"","user.content_disposition":"","user.content_encoding":"","user.content_language":"","user.content_type":"image/png","user.metadata":null}`), 0644)
@@ -469,5 +507,12 @@ func createTestDir(t *testing.T) string {
 	}
 	file5.Close()
 
+	file6, err := os.OpenFile(filepath.Join(dir, "image_! noext"), os.O_WRONLY|os.O_CREATE, 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	png.Encode(file6, image.Rect(0, 0, 1, 1)) // tiny 1x1 png
+	file6.Close()
+
 	return dir
 }

tools/list/list.go

@@ -10,6 +10,20 @@ import (
 
 var cachedPatterns = map[string]*regexp.Regexp{}
 
+// SubtractSlice returns a new slice with only the "base" elements
+// that don't exist in "subtract".
+func SubtractSlice[T comparable](base []T, subtract []T) []T {
+	var result = make([]T, 0, len(base))
+
+	for _, b := range base {
+		if !ExistInSlice(b, subtract) {
+			result = append(result, b)
+		}
+	}
+
+	return result
+}
+
 // ExistInSlice checks whether a comparable element exists in a slice of the same type.
 func ExistInSlice[T comparable](item T, list []T) bool {
 	if len(list) == 0 {
@@ -80,7 +94,10 @@ func NonzeroUniques[T comparable](list []T) []T {
 	var zeroVal T
 
 	for _, val := range list {
-		if _, ok := existMap[val]; ok || val == zeroVal {
+		if val == zeroVal {
+			continue
+		}
+		if _, ok := existMap[val]; ok {
 			continue
 		}
 		existMap[val] = struct{}{}
@@ -103,8 +120,13 @@ func ToUniqueStringSlice(value any) (result []string) {
 		}
 
 		// check if it is a json encoded array of strings
-		if err := json.Unmarshal([]byte(val), &result); err != nil {
-			// not a json array, just add the string as single array element
+		if strings.Contains(val, "[") {
+			if err := json.Unmarshal([]byte(val), &result); err != nil {
+				// not a json array, just add the string as single array element
+				result = append(result, val)
+			}
+		} else {
+			// just add the string as single array element
 			result = append(result, val)
 		}
 	case json.Marshaler: // eg. JsonArray

tools/list/list_test.go

@@ -1,12 +1,111 @@
 package list_test
 
 import (
+	"encoding/json"
 	"testing"
 
 	"github.com/pocketbase/pocketbase/tools/list"
 	"github.com/pocketbase/pocketbase/tools/types"
 )
 
+func TestSubtractSliceString(t *testing.T) {
+	scenarios := []struct {
+		base     []string
+		subtract []string
+		expected string
+	}{
+		{
+			[]string{},
+			[]string{},
+			`[]`,
+		},
+		{
+			[]string{},
+			[]string{"1", "2", "3", "4"},
+			`[]`,
+		},
+		{
+			[]string{"1", "2", "3", "4"},
+			[]string{},
+			`["1","2","3","4"]`,
+		},
+		{
+			[]string{"1", "2", "3", "4"},
+			[]string{"1", "2", "3", "4"},
+			`[]`,
+		},
+		{
+			[]string{"1", "2", "3", "4", "7"},
+			[]string{"2", "4", "5", "6"},
+			`["1","3","7"]`,
+		},
+	}
+
+	for i, s := range scenarios {
+		result := list.SubtractSlice(s.base, s.subtract)
+
+		raw, err := json.Marshal(result)
+		if err != nil {
+			t.Fatalf("(%d) Failed to serialize: %v", i, err)
+		}
+
+		strResult := string(raw)
+
+		if strResult != s.expected {
+			t.Fatalf("(%d) Expected %v, got %v", i, s.expected, strResult)
+		}
+	}
+}
+
+func TestSubtractSliceInt(t *testing.T) {
+	scenarios := []struct {
+		base     []int
+		subtract []int
+		expected string
+	}{
+		{
+			[]int{},
+			[]int{},
+			`[]`,
+		},
+		{
+			[]int{},
+			[]int{1, 2, 3, 4},
+			`[]`,
+		},
+		{
+			[]int{1, 2, 3, 4},
+			[]int{},
+			`[1,2,3,4]`,
+		},
+		{
+			[]int{1, 2, 3, 4},
+			[]int{1, 2, 3, 4},
+			`[]`,
+		},
+		{
+			[]int{1, 2, 3, 4, 7},
+			[]int{2, 4, 5, 6},
+			`[1,3,7]`,
+		},
+	}
+
+	for i, s := range scenarios {
+		result := list.SubtractSlice(s.base, s.subtract)
+
+		raw, err := json.Marshal(result)
+		if err != nil {
+			t.Fatalf("(%d) Failed to serialize: %v", i, err)
+		}
+
+		strResult := string(raw)
+
+		if strResult != s.expected {
+			t.Fatalf("(%d) Expected %v, got %v", i, s.expected, strResult)
+		}
+	}
+}
+
 func TestExistInSliceString(t *testing.T) {
 	scenarios := []struct {
 		item     string

tools/mailer/sendmail.go

@@ -10,8 +10,8 @@ import (
 
 var _ Mailer = (*Sendmail)(nil)
 
-// Sendmail implements `mailer.Mailer` interface and defines a mail
-// client that sends emails via the `sendmail` *nix command.
+// Sendmail implements [mailer.Mailer] interface and defines a mail
+// client that sends emails via the "sendmail" *nix command.
 //
 // This client is usually recommended only for development and testing.
 type Sendmail struct {

tools/mailer/smtp.go

@@ -1,6 +1,7 @@
 package mailer
 
 import (
+	"errors"
 	"fmt"
 	"net/smtp"
 	"strings"
@@ -11,7 +12,14 @@ import (
 
 var _ Mailer = (*SmtpClient)(nil)
 
-// NewSmtpClient creates new `SmtpClient` with the provided configuration.
+const (
+	SmtpAuthPlain = "PLAIN"
+	SmtpAuthLogin = "LOGIN"
+)
+
+// Deprecated: Use directly the SmtpClient struct literal.
+//
+// NewSmtpClient creates new SmtpClient with the provided configuration.
 func NewSmtpClient(
 	host string,
 	port int,
@@ -20,41 +28,47 @@ func NewSmtpClient(
 	tls bool,
 ) *SmtpClient {
 	return &SmtpClient{
-		host:     host,
-		port:     port,
-		username: username,
-		password: password,
-		tls:      tls,
+		Host:     host,
+		Port:     port,
+		Username: username,
+		Password: password,
+		Tls:      tls,
 	}
 }
 
 // SmtpClient defines a SMTP mail client structure that implements
 // `mailer.Mailer` interface.
 type SmtpClient struct {
-	host     string
-	port     int
-	username string
-	password string
-	tls      bool
+	Host       string
+	Port       int
+	Username   string
+	Password   string
+	Tls        bool
+	AuthMethod string // default to "PLAIN"
 }
 
 // Send implements `mailer.Mailer` interface.
 func (c *SmtpClient) Send(m *Message) error {
 	var smtpAuth smtp.Auth
-	if c.username != "" || c.password != "" {
-		smtpAuth = smtp.PlainAuth("", c.username, c.password, c.host)
+	if c.Username != "" || c.Password != "" {
+		switch c.AuthMethod {
+		case SmtpAuthLogin:
+			smtpAuth = &smtpLoginAuth{c.Username, c.Password}
+		default:
+			smtpAuth = smtp.PlainAuth("", c.Username, c.Password, c.Host)
+		}
 	}
 
 	// create mail instance
 	var yak *mailyak.MailYak
-	if c.tls {
+	if c.Tls {
 		var tlsErr error
-		yak, tlsErr = mailyak.NewWithTLS(fmt.Sprintf("%s:%d", c.host, c.port), smtpAuth, nil)
+		yak, tlsErr = mailyak.NewWithTLS(fmt.Sprintf("%s:%d", c.Host, c.Port), smtpAuth, nil)
 		if tlsErr != nil {
 			return tlsErr
 		}
 	} else {
-		yak = mailyak.New(fmt.Sprintf("%s:%d", c.host, c.port), smtpAuth)
+		yak = mailyak.New(fmt.Sprintf("%s:%d", c.Host, c.Port), smtpAuth)
 	}
 
 	if m.From.Name != "" {
@@ -108,3 +122,59 @@ func (c *SmtpClient) Send(m *Message) error {
 
 	return yak.Send()
 }
+
+// -------------------------------------------------------------------
+// AUTH LOGIN
+// -------------------------------------------------------------------
+
+var _ smtp.Auth = (*smtpLoginAuth)(nil)
+
+// smtpLoginAuth defines an AUTH that implements the LOGIN authentication mechanism.
+//
+// AUTH LOGIN is obsolete[1] but some mail services like outlook requires it [2].
+//
+// NB!
+// It will only send the credentials if the connection is using TLS or is connected to localhost.
+// Otherwise authentication will fail with an error, without sending the credentials.
+//
+// [1]: https://github.com/golang/go/issues/40817
+// [2]: https://support.microsoft.com/en-us/office/outlook-com-no-longer-supports-auth-plain-authentication-07f7d5e9-1697-465f-84d2-4513d4ff0145?ui=en-us&rs=en-us&ad=us
+type smtpLoginAuth struct {
+	username, password string
+}
+
+// Start initializes an authentication with the server.
+//
+// It is part of the [smtp.Auth] interface.
+func (a *smtpLoginAuth) Start(server *smtp.ServerInfo) (string, []byte, error) {
+	// Must have TLS, or else localhost server.
+	// Note: If TLS is not true, then we can't trust ANYTHING in ServerInfo.
+	// In particular, it doesn't matter if the server advertises LOGIN auth.
+	// That might just be the attacker saying
+	// "it's ok, you can trust me with your password."
+	if !server.TLS && !isLocalhost(server.Name) {
+		return "", nil, errors.New("unencrypted connection")
+	}
+
+	return "LOGIN", nil, nil
+}
+
+// Next "continues" the auth process by feeding the server with the requested data.
+//
+// It is part of the [smtp.Auth] interface.
+func (a *smtpLoginAuth) Next(fromServer []byte, more bool) ([]byte, error) {
+	if more {
+		switch strings.ToLower(string(fromServer)) {
+		case "username:":
+			return []byte(a.username), nil
+		case "password:":
+			return []byte(a.password), nil
+		}
+	}
+
+	return nil, nil
+}
+
+func isLocalhost(name string) bool {
+	return name == "localhost" || name == "127.0.0.1" || name == "::1"
+}