mirror of
https://github.com/pocketbase/pocketbase.git
synced 2026-09-20 13:30:50 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5cec579da9 | ||
|
|
5f496665c7 | ||
|
|
adf4eb822f | ||
|
|
114c01ac12 | ||
|
|
5684ee24f1 | ||
|
|
b3de909790 | ||
|
|
a53f189f74 | ||
|
|
7ff7e0f043 | ||
|
|
6e29284e6a | ||
|
|
1984952e4e | ||
|
|
f611256d90 | ||
|
|
10f4f28312 | ||
|
|
44f817e7f1 | ||
|
|
75c6a4fd1f | ||
|
|
57c0b034e6 | ||
|
|
4709f63183 | ||
|
|
53a6cd04e2 | ||
|
|
df4e6eeb35 | ||
|
|
dbea21f1b2 | ||
|
|
97f9d63a1b | ||
|
|
bf12ae0244 | ||
|
|
7ac159b61b | ||
|
|
8213ccffb4 | ||
|
|
e16a9f8b21 | ||
|
|
0206f6135c | ||
|
|
e417a26a05 | ||
|
|
ac4c30ccbc | ||
|
|
fed5ab9a13 | ||
|
|
6d9ca9bf9d | ||
|
|
56f1d1dfdd | ||
|
|
1b3edbbf5c | ||
|
|
522f9e0ab0 | ||
|
|
3c9427667d | ||
|
|
fd5b9e5f75 | ||
|
|
d0b441bc24 | ||
|
|
229240b374 | ||
|
|
f694893d31 | ||
|
|
8cb486f24a | ||
|
|
bc8ffed4e7 | ||
|
|
da18302785 | ||
|
|
b402f11a79 | ||
|
|
9f4f4d4e73 | ||
|
|
d763d3dff1 | ||
|
|
04ed202c78 | ||
|
|
756b326bfa | ||
|
|
50f5f83ace | ||
|
|
70d585759d | ||
|
|
9e15b464ba | ||
|
|
129d46f4c8 | ||
|
|
eae29c83a8 | ||
|
|
c532fd19b6 | ||
|
|
eade68d4a3 | ||
|
|
18cb3dc62f | ||
|
|
a293e1301d | ||
|
|
7f4b532f06 | ||
|
|
5e5ba383e8 | ||
|
|
b648b951b0 | ||
|
|
f544fa0c81 | ||
|
|
80608e8687 | ||
|
|
654af12d17 | ||
|
|
0e9dcfe1ab | ||
|
|
25dc8a8eea | ||
|
|
bf1f164014 | ||
|
|
50a7700450 | ||
|
|
252358e036 | ||
|
|
c2cbd9707f | ||
|
|
e71be7194a | ||
|
|
1a5f29818c | ||
|
|
25081f14eb | ||
|
|
f5695a7308 | ||
|
|
bad7cc00a1 | ||
|
|
14fc20a851 | ||
|
|
97dd775455 | ||
|
|
4d4275c9aa | ||
|
|
1247b24236 | ||
|
|
5d217ddb50 | ||
|
|
e0169684c4 | ||
|
|
f7229c502f | ||
|
|
82368a6152 | ||
|
|
d744647929 | ||
|
|
c86fa8890d | ||
|
|
3668e1c431 | ||
|
|
a4f1fa33d1 | ||
|
|
2262bb4cae | ||
|
|
6212276c82 | ||
|
|
9f1a4bcfa5 | ||
|
|
32fa2ddf77 | ||
|
|
dc7482ea67 | ||
|
|
b259534df7 | ||
|
|
4c6c166603 | ||
|
|
f9795da8c7 | ||
|
|
200c19b26f | ||
|
|
fd25a8ae32 | ||
|
|
4dc4387638 | ||
|
|
4076537a2c | ||
|
|
0a74d2f25d | ||
|
|
7789297065 | ||
|
|
17628e554c | ||
|
|
8ec80c9828 | ||
|
|
e83e911b2f | ||
|
|
6b00152847 | ||
|
|
9f32dfa7ba | ||
|
|
0cbfc046c7 | ||
|
|
d02016c5fd | ||
|
|
9425a5c82f | ||
|
|
244ae1a162 | ||
|
|
8af99615c4 |
@@ -26,9 +26,9 @@ jobs:
|
||||
node-version: '>=25.2.1'
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v6
|
||||
uses: actions/setup-go@v7
|
||||
with:
|
||||
go-version: '>=1.26.5'
|
||||
go-version: '>=1.27.1'
|
||||
|
||||
# This step usually is not needed because the /ui/dist is pregenerated locally
|
||||
# but its here to ensure that each release embeds the latest admin ui artifacts.
|
||||
|
||||
+50
-1173
File diff suppressed because it is too large
Load Diff
@@ -2,6 +2,44 @@
|
||||
> For the most recent versions, please refer to [CHANGELOG.md](./CHANGELOG.md)
|
||||
---
|
||||
|
||||
## v0.22.55
|
||||
|
||||
- (_Backported from v0.40.3_) Fixed collection index validator to allow expressions with parenthesis in the optional `WHERE` clause.
|
||||
|
||||
- (_Backported from v0.40.3_) Fixed nested cascade delete of self-referenced relation records.
|
||||
|
||||
- (_Backported from v0.40.3_) Bumped `golang.org/x/*` dependencies to silence security scanners ([#7829](https://github.com/pocketbase/pocketbase/discussions/7829)).
|
||||
|
||||
|
||||
## v0.22.54
|
||||
|
||||
- (_Backported from v0.40.2_) Bumped goja and its related dependencies _(regex unescaped dash error fix and base64 optimizations)_.
|
||||
|
||||
- (_Backported from v0.40.2_) Return an error when filter params fallback fails to json serialize and optimized params replacement to execute in a single pass.
|
||||
|
||||
- (_Backported from v0.40.2_) Bumped the min Go GitHub action version to 1.27.1 as it includes some [minor `database/sql` and `enconding/json/v2` bug fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.27.1).
|
||||
|
||||
|
||||
## v0.22.53
|
||||
|
||||
- Workaround `encoding/json` infinite recursion when used with Go 1.27 ([#7808](https://github.com/pocketbase/pocketbase/issues/7808)).
|
||||
|
||||
|
||||
## v0.22.52
|
||||
|
||||
- (_Backported from v0.39.11_) Bumped `golang.org/x/*` deps and the min Go GitHub action version to 1.26.6 because it comes with some [minor bug and security fixes](https://github.com/golang/go/issues?q=milestone%3AGo1.26.6).
|
||||
|
||||
|
||||
## v0.22.51
|
||||
|
||||
- (_Backported from v0.39.10_) Reverted the auto panic recover handling for the cli commands to preserve the old behavior and allow panic to force exit with non-zero code ([#7781](https://github.com/pocketbase/pocketbase/issues/7781)).
|
||||
|
||||
|
||||
## v0.22.50
|
||||
|
||||
- (_Backported from v0.39.9_) Bumped goja, fexpr and their related deps.
|
||||
|
||||
|
||||
## v0.22.49
|
||||
|
||||
- (_Backported from v0.39.8_) Bumped `golang.org/x/*` indirect dependencies as there are some minor security fixes.
|
||||
|
||||
+1243
File diff suppressed because it is too large
Load Diff
+1
-1
@@ -13,7 +13,7 @@ This document describes how to prepare a PR for a change in the main repository.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Go 1.25+ (for making changes in the Go code)
|
||||
- Go 1.27+ (for making changes in the Go code)
|
||||
- Node 24+ (for making changes in the Superuser UI)
|
||||
|
||||
If you haven't already, you can fork the main repository and clone your fork so that you can work locally:
|
||||
|
||||
@@ -49,7 +49,7 @@ your own custom app specific business logic and still have a single portable exe
|
||||
|
||||
Here is a minimal example:
|
||||
|
||||
0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
|
||||
0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
|
||||
|
||||
1. Create a new project directory with the following `main.go` file inside it:
|
||||
```go
|
||||
@@ -92,32 +92,37 @@ _For more details please refer to [Extend with Go](https://pocketbase.io/docs/go
|
||||
|
||||
To build the minimal standalone executable, like the prebuilt ones in the releases page, you can simply run `go build` inside the `examples/base` directory:
|
||||
|
||||
0. [Install Go 1.25+](https://go.dev/doc/install) (_if you haven't already_)
|
||||
0. [Install Go 1.27+](https://go.dev/doc/install) (_if you haven't already_)
|
||||
1. Clone/download the repo
|
||||
2. Navigate to `examples/base`
|
||||
3. Run `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`
|
||||
(_https://go.dev/doc/install/source#environment_)
|
||||
3. Run `CGO_ENABLED=0 go build` to build a binary for your current environment
|
||||
_(or to target other platforms use `GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build`; see https://go.dev/doc/install/source#environment)_
|
||||
4. Start the created executable by running `./base serve`.
|
||||
|
||||
Note that the supported build targets by the pure Go SQLite driver at the moment are:
|
||||
|
||||
```
|
||||
darwin amd64
|
||||
darwin arm64
|
||||
freebsd amd64
|
||||
freebsd arm64
|
||||
linux 386
|
||||
linux amd64
|
||||
linux arm
|
||||
linux arm64
|
||||
linux loong64
|
||||
linux ppc64le
|
||||
linux riscv64
|
||||
linux s390x
|
||||
windows 386
|
||||
windows amd64
|
||||
windows arm64
|
||||
```
|
||||
| GOOS | GOARCH |
|
||||
|---------|---------|
|
||||
| darwin | amd64 |
|
||||
| darwin | arm64 |
|
||||
| freebsd | 386 |
|
||||
| freebsd | amd64 |
|
||||
| freebsd | arm |
|
||||
| freebsd | arm64 |
|
||||
| linux | 386 |
|
||||
| linux | amd64 |
|
||||
| linux | arm |
|
||||
| linux | arm64 |
|
||||
| linux | loong64 |
|
||||
| linux | ppc64le |
|
||||
| linux | riscv64 |
|
||||
| linux | s390x |
|
||||
| netbsd | amd64 |
|
||||
| openbsd | amd64 |
|
||||
| openbsd | arm64 |
|
||||
| windows | 386 |
|
||||
| windows | amd64 |
|
||||
| windows | arm64 |
|
||||
|
||||
### Testing
|
||||
|
||||
@@ -134,7 +139,7 @@ Check also the [Testing guide](http://pocketbase.io/docs/testing) to learn how t
|
||||
|
||||
If you discover a security vulnerability within PocketBase, please send an e-mail to **support at pocketbase.io**.
|
||||
|
||||
All reports will be promptly addressed and you'll be credited in the fix release notes.
|
||||
You could find more details in the project [Security policy](https://github.com/pocketbase/pocketbase/security/policy).
|
||||
|
||||
## Contributing
|
||||
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package apis
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
|
||||
+66
-1
@@ -1225,6 +1225,71 @@ func TestCollectionUpdate(t *testing.T) {
|
||||
"OnModelValidate": 1,
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "add another OAuth2 provider to an auth collection",
|
||||
Method: http.MethodPatch,
|
||||
URL: "/api/collections/users",
|
||||
Body: strings.NewReader(`{
|
||||
"oauth2": {
|
||||
"providers": [
|
||||
{"name": "apple", "clientId": "a", "clientSecret": "b"},
|
||||
{
|
||||
"pkce": null,
|
||||
"name": "google",
|
||||
"authURL": "",
|
||||
"displayName": "existing",
|
||||
"extra": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
}`),
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
|
||||
},
|
||||
BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
|
||||
// verify that the collection has google and gitlab as OAuth2 providers
|
||||
users, err := app.FindCollectionByNameOrId("users")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if v := len(users.OAuth2.Providers); v != 2 {
|
||||
t.Fatalf("Expected 2 OAuth2 providers, got %d", v)
|
||||
}
|
||||
|
||||
if v := users.OAuth2.Providers[0].Name; v != "gitlab" {
|
||||
t.Fatalf("Expected provider 0 to be %s, got %s", "gitlab", v)
|
||||
}
|
||||
|
||||
if v := users.OAuth2.Providers[1].Name; v != "google" {
|
||||
t.Fatalf("Expected provider 1 to be %s, got %s", "google", v)
|
||||
}
|
||||
},
|
||||
ExpectedStatus: 200,
|
||||
ExpectedContent: []string{
|
||||
`"name":"google"`,
|
||||
`"name":"apple"`,
|
||||
`"displayName":"existing"`,
|
||||
`"clientId":"test"`,
|
||||
`"clientId":"a"`,
|
||||
},
|
||||
NotExpectedContent: []string{
|
||||
`"name":"gitlab"`,
|
||||
`clientSecret`,
|
||||
},
|
||||
ExpectedEvents: map[string]int{
|
||||
"*": 0,
|
||||
"OnCollectionUpdateRequest": 1,
|
||||
"OnCollectionUpdate": 1,
|
||||
"OnCollectionUpdateExecute": 1,
|
||||
"OnCollectionAfterUpdateSuccess": 1,
|
||||
"OnCollectionValidate": 1,
|
||||
"OnModelUpdate": 1,
|
||||
"OnModelUpdateExecute": 1,
|
||||
"OnModelAfterUpdateSuccess": 1,
|
||||
"OnModelValidate": 1,
|
||||
},
|
||||
},
|
||||
|
||||
// view
|
||||
// -----------------------------------------------------------
|
||||
@@ -1617,7 +1682,7 @@ func TestCollectionOAuth2Providers(t *testing.T) {
|
||||
},
|
||||
ExpectedStatus: 200,
|
||||
ExpectedContent: []string{
|
||||
`{"name":"oidc3","displayName":"OIDC","logo":"\u003csvg`,
|
||||
`{"name":"oidc3","displayName":"OIDC","logo":"<svg`,
|
||||
},
|
||||
NotExpectedContent: []string{
|
||||
`"order":`,
|
||||
|
||||
+3
-4
@@ -77,10 +77,13 @@ func TestCronsRun(t *testing.T) {
|
||||
app.Cron().Add("test", "* * * * *", func() {
|
||||
app.Store().Set("testJobCalls", cast.ToInt(app.Store().Get("testJobCalls"))+1)
|
||||
})
|
||||
app.Cron().Stop()
|
||||
}
|
||||
|
||||
expectedCalls := func(expected int) func(t testing.TB, app *tests.TestApp, res *http.Response) {
|
||||
return func(t testing.TB, app *tests.TestApp, res *http.Response) {
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
|
||||
total := cast.ToInt(app.Store().Get("testJobCalls"))
|
||||
if total != expected {
|
||||
t.Fatalf("Expected total testJobCalls %d, got %d", expected, total)
|
||||
@@ -93,7 +96,6 @@ func TestCronsRun(t *testing.T) {
|
||||
Name: "unauthorized",
|
||||
Method: http.MethodPost,
|
||||
URL: "/api/crons/test",
|
||||
Delay: 50 * time.Millisecond,
|
||||
BeforeTestFunc: beforeTestFunc,
|
||||
AfterTestFunc: expectedCalls(0),
|
||||
ExpectedStatus: 401,
|
||||
@@ -107,7 +109,6 @@ func TestCronsRun(t *testing.T) {
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
|
||||
},
|
||||
Delay: 50 * time.Millisecond,
|
||||
BeforeTestFunc: beforeTestFunc,
|
||||
AfterTestFunc: expectedCalls(0),
|
||||
ExpectedStatus: 403,
|
||||
@@ -121,7 +122,6 @@ func TestCronsRun(t *testing.T) {
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
|
||||
},
|
||||
Delay: 50 * time.Millisecond,
|
||||
BeforeTestFunc: beforeTestFunc,
|
||||
AfterTestFunc: expectedCalls(0),
|
||||
ExpectedStatus: 404,
|
||||
@@ -135,7 +135,6 @@ func TestCronsRun(t *testing.T) {
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
|
||||
},
|
||||
Delay: 50 * time.Millisecond,
|
||||
BeforeTestFunc: beforeTestFunc,
|
||||
AfterTestFunc: expectedCalls(1),
|
||||
ExpectedStatus: 204,
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ func healthCheck(e *core.RequestEvent) error {
|
||||
headersToCheck := append(
|
||||
slices.Clone(e.App.Settings().TrustedProxy.Headers),
|
||||
// common proxy headers
|
||||
"CF-Connecting-IP", "Fly-Client-IP", "X‑Forwarded-For",
|
||||
"CF-Connecting-IP", "Fly-Client-IP", "X-Forwarded-For",
|
||||
)
|
||||
for _, header := range headersToCheck {
|
||||
if e.Request.Header.Get(header) != "" {
|
||||
|
||||
+22
-1
@@ -11,8 +11,13 @@ import (
|
||||
|
||||
// bindLogsApi registers the request logs api endpoints.
|
||||
func bindLogsApi(app core.App, rg *router.RouterGroup[*core.RequestEvent]) {
|
||||
sub := rg.Group("/logs").Bind(RequireSuperuserAuth(), SkipSuccessActivityLog())
|
||||
sub := rg.Group("/logs").Bind(
|
||||
RequireSuperuserAuth(),
|
||||
SkipSuccessActivityLog(),
|
||||
)
|
||||
|
||||
sub.GET("", logsList)
|
||||
sub.DELETE("", logsTruncate)
|
||||
sub.GET("/stats", logsStats)
|
||||
sub.GET("/{id}", logsView)
|
||||
}
|
||||
@@ -71,3 +76,19 @@ func logsView(e *core.RequestEvent) error {
|
||||
|
||||
return e.JSON(http.StatusOK, log)
|
||||
}
|
||||
|
||||
func logsTruncate(e *core.RequestEvent) error {
|
||||
// delete all rows directly (aka. no model hooks will be fired)
|
||||
_, err := e.App.AuxNonconcurrentDB().Delete((&core.Log{}).TableName(), nil).Execute()
|
||||
if err != nil {
|
||||
return e.InternalServerError("Failed to truncate all logs.", err)
|
||||
}
|
||||
|
||||
// try to free the unused disk space
|
||||
err = e.App.AuxVacuum()
|
||||
if err != nil {
|
||||
e.App.Logger().Warn("Failed to VACUUM aux database", "error", err)
|
||||
}
|
||||
|
||||
return e.NoContent(http.StatusNoContent)
|
||||
}
|
||||
|
||||
@@ -210,3 +210,59 @@ func TestLogsStats(t *testing.T) {
|
||||
scenario.Test(t)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLogsTruncate(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
scenarios := []tests.ApiScenario{
|
||||
{
|
||||
Name: "unauthorized",
|
||||
Method: http.MethodDelete,
|
||||
URL: "/api/logs",
|
||||
ExpectedStatus: 401,
|
||||
ExpectedContent: []string{`"data":{}`},
|
||||
ExpectedEvents: map[string]int{"*": 0},
|
||||
},
|
||||
{
|
||||
Name: "authorized as regular user",
|
||||
Method: http.MethodDelete,
|
||||
URL: "/api/logs",
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6IjRxMXhsY2xtZmxva3UzMyIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoiX3BiX3VzZXJzX2F1dGhfIiwiZXhwIjoyNTI0NjA0NDYxLCJyZWZyZXNoYWJsZSI6dHJ1ZX0.ZT3F0Z3iM-xbGgSG3LEKiEzHrPHr8t8IuHLZGGNuxLo",
|
||||
},
|
||||
ExpectedStatus: 403,
|
||||
ExpectedContent: []string{`"data":{}`},
|
||||
ExpectedEvents: map[string]int{"*": 0},
|
||||
},
|
||||
{
|
||||
Name: "authorized as superuser",
|
||||
Method: http.MethodDelete,
|
||||
URL: "/api/logs",
|
||||
Headers: map[string]string{
|
||||
"Authorization": "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6InN5d2JoZWNuaDQ2cmhtMCIsInR5cGUiOiJhdXRoIiwiY29sbGVjdGlvbklkIjoicGJjXzMxNDI2MzU4MjMiLCJleHAiOjI1MjQ2MDQ0NjEsInJlZnJlc2hhYmxlIjp0cnVlfQ.UXgO3j-0BumcugrFjbd7j0M4MQvbrLggLlcu_YNGjoY",
|
||||
},
|
||||
BeforeTestFunc: func(t testing.TB, app *tests.TestApp, e *core.ServeEvent) {
|
||||
if err := tests.StubLogsData(app); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
},
|
||||
AfterTestFunc: func(t testing.TB, app *tests.TestApp, res *http.Response) {
|
||||
var found []core.Log
|
||||
|
||||
if err := app.LogQuery().All(&found); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if len(found) > 0 {
|
||||
t.Fatalf("Expected all logs to be deleted, found: %v", found)
|
||||
}
|
||||
},
|
||||
ExpectedStatus: 204,
|
||||
ExpectedEvents: map[string]int{"*": 0},
|
||||
},
|
||||
}
|
||||
|
||||
for _, scenario := range scenarios {
|
||||
scenario.Test(t)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -293,6 +293,7 @@ func securityHeaders() *hook.Handler[*core.RequestEvent] {
|
||||
e.Response.Header().Set("X-XSS-Protection", "1; mode=block")
|
||||
e.Response.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
e.Response.Header().Set("X-Frame-Options", "SAMEORIGIN")
|
||||
e.Response.Header().Set("Cross-Origin-Opener-Policy", "same-origin")
|
||||
|
||||
// @todo consider a default HSTS?
|
||||
// (see also https://webkit.org/blog/8146/protecting-against-hsts-abuse/)
|
||||
|
||||
@@ -84,49 +84,73 @@ func applyBodyLimit(e *core.RequestEvent, limitBytes int64) error {
|
||||
}
|
||||
|
||||
// replace the request body
|
||||
//
|
||||
// note: we don't use sync.Pool since the size of the elements could vary too much
|
||||
// and it might not be efficient (see https://github.com/golang/go/issues/23199)
|
||||
e.Request.Body = &limitedReader{ReadCloser: e.Request.Body, limit: limitBytes}
|
||||
e.Request.Body = newMaxBytesReader(e.Request.Body, limitBytes)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
type limitedReader struct {
|
||||
io.ReadCloser
|
||||
limit int64
|
||||
totalRead int64
|
||||
func newMaxBytesReader(body io.ReadCloser, limitBytes int64) *maxBytesReader {
|
||||
return &maxBytesReader{
|
||||
ReadCloser: body,
|
||||
limit: limitBytes,
|
||||
remaining: limitBytes,
|
||||
}
|
||||
}
|
||||
|
||||
func (r *limitedReader) Read(b []byte) (int, error) {
|
||||
// maxBytesReader is very similar to the http.MaxBytesReader but support
|
||||
// rereads and doesn't try to prematurely close the related response
|
||||
// to allow consequent middlewares to operate correctly.
|
||||
type maxBytesReader struct {
|
||||
io.ReadCloser
|
||||
limit int64
|
||||
remaining int64
|
||||
stickyErr error
|
||||
}
|
||||
|
||||
func (r *maxBytesReader) Read(b []byte) (int, error) {
|
||||
if r.stickyErr != nil {
|
||||
return 0, r.stickyErr
|
||||
}
|
||||
|
||||
if len(b) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
// if possible no need to read the entire chunk since
|
||||
// remaining+1 is enough to determine whether it exceed the limit
|
||||
if int64(len(b))-1 > r.remaining {
|
||||
b = b[:r.remaining+1]
|
||||
}
|
||||
|
||||
n, err := r.ReadCloser.Read(b)
|
||||
if err != nil {
|
||||
|
||||
if int64(n) <= r.remaining {
|
||||
r.remaining -= int64(n)
|
||||
r.stickyErr = err
|
||||
return n, err
|
||||
}
|
||||
|
||||
r.totalRead += int64(n)
|
||||
if r.totalRead > r.limit {
|
||||
return n, ErrRequestEntityTooLarge
|
||||
}
|
||||
n = int(r.remaining)
|
||||
|
||||
return n, nil
|
||||
r.remaining = 0
|
||||
r.stickyErr = ErrRequestEntityTooLarge
|
||||
|
||||
return n, r.stickyErr
|
||||
}
|
||||
|
||||
// explicit casts to ensure that the main struct methods will be invoked
|
||||
// (extra precautions in case of nested interface wrapping erasure)
|
||||
// ---
|
||||
|
||||
func (r *limitedReader) Reread() {
|
||||
func (r *maxBytesReader) Reread() {
|
||||
rereader, ok := r.ReadCloser.(router.Rereader)
|
||||
if ok {
|
||||
rereader.Reread()
|
||||
r.remaining = r.limit
|
||||
r.stickyErr = nil
|
||||
}
|
||||
}
|
||||
|
||||
func (r *limitedReader) Close() error {
|
||||
closer, ok := r.ReadCloser.(io.Closer)
|
||||
if ok {
|
||||
return closer.Close()
|
||||
}
|
||||
return nil
|
||||
func (r *maxBytesReader) Close() error {
|
||||
return r.ReadCloser.Close()
|
||||
}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package apis_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/pocketbase/pocketbase/apis"
|
||||
@@ -19,13 +20,35 @@ func TestBodyLimitMiddleware(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pbRouter.POST("/a", func(e *core.RequestEvent) error {
|
||||
return e.String(200, "a")
|
||||
}) // default global BodyLimit check
|
||||
|
||||
pbRouter.POST("/b", func(e *core.RequestEvent) error {
|
||||
return e.String(200, "b")
|
||||
}).Bind(apis.BodyLimit(20))
|
||||
testHandler := func(e *core.RequestEvent) error {
|
||||
// read the body multiple times to ensure that the limited
|
||||
// reader guards and rereads are invoked
|
||||
var result any
|
||||
if err := e.BindBody(&result); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := e.BindBody(&result); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return e.JSON(200, result)
|
||||
}
|
||||
|
||||
const customLimit = 20
|
||||
|
||||
pbRouter.POST("/a", testHandler) // default global BodyLimit check
|
||||
pbRouter.POST("/b", testHandler).Bind(apis.BodyLimit(customLimit))
|
||||
pbRouter.POST("/iof", func(e *core.RequestEvent) error {
|
||||
// ensure that normal io methods still operate correctly
|
||||
b, err := io.ReadAll(e.Request.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return e.String(http.StatusOK, string(b))
|
||||
}).Bind(apis.BodyLimit(customLimit))
|
||||
|
||||
mux, err := pbRouter.BuildMux()
|
||||
if err != nil {
|
||||
@@ -33,20 +56,90 @@ func TestBodyLimitMiddleware(t *testing.T) {
|
||||
}
|
||||
|
||||
scenarios := []struct {
|
||||
url string
|
||||
size int64
|
||||
expectedStatus int
|
||||
name string
|
||||
url string
|
||||
body string
|
||||
lazyContentLength bool
|
||||
expectedStatus int
|
||||
}{
|
||||
{"/a", 21, 200},
|
||||
{"/a", apis.DefaultMaxBodySize + 1, 413},
|
||||
{"/b", 20, 200},
|
||||
{"/b", 21, 413},
|
||||
{
|
||||
"(eager content-length check) with body = default limit",
|
||||
"/a",
|
||||
`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
|
||||
false,
|
||||
http.StatusOK,
|
||||
},
|
||||
{
|
||||
"(eager content-length check) with body > default limit",
|
||||
"/a",
|
||||
`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
|
||||
false,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
{
|
||||
"(lazy content-length check) with body = default limit",
|
||||
"/a",
|
||||
`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize-2)) + `"`,
|
||||
true,
|
||||
http.StatusOK,
|
||||
},
|
||||
{
|
||||
"(lazy content-length check) with body > default limit",
|
||||
"/a",
|
||||
`"` + strings.Repeat("a", int(apis.DefaultMaxBodySize)) + `"`,
|
||||
true,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
// ---
|
||||
{
|
||||
"(eager content-length check) with body = custom limit",
|
||||
"/b",
|
||||
`"` + strings.Repeat("a", customLimit-2) + `"`,
|
||||
false,
|
||||
http.StatusOK,
|
||||
},
|
||||
{
|
||||
"(eager content-length check) with body > custom limit",
|
||||
"/b",
|
||||
`"` + strings.Repeat("a", customLimit) + `"`,
|
||||
false,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
{
|
||||
"(lazy content-length check) with body = custom limit",
|
||||
"/b",
|
||||
`"` + strings.Repeat("a", customLimit-2) + `"`,
|
||||
true,
|
||||
http.StatusOK,
|
||||
},
|
||||
{
|
||||
"(lazy content-length check) with body > custom limit",
|
||||
"/b",
|
||||
`"` + strings.Repeat("a", customLimit) + `"`,
|
||||
true,
|
||||
http.StatusRequestEntityTooLarge,
|
||||
},
|
||||
// ---
|
||||
{
|
||||
"io.ReadAll io.EOF exact limit check",
|
||||
"/iof",
|
||||
`"` + strings.Repeat("a", customLimit-2) + `"`,
|
||||
true,
|
||||
http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, s := range scenarios {
|
||||
t.Run(fmt.Sprintf("%s_%d", s.url, s.size), func(t *testing.T) {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest("POST", s.url, bytes.NewReader(make([]byte, s.size)))
|
||||
|
||||
req := httptest.NewRequest("POST", s.url, strings.NewReader(s.body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
|
||||
if s.lazyContentLength {
|
||||
req.ContentLength = -1
|
||||
}
|
||||
|
||||
mux.ServeHTTP(rec, req)
|
||||
|
||||
result := rec.Result()
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
package apis_test
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"testing/synctest"
|
||||
"time"
|
||||
|
||||
"github.com/pocketbase/pocketbase/apis"
|
||||
@@ -127,27 +129,29 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
|
||||
{"/rate/guest", 0, true, 429},
|
||||
}
|
||||
|
||||
for _, s := range scenarios {
|
||||
t.Run(s.url, func(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
for i, s := range scenarios {
|
||||
prefix := fmt.Sprintf("[%s:%d] ", s.url, i+1)
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest("GET", s.url, nil)
|
||||
|
||||
if s.authenticated {
|
||||
auth, err := app.FindAuthRecordByEmail("users", "test@example.com")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
t.Fatalf(prefix+"%v", err)
|
||||
}
|
||||
|
||||
token, err := auth.NewAuthToken()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
t.Fatalf(prefix+"%v", err)
|
||||
}
|
||||
|
||||
req.Header.Add("Authorization", token)
|
||||
}
|
||||
|
||||
if s.wait > 0 {
|
||||
time.Sleep(time.Duration(s.wait) * time.Millisecond)
|
||||
synctest.Sleep(time.Duration(s.wait) * time.Millisecond)
|
||||
}
|
||||
|
||||
mux.ServeHTTP(rec, req)
|
||||
@@ -155,10 +159,10 @@ func TestDefaultRateLimitMiddleware(t *testing.T) {
|
||||
result := rec.Result()
|
||||
|
||||
if result.StatusCode != s.expectedStatus {
|
||||
t.Fatalf("Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
|
||||
t.Fatalf(prefix+"Expected response status %d, got %d", s.expectedStatus, result.StatusCode)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestDefaultRateLimitMiddlewareSkipChecks(t *testing.T) {
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package apis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
|
||||
@@ -2,7 +2,7 @@ package apis_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
@@ -1171,7 +1171,9 @@ func TestRealtimeRecordResolve(t *testing.T) {
|
||||
|
||||
var mu sync.Mutex
|
||||
notify := func(clientId string, eventData []byte) {
|
||||
data := struct{ Action string }{}
|
||||
data := struct {
|
||||
Action string `json:"action"`
|
||||
}{}
|
||||
_ = json.Unmarshal(eventData, &data)
|
||||
|
||||
mu.Lock()
|
||||
|
||||
@@ -54,7 +54,7 @@ func TestRecordAuthMethodsList(t *testing.T) {
|
||||
`"providers":[{`,
|
||||
`"name":"google"`,
|
||||
`"name":"gitlab"`,
|
||||
`"logo":"\u003csvg`,
|
||||
`"logo":"<svg`,
|
||||
`"logo":""`, // for the legacy fields
|
||||
`"state":`,
|
||||
`"displayName":`,
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package apis_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
|
||||
+28
-2
@@ -54,11 +54,16 @@ type App interface {
|
||||
// Bootstrap initializes the application
|
||||
// (aka. create data dir, open db connections, load settings, etc.).
|
||||
//
|
||||
// It will call ResetBootstrapState() if the application was already bootstrapped.
|
||||
// It calls ClearBootstrap() if the application was already bootstrapped.
|
||||
Bootstrap() error
|
||||
|
||||
// ResetBootstrapState releases the initialized core app resources
|
||||
// ClearBootstrap releases the initialized core app resources
|
||||
// (closing db connections, stopping cron ticker, etc.).
|
||||
//
|
||||
// This method is no-op if the application is not bootstrapped yet.
|
||||
ClearBootstrap() error
|
||||
|
||||
// Deprecated: use ClearBootstrap().
|
||||
ResetBootstrapState() error
|
||||
|
||||
// DataDir returns the app data directory path.
|
||||
@@ -713,6 +718,13 @@ type App interface {
|
||||
// resources (db, app settings, etc).
|
||||
OnBootstrap() *hook.Hook[*BootstrapEvent]
|
||||
|
||||
// OnBootstrapClear hook is triggered when clearing the main application
|
||||
// resources (db connections, cron, logger, etc.)
|
||||
//
|
||||
// It is usually invoked automatically right before app termination
|
||||
// or when manually calling app.ClearBootstrap().
|
||||
OnBootstrapClear() *hook.Hook[*BootstrapEvent]
|
||||
|
||||
// OnServe hook is triggered when the app web server is started
|
||||
// (after starting the TCP listener but before initializing the blocking serve task),
|
||||
// allowing you to adjust its options and attach new routes or middlewares.
|
||||
@@ -1262,6 +1274,20 @@ type App interface {
|
||||
// triggered and called only if their event data origin matches the tags.
|
||||
OnMailerRecordOTPSend(tags ...string) *hook.TaggedHook[*MailerRecordEvent]
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// Filesystem event hooks
|
||||
// (not publicly exposed until file_field refactoring)
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
// onFilesystemNewWriter is an internal hook for app.NewFilesystem()
|
||||
// instances that is triggered on every storage filesystem writer initialization
|
||||
// (aka. whenever attempting to create a new file).
|
||||
onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent]
|
||||
|
||||
// onFilesystemDelete is an internal hook for app.NewFilesystem()
|
||||
// instances that is triggered for every storage file delete call.
|
||||
onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent]
|
||||
|
||||
// ---------------------------------------------------------------
|
||||
// Realtime API event hooks
|
||||
// ---------------------------------------------------------------
|
||||
|
||||
+137
@@ -0,0 +1,137 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/inflector"
|
||||
)
|
||||
|
||||
const (
|
||||
StoreKeyActiveBackup = "@activeBackup"
|
||||
)
|
||||
|
||||
// generateBackupName generates a new backup name based on the app name and current date.
|
||||
func generateBackupName(app App, prefix string) string {
|
||||
appName := inflector.Snakecase(app.Settings().Meta.AppName)
|
||||
if len(appName) > 50 {
|
||||
appName = appName[:50]
|
||||
}
|
||||
|
||||
return fmt.Sprintf(
|
||||
"%s%s_%s.zip",
|
||||
prefix,
|
||||
appName,
|
||||
time.Now().UTC().Format("20060102150405"),
|
||||
)
|
||||
}
|
||||
|
||||
// registerAutobackupHooks registers the autobackup app serve hooks.
|
||||
func (app *BaseApp) registerAutobackupHooks() {
|
||||
const jobId = "__pbAutoBackup__"
|
||||
|
||||
loadJob := func() {
|
||||
rawSchedule := app.Settings().Backups.Cron
|
||||
if rawSchedule == "" {
|
||||
app.Cron().Remove(jobId)
|
||||
return
|
||||
}
|
||||
|
||||
app.Cron().Add(jobId, rawSchedule, func() {
|
||||
const autoPrefix = "@auto_pb_backup_"
|
||||
|
||||
name := generateBackupName(app, autoPrefix)
|
||||
|
||||
if err := app.CreateBackup(context.Background(), name); err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to create backup",
|
||||
slog.String("name", name),
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
|
||||
alertError := sendSystemAlertToAllSuperusers(
|
||||
app,
|
||||
"Autobackup failure",
|
||||
"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
|
||||
)
|
||||
if alertError != nil {
|
||||
app.Logger().Warn(
|
||||
"[Backup cron] Failed to send backup error alerts",
|
||||
slog.String("name", name),
|
||||
slog.String("error", alertError.Error()),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
maxKeep := app.Settings().Backups.CronMaxKeep
|
||||
|
||||
if maxKeep == 0 {
|
||||
return // no explicit limit
|
||||
}
|
||||
|
||||
fsys, err := app.NewBackupsFilesystem()
|
||||
if err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to initialize the backup filesystem",
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
defer fsys.Close()
|
||||
|
||||
files, err := fsys.List(autoPrefix)
|
||||
if err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to list autogenerated backups",
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if maxKeep >= len(files) {
|
||||
return // nothing to remove
|
||||
}
|
||||
|
||||
// sort desc
|
||||
sort.Slice(files, func(i, j int) bool {
|
||||
return files[i].ModTime.After(files[j].ModTime)
|
||||
})
|
||||
|
||||
// keep only the most recent n auto backup files
|
||||
toRemove := files[maxKeep:]
|
||||
|
||||
for _, f := range toRemove {
|
||||
if err := fsys.Delete(f.Key); err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to remove old autogenerated backup",
|
||||
slog.String("key", f.Key),
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
|
||||
if err := e.Next(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
loadJob()
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
|
||||
if err := e.Next(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
loadJob()
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,437 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"compress/flate"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/pocketbase/dbx"
|
||||
"github.com/pocketbase/pocketbase/tools/filesystem"
|
||||
"github.com/pocketbase/pocketbase/tools/hook"
|
||||
"github.com/pocketbase/pocketbase/tools/security"
|
||||
"github.com/pocketbase/pocketbase/tools/store"
|
||||
)
|
||||
|
||||
var errIsDir = errors.New("the specified path is a directory and not a regular file")
|
||||
|
||||
// CreateBackup creates a new backup of the current app pb_data directory.
|
||||
//
|
||||
// If name is empty, it will be autogenerated.
|
||||
// If backup with the same name exists, the new backup file will replace it.
|
||||
//
|
||||
// To safely perform the backup, it is recommended to have free disk space
|
||||
// for at least 2x the size of the pb_data directory.
|
||||
//
|
||||
// By default backups are stored in pb_data/backups
|
||||
// (the backups directory itself is excluded from the generated backup).
|
||||
//
|
||||
// Backups can be stored on S3 if it is configured in app.Settings().Backups.
|
||||
// When using S3 storage for the uploaded collection files, you have to
|
||||
// take care manually to backup those since they are not part of the pb_data.
|
||||
//
|
||||
// DB write locks are minimal and isolated only for the duration of the
|
||||
// VACUUM INTO statement that creates a live copy of the app database.
|
||||
//
|
||||
// The backup works as follow:
|
||||
//
|
||||
// 1. Start listening for DELETED storage files.
|
||||
// In case a file is being deleted while the backup is still ongoing,
|
||||
// we directly copy it in the zip before the deletion and mark it as "excluded".
|
||||
//
|
||||
// 2. Copy the main database with VACUUM INTO, write it in the zip and mark it as "excluded".
|
||||
//
|
||||
// 3. Stop listening for DELETED files.
|
||||
//
|
||||
// 4. Start listening for NEW storage files and mark all new files from this point as "excluded".
|
||||
//
|
||||
// 5. Copy the logs database with VACUUM INTO, write it in the zip and mark it as "excluded".
|
||||
//
|
||||
// 6. Copy the rest of the pb_data files in the zip while ignoring the "excluded" list (it should be concurrent safe).
|
||||
//
|
||||
// 7. Stop listening for NEW storage files.
|
||||
//
|
||||
// While there is a risk for a race condition between steps 1, 2 and 3, it is an
|
||||
// acceptable trade-off between performance and correctness because in
|
||||
// the worst case there will be some unused storage files in the backup that don't do any harm.
|
||||
func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
|
||||
if app.Store().Has(StoreKeyActiveBackup) {
|
||||
return errors.New("try again later - another backup/restore operation has already been started")
|
||||
}
|
||||
|
||||
app.Store().Set(StoreKeyActiveBackup, name)
|
||||
defer app.Store().Remove(StoreKeyActiveBackup)
|
||||
|
||||
startTime := time.Now()
|
||||
|
||||
event := new(BackupEvent)
|
||||
event.App = app
|
||||
event.Context = ctx
|
||||
event.Name = name
|
||||
// default root dir entries to exclude from the backup generation
|
||||
event.Exclude = []string{
|
||||
LocalBackupsDirName,
|
||||
LocalTempDirName,
|
||||
LocalNotifyDirName,
|
||||
LocalAutocertCacheDirName,
|
||||
lostFoundDirName,
|
||||
}
|
||||
|
||||
return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
|
||||
if e.Name == "" {
|
||||
e.Name = generateBackupName(e.App, "pb_backup_")
|
||||
}
|
||||
|
||||
app.Logger().Debug("[" + e.Name + "] zip archive started")
|
||||
|
||||
// create backup zip
|
||||
// (it needs to be inside the current pb_data to avoid "cross-device link" errors)
|
||||
// -----------------------------------------------------------
|
||||
tempZipPath := filepath.Join(app.DataDir(), LocalTempDirName, "pb_backup_"+security.PseudorandomString(6))
|
||||
err := createZip(e, tempZipPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(tempZipPath)
|
||||
|
||||
// persist the backup in the backups filesystem
|
||||
// -----------------------------------------------------------
|
||||
fsys, err := e.App.NewBackupsFilesystem()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer fsys.Close()
|
||||
|
||||
fsys.SetContext(e.Context)
|
||||
|
||||
file, err := filesystem.NewFileFromPath(tempZipPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
file.OriginalName = e.Name
|
||||
file.Name = file.OriginalName
|
||||
|
||||
err = fsys.UploadFile(file, file.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
app.Logger().Debug(
|
||||
"["+e.Name+"] zip archive completed",
|
||||
slog.Float64("execTime", float64(time.Since(startTime))/float64(time.Millisecond)),
|
||||
)
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func createZip(be *BackupEvent, tempZipPath string) error {
|
||||
logPrefix := "[" + be.Name + "] "
|
||||
|
||||
// make sure that the special temp directory exists
|
||||
localTempDir := filepath.Dir(tempZipPath)
|
||||
if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
|
||||
return fmt.Errorf(logPrefix+"failed to create temp dir: %w", err)
|
||||
}
|
||||
|
||||
const tempFilesHookId = "__pbTempBackupFilesystemWatcher__"
|
||||
defer func() {
|
||||
// unbind again in cacase of an error
|
||||
be.App.onFilesystemDelete().Unbind(tempFilesHookId)
|
||||
be.App.onFilesystemNewWriter().Unbind(tempFilesHookId)
|
||||
}()
|
||||
|
||||
zipper, err := newZipWriter(tempZipPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer zipper.close()
|
||||
|
||||
excluded := store.New[string, struct{}](nil)
|
||||
for _, name := range be.Exclude {
|
||||
excluded.Set(normalizePathExclude(name), struct{}{})
|
||||
}
|
||||
|
||||
// init deleted files tracker
|
||||
// ---------------------------------------------------------------
|
||||
be.App.onFilesystemDelete().Bind(&hook.Handler[*FilesystemDeleteEvent]{
|
||||
Id: tempFilesHookId,
|
||||
Priority: -99,
|
||||
Func: func(e *FilesystemDeleteEvent) error {
|
||||
// note: the zip header name allow only forward slashes
|
||||
zipPath := path.Join(LocalStorageDirName, e.FileKey)
|
||||
|
||||
if excluded.Has(normalizePathExclude(zipPath)) || be.App.Settings().S3.Enabled {
|
||||
return e.Next()
|
||||
}
|
||||
|
||||
localPath := filepath.Join(
|
||||
be.App.DataDir(),
|
||||
LocalStorageDirName,
|
||||
e.FileKey,
|
||||
)
|
||||
|
||||
// copy to zip before delete
|
||||
err := zipper.copyFileToZip(localPath, zipPath)
|
||||
if err != nil {
|
||||
// it is ok to ignore directories
|
||||
if !errors.Is(err, errIsDir) {
|
||||
be.App.Logger().Warn(
|
||||
logPrefix+"failed to copy file in backup zip before delete",
|
||||
slog.Any("error", err),
|
||||
slog.String("file", e.FileKey),
|
||||
)
|
||||
}
|
||||
} else {
|
||||
// mark that it was already copied
|
||||
excluded.Set(normalizePathExclude(zipPath), struct{}{})
|
||||
}
|
||||
|
||||
// proceed with the normal deletion
|
||||
return e.Next()
|
||||
},
|
||||
})
|
||||
|
||||
// copy data.db
|
||||
// ---------------------------------------------------------------
|
||||
dataStartTime := time.Now()
|
||||
tempDataDBPath := filepath.Join(localTempDir, dataDBFilename)
|
||||
|
||||
_, err = be.App.ConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempDataDBPath}).Execute()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// eagerly stop listening for deleted files since we already have what we needed
|
||||
be.App.onFilesystemDelete().Unbind(tempFilesHookId)
|
||||
|
||||
be.App.Logger().Debug(
|
||||
logPrefix+dataDBFilename+" copy completed",
|
||||
slog.Float64("execTime", float64(time.Since(dataStartTime))/float64(time.Millisecond)),
|
||||
)
|
||||
|
||||
err = zipper.copyFileToZip(tempDataDBPath, dataDBFilename)
|
||||
if err != nil {
|
||||
_ = os.Remove(tempDataDBPath)
|
||||
return err
|
||||
}
|
||||
_ = os.Remove(tempDataDBPath)
|
||||
|
||||
excluded.Set(normalizePathExclude(dataDBFilename), struct{}{})
|
||||
excluded.Set(normalizePathExclude(dataDBFilename+"-wal"), struct{}{})
|
||||
excluded.Set(normalizePathExclude(dataDBFilename+"-shm"), struct{}{})
|
||||
|
||||
// init to-be-created files tracker
|
||||
// ---------------------------------------------------------------
|
||||
be.App.onFilesystemNewWriter().Bind(&hook.Handler[*FilesystemNewWriterEvent]{
|
||||
Id: tempFilesHookId,
|
||||
Priority: -99,
|
||||
Func: func(e *FilesystemNewWriterEvent) error {
|
||||
if !be.App.Settings().S3.Enabled {
|
||||
// mark for exclude even if the writer eventually fails
|
||||
// (all record files have random name so collisions are unlikely)
|
||||
name := normalizePathExclude(filepath.Join(LocalStorageDirName, e.FileKey))
|
||||
excluded.Set(name, struct{}{})
|
||||
}
|
||||
|
||||
return e.Next()
|
||||
},
|
||||
})
|
||||
|
||||
// copy auxiliary.db
|
||||
// ---------------------------------------------------------------
|
||||
auxStartTime := time.Now()
|
||||
tempAuxDBPath := filepath.Join(localTempDir, auxDBFilename)
|
||||
|
||||
_, err = be.App.AuxConcurrentDB().NewQuery("VACUUM INTO {:path}").Bind(dbx.Params{"path": tempAuxDBPath}).Execute()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
be.App.Logger().Debug(
|
||||
logPrefix+auxDBFilename+" copy completed",
|
||||
slog.Float64("execTime", float64(time.Since(auxStartTime))/float64(time.Millisecond)),
|
||||
)
|
||||
|
||||
err = zipper.copyFileToZip(tempAuxDBPath, auxDBFilename)
|
||||
if err != nil {
|
||||
_ = os.Remove(tempAuxDBPath)
|
||||
return err
|
||||
}
|
||||
_ = os.Remove(tempAuxDBPath)
|
||||
|
||||
excluded.Set(normalizePathExclude(auxDBFilename), struct{}{})
|
||||
excluded.Set(normalizePathExclude(auxDBFilename+"-wal"), struct{}{})
|
||||
excluded.Set(normalizePathExclude(auxDBFilename+"-shm"), struct{}{})
|
||||
|
||||
// try to run manual checkpoints to ensure that all wal writes during the
|
||||
// previous VACUUM INTO are transferred and don't accumulate
|
||||
// (errors are ignore because some drivers may not support the wal_checkpoint pragma)
|
||||
// ---------------------------------------------------------------
|
||||
_, _ = be.App.NonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
|
||||
_, _ = be.App.AuxNonconcurrentDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
|
||||
|
||||
// copy the rest of the pb_data
|
||||
// ---------------------------------------------------------------
|
||||
err = zipper.copyDirToZip(os.DirFS(be.App.DataDir()), excluded)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return zipper.close()
|
||||
}
|
||||
|
||||
// normalize the provided file path to always end with forward slash
|
||||
func normalizePathExclude(filePath string) string {
|
||||
return path.Clean(filePath) + "/"
|
||||
}
|
||||
|
||||
type zipWriter struct {
|
||||
mu sync.Mutex
|
||||
w *zip.Writer
|
||||
f *os.File
|
||||
closed bool
|
||||
}
|
||||
|
||||
func newZipWriter(zipFilePath string) (*zipWriter, error) {
|
||||
f, err := os.Create(zipFilePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
w := zip.NewWriter(f)
|
||||
w.RegisterCompressor(zip.Deflate, func(out io.Writer) (io.WriteCloser, error) {
|
||||
return flate.NewWriter(out, flate.BestSpeed)
|
||||
})
|
||||
|
||||
return &zipWriter{
|
||||
w: w,
|
||||
f: f,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (z *zipWriter) close() error {
|
||||
z.mu.Lock()
|
||||
defer z.mu.Unlock()
|
||||
|
||||
if z.closed {
|
||||
return nil
|
||||
}
|
||||
|
||||
z.closed = true
|
||||
|
||||
return errors.Join(z.w.Close(), z.f.Close())
|
||||
}
|
||||
|
||||
func (z *zipWriter) copyFileToZip(localPath string, zipPath string) error {
|
||||
info, err := os.Stat(localPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if info.IsDir() {
|
||||
return errIsDir
|
||||
}
|
||||
|
||||
h, err := zip.FileInfoHeader(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
h.Name = zipPath
|
||||
h.Method = zip.Deflate
|
||||
|
||||
z.mu.Lock()
|
||||
defer z.mu.Unlock()
|
||||
|
||||
if z.closed {
|
||||
return errors.New("zip writer is already closed")
|
||||
}
|
||||
|
||||
fw, err := z.w.CreateHeader(h)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
f, err := os.Open(localPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
_, err = io.Copy(fw, f)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
func (z *zipWriter) copyDirToZip(fsys fs.FS, excludedPrefixes *store.Store[string, struct{}]) error {
|
||||
return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// skip excluded prefixes
|
||||
if excludedPrefixes != nil {
|
||||
check := normalizePathExclude(name)
|
||||
prefixes := excludedPrefixes.Keys() // refetch to avoid races
|
||||
for _, prefix := range prefixes {
|
||||
if strings.HasPrefix(check, prefix) {
|
||||
if d.IsDir() {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
|
||||
info, err := d.Info()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
h, err := zip.FileInfoHeader(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
h.Name = name
|
||||
h.Method = zip.Deflate
|
||||
|
||||
z.mu.Lock()
|
||||
defer z.mu.Unlock()
|
||||
|
||||
if z.closed {
|
||||
// note: fs.WalkDir perform direct comparison with the value
|
||||
return fs.SkipAll
|
||||
}
|
||||
|
||||
fw, err := z.w.CreateHeader(h)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
f, err := fsys.Open(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
_, err = io.Copy(fw, f)
|
||||
|
||||
return err
|
||||
})
|
||||
}
|
||||
@@ -9,117 +9,12 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/archive"
|
||||
"github.com/pocketbase/pocketbase/tools/filesystem"
|
||||
"github.com/pocketbase/pocketbase/tools/inflector"
|
||||
"github.com/pocketbase/pocketbase/tools/osutils"
|
||||
"github.com/pocketbase/pocketbase/tools/security"
|
||||
)
|
||||
|
||||
const (
|
||||
StoreKeyActiveBackup = "@activeBackup"
|
||||
)
|
||||
|
||||
// CreateBackup creates a new backup of the current app pb_data directory.
|
||||
//
|
||||
// If name is empty, it will be autogenerated.
|
||||
// If backup with the same name exists, the new backup file will replace it.
|
||||
//
|
||||
// The backup is executed within a transaction, meaning that new writes
|
||||
// will be temporary "blocked" until the backup file is generated.
|
||||
//
|
||||
// To safely perform the backup, it is recommended to have free disk space
|
||||
// for at least 2x the size of the pb_data directory.
|
||||
//
|
||||
// By default backups are stored in pb_data/backups
|
||||
// (the backups directory itself is excluded from the generated backup).
|
||||
//
|
||||
// When using S3 storage for the uploaded collection files, you have to
|
||||
// take care manually to backup those since they are not part of the pb_data.
|
||||
//
|
||||
// Backups can be stored on S3 if it is configured in app.Settings().Backups.
|
||||
func (app *BaseApp) CreateBackup(ctx context.Context, name string) error {
|
||||
if app.Store().Has(StoreKeyActiveBackup) {
|
||||
return errors.New("try again later - another backup/restore operation has already been started")
|
||||
}
|
||||
|
||||
app.Store().Set(StoreKeyActiveBackup, name)
|
||||
defer app.Store().Remove(StoreKeyActiveBackup)
|
||||
|
||||
event := new(BackupEvent)
|
||||
event.App = app
|
||||
event.Context = ctx
|
||||
event.Name = name
|
||||
// default root dir entries to exclude from the backup generation
|
||||
event.Exclude = []string{
|
||||
LocalBackupsDirName,
|
||||
LocalTempDirName,
|
||||
LocalNotifyDirName,
|
||||
LocalAutocertCacheDirName,
|
||||
lostFoundDirName,
|
||||
}
|
||||
|
||||
return app.OnBackupCreate().Trigger(event, func(e *BackupEvent) error {
|
||||
// generate a default name if missing
|
||||
if e.Name == "" {
|
||||
e.Name = generateBackupName(e.App, "pb_backup_")
|
||||
}
|
||||
|
||||
// make sure that the special temp directory exists
|
||||
// note: it needs to be inside the current pb_data to avoid "cross-device link" errors
|
||||
localTempDir := filepath.Join(e.App.DataDir(), LocalTempDirName)
|
||||
if err := os.MkdirAll(localTempDir, os.ModePerm); err != nil {
|
||||
return fmt.Errorf("failed to create a temp dir: %w", err)
|
||||
}
|
||||
|
||||
// archive pb_data in a temp directory, excluding the "backups" and the temp dirs
|
||||
//
|
||||
// run in transaction to temporary block other writes (transactions uses the NonconcurrentDB connection)
|
||||
// ---
|
||||
tempPath := filepath.Join(localTempDir, "pb_backup_"+security.PseudorandomString(6))
|
||||
createErr := e.App.RunInTransaction(func(txApp App) error {
|
||||
return txApp.AuxRunInTransaction(func(txApp App) error {
|
||||
// run manual checkpoint and truncate the WAL files
|
||||
// (errors are ignored because it is not that important and the PRAGMA may not be supported by the used driver)
|
||||
txApp.DB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
|
||||
txApp.AuxDB().NewQuery("PRAGMA wal_checkpoint(TRUNCATE)").Execute()
|
||||
|
||||
return archive.Create(txApp.DataDir(), tempPath, e.Exclude...)
|
||||
})
|
||||
})
|
||||
if createErr != nil {
|
||||
return createErr
|
||||
}
|
||||
defer os.Remove(tempPath)
|
||||
|
||||
// persist the backup in the backups filesystem
|
||||
// ---
|
||||
fsys, err := e.App.NewBackupsFilesystem()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer fsys.Close()
|
||||
|
||||
fsys.SetContext(e.Context)
|
||||
|
||||
file, err := filesystem.NewFileFromPath(tempPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
file.OriginalName = e.Name
|
||||
file.Name = file.OriginalName
|
||||
|
||||
if err := fsys.UploadFile(file, file.Name); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// RestoreBackup restores the backup with the specified name and restarts
|
||||
// the current running application process.
|
||||
//
|
||||
@@ -299,124 +194,3 @@ func (app *BaseApp) RestoreBackup(ctx context.Context, name string) error {
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// registerAutobackupHooks registers the autobackup app serve hooks.
|
||||
func (app *BaseApp) registerAutobackupHooks() {
|
||||
const jobId = "__pbAutoBackup__"
|
||||
|
||||
loadJob := func() {
|
||||
rawSchedule := app.Settings().Backups.Cron
|
||||
if rawSchedule == "" {
|
||||
app.Cron().Remove(jobId)
|
||||
return
|
||||
}
|
||||
|
||||
app.Cron().Add(jobId, rawSchedule, func() {
|
||||
const autoPrefix = "@auto_pb_backup_"
|
||||
|
||||
name := generateBackupName(app, autoPrefix)
|
||||
|
||||
if err := app.CreateBackup(context.Background(), name); err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to create backup",
|
||||
slog.String("name", name),
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
|
||||
alertError := sendSystemAlertToAllSuperusers(
|
||||
app,
|
||||
"Autobackup failure",
|
||||
"Failed to create/upload automated backup. Raw error:\n"+err.Error(),
|
||||
)
|
||||
if alertError != nil {
|
||||
app.Logger().Warn(
|
||||
"[Backup cron] Failed to send backup error alerts",
|
||||
slog.String("name", name),
|
||||
slog.String("error", alertError.Error()),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
maxKeep := app.Settings().Backups.CronMaxKeep
|
||||
|
||||
if maxKeep == 0 {
|
||||
return // no explicit limit
|
||||
}
|
||||
|
||||
fsys, err := app.NewBackupsFilesystem()
|
||||
if err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to initialize the backup filesystem",
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
defer fsys.Close()
|
||||
|
||||
files, err := fsys.List(autoPrefix)
|
||||
if err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to list autogenerated backups",
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
if maxKeep >= len(files) {
|
||||
return // nothing to remove
|
||||
}
|
||||
|
||||
// sort desc
|
||||
sort.Slice(files, func(i, j int) bool {
|
||||
return files[i].ModTime.After(files[j].ModTime)
|
||||
})
|
||||
|
||||
// keep only the most recent n auto backup files
|
||||
toRemove := files[maxKeep:]
|
||||
|
||||
for _, f := range toRemove {
|
||||
if err := fsys.Delete(f.Key); err != nil {
|
||||
app.Logger().Error(
|
||||
"[Backup cron] Failed to remove old autogenerated backup",
|
||||
slog.String("key", f.Key),
|
||||
slog.String("error", err.Error()),
|
||||
)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
app.OnBootstrap().BindFunc(func(e *BootstrapEvent) error {
|
||||
if err := e.Next(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
loadJob()
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
app.OnSettingsReload().BindFunc(func(e *SettingsReloadEvent) error {
|
||||
if err := e.Next(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
loadJob()
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func generateBackupName(app App, prefix string) string {
|
||||
appName := inflector.Snakecase(app.Settings().Meta.AppName)
|
||||
if len(appName) > 50 {
|
||||
appName = appName[:50]
|
||||
}
|
||||
|
||||
return fmt.Sprintf(
|
||||
"%s%s_%s.zip",
|
||||
prefix,
|
||||
appName,
|
||||
time.Now().UTC().Format("20060102150405"),
|
||||
)
|
||||
}
|
||||
@@ -126,11 +126,7 @@ func verifyBackupContent(app core.App, path string) error {
|
||||
expectedRootEntries := []string{
|
||||
"storage",
|
||||
"data.db",
|
||||
"data.db-shm",
|
||||
"data.db-wal",
|
||||
"auxiliary.db",
|
||||
"auxiliary.db-shm",
|
||||
"auxiliary.db-wal",
|
||||
".gitignore",
|
||||
}
|
||||
|
||||
+188
-71
@@ -12,6 +12,7 @@ import (
|
||||
"regexp"
|
||||
"runtime"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/fatih/color"
|
||||
@@ -44,6 +45,9 @@ const (
|
||||
|
||||
// @todo consider removing after backups refactoring
|
||||
lostFoundDirName string = "lost+found"
|
||||
|
||||
dataDBFilename string = "data.db"
|
||||
auxDBFilename string = "auxiliary.db"
|
||||
)
|
||||
|
||||
// FilesManager defines an interface with common methods that files manager models should implement.
|
||||
@@ -86,11 +90,12 @@ type BaseApp struct {
|
||||
auxNonconcurrentDB dbx.Builder
|
||||
|
||||
// app event hooks
|
||||
onBootstrap *hook.Hook[*BootstrapEvent]
|
||||
onServe *hook.Hook[*ServeEvent]
|
||||
onTerminate *hook.Hook[*TerminateEvent]
|
||||
onBackupCreate *hook.Hook[*BackupEvent]
|
||||
onBackupRestore *hook.Hook[*BackupEvent]
|
||||
onBootstrap *hook.Hook[*BootstrapEvent]
|
||||
onBootstrapClear *hook.Hook[*BootstrapEvent]
|
||||
onServe *hook.Hook[*ServeEvent]
|
||||
onTerminate *hook.Hook[*TerminateEvent]
|
||||
onBackupCreate *hook.Hook[*BackupEvent]
|
||||
onBackupRestore *hook.Hook[*BackupEvent]
|
||||
|
||||
// db model hooks
|
||||
onModelValidate *hook.Hook[*ModelEvent]
|
||||
@@ -146,6 +151,17 @@ type BaseApp struct {
|
||||
onMailerRecordOTPSend *hook.Hook[*MailerRecordEvent]
|
||||
onMailerRecordAuthAlertSend *hook.Hook[*MailerRecordEvent]
|
||||
|
||||
// filesystem event hooks
|
||||
//
|
||||
// @todo 1:
|
||||
// intentionally not exposed since the events are too "chatty" and
|
||||
// can cause unnecessary userland tests breaking changes;
|
||||
// reevaluate once refactoring the file_field
|
||||
//
|
||||
// @todo 2: if exposed consider registering the same for the backup filesystem
|
||||
_onFilesystemNewWriter *hook.Hook[*FilesystemNewWriterEvent]
|
||||
_onFilesystemDelete *hook.Hook[*FilesystemDeleteEvent]
|
||||
|
||||
// realtime api event hooks
|
||||
onRealtimeConnectRequest *hook.Hook[*RealtimeConnectRequestEvent]
|
||||
onRealtimeMessageSend *hook.Hook[*RealtimeMessageEvent]
|
||||
@@ -235,6 +251,7 @@ func NewBaseApp(config BaseAppConfig) *BaseApp {
|
||||
func (app *BaseApp) initHooks() {
|
||||
// app event hooks
|
||||
app.onBootstrap = &hook.Hook[*BootstrapEvent]{}
|
||||
app.onBootstrapClear = &hook.Hook[*BootstrapEvent]{}
|
||||
app.onServe = &hook.Hook[*ServeEvent]{}
|
||||
app.onTerminate = &hook.Hook[*TerminateEvent]{}
|
||||
app.onBackupCreate = &hook.Hook[*BackupEvent]{}
|
||||
@@ -294,6 +311,10 @@ func (app *BaseApp) initHooks() {
|
||||
app.onMailerRecordOTPSend = &hook.Hook[*MailerRecordEvent]{}
|
||||
app.onMailerRecordAuthAlertSend = &hook.Hook[*MailerRecordEvent]{}
|
||||
|
||||
// filesystem event hooks
|
||||
app._onFilesystemNewWriter = &hook.Hook[*FilesystemNewWriterEvent]{}
|
||||
app._onFilesystemDelete = &hook.Hook[*FilesystemDeleteEvent]{}
|
||||
|
||||
// realtime API event hooks
|
||||
app.onRealtimeConnectRequest = &hook.Hook[*RealtimeConnectRequestEvent]{}
|
||||
app.onRealtimeMessageSend = &hook.Hook[*RealtimeMessageEvent]{}
|
||||
@@ -387,14 +408,14 @@ func (app *BaseApp) IsBootstrapped() bool {
|
||||
// Bootstrap initializes the application
|
||||
// (aka. create data dir, open db connections, load settings, etc.).
|
||||
//
|
||||
// It will call ResetBootstrapState() if the application was already bootstrapped.
|
||||
// It calls ClearBootstrap() if the application was already bootstrapped.
|
||||
func (app *BaseApp) Bootstrap() error {
|
||||
event := &BootstrapEvent{}
|
||||
event.App = app
|
||||
|
||||
err := app.OnBootstrap().Trigger(event, func(e *BootstrapEvent) error {
|
||||
// clear resources of previous core state (if any)
|
||||
if err := app.ResetBootstrapState(); err != nil {
|
||||
// clear previous bootstrap state (if any)
|
||||
if err := app.ClearBootstrap(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -442,41 +463,55 @@ func (app *BaseApp) Bootstrap() error {
|
||||
return err
|
||||
}
|
||||
|
||||
type closer interface {
|
||||
Close() error
|
||||
// Deprecated: use [ClearBootstrap].
|
||||
func (app *BaseApp) ResetBootstrapState() error {
|
||||
return app.ClearBootstrap()
|
||||
}
|
||||
|
||||
// ResetBootstrapState releases the initialized core app resources
|
||||
// ClearBootstrap releases the initialized core app resources
|
||||
// (closing db connections, stopping cron ticker, etc.).
|
||||
func (app *BaseApp) ResetBootstrapState() error {
|
||||
app.Cron().Stop()
|
||||
|
||||
var errs []error
|
||||
|
||||
dbs := []*dbx.Builder{
|
||||
&app.concurrentDB,
|
||||
&app.nonconcurrentDB,
|
||||
&app.auxConcurrentDB,
|
||||
&app.auxNonconcurrentDB,
|
||||
//
|
||||
// This method is no-op if the application is not bootstrapped yet.
|
||||
func (app *BaseApp) ClearBootstrap() error {
|
||||
if !app.IsBootstrapped() {
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, db := range dbs {
|
||||
if db == nil {
|
||||
continue
|
||||
event := &BootstrapEvent{}
|
||||
event.App = app
|
||||
|
||||
return app.OnBootstrapClear().Trigger(event, func(e *BootstrapEvent) error {
|
||||
type closer interface {
|
||||
Close() error
|
||||
}
|
||||
if v, ok := (*db).(closer); ok {
|
||||
if err := v.Close(); err != nil {
|
||||
errs = append(errs, err)
|
||||
|
||||
var errs []error
|
||||
|
||||
dbs := []*dbx.Builder{
|
||||
&app.concurrentDB,
|
||||
&app.nonconcurrentDB,
|
||||
&app.auxConcurrentDB,
|
||||
&app.auxNonconcurrentDB,
|
||||
}
|
||||
|
||||
for _, db := range dbs {
|
||||
if db == nil {
|
||||
continue
|
||||
}
|
||||
if v, ok := (*db).(closer); ok {
|
||||
if err := v.Close(); err != nil {
|
||||
errs = append(errs, err)
|
||||
}
|
||||
}
|
||||
*db = nil
|
||||
}
|
||||
*db = nil
|
||||
}
|
||||
|
||||
if len(errs) > 0 {
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
if len(errs) > 0 {
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
return nil
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// DB returns the default app data.db builder instance.
|
||||
@@ -712,9 +747,10 @@ func (app *BaseApp) NewMailClient() mailer.Mailer {
|
||||
//
|
||||
// NB! Make sure to call Close() on the returned result
|
||||
// after you are done working with it.
|
||||
func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
|
||||
func (app *BaseApp) NewFilesystem() (fsys *filesystem.System, err error) {
|
||||
if app.settings != nil && app.settings.S3.Enabled {
|
||||
return filesystem.NewS3(
|
||||
// S3
|
||||
fsys, err = filesystem.NewS3(
|
||||
app.settings.S3.Bucket,
|
||||
app.settings.S3.Region,
|
||||
app.settings.S3.Endpoint,
|
||||
@@ -722,10 +758,41 @@ func (app *BaseApp) NewFilesystem() (*filesystem.System, error) {
|
||||
app.settings.S3.Secret,
|
||||
app.settings.S3.ForcePathStyle,
|
||||
)
|
||||
} else {
|
||||
// local filesystem
|
||||
fsys, err = filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// fallback to local filesystem
|
||||
return filesystem.NewLocal(filepath.Join(app.DataDir(), LocalStorageDirName))
|
||||
// attach delete hook
|
||||
if app._onFilesystemDelete.Length() > 0 {
|
||||
fsys.OnDelete().BindFunc(func(originalEvent *filesystem.DeleteEvent) error {
|
||||
appEvent := new(FilesystemDeleteEvent)
|
||||
appEvent.DeleteEvent = originalEvent
|
||||
appEvent.App = app
|
||||
|
||||
return app._onFilesystemDelete.Trigger(appEvent, func(fde *FilesystemDeleteEvent) error {
|
||||
return originalEvent.Next()
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
// attach write hook
|
||||
if app._onFilesystemNewWriter.Length() > 0 {
|
||||
fsys.OnNewWriter().BindFunc(func(originalEvent *filesystem.NewWriterEvent) error {
|
||||
appEvent := new(FilesystemNewWriterEvent)
|
||||
appEvent.NewWriterEvent = originalEvent
|
||||
appEvent.App = app
|
||||
|
||||
return app._onFilesystemNewWriter.Trigger(appEvent, func(fwe *FilesystemNewWriterEvent) error {
|
||||
return originalEvent.Next()
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
return fsys, nil
|
||||
}
|
||||
|
||||
// NewBackupsFilesystem creates a new local or S3 filesystem instance
|
||||
@@ -767,7 +834,7 @@ func (app *BaseApp) Restart() error {
|
||||
event.IsRestart = true
|
||||
|
||||
return app.OnTerminate().Trigger(event, func(e *TerminateEvent) error {
|
||||
_ = e.App.ResetBootstrapState()
|
||||
_ = e.App.ClearBootstrap()
|
||||
|
||||
// attempt to restart the bootstrap process in case execve returns an error for some reason
|
||||
defer func() {
|
||||
@@ -810,6 +877,10 @@ func (app *BaseApp) OnBootstrap() *hook.Hook[*BootstrapEvent] {
|
||||
return app.onBootstrap
|
||||
}
|
||||
|
||||
func (app *BaseApp) OnBootstrapClear() *hook.Hook[*BootstrapEvent] {
|
||||
return app.onBootstrapClear
|
||||
}
|
||||
|
||||
func (app *BaseApp) OnServe() *hook.Hook[*ServeEvent] {
|
||||
return app.onServe
|
||||
}
|
||||
@@ -1016,6 +1087,18 @@ func (app *BaseApp) OnMailerRecordAuthAlertSend(tags ...string) *hook.TaggedHook
|
||||
return hook.NewTaggedHook(app.onMailerRecordAuthAlertSend, tags...)
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Filesystem event hooks
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
func (app *BaseApp) onFilesystemNewWriter() *hook.Hook[*FilesystemNewWriterEvent] {
|
||||
return app._onFilesystemNewWriter
|
||||
}
|
||||
|
||||
func (app *BaseApp) onFilesystemDelete() *hook.Hook[*FilesystemDeleteEvent] {
|
||||
return app._onFilesystemDelete
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Realtime API event hooks
|
||||
// -------------------------------------------------------------------
|
||||
@@ -1173,7 +1256,7 @@ func (app *BaseApp) OnBatchRequest() *hook.Hook[*BatchRequestEvent] {
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
func (app *BaseApp) initDataDB() error {
|
||||
dbPath := filepath.Join(app.DataDir(), "data.db")
|
||||
dbPath := filepath.Join(app.DataDir(), dataDBFilename)
|
||||
|
||||
concurrentDB, err := app.config.DBConnect(dbPath)
|
||||
if err != nil {
|
||||
@@ -1235,7 +1318,7 @@ func normalizeSQLLog(sql string) string {
|
||||
func (app *BaseApp) initAuxDB() error {
|
||||
// note: renamed to "auxiliary" because "aux" is a reserved Windows filename
|
||||
// (see https://github.com/pocketbase/pocketbase/issues/5607)
|
||||
dbPath := filepath.Join(app.DataDir(), "auxiliary.db")
|
||||
dbPath := filepath.Join(app.DataDir(), auxDBFilename)
|
||||
|
||||
concurrentDB, err := app.config.DBConnect(dbPath)
|
||||
if err != nil {
|
||||
@@ -1351,7 +1434,15 @@ func (app *BaseApp) registerBaseHooks() {
|
||||
Id: "__pbCronStart__",
|
||||
Func: func(e *ServeEvent) error {
|
||||
app.Cron().Start()
|
||||
return e.Next()
|
||||
},
|
||||
Priority: 999,
|
||||
})
|
||||
|
||||
app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
|
||||
Id: "__pbCronStop__",
|
||||
Func: func(e *BootstrapEvent) error {
|
||||
app.Cron().Stop()
|
||||
return e.Next()
|
||||
},
|
||||
Priority: 999,
|
||||
@@ -1408,9 +1499,41 @@ func getLoggerMinLevel(app App) slog.Level {
|
||||
}
|
||||
|
||||
func (app *BaseApp) initLogger() error {
|
||||
var stopped atomic.Bool
|
||||
|
||||
duration := 3 * time.Second
|
||||
ticker := time.NewTicker(duration)
|
||||
done := make(chan bool, 1)
|
||||
|
||||
done := make(chan struct{}, 1)
|
||||
|
||||
runLogsWrite := func(logs []*logger.Log) {
|
||||
if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
// write the accumulated logs
|
||||
//
|
||||
// note: based on several local tests there is no
|
||||
// significant performance difference between small number
|
||||
// of separate write queries vs 1 big INSERT
|
||||
app.AuxRunInTransaction(func(txApp App) error {
|
||||
model := &Log{}
|
||||
for _, l := range logs {
|
||||
model.MarkAsNew()
|
||||
model.Id = GenerateDefaultRandomId()
|
||||
model.Level = int(l.Level)
|
||||
model.Message = l.Message
|
||||
model.Data = l.Data
|
||||
model.Created, _ = types.ParseDateTime(l.Time)
|
||||
|
||||
if err := txApp.AuxSave(model); err != nil {
|
||||
log.Println("Failed to write log", model, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
handler := logger.NewBatchHandler(logger.BatchOptions{
|
||||
Level: getLoggerMinLevel(app),
|
||||
@@ -1425,35 +1548,25 @@ func (app *BaseApp) initLogger() error {
|
||||
}
|
||||
}
|
||||
|
||||
ticker.Reset(duration)
|
||||
if !stopped.Load() {
|
||||
ticker.Reset(duration)
|
||||
}
|
||||
|
||||
return app.Settings().Logs.MaxDays > 0
|
||||
},
|
||||
WriteFunc: func(ctx context.Context, logs []*logger.Log) error {
|
||||
if !app.IsBootstrapped() || app.Settings().Logs.MaxDays == 0 {
|
||||
return nil
|
||||
// don't block and wait for the write transaction to complete
|
||||
// when we can't be sure if the logs write wasn't triggered while
|
||||
// inside another AUX db transaction (ticker or batch threshold reached)
|
||||
// which can block indefinitely and cause deadlock
|
||||
// (https://github.com/pocketbase/pocketbase/issues/7836)
|
||||
shouldBlock, _ := ctx.Value(logger.BlockKey).(bool)
|
||||
if shouldBlock {
|
||||
runLogsWrite(logs)
|
||||
} else {
|
||||
routine.FireAndForget(func() { runLogsWrite(logs) })
|
||||
}
|
||||
|
||||
// write the accumulated logs
|
||||
// (note: based on several local tests there is no significant performance difference between small number of separate write queries vs 1 big INSERT)
|
||||
app.AuxRunInTransaction(func(txApp App) error {
|
||||
model := &Log{}
|
||||
for _, l := range logs {
|
||||
model.MarkAsNew()
|
||||
model.Id = GenerateDefaultRandomId()
|
||||
model.Level = int(l.Level)
|
||||
model.Message = l.Message
|
||||
model.Data = l.Data
|
||||
model.Created, _ = types.ParseDateTime(l.Time)
|
||||
|
||||
if err := txApp.AuxSave(model); err != nil {
|
||||
log.Println("Failed to write log", model, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
return nil
|
||||
},
|
||||
})
|
||||
@@ -1473,17 +1586,21 @@ func (app *BaseApp) initLogger() error {
|
||||
|
||||
app.logger = slog.New(handler)
|
||||
|
||||
// write all remaining logs before ticker.Stop to avoid races with ResetBootstrap user calls
|
||||
app.OnTerminate().Bind(&hook.Handler[*TerminateEvent]{
|
||||
Id: "__pbAppLoggerOnTerminate__",
|
||||
Func: func(e *TerminateEvent) error {
|
||||
handler.WriteAll(context.Background())
|
||||
// attempt to write all queued logs before clearing the application bootstrap state
|
||||
app.OnBootstrapClear().Bind(&hook.Handler[*BootstrapEvent]{
|
||||
Id: "__pbAppLoggerFlushBeforeStop__",
|
||||
Func: func(e *BootstrapEvent) error {
|
||||
// extra precaution in case the hook was manually triggered while inside aux db transaction
|
||||
_, isTx := e.App.AuxNonconcurrentDB().(*dbx.Tx)
|
||||
ctx := context.WithValue(context.Background(), logger.BlockKey, !isTx)
|
||||
handler.WriteAll(ctx)
|
||||
|
||||
stopped.Store(true)
|
||||
ticker.Stop()
|
||||
|
||||
// don't block in case OnTerminate is triggered more than once
|
||||
// don't block in case the hook is triggered more than once
|
||||
select {
|
||||
case done <- true:
|
||||
case done <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
|
||||
|
||||
+153
-48
@@ -7,6 +7,7 @@ import (
|
||||
"os"
|
||||
"slices"
|
||||
"testing"
|
||||
"testing/synctest"
|
||||
"time"
|
||||
|
||||
_ "unsafe"
|
||||
@@ -64,7 +65,7 @@ func TestBaseAppBootstrap(t *testing.T) {
|
||||
app := core.NewBaseApp(core.BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
if app.IsBootstrapped() {
|
||||
t.Fatal("Didn't expect the application to be bootstrapped.")
|
||||
@@ -114,7 +115,7 @@ func TestBaseAppBootstrap(t *testing.T) {
|
||||
runNilChecks(nilChecksBeforeReset)
|
||||
|
||||
// reset
|
||||
if err := app.ResetBootstrapState(); err != nil {
|
||||
if err := app.ClearBootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -140,7 +141,7 @@ func TestNewBaseAppTx(t *testing.T) {
|
||||
app := core.NewBaseApp(core.BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
if err := app.Bootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -184,7 +185,7 @@ func TestBaseAppNewMailClient(t *testing.T) {
|
||||
DataDir: testDataDir,
|
||||
EncryptionEnv: "pb_test_env",
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
client1 := app.NewMailClient()
|
||||
m1, ok := client1.(*mailer.Sendmail)
|
||||
@@ -214,7 +215,7 @@ func TestBaseAppNewFilesystem(t *testing.T) {
|
||||
app := core.NewBaseApp(core.BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
// local
|
||||
local, localErr := app.NewFilesystem()
|
||||
@@ -243,7 +244,7 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
|
||||
app := core.NewBaseApp(core.BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
// local
|
||||
local, localErr := app.NewBackupsFilesystem()
|
||||
@@ -265,68 +266,172 @@ func TestBaseAppNewBackupsFilesystem(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
const logsThreshold = 200
|
||||
|
||||
func assertLogsCount(t *testing.T, app core.App, expected int) {
|
||||
var total int
|
||||
|
||||
err := app.LogQuery().Select("count(*)").Row(&total)
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to fetch total logs: %v", err)
|
||||
}
|
||||
|
||||
if total != expected {
|
||||
t.Fatalf("Expected %d log(s), got %d", expected, total)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBaseAppLoggerWrites(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// note: outside of synctest because the bootstrap tickers could deadlock
|
||||
app, _ := tests.NewTestApp()
|
||||
defer app.Cleanup()
|
||||
|
||||
// clear old logs
|
||||
err := app.DeleteOldLogs(time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
t.Run("disabled logs retention", func(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
app.Settings().Logs.MaxDays = 0
|
||||
|
||||
for i := 0; i < logsThreshold+1; i++ {
|
||||
app.Logger().Error("test")
|
||||
}
|
||||
|
||||
// short delay for the non-blocking write goroutine
|
||||
synctest.Sleep(time.Nanosecond)
|
||||
|
||||
assertLogsCount(t, app, 0)
|
||||
})
|
||||
})
|
||||
|
||||
t.Run("test batch logs writes", func(t *testing.T) {
|
||||
synctest.Test(t, func(t *testing.T) {
|
||||
app.Settings().Logs.MaxDays = 2
|
||||
|
||||
for i := 0; i < logsThreshold-1; i++ {
|
||||
app.Logger().Error("test")
|
||||
}
|
||||
|
||||
// short delay for the non-blocking write goroutine
|
||||
synctest.Sleep(time.Nanosecond)
|
||||
|
||||
// below threshold
|
||||
assertLogsCount(t, app, 0)
|
||||
|
||||
// threshold reached -> should trigger batch write
|
||||
app.Logger().Error("test")
|
||||
|
||||
// should be skipped from this batch and added for the next
|
||||
app.Logger().Error("test")
|
||||
|
||||
// short delay for the non-blocking write goroutine
|
||||
synctest.Sleep(time.Nanosecond)
|
||||
|
||||
assertLogsCount(t, app, logsThreshold)
|
||||
|
||||
// note: we can't test the flush timer here because the ticker
|
||||
// was started out of the synctest buble to avoid deadlocks
|
||||
// (see TestBaseAppLoggerWritesAwaited for a flaky but real timer test)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
func TestBaseAppLoggerWritesAwaited(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app, _ := tests.NewTestApp()
|
||||
defer app.Cleanup()
|
||||
|
||||
// reset
|
||||
if err := app.DeleteOldLogs(time.Now()); err != nil {
|
||||
// clear old logs
|
||||
err := app.DeleteOldLogs(time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
const logsThreshold = 200
|
||||
|
||||
totalLogs := func(app core.App, t *testing.T) int {
|
||||
var total int
|
||||
|
||||
err := app.LogQuery().Select("count(*)").Row(&total)
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to fetch total logs: %v", err)
|
||||
}
|
||||
|
||||
return total
|
||||
// enable logs persistence
|
||||
app.Settings().Logs.MaxDays = 1
|
||||
err = app.Save(app.Settings())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
t.Run("disabled logs retention", func(t *testing.T) {
|
||||
app.Settings().Logs.MaxDays = 0
|
||||
t.Run("flush on timer tick", func(t *testing.T) {
|
||||
timeout := time.After(5 * time.Second)
|
||||
done := make(chan struct{})
|
||||
|
||||
for i := 0; i < logsThreshold+1; i++ {
|
||||
app.Logger().Error("test")
|
||||
logsHook := app.OnModelAfterCreateSuccess("_logs")
|
||||
hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
|
||||
done <- struct{}{}
|
||||
return e.Next()
|
||||
})
|
||||
defer logsHook.Unbind(hookId)
|
||||
|
||||
app.Logger().Error("test")
|
||||
|
||||
// short wait to ensure that there is no non-blocking write
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
|
||||
assertLogsCount(t, app, 0)
|
||||
|
||||
// wait for the ticker to write the db record
|
||||
select {
|
||||
case <-timeout:
|
||||
t.Fatal("ticker wait timeout")
|
||||
case <-done:
|
||||
}
|
||||
|
||||
if total := totalLogs(app, t); total != 0 {
|
||||
t.Fatalf("Expected no logs, got %d", total)
|
||||
}
|
||||
assertLogsCount(t, app, 1)
|
||||
})
|
||||
|
||||
t.Run("test batch logs writes", func(t *testing.T) {
|
||||
app.Settings().Logs.MaxDays = 1
|
||||
|
||||
for i := 0; i < logsThreshold-1; i++ {
|
||||
app.Logger().Error("test")
|
||||
}
|
||||
|
||||
if total := totalLogs(app, t); total != 0 {
|
||||
t.Fatalf("Expected no logs, got %d", total)
|
||||
}
|
||||
|
||||
// should trigger batch write
|
||||
t.Run("before ClearBootstrap flush", func(t *testing.T) {
|
||||
app.Logger().Error("test")
|
||||
|
||||
// should be added for the next batch write
|
||||
app.Logger().Error("test")
|
||||
app.Bootstrap()
|
||||
|
||||
if total := totalLogs(app, t); total != logsThreshold {
|
||||
t.Fatalf("Expected %d logs, got %d", logsThreshold, total)
|
||||
assertLogsCount(t, app, 2)
|
||||
})
|
||||
|
||||
t.Run("batch flush inside aux transaction shouldn't hang", func(t *testing.T) {
|
||||
timeout := time.After(1 * time.Second)
|
||||
done := make(chan struct{})
|
||||
totalCreated := 0
|
||||
|
||||
logsHook := app.OnModelAfterCreateSuccess("_logs")
|
||||
hookId := logsHook.BindFunc(func(e *core.ModelEvent) error {
|
||||
totalCreated++
|
||||
if totalCreated == 200 {
|
||||
done <- struct{}{}
|
||||
}
|
||||
return e.Next()
|
||||
})
|
||||
defer logsHook.Unbind(hookId)
|
||||
|
||||
app.AuxRunInTransaction(func(txApp core.App) error {
|
||||
for range logsThreshold {
|
||||
txApp.Logger().Error("test")
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
// wait for the non-blocking write
|
||||
select {
|
||||
case <-timeout:
|
||||
t.Fatal("non-blocking write timeout")
|
||||
case <-done:
|
||||
}
|
||||
|
||||
// wait for ~3 secs to check the timer trigger
|
||||
time.Sleep(3200 * time.Millisecond)
|
||||
if total := totalLogs(app, t); total != logsThreshold+1 {
|
||||
t.Fatalf("Expected %d logs, got %d", logsThreshold+1, total)
|
||||
}
|
||||
assertLogsCount(t, app, 202)
|
||||
|
||||
// force clear to ensure that there are no other logs
|
||||
app.Bootstrap()
|
||||
|
||||
assertLogsCount(t, app, 202)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -369,7 +474,7 @@ func TestBaseAppRefreshSettingsLoggerMinLevelEnabled(t *testing.T) {
|
||||
DataDir: testDataDir,
|
||||
IsDev: s.isDev,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
if err := app.Bootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -4,7 +4,8 @@ import (
|
||||
"cmp"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/jsontext"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -186,7 +187,11 @@ func (app *BaseApp) ImportCollections(toImport []map[string]any, deleteMissing b
|
||||
)
|
||||
if err := validator.run(); err != nil {
|
||||
// serialize the validation error(s)
|
||||
serializedErr, _ := json.MarshalIndent(err, "", " ")
|
||||
serializedErr, _ := json.Marshal(
|
||||
err,
|
||||
jsontext.WithIndentPrefix(""),
|
||||
jsontext.WithIndent(" "),
|
||||
)
|
||||
|
||||
return validation.Errors{"collections": validation.NewError(
|
||||
"validation_collections_import_failure",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -449,7 +449,7 @@ func TestImportCollectionsCreateRules(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(collection)
|
||||
raw, err := json.Marshal(collection, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -522,8 +522,6 @@ func (m *Collection) unmarshalRawOptions() error {
|
||||
// For new/"blank" Collection models it replaces the model with a factory
|
||||
// instance and then unmarshal the provided data one on top of it.
|
||||
func (m *Collection) UnmarshalJSON(b []byte) error {
|
||||
type alias *Collection
|
||||
|
||||
// initialize the default fields
|
||||
// (e.g. in case the collection was NOT created using the designated factories)
|
||||
if m.IsNew() && m.Type == "" {
|
||||
@@ -540,7 +538,8 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
|
||||
*m = *blank
|
||||
}
|
||||
|
||||
return json.Unmarshal(b, alias(m))
|
||||
type alias Collection
|
||||
return json.Unmarshal(b, (*alias)(m))
|
||||
}
|
||||
|
||||
// MarshalJSON implements the [json.Marshaler] interface.
|
||||
@@ -550,10 +549,12 @@ func (m *Collection) UnmarshalJSON(b []byte) error {
|
||||
func (m Collection) MarshalJSON() ([]byte, error) {
|
||||
switch m.Type {
|
||||
case CollectionTypeView:
|
||||
return json.Marshal(struct {
|
||||
alias := struct {
|
||||
baseCollection
|
||||
collectionViewOptions
|
||||
}{m.baseCollection, m.collectionViewOptions})
|
||||
}{m.baseCollection, m.collectionViewOptions}
|
||||
|
||||
return json.Marshal(alias, json.Deterministic(true))
|
||||
case CollectionTypeAuth:
|
||||
alias := struct {
|
||||
baseCollection
|
||||
@@ -582,15 +583,15 @@ func (m Collection) MarshalJSON() ([]byte, error) {
|
||||
alias.OAuth2.Providers = redactedProviders
|
||||
}
|
||||
|
||||
return json.Marshal(alias)
|
||||
return json.Marshal(alias, json.Deterministic(true))
|
||||
default:
|
||||
return json.Marshal(m.baseCollection)
|
||||
return json.Marshal(m.baseCollection, json.Deterministic(true))
|
||||
}
|
||||
}
|
||||
|
||||
// String returns a string representation of the current collection.
|
||||
func (m Collection) String() string {
|
||||
raw, _ := json.Marshal(m)
|
||||
raw, _ := m.MarshalJSON()
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -409,6 +412,71 @@ type OAuth2Config struct {
|
||||
Enabled bool `form:"enabled" json:"enabled"`
|
||||
}
|
||||
|
||||
// UnmarshalJSON implements the [json.Unmarshaler] interface.
|
||||
//
|
||||
// The main difference from the standard unmarshalization is that
|
||||
// instead of replacing the entire providers config slice, we ensure
|
||||
// that partially submitted provider data (e.g. without clientSecret)
|
||||
// is merged on per config level based on the provider name
|
||||
// (https://github.com/pocketbase/pocketbase/issues/7815).
|
||||
func (c *OAuth2Config) UnmarshalJSON(b []byte) error {
|
||||
originalProviders := slices.Clone(c.Providers)
|
||||
|
||||
type alias OAuth2Config
|
||||
err := json.Unmarshal(b, (*alias)(c))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(c.Providers) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// unmarshal again but this time into a plain array of objects
|
||||
// so that we have only the submitted fields and no zero defaults
|
||||
plain := struct {
|
||||
Providers []map[string]any `json:"providers"`
|
||||
}{}
|
||||
err = json.Unmarshal(b, &plain)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// no providers were submitted
|
||||
if len(plain.Providers) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
if len(c.Providers) != len(plain.Providers) {
|
||||
return errors.New("the length of the plain unmarshalized providers and the ones from the config doesn't match")
|
||||
}
|
||||
|
||||
ProvidersMergeLoop:
|
||||
for i, plain := range plain.Providers {
|
||||
for _, original := range originalProviders {
|
||||
if original.Name == plain["name"] {
|
||||
raw, err := json.Marshal(plain)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// unmarshal the new plain data on top of the original one
|
||||
err = json.Unmarshal(raw, &original)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// reassigne to the updated original
|
||||
c.Providers[i] = original
|
||||
|
||||
continue ProvidersMergeLoop
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetProviderConfig returns the first OAuth2ProviderConfig that matches the specified name.
|
||||
//
|
||||
// Returns false and zero config if no such provider is available in c.Providers.
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -712,6 +712,76 @@ func TestPasswordAuthConfigValidate(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestOAuth2ConfigUnmarshalJSON(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
scenarios := []struct {
|
||||
name string
|
||||
newJSON string
|
||||
expected string
|
||||
}{
|
||||
{
|
||||
"missing",
|
||||
`{
|
||||
"enabled": true,
|
||||
"mappedFields": {"username": "username_test"}
|
||||
}`,
|
||||
`{"providers":[{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"b","clientId":"b_clientId","clientSecret":"b_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
|
||||
},
|
||||
{
|
||||
"empty",
|
||||
`{
|
||||
"enabled": true,
|
||||
"mappedFields": {"username": "username_test"},
|
||||
"providers": []
|
||||
}`,
|
||||
`{"providers":[],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
|
||||
},
|
||||
{
|
||||
"non-empty",
|
||||
`{
|
||||
"enabled": true,
|
||||
"mappedFields": {"username": "username_test"},
|
||||
"providers": [
|
||||
{"name": "c", "clientId": "c_clientId", "clientSecret": "c_clientSecret"},
|
||||
{"name": "a", "displayName": "a_displayName"}
|
||||
]
|
||||
}`,
|
||||
`{"providers":[{"pkce":null,"name":"c","clientId":"c_clientId","clientSecret":"c_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"","extra":{}},{"pkce":null,"name":"a","clientId":"a_clientId","clientSecret":"a_clientSecret","authURL":"","tokenURL":"","userInfoURL":"","displayName":"a_displayName","extra":{}}],"mappedFields":{"id":"","name":"name_test","username":"username_test","avatarURL":""},"enabled":true}`,
|
||||
},
|
||||
}
|
||||
|
||||
for _, s := range scenarios {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
config := core.OAuth2Config{
|
||||
Enabled: false,
|
||||
MappedFields: core.OAuth2KnownFields{
|
||||
Name: "name_test",
|
||||
},
|
||||
Providers: []core.OAuth2ProviderConfig{
|
||||
{Name: "a", ClientId: "a_clientId", ClientSecret: "a_clientSecret"},
|
||||
{Name: "b", ClientId: "b_clientId", ClientSecret: "b_clientSecret"},
|
||||
},
|
||||
}
|
||||
|
||||
err := json.Unmarshal([]byte(s.newJSON), &config)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(config, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rawStr := string(raw)
|
||||
|
||||
if rawStr != s.expected {
|
||||
t.Fatalf("Expected OAuth2ProviderConfig\n%s\ngot\n%s", s.expected, rawStr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestOAuth2ConfigGetProviderConfig(t *testing.T) {
|
||||
scenarios := []struct {
|
||||
name string
|
||||
@@ -1016,8 +1086,8 @@ func TestOAuth2ProviderConfigInitProvider(t *testing.T) {
|
||||
t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
|
||||
}
|
||||
|
||||
rawMeta, _ := json.Marshal(provider.Extra())
|
||||
expectedMeta, _ := json.Marshal(s.expectedConfig.Extra)
|
||||
rawMeta, _ := json.Marshal(provider.Extra(), json.Deterministic(true))
|
||||
expectedMeta, _ := json.Marshal(s.expectedConfig.Extra, json.Deterministic(true))
|
||||
if !bytes.Equal(rawMeta, expectedMeta) {
|
||||
t.Fatalf("Expected PKCE %v, got %v", *s.expectedConfig.PKCE, provider.PKCE())
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -618,7 +618,7 @@ func TestCollectionUnmarshalJSON(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rawResult, err := json.Marshal(collection)
|
||||
rawResult, err := json.Marshal(collection, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -829,7 +829,7 @@ func TestCollectionDBExport(t *testing.T) {
|
||||
},
|
||||
{
|
||||
core.CollectionTypeAuth,
|
||||
`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":null,"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":null},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
|
||||
`{"createRule":"1=3","created":"2024-07-01 01:02:03.456Z","deleteRule":"1=5","fields":[{"help":"","hidden":false,"id":"f1_id","name":"f1","presentable":false,"required":false,"system":true,"type":"bool"},{"help":"","hidden":false,"id":"f2_id","name":"f2","presentable":false,"required":true,"system":false,"type":"bool"}],"id":"test_id","indexes":["CREATE INDEX idx1 on test_name(id)","CREATE INDEX idx2 on test_name(id)"],"listRule":"1=1","name":"test_name","options":{"authRule":null,"manageRule":"1=6","authAlert":{"enabled":false,"emailTemplate":{"subject":"","body":""}},"oauth2":{"providers":[],"mappedFields":{"id":"","name":"","username":"","avatarURL":""},"enabled":false},"passwordAuth":{"enabled":false,"identityFields":[]},"mfa":{"enabled":false,"duration":0,"rule":""},"otp":{"enabled":false,"duration":0,"length":0,"emailTemplate":{"subject":"","body":""}},"authToken":{"duration":0},"passwordResetToken":{"duration":0},"emailChangeToken":{"duration":0},"verificationToken":{"duration":0},"fileToken":{"duration":0},"verificationTemplate":{"subject":"","body":""},"resetPasswordTemplate":{"subject":"","body":""},"confirmEmailChangeTemplate":{"subject":"","body":""}},"system":true,"type":"auth","updateRule":"1=4","updated":"2024-07-01 01:02:03.456Z","viewRule":"1=7"}`,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -860,7 +860,7 @@ func TestCollectionDBExport(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(result)
|
||||
raw, err := json.Marshal(result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -3,7 +3,7 @@ package core
|
||||
import (
|
||||
"bytes"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -324,12 +324,12 @@ func resaveViewsWithChangedFields(app App, excludeIds ...string) error {
|
||||
f.SetId("")
|
||||
}
|
||||
|
||||
encodedNewFields, err := json.Marshal(newFields)
|
||||
encodedNewFields, err := json.Marshal(newFields, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
encodedOldFields, err := json.Marshal(oldFields)
|
||||
encodedOldFields, err := json.Marshal(oldFields, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"testing"
|
||||
|
||||
"github.com/pocketbase/dbx"
|
||||
@@ -278,12 +278,12 @@ func TestSingleVsMultipleValuesNormalization(t *testing.T) {
|
||||
t.Fatalf("Failed to load record: %v", err)
|
||||
}
|
||||
|
||||
encodedResult, err := json.Marshal(result)
|
||||
encodedResult, err := json.Marshal(result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to encode result: %v", err)
|
||||
}
|
||||
|
||||
encodedExpectation, err := json.Marshal(s.expected)
|
||||
encodedExpectation, err := json.Marshal(s.expected, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to encode expectation: %v", err)
|
||||
}
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ func DefaultDBConnect(dbPath string) (*dbx.DB, error) {
|
||||
// Note: the busy_timeout pragma must be first because
|
||||
// the connection needs to be set to block on busy before WAL mode
|
||||
// is set in case it hasn't been already set by another connection.
|
||||
pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)"
|
||||
pragmas := "?_pragma=busy_timeout(10000)&_pragma=journal_mode(WAL)&_pragma=journal_size_limit(200000000)&_pragma=synchronous(NORMAL)&_pragma=foreign_keys(ON)&_pragma=temp_store(MEMORY)&_pragma=cache_size(-32000)&_defensive=1"
|
||||
|
||||
db, err := dbx.Open("sqlite", dbPath+pragmas)
|
||||
if err != nil {
|
||||
|
||||
@@ -3,7 +3,7 @@ package core_test
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"slices"
|
||||
"testing"
|
||||
@@ -120,7 +120,11 @@ func TestTableInfo(t *testing.T) {
|
||||
t.Run(fmt.Sprintf("%d_%s", i, s.tableName), func(t *testing.T) {
|
||||
rows, _ := app.TableInfo(s.tableName)
|
||||
|
||||
raw, err := json.Marshal(rows)
|
||||
raw, err := json.Marshal(
|
||||
rows,
|
||||
json.Deterministic(true),
|
||||
json.FormatNilSliceAsNull(true),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -183,7 +183,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
|
||||
t.Fatalf("Failed to resolve request info: %v", err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(info)
|
||||
raw, err := json.Marshal(info, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to serialize request info: %v", err)
|
||||
}
|
||||
@@ -205,7 +205,7 @@ func TestRequestEventRequestInfo(t *testing.T) {
|
||||
t.Fatalf("Failed to resolve request info: %v", err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(info)
|
||||
raw, err := json.Marshal(info, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to serialize request info: %v", err)
|
||||
}
|
||||
@@ -308,7 +308,7 @@ func TestRequestInfoClone(t *testing.T) {
|
||||
|
||||
// check the original data
|
||||
// ---
|
||||
originalRaw, err := json.Marshal(info)
|
||||
originalRaw, err := json.Marshal(info, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to serialize original request info: %v", err)
|
||||
}
|
||||
@@ -321,7 +321,7 @@ func TestRequestInfoClone(t *testing.T) {
|
||||
|
||||
// check the clone data
|
||||
// ---
|
||||
cloneRaw, err := json.Marshal(clone)
|
||||
cloneRaw, err := json.Marshal(clone, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to serialize clone request info: %v", err)
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/auth"
|
||||
"github.com/pocketbase/pocketbase/tools/filesystem"
|
||||
"github.com/pocketbase/pocketbase/tools/hook"
|
||||
"github.com/pocketbase/pocketbase/tools/mailer"
|
||||
"github.com/pocketbase/pocketbase/tools/router"
|
||||
@@ -188,6 +189,24 @@ type MailerRecordEvent struct {
|
||||
Meta map[string]any
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Filesystem events data
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
type FilesystemNewWriterEvent struct {
|
||||
hook.Event
|
||||
*filesystem.NewWriterEvent
|
||||
|
||||
App App
|
||||
}
|
||||
|
||||
type FilesystemDeleteEvent struct {
|
||||
hook.Event
|
||||
*filesystem.DeleteEvent
|
||||
|
||||
App App
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
// Model events data
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
+6
-2
@@ -732,12 +732,16 @@ func (f *FileField) toSliceValue(raw any) []any {
|
||||
case nil:
|
||||
// nothing to cast
|
||||
case *filesystem.File:
|
||||
result = append(result, value)
|
||||
if value != nil {
|
||||
result = append(result, value)
|
||||
}
|
||||
case filesystem.File:
|
||||
result = append(result, &value)
|
||||
case []*filesystem.File:
|
||||
for _, v := range value {
|
||||
result = append(result, v)
|
||||
if v != nil {
|
||||
result = append(result, v)
|
||||
}
|
||||
}
|
||||
case []filesystem.File:
|
||||
for _, v := range value {
|
||||
|
||||
+22
-18
@@ -3,7 +3,7 @@ package core_test
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -98,11 +98,13 @@ func TestFileFieldPrepareValue(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f1Raw, err := json.Marshal(f1)
|
||||
f1Raw, err := json.Marshal(f1, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var nilFile *filesystem.File
|
||||
|
||||
scenarios := []struct {
|
||||
raw any
|
||||
field *core.FileField
|
||||
@@ -114,8 +116,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
|
||||
{123, &core.FileField{MaxSelect: 1}, `"123"`},
|
||||
{"a", &core.FileField{MaxSelect: 1}, `"a"`},
|
||||
{`["a"]`, &core.FileField{MaxSelect: 1}, `"a"`},
|
||||
{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
|
||||
{f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
|
||||
{*f1, &core.FileField{MaxSelect: 1}, string(f1Raw)},
|
||||
{nilFile, &core.FileField{MaxSelect: 1}, `""`},
|
||||
{[]string{}, &core.FileField{MaxSelect: 1}, `""`},
|
||||
{[]string{"a", "b"}, &core.FileField{MaxSelect: 1}, `"b"`},
|
||||
|
||||
@@ -126,8 +129,9 @@ func TestFileFieldPrepareValue(t *testing.T) {
|
||||
{"a", &core.FileField{MaxSelect: 2}, `["a"]`},
|
||||
{`["a"]`, &core.FileField{MaxSelect: 2}, `["a"]`},
|
||||
{[]any{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
|
||||
{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
|
||||
{[]filesystem.File{*f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
|
||||
{[]*filesystem.File{f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
|
||||
{[]any{nilFile, f1}, &core.FileField{MaxSelect: 2}, `[` + string(f1Raw) + `]`},
|
||||
{[]string{}, &core.FileField{MaxSelect: 2}, `[]`},
|
||||
{[]string{"a", "b", "c"}, &core.FileField{MaxSelect: 2}, `["a","b","c"]`},
|
||||
}
|
||||
@@ -139,7 +143,7 @@ func TestFileFieldPrepareValue(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -208,7 +212,7 @@ func TestFileFieldDriverValue(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -695,7 +699,7 @@ func TestFileFieldFindGetter(t *testing.T) {
|
||||
|
||||
v := getter(record)
|
||||
|
||||
raw, err := json.Marshal(v)
|
||||
raw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -812,7 +816,7 @@ func TestFileFieldFindSetter(t *testing.T) {
|
||||
|
||||
setter(record, s.value)
|
||||
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()))
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -911,8 +915,8 @@ func TestFileFieldIntercept(t *testing.T) {
|
||||
|
||||
tests.TestValidationErrors(t, err, []string{"text"})
|
||||
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3})
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3}, json.Deterministic(true))
|
||||
if !bytes.Equal(expectedRaw, raw) {
|
||||
t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
|
||||
}
|
||||
@@ -931,8 +935,8 @@ func TestFileFieldIntercept(t *testing.T) {
|
||||
t.Fatalf("Expected save to succeed, got %v", err)
|
||||
}
|
||||
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
|
||||
if !bytes.Equal(expectedRaw, raw) {
|
||||
t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
|
||||
}
|
||||
@@ -952,8 +956,8 @@ func TestFileFieldIntercept(t *testing.T) {
|
||||
t.Fatalf("Expected save to succeed, got %v", err)
|
||||
}
|
||||
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"))
|
||||
expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
|
||||
expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
|
||||
if !bytes.Equal(expectedRaw, raw) {
|
||||
t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
|
||||
}
|
||||
@@ -1063,8 +1067,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
|
||||
t.Fatalf("Expected save to succeed, got %v", err)
|
||||
}
|
||||
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name})
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
|
||||
expectedRaw, _ := json.Marshal([]any{f1.Name, f3.Name}, json.Deterministic(true))
|
||||
if !bytes.Equal(expectedRaw, raw) {
|
||||
t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
|
||||
}
|
||||
@@ -1086,8 +1090,8 @@ func TestFileFieldInterceptTx(t *testing.T) {
|
||||
t.Fatalf("Expected save to succeed, got %v", err)
|
||||
}
|
||||
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"))
|
||||
expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name})
|
||||
raw, _ := json.Marshal(record.GetRaw("file_many"), json.Deterministic(true))
|
||||
expectedRaw, _ := json.Marshal([]any{f3.Name, f4.Name}, json.Deterministic(true))
|
||||
if !bytes.Equal(expectedRaw, raw) {
|
||||
t.Fatalf("Expected file field value\n%s\ngot\n%s", expectedRaw, raw)
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
@@ -57,7 +57,7 @@ func TestGeoPointFieldPrepareValue(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(v)
|
||||
raw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -2,6 +2,8 @@ package core
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json/jsontext"
|
||||
"encoding/json/v2"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -171,6 +173,18 @@ func (f *JSONField) ValidateValue(ctx context.Context, app App, record *Record)
|
||||
return validation.NewError("validation_invalid_json", "Must be a valid json value")
|
||||
}
|
||||
|
||||
// temp extra encoding/json/v2 check since the above validator is
|
||||
// still using the v1 semantics
|
||||
//
|
||||
// @todo remove after updating the string validator
|
||||
if len(raw) > 0 {
|
||||
var dummy any
|
||||
err := json.Unmarshal(raw, &dummy, jsontext.AllowInvalidUTF8(true))
|
||||
if err != nil {
|
||||
return validation.NewError("validation_invalid_json", "Must be a valid json value")
|
||||
}
|
||||
}
|
||||
|
||||
rawStr := strings.TrimSpace(raw.String())
|
||||
|
||||
if f.Required && slices.Contains(emptyJSONValues, rawStr) {
|
||||
|
||||
@@ -101,6 +101,16 @@ func TestJSONFieldValidateValue(t *testing.T) {
|
||||
},
|
||||
true,
|
||||
},
|
||||
{
|
||||
"invalid raw value (v2 semantics)",
|
||||
&core.JSONField{Name: "test"},
|
||||
func() *core.Record {
|
||||
record := core.NewRecord(collection)
|
||||
record.SetRaw("test", types.JSONRaw(`{"a": 1, "a": 2}`))
|
||||
return record
|
||||
},
|
||||
true,
|
||||
},
|
||||
{
|
||||
"zero field value (not required)",
|
||||
&core.JSONField{Name: "test"},
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
@@ -496,7 +496,7 @@ func TestPasswordFieldFindSetter(t *testing.T) {
|
||||
|
||||
setter(record, s.value)
|
||||
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()))
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
@@ -119,7 +119,7 @@ func TestRelationFieldPrepareValue(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -181,7 +181,7 @@ func TestRelationFieldDriverValue(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -590,7 +590,7 @@ func TestRelationFieldFindSetter(t *testing.T) {
|
||||
|
||||
setter(record, s.value)
|
||||
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()))
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
@@ -119,7 +119,7 @@ func TestSelectFieldPrepareValue(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -181,7 +181,7 @@ func TestSelectFieldDriverValue(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
vRaw, err := json.Marshal(v)
|
||||
vRaw, err := json.Marshal(v, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -503,7 +503,7 @@ func TestSelectFieldFindSetter(t *testing.T) {
|
||||
|
||||
setter(record, s.value)
|
||||
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()))
|
||||
raw, err := json.Marshal(record.Get(s.field.GetName()), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
+4
-4
@@ -2,7 +2,7 @@ package core
|
||||
|
||||
import (
|
||||
"database/sql/driver"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strconv"
|
||||
@@ -277,7 +277,7 @@ func (l *FieldsList) add(pos int, newField Field) {
|
||||
|
||||
// String returns the string representation of the current list.
|
||||
func (l FieldsList) String() string {
|
||||
v, _ := json.Marshal(l)
|
||||
v, _ := l.MarshalJSON()
|
||||
return string(v)
|
||||
}
|
||||
|
||||
@@ -355,12 +355,12 @@ func (l FieldsList) MarshalJSON() ([]byte, error) {
|
||||
wrapper = append(wrapper, data)
|
||||
}
|
||||
|
||||
return json.Marshal(wrapper)
|
||||
return json.Marshal(wrapper, json.Deterministic(true))
|
||||
}
|
||||
|
||||
// Value implements the [driver.Valuer] interface.
|
||||
func (l FieldsList) Value() (driver.Value, error) {
|
||||
data, err := json.Marshal(l)
|
||||
data, err := l.MarshalJSON()
|
||||
|
||||
return string(data), err
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package core_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
+70
-3
@@ -1,13 +1,23 @@
|
||||
package core
|
||||
|
||||
import "github.com/pocketbase/pocketbase/tools/types"
|
||||
import (
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/types"
|
||||
)
|
||||
|
||||
var (
|
||||
_ Model = (*Log)(nil)
|
||||
_ Model = (*Log)(nil)
|
||||
_ DBExporter = (*Log)(nil)
|
||||
)
|
||||
|
||||
const LogsTableName = "_logs"
|
||||
|
||||
const (
|
||||
defaultMaxLogDataSize = 16 << 10 // ~16kb
|
||||
defaultMaxLogMessageSize = 8000
|
||||
)
|
||||
|
||||
type Log struct {
|
||||
BaseModel
|
||||
|
||||
@@ -17,6 +27,63 @@ type Log struct {
|
||||
Level int `db:"level" json:"level"`
|
||||
}
|
||||
|
||||
func (m *Log) TableName() string {
|
||||
func (l *Log) TableName() string {
|
||||
return LogsTableName
|
||||
}
|
||||
|
||||
// DBExport prepares and exports the current log model for db persistence.
|
||||
//
|
||||
// It also truncates the log's message and data to ensure that it is
|
||||
// under app.Settings().Logs.MaxDataSize.
|
||||
func (l *Log) DBExport(app App) (map[string]any, error) {
|
||||
result := map[string]any{
|
||||
"id": l.Id,
|
||||
"created": l.Created,
|
||||
"level": l.Level,
|
||||
}
|
||||
|
||||
// truncate the raw message bytes
|
||||
// (this is expected to be very rare so it is ok even if multi-byte chars)
|
||||
if int64(len(l.Message)) > defaultMaxLogMessageSize {
|
||||
result["message"] = l.Message[:defaultMaxLogMessageSize]
|
||||
} else {
|
||||
result["message"] = l.Message
|
||||
}
|
||||
|
||||
// @todo once added in the standard library consider replacing with
|
||||
// WithByteLimit and WithDepthLimit as suggested in https://github.com/golang/go/issues/56733
|
||||
if len(l.Data) == 0 {
|
||||
result["data"] = l.Data
|
||||
} else {
|
||||
maxDataSize := app.Settings().Logs.MaxDataSize
|
||||
if maxDataSize == 0 {
|
||||
maxDataSize = defaultMaxLogDataSize
|
||||
}
|
||||
|
||||
rawData, err := l.Data.MarshalJSON()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if int64(len(rawData)) > maxDataSize {
|
||||
truncatedData := types.JSONMap[any]{}
|
||||
|
||||
// ignore syntax errors in case of truncated incomplete json
|
||||
//
|
||||
// jsonv2 stream decodes and all "valid" attrs read up to the
|
||||
// invalid part will be populated in truncatedData
|
||||
_ = json.Unmarshal(rawData[:maxDataSize], &truncatedData)
|
||||
|
||||
truncatedData["__pb_truncated__"] = true
|
||||
|
||||
rawData, err = truncatedData.MarshalJSON()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
result["data"] = types.JSONRaw(rawData)
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json/v2"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
"github.com/pocketbase/pocketbase/tests"
|
||||
"github.com/pocketbase/pocketbase/tools/types"
|
||||
)
|
||||
|
||||
func TestLogTableName(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
var log core.Log
|
||||
|
||||
if name := log.TableName(); name != core.LogsTableName {
|
||||
t.Fatalf("Expected Log table name %q, got %q", core.LogsTableName, name)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLogDBExport(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
testApp, _ := tests.NewTestApp()
|
||||
defer testApp.Cleanup()
|
||||
|
||||
date, err := types.ParseDateTime("2026-08-18 10:20:30.456Z")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
messageLimit := 8000
|
||||
dataLimit := 16 << 10
|
||||
|
||||
scenarios := []struct {
|
||||
name string
|
||||
log core.Log
|
||||
limit int64 // 0 -> use default
|
||||
expectJSON string
|
||||
}{
|
||||
{
|
||||
"empty log",
|
||||
core.Log{},
|
||||
0,
|
||||
`{"created":"","data":{},"id":"","level":0,"message":""}`,
|
||||
},
|
||||
{
|
||||
"with message and data below the default limits",
|
||||
core.Log{
|
||||
BaseModel: core.BaseModel{Id: "test_id"},
|
||||
Created: date,
|
||||
Level: 123,
|
||||
Message: "test_message",
|
||||
Data: types.JSONMap[any]{"a": "test1", "b": "test2"},
|
||||
},
|
||||
0,
|
||||
`{"created":"2026-08-18 10:20:30.456Z","data":{"a":"test1","b":"test2"},"id":"test_id","level":123,"message":"test_message"}`,
|
||||
},
|
||||
{
|
||||
"with message and data exactly the default limits",
|
||||
core.Log{
|
||||
BaseModel: core.BaseModel{Id: "test_id"},
|
||||
Created: date,
|
||||
Level: 123,
|
||||
Message: strings.Repeat("a", messageLimit),
|
||||
Data: types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", dataLimit-32)},
|
||||
},
|
||||
0,
|
||||
`{"created":"2026-08-18 10:20:30.456Z","data":{"a":"test1","b":"test2","c":"` + strings.Repeat("a", dataLimit-32) + `"},"id":"test_id","level":123,"message":"` + strings.Repeat("a", messageLimit) + `"}`,
|
||||
},
|
||||
{
|
||||
"with message and data above the default limits",
|
||||
core.Log{
|
||||
BaseModel: core.BaseModel{Id: "test_id"},
|
||||
Created: date,
|
||||
Level: 123,
|
||||
Message: strings.Repeat("a", messageLimit) + "x", // "x" should be omitted
|
||||
Data: types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", dataLimit-32) + "x"}, // the end will be incomplete and something like `"c":"...aaaaaax`
|
||||
},
|
||||
0,
|
||||
`{"created":"2026-08-18 10:20:30.456Z","data":{"__pb_truncated__":true,"a":"test1","b":"test2","c":"` + strings.Repeat("a", dataLimit-32) + `x"},"id":"test_id","level":123,"message":"` + strings.Repeat("a", messageLimit) + `"}`,
|
||||
},
|
||||
{
|
||||
"with data above custom limit",
|
||||
core.Log{
|
||||
BaseModel: core.BaseModel{Id: "test_id"},
|
||||
Created: date,
|
||||
Level: 123,
|
||||
Message: "test_message",
|
||||
Data: types.JSONMap[any]{"a": "test1", "b": "test2", "c": strings.Repeat("a", (2<<10)-32) + "x"}, // the end will be incomplete and something like `"c":"...aaaaaax`
|
||||
},
|
||||
2 << 10,
|
||||
`{"created":"2026-08-18 10:20:30.456Z","data":{"__pb_truncated__":true,"a":"test1","b":"test2","c":"` + strings.Repeat("a", (2<<10)-32) + `x"},"id":"test_id","level":123,"message":"test_message"}`,
|
||||
},
|
||||
}
|
||||
|
||||
for _, s := range scenarios {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
testApp.Settings().Logs.MaxDataSize = s.limit
|
||||
|
||||
result, err := s.log.DBExport(testApp)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if !bytes.Equal(raw, []byte(s.expectJSON)) {
|
||||
t.Fatalf("Expected export data\n%s\ngot\n%s", s.expectJSON, raw)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -3,17 +3,24 @@ package core
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/fatih/color"
|
||||
"github.com/pocketbase/dbx"
|
||||
"github.com/pocketbase/pocketbase/tools/list"
|
||||
"github.com/pocketbase/pocketbase/tools/logger"
|
||||
)
|
||||
|
||||
func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
|
||||
// temp unset to avoid littering the stdout if the test fails when in dev mode
|
||||
colorOutput := color.Output
|
||||
color.Output = io.Discard
|
||||
defer func() { color.Output = colorOutput }()
|
||||
|
||||
testLogLevel := 4
|
||||
|
||||
scenarios := []struct {
|
||||
@@ -48,7 +55,7 @@ func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
|
||||
DataDir: testDataDir,
|
||||
IsDev: s.isDev,
|
||||
})
|
||||
defer app.ResetBootstrapState()
|
||||
defer app.ClearBootstrap()
|
||||
|
||||
if err := app.Bootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -68,7 +75,7 @@ func TestBaseAppLoggerLevelDevPrint(t *testing.T) {
|
||||
var printedLevels []int
|
||||
var persistedLevels []int
|
||||
|
||||
ctx := context.Background()
|
||||
ctx := context.WithValue(context.Background(), logger.BlockKey, true)
|
||||
|
||||
// track printed logs
|
||||
originalPrintLog := printLog
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -63,7 +63,7 @@ func TestLogsStats(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
encoded, _ := json.Marshal(result)
|
||||
encoded, _ := json.Marshal(result, json.Deterministic(true))
|
||||
if string(encoded) != expected {
|
||||
t.Fatalf("Expected\n%q\ngot\n%q", expected, string(encoded))
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -354,7 +354,7 @@ func (r *RecordFieldResolver) resolveStaticRequestField(path ...string) (*search
|
||||
|
||||
// if that doesn't work, try encoding it
|
||||
if castErr != nil {
|
||||
encoded, jsonErr := json.Marshal(v)
|
||||
encoded, jsonErr := json.Marshal(v, json.Deterministic(true))
|
||||
if jsonErr == nil {
|
||||
val = string(encoded)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
@@ -333,7 +333,7 @@ func (r *runner) processRequestBodyEachModifier(bodyField Field) (*search.Resolv
|
||||
}
|
||||
|
||||
bodyItems := toSlice(r.resolver.requestInfo.Body[bodyField.GetName()])
|
||||
bodyItemsRaw, err := json.Marshal(bodyItems)
|
||||
bodyItemsRaw, err := json.Marshal(bodyItems, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cannot serialize the data for field %q", r.activeProps[2])
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
@@ -940,7 +940,7 @@ func TestRecordFieldResolverResolveStaticRequestInfoFields(t *testing.T) {
|
||||
t.Fatalf("Expected parameter r.Identifier %q, got %q", paramName, r.Identifier)
|
||||
}
|
||||
|
||||
encodedParamValue, _ := json.Marshal(paramValue)
|
||||
encodedParamValue, _ := json.Marshal(paramValue, json.Deterministic(true))
|
||||
if string(encodedParamValue) != s.expectParamValue {
|
||||
t.Fatalf("Expected r.Params %#v for %s, got %#v", s.expectParamValue, r.Identifier, string(encodedParamValue))
|
||||
}
|
||||
|
||||
+41
-16
@@ -3,7 +3,9 @@ package core
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"database/sql"
|
||||
"encoding/json/jsontext"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
@@ -959,6 +961,11 @@ func (m *Record) GetInt(key string) int {
|
||||
return cast.ToInt(m.Get(key))
|
||||
}
|
||||
|
||||
// GetInt64 returns the data value for "key" as an int64.
|
||||
func (m *Record) GetInt64(key string) int64 {
|
||||
return cast.ToInt64(m.Get(key))
|
||||
}
|
||||
|
||||
// GetFloat returns the data value for "key" as a float64.
|
||||
func (m *Record) GetFloat(key string) float64 {
|
||||
return cast.ToFloat64(m.Get(key))
|
||||
@@ -1218,12 +1225,12 @@ func areValuesEqual(a any, b any) bool {
|
||||
bv, ok := b.(types.JSONRaw)
|
||||
return ok && bytes.Equal(av, bv)
|
||||
default:
|
||||
aRaw, err := json.Marshal(a)
|
||||
aRaw, err := json.Marshal(a, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
|
||||
bRaw, err := json.Marshal(b)
|
||||
bRaw, err := json.Marshal(b, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
@@ -1324,7 +1331,14 @@ func (record *Record) PublicExport() map[string]any {
|
||||
//
|
||||
// Only the data exported by `PublicExport()` will be serialized.
|
||||
func (m Record) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(m.PublicExport())
|
||||
return json.Marshal(
|
||||
m.PublicExport(),
|
||||
json.Deterministic(true),
|
||||
// for compliance with old data (and slightly better performance)
|
||||
jsontext.AllowDuplicateNames(true),
|
||||
// preserve the old jsonv1 behavior in case of invalid data
|
||||
jsontext.AllowInvalidUTF8(true),
|
||||
)
|
||||
}
|
||||
|
||||
// UnmarshalJSON implements the [json.Unmarshaler] interface.
|
||||
@@ -1521,12 +1535,14 @@ func cascadeRecordDelete(app App, mainRecord *Record, refs map[*Collection][]Fie
|
||||
continue // skip missing or view collections
|
||||
}
|
||||
|
||||
recordTableName := inflector.Columnify(refCollection.Name)
|
||||
refTableName := inflector.Columnify(refCollection.Name)
|
||||
|
||||
for _, field := range fields {
|
||||
prefixedFieldName := recordTableName + "." + inflector.Columnify(field.GetName())
|
||||
prefixedFieldName := refTableName + "." + inflector.Columnify(field.GetName())
|
||||
|
||||
query := app.RecordQuery(refCollection)
|
||||
// fetch only the related ids because they will be queried anyway right
|
||||
// before delete to ensure that we are working with fresh record data
|
||||
query := app.DB().Select(refTableName + ".id").From(refTableName)
|
||||
|
||||
if opt, ok := field.(MultiValuer); !ok || !opt.IsMultiple() {
|
||||
query.AndWhere(dbx.HashExp{prefixedFieldName: mainRecord.Id})
|
||||
@@ -1540,23 +1556,24 @@ func cascadeRecordDelete(app App, mainRecord *Record, refs map[*Collection][]Fie
|
||||
}
|
||||
|
||||
if refCollection.Id == mainRecord.Collection().Id {
|
||||
query.AndWhere(dbx.Not(dbx.HashExp{recordTableName + ".id": mainRecord.Id}))
|
||||
query.AndWhere(dbx.Not(dbx.HashExp{refTableName + ".id": mainRecord.Id}))
|
||||
}
|
||||
|
||||
// trigger cascade for each batchSize rel items until there is none
|
||||
batchSize := 4000
|
||||
rows := make([]*Record, 0, batchSize)
|
||||
batchSize := 8000
|
||||
refIds := make([]string, 0, batchSize)
|
||||
for {
|
||||
if err := query.Limit(int64(batchSize)).All(&rows); err != nil {
|
||||
err := query.Limit(int64(batchSize)).Column(&refIds)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
total := len(rows)
|
||||
total := len(refIds)
|
||||
if total == 0 {
|
||||
break
|
||||
}
|
||||
|
||||
err := deleteRefRecords(app, mainRecord, rows, field)
|
||||
err = deleteRefRecords(app, mainRecord, refCollection, refIds, field)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -1565,7 +1582,7 @@ func cascadeRecordDelete(app App, mainRecord *Record, refs map[*Collection][]Fie
|
||||
break // no more items
|
||||
}
|
||||
|
||||
rows = rows[:0] // keep allocated memory
|
||||
refIds = refIds[:0] // keep allocated memory
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1578,13 +1595,21 @@ func cascadeRecordDelete(app App, mainRecord *Record, refs map[*Collection][]Fie
|
||||
// just unset the record id from any relation field values (if they are not required).
|
||||
//
|
||||
// NB! This method is expected to be called from inside of a transaction.
|
||||
func deleteRefRecords(app App, mainRecord *Record, refRecords []*Record, field Field) error {
|
||||
func deleteRefRecords(app App, mainRecord *Record, refCollection *Collection, refIds []string, field Field) error {
|
||||
relField, _ := field.(*RelationField)
|
||||
if relField == nil {
|
||||
return errors.New("only RelationField is supported at the moment, got " + field.Type())
|
||||
}
|
||||
|
||||
for _, refRecord := range refRecords {
|
||||
for _, refId := range refIds {
|
||||
refRecord, err := app.FindRecordById(refCollection, refId)
|
||||
if err != nil {
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
continue // already deleted
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
ids := refRecord.GetStringSlice(relField.Name)
|
||||
|
||||
// unset the record id
|
||||
|
||||
+164
-28
@@ -4,7 +4,8 @@ import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/jsontext"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
@@ -31,7 +32,7 @@ func TestNewRecord(t *testing.T) {
|
||||
|
||||
m := core.NewRecord(collection)
|
||||
|
||||
rawData, err := json.Marshal(m.FieldsData()) // should be initialized with the defaults
|
||||
rawData, err := json.Marshal(m.FieldsData(), json.Deterministic(true)) // should be initialized with the defaults
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -520,7 +521,7 @@ func TestRecordMergeExpand(t *testing.T) {
|
||||
|
||||
result := m.Expand()
|
||||
|
||||
raw, err := json.Marshal(result)
|
||||
raw, err := json.Marshal(result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -566,7 +567,7 @@ func TestRecordMergeExpandNilCheck(t *testing.T) {
|
||||
m := core.NewRecord(collection)
|
||||
m.MergeExpand(s.expand)
|
||||
|
||||
raw, err := json.Marshal(m)
|
||||
raw, err := json.Marshal(m, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -663,7 +664,7 @@ func TestRecordFieldsData(t *testing.T) {
|
||||
m.Set("field2", 456)
|
||||
m.Set("unknown", 789)
|
||||
|
||||
raw, err := json.Marshal(m.FieldsData())
|
||||
raw, err := json.Marshal(m.FieldsData(), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -691,7 +692,7 @@ func TestRecordCustomData(t *testing.T) {
|
||||
m.Set("field2", 456)
|
||||
m.Set("unknown", 789)
|
||||
|
||||
raw, err := json.Marshal(m.CustomData())
|
||||
raw, err := json.Marshal(m.CustomData(), json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -899,6 +900,43 @@ func TestRecordGetInt(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordGetInt64(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
scenarios := []struct {
|
||||
value any
|
||||
expected int64
|
||||
}{
|
||||
{nil, 0},
|
||||
{"", 0},
|
||||
{[]string{"true"}, 0},
|
||||
{map[string]int{"test": 1}, 0},
|
||||
{time.Now(), 0},
|
||||
{"test", 0},
|
||||
{123, 123},
|
||||
{2.4, 2},
|
||||
{1<<63 - 1, 1<<63 - 1},
|
||||
{"123", 123},
|
||||
{"123.5", 123},
|
||||
{false, 0},
|
||||
{true, 1},
|
||||
}
|
||||
|
||||
collection := core.NewBaseCollection("test")
|
||||
record := core.NewRecord(collection)
|
||||
|
||||
for i, s := range scenarios {
|
||||
t.Run(fmt.Sprintf("%d_%#v", i, s.value), func(t *testing.T) {
|
||||
record.Set("test", s.value)
|
||||
|
||||
result := record.GetInt64("test")
|
||||
if result != s.expected {
|
||||
t.Fatalf("Expected %v, got %v", s.expected, result)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordGetFloat(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -1103,14 +1141,16 @@ func TestRecordGetUnsavedFiles(t *testing.T) {
|
||||
t.Run(fmt.Sprintf("%d_%#v", i, s.key), func(t *testing.T) {
|
||||
v := record.GetUnsavedFiles(s.key)
|
||||
|
||||
raw, err := json.Marshal(v)
|
||||
raw, err := json.Marshal(v,
|
||||
json.Deterministic(true),
|
||||
json.FormatNilSliceAsNull(true),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rawStr := string(raw)
|
||||
|
||||
if rawStr != s.expected {
|
||||
t.Fatalf("Expected\n%s\ngot\n%s", s.expected, rawStr)
|
||||
if str := string(raw); str != s.expected {
|
||||
t.Fatalf("Expected\n%s\ngot\n%s", s.expected, str)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1164,7 +1204,7 @@ func TestRecordUnmarshalJSONField(t *testing.T) {
|
||||
t.Fatalf("Expected hasErr %v, got %v", s.expectError, hasErr)
|
||||
}
|
||||
|
||||
raw, _ := json.Marshal(s.destination)
|
||||
raw, _ := json.Marshal(s.destination, json.Deterministic(true))
|
||||
if v := string(raw); v != s.expectedJSON {
|
||||
t.Fatalf("Expected %q, got %q", s.expectedJSON, v)
|
||||
}
|
||||
@@ -1271,7 +1311,7 @@ func TestRecordDBExport(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(result)
|
||||
raw, err := json.Marshal(result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -1373,14 +1413,15 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
f3 := &core.SelectField{Name: "field3", MaxSelect: 2, Values: []string{"test1", "test2", "test3"}}
|
||||
f4 := &core.TextField{Name: "field4", Hidden: true}
|
||||
f5 := &core.TextField{Name: "field5", Hidden: true}
|
||||
f6 := &core.JSONField{Name: "field6"}
|
||||
|
||||
colBase := core.NewBaseCollection("test_base")
|
||||
colBase.Id = "_pbc_base_123"
|
||||
colBase.Fields.Add(f1, f2, f3, f4, f5)
|
||||
colBase.Fields.Add(f1, f2, f3, f4, f5, f6)
|
||||
|
||||
colAuth := core.NewAuthCollection("test_auth")
|
||||
colAuth.Id = "_pbc_auth_123"
|
||||
colAuth.Fields.Add(f1, f2, f3, f4, f5)
|
||||
colAuth.Fields.Add(f1, f2, f3, f4, f5, f6)
|
||||
|
||||
scenarios := []struct {
|
||||
name string
|
||||
@@ -1399,7 +1440,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
false,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id"}`,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id"}`,
|
||||
},
|
||||
{
|
||||
"[base] with email visibility",
|
||||
@@ -1408,7 +1449,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
false,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id"}`,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id"}`,
|
||||
},
|
||||
{
|
||||
"[base] with custom data",
|
||||
@@ -1417,7 +1458,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","email":"test_email","emailVisibility":"test_invalid","expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id","password":"test_passwordHash","tokenKey":"test_tokenKey","unknown":"test_unknown","verified":true}`,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","email":"test_email","emailVisibility":"test_invalid","expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id","password":"test_passwordHash","tokenKey":"test_tokenKey","unknown":"test_unknown","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[base] with explicit hide and unhide fields",
|
||||
@@ -1426,7 +1467,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
[]string{"field3", "field1", "expand", "collectionId", "collectionName", "email", "tokenKey", "unknown"},
|
||||
[]string{"field4", "@pbInternalAbc"},
|
||||
`{"emailVisibility":"test_invalid","field2":"field_2.png","field4":"field_4","id":"test_id","password":"test_passwordHash","verified":true}`,
|
||||
`{"emailVisibility":"test_invalid","field2":"field_2.png","field4":"field_4","field6":{"a":1,"a":2},"id":"test_id","password":"test_passwordHash","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[base] trying to unhide custom fields without explicit WithCustomData",
|
||||
@@ -1435,7 +1476,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
nil,
|
||||
[]string{"field5", "@pbInternalAbc", "email", "tokenKey", "unknown"},
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","email":"test_email","emailVisibility":"test_invalid","expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"field5":"field_5","id":"test_id","password":"test_passwordHash","tokenKey":"test_tokenKey","unknown":"test_unknown","verified":true}`,
|
||||
`{"collectionId":"_pbc_base_123","collectionName":"test_base","email":"test_email","emailVisibility":"test_invalid","expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field5":"field_5","field6":{"a":1,"a":2},"id":"test_id","password":"test_passwordHash","tokenKey":"test_tokenKey","unknown":"test_unknown","verified":true}`,
|
||||
},
|
||||
|
||||
// auth
|
||||
@@ -1446,7 +1487,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
false,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id","verified":true}`,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[auth] with email visibility",
|
||||
@@ -1455,7 +1496,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
false,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","email":"test_email","emailVisibility":false,"expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id","verified":true}`,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","email":"test_email","emailVisibility":false,"expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[auth] with custom data",
|
||||
@@ -1464,7 +1505,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
nil,
|
||||
nil,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"id":"test_id","unknown":"test_unknown","verified":true}`,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field6":{"a":1,"a":2},"id":"test_id","unknown":"test_unknown","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[auth] with explicit hide and unhide fields",
|
||||
@@ -1473,7 +1514,7 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
[]string{"field3", "field1", "expand", "collectionId", "collectionName", "email", "unknown"},
|
||||
[]string{"field4", "@pbInternalAbc"},
|
||||
`{"emailVisibility":false,"field2":"field_2.png","field4":"field_4","id":"test_id","verified":true}`,
|
||||
`{"emailVisibility":false,"field2":"field_2.png","field4":"field_4","field6":{"a":1,"a":2},"id":"test_id","verified":true}`,
|
||||
},
|
||||
{
|
||||
"[auth] trying to unhide custom fields without explicit WithCustomData",
|
||||
@@ -1482,17 +1523,18 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
true,
|
||||
nil,
|
||||
[]string{"field5", "@pbInternalAbc", "tokenKey", "unknown", "email"}, // emailVisibility:false has higher priority
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1","field2":"field_2.png","field3":["test1","test2"],"field5":"field_5","id":"test_id","unknown":"test_unknown","verified":true}`,
|
||||
`{"collectionId":"_pbc_auth_123","collectionName":"test_auth","emailVisibility":false,"expand":{"test":123},"field1":"field_1�","field2":"field_2.png","field3":["test1","test2"],"field5":"field_5","field6":{"a":1,"a":2},"id":"test_id","unknown":"test_unknown","verified":true}`,
|
||||
},
|
||||
}
|
||||
|
||||
data := map[string]any{
|
||||
"id": "test_id",
|
||||
"field1": "field_1",
|
||||
"field1": "field_1\xc3", /* invalid utf8 suffix to test mangling */
|
||||
"field2": "field_2.png",
|
||||
"field3": []string{"test1", "test2"},
|
||||
"field4": "field_4",
|
||||
"field5": "field_5",
|
||||
"field6": types.JSONRaw(`{"a":1,"a":2}`), // intentionally duplicated to check serialization
|
||||
"expand": map[string]any{"test": 123},
|
||||
"collectionId": "m_id", // should be always ignored
|
||||
"collectionName": "m_name", // should be always ignored
|
||||
@@ -1515,7 +1557,12 @@ func TestRecordPublicExportAndMarshalJSON(t *testing.T) {
|
||||
m.Unhide(s.unhideFields...)
|
||||
m.Hide(s.hideFields...)
|
||||
|
||||
exportResult, err := json.Marshal(m.PublicExport())
|
||||
exportResult, err := json.Marshal(
|
||||
m.PublicExport(),
|
||||
json.Deterministic(true),
|
||||
jsontext.AllowDuplicateNames(true),
|
||||
jsontext.AllowInvalidUTF8(true),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -2280,16 +2327,105 @@ func TestRecordDelete(t *testing.T) {
|
||||
}
|
||||
// ensure that the json rel fields were prefixed
|
||||
joinedQueries := strings.Join(calledQueries, " ")
|
||||
expectedRelManyPart := "SELECT `demo1`.* FROM `demo1` WHERE EXISTS (SELECT 1 FROM json_each(CASE WHEN iif(json_valid([[demo1.rel_many]]), json_type([[demo1.rel_many]])='array', FALSE) THEN [[demo1.rel_many]] ELSE json_array([[demo1.rel_many]]) END) {{__je__}} WHERE [[__je__.value]]='"
|
||||
expectedRelManyPart := "SELECT `demo1`.`id` FROM `demo1` WHERE EXISTS (SELECT 1 FROM json_each(CASE WHEN iif(json_valid([[demo1.rel_many]]), json_type([[demo1.rel_many]])='array', FALSE) THEN [[demo1.rel_many]] ELSE json_array([[demo1.rel_many]]) END) {{__je__}} WHERE [[__je__.value]]='"
|
||||
if !strings.Contains(joinedQueries, expectedRelManyPart) {
|
||||
t.Fatalf("(rec3) Expected the cascade delete to call the query \n%v, got \n%v", expectedRelManyPart, calledQueries)
|
||||
}
|
||||
expectedRelOnePart := "SELECT `demo1`.* FROM `demo1` WHERE (`demo1`.`rel_one`='"
|
||||
expectedRelOnePart := "SELECT `demo1`.`id` FROM `demo1` WHERE (`demo1`.`rel_one`='"
|
||||
if !strings.Contains(joinedQueries, expectedRelOnePart) {
|
||||
t.Fatalf("(rec3) Expected the cascade delete to call the query \n%v, got \n%v", expectedRelOnePart, calledQueries)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordDeleteWithMultipleRelationCascade(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
app, _ := tests.NewTestApp()
|
||||
defer app.Cleanup()
|
||||
|
||||
// create a mock collection with self referencing multiple relation field
|
||||
// ---
|
||||
collection := core.NewBaseCollection("test")
|
||||
err := app.Save(collection)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// for simpler mocks
|
||||
collection.Fields.GetByName("id").(*core.TextField).Min = 1
|
||||
|
||||
collection.Fields.Add(&core.RelationField{
|
||||
Name: "rels",
|
||||
CollectionId: collection.Id,
|
||||
MaxSelect: 99,
|
||||
CascadeDelete: true,
|
||||
})
|
||||
|
||||
err = app.Save(collection)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// create mock records
|
||||
// ---
|
||||
relsData := map[string][]string{
|
||||
"a": nil,
|
||||
"b": {"a"},
|
||||
"c": {"a", "b"},
|
||||
"d": {},
|
||||
"e": {"c", "d"},
|
||||
}
|
||||
for id, rels := range relsData {
|
||||
record := core.NewRecord(collection)
|
||||
record.Set("id", id)
|
||||
record.Set("rels", rels)
|
||||
err = app.SaveNoValidate(record) // map is not ordered
|
||||
if err != nil {
|
||||
t.Fatalf("failed to create mock record: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// trigger cascade delete for the top record
|
||||
// ---
|
||||
aRecord, err := app.FindRecordById(collection, "a")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err = app.Delete(aRecord)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// verify cascade delete
|
||||
// ---
|
||||
expectedRels := map[string][]string{
|
||||
"d": {},
|
||||
"e": {"d"},
|
||||
}
|
||||
|
||||
allRecords, err := app.FindAllRecords(collection)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if len(allRecords) != len(expectedRels) {
|
||||
t.Fatalf("Expected %d remaining records, got %d", len(expectedRels), len(allRecords))
|
||||
}
|
||||
|
||||
for _, r := range allRecords {
|
||||
expected, ok := expectedRels[r.Id]
|
||||
if !ok {
|
||||
t.Fatalf("Record %q wasn't found in %v", r.Id, expectedRels)
|
||||
}
|
||||
|
||||
rels := r.GetStringSlice("rels")
|
||||
if !slices.Equal(rels, expected) {
|
||||
t.Fatalf("Record %q expected rels\n%v\ngot\n%v", r.Id, expected, rels)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordDeleteBatchProcessing(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -104,7 +104,7 @@ func (app *BaseApp) RecordQuery(collectionModelOrIdentifier any) *dbx.SelectQuer
|
||||
return nil
|
||||
default: // expects []RecordProxy slice
|
||||
rv := reflect.ValueOf(v)
|
||||
if rv.Kind() != reflect.Ptr || rv.IsNil() {
|
||||
if rv.Kind() != reflect.Pointer || rv.IsNil() {
|
||||
return errors.New("must be a pointer")
|
||||
}
|
||||
|
||||
@@ -117,7 +117,7 @@ func (app *BaseApp) RecordQuery(collectionModelOrIdentifier any) *dbx.SelectQuer
|
||||
et := rv.Type().Elem()
|
||||
|
||||
var isSliceOfPointers bool
|
||||
if et.Kind() == reflect.Ptr {
|
||||
if et.Kind() == reflect.Pointer {
|
||||
isSliceOfPointers = true
|
||||
et = et.Elem()
|
||||
}
|
||||
@@ -182,7 +182,7 @@ func resolveRecordAllHook(collection *Collection, op func(dst any) error) ([]*Re
|
||||
|
||||
// dereference returns the underlying value v points to.
|
||||
func dereference(v reflect.Value) reflect.Value {
|
||||
for v.Kind() == reflect.Ptr {
|
||||
for v.Kind() == reflect.Pointer {
|
||||
if v.IsNil() {
|
||||
// initialize with a new value and continue searching
|
||||
v.Set(reflect.New(v.Type().Elem()))
|
||||
|
||||
@@ -3,7 +3,7 @@ package core_test
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -240,7 +240,7 @@ func TestExpandRecords(t *testing.T) {
|
||||
t.Errorf("Expected %d failures, got %d\n%v", s.expectExpandFailures, len(failed), failed)
|
||||
}
|
||||
|
||||
encoded, _ := json.Marshal(records)
|
||||
encoded, _ := json.Marshal(records, json.Deterministic(true))
|
||||
encodedStr := string(encoded)
|
||||
totalExpandProps := strings.Count(encodedStr, `"`+core.FieldNameExpand+`":`)
|
||||
totalEmptyExpands := strings.Count(encodedStr, `"`+core.FieldNameExpand+`":{}`)
|
||||
@@ -420,7 +420,7 @@ func TestExpandRecord(t *testing.T) {
|
||||
t.Errorf("Expected %d failures, got %d\n%v", s.expectExpandFailures, len(failed), failed)
|
||||
}
|
||||
|
||||
encoded, _ := json.Marshal(record)
|
||||
encoded, _ := json.Marshal(record, json.Deterministic(true))
|
||||
encodedStr := string(encoded)
|
||||
totalExpandProps := strings.Count(encodedStr, `"`+core.FieldNameExpand+`":`)
|
||||
totalEmptyExpands := strings.Count(encodedStr, `"`+core.FieldNameExpand+`":{}`)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"slices"
|
||||
@@ -107,7 +107,7 @@ func TestRecordQueryOne(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(s.model)
|
||||
raw, err := json.Marshal(s.model, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -192,7 +192,7 @@ func TestRecordQueryAll(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(s.result)
|
||||
raw, err := json.Marshal(s.result, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
+13
-45
@@ -2,7 +2,7 @@ package core
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -237,7 +237,7 @@ func (s *Settings) String() string {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
|
||||
raw, _ := json.Marshal(s)
|
||||
raw, _ := s.MarshalJSON()
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
@@ -257,14 +257,7 @@ func (s *Settings) DBExport(app App) (map[string]any, error) {
|
||||
}
|
||||
result["updated"] = now
|
||||
|
||||
// @todo remove with encoding/json/2
|
||||
// serialize as empty array
|
||||
//nolint:staticcheck
|
||||
if s.settings.SuperuserIPs == nil {
|
||||
s.settings.SuperuserIPs = []string{}
|
||||
}
|
||||
|
||||
encoded, err := json.Marshal(s.settings)
|
||||
encoded, err := json.Marshal(s.settings, json.Deterministic(true))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -355,13 +348,7 @@ func (s *Settings) MarshalJSON() ([]byte, error) {
|
||||
}
|
||||
}
|
||||
|
||||
// @todo remove with encoding/json/2
|
||||
// serialize as empty array
|
||||
if copy.SuperuserIPs == nil {
|
||||
copy.SuperuserIPs = []string{}
|
||||
}
|
||||
|
||||
return json.Marshal(copy)
|
||||
return json.Marshal(copy, json.Deterministic(true))
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------
|
||||
@@ -559,6 +546,12 @@ func (c MetaConfig) Validate() error {
|
||||
// -------------------------------------------------------------------
|
||||
|
||||
type LogsConfig struct {
|
||||
// MaxDataSize specifies the maximum allowed serialized log data
|
||||
// size before it gets truncated (see [Log.DBExport]).
|
||||
//
|
||||
// If zero, fallbacks to ~16kb by default.
|
||||
MaxDataSize int64 `form:"maxDataSize" json:"maxDataSize"`
|
||||
|
||||
MaxDays int `form:"maxDays" json:"maxDays"`
|
||||
MinLevel int `form:"minLevel" json:"minLevel"`
|
||||
LogIP bool `form:"logIP" json:"logIP"`
|
||||
@@ -568,7 +561,9 @@ type LogsConfig struct {
|
||||
// Validate makes LogsConfig validatable by implementing [validation.Validatable] interface.
|
||||
func (c LogsConfig) Validate() error {
|
||||
return validation.ValidateStruct(&c,
|
||||
validation.Field(&c.MaxDays, validation.Min(0)),
|
||||
validation.Field(&c.MaxDataSize, validation.Min(0), validation.Max(maxSafeJSONInt)),
|
||||
validation.Field(&c.MaxDays, validation.Min(0), validation.Max(maxSafeJSONInt)),
|
||||
validation.Field(&c.MinLevel, validation.Max(maxSafeJSONInt)),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -586,18 +581,6 @@ type TrustedProxyConfig struct {
|
||||
UseLeftmostIP bool `form:"useLeftmostIP" json:"useLeftmostIP"`
|
||||
}
|
||||
|
||||
// MarshalJSON implements the [json.Marshaler] interface.
|
||||
func (c TrustedProxyConfig) MarshalJSON() ([]byte, error) {
|
||||
type alias TrustedProxyConfig
|
||||
|
||||
// serialize as empty array
|
||||
if c.Headers == nil {
|
||||
c.Headers = []string{}
|
||||
}
|
||||
|
||||
return json.Marshal(alias(c))
|
||||
}
|
||||
|
||||
// Validate makes RateLimitRule validatable by implementing [validation.Validatable] interface.
|
||||
func (c TrustedProxyConfig) Validate() error {
|
||||
return nil
|
||||
@@ -645,21 +628,6 @@ func (c *RateLimitsConfig) FindRateLimitRule(searchLabels []string, optOnlyAudie
|
||||
return RateLimitRule{}, false
|
||||
}
|
||||
|
||||
// MarshalJSON implements the [json.Marshaler] interface.
|
||||
func (c RateLimitsConfig) MarshalJSON() ([]byte, error) {
|
||||
type alias RateLimitsConfig
|
||||
|
||||
// serialize as empty array
|
||||
if c.Rules == nil {
|
||||
c.Rules = []RateLimitRule{}
|
||||
}
|
||||
if c.ExcludedIPs == nil {
|
||||
c.ExcludedIPs = []string{}
|
||||
}
|
||||
|
||||
return json.Marshal(alias(c))
|
||||
}
|
||||
|
||||
// Validate makes RateLimitsConfig validatable by implementing [validation.Validatable] interface.
|
||||
func (c RateLimitsConfig) Validate() error {
|
||||
return validation.ValidateStruct(&c,
|
||||
|
||||
+14
-11
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
@@ -84,7 +84,7 @@ func TestSettings_DBExport(t *testing.T) {
|
||||
valueStr = string(export["value"].([]byte))
|
||||
}
|
||||
|
||||
expected := `{"superuserIPs":[],"smtp":{"enabled":false,"port":0,"host":"smtp_host","username":"smtp_username","password":"","authMethod":"","tls":false,"localName":""},"backups":{"cron":"* * * * *","cronMaxKeep":0,"s3":{"enabled":true,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false}},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"s3_endpoint","accessKey":"","secret":"s3_secret","forcePathStyle":false},"meta":{"accentColor":"","appName":"test_app_name","appURL":"","senderName":"","senderAddress":"","hideControls":false},"rateLimits":{"rules":[],"excludedIPs":[],"enabled":true},"trustedProxy":{"headers":[],"useLeftmostIP":true},"batch":{"enabled":false,"maxRequests":0,"timeout":15,"maxBodySize":0},"logs":{"maxDays":123,"minLevel":0,"logIP":false,"logAuthId":false}}`
|
||||
expected := `{"superuserIPs":[],"smtp":{"enabled":false,"port":0,"host":"smtp_host","username":"smtp_username","password":"","authMethod":"","tls":false,"localName":""},"backups":{"cron":"* * * * *","cronMaxKeep":0,"s3":{"enabled":true,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false}},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"s3_endpoint","accessKey":"","secret":"s3_secret","forcePathStyle":false},"meta":{"accentColor":"","appName":"test_app_name","appURL":"","senderName":"","senderAddress":"","hideControls":false},"rateLimits":{"rules":[],"excludedIPs":[],"enabled":true},"trustedProxy":{"headers":[],"useLeftmostIP":true},"batch":{"enabled":false,"maxRequests":0,"timeout":15,"maxBodySize":0},"logs":{"maxDataSize":0,"maxDays":123,"minLevel":0,"logIP":false,"logAuthId":false}}`
|
||||
if valueStr != expected {
|
||||
t.Fatalf("Expected exported settings\n%s\ngot\n%s", expected, valueStr)
|
||||
}
|
||||
@@ -112,12 +112,12 @@ func TestSettingsMerge(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
s1Encoded, err := json.Marshal(s1)
|
||||
s1Encoded, err := json.Marshal(s1, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
s2Encoded, err := json.Marshal(s2)
|
||||
s2Encoded, err := json.Marshal(s2, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -138,12 +138,12 @@ func TestSettingsClone(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
s1Bytes, err := json.Marshal(s1)
|
||||
s1Bytes, err := json.Marshal(s1, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
s2Bytes, err := json.Marshal(s2)
|
||||
s2Bytes, err := json.Marshal(s2, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -174,13 +174,13 @@ func TestSettingsMarshalJSON(t *testing.T) {
|
||||
settings.S3.Secret = testSecret
|
||||
settings.Backups.S3.Secret = testSecret
|
||||
|
||||
raw, err := json.Marshal(settings)
|
||||
raw, err := json.Marshal(settings, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rawStr := string(raw)
|
||||
|
||||
expected := `{"superuserIPs":[],"smtp":{"enabled":false,"port":0,"host":"","username":"abc","authMethod":"","tls":false,"localName":""},"backups":{"cron":"","cronMaxKeep":0,"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false}},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false},"meta":{"accentColor":"","appName":"test123","appURL":"","senderName":"","senderAddress":"","hideControls":false},"rateLimits":{"rules":[],"excludedIPs":[],"enabled":false},"trustedProxy":{"headers":[],"useLeftmostIP":false},"batch":{"enabled":false,"maxRequests":0,"timeout":0,"maxBodySize":0},"logs":{"maxDays":0,"minLevel":0,"logIP":false,"logAuthId":false}}`
|
||||
expected := `{"superuserIPs":[],"smtp":{"enabled":false,"port":0,"host":"","username":"abc","authMethod":"","tls":false,"localName":""},"backups":{"cron":"","cronMaxKeep":0,"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false}},"s3":{"enabled":false,"bucket":"","region":"","endpoint":"","accessKey":"","forcePathStyle":false},"meta":{"accentColor":"","appName":"test123","appURL":"","senderName":"","senderAddress":"","hideControls":false},"rateLimits":{"rules":[],"excludedIPs":[],"enabled":false},"trustedProxy":{"headers":[],"useLeftmostIP":false},"batch":{"enabled":false,"maxRequests":0,"timeout":0,"maxBodySize":0},"logs":{"maxDataSize":0,"maxDays":0,"minLevel":0,"logIP":false,"logAuthId":false}}`
|
||||
|
||||
if rawStr != expected {
|
||||
t.Fatalf("Expected\n%v\ngot\n%v", expected, rawStr)
|
||||
@@ -228,7 +228,7 @@ func TestSettingsValidate(t *testing.T) {
|
||||
`"rateLimits":{`,
|
||||
}
|
||||
|
||||
errBytes, _ := json.Marshal(err)
|
||||
errBytes, _ := json.Marshal(err, json.Deterministic(true))
|
||||
jsonErr := string(errBytes)
|
||||
for _, expected := range expectations {
|
||||
if !strings.Contains(jsonErr, expected) {
|
||||
@@ -309,8 +309,11 @@ func TestLogsConfigValidate(t *testing.T) {
|
||||
},
|
||||
{
|
||||
"invalid data",
|
||||
core.LogsConfig{MaxDays: -1},
|
||||
[]string{"maxDays"},
|
||||
core.LogsConfig{
|
||||
MaxDays: -1,
|
||||
MaxDataSize: -1,
|
||||
},
|
||||
[]string{"maxDays", "maxDataSize"},
|
||||
},
|
||||
{
|
||||
"valid data",
|
||||
|
||||
@@ -2,7 +2,7 @@ package core
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
@@ -19,7 +19,7 @@ func TestSendSystemAlert(t *testing.T) {
|
||||
testApp := NewBaseApp(BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer testApp.ResetBootstrapState()
|
||||
defer testApp.ClearBootstrap()
|
||||
|
||||
if err := testApp.Bootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -72,7 +72,7 @@ func TestSendSystemAlertToAllSuperusers(t *testing.T) {
|
||||
testApp := NewBaseApp(BaseAppConfig{
|
||||
DataDir: testDataDir,
|
||||
})
|
||||
defer testApp.ResetBootstrapState()
|
||||
defer testApp.ClearBootstrap()
|
||||
|
||||
if err := testApp.Bootstrap(); err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
+3
-3
@@ -1,7 +1,7 @@
|
||||
package core_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
"slices"
|
||||
"testing"
|
||||
@@ -530,7 +530,7 @@ func TestCreateViewFields(t *testing.T) {
|
||||
}
|
||||
|
||||
if len(s.expectFields) != len(result) {
|
||||
serialized, _ := json.Marshal(result)
|
||||
serialized, _ := json.Marshal(result, json.Deterministic(true))
|
||||
t.Fatalf("Expected %d fields, got %d: \n%s", len(s.expectFields), len(result), serialized)
|
||||
}
|
||||
|
||||
@@ -835,7 +835,7 @@ func TestDryRunView(t *testing.T) {
|
||||
// check fields
|
||||
// ---
|
||||
if len(s.expectFields) != len(result.Fields) {
|
||||
serialized, _ := json.Marshal(result.Fields)
|
||||
serialized, _ := json.Marshal(result.Fields, json.Deterministic(true))
|
||||
t.Fatalf("Expected %d fields, got %d: \n%s", len(s.expectFields), len(result.Fields), serialized)
|
||||
}
|
||||
for name, typ := range s.expectFields {
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/x509"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"encoding/pem"
|
||||
"testing"
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ package forms_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"maps"
|
||||
"os"
|
||||
|
||||
@@ -1,49 +1,50 @@
|
||||
module github.com/pocketbase/pocketbase
|
||||
|
||||
go 1.25.0
|
||||
go 1.27
|
||||
|
||||
require (
|
||||
github.com/disintegration/imaging v1.6.2
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2
|
||||
github.com/dop251/goja v0.0.0-20260701091749-b07b74453ea9
|
||||
github.com/dop251/goja v0.0.0-20260901132549-43234fa61381
|
||||
github.com/dop251/goja_nodejs v0.0.0-20260212111938-1f56ff5bcf14
|
||||
github.com/fatih/color v1.19.0
|
||||
github.com/fsnotify/fsnotify v1.10.1
|
||||
github.com/gabriel-vasile/mimetype v1.4.13
|
||||
github.com/ganigeorgiev/fexpr v0.5.0
|
||||
github.com/gabriel-vasile/mimetype v1.4.15
|
||||
github.com/ganigeorgiev/fexpr v0.6.0
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
github.com/pocketbase/dbx v1.12.0
|
||||
github.com/pocketbase/ozzo-validation/v4 v4.3.0
|
||||
github.com/pocketbase/tygoja v0.1.0
|
||||
github.com/spf13/cast v1.10.0
|
||||
github.com/spf13/cobra v1.10.2
|
||||
golang.org/x/crypto v0.54.0
|
||||
golang.org/x/image v0.44.0
|
||||
golang.org/x/net v0.57.0
|
||||
golang.org/x/crypto v0.57.0
|
||||
golang.org/x/image v0.46.0
|
||||
golang.org/x/net v0.59.0
|
||||
golang.org/x/oauth2 v0.36.0
|
||||
golang.org/x/sync v0.22.0
|
||||
modernc.org/sqlite v1.54.0
|
||||
golang.org/x/sync v0.23.0
|
||||
modernc.org/sqlite v1.57.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
|
||||
github.com/dlclark/regexp2/v2 v2.2.2 // indirect
|
||||
github.com/dlclark/regexp2/v2 v2.7.1 // indirect
|
||||
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
|
||||
github.com/google/pprof v0.0.0-20260604005048-7023385849c0 // indirect
|
||||
github.com/google/pprof v0.0.0-20260902005441-ca85771921e4 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.15 // indirect
|
||||
github.com/mattn/go-isatty v0.0.23 // indirect
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/spf13/pflag v1.0.10 // indirect
|
||||
golang.org/x/mod v0.37.0 // indirect
|
||||
golang.org/x/sys v0.47.0 // indirect
|
||||
golang.org/x/text v0.40.0 // indirect
|
||||
golang.org/x/tools v0.47.0 // indirect
|
||||
modernc.org/libc v1.74.1 // indirect
|
||||
github.com/stretchr/testify v1.8.0 // indirect
|
||||
golang.org/x/mod v0.41.0 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/text v0.42.0 // indirect
|
||||
golang.org/x/tools v0.49.0 // indirect
|
||||
modernc.org/libc v1.74.4 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
modernc.org/memory v1.12.0 // indirect
|
||||
)
|
||||
|
||||
@@ -4,18 +4,19 @@ github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:o
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
|
||||
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
|
||||
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
|
||||
github.com/dlclark/regexp2/v2 v2.2.2 h1:MYWvNYw8okuqNhwTYO587EZMiDruVa2vhV6fsGpfya0=
|
||||
github.com/dlclark/regexp2/v2 v2.2.2/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
|
||||
github.com/dlclark/regexp2/v2 v2.7.1 h1:yqDtwI1ptXXvEUNpYTk2lad4jLtAcKqkzepn4savSk4=
|
||||
github.com/dlclark/regexp2/v2 v2.7.1/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2 h1:x3tGMsyFhTCaxp6ycgR0FE/bu5QiNp+hetUuCOBXMn8=
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2/go.mod h1:lOm/u9CyCVWHeaAmHIdF4RiKVxKUT/H5XX10lIKAL6c=
|
||||
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 h1:16iT9CBDOniJwFGPI41MbUDfEk74hFaKTqudrX8kenY=
|
||||
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217/go.mod h1:eIb+f24U+eWQCIsj9D/ah+MD9UP+wdxuqzsdLD+mhGM=
|
||||
github.com/dop251/goja v0.0.0-20260701091749-b07b74453ea9 h1:q33zakIx+wEp1Ko5NpDyDBICuXL4JeHUaHbhPowcMEk=
|
||||
github.com/dop251/goja v0.0.0-20260701091749-b07b74453ea9/go.mod h1:Sc+QOu1WruvaaeT/cxFez/pXHpI9ZDjg/E8QNfSVveI=
|
||||
github.com/dop251/goja v0.0.0-20260901132549-43234fa61381 h1:vzmJ+R9qxuaksCAB+Kn6wtGgQ01cIfKzUGwWWcX5zXI=
|
||||
github.com/dop251/goja v0.0.0-20260901132549-43234fa61381/go.mod h1:u8yZRUavu+N4EnFFy6J5fVtjE7lEcZ2YyV2GcBXY9c8=
|
||||
github.com/dop251/goja_nodejs v0.0.0-20260212111938-1f56ff5bcf14 h1:3U8dTgyNBhEQ/GVw0jZW5q+93Zw2gAZPRWhJ9TwV3rM=
|
||||
github.com/dop251/goja_nodejs v0.0.0-20260212111938-1f56ff5bcf14/go.mod h1:Tb7Xxye4LX7cT3i8YLvmPMGCV92IOi4CDZvm/V8ylc0=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
@@ -26,10 +27,10 @@ github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHk
|
||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||
github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
|
||||
github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
|
||||
github.com/ganigeorgiev/fexpr v0.5.0 h1:XA9JxtTE/Xm+g/JFI6RfZEHSiQlk+1glLvRK1Lpv/Tk=
|
||||
github.com/ganigeorgiev/fexpr v0.5.0/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
|
||||
github.com/ganigeorgiev/fexpr v0.6.0 h1:Fza3O/QMBKEudUvxV862qe6GjxM60GJjjKytdp+VQus=
|
||||
github.com/ganigeorgiev/fexpr v0.6.0/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
|
||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible h1:a+iTbH5auLKxaNwQFg0B+TCYl6lbukKPc7b5x0n1s6Q=
|
||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
|
||||
github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
|
||||
@@ -41,8 +42,8 @@ github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArs
|
||||
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/pprof v0.0.0-20260604005048-7023385849c0 h1:h1QTMDl6q9wDvDCJVpKQSjgleGFYnd2fOxmg2K+6BGE=
|
||||
github.com/google/pprof v0.0.0-20260604005048-7023385849c0/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
|
||||
github.com/google/pprof v0.0.0-20260902005441-ca85771921e4 h1:/6mPXfWmhv8eKck12I0YNIcIjwHtxP3YRIMKiEgTjWg=
|
||||
github.com/google/pprof v0.0.0-20260902005441-ca85771921e4/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||
@@ -55,8 +56,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
|
||||
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.23 h1:cYwCQTQf3HB6xUC+BtyCLZNr7IzbOmoZbmssVNzSyiQ=
|
||||
github.com/mattn/go-isatty v0.0.23/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
|
||||
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
@@ -80,41 +81,46 @@ github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An
|
||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
|
||||
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
|
||||
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
||||
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
||||
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
|
||||
golang.org/x/image v0.44.0 h1:+tDekMZED9+LrtB3G5xzRggpVh9CARjZqROla3R3R+I=
|
||||
golang.org/x/image v0.44.0/go.mod h1:V8K3KE9KKKE+pLpQDOeN18w9oacNSvy1tDOirTu4xtY=
|
||||
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
|
||||
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
|
||||
golang.org/x/image v0.46.0 h1:b1+oYj0Jbp6K5MDT4i4/eZpYlk3V8SJhhDKh6LBHAyQ=
|
||||
golang.org/x/image v0.46.0/go.mod h1:3B3W05VGVQyuXucLINLjXKrqISASfi4Xj+iCVkLMwew=
|
||||
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
|
||||
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
|
||||
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
|
||||
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
|
||||
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
|
||||
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
|
||||
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
|
||||
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
|
||||
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
|
||||
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
|
||||
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
|
||||
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
||||
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
|
||||
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
|
||||
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
|
||||
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
|
||||
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
|
||||
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
|
||||
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
|
||||
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
|
||||
google.golang.org/appengine v1.6.5 h1:tycE03LOZYQNhDpS27tcQdAzLCVMaj7QT2SXxebnpCM=
|
||||
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
modernc.org/cc/v4 v4.29.0 h1:CXgwL8cvxmyzBQZzbSl/6xFtMCryb6u8IOqDci39cgc=
|
||||
modernc.org/cc/v4 v4.29.0/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
modernc.org/cc/v4 v4.29.1 h1:MKgdCV3WykTSPqpVrnxdEDS0HEd2FHpKZDzxzU5LyeI=
|
||||
modernc.org/cc/v4 v4.29.1/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
|
||||
modernc.org/ccgo/v4 v4.34.6 h1:sBgfIwyN0TQ9C5hwIeuqyeAKyMWnbvj2fvpF4L11uzU=
|
||||
modernc.org/ccgo/v4 v4.34.6/go.mod h1:SZ8YcN9NG7XVsQYdm6jYBvi8PQP1qi+kqB6OhjqI3Fk=
|
||||
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
|
||||
@@ -125,18 +131,18 @@ modernc.org/gc/v3 v3.1.4 h1:2g65LGVSmFQrXeITAw97x7hCRvZFcyE1uDP+7Vng7JI=
|
||||
modernc.org/gc/v3 v3.1.4/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
|
||||
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
|
||||
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
|
||||
modernc.org/libc v1.74.1 h1:bdR4VTKFMC4966QSNZ05XLGI/VwzVa2kTUX51Dm0riQ=
|
||||
modernc.org/libc v1.74.1/go.mod h1:uH4t5bOx3G3g9Xcmj10YKlTcVISlRDwv8VoQJG9n8Os=
|
||||
modernc.org/libc v1.74.4 h1:fX1Omw4o2/1C2iRkkIsrQTasJQldLhRmuPreXLoWs9k=
|
||||
modernc.org/libc v1.74.4/go.mod h1:eeQAS9W3sZeKYMFubydxJpII9ybHWshk+7or7bLG9co=
|
||||
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
|
||||
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
|
||||
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
|
||||
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||
modernc.org/memory v1.12.0 h1:twkmYNkGXCvtYWzoux02jtK6eovjZbdI0uHFUYp6kuU=
|
||||
modernc.org/memory v1.12.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
|
||||
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
|
||||
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
|
||||
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
|
||||
modernc.org/sqlite v1.54.0 h1:JCxR4qwkJvOaqAoYcgDoO25Nc+ROg6EJ2LfBVzdrgog=
|
||||
modernc.org/sqlite v1.54.0/go.mod h1:4ntCLuNmnH8+GNqjka1wNg7KJd5/Hi5FYp8K+XQ7GZw=
|
||||
modernc.org/sqlite v1.57.0 h1:qNQP6xnx5M0ISNtlnxoOX0+cD5bJ0/gr9aMmndFczzg=
|
||||
modernc.org/sqlite v1.57.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
|
||||
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
|
||||
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
|
||||
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
|
||||
|
||||
+6
-1
@@ -7,7 +7,7 @@ linters:
|
||||
enable:
|
||||
- asasalint
|
||||
- asciicheck
|
||||
- gomodguard
|
||||
- gomodguard_v2
|
||||
- goprintffuncname
|
||||
- govet
|
||||
- ineffassign
|
||||
@@ -20,6 +20,11 @@ linters:
|
||||
- unconvert
|
||||
- unused
|
||||
- whitespace
|
||||
exclusions:
|
||||
rules:
|
||||
- path: _test\.go
|
||||
linters:
|
||||
- prealloc
|
||||
formatters:
|
||||
enable:
|
||||
- gofmt
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package migrations
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
@@ -10,8 +10,8 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
expectedDriverVersion = "v1.54.0"
|
||||
expectedLibcVersion = "v1.74.1"
|
||||
expectedDriverVersion = "v1.57.0"
|
||||
expectedLibcVersion = "v1.74.4"
|
||||
|
||||
// ModerncDepsCheckHookId is the id of the hook that performs the modernc.org/* deps checks.
|
||||
// It could be used for removing/unbinding the hook if you don't want the checks.
|
||||
|
||||
@@ -8,7 +8,7 @@ package ghupdate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
@@ -3,7 +3,7 @@ package jsvm
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"io"
|
||||
"io/fs"
|
||||
@@ -352,7 +352,7 @@ func BindCore(vm *goja.Runtime) {
|
||||
}
|
||||
|
||||
// as a last attempt try to json encode the value
|
||||
rawBytes, _ := json.Marshal(raw)
|
||||
rawBytes, _ := json.Marshal(raw, json.Deterministic(true))
|
||||
|
||||
return rawBytes, nil
|
||||
}
|
||||
@@ -381,7 +381,7 @@ func BindCore(vm *goja.Runtime) {
|
||||
}
|
||||
|
||||
// as a last attempt try to json encode the value
|
||||
rawBytes, _ := json.Marshal(raw)
|
||||
rawBytes, _ := json.Marshal(raw, json.Deterministic(true))
|
||||
|
||||
return string(rawBytes), nil
|
||||
}
|
||||
@@ -1217,13 +1217,13 @@ func newDynamicModel(shape map[string]any) any {
|
||||
case reflect.Map:
|
||||
raw, _ := json.Marshal(v)
|
||||
newV := types.JSONMap[any]{}
|
||||
newV.Scan(raw)
|
||||
_ = newV.Scan(raw)
|
||||
v = newV
|
||||
vt = reflect.TypeOf(v)
|
||||
case reflect.Slice, reflect.Array:
|
||||
raw, _ := json.Marshal(v)
|
||||
newV := types.JSONArray[any]{}
|
||||
newV.Scan(raw)
|
||||
_ = newV.Scan(raw)
|
||||
v = newV
|
||||
vt = reflect.TypeOf(newV)
|
||||
case reflect.Pointer:
|
||||
|
||||
@@ -2,7 +2,7 @@ package jsvm
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -505,12 +505,12 @@ func TestBindCoreMailerMessage(t *testing.T) {
|
||||
t.Fatalf("Expected mailer.Message, got %v", m)
|
||||
}
|
||||
|
||||
raw, err := json.Marshal(m)
|
||||
raw, err := json.Marshal(m, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
expected := `{"from":{"Name":"test_from","Address":"test_from@example.com"},"to":[{"Name":"test_to1","Address":"test_to1@example.com"},{"Name":"test_to2","Address":"test_to2@example.com"}],"bcc":[{"Name":"test_bcc1","Address":"test_bcc1@example.com"},{"Name":"test_bcc2","Address":"test_bcc2@example.com"}],"cc":[{"Name":"test_cc1","Address":"test_cc1@example.com"},{"Name":"test_cc2","Address":"test_cc2@example.com"}],"subject":"test_subject","html":"test_html","text":"test_text","headers":{"header1":"a","header2":"b"},"attachments":null,"inlineAttachments":null}`
|
||||
expected := `{"from":{"Name":"test_from","Address":"test_from@example.com"},"to":[{"Name":"test_to1","Address":"test_to1@example.com"},{"Name":"test_to2","Address":"test_to2@example.com"}],"bcc":[{"Name":"test_bcc1","Address":"test_bcc1@example.com"},{"Name":"test_bcc2","Address":"test_bcc2@example.com"}],"cc":[{"Name":"test_cc1","Address":"test_cc1@example.com"},{"Name":"test_cc2","Address":"test_cc2@example.com"}],"subject":"test_subject","html":"test_html","text":"test_text","headers":{"header1":"a","header2":"b"},"attachments":{},"inlineAttachments":{}}`
|
||||
|
||||
if string(raw) != expected {
|
||||
t.Fatalf("Expected \n%s, \ngot \n%s", expected, raw)
|
||||
@@ -1178,7 +1178,7 @@ func TestBindApisErrors(t *testing.T) {
|
||||
t.Errorf("[%s] Expected Message %q, got %q", s.js, s.expectMessage, apiErr.Message)
|
||||
}
|
||||
|
||||
dataRaw, _ := json.Marshal(apiErr.RawData())
|
||||
dataRaw, _ := json.Marshal(apiErr.RawData(), json.Deterministic(true))
|
||||
if string(dataRaw) != s.expectData {
|
||||
t.Errorf("[%s] Expected Data %q, got %q", s.js, s.expectData, dataRaw)
|
||||
}
|
||||
@@ -1439,7 +1439,7 @@ func TestBindHTTPSend(t *testing.T) {
|
||||
res.Header().Add("X-Custom", "custom_header")
|
||||
res.Header().Add("Set-Cookie", "sessionId=123456")
|
||||
|
||||
infoRaw, _ := json.Marshal(info)
|
||||
infoRaw, _ := json.Marshal(info, json.Deterministic(true))
|
||||
|
||||
// write back the submitted request
|
||||
res.Write(infoRaw)
|
||||
@@ -1613,7 +1613,7 @@ func TestHooksBindsCount(t *testing.T) {
|
||||
vm := goja.New()
|
||||
hooksBinds(app, vm, nil)
|
||||
|
||||
testBindsCount(vm, "this", 82, t)
|
||||
testBindsCount(vm, "this", 83, t)
|
||||
}
|
||||
|
||||
func TestHooksBinds(t *testing.T) {
|
||||
|
||||
@@ -2,7 +2,7 @@ package jsvm
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -166,7 +166,7 @@ func TestFormDataEntries(t *testing.T) {
|
||||
|
||||
entries := data.Entries()
|
||||
|
||||
rawEntries, err := json.Marshal(entries)
|
||||
rawEntries, err := json.Marshal(entries, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
+5483
-5079
File diff suppressed because it is too large
Load Diff
@@ -129,7 +129,7 @@ type excludeHooks<Type> = {
|
||||
type CoreApp = excludeHooks<ORIGINAL_CORE_APP>
|
||||
|
||||
// pocketbase.PocketBase without the on* hook methods
|
||||
type PocketBase = excludeHooks<ORIGINAL_POCKETBASE>
|
||||
interface PocketBase extends excludeHooks<ORIGINAL_POCKETBASE>{}
|
||||
|
||||
/**
|
||||
* ` + "`$app`" + ` is the current running PocketBase instance that is globally
|
||||
|
||||
@@ -194,7 +194,8 @@ func (p *plugin) migrateCreateHandler(template string, args []string, interactiv
|
||||
}
|
||||
|
||||
func (p *plugin) migrateCollectionsHandler(args []string, interactive bool) (string, error) {
|
||||
createArgs := []string{"collections_snapshot"}
|
||||
createArgs := make([]string, 0, len(args)+1)
|
||||
createArgs = append(createArgs, "collections_snapshot")
|
||||
createArgs = append(createArgs, args...)
|
||||
|
||||
collections := []*core.Collection{}
|
||||
|
||||
@@ -93,12 +93,12 @@ migrate((app) => {
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"exceptDomains": null,
|
||||
"exceptDomains": [],
|
||||
"help": "",
|
||||
"hidden": false,
|
||||
"id": "email@TEST_RANDOM",
|
||||
"name": "email",
|
||||
"onlyDomains": null,
|
||||
"onlyDomains": [],
|
||||
"presentable": false,
|
||||
"required": true,
|
||||
"system": true,
|
||||
@@ -200,7 +200,7 @@ migrate((app) => {
|
||||
package _test_migrations
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
m "github.com/pocketbase/pocketbase/migrations"
|
||||
@@ -275,12 +275,12 @@ func init() {
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"exceptDomains": null,
|
||||
"exceptDomains": [],
|
||||
"help": "",
|
||||
"hidden": false,
|
||||
"id": "email@TEST_RANDOM",
|
||||
"name": "email",
|
||||
"onlyDomains": null,
|
||||
"onlyDomains": [],
|
||||
"presentable": false,
|
||||
"required": true,
|
||||
"system": true,
|
||||
@@ -546,12 +546,12 @@ migrate((app) => {
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"exceptDomains": null,
|
||||
"exceptDomains": [],
|
||||
"help": "",
|
||||
"hidden": false,
|
||||
"id": "email3885137012",
|
||||
"name": "email",
|
||||
"onlyDomains": null,
|
||||
"onlyDomains": [],
|
||||
"presentable": false,
|
||||
"required": true,
|
||||
"system": true,
|
||||
@@ -649,7 +649,7 @@ migrate((app) => {
|
||||
package _test_migrations
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
m "github.com/pocketbase/pocketbase/migrations"
|
||||
@@ -731,12 +731,12 @@ func init() {
|
||||
"type": "text"
|
||||
},
|
||||
{
|
||||
"exceptDomains": null,
|
||||
"exceptDomains": [],
|
||||
"help": "",
|
||||
"hidden": false,
|
||||
"id": "email3885137012",
|
||||
"name": "email",
|
||||
"onlyDomains": null,
|
||||
"onlyDomains": [],
|
||||
"presentable": false,
|
||||
"required": true,
|
||||
"system": true,
|
||||
@@ -1041,7 +1041,7 @@ migrate((app) => {
|
||||
package _test_migrations
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
m "github.com/pocketbase/pocketbase/migrations"
|
||||
|
||||
@@ -2,12 +2,12 @@ package migratecmd
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/json/jsontext"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
@@ -383,7 +383,7 @@ func (p *plugin) goCreateTemplate(collection *core.Collection) (string, error) {
|
||||
const template = `package %s
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
m "github.com/pocketbase/pocketbase/migrations"
|
||||
@@ -436,7 +436,7 @@ func (p *plugin) goDeleteTemplate(collection *core.Collection) (string, error) {
|
||||
const template = `package %s
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
m "github.com/pocketbase/pocketbase/migrations"
|
||||
@@ -621,7 +621,7 @@ func (p *plugin) goDiffTemplate(new *core.Collection, old *core.Collection) (str
|
||||
|
||||
if strings.Contains(combined, "json.Unmarshal(") ||
|
||||
strings.Contains(combined, "json.Marshal(") {
|
||||
imports += "\n\t\"encoding/json\"\n"
|
||||
imports += "\n\t\"encoding/json/v2\"\n"
|
||||
}
|
||||
|
||||
imports += "\n\t\"github.com/pocketbase/pocketbase/core\""
|
||||
@@ -666,18 +666,11 @@ func init() {
|
||||
}
|
||||
|
||||
func marhshalWithoutEscape(v any, prefix string, indent string) ([]byte, error) {
|
||||
raw, err := json.MarshalIndent(v, prefix, indent)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// unescape escaped unicode characters
|
||||
unescaped, err := strconv.Unquote(strings.ReplaceAll(strconv.Quote(string(raw)), `\\u`, `\u`))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return []byte(unescaped), nil
|
||||
return json.Marshal(v,
|
||||
jsontext.WithIndentPrefix(prefix),
|
||||
jsontext.WithIndent(indent),
|
||||
json.Deterministic(true),
|
||||
)
|
||||
}
|
||||
|
||||
func escapeBacktick(v string) string {
|
||||
@@ -720,8 +713,8 @@ func diffMaps(old, new map[string]any, excludeKeys ...string) map[string]any {
|
||||
}
|
||||
|
||||
// compare the serialized version of the values in case of slice or other custom type
|
||||
rawOld, _ := json.Marshal(vOld)
|
||||
rawNew, _ := json.Marshal(vNew)
|
||||
rawOld, _ := json.Marshal(vOld, json.Deterministic(true))
|
||||
rawNew, _ := json.Marshal(vNew, json.Deterministic(true))
|
||||
|
||||
if !bytes.Equal(rawOld, rawNew) {
|
||||
// if both are maps add recursively only the changed fields
|
||||
|
||||
+22
-36
@@ -1,6 +1,7 @@
|
||||
package pocketbase
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
@@ -9,7 +10,6 @@ import (
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/fatih/color"
|
||||
"github.com/pocketbase/pocketbase/cmd"
|
||||
"github.com/pocketbase/pocketbase/core"
|
||||
"github.com/pocketbase/pocketbase/tools/hook"
|
||||
@@ -117,8 +117,9 @@ func NewWithConfig(config Config) *PocketBase {
|
||||
hideStartBanner: config.HideStartBanner,
|
||||
}
|
||||
|
||||
// replace with a colored stderr writer
|
||||
pb.RootCmd.SetErr(newErrWriter())
|
||||
// don't write command errors to the stderr because the error is
|
||||
// propagated back to the app.Start() and could result in duplication
|
||||
pb.RootCmd.SetErr(&nopWrite{})
|
||||
|
||||
// parse base flags
|
||||
// (errors are ignored, since the full flags parsing happens on Execute())
|
||||
@@ -183,26 +184,25 @@ func (pb *PocketBase) Execute() error {
|
||||
}
|
||||
}
|
||||
|
||||
done := make(chan bool, 1)
|
||||
execCh := make(chan error, 1)
|
||||
sigCh := make(chan os.Signal, 1)
|
||||
|
||||
// listen for interrupt signal to gracefully shutdown the application
|
||||
routine.FireAndForget(func() {
|
||||
sigch := make(chan os.Signal, 1)
|
||||
signal.Notify(sigch, os.Interrupt, syscall.SIGTERM)
|
||||
<-sigch
|
||||
|
||||
done <- true
|
||||
})
|
||||
signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM)
|
||||
|
||||
// execute the root command
|
||||
routine.FireAndForget(func() {
|
||||
// note: leave to the commands to decide whether to print their error
|
||||
pb.RootCmd.Execute()
|
||||
go func() {
|
||||
execCh <- routine.SafeWrap(pb.RootCmd.Execute)()
|
||||
}()
|
||||
|
||||
done <- true
|
||||
})
|
||||
// wait for either an OS signal or the command to complete
|
||||
var execErr error
|
||||
select {
|
||||
case <-sigCh:
|
||||
case execErr = <-execCh:
|
||||
}
|
||||
|
||||
<-done
|
||||
signal.Stop(sigCh)
|
||||
|
||||
// trigger cleanups
|
||||
//
|
||||
@@ -210,7 +210,7 @@ func (pb *PocketBase) Execute() error {
|
||||
event := new(core.TerminateEvent)
|
||||
event.App = pb
|
||||
return pb.OnTerminate().Trigger(event, func(e *core.TerminateEvent) error {
|
||||
return e.App.ResetBootstrapState()
|
||||
return errors.Join(e.App.ClearBootstrap(), execErr)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -308,24 +308,10 @@ func inspectRuntime() (baseDir string, withGoRun bool) {
|
||||
return
|
||||
}
|
||||
|
||||
// newErrWriter returns a red colored stderr writter.
|
||||
func newErrWriter() *coloredWriter {
|
||||
return &coloredWriter{
|
||||
w: os.Stderr,
|
||||
c: color.New(color.FgRed),
|
||||
}
|
||||
}
|
||||
var _ io.Writer = (*nopWrite)(nil)
|
||||
|
||||
// coloredWriter is a small wrapper struct to construct a [color.Color] writter.
|
||||
type coloredWriter struct {
|
||||
w io.Writer
|
||||
c *color.Color
|
||||
}
|
||||
type nopWrite struct{}
|
||||
|
||||
// Write writes the p bytes using the colored writer.
|
||||
func (colored *coloredWriter) Write(p []byte) (n int, err error) {
|
||||
colored.c.SetWriter(colored.w)
|
||||
defer colored.c.UnsetWriter(colored.w)
|
||||
|
||||
return colored.c.Print(string(p))
|
||||
func (w *nopWrite) Write(p []byte) (n int, err error) {
|
||||
return
|
||||
}
|
||||
|
||||
+7
-4
@@ -3,7 +3,7 @@ package tests
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/jsontext"
|
||||
"fmt"
|
||||
"io"
|
||||
"maps"
|
||||
@@ -249,6 +249,7 @@ func (scenario *ApiScenario) test(t testing.TB) {
|
||||
t.Errorf("Expected status code %d, got %d", scenario.ExpectedStatus, res.StatusCode)
|
||||
}
|
||||
|
||||
// @todo consider removing in favour of synctest.Wait()
|
||||
if scenario.Delay > 0 {
|
||||
time.Sleep(scenario.Delay)
|
||||
}
|
||||
@@ -259,14 +260,16 @@ func (scenario *ApiScenario) test(t testing.TB) {
|
||||
}
|
||||
} else {
|
||||
// normalize json response format
|
||||
buffer := new(bytes.Buffer)
|
||||
err := json.Compact(buffer, recorder.Body.Bytes())
|
||||
var normalizedBody string
|
||||
|
||||
buf := new(bytes.Buffer)
|
||||
enc := jsontext.NewEncoder(buf)
|
||||
err := enc.WriteValue(recorder.Body.Bytes())
|
||||
if err != nil {
|
||||
// not a json...
|
||||
normalizedBody = recorder.Body.String()
|
||||
} else {
|
||||
normalizedBody = buffer.String()
|
||||
normalizedBody = buf.String()
|
||||
}
|
||||
|
||||
for _, item := range scenario.ExpectedContent {
|
||||
|
||||
+24
-1
@@ -37,9 +37,9 @@ func (t *TestApp) Cleanup() {
|
||||
event.App = t
|
||||
|
||||
t.OnTerminate().Trigger(event, func(e *core.TerminateEvent) error {
|
||||
t.ClearBootstrap()
|
||||
t.TestMailer.Reset()
|
||||
t.ResetEventCalls()
|
||||
t.ResetBootstrapState()
|
||||
|
||||
return e.Next()
|
||||
})
|
||||
@@ -144,6 +144,14 @@ func NewTestAppWithConfig(config core.BaseAppConfig) (*TestApp, error) {
|
||||
Priority: -99999,
|
||||
})
|
||||
|
||||
t.OnBootstrapClear().Bind(&hook.Handler[*core.BootstrapEvent]{
|
||||
Func: func(e *core.BootstrapEvent) error {
|
||||
t.registerEventCall("OnBootstrapClear")
|
||||
return e.Next()
|
||||
},
|
||||
Priority: -99999,
|
||||
})
|
||||
|
||||
t.OnServe().Bind(&hook.Handler[*core.ServeEvent]{
|
||||
Func: func(e *core.ServeEvent) error {
|
||||
t.registerEventCall("OnServe")
|
||||
@@ -549,6 +557,21 @@ func NewTestAppWithConfig(config core.BaseAppConfig) (*TestApp, error) {
|
||||
Priority: -99999,
|
||||
})
|
||||
|
||||
// t.OnFilesystemDelete().Bind(&hook.Handler[*core.FilesystemDeleteEvent]{
|
||||
// Func: func(e *core.FilesystemDeleteEvent) error {
|
||||
// t.registerEventCall("OnFilesystemDelete")
|
||||
// return e.Next()
|
||||
// },
|
||||
// Priority: -99999,
|
||||
// })
|
||||
// t.OnFilesystemNewWriter().Bind(&hook.Handler[*core.FilesystemNewWriterEvent]{
|
||||
// Func: func(e *core.FilesystemNewWriterEvent) error {
|
||||
// t.registerEventCall("OnFilesystemNewWriter")
|
||||
// return e.Next()
|
||||
// },
|
||||
// Priority: -99999,
|
||||
// })
|
||||
|
||||
t.OnRealtimeConnectRequest().Bind(&hook.Handler[*core.RealtimeConnectRequestEvent]{
|
||||
Func: func(e *core.RealtimeConnectRequestEvent) error {
|
||||
t.registerEventCall("OnRealtimeConnectRequest")
|
||||
|
||||
@@ -48,7 +48,7 @@ func TestCreateSuccess(t *testing.T) {
|
||||
t.Fatalf("Expected zip with name %q, got %q", zipName, name)
|
||||
}
|
||||
|
||||
expectedSize := int64(544)
|
||||
expectedSize := int64(532)
|
||||
if size := info.Size(); size != expectedSize {
|
||||
t.Fatalf("Expected zip with size %d, got %d", expectedSize, size)
|
||||
}
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"net/http"
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ package auth
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/oauth2"
|
||||
@@ -215,12 +215,12 @@ func TestExtra(t *testing.T) {
|
||||
|
||||
after := b.Extra()
|
||||
|
||||
rawExtra, err := json.Marshal(extra)
|
||||
rawExtra, err := json.Marshal(extra, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rawAfter, err := json.Marshal(after)
|
||||
rawAfter, err := json.Marshal(after, json.Deterministic(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"io"
|
||||
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/types"
|
||||
|
||||
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"fmt"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/types"
|
||||
|
||||
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
|
||||
"github.com/pocketbase/pocketbase/tools/types"
|
||||
"golang.org/x/oauth2"
|
||||
@@ -53,12 +53,14 @@ func (p *Facebook) FetchAuthUser(token *oauth2.Token) (*AuthUser, error) {
|
||||
}
|
||||
|
||||
extracted := struct {
|
||||
Id string
|
||||
Name string
|
||||
Email string
|
||||
Id string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Email string `json:"email"`
|
||||
Picture struct {
|
||||
Data struct{ Url string }
|
||||
}
|
||||
Data struct {
|
||||
Url string `json:"url"`
|
||||
} `json:"data"`
|
||||
} `json:"picture"`
|
||||
}{}
|
||||
if err := json.Unmarshal(data, &extracted); err != nil {
|
||||
return nil, err
|
||||
|
||||
+4
-4
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -119,9 +119,9 @@ func (p *Gitea) fetchVerifiedPrimaryEmail(token *oauth2.Token) (string, error) {
|
||||
}
|
||||
|
||||
emails := []struct {
|
||||
Email string
|
||||
Verified bool
|
||||
Primary bool
|
||||
Email string `json:"email"`
|
||||
Verified bool `json:"verified"`
|
||||
Primary bool `json:"primary"`
|
||||
}{}
|
||||
if err := json.Unmarshal(content, &emails); err != nil {
|
||||
return "", err
|
||||
|
||||
+4
-4
@@ -2,7 +2,7 @@ package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/json/v2"
|
||||
"io"
|
||||
"strconv"
|
||||
|
||||
@@ -120,9 +120,9 @@ func (p *Gitee) fetchPrimaryEmail(token *oauth2.Token) (string, error) {
|
||||
}
|
||||
|
||||
emails := []struct {
|
||||
Email string
|
||||
State string
|
||||
Scope []string
|
||||
Email string `json:"email"`
|
||||
State string `json:"state"`
|
||||
Scope []string `json:"scope"`
|
||||
}{}
|
||||
if err := json.Unmarshal(content, &emails); err != nil {
|
||||
// ignore unmarshal error in case "Keep my email address private"
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user