* postgres2: default the filemeta CREATE TABLE when createTable is unset
An empty createTable rendered through fmt.Sprintf produced
%!(EXTRA string=filemeta), which Postgres rejected with a syntax error at
init. Fall back to a working template so a minimal config bootstraps.
* mysql2: default the filemeta CREATE TABLE when createTable is unset
Same empty-template failure the postgres2 path had: an unset createTable
rendered to %!(EXTRA string=filemeta) and MySQL rejected it at init.
Fall back to a working template.
Concurrent bucket deletion across multiple filer replicas races on the
per-bucket DROP TABLE. The first replica drops the table; the rest hit
an undefined-table error (postgres 42P01, mysql 1051) which propagates
out of DeleteFolderChildren and panics the filer. On restart the same
pending DROP re-runs and the filer crash-loops.
Make the drop idempotent. Same defect class in all SQL backends, so fix
postgres, postgres2, mysql, mysql2, and sqlite together.
* mysql: keep S3 list order byte-lexicographic regardless of name column collation
ORDER BY name and the name > ? pagination predicate follow the column
collation, so a case-insensitive filemeta.name (e.g. utf8mb3_general_ci)
returns S3 keys out of byte order and breaks clients that merge two sorted
listings.
Detect the live name collation at startup; only when it isn't binary, wrap
the list comparison, prefix, and ORDER BY in BINARY name so order and
pagination stay consistent. Correctly configured utf8mb4_bin tables keep
their indexed range scan unchanged, and the operator gets a warning to
convert the column.
* postgres: keep S3 list order byte-lexicographic regardless of name column collation
ORDER BY name and the name > $n pagination predicate follow the column or
database collation, so a locale-aware filemeta.name (e.g. the en_US.UTF-8
database default) returns S3 keys out of byte order and breaks clients that
merge two sorted listings.
Detect the live name collation at startup; only when it isn't byte-ordered,
wrap the list comparison, prefix, and ORDER BY in COLLATE "C" so order and
pagination stay consistent. A byte-ordered (C/POSIX/C.UTF-8) column keeps its
indexed range scan unchanged, and the operator gets a warning to declare the
column COLLATE "C".
* fix(filer/postgres): default to ON CONFLICT upsert to keep tx alive
A KvPut from the inode-index secondary write could fail with 23505
(duplicate key) inside a rename's transaction, after which the next
statement returned 25P02 and rename surfaced to FUSE as EIO. Default
the postgres upsert query when enableUpsert=true so INSERTs are
idempotent; the enableUpsert=false escape hatch is preserved for
non-PG-compatible backends.
* fix(filer/mysql): default to ON DUPLICATE KEY UPDATE upsert
Same shape as the postgres default: when enableUpsert=true but no
upsertQuery is configured, install a sensible default so the
inode-index KvPut does not waste a duplicate-key roundtrip on every
entry write. Uses the VALUES() form so the default works on MariaDB
and MySQL >=5.7; the MySQL 8.0.19 row-alias form is left to explicit
config.
* fix(filer): default enableUpsert=true for sql stores
The default-template fallback only kicks in when enableUpsert=true,
so minimal configs that omit the flag entirely were still exposed.
Default it on for postgres/postgres2/mysql/mysql2; an explicit false
in filer.toml still wins because SetDefault only fills absent keys.
* fix(filer): don't disable the SQL idle connection pool when unconfigured
The mysql/mysql2/postgres stores called SetMaxIdleConns(maxIdle)
unconditionally, so an unset connection_max_idle (0) actively kept zero
idle connections - every query opened and closed a fresh connection
instead of reusing the pool.
Only apply the value when it's set; otherwise leave database/sql's
default idle pool of 2 in place.
* comments: shorten idle-pool note
* fix(filer): default the SQL idle pool via config, keep explicit 0 honored
Apply the idle-pool default at the config layer with SetDefault instead of
guarding the SetMaxIdleConns call. An absent connection_max_idle now reads
back as 2 (pool stays on), while an explicit 0 flows through to
SetMaxIdleConns(0) so operators can still disable idle pooling on purpose.
* refactor(filer/mysql): set tls.Config per-instance via Connector instead of global registry
Replace the use of `mysql.RegisterTLSConfig("mysql-tls", ...)` and the
`&tls=mysql-tls` DSN suffix with a per-instance setup that assigns the
`*tls.Config` directly to `mysql.Config.TLS` and opens the database via
`mysql.NewConnector` + `sql.OpenDB`.
The driver's TLS-config registry is process-wide; if a second `MysqlStore`
were ever initialized with different TLS settings (e.g., a filer plus a
separately configured store) the second registration would silently
overwrite the first. The connector pattern keeps the TLS configuration
attached to the connector and avoids that global side effect.
Behavior is otherwise unchanged: TLS is enabled when `enable_tls=true`,
the same `ca_crt`/`client_crt`/`client_key` knobs are honored, and the
TLS minimum version remains 1.2.
* filer(mysql): use system root CAs when ca_crt is empty
Previously, enabling `enable_tls=true` without setting `ca_crt` returned an
unhelpful empty-path read error. Many managed MySQL/MariaDB providers serve
certificates that chain to a public CA already in the host's trust store, so
requiring an explicit CA bundle adds friction with no security benefit.
Leave `RootCAs` unset when `ca_crt` is empty so Go's `tls.Config` falls back
to the system trust store, matching the standard behavior of `mysql --ssl`.
Existing setups with `ca_crt` configured are unaffected.
Also wraps the CA read/parse errors with the file path for easier diagnosis.
* filer(mysql): fail loudly when client_crt / client_key are unreadable
The previous implementation called `tls.LoadX509KeyPair` and silently
discarded any error, falling back to a non-mTLS connection. A typo or
permissions problem in `client_crt` / `client_key` therefore appeared as a
confusing server-side handshake error rather than as a config error,
because the server was expecting a client cert that the filer never sent.
Treat the keypair as required when either path is set, and surface the
underlying load error with both filenames so the misconfiguration is
obvious. The default (both paths empty) is unchanged: no client cert is
sent.
* filer(mysql): add tls_insecure_skip_verify and tls_server_name knobs
When the filer connects to a MySQL/MariaDB cluster whose server
certificate's SAN does not match the connection address (common with
internal load balancers, IP-only connection strings, or self-signed
cluster certs), the TLS handshake fails with `x509: certificate is valid
for X, not Y`. There was previously no way to fix this short of reissuing
the cert.
Expose two new optional knobs on `[mysql]`:
- `tls_server_name` overrides the SNI / cert hostname used for
verification — the standard fix when the cert SAN is correct but the
connection address is not.
- `tls_insecure_skip_verify` disables verification entirely as an escape
hatch for testing or for clusters with no usable SAN.
Both default to off, so existing configurations continue to verify the
server certificate against the connection address as before.
* docs(scaffold/filer.toml): document mysql TLS knobs and MariaDB upsert override
- Document the new `tls_insecure_skip_verify` and `tls_server_name` options.
- Update the `ca_crt` comment to reflect that it is optional and that the
system trust store is used when the path is empty (matches the runtime
behavior in mysql_store.go).
- Reword the client cert comments to make the mTLS pairing requirement
explicit (both `client_crt` and `client_key` must be set together).
- Add a commented-out MariaDB / MySQL 5.7 alternative for `upsertQuery`,
noting that the default (`AS new` row alias) requires MySQL 8.0.19+.
* filer(mysql): drop redundant blank import of go-sql-driver/mysql
The package was imported twice: once with the `mysql` alias (used for
`mysql.MySQLError`, `mysql.Config`, `mysql.NewConnector`, etc.) and once
as `_` to register the driver. The named import already triggers
`init()` and registers the driver, so the blank import is dead weight.
* Fix chown Input/output error on large file sets (Fixes#7911)
Implemented retry logic for MySQL/MariaDB backend to handle transient errors like deadlocks and timeouts.
* Fix syntax error: missing closing brace
* Refactor: Use %w for error wrapping and errors.As for extraction
* Fix: Disable retry logic inside transactions
Chris Lu