mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-23 22:40:39 +02:00
Jaehoon KimandChris Lu
744ab586c5
* shell: volume.fsck reads the filer from one store snapshot when it can volume.fsck collects the filer's chunk references by walking directory by directory, so each directory is seen at a different time. A file renamed from a directory not yet listed into one already listed is never seen, its chunks look like orphans, and -reallyDeleteFromVolume deletes live data. Checkpoint rotation and write-to-tmp-then-rename hit this. Add an optional filer store capability, EntryScanner, that visits every entry below a directory from one consistent snapshot. The SQL stores implement it with one REPEATABLE READ read-only transaction read in primary-key pages, so a concurrent rename is seen under exactly one of its names. Key-value rows are skipped by their dirhash, a bucket directory whose table is missing fails the scan (from the snapshot it cannot be told from one dropped a moment ago), and hard links are resolved in the same transaction. SQLite offers it only in WAL mode, on its own connection. TraverseBfsMetadata serves the snapshot when asked via request metadata sw-traverse-mode=snapshot and reports the mode in a response header; without the metadata it walks as before, so peer bootstrap and admin export are unchanged. Snapshot responses carry no entry_ts_ns, whose per-entry lookup would cost more than the scan. volume.fsck requests the snapshot and falls back to the walk on an older filer or a store that cannot scan. * filer, shell: fail a scan that cannot start, not fall back to the walk A transient BeginTx failure and a preliminary GetEntry error were treated like a missing capability or missing root, so fsck would take the directory walk - the very path that misses renamed-away entries. Only a genuinely unsupported scan or a confirmed-absent root may fall back. * filer: only snapshot-listed bucket names keep a missing table fatal A cached-only name, whose bucket may have been deleted between copying the cache and opening the snapshot, is skipped when its table is gone. * filer: roll back to a savepoint before skipping a cached-only table On PostgreSQL a failed statement aborts the snapshot transaction, so ignoring a missing-table error without the savepoint still left every later cached-only table unreadable. --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
1. create "seaweedfs" database export PGHOME=/Library/PostgreSQL/10 $PGHOME/bin/createdb --username=postgres --password seaweedfs 2. create "filemeta" table $PGHOME/bin/psql --username=postgres --password seaweedfs CREATE TABLE IF NOT EXISTS filemeta ( dirhash BIGINT, name VARCHAR(65535), directory VARCHAR(65535), meta bytea, PRIMARY KEY (dirhash, name) );