mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-07 04:27:49 +01:00
Chris LuandClaude Opus 5.5
7b3983f154
* s3api: look up a trailing-slash prefix's directory instead of listing its parent A prefix such as "account-0001/" names one directory, but the walk found it by listing the parent with the name as a prefix. SQL stores answer that with a LIKE that bounds the scan from below only, so in a wide bucket root the query reads every sibling after the name: seconds for a prefix holding a few keys. Look the directory up by name and serve it as the level's only entry. A miss, an error or a non-directory falls back to the prefix listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: track failover receives atomically The failover wrapper flags a received reply through a plain bool, which a callback reading from the filer on several goroutines would race on. Use an atomic.Bool so the callback may share its filer client. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: read directories ahead of a recursive listing walk A recursive ListObjects walk enters directories one at a time, so a bucket made of many directories holding one or two keys each costs one filer round trip per directory, in series: seconds per 1000-key page on a store whose reads wait on disk. Read the first window of the directories the walk will enter on a few workers, nearest first in walk order, and hand each window over when the walk gets there. Every window the walk reads itself is drained first so its subdirectories are queued before the walk descends into the first one. The number of windows held ahead grows while directories end inside their first window and falls back when one does not, so a page ending inside a large directory leaves few reads wasted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: drop the prefetcher's record of every directory it was handed A page can cross any number of empty directories without spending its key budget, so a set of every announced directory grows with the tree. A directory is already handled when it is at or behind the walk, held or in flight, or queued; check those, which the queue and window caps bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: end a trailing-slash prefix listing at the file its lookup finds A file named by the prefix lists nothing under it, the same as the sibling listing it fell back to, which threw the lookup's answer away and paid the parent scan anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: read a walk's own listing ahead instead of draining it Draining the window before the walk took its first entry made a page wait for the end of the listing, which a SQL store answering a name prefix reaches only after scanning the rest of the directory; a page that fills before then used to return at once. A background reader now hands the window's directories to the prefetcher as they arrive while the walk takes each entry, holding at most 1 MiB of entries ahead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: bound the bytes a prefetched listing window holds Entries with long chunk lists or inline content made the up to 64 held windows of 32 entries an unbounded amount of memory per listing. A window now stops at 128 KiB of entries and the walk lists the rest of that directory from the window's last entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * s3api: test recursive listings whose streams break partway Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>