2 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandClaude Opus 5.5	533f5060b9	filer: scope a prefix JWT to the directory a zip extract writes under (#12305) ... * filer: scope a prefix JWT to the directory a zip extract writes under Without a trailing slash, an extract request writes members beside the named path, in its parent directory, while the JWT check only saw the request path. A token scoped to /tenant1 could therefore extract into / by posting to /tenant1?extract=zip. Add the extract root to the paths a prefix-restricted token must cover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * filer: do not keep an extracted archive under the name ".." A multipart file name of ".." passes path.Base unchanged, so extract.keep stored the archive as an entry literally named ".." in the target directory, a name that local sinks resolve to the parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * leveldb3: keep a non-local bucket name out of the per-bucket folder findDB splits the bucket out of a /buckets/ path and uses it as a folder name under the store directory. The name cannot hold a slash and dot names already go to the default DB, so this only rejects an empty or non-local name, but it also lets static analysis see the folder cannot escape the store directory. * filer: clean the zip extract root before scoping and writing A percent-encoded dot segment survives the path cleaning in front of the mux, so a request path ending in "/" reached extractZipRoot with ".." in it. The JWT scope check cleans the path it compares, but the members were written under the raw root, so the two could disagree. * leveldb3: apply the bucket folder check to bucket create and delete events findDB already keeps dot and non-local bucket names out of the per-bucket folders, but OnBucketCreation and OnBucketDeletion used the name as is. A directory event for /buckets/.. opened a DB in the parent of the store directory, and its delete removed that parent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * leveldb3: keep a bucket delete event off the default DB A bucket named _main maps to the default DB in findDB, so deleting /buckets/_main closed the default DB and removed its folder, taking every non-bucket entry with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * leveldb3: keep bucket names with a path separator out of the store dir A bucket event named x/../_main passed the local-name check, and its delete removed the default DB folder once store.dir/x existed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * leveldb3: only reject the platform separator in a bucket name A backslash is an ordinary file name character on Unix, and a bucket folder named that way already exists for anyone who created one. Rejecting it on every platform would route those entries to the default DB after an upgrade. The slash can never appear in a bucket name here, and on Windows the backslash is the separator, so rejecting os.PathSeparator alongside the slash keeps the folder flat without changing Unix routing. * leveldb3: check for an escaped bucket db before its delete event --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-01 18:36:31 -08:00
Chris Lu	a8affc851d	filer: extract zip members on ?extract=zip upload (#11782) ... * filer: add zip archive extraction helpers SpoolZipReader makes an upload body seekable for archive/zip, SanitizeZipMemberName rejects zip-slip and non-regular members, and CheckZipLimits caps member count and total uncompressed size. For seaweedfs/seaweedfs#11722 * filer: extract zip members on ?extract=zip upload POST/PUT to a directory path with ?extract=zip (or X-Seaweedfs-Extract: zip) writes each archive member as its own entry, so a bundle of small files lands in one request instead of one roundtrip each. Members keep their mtime, mode and mime-by-extension; directories are created; unsafe names are skipped and reported. extract.keep=true also stores the archive itself. For seaweedfs/seaweedfs#11722 * filer: bound zip spool memory by actual bytes A chunked upload has no Content-Length, so the old size hint let an unbounded body go through io.ReadAll. Read up to the memory threshold plus one byte first, then spill the prefix and the rest to a temp file, capped at ZipExtractMaxBytes total. * filer: guard zip member size total against overflow Declared member sizes summing past math.MaxUint64 wrapped the running total and slipped past the 1 TiB cap. * server: only take the inline fast path at a short first read uploadReaderToChunks read at most chunkSize bytes, then treated any read below SaveToFilerLimit as the whole file. When SaveToFilerLimit exceeded chunkSize, longer uploads were stored truncated. * server: resolve storage options per extracted zip member A member can land under a storage rule more specific than the request path. Re-running detectStorageOption0 per destination applies that rule's collection, replication, TTL, filename limit, and read-only guard to files and directories alike, while explicit query overrides still win. * server: accept raw request body for POST extract uploads Every POST was forced through MultipartReader, so a raw zip body always got a 400. Only parse multipart when the Content-Type is actually multipart. * server: extract zip directories before files and stop on cancel Processing directory members first keeps their stored mtime/mode from being overwritten by parents auto-created for an earlier file member. Checking ctx.Err() between members keeps a disconnected client from driving thousands more writes. * server: assert stored mtime and mode for extracted members * filer: bound compressed zip spool and reject non-local member names The 1 TiB uncompressed limit was also capping the compressed bytes spooled to the host's temp directory, so a handful of concurrent extractions could fill the filer's disk before any ZIP validation ran. SpoolZipReader now takes a separate compressed-size limit (the filer passes its TUS max upload size) with a 5 GiB default. SanitizeZipMemberName gains a filepath.IsLocal check: the segment loop already rejects "..", but CodeQL only treats member names as sanitized through a recognized guard, and the extra check also covers reserved Windows names. * server: honor destination rules for zip members and clamp maxMB An x-seaweedfs-destination header picks the storage rules for a logical destination while members still land under the request path, matching PostHandler. Directory members now also enforce the actual path's read-only rule instead of relying on option resolution to fail. maxMB up to 2048 overflowed the int32 chunk size, so an oversized query value made extraction (and ordinary autoChunk uploads) store members empty yet report success. The parsing is shared and clamped at 2047 MB. * filer: return 413 for oversized archives and honor saveInside in extract Map filer.ErrZipTooLarge to 413 instead of 500 for client-side archive size violations. Extracted members written under saveInside now store as inline entry content instead of unreachable volume chunks.	2026-10-16 21:04:54 -07:00