mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-06 20:17:38 +01:00
s3tables: accept the external-volume verification object at a table bucket's root (#11900)
* s3tables: accept the external-volume verification object at a table bucket's root Snowflake verifies an S3-compatible external volume before it creates or writes Iceberg tables in it (SYSTEM$VERIFY_EXTERNAL_VOLUME, also run implicitly): it writes, reads, lists and deletes one small object, <base>/verify_<epoch milliseconds>_<number>, a 21-byte body. A table bucket refuses every key outside the Iceberg layout, so the check failed and Snowflake could not use a table bucket as an external volume. PUT, GET, HEAD, DELETE and DeleteObjects now accept exactly that key in a table bucket, narrowly enough that it cannot be used to store data: - at the bucket root only, so nothing is encoded in a prefix, written into a hidden tree or placed on a namespace or table name; - one shape: verify_<13 digits of epoch ms within one hour of the server's clock>_<1-10 digits>; - a PUT must declare a Content-Length of at most 1 KiB (no chunked or undeclared body) and carry no user metadata, tagging, Content-Disposition, Cache-Control, Expires or website redirect. Multipart, copy, rename, ACL, tagging and POST policy writes keep the plain layout rule. Buckets that are not table buckets are unaffected. Tests: table_bucket_verify_object_test.go, the key accepted (with the Iceberg layout as the positive control and the plain rule still refusing it), 15 look-alikes refused (prefixed, hidden tree, wrong shape, outside the window, ordinary keys), and seven PUT refusals through PutObjectHandler. * s3tables: keep the verification key off namespaces and narrow its PUT A namespace may be named like a verification key (verify_<ms>_<n>), and the key then names the namespace's directory. A PUT replaced that directory with a prefix object, dropping the namespace's catalog metadata; a DELETE removed an empty namespace. The key now gets the plain layout rule when it names a directory or its entry cannot be read, and a PUT of it never falls back to a prefix object, so a namespace created after the check is not replaced either. The time window applies to PUT only: GET, HEAD and DELETE take a key of any date, so an object a check left behind for more than an hour can still be read and removed. A verification PUT also refuses an extraction request and the stored Content-Language, Response-Content-Disposition and Content-Encoding headers (aws-chunked framing alone is still accepted). The integration test runs the check's PUT, HEAD, GET, list and DELETE against a table bucket, and the PUT and DELETE over a namespace of the key's shape. --------- Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
This commit is contained in:
1 parent
21add33b6e
commit
de2fc03c7d
6 files changed
+427
-6
No files matched your search
@@ -210,7 +210,7 @@ func (s3a *S3ApiServer) DeleteObjectHandler(w http.ResponseWriter, r *http.Reque
|
||||
// The filer ops below each retry, and a failover walk runs the whole set
|
||||
// once per filer, so the backoff comes out of one allowance held here.
|
||||
r = r.WithContext(withFilerRetryBudget(r.Context(), filerRetryRequestBudget))
|
||||
if err := s3a.validateTableBucketObjectPath(bucket, object); err != nil {
|
||||
if _, err := s3a.validateTableBucketObjectPathOrVerify(bucket, object); err != nil {
|
||||
s3err.WriteErrorResponse(w, r, s3err.ErrAccessDenied)
|
||||
return
|
||||
}
|
||||
@@ -479,7 +479,7 @@ func (s3a *S3ApiServer) DeleteMultipleObjectsHandler(w http.ResponseWriter, r *h
|
||||
deleteErrors = append(deleteErrors, deleteErrorFromCode(validationCode, object.Key, object.VersionId))
|
||||
continue
|
||||
}
|
||||
if err := s3a.validateTableBucketObjectPath(bucket, object.Key); err != nil {
|
||||
if _, err := s3a.validateTableBucketObjectPathOrVerify(bucket, object.Key); err != nil {
|
||||
deleteErrors = append(deleteErrors, deleteErrorFromCode(s3err.ErrAccessDenied, object.Key, object.VersionId))
|
||||
continue
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user