s3tables: accept the external-volume verification object at a table bucket's root (#11900)

* s3tables: accept the external-volume verification object at a table bucket's root

Snowflake verifies an S3-compatible external volume before it creates or
writes Iceberg tables in it (SYSTEM$VERIFY_EXTERNAL_VOLUME, also run
implicitly): it writes, reads, lists and deletes one small object,
<base>/verify_<epoch milliseconds>_<number>, a 21-byte body. A table bucket
refuses every key outside the Iceberg layout, so the check failed and
Snowflake could not use a table bucket as an external volume.

PUT, GET, HEAD, DELETE and DeleteObjects now accept exactly that key in a
table bucket, narrowly enough that it cannot be used to store data:
- at the bucket root only, so nothing is encoded in a prefix, written into
  a hidden tree or placed on a namespace or table name;
- one shape: verify_<13 digits of epoch ms within one hour of the server's
  clock>_<1-10 digits>;
- a PUT must declare a Content-Length of at most 1 KiB (no chunked or
  undeclared body) and carry no user metadata, tagging, Content-Disposition,
  Cache-Control, Expires or website redirect.
Multipart, copy, rename, ACL, tagging and POST policy writes keep the plain
layout rule. Buckets that are not table buckets are unaffected.

Tests: table_bucket_verify_object_test.go, the key accepted (with the
Iceberg layout as the positive control and the plain rule still refusing
it), 15 look-alikes refused (prefixed, hidden tree, wrong shape, outside the
window, ordinary keys), and seven PUT refusals through PutObjectHandler.

* s3tables: keep the verification key off namespaces and narrow its PUT

A namespace may be named like a verification key (verify_<ms>_<n>), and
the key then names the namespace's directory. A PUT replaced that
directory with a prefix object, dropping the namespace's catalog
metadata; a DELETE removed an empty namespace. The key now gets the plain
layout rule when it names a directory or its entry cannot be read, and a
PUT of it never falls back to a prefix object, so a namespace created
after the check is not replaced either.

The time window applies to PUT only: GET, HEAD and DELETE take a key of
any date, so an object a check left behind for more than an hour can
still be read and removed. A verification PUT also refuses an extraction
request and the stored Content-Language, Response-Content-Disposition and
Content-Encoding headers (aws-chunked framing alone is still accepted).

The integration test runs the check's PUT, HEAD, GET, list and DELETE
against a table bucket, and the PUT and DELETE over a namespace of the
key's shape.

---------

Co-authored-by: Javier de la Torre <jatorre@lakehousebox.com>
This commit is contained in:
Javier de la TorreandJavier de la Torre authored and GitHub committed 2026-10-21 14:49:36 -07:00
1 parent 21add33b6e
commit de2fc03c7d
6 files changed
+427 -6

No files matched your search

+2 -2
View File
@@ -210,7 +210,7 @@ func (s3a *S3ApiServer) DeleteObjectHandler(w http.ResponseWriter, r *http.Reque
// The filer ops below each retry, and a failover walk runs the whole set
// once per filer, so the backoff comes out of one allowance held here.
r = r.WithContext(withFilerRetryBudget(r.Context(), filerRetryRequestBudget))
if err := s3a.validateTableBucketObjectPath(bucket, object); err != nil {
if _, err := s3a.validateTableBucketObjectPathOrVerify(bucket, object); err != nil {
s3err.WriteErrorResponse(w, r, s3err.ErrAccessDenied)
return
}
@@ -479,7 +479,7 @@ func (s3a *S3ApiServer) DeleteMultipleObjectsHandler(w http.ResponseWriter, r *h
deleteErrors = append(deleteErrors, deleteErrorFromCode(validationCode, object.Key, object.VersionId))
continue
}
if err := s3a.validateTableBucketObjectPath(bucket, object.Key); err != nil {
if _, err := s3a.validateTableBucketObjectPathOrVerify(bucket, object.Key); err != nil {
deleteErrors = append(deleteErrors, deleteErrorFromCode(s3err.ErrAccessDenied, object.Key, object.VersionId))
continue
}