2 Commits

Include renames

Author	SHA1	Message	Date
Chris Lu	8236c8d136	vector: integrate engine health, admin management, and local deployment (#12517) ... * vector: expose request metrics and registration-aware health endpoints * vector: report immutable read cache and storage fetch metrics * docs: describe vector engine health, metrics, and maintenance * vector: count canceled requests when their handler is dropped * vector: account for uncached large immutable storage reads * vector: reliably release the stalled credential reader in tests * vector: observe large reads as their streamed body is consumed * vector: bound heartbeats on established master registration streams * test: exercise hardened vector maintenance on Go and Rust volumes * vector: register engines in their configured filer group * vector: make retried index creation preserve compatible datasets * vector: allow registration to advertise a separate address * test: release stalled credential readers without blocking FIFO cleanup * test: allow hardened vector coverage in a named filer group * vector: discover engines in the same filer group as their gateway * vector: fall back when the advertised address is empty * admin: expose vector readiness, maintenance, and management tools (#12520) * s3vectors: share vector discovery with trusted management clients * admin: show vector engine readiness and recent maintenance activity * admin: suggest S3 Vectors IAM actions from the supported operations * admin: manage vector indexes, policies, tags, encryption, and query data * admin: validate SSE-S3 support before creating encrypted vector buckets * admin: discover vector engines in their registration group * admin: probe vector engines concurrently with independent time budgets * admin: verify encryption capability on the remote S3 gateway * admin: defer discovery until after the HTTP server can start * admin: distinguish unknown vector health services from failed engines * admin: count workers that can execute vector index maintenance * admin: allow read-only users to query vectors and inspect tags * vector: scope gateway and admin discovery to the configured filer group * admin: refresh policy state after mutations and serialize policy controls * vector: drain active RPCs before closing replaced discovery connections * vector: respect request cancellation while waiting for discovery * vector: prefer another registered engine after a transport failure * admin: check all masters before reporting no registered vector engines * iam: honor attached vector policies without broadening legacy grants * vector: allow short-lived discovery clients to close their connections * vector: refresh cached master addresses for each discovery lookup * iam: enforce vector bucket tag conditions in attached policies * iam: authorize vector bucket creation with the requested tags * test: exercise vector IAM grants and tag denies on Go and Rust volumes * admin: reject authenticated vector mutations without an admin role * test: run vector IAM assertions before the shared engine is stopped * vector: add native mini startup, shell tools, and a runnable Compose example (#12522) * s3vectors: share vector discovery with trusted management clients * admin: show vector engine readiness and recent maintenance activity * admin: suggest S3 Vectors IAM actions from the supported operations * admin: manage vector indexes, policies, tags, encryption, and query data * admin: validate SSE-S3 support before creating encrypted vector buckets * admin: discover vector engines in their registration group * admin: probe vector engines concurrently with independent time budgets * admin: verify encryption capability on the remote S3 gateway * admin: defer discovery until after the HTTP server can start * admin: distinguish unknown vector health services from failed engines * admin: count workers that can execute vector index maintenance * admin: allow read-only users to query vectors and inspect tags * vector: scope gateway and admin discovery to the configured filer group * admin: refresh policy state after mutations and serialize policy controls * vector: drain active RPCs before closing replaced discovery connections * vector: respect request cancellation while waiting for discovery * vector: prefer another registered engine after a transport failure * admin: check all masters before reporting no registered vector engines * iam: honor attached vector policies without broadening legacy grants * vector: allow short-lived discovery clients to close their connections * vector: refresh cached master addresses for each discovery lookup * iam: enforce vector bucket tag conditions in attached policies * iam: authorize vector bucket creation with the requested tags * test: exercise vector IAM grants and tag denies on Go and Rust volumes * admin: reject authenticated vector mutations without an admin role * test: run vector IAM assertions before the shared engine is stopped * mini: optionally start and supervise the vector engine * mini: bootstrap vector buckets with existing Lance catalog helpers * shell: manage vector buckets, indexes, and vector data * docs: provide a runnable vector and maintenance worker Compose stack * shell: scope vector discovery to the session and close its connections * mini: honor the bind address for vector and metrics listeners * mini: reserve vector ports before assigning gRPC and compatibility listeners * mini: wait for vector storage setup and master registration * mini: create requested vector buckets without requiring the Lance HTTP listener * shell: refresh master addresses during vector discovery * mini: resolve sibling vector executables with platform extensions * vector: preserve uncertain creates and fix mini listener allocation (#12525) * vector: retain catalog records when index creation has an unknown outcome * mini: keep chosen listener ports reserved throughout allocation * mini: resolve hostname binds before starting vector listeners * vector: fix integration test imports and recover interrupted index creation (#12533) * shell: register vector operations from the command package * s3vectors: resume interrupted index initialization safely * mini: exclude disabled WebDAV from listener allocation * s3vectors: retain uncertain creates across refused retries * s3vectors: claim pending indexes before retrying initialization * s3vectors: compare resolved distance metrics during recovery * s3vectors: preserve metadata keys when retrying initialization * s3vectors: verify ownership before initializing new catalog records	2026-11-10 04:21:18 -08:00
Chris LuandClaude Opus 5.5	a074538c8f	admin: manage S3 vector buckets (#12334) ... * s3vectors: run an operation for trusted local tooling Like s3tables.Manager.Execute, Server.Execute runs one S3 Vectors operation through its handler as a given identity, without a signed request. The admin console can then manage vector buckets through the same validation, authorization and catalog steps as the API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * admin: list, inspect, create and delete vector buckets over the API /api/s3vectors runs the S3 Vectors operations through Server.Execute as the admin account: list vector buckets with their encryption, read one bucket with its policy and each index's data type, dimension, distance metric and non-filterable keys, create and delete a bucket, and delete an index. Going through the operations keeps the API's rules, such as ICEBERG buckets not being vector buckets and the implicit index namespace not blocking a delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * s3vectors: export the rule for which table buckets are vector buckets IsVectorBucketFormat names the check ListVectorBuckets and isVectorBucket already make, so the admin console can tell the same buckets apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * admin: vector bucket pages Object Store > Vector Buckets lists the vector buckets with their ARN, encryption and creation time, creates and deletes them, and opens one bucket with its indexes (data type, dimension, distance metric, non-filterable keys) and its policy, where an index can be deleted. A LANCE row on the Table Buckets page links to the same bucket's vector page, since both APIs serve it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK * admin: stop the vector bucket and index lists at the list limit A page cut short by filtered ICEBERG buckets let the next page carry the list past s3TablesAdminListLimit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: say when a vector bucket or index list is cut at the limit The lists stop at s3TablesAdminListLimit while more remain, so mark them truncated and point to an S3 Vectors client for the rest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: indent the vector bucket policy A policy stored as one line ran off the right edge of the details page. Policies that are not valid JSON are shown as stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * admin: show a deeply nested vector bucket policy as stored Indenting multiplies the size of a deeply nested document, so a policy past 32 levels of nesting is shown without indenting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 11:25:24 -08:00