mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-11-10 13:58:24 +01:00
Chris Lu
8236c8d136
* vector: expose request metrics and registration-aware health endpoints * vector: report immutable read cache and storage fetch metrics * docs: describe vector engine health, metrics, and maintenance * vector: count canceled requests when their handler is dropped * vector: account for uncached large immutable storage reads * vector: reliably release the stalled credential reader in tests * vector: observe large reads as their streamed body is consumed * vector: bound heartbeats on established master registration streams * test: exercise hardened vector maintenance on Go and Rust volumes * vector: register engines in their configured filer group * vector: make retried index creation preserve compatible datasets * vector: allow registration to advertise a separate address * test: release stalled credential readers without blocking FIFO cleanup * test: allow hardened vector coverage in a named filer group * vector: discover engines in the same filer group as their gateway * vector: fall back when the advertised address is empty * admin: expose vector readiness, maintenance, and management tools (#12520) * s3vectors: share vector discovery with trusted management clients * admin: show vector engine readiness and recent maintenance activity * admin: suggest S3 Vectors IAM actions from the supported operations * admin: manage vector indexes, policies, tags, encryption, and query data * admin: validate SSE-S3 support before creating encrypted vector buckets * admin: discover vector engines in their registration group * admin: probe vector engines concurrently with independent time budgets * admin: verify encryption capability on the remote S3 gateway * admin: defer discovery until after the HTTP server can start * admin: distinguish unknown vector health services from failed engines * admin: count workers that can execute vector index maintenance * admin: allow read-only users to query vectors and inspect tags * vector: scope gateway and admin discovery to the configured filer group * admin: refresh policy state after mutations and serialize policy controls * vector: drain active RPCs before closing replaced discovery connections * vector: respect request cancellation while waiting for discovery * vector: prefer another registered engine after a transport failure * admin: check all masters before reporting no registered vector engines * iam: honor attached vector policies without broadening legacy grants * vector: allow short-lived discovery clients to close their connections * vector: refresh cached master addresses for each discovery lookup * iam: enforce vector bucket tag conditions in attached policies * iam: authorize vector bucket creation with the requested tags * test: exercise vector IAM grants and tag denies on Go and Rust volumes * admin: reject authenticated vector mutations without an admin role * test: run vector IAM assertions before the shared engine is stopped * vector: add native mini startup, shell tools, and a runnable Compose example (#12522) * s3vectors: share vector discovery with trusted management clients * admin: show vector engine readiness and recent maintenance activity * admin: suggest S3 Vectors IAM actions from the supported operations * admin: manage vector indexes, policies, tags, encryption, and query data * admin: validate SSE-S3 support before creating encrypted vector buckets * admin: discover vector engines in their registration group * admin: probe vector engines concurrently with independent time budgets * admin: verify encryption capability on the remote S3 gateway * admin: defer discovery until after the HTTP server can start * admin: distinguish unknown vector health services from failed engines * admin: count workers that can execute vector index maintenance * admin: allow read-only users to query vectors and inspect tags * vector: scope gateway and admin discovery to the configured filer group * admin: refresh policy state after mutations and serialize policy controls * vector: drain active RPCs before closing replaced discovery connections * vector: respect request cancellation while waiting for discovery * vector: prefer another registered engine after a transport failure * admin: check all masters before reporting no registered vector engines * iam: honor attached vector policies without broadening legacy grants * vector: allow short-lived discovery clients to close their connections * vector: refresh cached master addresses for each discovery lookup * iam: enforce vector bucket tag conditions in attached policies * iam: authorize vector bucket creation with the requested tags * test: exercise vector IAM grants and tag denies on Go and Rust volumes * admin: reject authenticated vector mutations without an admin role * test: run vector IAM assertions before the shared engine is stopped * mini: optionally start and supervise the vector engine * mini: bootstrap vector buckets with existing Lance catalog helpers * shell: manage vector buckets, indexes, and vector data * docs: provide a runnable vector and maintenance worker Compose stack * shell: scope vector discovery to the session and close its connections * mini: honor the bind address for vector and metrics listeners * mini: reserve vector ports before assigning gRPC and compatibility listeners * mini: wait for vector storage setup and master registration * mini: create requested vector buckets without requiring the Lance HTTP listener * shell: refresh master addresses during vector discovery * mini: resolve sibling vector executables with platform extensions * vector: preserve uncertain creates and fix mini listener allocation (#12525) * vector: retain catalog records when index creation has an unknown outcome * mini: keep chosen listener ports reserved throughout allocation * mini: resolve hostname binds before starting vector listeners * vector: fix integration test imports and recover interrupted index creation (#12533) * shell: register vector operations from the command package * s3vectors: resume interrupted index initialization safely * mini: exclude disabled WebDAV from listener allocation * s3vectors: retain uncertain creates across refused retries * s3vectors: claim pending indexes before retrying initialization * s3vectors: compare resolved distance metrics during recovery * s3vectors: preserve metadata keys when retrying initialization * s3vectors: verify ownership before initializing new catalog records
Docker
Compose V2
SeaweedFS now uses the v2 syntax docker compose
If you rely on using Docker Compose as docker-compose (with a hyphen), you can set up Compose V2 to act as a drop-in replacement of the previous docker-compose. Refer to the Installing Compose section for detailed instructions on upgrading.
Confirm your system has docker compose v2 with a version check
$ docker compose version
Docker Compose version v2.10.2
Try it out
wget https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/seaweedfs-compose.yml
docker compose -f seaweedfs-compose.yml -p seaweedfs up
Try latest tip
wget https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/docker/seaweedfs-dev-compose.yml
docker compose -f seaweedfs-dev-compose.yml -p seaweedfs up
Verify an image signature
Every image CI pushes to chrislusf/seaweedfs and ghcr.io/chrislusf/seaweedfs is signed with cosign, keyless, by the GitHub Actions workflow that built it, so there is no key to fetch or pin. The signature is attached to the image digest and covers the multi-arch index and each platform image in it; latest is the release image under another tag and verifies the same way. Images published before September 2026 predate signing.
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp '^https://github.com/seaweedfs/seaweedfs/\.github/workflows/container_release_unified\.yml@' \
chrislusf/seaweedfs:latest
cosign prints the digest it verified. Deploy by that digest, or let an admission controller resolve the tag, so what runs is what was checked.
The identity is https://github.com/seaweedfs/seaweedfs/.github/workflows/<workflow>@<ref>. The ref is refs/tags/<version> for a release and refs/heads/master when a variant was republished by hand. The workflow is container_release_unified.yml for the release images, container_dev.yml for dev, container_latest.yml for a latest rebuilt by hand, container_release_foundationdb.yml for the _large_disk_foundationdb release image, and container_foundationdb_version.yml or container_rocksdb_version.yml for the per-version builds. The regexp above accepts release images only; container_[a-z_]+\.yml@ accepts everything this repository publishes, dev included.
The same check as a Kyverno policy, release images only:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: verify-seaweedfs-images
spec:
validationFailureAction: Enforce
webhookTimeoutSeconds: 30
rules:
- name: signed-by-the-release-workflow
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "docker.io/chrislusf/seaweedfs:*"
- "ghcr.io/chrislusf/seaweedfs:*"
attestors:
- entries:
- keyless:
issuer: https://token.actions.githubusercontent.com
subject: https://github.com/seaweedfs/seaweedfs/.github/workflows/container_release_unified.yml@refs/tags/*
Local Development
cd $GOPATH/src/github.com/seaweedfs/seaweedfs/docker
make
S3 cmd
list
s3cmd --no-ssl --host=127.0.0.1:8333 ls s3://
Build and push a multiarch build
Make sure that docker buildx is supported (might be an experimental docker feature)
BUILDER=$(docker buildx create --driver docker-container --use)
docker buildx build --pull --push --platform linux/386,linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6 . -t chrislusf/seaweedfs
docker buildx stop $BUILDER
Minio debugging
mc config host add local http://127.0.0.1:9000 some_access_key1 some_secret_key1
mc admin trace --all --verbose local