Github-Mirror/seaweedfs
1
0 Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-10-23 22:40:39 +02:00
Code Issues Packages Projects Releases Wiki Activity
15,670 Commits 224 Branches 362 Tags
f7af816ab415dcf6da6a834dbd4269db486388ef
Commit Graph
756 Commits
Author SHA1 Message Date
Chris LuandDevin f7af816ab4 ci: cancel superseded workflow runs on PR pushes (#11979) 
* ci: cancel superseded workflow runs on PR pushes

PR-triggered workflows without a concurrency group keep running (and
holding runner slots) after a force-push or quick follow-up commit
supersedes them. Add the same group/cancel-in-progress block the go
workflow already uses, keyed on head_ref, to the remaining PR workflows.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: key concurrency groups on PR number or ref, not head_ref

github.head_ref is empty on tag pushes and manual runs (every run then
shares one group and cancels the last) and collides across forks when
two PRs share a branch name. pull_request.number is unique per PR;
github.ref covers push/tag/dispatch. Applied to all workflows using the
pattern, including pre-existing ones.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: restrict cancel-in-progress to PR and branch-push runs

* ci: serialize container_latest manual runs by destination tag

Two workflow_dispatch runs sharing an image_tag could otherwise build
concurrently: their per-arch images publish to the same platform tags,
so one run's create-manifest job can combine platforms from different
source refs. Group dispatch runs by the destination tag (with
cancel-in-progress already disabled for non-PR/non-branch events) so
same-tag publishes queue instead of interleaving.

* ci: serialize manual publish runs by destination tag in remaining container workflows

* ci: group foundationdb release dispatch runs by release tag

A manual run now shares the tag-push concurrency group for the same
release, so a re-publish of a tag queues behind (or ahead of) the push
run instead of racing it on the same image tag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: serialize foundationdb_version dispatch runs by destination tag

Two manual runs resolving to the same docker tag now share a concurrency
group (explicit image_tag, else the computed
foundationdb_<fdb>_seaweedfs_<ref> default), so they queue instead of
racing pushes to the same tag.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: drop unsupported toLower() from foundationdb_version concurrency group

* ci: key rocksdb_version dispatch group on its published tag

* ci: queue publish runs via turnstyle instead of cancel-free shared groups

* Revert "ci: queue publish runs via turnstyle instead of cancel-free shared groups"

This reverts commit 039148f8d5.

* ci: queue same-tag publish runs instead of replacing the pending one

Dispatch/tag-only workflows can never cancel in progress, so opt their
concurrency groups into queue: max — up to 100 same-tag publishes wait
in line rather than each new dispatch cancelling the pending one.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-23 04:49:52 -07:00
Chris Lu 7c478fd89a fuse integration: run posix suites on weed mini and allow allow_other mounts (#11984) 
The separate master/volume/filer processes in framework.go bound fixed
ports, discarded child output, and never noticed an early process exit,
so a slow or dead volume server burned the whole 30s timeout with no
logs. Switch to the same weed-mini approach the fuse_test package
framework already uses: dynamic filer ports, child output captured to
log files, and readiness waits that bail out the moment the process
dies. The external-mount path (TEST_MOUNT_POINT) is unchanged.

The posix-compliance workflow mounts with -allowOthers, but fusermount
refuses allow_other unless user_allow_other is set in /etc/fuse.conf,
mirroring the fuse-integration workflow FUSE setup.
 2026-10-23 04:33:14 -07:00
Chris LuandDevin bf3081e23a s3 lifecycle: route version-file writes to current-version expiration (#11956) 
* s3lifecycle: schedule ExpirationDays from version-file writes

On a versioned bucket a PUT lands in the meta-log as the version file
plus the .versions/ directory update — there is no bare-key event, so
the version-folder skip dropped every version-file event for all kinds
but ExpiredDeleteMarker. ExpirationDays therefore only fired via the
recovery walker's cold-start / rule-change pass: once a shard had a
saved cursor, no daily run ever scheduled another delete marker.

A version-file write is the latest for its logical key at write time,
so route it through the same event-driven evaluation the regular path
runs. The match carries the file's identity with an empty VersionID —
the dispatcher resolves the current latest at due time, and a fresher
PUT meanwhile resolves to STALE_IDENTITY there.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* lifecycle test: versioned Expiration fires on the second daily run

Establishes the shard cursor with a first pass, then PUTs and backdates
a new versioned object and runs again. The delete marker must appear
without a rule change forcing a recovery walk.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: run TestLifecycleVersionedBucketExpiresOnSecondRun in the lifecycle matrix

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-23 03:19:41 -07:00
Chris LuandDevin 7c980774e7 FUSE mount: add posix compliance reports (#7183) 
* fuse integration: add POSIX compliance test suite

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: add posix-compliance workflow

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: skip non-executable weed path candidates

* fuse integration: size readv buffer to the written segment

* fuse integration: reference posix_Makefile explicitly in recursive make calls

GNU make does not propagate -f through MAKEFLAGS, so bare $(MAKE) re-read the default Makefile in this directory.

* fuse integration: fail posix test targets when go test fails

Every suite pipes go test through tee under /bin/sh, so the recipe status was tee's and test failures stayed green. Run recipes under bash -o pipefail and emit the color escapes with printf since bash echo does not expand them.

* fuse integration: run only benchmarks in benchmark-posix

go test -bench=. without -run also executes the whole suite, mixing compliance output into the benchmark log.

* fuse integration: emit single-number test counts in reports

grep -c prints 0 and exits nonzero when nothing matches, so '|| echo 0' produced two lines that broke the expr arithmetic and JSON fields. awk prints exactly one count either way.

* fuse integration: skip the sendfile test on macOS

macOS sendfile(2) only accepts a socket destination; the test copies into a regular file and always gets ENOTSOCK.

* fuse integration: honor TestConfig and own only self-started resources

NewFuseTestFramework dropped its config argument and Cleanup read a fresh
DefaultTestConfig, so SkipCleanup never did anything. Cleanup also skipped
teardown whenever TEST_SKIP_CLUSTER_SETUP was set, even when TEST_MOUNT_POINT
was empty and Setup had started the cluster itself. The framework now keeps
the config it was built with, runs external-mount tests inside a per-run work
dir under the mount, and only skips teardown when an external mount was
actually used.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: pass the debug flag before the subcommand

-v is a weed global flag; appended after master/volume/filer/mount it lands
on the subcommand flagset, which rejects it and the process dies at flag
parsing whenever EnableDebug is on.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: require a real mount before running tests

Setup creates the mount point before weed mount starts, so statting and
listing it succeeded on the plain directory and tests ran against local
disk whenever the mount was slow or died. waitForMount now waits until the
mount point's device id differs from its parent's.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: resolve the weed binary to an absolute path

The started processes run with cmd.Dir at the framework's temp dir, so a
relative candidate like ../weed no longer resolves. Prefer PATH, then probe
the usual build outputs and return absolute paths only.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: build the external test work dir with MkdirTemp

The suite staged cloned tools and generated scripts under a predictable
/tmp path another user could pre-create and swap mid-run.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: use the pjdfstest tree setup-pjdfstest installs

The test gated on a pjdfstest binary in PATH but then proved a suite it
cloned itself, so a tree installed by make setup-pjdfstest was never found.
Look for the built suite under external_tools first, then clone into the
work dir as before.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: require a real mount before posix tests

The wait loop fell back to listing the mount point, which passes on the
directory created before weed mount runs — a dead or slow mount meant the
suite exercised local disk. Only mountpoint counts now, and a dead mount
process fails fast instead of timing out.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: collect failure logs before cleanup deletes them

The always-on cleanup step removed /tmp/seaweedfs before the failure-only
debug step could copy the service logs, so failed runs uploaded nothing.
Run cleanup after log collection.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: point MOUNT_POINT at TEST_MOUNT_POINT

Targets that work against MOUNT_POINT (pjdfstest, nfstest, fio configs)
ignored TEST_MOUNT_POINT and ran against the local-disk default while the
go tests used the external mount.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: skip the permission-denied check as root

Root bypasses mode-bit checks, so writing the read-only file succeeded and
the assertion failed in root-run environments like containers.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: align the posix docs with the suite

The doc listed a docker-test-posix target and make variables that do not
exist, and ran go test against single files that cannot compile alone. The
roadmap also still claimed the platform-specific tests this suite now
implements were all skipped.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: honor SkipCleanup in the mini-based framework too

Same gap as the fuse-package framework: Cleanup read a fresh
DefaultTestConfig, so a test's SkipCleanup setting was ignored.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fuse integration: normalize config in Setup and clean the work dir too

Setup(nil) panicked on config.EnableDebug and external-mount Cleanup left
the temp dir behind; use the stored config for the start* calls and remove
tempDir alongside the external work dir.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: point external posix suites at the workflow FUSE mount

* fuse integration: create the fio target directory before running

* fuse integration: use unix.O_DIRECT instead of a hard-coded x86 value

0x4000 is O_DIRECT only on x86 and amd64; on arm64 it is O_DIRECTORY.
The test module already depends on golang.org/x/sys.

* fuse integration: run autoreconf and configure before building pjdfstest

A fresh pjdfstest clone is autotools-based and has no Makefile, so the
Go-side setup fallback always failed and the suite silently skipped.

* ci: require external posix tools only when setup installed them

setup-external-tools always exits 0 because each sub-target tolerates
failure, so POSIX_REQUIRE_EXTERNAL_TOOLS was set even with no tools
installed and the suite then hard-failed on the missing suites.

* ci: fail the posix workflow when the mount point is not a FUSE mount

The file write/read check passes on the plain directory created before
weed mount, so a dead mount let the whole suite grade the local disk.

* fuse integration: skip when the host has no FUSE device node

Setup failed the test outright on machines without /dev/fuse instead of
skipping; cover every suite entry point by skipping inside Setup.

* fuse integration: drop the chmod 666 /dev/fuse troubleshooting tip

World-writable device access is a security risk; the fuse group
membership on the line below is the safe fix.

* fuse integration: require go 1.21+ in check-go, not 1.20+

* fuse integration: refuse external suites on an unmounted path

* fuse integration: verify TEST_MOUNT_POINT is a real mount

* fuse integration: clean TEST_MOUNT_POINT before the parent-device check

filepath.Dir of a trailing-slash path returns the mount itself, so a mounted TEST_MOUNT_POINT ending in / was refused.

* fuse integration: detect WinFsp mounts on Windows in mountedAt

A directory mount is a reparse point and a drive mount is a volume root; a plain directory is now refused instead of accepted.

* fuse integration: strip a trailing slash off MOUNT_POINT in require-mounted

The mount-table grep fallback on hosts without mountpoint(8) would not match a slash-suffixed entry.

* fuse integration: require an external mount to be a FUSE filesystem

A device-ID boundary alone does not prove FUSE: a tmpfs mount would pass mountedAt and report its results as FUSE compliance. Check the filesystem type (statfs magic on linux, fstypename on the BSDs); platforms without a check keep the mount and write verification.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-22 14:37:03 -07:00
Chris LuandDevin dbcf6850b6 ci: fail the awscli multipart retry loop on the last attempt (#11848) 
Co-authored-by: Devin <devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-19 17:59:50 -07:00
Chris LuandDevin 04eff7d613 ci: retry the awscli multipart download on transient failures (#11846) 
The HEAD on the completed multipart object occasionally gets a 403 from a
signature verification mismatch before the object read settles; retry the
download instead of failing the run.

Co-authored-by: Devin <devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-19 17:09:58 -07:00
dependabot[bot] 109dd24f52 build(deps): bump github/codeql-action from 4.38.3 to 4.39.0 (#11840) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.38.3 to 4.39.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.38.3...v4.39.0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-19 16:05:07 -07:00
dependabot[bot] 9aae0bb702 build(deps): bump helm/kind-action from 1.15.0 to 1.15.1 (#11747) 
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.15.0 to 1.15.1.
- [Release notes](https://github.com/helm/kind-action/releases)
- [Commits](https://github.com/helm/kind-action/compare/v1.15.0...v1.15.1)

---
updated-dependencies:
- dependency-name: helm/kind-action
  dependency-version: 1.15.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-15 21:53:01 -07:00
dependabot[bot] d167521270 build(deps): bump helm/chart-testing-action from 2.8.0 to 2.9.0 (#11745) 
Bumps [helm/chart-testing-action](https://github.com/helm/chart-testing-action) from 2.8.0 to 2.9.0.
- [Release notes](https://github.com/helm/chart-testing-action/releases)
- [Commits](https://github.com/helm/chart-testing-action/compare/v2.8.0...v2.9.0)

---
updated-dependencies:
- dependency-name: helm/chart-testing-action
  dependency-version: 2.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-15 06:20:36 -07:00
dependabot[bot] 896e4dd192 build(deps): bump github/codeql-action from 4.38.2 to 4.38.3 (#11746) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.38.2 to 4.38.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.38.2...v4.38.3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.38.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-15 06:18:54 -07:00
Ege ZambelliDevinEge ZambelliChris Lu
 		f11cecc697 helm: keep the S3 config checksum stable across the first upgrade (fixes #11719) (#11720) 
* helm: keep the S3 config checksum stable across the first upgrade (fixes #11719)

checksum/s3config hashed the rendered s3-secret.yaml. That template
keeps its generated S3 keys with lookup, so rendering it again for the
checksum drew new keys whenever lookup found nothing: on install the
filer, S3 and all-in-one pods each carried a checksum of a different
set of keys than the Secret got, and the first upgrade with unchanged
values corrected it and restarted them.

Hash the values s3-secret.yaml is built from instead: the enable and
auth switches, existingConfigSecret, credentials, the legacy-secret
settings, the audit log configs and the Secret name. Only those, so
scaling the S3 gateway or changing its resources does not restart the
filer. A change to any of them still restarts the pods. Keys that the
chart generated stay out of the hash; they only change if the Secret
is deleted by hand.

The new render test fails on the old template (two checksums for one
Secret, different on every render) and passes now; it also checks that
the checksum follows the credentials and filer.s3.auditLogConfig and
ignores s3.replicas and s3.resources. On kind, an upgrade with unchanged
values no longer replaces the filer pod.

* helm: check the S3 checksum per workload, all-in-one included

Each workload's checksum/s3config is now read on its own (--show-only),
so a workload that loses the annotation fails the test instead of being
hidden by sort -u, and the all-in-one deployment gets the same checks.
Verified against two broken copies: one without the filer annotation,
one with the old all-in-one checksum; the test fails on both.

The helper comment now says what happens when the Secret is deleted to
rotate generated keys.

* ci(helm): fail the S3 checksum test on a missing or unrendered checksum

A failed helm template or a render without the annotation produced an
empty checksum, and the inequality assertions then passed against it.
s3_checksum now returns non-zero unless it found a checksum, and the
changed-value renders are assigned before comparing so failures cannot
hide inside test.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Ege Zambelli <egezamb@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-12 15:43:53 +08:00
yi111Yi-111-aChris LuChris Lu
 		c96eb7b8ee kms/azure: fix encrypt/decrypt round trip for Azure Key Vault (#11692) 
* kms/azure: fix encrypt/decrypt round trip for Azure Key Vault

The provider stored the wrapped data key as string(encryptResult.Result)
in the JSON envelope, sent the encryption context as AAD to an RSA-OAEP
key, and passed a full key URL to the azkeys client in the name
position. Each of those breaks a round trip on its own.

- split the key URL into the (name, version) pair azkeys expects before
  Encrypt, Decrypt and GetKey; a plain name still resolves to the latest
  version, and decrypt keeps the stored version so objects stay readable
  after a key rotation
- store the wrapped key base64-encoded and decode it strictly, so the JSON
  envelope no longer replaces the raw bytes with U+FFFD
- stop sending AAD: Key Vault rejects it on RSA-OAEP with BadParameter,
  so the encryption context is logged and dropped instead

* kms/azure: reject a key URL that names another vault

splitKeyID dropped the host of a full Key Vault URL, so a key ID from a
different vault silently resolved to this vault's same-named key. Return
an error instead of encrypting under a key the caller did not ask for.

* kms/azure: bind encryption context via an envelope digest

RSA-OAEP rejects AAD, so removing it left the encryption context
unauthenticated: a wrapped key could be decrypted under a different
object's context. Record a sha256 digest of the marshaled context in
the envelope's provider_specific field on encrypt and verify it before
calling Decrypt, restoring the binding without AAD.

* kms/azure: treat an explicit :443 port as the same vault

* kms/azure: accept an absent context digest only for empty contexts

* kms/azure: normalize both hosts when comparing key URLs to the vault

splitKeyID stripped :443 and a trailing dot from the configured vault but
only :443 from the key URL, so a key URL naming the same vault with a
trailing DNS dot was rejected before Azure was ever called. Compare both
sides through vaultHost so they are normalized identically.

* kms/azure: reject non-key vault URLs in splitKeyID, document digest limits

A Key Vault URL that does not name a key under /keys/, has an empty key
name, or carries extra path segments now fails fast instead of being
passed to the client as a key name, where it would surface as a
confusing vault-side error.

Also note that the envelope context digest is a client-side mismatch
check, not vault-authenticated AAD, and only allocate providerSpecific
when a context is present.

* ci: compile and test azurekms-gated code

weed/kms/azure is excluded from every default build, so nothing in CI
compiled it; that is how the provider shipped unregistered. Build the
tree and run the kms tests with -tags azurekms on every Go change.

---------

Co-authored-by: Yi-111-a <>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
 2026-10-10 23:40:19 +08:00
Javier GarciaChris LuChris LuDevin
 		838c554e33 s3api: reject SSE headers PutObject cannot honor, as CopyObject does (#11642) 
* s3api: reject SSE headers PutObject cannot honor, as CopyObject does

PutObject and CreateMultipartUpload did not check the server-side
encryption headers they were given. An x-amz-server-side-encryption
value that names no method, such as "aes:kms", matched none of the SSE
paths, so the object was stored unencrypted and the request answered
200. SSE-C together with x-amz-server-side-encryption was also accepted,
and one of the two silently won.

CopyObject already rejects both through validateEncryptionCompatibility.
Run the same check, with the same error codes, before PutObject and
CreateMultipartUpload store anything.

* s3api: answer rejected SSE headers with InvalidArgument, as S3 does

S3 rejects an unknown x-amz-server-side-encryption value and SSE-C
combined with another method with InvalidArgument. PutObject and
CreateMultipartUpload now return that code, with S3's messages, through
two new error codes; CopyObject keeps its own.

Also run ceph/s3-tests' test_put_obj_enc_conflict_c_s3, _c_kms and
_bad_enc_kms in CI, which check both handlers' responses end to end.

* s3api: close the remaining ways a PUT could skip requested encryption

- A repeated x-amz-server-side-encryption header is rejected: the
  encryption paths apply only the first value, so extra values could
  hide the method the client asked for.
- KMS options (key id, encryption context, bucket key) are rejected
  unless the method is aws:kms, matching the S3 InvalidArgument error.
- CreateMultipartUpload validates before auto-create, so a refused
  upload cannot leave a bucket behind.
- Directory markers encrypt their inline content through the shared
  SSE path and record the same entry metadata as regular objects,
  instead of storing requested-encrypted bytes in plaintext.

* s3api: read back what the SSE marker write stores

- Repeated SSE-C and KMS option headers are rejected alongside a
  repeated x-amz-server-side-encryption, closing the same first-value
  bypass for customer-key and KMS fields.
- Algorithm validity is checked before KMS options so an unsupported
  value keeps the "not supported" error.
- serveDirectoryContent decrypts marker content with the stored SSE
  metadata and returns the SSE headers, so an encrypted marker reads
  back what was written; its Content-Length now reflects the bytes
  actually served.

* s3api: HEAD of a marker skips decryption and keeps the stored size

HEAD returns no body, so it now runs only the SSE-C key check instead of
decrypting — matching HeadObjectHandler and avoiding a KMS round-trip —
and chunk-backed directory entries report Attributes.FileSize again
rather than the length of their (empty) inline content.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3api: keep GET Content-Length to the bytes serveDirectoryContent writes

The FileSize override described chunk-backed markers on HEAD, but GET
sends only the inline content, so it promised bytes it never wrote.

* s3api: stream a chunk-backed directory on GET like any object

A directory promoted over an uploaded object keeps the object chunks
with empty inline content, so serving only entry.Content made GET
deliver nothing while HEAD reported FileSize. GET now routes those
entries through the regular volume-server stream, keeping the two
methods consistent.

* s3api: answer a busy volume read with RequestBytesExceed

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-10-09 12:43:04 +08:00
Younsung Lee 68af030e38 helm: add Gateway API HTTPRoute for master, volume, filer, s3 and admin (#11625) 
* helm: add Gateway API HTTPRoute for master, volume, filer, s3 and admin

Each component that offers an Ingress can now be exposed through a
Gateway API HTTPRoute instead, via <component>.httpRoute. It is disabled
by default, so existing renders are unchanged.

parentRefs, hostnames, labels and annotations pass through as given.
Each rule may set matches, filters, timeouts and backendRefs, and a rule
without backendRefs routes to the component's own service and port,
following all-in-one mode the same way the Ingress templates do. An
empty rules list yields a single rule sending all traffic there.

Signed-off-by: younsl <cysl@kakao.com>

* helm: fix HTTPRoute backend selection for S3 on filer and all-in-one

- s3: with S3 on the filer the route targets filer.s3.port, the port the
  s3 Service exposes. The all-in-one Service is chosen only when
  allInOne.s3 is enabled, so a standalone S3 next to all-in-one routes
  to the s3 Service.
- master: render in all-in-one mode and route to the all-in-one Service,
  like the volume and filer routes.
- Name routes with seaweedfs.componentName, keeping them within 63
  characters like the Services they point to.
- CI: cover the filer S3 port and the all-in-one master route.

Signed-off-by: younsl <cysl@kakao.com>

* helm: document the Gateway API HTTPRoute values in the chart README

Signed-off-by: younsl <cysl@kakao.com>

* helm: link the Gateway API docs from the chart README

Signed-off-by: younsl <cysl@kakao.com>

* helm: scope the filer S3 port note in the chart README

Signed-off-by: younsl <cysl@kakao.com>

---------

Signed-off-by: younsl <cysl@kakao.com>
 2026-10-09 11:37:10 +08:00
Upayan b14dd1cee4 helm: drop fromToml dependency in security-configmap.yaml (fixes #11611) (#11614) 
* helm: drop fromToml dependency in security-configmap.yaml (fixes #11611)

fromToml was added to Helm in v3.17.0 (helm/helm#12026, merged
2024-09-12, one day after v3.16.0 was cut). This chart declares no
minimum Helm version (no Chart.yaml kubeVersion, nothing in the
README), and the call in security-configmap.yaml:21 is an
unconditional *parse*-time failure on Helm < v3.17.0 - Go's
text/template parses a file's entire body before evaluating any
{{if}}, so this breaks the chart (any topology, any values) even when
securityConfigEnabled is false and the ConfigMap would render nothing.

Replaces the fromToml-based dig lookup with a small regex-based helper
(seaweedfs.existingTomlKey) that reads the same four "key = ..." JWT
signing-key values out of a previously-rendered security.toml,
preserving the existing fallback-to-random behavior exactly.

Verified:
- helm lint (v3.16.3 and v4.3.0): clean
- helm template with chart defaults: byte-identical output to the
  unpatched chart rendered via Helm v4 (which has fromToml) - the
  disabled/default path is untouched
- helm template with security enabled, no prior ConfigMap: identical
  structure to the unpatched chart (helm v4), modulo the expected
  random key
- Real helm install + helm upgrade round trip (live lookup, since
  "helm template" never evaluates lookup, even under the original
  fromToml code): the JWT signing key is identical across both
  releases - confirms key persistence across upgrades is preserved,
  not just "renders without erroring"
- helm template with chart defaults, Helm v3.16.3: previously failed
  with a parse error naming fromToml as undefined; now renders
  successfully

Fixes #11611.

* helm: harden existingTomlKey against commented key lines and CRLF

Addresses two review findings from greptile-apps on PR #11614:

- The key-line regex matched the first "key = ..." anywhere in the
  section block, including a commented-out "# key = ..." line, which
  would shadow a real active key on a hand-edited or otherwise
  non-chart-generated ConfigMap. Anchored to line start with the
  Go regexp multiline flag ((?m)^key...), which a line starting with
  "#" cannot match.
- The section-header match required an exact "]\n", so a ConfigMap
  with CRLF line endings would fail to match the block at all and
  regenerate the key instead of reusing it. Changed to "]\r?\n".

Also adds a CI test ("Verify JWT signing key persistence across
upgrades") exercising all of this end to end with a real
helm install -> edit the live ConfigMap -> helm upgrade cycle,
matching the existing "Verify SFTP host key secret lifecycle" test's
shape: both edge cases are reproduced against a real ConfigMap and
asserted on the post-upgrade rendered security.toml.

Verified locally (same commands as the new CI step) against a real
cluster before pushing.

* helm: preserve JWT keys across supported TOML layouts

* ci: use setup-python interpreter for JWT upgrade checks

* helm: preserve keys under quoted TOML section headers

* helm: ignore unrelated quoted TOML section headers
 2026-10-06 12:56:08 +08:00
Peter Dodd 8ebef03509 helm: expose s3 update strategy, lifecycle, and termination grace period (#11602) 
The standalone s3 Deployment hardcoded terminationGracePeriodSeconds and
offered no way to set a container lifecycle or the Deployment strategy, so
operators could not add a preStop delay to drain endpoints before SIGTERM
or hold maxUnavailable at 0 during rollouts.

Add s3.updateStrategy, s3.lifecycle and s3.terminationGracePeriodSeconds.
Defaults render the same manifest as before.
 2026-10-06 10:10:44 +08:00
zhao-ycandChris Lu 16e66b1bad fix(s3): initialize destination ACLs for CopyObject (#11599) 
* fix(s3): initialize destination ACLs for CopyObject

Signed-off-by: zhaoyuchen <43179751+zhao-yc@users.noreply.github.com>

* fix(s3): re-check routed self-copy eligibility on the locked read

routeInPlace was decided on the pre-lock entry, but the PATCH body
re-reads the entry. A concurrent write changing file mode or MIME in
between left the routed PATCH installing new ACL keys while Attributes
kept the stale mode. Evaluate eligibility against the re-read entry and
retry the self-copy under the distributed lock when it no longer
qualifies.

* fix(s3): guard metadata self-copies against concurrent writes

Signed-off-by: zhaoyuchen <43179751+zhao-yc@users.noreply.github.com>

* ci: raise s3api unit-test timeout to 9m

The suite crossed the 5m binary timeout on the hosted runner (local run
is ~4.3m and still growing). The job-level limit is already 10m.

* ci: allow setup time before the S3 API test suite

Signed-off-by: zhaoyuchen <43179751+zhao-yc@users.noreply.github.com>

---------

Signed-off-by: zhaoyuchen <43179751+zhao-yc@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2026-10-05 18:22:16 +08:00
hajimohammadi.net 0d93dec145 helm: mount TLS certificates in bucket hook (#11589) 2026-10-04 20:35:08 +08:00
deepnemesisSubhadeep MaityCursorChris Lu
 		43abe21ffa Helm: Add Opt-in Read-Only Root Filesystem Support (#11562) 
* security context changes

Signed-off-by: Subhadeep Maity <smaity@slb.com>

* root file changes

Signed-off-by: Subhadeep Maity <smaity@slb.com>

* Skip tmp mount when extras provide one; use allInOne context for the bucket hook

* Mount tmp for secondary containers; keep user /tmp on the main container

---------

Signed-off-by: Subhadeep Maity <smaity@slb.com>
Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>
Co-authored-by: Subhadeep Maity <smaity@slb.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2026-10-02 22:31:48 +08:00
deepnemesiscoderabbitai[bot]greptile-apps[bot]Subhadeep Maity
 		7c7834c98e Helm: Add Configurable Security Contexts for Chart-Managed Workloads (#11558) 
* security context changes

Signed-off-by: Subhadeep Maity <smaity@slb.com>

* extending examples

Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>

* updating examples

Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>

* examples

Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>

* Update k8s/charts/seaweedfs/values.yaml

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update k8s/charts/seaweedfs/values.yaml

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* review comments

Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>

---------

Signed-off-by: Subhadeep Maity <smaity@slb.com>
Signed-off-by: Subhadeep Maity <322813880+deepnemesis@users.noreply.github.com>
Co-authored-by: Subhadeep Maity <smaity@slb.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-10-01 22:39:54 +08:00
Javier Garcia 8fdcf69eb0 s3api: report the stored checksum in GetObjectAttributes (#11529) 
GetObjectAttributes accepted the Checksum attribute but never filled it
in, as its comment said SeaweedFS did not store S3 checksums. PutObject
and CompleteMultipartUpload store them now, and HeadObject returns them.
Fill in Checksum from the same entry fields, with the ChecksumType and
ChecksumCRC64NVME members the response did not have.

Also run ceph/s3-tests' test_get_checksum_object_attributes in CI.
 2026-10-01 01:11:19 +08:00
Javier Garcia 988fc4f7ba s3api: do not store aws-chunked in an object's Content-Encoding (#11528) 
* s3api: do not store aws-chunked in an object's Content-Encoding

aws-chunked in Content-Encoding names the SigV4 streaming framing of
the request body, which the gateway decodes on upload. PutObject and
CreateMultipartUpload stored the header as sent, so an object uploaded
with "gzip, aws-chunked" was served with that Content-Encoding, and one
uploaded with "aws-chunked" alone was served as aws-chunked. S3 drops
aws-chunked and keeps the other encodings.

Also run ceph/s3-tests' test_object_content_encoding_aws_chunked in CI.

* s3api: read every Content-Encoding field, and drop aws-chunked on copy

A client can send aws-chunked and the object's own encoding as separate
Content-Encoding fields. Only the first was read, so "aws-chunked"
followed by "gzip" left the object without its gzip. Combine all the
fields before dropping aws-chunked. CopyObject with the REPLACE
directive stored the requested Content-Encoding as sent: drop
aws-chunked there too.
 2026-10-01 01:10:38 +08:00
Ishwar Kanse 5fcf12c080 docker: publish release and latest images for linux/ppc64le and linux/s390x (#11517) 
Add ppc64le and s390x to the normal and large_disk release image
matrices, next to arm/v7 and 386, and to the manual container_latest
fallback so that rebuilding `latest` does not drop them from the
manifest.

The Go binary is cross-compiled without cgo on the build host, and
Dockerfile.go_build already falls back to empty Rust volume server and
worker placeholders on platforms without pre-built Rust binaries. Only
the small final-stage RUN steps run emulated, so these two platforms use
QEMU on ubuntu-latest like arm/v7.

Refs #5558
 2026-09-29 21:04:19 +08:00
dependabot[bot] c54f1fe69c build(deps): bump github/codeql-action from 4.38.1 to 4.38.2 (#11509) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.38.1 to 4.38.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.38.1...v4.38.2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-29 10:10:07 +08:00
Ping QiuandClaude Opus 5 635f69a821 test(s3tables): add an OLake Iceberg catalog integration test (#11441) 
OLake (github.com/datazip-inc/olake) is a CDC and ingestion engine that writes
Apache Iceberg. It covers two paths none of the existing catalog suites reach.

It is a strict Java Iceberg client. OLake does not write Iceberg from Go — its
Go process spawns a Java sidecar over gRPC and writes through the official
Apache Iceberg library, because the Go library has no equality deletes and CDC
needs them. That makes it the client class weed/s3api/iceberg's metadata
compliance backfill exists to serve: the one that fails with "Cannot parse
missing long current-snapshot-id" when spec-required keys are omitted.

And it produces equality deletes. Its upsert path commits operation=overwrite
with an equality-delete file and a delete manifest. ClickHouse, Doris, Trino,
Spark and DuckDB all only append, so nothing else in this directory exercises
a delete manifest at all.

Six subtests: the destination check reaches SUCCEEDED and actually loads the
REST catalog; discover enumerates the source; a full sync commits a snapshot;
PyIceberg reads back what the Java writer committed; an update plus a re-sync
records an overwrite carrying equality deletes with a delete manifest in the
current snapshot; and the catalog does not rewrite manifests the official Java
writer produced, which gives the manifest-repair path a negative test to go
with the ClickHouse positive one.

What it deliberately does not assert is a delete-applied read. PyIceberg
refuses to scan a table carrying equality deletes (apache/iceberg#6568) while
reading its metadata fine, and an engine that can apply them costs a
multi-gigabyte image. Recording the commit correctly is the catalog's
contract; applying deletes on read is the engine's. The README says so, says
the read half was verified by hand once with StarRocks, and warns against
later "upgrading" this to a PyIceberg rows read — which would either fail or,
if PyIceberg ever starts skipping deletes instead of raising, pass by not
looking.

The workflow job asserts the suite actually ran rather than trusting a green
exit, for the same reason: at least one top-level PASS and zero SKIP.

No product change — the destination config is the generic catalog_type=rest
with the standard OAuth2 client-credentials flow, and s3_path_style is not even
set, since OLake turns it on itself whenever s3_endpoint is non-empty.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-25 07:39:27 +08:00
dependabot[bot] 559ec33498 build(deps): bump github/codeql-action from 4.38.0 to 4.38.1 (#11423) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.38.0 to 4.38.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.38.0...v4.38.1)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.38.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-22 16:14:47 +09:00
dependabot[bot] d6397fdc50 build(deps): bump dorny/paths-filter from 3 to 4 (#11422) 
Bumps [dorny/paths-filter](https://github.com/dorny/paths-filter) from 3 to 4.
- [Release notes](https://github.com/dorny/paths-filter/releases)
- [Changelog](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/dorny/paths-filter/compare/v3...v4)

---
updated-dependencies:
- dependency-name: dorny/paths-filter
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-22 16:14:38 +09:00
dependabot[bot] 264030c08c build(deps): bump docker/setup-qemu-action from 4.3.0 to 4.4.0 (#11421) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.3.0 to 4.4.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v4.3.0...v4.4.0)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-22 16:14:30 +09:00
Eliah RusinClaude Fable 5.1Chris Lu
 		7643f4f541 rust: a seaweed-common crate for the address and TLS helpers both crates carry (#11358) 
* rust: a seaweed-common crate for the address and TLS helpers both crates carry

seaweed-volume and seaweed-worker are separate cargo trees with separate
lockfiles and no root manifest, so anything both of them need has had to be
written twice. Two of those copies are a correctness risk rather than a typing
cost, and this crate is where they stop being copies.

address.rs is the HTTP<->gRPC port rule: `host:port` means gRPC on port+10000,
`host:port.grpcPort` names it outright. The two copies had already drifted —
the worker's bracketed IPv6 literals, the volume server's did not — so the rule
lives here once, returning a typed AddressError whose Display text is the volume
server's original wording, with join_host_port public beside it. A test asserts
two of those messages in full rather than by substring, because the wording is
the contract its callers hand to a Status or an io::Error; the other three end
in a std ParseIntError message, which is std's to reword. The enum is
#[non_exhaustive] so a future variant is not a breaking change for either
consumer. The tests are both crates' cases together, plus the IPv6,
already-bracketed and normalisation cases neither copy covered on its own.

tls.rs is install_default_crypto_provider. Both binaries link aws-lc-rs and ring
transitively, so rustls cannot auto-select and tonic's client TLS panics on
first use; each binary has to pin one and it has to be the same one, which is
exactly the kind of choice that should not exist twice. It is safe to share
because `cargo tree -i rustls` resolves a single rustls in each tree (0.23.37 in
seaweed-volume, 0.23.43 in seaweed-worker) and cargo unifies all
semver-compatible `rustls = "0.23"` requirements into one crate per binary, so
this crate writes the same process-wide static its consumer reads. rustls is
already in both graphs — directly in the volume server, through tonic's
tls-aws-lc in seaweed-worker-core — so the dependency adds no crate to either.

rust-version is 1.91.1, the lower of the two consumers' floors, so depending on
this crate cannot raise either tree's MSRV; verified with
`cargo +1.91.1 check --all-targets`. The lockfile is committed even though this
is a library: CI builds it directly, so a committed lock is what makes those
runs reproducible and their caches stable.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* rust: take the address and TLS helpers from seaweed-common

Both public signatures are kept, so no caller outside the two wrapper files
changes. parse_grpc_address stays `Result<String, String>` and maps the typed
error through Display; server_to_grpc_address stays `Option<String>` and drops
it with .ok(). Their doc comments and the volume server's 13 call sites are
otherwise untouched.

Three behaviours change, each in the direction of the copy that was already
right:

- The volume server now brackets IPv6 literals. `::1:19333` used to come back as
  `::1:29333`, which build_grpc_endpoint rejects with "invalid gRPC endpoint
  http://::1:19333: invalid authority" — an IPv6 master or EC peer could not be
  dialled at all. Two tests in grpc_client.rs pin it, one on the string and one
  on the endpoint the string builds.
- The volume server now emits the *parsed* gRPC port of the dotted form instead
  of the original text it had just validated, so `host:8080.018080` and
  `host:8080.+18080` come back as `host:18080` rather than as authorities the
  URI parser rejects. Same port either way; only malformed spellings change.
- The worker's dotted form now validates the HTTP port it discards.
  `server_to_grpc_address("host:abc.18080")` used to answer Some("host:18080");
  it now answers None, which is what the volume server's copy has always done.

install_default_crypto_provider becomes a re-export in both trees, so
`crate::security::tls::install_default_crypto_provider` and
`weed_lance_worker::tls::install_default_crypto_provider` still resolve. The
lance crate's `rustls = "0.23"` was its only direct use of rustls and goes away
with the body; seaweed-common states the same requirement, so neither the
resolved version nor the enabled features move in either lockfile.

The PEM test fixtures stay where they are. The two tests that use them are not
duplicates: the volume server's exercises build_grpc_endpoint, and the lance one
exists precisely because aws-lc-rs and ring are both linked in that crate's
graph. Only the literals are shared, and exporting test fixtures from a library
to dedupe two constants costs more than it saves.

A path dependency outside both trees means every build context that copies one
crate directory has to copy the other. The repo has one: the Rust source-build
stage of docker/Dockerfile.go_build, which now copies seaweed-common beside
seaweed-volume. Every workflow whose `paths:` filter keys on a crate directory
gains `seaweed-common/**` — the two Rust test workflows, rust_binaries_dev,
container_dev and performance. The tag- and dispatch-triggered ones
(rust_binaries_release, container_release_unified, container_latest) have no
`paths:` filter and need nothing.

The two Rust test workflows also run `cargo test` in seaweed-common, from their
unit-test job, because a path dependency is not a workspace member and neither
tree's own `cargo test` reaches it. Each step builds into its job's cached
target directory, and both cache keys now hash seaweed-common/Cargo.lock as well
so a change there invalidates the cache it would otherwise silently reuse.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docker: keep go_build working for BRANCH revisions without seaweed-common

The rust_builder stage copies seaweed-common unconditionally now that seaweed-volume path-depends on it, but BRANCH can name any revision — including ones that predate the crate. Create the directory in the builder stage so the COPY always has a source; an empty dir beside an old seaweed-volume is harmless.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
 2026-09-20 13:30:58 -07:00
Chris Lu 44ba070d83 helm: document admin.allowInsecureBind and drop stale comment 
- README and values.yaml now describe the allowInsecureBind escape
  hatch alongside the non-loopback bind guard
- remove a PR reference from the CI test comment
 2026-09-20 12:28:53 -07:00
Yongun SeongandClaude Sonnet 5 062238bb5c helm: let admin.allowInsecureBind satisfy the admin auth render guard 
PR #11185 made `weed admin` refuse to bind a non-loopback address
without -adminPassword or mTLS. PR #11228 added -allowInsecureNoAuth
as an explicit opt-out for operators who restrict admin access some
other way (e.g. a NetworkPolicy plus an authenticating reverse proxy).

The chart's render-time guard added by #11236 (admin-statefulset.yaml,
seaweedfs.admin.authEnabled) predates -allowInsecureNoAuth and only
recognizes password-based auth, so there was no values.yaml path to
express that choice: the chart would fail(...) even though the binary
itself would start fine with a warning.

Add admin.allowInsecureBind (default false) to the seaweedfs.admin.authEnabled
helper's checks; when true it renders -allowInsecureNoAuth on the admin
command and satisfies the render guard alongside the existing
password-based checks.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-20 12:28:53 -07:00
Chris Lu 01bb3b3053 s3api: add Snowflake s3compat API integration tests (#11394) 
* s3api: add Snowflake s3compat API integration tests

Run the upstream snowflakedb/snowflake-s3compat-api-test-suite against a
local SeaweedFS server in CI. test/s3/snowflake/run.sh starts weed server
with S3 (-s3.autoCreateBucket=false so missing-bucket PUTs return
NoSuchBucket), prepares the fixtures the suite needs (versioned bucket,
deny-all-policy bucket, >1000-object prefix), clones the suite, patches
it to path-style addressing, and runs mvn -Dtest=S3CompatApiTest.

The suite also exposed that GetBucketLocation returned 404 NoSuchBucket
for a malformed bucket name; validate the name first and return
400 InvalidBucketName like AWS.

* test: harden snowflake s3compat runner per review

- Pin the upstream suite to a tested commit (SUITE_REV) instead of the
  moving default branch
- Bind the test server to loopback only
- Require the AccessDenied error code when verifying the denied bucket
- Fix README so go install runs in a subshell
- checkout with persist-credentials: false
- Make the concurrency group unique per PR, and widen path filters to
  the storage/operation/wdclient/cluster/pb packages the S3 stack uses

* test: advertise loopback ip for snowflake test server

-ip.bind 127.0.0.1 alone left the volume server advertising the host's
primary address, so chunk uploads were refused. Also set -ip 127.0.0.1
and disable the Iceberg/Lance listeners so the harness is loopback-only
and does not collide with other local services.
 2026-09-19 03:29:51 -07:00
Chris Lu 2d2619f0b4 ci: make telemetry deploy work on Oracle Linux 7 (#11377) 
* telemetry/server: tidy module dependencies

* ci: make telemetry deploy work on Oracle Linux 7

* ci: install telemetry unit and logrotate on every deploy

* ci: abort telemetry deploy on install failures
 2026-09-17 20:00:25 -07:00
Eliah Rusin 4fc9ada2ec ci: run seaweed-volume unit tests on Windows (#11349) 2026-09-16 08:06:21 -07:00
Eliah RusinClaude Fable 5.1Chris Lu
 		166af06a2b rust: cargo fmt both crates, with a commented-out fmt --check CI step (#11329) 
* rust: migrate seaweed-volume and seaweed-worker to tonic 0.14 / prost 0.14

tonic 0.14 boxes the contents of tonic::Status, which is what made every
RPC path trip clippy's result_large_err; the allow for that lint goes in
the next commit. The prost codec moved out of tonic into tonic-prost and
tonic-prost-build, so both build scripts now call
tonic_prost_build::configure() and both crates depend on tonic-prost for
the generated code. The `tls` feature was split into a per-backend
feature; `tls-aws-lc` is the same backend both crates already install
through rustls::crypto::aws_lc_rs.

tonic 0.14 depends on axum 0.8 and tower 0.5, which would have left a
second axum and a second tower in each tree next to the 0.7 / 0.4 the
crates named themselves. Bumping them keeps one copy of each: axum 0.8
only changes the path-parameter syntax for the routes here (`/:vid` ->
`/{vid}`, `/*path` -> `/{*path}`), tower 0.5 needs the `util` feature
named explicitly for ServiceExt::oneshot (it used to arrive through
tonic's feature unification), and tower-http 0.6 is the matching
release.

Lock files move only through cargo's own resolution for the new
versions; no other dependency was refreshed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust: drop the result_large_err allow now that tonic::Status is boxed

tonic 0.14 stores Status behind a Box, so Result<_, Status> is no longer
a large-Err type and clippy has nothing to say about it. Both crates
pass `cargo clippy --all-targets -- -D warnings` without the allow
(seaweed-volume in both feature sets), so the policy entry and its
comment go.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: drop the unused headers argument of try_expand_chunk_manifest

The parameter was already named `_headers`; nothing in the body reads it.
With it gone the function is under clippy's argument threshold and the
expect goes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: pass EC peer reads an EcInterval instead of ten arguments

fetch_one_interval, read_remote_ec_shard_interval,
do_read_remote_ec_shard_interval and recover_one_remote_ec_shard_interval
all took the same (vid, needle_id, shard_id, shard_offset, size,
expected_encode_ts_ns) tuple, and the two that reconstruct also took the
location map with the data/parity counts. Those are now EcInterval (Copy)
and EcShardMap (a borrow of the map plus the counts). The fan-out inside
recovery builds its per-shard request with `EcInterval { shard_id: sid,
..iv }`, which is the one place the old argument list was easy to get
wrong. Bodies destructure at the top, so the code below the signatures
is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: give the EC encoder an EcEncodeLayout and an EncodeRun

encode_dat_file took the Reed-Solomon shape and three block sizes as five
loose integers; they are now one Copy struct, EcEncodeLayout, which is
what Go calls ECContext. The per-row and per-batch helpers took the same
six sinks and the offsets; they become methods on EncodeRun, which owns
the borrows for one run, so each call names only the offset and block
size that vary. The byte-level work is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: describe a .dat rebuild with DatRebuild instead of nine arguments

write_dat_file_from_shards, its _with_dirs twin and the private
write_dat_file were three layers over one nine-argument signature. One
public function now takes a DatRebuild, whose shard_dirs is None when
every shard sits beside the .dat and Some(dirs) for the cross-disk
reconciled layout. The field docs carry what the function doc used to
say about the encode-time size and the block layout.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: split copy_file_from_source's fifteen arguments into two structs

CopyFileSpec is the per-file request (what to ask the source for, where
it lands, whether its bytes count as progress); CopyProgress is the
sender, throttler and report state that all three files of one
VolumeCopy share, held by &mut across the calls. The three production
call sites now read as the .dat/.idx/.vif literals they are, instead of
positional trues and falses.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: create volumes from a VolumeSpec

Volume::new, DiskLocation::create_volume and Store::add_volume each
took the same five-value tail of Go's NewVolume argument list:
collection, replica placement, TTL, preallocation and needle version.
That tail is now VolumeSpec, a Copy struct whose Default is what almost
every test wanted anyway (empty collection, no replication, no TTL, no
preallocation, current version), so most of the 104 call sites shrink
to `&VolumeSpec::default()` or name the one field they set. The id,
directories, index kind and disk type stay positional because they
differ at every site.

Two imports that only test modules use moved into those modules, and
DiskLocation no longer imports ReplicaPlacement.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-worker: run cargo fmt

Layout only; no token in the workspace changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust-volume: run cargo fmt

Layout only; no token in the crate changes. Every earlier Rust PR here
formatted only the blocks it touched so as not to drown its diff in
this one, and this commit is that debt paid in a single place. rustfmt
needed two passes to settle one block in handlers.rs; the committed
form is the fixed point, so `cargo fmt --check` is clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* ci: add a commented-out cargo fmt --check step to both Rust workflows

Same shape as the commented clippy step from #11312: the check is
written out so that making formatting a gate is a one-line uncomment,
and whether to do that stays a maintainer call.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2026-09-15 09:29:22 -07:00
Eliah RusinClaude Fable 5.1Chris Lu
 		adaf3534fa rust: clippy-clean both crates and adopt the std APIs the 1.91 MSRV allows (#11312) 
* rust: apply clippy --fix to both crates

The mechanical part of a clippy sweep: `cargo clippy --all-targets --fix`
on seaweed-volume and the seaweed-worker workspace, hand-reviewed. Both
manifests declare their MSRV (1.91.1 and 1.94.1), so every suggestion
clippy applied is within it: the collapsible_if sites become let chains
(1.88, edition 2024), `% n == 0` becomes is_multiple_of (1.87),
chunks_exact with a constant becomes as_chunks (1.88), repeat().take()
becomes repeat_n (1.82), and io::Error::new(Other, ..) becomes
io::Error::other (1.74). The rest is redundant clones, borrows, casts,
closures and field names.

Nothing here changes behaviour. The three let_and_return sites in
needle_map.rs and store_ec.rs deserve a note: the `let result = ..;
result` shape was a deliberate edition-2021 workaround to drop a redb
guard before the table it borrows. Edition 2024 drops tail-expression
temporaries before locals, which is why clippy now flags it, and the
two comments that described the workaround say so instead.

Manual edits on top of the tool output: the blocks clippy rewrote are
re-indented the way rustfmt lays them out (only those blocks — the
crate is not rustfmt-clean and a whole-crate fmt would bury this diff),
the blank lines let_and_return left behind are removed, and the CRC
legacy_value test compares against a literal worked out from the
original shift formula rather than restating rotate_right.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust: clear the clippy warnings --fix cannot apply, and say why the rest stay

Hand fixes for the lints clippy only reports. Behaviour is unchanged
throughout; each rewrite is the one clippy names.

- needless_range_loop (7): index loops over shard vectors become
  iterator loops. Where the old code indexed `v[..n]` the new loop
  iterates `v[..n]` so an undersized vector still panics the same way.
- field_reassign_with_default (6): struct literals with `..Default`.
- redundant_pattern_matching (3): `if let Err(_) = guard.check()` becomes
  `.is_err()`, which also releases the read guard at the end of the
  condition instead of at the end of the block.
- manual_strip (2), manual_checked_ops, format_in_format_args,
  redundant_locals, wrong_self_convention (to_vif takes self by value,
  so it is into_vif; CompactEntry is Copy, so to_needle_value takes self).
- type_complexity (2): `OrphanShardLoad` and `RawNeedleEntry` name two
  tuples that were spelled out inline.
- new_without_default: CompactNeedleMap gets a Default that calls new().
- suspicious_open_options: a test helper spells out `.truncate(false)`,
  which is what `.create(true).write(true)` already did.

What stays, and the attribute that says so:

- too_many_arguments (10): `#[expect]` on each function. Folding 8–15
  parameters into a struct is a design change, not a lint fix.
- await_holding_lock / readonly_write_lock: one test holds the store
  write guard across a sleep on purpose, as a barrier that parks the
  copy task at the mount block. `#[expect(.., reason = ..)]` records it.
- module_inception: needle/needle.rs mirrors the Go package layout.

Two lints become crate-wide policy in `[lints.clippy]`, with the reason
next to each: result_large_err, because every RPC path returns
tonic::Status (176 bytes) and boxing it would change every handler
signature; and needless_update, because `..Default::default()` on a
protobuf message literal is what lets a proto gain a field without
touching every constructor (all 11 sites are pb messages). The worker
workspace gets the same table and its members opt in with
`lints.workspace = true`; its generated plugin.rs also allows
large_enum_variant on prost's oneof enums.

Both crates are now clean under `cargo clippy --all-targets -- -D warnings`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust volume: use the std APIs the 1.91 MSRV already pays for

The crate declares rust-version 1.91.1, so a few things the code still
worked around are plain std now. All of them come from the 1.85–1.91
release notes; nothing here needs a newer toolchain than the manifest
already requires.

- std::sync::LazyLock (1.80) replaces the lazy_static! block in
  metrics.rs, and the lazy_static dependency goes. Every use site reads
  the same through Deref, so no caller changes.
- Duration::from_mins / from_hours (1.91) replace `from_secs(v * 60)`
  and `from_secs(v * 3600)` in the option parser and the shard-location
  refresh TTLs. One difference for the parser: an absurd count that
  overflows u64 seconds now panics in release builds too, where the
  multiplication used to wrap.
- Result::flatten (1.89) replaces `.and_then(|r| r)` on the replication
  join handle.
- OsStr::display (1.87) replaces `to_string_lossy()` where the name was
  only being formatted; the output is byte-identical.
- `#[allow]` becomes `#[expect]` (1.81) on the suppressions that are
  meant to be permanent, so a suppression that stops being needed
  becomes a warning rather than lingering. Doing that found four that
  already had: dead_code on ChunkManifest, base_name and last_io_error,
  and too_many_arguments on read_from_data_shards, which is down to
  seven parameters. Those attributes are deleted. The three allows that
  depend on cfg (a unix-only mutation, a linux-only field set, a
  profiling-only parameter) stay as allow, because expect would be
  unfulfilled on the other platforms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* ci: add a commented-out clippy step to both Rust workflows

Both crates are warning-free under `cargo clippy --all-targets
-D warnings` now. Whether that becomes a gate is a policy call, so the
step is present but commented out; uncommenting it is the whole change.
The comment points at the `[lints.clippy]` table where crate-wide
exceptions are recorded, so the gate does not become a reason to
sprinkle allows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CjZY429aVU74SLDmo1wiuU

* rust volume: guard parse_duration against overflow panics

Duration::from_mins/from_hours panic when the count overflows u64
seconds. Use checked_mul so an oversized CLI value falls back to the
parser default instead of crashing volume startup.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
 2026-09-14 11:29:29 -07:00
dependabot[bot] fd33c07843 build(deps): bump github/codeql-action from 4.37.9 to 4.38.0 (#11311) 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.37.9 to 4.38.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.37.9...v4.38.0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-14 09:41:36 -07:00
Chris Lu f1f6886d0e fix: rebase on latest master before pushing star history chart (#11240) 
* fix: rebase on latest master before pushing star history chart

The daily star history workflow git push was rejected with a
non-fast-forward error because new commits landed on master between
the checkout and the push. Fetch full history (fetch-depth: 0) and
rebase the generated commit on top of the latest remote branch before
pushing so the workflow no longer fails when master has moved.

* fix: retry rebase-and-push to handle concurrent master updates

Address review feedback: a one-shot rebase still races if master advances
between the rebase and the push. Match the bounded retry loop used by
java_release.yml — push first, and on rejection rebase and retry up to
five times before failing.

* fix: serialize runs and ensure every rebase is followed by a push

Address review feedback:
- Devin (line 54-55): the old loop rebased after the 5th failed push but
  never pushed the rebased commit. Restructure so each rebase (attempts
  2-5) is followed by a push attempt, with a clear 5-attempt cap.
- Greptile: overlapping runs could conflict on the SVG during rebase.
  Add a concurrency group (cancel-in-progress: true, matching the repo
  convention) so only one chart regeneration runs at a time.

* fix: scope concurrency by ref and guard rebase against transient failures

Address review feedback:
- Devin (line 16): the global star-history concurrency group let a manual
  run on another branch cancel an in-flight daily master update. Scope
  the group by github.ref so only same-branch runs cancel each other.
- Greptile (line 58): git pull --rebase runs under the fail-fast shell,
  so a transient fetch error or conflict aborted the whole step before
  remaining attempts ran. Guard the rebase so a failure aborts the
  in-progress rebase and continues to the next attempt instead.
 2026-09-08 21:43:24 -07:00
Chris Lu 0ce5ca42ea helm: supply admin auth in CI renders that enable admin (#11239) 
PR #11236 added a render-time guard that fails the chart when admin.ip
is non-loopback (default 0.0.0.0) and admin auth is not configured,
since weed admin 4.46 refuses to bind a non-loopback address without
authentication. Several pre-existing helm_ci.yml test cases enable
admin.enabled=true as part of "everything on" renders without a
password, so helm template now exits non-zero and the Verify template
rendering step fails.

Add admin.secret.adminPassword to the four render calls that turn on
admin without auth (IAM gRPC opt-in, NetworkPolicy EVERYTHING, egress
without kubeApiServer.cidrs, and the license ALL_ON dict), using the
same key ci/admin-values.yaml already uses.
 2026-09-08 20:51:53 -07:00
Chris LuandCopilot 213f4c5d5c release: wait for Go proxy propagation (#11219) 
Allow normal post-tag proxy propagation before dispatching downstream releases, while preserving the check that prevents them from pinning the previous commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-09-07 21:38:49 -07:00
dependabot[bot] 34936c610d build(deps): bump helm/kind-action from 1.14.0 to 1.15.0 (#11212) 
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/helm/kind-action/releases)
- [Commits](https://github.com/helm/kind-action/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: helm/kind-action
  dependency-version: 1.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-07 13:27:09 -07:00
dependabot[bot] 5e3af51d91 build(deps): bump docker/setup-qemu-action from 4.2.0 to 4.3.0 (#11213) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.2.0 to 4.3.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-07 13:27:01 -07:00
Eliah Rusin ed5b342f0c rust volume: optional redb insert_before bulk load (#11205) 
* rust volume: quick-repair redb on durable checkpoints

set_quick_repair(true) on the durable checkpoint transaction so an
OOM-killed volume server opens without a full-file repair scan.

* rust volume: optional redb insert_before bulk load

Behind redb-experimental-cursor (default off). Production binary
stays on sorted insert(). CI unit tests run both feature settings.

* rust volume: exercise insert_before across leaf splits

Replace the 5-key cfg clone with a 4000-key reverse-order rebuild
so CursorMut::insert_before hits page splits. CI runs the feature
only on storage::needle_map unit tests.
 2026-09-07 13:15:47 -07:00
Chris Lu d8a40ef750 ci: pin actions/setup-python to v7 in star_history workflow (#11191) 
The star_history workflow referenced actions/setup-python@v8, which
does not exist, causing the workflow to fail at the "Set up job" step.
Pin to v7, matching the version used across the other workflows.
 2026-09-05 21:44:12 -07:00
Chris Lu ff0d5a9adf ci(mount-windows): clean up processes and don't let Logs step fail job (#11174) 
The "Mount and exercise" step left the weed.exe mini server and the final
WinFsp mount running when it exited. The next step's pwsh.exe then failed
with STATUS_DLL_INIT_FAILED (0xC0000142), failing a job whose actual test
step had passed. The same code passed on both the PR branch and the next
master run, so this was a transient launch failure — but it was caused by
an unclean environment and made fatal by a diagnostic step.

Tear down all weed.exe processes at the end of the test step so subsequent
steps launch into a clean environment, and mark the Logs step
continue-on-error so a diagnostic step can never fail the job on its own.
 2026-09-05 00:10:40 -07:00
Chris Lu 567578f08d docs(readme): replace star-history.com with self-generated chart (#11171) 
* docs(readme): replace star-history.com with self-generated chart

The star-history.com SVG is a third-party dependency that can rate
limit or go down. Replace it with a GitHub Action that fetches
stargazers via the REST API and renders an SVG with matplotlib,
committing note/star_history.svg weekly. The README references the
committed file directly, so the chart has no runtime dependency on
any external service.

* ci(star-history): run daily instead of weekly
 2026-09-04 23:45:50 -07:00
Chris Lu 5a515adab2 s3: HeadObject with partNumber returns the part's size and 206 (#11166) 
* s3: HEAD with partNumber reports the part's size and range

HeadObject set its headers from the total object size and then only
validated the partNumber, so a client probing part 1 with HEAD got the
whole object's Content-Length and a 200 while the same GET returned the
part's size, a Content-Range and a 206.

Resolve the part's byte range before the headers are written, through the
range logic GetObject already used, and answer a partNumber HEAD as the
ranged HEAD that AWS documents.

* s3: answer an unsatisfiable partNumber with 416 InvalidPartNumber

GET and HEAD rejected a partNumber past the number of parts with 400
InvalidPart, the code for a missing part in CompleteMultipartUpload. AWS
answers a read of a part that does not exist with 416 InvalidPartNumber,
which lets a client probing for the part count tell the two apart.

The ceph suite pins RGW's 400 InvalidPart here, so the s3tests jobs patch
that expectation the way they already patch prefix ordering.

* s3: keep the whole-object checksum off a partNumber response

The stored checksum covers the whole object, so it is already withheld
from a ranged read. A partNumber HEAD now describes one part while the
request carries no Range header, so exclude it there too rather than
handing a client a checksum that does not match the bytes described.

* s3: resolve a partNumber against the parts the object records

Completion accepts ascending, not consecutive, part numbers, so the part
count is not the highest part number. Comparing the two rejected an
uploaded part 3 of a two-part object, and let a request for the absent
part 2 fall through to the positional chunk lookup and serve part 3's
bytes. Ask the recorded boundaries for the part instead, and keep the
count comparison for objects written before boundaries were stored.

* s3: apply a client Range within the part on HEAD too

GET narrowed the part by a Range sent alongside partNumber; HEAD reported
the whole part, so the two disagreed again for a request that carries
both. Move the narrowing into the shared range lookup so either verb
describes the same bytes.
 2026-09-04 23:10:41 -07:00
Chris Lu 1205630c43 ci: track the actions in the signing jobs by tag again (#11137) 
The repository tracks actions by tag with dependabot; the signing jobs
follow the same convention.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
 2026-09-03 12:36:36 -07:00
Chris Lu a8f763e717 Sign the published Docker images with cosign (#11129) 
* ci: composite action that signs and verifies an image with cosign

Keyless, by digest, with a verification pass against the calling workflow's
own identity right after signing. Signatures use the .sig tag layout rather
than the OCI-referrer bundle cosign 3 writes by default, since that is what
the verifiers people run today read. Dependabot is pointed at the action so
the cosign-installer pin keeps moving.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker release: sign every variant on both registries

The merge job signs each variant's multi-arch index on GHCR and Docker Hub
once the tag exists, recursively so the platform images are covered too.
latest re-tags the same manifest and inherits the signature.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker dev: sign the dev image

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker latest: sign a latest rebuilt by hand

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker release: sign the foundationdb image

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker: sign the per-version foundationdb and rocksdb builds

They push to the same repository as the releases, so an admission policy
that verifies chrislusf/seaweedfs would otherwise reject them.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker: document image signature verification

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* ci: pin the actions the signing jobs newly run by commit

These run with registry credentials and the OIDC token that signs under
the repository's identity, so a retargeted tag upstream must not be able
to reach them.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker latest: pass the dispatch tag through env, not the script

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker: complete Kyverno policy, digest note, identity scope

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker latest: keep the dispatch tag out of the manifest script too

The step predates signing, but the job now holds the OIDC identity.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* ci: pin every action in the jobs that sign

The jobs that hold the OIDC identity run these with registry credentials,
so a retargeted tag upstream must not reach them.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker release: copy and sign the digest the run created, pin the rest

crane copy and the signature both resolved the tag, which another
publisher could move between the two steps. The index digest is read once,
right after it is created, and the Docker Hub copy and both signatures use
it. The manual latest rebuild gets the same treatment. The actions in these
jobs are pinned to commits, crane to v0.22.0 by checksum, and the sparse
checkout no longer keeps the token.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker latest: the signing job checks out the workflow's own commit

The job only assembles and signs manifests, so nothing there needs the
source_ref checkout; the local signing action now comes from the same
revision as the workflow file that calls it.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa

* docker release: take the index digest from the create result

imagetools create writes the descriptor it pushed with --metadata-file
(buildx 0.32+, the runners ship 0.36), so the digest no longer comes from
re-resolving the tag even within the same step.

Claude-Session: https://claude.ai/code/session_01A5zMqzaUg1Snur4Yg8xJGa
 2026-09-03 11:52:10 -07:00
Chris Lu 59916d8978 helm: values-driven labels on every ingress (#11127) 
* helm: values-driven labels on every ingress

Each ingress already takes annotations from values, but its labels were
a fixed block, so tools that select ingresses by label (ExternalDNS
label filters, for one) had nothing to key on. Every ingress block now
has a labels map rendered after the standard app.kubernetes.io labels,
including the Traefik IngressRouteTCP that shares the filer gRPC values.

Claude-Session: https://claude.ai/code/session_01L6eJGXtYkwe1W9QjGeUgr1

* helm ci: render check for ingress labels

Claude-Session: https://claude.ai/code/session_01L6eJGXtYkwe1W9QjGeUgr1
 2026-09-03 09:17:27 -07:00