* security: require go 1.26.6 and bump vulnerable deps
A dependency scan of the 4.47 release flagged the bundled toolchain
and modules:
- github.com/golang/go < 1.26.6 (CVE-2026-39821, CVE-2026-56853,
CVE-2026-56859, CVE-2026-56862, CVE-2026-56864, CVE-2026-56865,
CVE-2026-33818, CVE-2026-46600): raise the go directive to 1.26.6
so every built artifact requires the fixed toolchain.
- google.golang.org/grpc (CVE-2026-84445, CVE-2026-84304): move to
the fixed dev pseudo-version; released tags through v1.85.0-dev
remain in the affected range.
- github.com/pelletier/go-toml/v2 <= v2.4.2 (unbounded parser
recursion): v2.4.3.
- alpine libcrypto3/libssl3 < 3.5.8-r0 (CVE-2026-75803,
CVE-2026-63073, CVE-2026-63075, CVE-2026-63076, CVE-2026-63072,
CVE-2026-54874, CVE-2026-18798, CVE-2026-14456, CVE-2026-14457):
the release images already apk-upgrade the final stage; extend the
same to the telemetry and admin-integration images.
Same bumps applied to the test/kafka, test/sftp,
kafka-client-loadtest, and telemetry/server modules.
* telemetry: send integration test report above the 10 GiB floor
The collect endpoint keeps reports only when TotalDiskBytes >=
proto.MinDiskBytes, but the integration test still sent 1 GiB, so the
server counted the report and skipped storing it. No cluster_id series
was ever created and /metrics lacked seaweedfs_telemetry_volume_servers.
Send just above the floor (via proto.MinDiskBytes so it cannot silently
drift again) so the expected per-cluster metrics are exported.
* build(deps): bump github.com/redis/go-redis/v9 from 9.21.0 to 9.22.0
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.21.0 to 9.22.0.
- [Release notes](https://github.com/redis/go-redis/releases)
- [Changelog](https://github.com/redis/go-redis/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/redis/go-redis/compare/v9.21.0...v9.22.0)
---
updated-dependencies:
- dependency-name: github.com/redis/go-redis/v9
dependency-version: 9.22.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* test(redis_conf): track go-redis 9.22.0 default read timeout of 5s
go-redis 9.22.0 raised the default ReadTimeout from 3s to 5s (part of the
cross-SDK configuration alignment). Update TestUnsetKeepsGoRedisDefaults to
expect the new default so the bump in #11306 stops failing CI.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
MasterServer.Shutdown only stopped the Hashicorp raft implementation;
when using the default goraft backend, the raft event-loop goroutine
(leaderLoop/followerLoop) kept running after the master shut down. In
the in-process test harness this leaked goroutines across sequential
test runs, and a stale event occasionally reached a leader at term 0
and tripped the goraft "leader.elected.at.same.term" assertion,
crashing the whole test binary (CI run 34670959967, PR 11279).
Stop the goraft server in Shutdown() so its goroutines exit cleanly,
and bump seaweedfs/raft to v1.2.1 which replaces that assertion with a
graceful step-down to Follower instead of a panic.
* build(deps): bump github.com/twmb/avro from 1.7.2 to 1.8.0
Bumps [github.com/twmb/avro](https://github.com/twmb/avro) from 1.7.2 to 1.8.0.
- [Commits](https://github.com/twmb/avro/compare/v1.7.2...v1.8.0)
---
updated-dependencies:
- dependency-name: github.com/twmb/avro
dependency-version: 1.8.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* iceberg: adapt to twmb/avro v1.8.0 and iceberg-go defensive copies
avro v1.8.0 changes Schema.Root() to return *SchemaNode, which breaks
iceberg-go v0.6.0's internal avro_schemas.go. The fix (apache/iceberg-go#1843)
is only on iceberg-go's main branch, unreleased, so bump iceberg-go to
that commit (c210509) alongside the avro bump.
That iceberg-go revision also changes two behaviors seaweedfs worked
around:
- It now infers a manifest list's format version from the embedded
writer schema, so a list missing the "format-version" header entry
(DuckDB's shape is read as v2, not v1. ReadManifestList's header
patching is now a redundant safety net; tests updated to expect v2.
- It returns defensive copies from DataFile.Partition(), so the
ReadManifest shim's in-place partition normalization was silently
discarded. Rebuild the entry through NewDataFileBuilder when any
partition value is normalized, copying every other DataFile field so
manifest round-trips are preserved.
- It converts day-transform partitions to iceberg.Date on read
(applyDayTransformDates), so the day-partition cases the shim and
tests guarded now convert without help; tests updated to expect
iceberg.Date from the raw read.
EOF
)
* iceberg: accept assert-ref-snapshot-id without snapshot-id
iceberg-go's new nullableInt64 parser rejects an assert-ref-snapshot-id
requirement whose "snapshot-id" field is absent from the JSON, even
though the Iceberg REST spec makes it optional (null means the ref must
not already exist). v0.6.0 used a plain *int64, so absent was nil and
accepted. ClickHouse sends the requirement without snapshot-id when
asserting a branch does not yet exist, so its writes fail with
"missing required field \"snapshot-id\"".
normalizeRequirements splices an explicit null into any
assert-ref-snapshot-id requirement missing the field before handing
the JSON to iceberg-go's parser, restoring the v0.6.0 behavior across
both iceberg-go versions.
* iceberg: fix v1 block_size_in_bytes default in rebuilt manifest entries
rebuildManifestEntry set block_size_in_bytes to 0, but the v1 manifest
schema requires the default of 64 MiB ("Always write default in v1").
The original value is not exposed on the DataFile interface, so use the
spec default. Also clarify the fallback comment to note that empty
(zero-record / zero-byte) files also trigger it, not just a nil spec.
Add a round-trip test that writes a rebuilt entry as v1 and verifies
block_size_in_bytes is 64 MiB via Avro decoding.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
golang.org/x/image v0.44.0 is affected by CVE-2026-46603 (GO-2026-6222):
a denial of service via excessive memory allocation when decoding
malformed VP8L (lossless WebP) data. It is fixed in v0.45.0, released
2026-08-11.
The decoder is reachable from SeaweedFS: weed/images/resizing.go
blank-imports golang.org/x/image/webp, which registers the VP8L decoder
with image.Decode, so the filer image resizing path decodes attacker
supplied WebP data with the affected version.
This is a go.mod/go.sum only change produced by
`go get golang.org/x/image@v0.45.0 && go mod tidy`; no other dependency
moved. `go build ./weed/`, `go vet ./weed/images/...`,
`go test ./weed/images/...` and `go mod verify` all pass.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* iceberg: normalize foreign day partitions during manifest rewrite
* test: cover manifest rewrite with foreign day partitions
* iceberg: restore every foreign partition value, not just day transforms
iceberg-go takes a partition field's logical type from the last branch of
its Avro union, so a writer that spells an optional partition [<type>, null]
rather than [null, <type>] leaves the value as whatever the Avro decoder
produced. A day or date partition then arrives as a time.Time the manifest
writer cannot encode, and a time partition is worse: time.Duration converts
to int64 nanoseconds and silently records the wrong value.
ReadManifest sits next to ReadManifestList, the other shim for what foreign
writers put on the wire, and converts each partition value back to the
Iceberg representation for its field type.
Claude-Session: https://claude.ai/code/session_01FdQyRuWF9SuCnPn21iH9yR
* iceberg: read manifests that carry partition values through the shim
Compaction, delete rewrite and their detection passes read entries and write
the same partition values back into new manifests, so they fail on a foreign
day partition exactly as manifest rewrite does. Where filters see it too:
literalMatchesActual falls through to fmt.Sprint, so a time.Time renders as a
timestamp and never matches the day the user asked for.
The two remaining readers, orphan collection and the admin preview, only look
at file paths and stay on iceberg.ReadManifest.
Claude-Session: https://claude.ai/code/session_01FdQyRuWF9SuCnPn21iH9yR
* iceberg: convert partition values before the writer rebinds logical types
Dimonyga checked the manifests of a live Doris table: every input spells the
partition union null-first, with the date logical type present, so the union
ordering is not what breaks the merge.
The conversion is lazy. iceberg-go converts what the Avro decoder returned on
the first Partition() call, using the logical types read from the manifest
being parsed, and ManifestWriter.addEntry rebinds them to the manifest it is
about to write before it makes that call. A day partition is where the two
disagree -- iceberg-go's day transform reports an int32 result type, so the
manifest it writes carries no date logical type at all -- and an entry nobody
looked at in between converts against that and keeps its time.Time.
That is why only rewrite_manifests failed: compaction and delete rewrite group
entries by partitionKey(df.Partition()) first, which converts them, and a where
filter does the same. Reading every entry's partition here converts them all
while the manifest's own logical types are still in place.
Claude-Session: https://claude.ai/code/session_01FdQyRuWF9SuCnPn21iH9yR
---------
Co-authored-by: Chris Lu <chris.lu@gmail.com>
The replace pinned thrift to a post-v0.23.0 master commit so 32-bit
GOARCHes would compile. That fix shipped in v0.24.0, so the replace was
only overriding the v0.24.0 require back to the pseudo-version and
holding the build below the CVE-2026-43871 fix.
Claude-Session: https://claude.ai/code/session_01C5BpSeYD3yULWmfVXwPRmB
* deps: upgrade parquet-go to v0.32.0
Iceberg compaction writes the merged file with the schema of its first
input, encodings included. parquet-go before v0.31.0 took the deprecated
PLAIN_DICTIONARY encoding that DuckDB writes at face value and encoded
those pages as plain int32 indices, but the spec gives PLAIN_DICTIONARY
the same bit-width-prefixed RLE layout as RLE_DICTIONARY. Every
dictionary-encoded column in a compacted file then decoded onto a single
dictionary entry, and anything past one page failed to decode at all.
* iceberg: cover compaction of dictionary-encoded input
The fixture is a DuckDB-written file, so it carries the PLAIN_DICTIONARY
encoding a Go writer will not produce.
* iceberg: tally whole rows in the dictionary merge test
Counting each column on its own passes a merge that remaps names while
leaving their cardinality intact.
* topology: digest the volumes a master believes each node holds
A volume server resends its whole volume list every heartbeat because that list
is the only way the master can notice a volume that vanished without a delta.
A digest gives the master the same guarantee without the list: the two ends
agree iff the master's copy is current.
VolumeInfo.ReportHash covers every field of VolumeInformationMessage, so a
change the hash misses is a change the master would never hear about. Both ends
run it over the same converted VolumeInfo, so they cannot drift apart.
Disk keeps the xor of its volumes' hashes, which is order-independent and its
own inverse, so add, update and remove each stay O(1) and the running value
needs no per-volume storage.
Nothing reads the digest yet; the heartbeat protocol change comes next.
* topology: test that a changed-volumes-only heartbeat reconciles
The digest is not a change detector -- in a live cluster some volumes always
have changed. It answers whether the master holds what the volume server holds
once the heartbeat's own changes are applied, so reporting three volumes out of
fifty has to reconcile while a volume lost without a delta must not.
* topology: digest the lookup index too, not just the disk maps
The reported digest answers whether the master holds what the volume server
holds. It cannot answer whether the master can serve those volumes: the disk
map and the lookup index are maintained separately, and a disconnect racing a
reconnect drops a volume from the index while leaving it on the node. The
server's report is identical either way, so a digest built from the disk maps
alone matches while the volume answers 'volume id not found'.
Track a second digest over volume ids on both sides of that split, so the
master can see its own indexes disagree without the volume server's help, and
without the O(volumes) scan the full heartbeat currently relies on.
* topology: exclude nodes reporting a duplicate volume id from the digest
A volume id can end up mounted on two disks of one server -- a stale twin
re-attached after a disk repair, which the store handles rather than rejects.
The server reports both copies with different disk ids, but the master keys
volumes by id alone within a disk type and keeps only the last one. Its digest
can then never equal the server's, and no amount of resending the full list
would fix it.
Detect it from the report itself, where deduplicating the ids already tells us
the count, and mark the node. A marked node has to keep sending full lists;
representing both copies is a separate question, and nesting the volume map by
disk id would cost more memory than the digest saves.
* topology: move the lookup digest with the entry, not the node passed in
Two volume servers can hold one address: GetOrCreateDataNode keys on the id a
server reports and refuses to merge a new id onto an address an older node
still claims, while the lookup list keys on address alone. Registering the
second server therefore displaces the first from the entry, and unregistering
through either removes whichever node the entry named.
Crediting the node handed to Set and Remove instead of the one actually
displaced or removed left the digest on the wrong node. A displaced node went
on reporting a consistent index while it could no longer serve the volume,
which is exactly the silent unavailability the digest exists to catch.
Set and Remove now return the node they displaced and removed, so ownership
can be transferred rather than assumed.
