Chris LuandClaude Opus 5.5 cdea7e79e0 filer: fold small chunks into manifests 256 at a time (#12462) ...

* filer: fold small chunks into manifests 256 at a time

Every metadata event carries the file's whole chunk list, so a file grown
by small appends re-sent each chunk record on every append until the
list reached ManifestBatch (10000). Chunks under 1 MiB now fold once 256
of them build up; full-size chunks still fold at ManifestBatch, so large
files keep their flat lists and read without a manifest fetch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: merge appended small chunks at the small-chunk fold count

The append path merges small chunks into full-size ones, but only at
1000 of them; the fold now hides small chunks in a manifest at 256, so
the merge would never see enough of them. Merging at the same count
keeps rewriting them into full chunks first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-06 13:41:02 -08:00

..

admin

s3: require their own actions to change a bucket's CORS or versioning (#12433)

2026-11-06 09:12:00 -08:00

cluster

vector: weed-vector registers to master; s3 discovers it there (#12146)

2026-10-28 00:58:31 -07:00

command

filer.remote.gateway: keep the remote object while a version under a delete marker is locked (#12459)

2026-11-06 13:37:08 -08:00

credential

plugin worker: stop forwarding the stream's admin token to job RPCs (#12391)

2026-11-04 05:26:12 -08:00

ec

weed/shell: resolve collection from topology when ec.decode is given only volumeId (#12424)

2026-11-05 05:01:25 -08:00

filer

filer: fold small chunks into manifests 256 at a time (#12462)

2026-11-06 13:41:02 -08:00

filer_client

fix: resolve gRPC DNS resolution issues in Kubernetes #8384 (#8387)

2026-02-19 15:46:02 -08:00

glog

go fmt

2026-04-10 17:31:14 -07:00

iam

s3: require their own actions to change a bucket's CORS or versioning (#12433)

2026-11-06 09:12:00 -08:00

iamapi

iam: optionally require an iam: grant for self-service access-key actions (#11937)

2026-10-21 20:06:30 -07:00

images

image: add an optional public image processing gateway (#11593)

2026-10-05 19:49:02 +08:00

kms

kms/azure: fix encrypt/decrypt round trip for Azure Key Vault (#11692)

2026-10-10 23:40:19 +08:00

mount

mount: refuse to attach opens to a draining file handle (#12219)

2026-10-29 16:06:57 -07:00

mq

mq/kafka: wait for a filer to register instead of panicking on an empty list (#12019)

2026-10-24 07:09:43 -07:00

notification

filer,pb,mount: keep per-chunk cipher keys out of logs (#12085)

2026-10-26 11:52:56 -07:00

operation

master: tell a warming-up Assign or LookupVolume client when warm-up ends (#12386)

2026-11-03 20:46:58 -08:00

pb

vector: keep ListVectors pages and segments flat as an index grows; read only keys in GetVectors (#12300)

2026-11-01 11:57:44 -08:00

placement

placement: let callers reject candidates placement cannot judge (#10581)

2026-08-05 00:18:22 -07:00

plugin/worker

plugin worker: stop forwarding the stream's admin token to job RPCs (#12391)

2026-11-04 05:26:12 -08:00

query

filer: fail the request, not the process, when chunks cannot be resolved (#11685)

2026-10-10 23:39:29 +08:00

remote_storage

filer.remote.sync: bounded filer lookups, one chunk cache per process (#12441)

2026-11-06 09:35:16 -08:00

replication

filer.sync: stop chunk retries with the same supersession check the skip uses (#12367)

2026-11-03 10:43:22 -08:00

s3api

s3api: mint a versioned PUT's id at commit, after the lock wait and the upload (#12454)

2026-11-06 13:20:36 -08:00

security

fix(s3): accept fresh admin tokens when context carries stale authorization (#12390) (#12397)

2026-11-04 09:42:25 -08:00

sequence

[nfs] Add NFS (#9067)

2026-04-14 20:48:24 -07:00

server

filer: fold small chunks into manifests 256 at a time (#12462)

2026-11-06 13:41:02 -08:00

sftpd

sftp: fail rm and rmdir of a missing path, and mkdir of an existing one (#12168)

2026-10-28 08:11:21 -07:00

shell

weed/shell: resolve collection from topology when ec.decode is given only volumeId (#12424)

2026-11-05 05:01:25 -08:00

static

adjust logo

2026-04-23 10:05:51 -07:00

stats

filer: a full remote-deletion tombstone table keeps the lazy fetch off (#12440)

2026-11-06 13:28:47 -08:00

storage

volume: stop serving EC reads from quarantined shards (#12450)

2026-11-06 09:38:15 -08:00

telemetry

telemetry: keep only clusters that store at least 10 GiB (#11138)

2026-09-03 12:36:00 -07:00

topology

admin: re-send the master vacuum disable every poll so it survives leader changes (#12430)

2026-11-06 09:07:50 -08:00

util

master: tell a warming-up Assign or LookupVolume client when warm-up ends (#12386)

2026-11-03 20:46:58 -08:00

wdclient

s3: drop retired volume servers from the gateway location cache (#12281)

2026-10-31 15:05:00 -07:00

worker

iceberg maintenance: cache a remote-only file before reading it (#12191)

2026-10-29 11:13:23 -07:00

Makefile

Move SQL engine and PostgreSQL server to their own binaries (#8417)

2026-02-23 16:27:08 -08:00

weed.go

shell: exit non-zero when a piped command fails (#11117)

2026-09-02 22:12:51 -07:00