mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-10 00:20:42 +02:00
fd4fa72289ef09f9012c1dbdfeda16dc5e646ab3
15095
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris Lu
|
fd4fa72289 |
helm: pass -ip to admin so StatefulSet becomes Ready on 4.46 (#11236)
* helm: pass -ip to admin so StatefulSet becomes Ready on 4.46 Since weed admin 4.46 changed its default listen address from all interfaces to loopback (127.0.0.1), the admin StatefulSet template never passed -ip, so the admin server bound to loopback only. The chart's httpGet readiness/liveness probes dial the pod IP, not loopback, so the probes never succeeded and the admin StatefulSet stayed 0/1 forever — breaking upgrades with helm --wait or GitOps controllers. Add an admin.ip value (default "0.0.0.0", restoring the pre-4.46 behaviour) and render it as -ip. A non-loopback bind requires authentication, so fail at render time when admin.ip is non-loopback and neither admin.secret.adminPassword nor admin.secret.existingSecret is set, instead of letting the pod crash-loop. Document the value and add a chart-testing CI values file. Bumps chart to 4.46.1. Fixes #11234 * helm: address review feedback on admin bind validation Align the chart's loopback classification with weed admin's isLoopbackIp (net.ParseIP + IsLoopback): the whole 127.0.0.0/8 range and ::1 are loopback; localhost and wildcard addresses are non-loopback, matching the binary. Previously the exact-string check rejected valid loopback addresses like 127.0.0.2 while permitting localhost (which the binary treats as non-loopback). Recognize WEED_ADMIN_PASSWORD supplied via admin.extraEnvironmentVars / admin.secretExtraEnvironmentVars as authentication, since weed admin picks it up through viper's AutomaticEnv. Previously such deployments were wrongly rejected at render time. Remove [https.admin] mTLS from the validation message and docs: the chart only generates [grpc.admin] (gRPC mTLS), not [https.admin] (HTTP mTLS), so mentioning it as an alternative was misleading. Document that the -ip flag requires SeaweedFS 4.46 or newer, so pinning admin.imageOverride to an older image is not supported with this chart. Extracted the loopback and auth checks into reusable helpers (seaweedfs.admin.isLoopbackIp, seaweedfs.admin.authEnabled) following the existing seaweedfs.filer.mysqlEnabled pattern. * helm: tighten loopback classification to reject malformed 127.x addresses Use regexMatch instead of hasPrefix for the IPv4 loopback check so malformed values like "127.not-an-ip" are not accepted as loopback (net.ParseIP returns nil for them, so weed admin treats them as non-loopback). Also recognize the expanded IPv6 loopback form "0:0:0:0:0:0:0:1" in addition to "::1", matching net.ParseIP behavior for the two common representations. |
||
|
Chris Lu
|
cb9fcd39d2 |
filer/postgres: create filemeta table on startup via createTable config (#11229)
* filer/postgres: create default filemeta table on startup The postgres filer store hardcoded CreateTableSqlTemplate to empty and never created the filemeta table, unlike postgres2/mysql2/sqlite which all create it during Initialize. Users had to create the table manually or the filer would crash loop with "relation filemeta does not exist". Read the createTable config option (same as postgres2), default to DefaultCreateTableQuery when unset, and execute CREATE TABLE IF NOT EXISTS on the default table after the connection pool is established. SupportBucketTable stays false so per-bucket table creation remains a no-op; only the shared filemeta table is created, via a direct ExecContext since AbstractSqlStore.CreateTable short-circuits without bucket support. * filer/postgres: accept boolean createTable = true/false viper reads a TOML boolean as the string "true"/"false" via GetString, so createTable = true was being used as a SQL template and failed. Add ResolveCreateTableQuery to normalize the value: true and empty select the default template, false disables table creation, anything else is a custom template. Both postgres and postgres2 now use it, and both skip the CREATE TABLE call when the resolved template is empty. * scaffold: document createTable option for postgres filer store Replace the commented-out CREATE TABLE SQL in the [postgres] scaffold with a createTable config hint, matching the [postgres2] section. Users no longer need to manually create the filemeta table before starting the filer. * filer/postgres: make createTable opt-in for postgres, keep postgres2 default The previous commit defaulted postgres to create the filemeta table even when createTable was unset, which could break existing deployments whose DB user lacks CREATE TABLE privileges. ResolveCreateTableQuery now returns empty for an unset value so postgres only creates the table when createTable is explicitly true or a custom template — preserving the prior no-DDL behaviour for existing configurations. postgres2 keeps its existing always-create default: it defaults an empty resolved value to DefaultCreateTableQuery, and only skips when createTable is explicitly false. * filer/postgres2: simplify createTable handling, document all modes Drop the false opt-out from postgres2 — it only skipped the default table while per-bucket CreateTable still ran, leaving restricted DB roles broken on bucket access. postgres2 now accepts true the same way (defaulting to DefaultCreateTableQuery) and keeps its existing always-create behaviour for every other value, matching the original semantics. The scaffold comment now documents true/false/custom for the postgres section so users know false (or unset) is the backward-compatible default. * filer/postgres2: normalize false via ResolveCreateTableQuery postgres2 only handled "" and "true", leaving createTable = false as the literal string "false" which CreateTable then executed as invalid SQL. Route it through ResolveCreateTableQuery (which maps false to empty) and default the empty result to DefaultCreateTableQuery, so false is treated the same as unset for the bucket-aware store. * filer/postgres2: honor createTable = false for default table postgres2 treated false the same as unset and always created the default filemeta table, failing startup for restricted DB roles that explicitly opted out. Track the original false value before ResolveCreateTableQuery collapses it to empty, and skip the default CreateTable call when set. Per-bucket table creation is unaffected — it is a runtime requirement of the bucket-aware store. Users who need to suppress all DDL should use the postgres (non-bucket) store with createTable unset. * filer/postgres2: disable bucket tables when createTable = false Setting SupportBucketTable = false when createTable is explicitly false makes AbstractSqlStore.CreateTable a no-op (it already returns nil when SupportBucketTable is false), so neither the default filemeta table nor per-bucket tables are created. The template stays empty and no DDL runs, honouring the opt-out for restricted DB roles. All data routes to the pre-provisioned filemeta table, matching the postgres (non-bucket) store. * filer: suppress DDL without disabling bucket routing Setting SupportBucketTable = false when createTable = false also disabled per-bucket routing, hiding objects in pre-provisioned per-bucket tables. Keep SupportBucketTable true and instead skip the CREATE TABLE execution when the resolved template is empty. GetSqlCreateTable now returns empty for both postgres and mysql SQL generators when CreateTableSqlTemplate is empty, and AbstractSqlStore.CreateTable skips the ExecContext call when the SQL is empty. This preserves bucket routing while suppressing all DDL for users who explicitly set createTable = false and pre-provision their tables. * filer: add SkipDDL to suppress CREATE and DROP without disabling routing createTable = false with SupportBucketTable = true preserved bucket routing but deleteTable still executed DROP TABLE on bucket deletion, dropping externally managed tables. CanDropWholeBucket also returned true, so the S3 layer tried whole-table drops instead of row-by-row deletes. Add a SkipDDL flag to AbstractSqlStore, independent of SupportBucketTable. CreateTable and deleteTable both skip when SkipDDL is set, and CanDropWholeBucket returns false so bucket deletion falls back to row-by-row metadata deletes. postgres2 sets SkipDDL when createTable is explicitly false — bucket routing is preserved, no DDL runs. * filer: fall back to row-by-row delete when CanDropWholeBucket is false DeleteFolderChildren took the whole-table drop path whenever the path was a bucket root, even when SkipDDL made deleteTable a no-op. The no-op returned nil, the caller returned early, and rows inserted after the recursive enumeration survived the bucket deletion. Gate the whole-table drop on CanDropWholeBucket so the row-by-row DeleteFolderChildren SQL runs when SkipDDL is set, removing all metadata without issuing DROP TABLE. |
||
|
Chris Lu
|
8782749f26 |
admin: reject IAM policy deletion while still attached to a user/group (#11230)
* admin: add IsPolicyAttached helper to detect user/group attachments Introduces AdminServer.IsPolicyAttached, which lists the users and groups that still have a managed policy attached, reusing the existing credential manager ListUsers / ListAttachedUserPolicies / ListGroups / GetGroup methods. This is the building block for rejecting policy deletion while a policy is still referenced, so deleted policy names stop lingering in a user attached policy names list (issue #11225). * admin: reject IAM policy deletion while still attached Guards AdminServer.DeletePolicy with the new IsPolicyAttached check and returns the typed ErrPolicyStillAttached error when the policy is still referenced by a user or group. This matches AWS IAM and the existing IAM API handler behavior, fixing the stale reference where a deleted policy name kept showing up in a user attached policy names list (issue #11225). * admin: return 409 Conflict when deleting an attached IAM policy The admin UI DeletePolicy handler now maps ErrPolicyStillAttached to HTTP 409 Conflict instead of 500, so the dashboard can surface the attachment conflict to the user rather than reporting a generic server error. * admin: skip vanished groups when checking policy attachments IsPolicyAttached now treats a group that disappears between ListGroups and GetGroup (credential.ErrGroupNotFound) as no longer attached instead of failing the whole deletion with HTTP 500, matching the IAM API handler which skips vanished groups. * test: assert policy state after deletion paths Strengthen GetPolicy assertions in the policy deletion tests to check the returned policy is non-nil after a rejected deletion and nil after a successful one, not just that no lookup error occurred (GetPolicy returns nil, nil when a policy is absent). |
||
|
Chris Lu
|
b88156fe6b |
fix(s3api): evaluate aws:SourceIp from the direct TCP peer, not forwarded headers (#11231)
* fix(s3api): use direct peer IP for aws:SourceIp in bucket policy engine extractSourceIP in the bucket-policy engine trusted X-Forwarded-For and X-Real-Ip whenever the TCP peer looked private (loopback/RFC1918/link-local), with no configurable trusted-proxy allowlist. In containerized deployments the gateway peer is almost always private, so any caller reaching it directly or from a co-located workload could spoof aws:SourceIp and bypass IpAddress/NotIpAddress bucket-policy restrictions. Always return the direct peer address (r.RemoteAddr), matching AWS S3 semantics. Remove the now-unused isPrivateIP helper and header-trust branch. Update TestExtractConditionValuesFromRequestSourceIPPrecedence to assert the peer IP is used regardless of forwarding headers, and add regression tests TestExtractSourceIP_IgnoresForwardedHeaders and TestExtractSourceIP_EnforcesIPRestrictionPolicy. * fix(s3api): use direct peer IP for aws:SourceIp in IAM role/session policies The IAM middleware's extractSourceIP trusted X-Forwarded-For and X-Real-IP whenever the TCP peer looked private (loopback/RFC1918/link-local), with no configurable trusted-proxy allowlist. In containerized deployments the gateway peer is almost always private, so any caller reaching it directly or from a co-located workload could spoof aws:SourceIp and bypass IpAddress/NotIpAddress conditions on role and session policies (IsPrincipalActionExplicitlyDenied). Always return the direct peer address (r.RemoteAddr), matching AWS S3 semantics. Remove the now-unused isPrivateIP helper, privateNetworks table, and its init(). Update TestRequestContextExtraction and TestIPBasedPolicyEnforcement to assert the peer IP is enforced regardless of forwarding headers, and add regression test TestUserInlinePolicySourceIpCondition_IgnoresForwardedHeaders. |
||
|
Chris Lu
|
557fffa350 |
iam: preserve native Admin when IAM policies are attached (#11226) (#11232)
* iam: expose tri-state result from attached policy evaluation evaluateIAMPolicies returned a bool that collapsed explicit Deny and no-match into a single false, so the authorization path could not tell "policies forbid this" from "policies say nothing". Introduce evaluateAttachedIAMPolicies returning Allow/Deny/NoMatch and keep evaluateIAMPolicies as a bool projection for existing callers. This is preparation for unioning native permissions with attached policies while preserving deny-always-wins. * iam: preserve native Admin when IAM policies are attached Attaching an IAM policy routed authorization exclusively to the attached policies, dropping the identity native permissions. A user with native Admin lost all access after attaching a non-granting policy, and stayed locked out if that policy was deleted without being detached first (#11226). Treat a native bare Admin grant as a permission floor that survives attached policies: when the attached policies do not explicitly allow, fall back to isAdmin() on the attached-policy path, and on the IAM integration path allow unless an attached policy explicitly denies. Explicit Deny still wins on both paths. Only bare Admin is consulted because inline policies flatten lossily into Actions (dropping conditions), so scoped actions are not unambiguously native and must keep flowing through the policy engine. * iam: regression tests for native Admin surviving attached policies Reproduces issue #11226: - TestNativeAdminSurvivesAttachedPolicy: a user with native Admin keeps Write access after attaching a policy that does not grant it. - TestNativeAdminSurvivesDeletedPolicy: the same user keeps Write access after the attached policy is deleted without being detached. - TestAttachedPolicyExplicitDenyOverridesNativeAdmin: an explicit Deny in an attached policy still constrains a native admin (deny-always-wins). * iam: apply native Admin floor before IAM principal validation The native Admin floor in authorizeWithIAM ran after the auth-path switch, which denies when no session principal or PrincipalArn is present. An Admin identity without a PrincipalArn (no session token) was therefore denied before the floor executed. Move the floor ahead of the switch and derive the principal for its explicit-deny check with buildPrincipalARN, which already handles identities without a PrincipalArn. Adds a regression case for an Admin identity with an empty PrincipalArn. Addresses CodeRabbit review feedback on PR #11232. |
||
|
Chris Lu
|
4a1d65939f |
fix(mount): bound reader cache memory across open files (#11220)
* fix(filer): bound retained reader cache buffers by bytes * test(filer): keep in-flight downloads during cache trimming * feat(mem): expose pooled allocation capacity for byte reservations * fix(mount): share a configurable reader buffer budget across files * fix(filer): release failed prefetch slots and memory reservations * feat(mount): expose a soft Go runtime memory limit * docs(filer): restore shared-download rationale in startCaching The one-line comment replacing the original context.Background() explanation was too thin for readChunkAt to cross-reference shared resource semantics. Restore a concise note on why request cancellation must not abort a download shared by concurrent readers. * test(filer): loosen reader cache test deadlines to 5s Three tests used 1-second deadlines that can flake on CI under load: TestReaderCacheBudgetInFlight, TestReaderCacheEvictionDoesNotHoldCacheLock, and TestReaderCacheFailedPrefetchReleasesBudget. Increase to 5 seconds. * test(filer): cover re-read after reader cache eviction Add TestReaderCacheReReadAfterEviction: reads chunk 'a', reads chunk 'b' (evicting 'a' via budget pressure), then re-reads 'a' and asserts a fresh download returns correct data. Verifies the core correctness property that eviction never exposes missing or stale data to readers. |
||
|
ssshr-66
|
c6b330be2b |
[Mount] Add ChunkGroup seeking tests and fix boundary handling (#11223)
* fix issue 11221 * reply ai comments |
||
|
Chris LuandCopilot
|
213f4c5d5c |
release: wait for Go proxy propagation (#11219)
Allow normal post-tag proxy propagation before dispatching downstream releases, while preserving the check that prevents them from pinning the previous commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
github-actions[bot]
|
d997fba157 | 4.46 4.46 | ||
|
Chris Lu
|
c0a7dbb2bb |
iam: bind CreateServiceAccount ParentUser to the caller (#11218)
* iam: bind CreateServiceAccount target to caller in AuthorizeIamAction A non-admin holding iam:CreateServiceAccount could pass an arbitrary ParentUser and mint a service account for any identity, inheriting that identity permissions. Add a self-target category so a granted non-admin may only target their own identity; admins remain unrestricted. * iam: authorize CreateServiceAccount against its ParentUser target AuthIamManagement passed UserName as the authorization target for every action, so CreateServiceAccount was authorized with an empty target and the self-target binding never saw the caller-supplied ParentUser. Pass ParentUser for that action so the binding takes effect on the live path. * iam: test CreateServiceAccount binds target to caller Regression test: a non-admin holding iam:CreateServiceAccount may target itself but is denied targeting another identity; admins remain unrestricted. * iam: authorize CreateServiceAccount against ParentUser on the S3 port UnifiedPostHandler passed UserName as the authorization target for every IAM action, so CreateServiceAccount was authorized with an empty target on the S3-port route and the self-target binding never saw the caller ParentUser. Extract iamTargetUserName (ParentUser for CreateServiceAccount, UserName otherwise) and use it from both IAM dispatch surfaces so the binding applies on the live S3-port path as well as the standalone iam server. * iam: test CreateServiceAccount ParentUser binding on the S3 port End-to-end regression test through UnifiedPostHandler: a non-admin holding iam:CreateServiceAccount is denied (403) when targeting another identity and passes authorization when targeting itself. |
||
|
ssshr-66andChris Lu
|
361fd6b263 |
[Filer] Parallelize Chunk Manifest Resolution to Reduce Large File Read Latency (#11215)
* fix issue-11214 * fix(filer): cancel sibling manifest reads on failure * fix(filer): scope manifest cancellation to read batch and propagate context to encrypted reads Address PR review comments on #11215: - Scope cancellation to each parallel read batch instead of the resolver-wide context, so a later manifest failure does not cancel recursive work for an earlier successful manifest (CodeRabbit #3952446554). - Propagate the resolver context through GetAuthenticatedWithContext so encrypted sibling reads observe cancellation and stop promptly when another manifest fails (Greptile #3952422343). - Use net.ListenConfig.Listen with an explicit context in the test fixture to satisfy the noctx linter (CodeRabbit #3952111696). - Add regression tests for encrypted sibling cancellation and for preserving earlier manifest children on later failure. * fix(filer): return known manifest errors without blocking on earlier children Address Greptile review comment on #11215: After all parallel reads complete, pre-scan slots for the first real (non-internal-cancel) error before recursing into earlier manifests' children. If a later manifest already failed, return its error promptly with data chunks already in hand, instead of blocking on recursive network reads of earlier manifests' children. Updated the regression test to verify the error returns within 1 second when an earlier manifest's child has a 2-second delay, and that the child is never loaded. * fix(filer): filter partial child chunks by requested range on error path Address CodeRabbit review comment on #11215: The pre-scan error path appended non-manifest child chunks from earlier manifests without applying the [startOffset, stopOffset) overlap check used for top-level chunks. A child outside the requested range could be returned in dataChunks alongside the later manifest's error. Apply the same range predicate before appending. Add regression test with an out-of-range child chunk. * fix(filer): buffer job channel and abort submission on batch cancellation Address Greptile review comment on #11215: - Use a buffered job channel (capacity 128) so submission does not block when all workers are busy. This ensures a promptly-failing manifest is always queued and can cancel stalled sibling reads once a worker picks it up, instead of blocking the caller on the unbuffered channel send. - Add batchCtx.Done() to the submit select so submission aborts promptly when the batch is already cancelled by a sibling failure. - Add regression test with 5 manifests (4 stalled + 1 failing) verifying the failing job is queued and picked up after a stalled worker is freed. * fix(filer): avoid double WaitGroup decrement on batch cancellation in submit Address Devin review comment on #11215: When batchCtx.Done() fired in submit, it called job.done.Done() and returned false. The caller in resolve also called reads.Done() on the same WaitGroup, causing a double decrement that would panic with a negative counter. Fix: submit sets the result error but does not decrement the WaitGroup. The caller always owns the decrement and skips overwriting the result when submit already set it. * fix(filer): overflow execution when job queue buffer is full Address Greptile follow-up review comment on #11215: With a 128-entry buffer, if more than 132 in-range manifests (4 workers + 128 buffer) stall at one level, a promptly-failing manifest beyond the buffer cannot be submitted and cannot cancel the stalled reads. Fix: when the buffer is full, run the job directly in a goroutine instead of blocking on the channel send. This only triggers for >132 manifests at one level (exceedingly rare), so the bounded concurrency guarantee (4 workers) holds for all normal workloads. Extracted executeJob method shared by both workers and overflow goroutines. * fix(filer): bound overflow execution with a semaphore Address Devin review comment on #11215: The unbounded overflow goroutines could create thousands of concurrent reads for large files, defeating the four-worker resource bound. Fix: add a semaphore (capacity = maxChunkManifestResolveWorkers) that overflow goroutines must acquire before doing the read. While waiting for the semaphore, they also watch batchCtx and r.ctx so they exit promptly on cancellation. Total concurrency is now bounded to 2 * workers (4 workers + 4 overflow) in the degenerate case. * refactor(http): add ctx to GetAuthenticated signature instead of new function Reuse the existing GetAuthenticated name by adding ctx as the first parameter, matching the pattern of ReadUrl, ReadUrlAsStream, and RetriedFetchChunkData. Removes the GetAuthenticatedWithContext wrapper. --------- Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Chris Lu
|
75ec5ec193 |
admin: allow setting volume read-only and read/write modes (#11217)
* admin: support setting volume read-only and read/write modes
* admin: address PR review on volume access-mode persistence
Reject trailing JSON values in the SetVolumeReadOnly handler so
requests like {"read_only":true}{} no longer pass validation, and add
a trailing-value case to the invalid-request test.
Propagate .vif persistence failures through the access-mode chain.
PersistReadOnly now returns the SaveVolumeInfo error and rolls back
the in-memory volumeInfo on failure; Store.MarkVolumeReadonly and
Store.MarkVolumeWritable propagate that error and roll back their
noWrite flags, so the API reports failure instead of success while
restart would revert the mode.
* admin: make .vif persistence atomic and preserve error chain
SaveVolumeInfo now writes to a .vif.tmp file, syncs it, renames it
over the target, and fsyncs the directory. A write/sync/close failure
leaves the existing .vif intact, so the PersistReadOnly in-memory
rollback matches the durable state instead of diverging from a
partially written file that restart would apply.
Switch the error wrappers in PersistReadOnly, MarkVolumeReadonly, and
MarkVolumeWritable from %v to %w so callers can use errors.Is and
errors.As to classify persistence failures.
* admin: treat post-rename dir fsync failure as a warning
After os.Rename commits the new .vif, the on-disk file already holds
the requested mode. A directory fsync failure only risks losing the
rename across a crash; returning an error here would make
PersistReadOnly roll back in-memory state while the durable file keeps
the new mode, splitting the replica. Log the failure as a warning
instead, matching the best-effort nature of FsyncDir (already skipped
on Windows).
* admin: distinguish post-rename durability failures and use unique temp files
SaveVolumeInfo now uses os.CreateTemp for the staging file, preventing
concurrent saves for the same volume from colliding on a shared .tmp
path.
A directory fsync failure after os.Rename returns a
NotCrashDurableError instead of being silently swallowed. The rename
already committed the new metadata to disk, so PersistReadOnly,
MarkVolumeReadonly, and MarkVolumeWritable skip the in-memory rollback
for this error type (keeping state aligned with the durable file) while
still propagating the failure to the API. Pre-commit failures continue
to roll back as before.
* admin: continue post-commit work after NotCrashDurableError
MarkVolumeWritable now clears the EIO quarantine and the gRPC handlers
(makeVolumeReadonly step 3, makeVolumeWritable master notification)
proceed with their post-commit work when SaveVolumeInfo returns a
NotCrashDurableError, instead of aborting and leaving the volume
unavailable or the master unaware of the mode change. The durability
warning is still propagated to the API caller. Pre-commit failures
continue to abort early as before.
* admin: handle NotCrashDurableError in tier and EC callers
VolumeTierMoveDatFromRemote and VolumeEcShardsGenerate now check for
NotCrashDurableError from SaveVolumeInfo. When the rename has already
committed the new .vif, they continue with their post-commit work
(backend switch, remote deletion, keeping generated EC shards) instead
of aborting and leaving the on-disk metadata inconsistent with the
file layout. The durability warning is logged for the operator.
|
||
|
github-actions[bot]
|
53a18ecadd | docs: regenerate star history chart | ||
|
dependabot[bot]andChris Lu
|
2d4b730a2f |
build(deps): bump github.com/twmb/avro from 1.7.2 to 1.8.0 (#11210)
* build(deps): bump github.com/twmb/avro from 1.7.2 to 1.8.0 Bumps [github.com/twmb/avro](https://github.com/twmb/avro) from 1.7.2 to 1.8.0. - [Commits](https://github.com/twmb/avro/compare/v1.7.2...v1.8.0) --- updated-dependencies: - dependency-name: github.com/twmb/avro dependency-version: 1.8.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * iceberg: adapt to twmb/avro v1.8.0 and iceberg-go defensive copies avro v1.8.0 changes Schema.Root() to return *SchemaNode, which breaks iceberg-go v0.6.0's internal avro_schemas.go. The fix (apache/iceberg-go#1843) is only on iceberg-go's main branch, unreleased, so bump iceberg-go to that commit (c210509) alongside the avro bump. That iceberg-go revision also changes two behaviors seaweedfs worked around: - It now infers a manifest list's format version from the embedded writer schema, so a list missing the "format-version" header entry (DuckDB's shape is read as v2, not v1. ReadManifestList's header patching is now a redundant safety net; tests updated to expect v2. - It returns defensive copies from DataFile.Partition(), so the ReadManifest shim's in-place partition normalization was silently discarded. Rebuild the entry through NewDataFileBuilder when any partition value is normalized, copying every other DataFile field so manifest round-trips are preserved. - It converts day-transform partitions to iceberg.Date on read (applyDayTransformDates), so the day-partition cases the shim and tests guarded now convert without help; tests updated to expect iceberg.Date from the raw read. EOF ) * iceberg: accept assert-ref-snapshot-id without snapshot-id iceberg-go's new nullableInt64 parser rejects an assert-ref-snapshot-id requirement whose "snapshot-id" field is absent from the JSON, even though the Iceberg REST spec makes it optional (null means the ref must not already exist). v0.6.0 used a plain *int64, so absent was nil and accepted. ClickHouse sends the requirement without snapshot-id when asserting a branch does not yet exist, so its writes fail with "missing required field \"snapshot-id\"". normalizeRequirements splices an explicit null into any assert-ref-snapshot-id requirement missing the field before handing the JSON to iceberg-go's parser, restoring the v0.6.0 behavior across both iceberg-go versions. * iceberg: fix v1 block_size_in_bytes default in rebuilt manifest entries rebuildManifestEntry set block_size_in_bytes to 0, but the v1 manifest schema requires the default of 64 MiB ("Always write default in v1"). The original value is not exposed on the DataFile interface, so use the spec default. Also clarify the fallback comment to note that empty (zero-record / zero-byte) files also trigger it, not just a nil spec. Add a round-trip test that writes a rebuilt entry as v1 and verifies block_size_in_bytes is 64 MiB via Avro decoding. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
|
005012edcf |
rust volume: quick-repair redb on durable checkpoints (#11203)
* rust volume: insert redb rebuilds in needle-id order Unlink the .rdb before create (create does not truncate). Collapse last-write-wins, then insert live keys sorted so 4.2.0 packs leaves. * rust volume: rebuild redb from a BTreeMap and clear leftover keys Peak rebuild memory is one ordered map instead of HashMap + Vec + stable-sort scratch. Unlink stays best-effort: if it fails, retain clears the leftover table before sorted insert. Compute idx metrics before the write so a read error does not unlink a committed .rdb. * rust volume: drop the extra redb read transaction on put/delete put uses insert()'s previous value. delete gets then inserts the tombstone in the same write transaction. Truncate the .idx row on any failed redb write after the append. * rust volume: unpack redb blobs through packed_to_needle_value save_to_idx, ascending_visit, and collect_entries used the same length-check copy as get. Route them through the helper so a wrong-length value is absent everywhere, not a panic. * rust volume: quick-repair redb on durable checkpoints set_quick_repair(true) on the durable checkpoint transaction so an OOM-killed volume server opens without a full-file repair scan. * rust volume: reopen redb from .idx on non-poisoned commit error redb 4.2.0 can make a Durability::None commit visible before returning Err(CommitError::Storage(..)). In that state the database refuses further write transactions, so truncating the .idx row (the old behavior) would leave a redb-only put or tombstone that the stored idx_size makes the reload skip. Distinguish CommitError::TransactionPoisoned (txn rolled back, db still usable -- truncate the orphan .idx row as before) from other commit errors (change may be visible, db refuses writes -- keep the .idx row, close the database, and reopen from .idx to repair redb's internal state). db becomes Option<Database> so reopen_from_idx can drop the old file lock before load_from_idx opens the same path. rdb_path, version, and cache_bytes are stored so the reopen uses the same configuration. * rust volume: truncate .idx row when redb is closed in put put appends the .idx entry before acquiring the write transaction. When db_or_err() fails (db is None after a failed reopen), the ? returned without calling truncate_idx_to_offset, so a write reported failed remained in the authoritative .idx and was replayed on restart. Handle db_or_err() explicitly and truncate the orphan .idx row before returning the error, matching the existing handling for begin_write, open_table, and insert failures. --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Chris Lu
|
5d5ea18287 |
topology: fix fatal concurrent map read/write on VolumeLayout.crowded (#11216)
SetVolumeCrowded mutated the crowded map under accessLock.RLock(), while GetWritableVolumeCount reads the same map under RLock() on the Assign hot path. Two concurrent RLock holders with one writing and one reading the map triggers a fatal "concurrent map read and map write" that kills the master process (unrecoverable, bypasses recover). Take the write lock in SetVolumeCrowded instead. This event path is a low-frequency single consumer driven by the crowded-volume event loop, and every other mutation of crowded already holds Lock(); setVolumeCrowded takes no nested locks, so there is no deadlock path. The hot readers (GetWritableVolumeCount, CloneWritableVolumes) keep using RLock. Adds a -race regression test that fails (race detected) on the old RLock and passes with the write lock. Fixes #11211 |
||
|
dependabot[bot]
|
34936c610d |
build(deps): bump helm/kind-action from 1.14.0 to 1.15.0 (#11212)
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.14.0 to 1.15.0. - [Release notes](https://github.com/helm/kind-action/releases) - [Commits](https://github.com/helm/kind-action/compare/v1.14.0...v1.15.0) --- updated-dependencies: - dependency-name: helm/kind-action dependency-version: 1.15.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
5e3af51d91 |
build(deps): bump docker/setup-qemu-action from 4.2.0 to 4.3.0 (#11213)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.2.0 to 4.3.0. - [Release notes](https://github.com/docker/setup-qemu-action/releases) - [Commits](https://github.com/docker/setup-qemu-action/compare/v4.2.0...v4.3.0) --- updated-dependencies: - dependency-name: docker/setup-qemu-action dependency-version: 4.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Junker der ProvinzandJunker der Provinz
|
a4885b7975 |
log_buffer: stop closing a notification channel another reader still holds (#11177)
* fix(log_buffer): stop closing a notification channel another reader still holds - #10810 The report blames the polling loop for the busy spin, but that loop is not what burns the core. LogBuffer keeps one notification channel per subscriberID, and UnregisterSubscriber closes it. Two registrations that share a subscriberID share that channel, which happens whenever a client opens a second stream or an old stream has not yet noticed it was replaced, so the first unregister closes a channel the other reader is parked on. A closed channel makes every receive in awaitNotificationOrTimeoutFor return instantly, and that reader then spins at full speed for the rest of its life. Subscriptions are now reference counted. Registering an existing subscriberID hands back the same channel and raises the count; the channel is only closed when the last holder unregisters. * test: fail if the surviving reader stops instead of keeps reading Review caught that the iteration count alone proves nothing: had LoopProcessLogData returned when the duplicate reader unregistered, the counter would sit at 0 and the assertion would pass without a reader ever having been there to spin. Check the reader is still running before trusting its low count. --------- Co-authored-by: Junker der Provinz <jdp@braethoria.com> |
||
|
dependabot[bot]
|
e0f9e02761 |
build(deps): bump github.com/prometheus/client_model from 0.6.2 to 0.6.3 (#11208)
Bumps [github.com/prometheus/client_model](https://github.com/prometheus/client_model) from 0.6.2 to 0.6.3. - [Release notes](https://github.com/prometheus/client_model/releases) - [Commits](https://github.com/prometheus/client_model/compare/v0.6.2...v0.6.3) --- updated-dependencies: - dependency-name: github.com/prometheus/client_model dependency-version: 0.6.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
3f8cc380fb |
build(deps): bump cloud.google.com/go/storage from 1.64.0 to 1.67.0 (#11206)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.64.0 to 1.67.0. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/compute/v1.64.0...compute/v1.67.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/storage dependency-version: 1.67.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
3eba8ebb9b |
build(deps): bump github.com/go-ldap/ldap/v3 from 3.4.13 to 3.4.14 (#11209)
Bumps [github.com/go-ldap/ldap/v3](https://github.com/go-ldap/ldap) from 3.4.13 to 3.4.14. - [Release notes](https://github.com/go-ldap/ldap/releases) - [Commits](https://github.com/go-ldap/ldap/compare/v3.4.13...v3.4.14) --- updated-dependencies: - dependency-name: github.com/go-ldap/ldap/v3 dependency-version: 3.4.14 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Eliah Rusin
|
ed5b342f0c |
rust volume: optional redb insert_before bulk load (#11205)
* rust volume: quick-repair redb on durable checkpoints set_quick_repair(true) on the durable checkpoint transaction so an OOM-killed volume server opens without a full-file repair scan. * rust volume: optional redb insert_before bulk load Behind redb-experimental-cursor (default off). Production binary stays on sorted insert(). CI unit tests run both feature settings. * rust volume: exercise insert_before across leaf splits Replace the 5-key cfg clone with a 4000-key reverse-order rebuild so CursorMut::insert_before hits page splits. CI runs the feature only on storage::needle_map unit tests. |
||
|
dependabot[bot]
|
99b84eeb88 |
build(deps): bump github.com/getsentry/sentry-go from 0.48.0 to 0.49.0 (#11207)
Bumps [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) from 0.48.0 to 0.49.0. - [Release notes](https://github.com/getsentry/sentry-go/releases) - [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md) - [Commits](https://github.com/getsentry/sentry-go/compare/v0.48.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/getsentry/sentry-go dependency-version: 0.49.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Eliah RusinandChris Lu
|
b6690cfbc8 |
rust volume: drop the extra redb read transaction on put/delete (#11204)
* rust volume: insert redb rebuilds in needle-id order Unlink the .rdb before create (create does not truncate). Collapse last-write-wins, then insert live keys sorted so 4.2.0 packs leaves. * rust volume: rebuild redb from a BTreeMap and clear leftover keys Peak rebuild memory is one ordered map instead of HashMap + Vec + stable-sort scratch. Unlink stays best-effort: if it fails, retain clears the leftover table before sorted insert. Compute idx metrics before the write so a read error does not unlink a committed .rdb. * rust volume: drop the extra redb read transaction on put/delete put uses insert()'s previous value. delete gets then inserts the tombstone in the same write transaction. Truncate the .idx row on any failed redb write after the append. * rust volume: unpack redb blobs through packed_to_needle_value save_to_idx, ascending_visit, and collect_entries used the same length-check copy as get. Route them through the helper so a wrong-length value is absent everywhere, not a panic. --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> |
||
|
Eliah Rusin
|
94b10c006d |
rust volume: insert redb rebuilds in needle-id order (#11202)
* rust volume: insert redb rebuilds in needle-id order Unlink the .rdb before create (create does not truncate). Collapse last-write-wins, then insert live keys sorted so 4.2.0 packs leaves. * rust volume: rebuild redb from a BTreeMap and clear leftover keys Peak rebuild memory is one ordered map instead of HashMap + Vec + stable-sort scratch. Unlink stays best-effort: if it fails, retain clears the leftover table before sorted insert. Compute idx metrics before the write so a read error does not unlink a committed .rdb. |
||
|
Chris Lu
|
15e4da65f7 |
volume: avoid read-only replica write targets (#11195)
* master: carry replica read-only state in volume lookups * volume: refresh writable replica targets * volume: preserve read-only replicas for deletes * master: propagate read-only delete capability * volume: target delete-capable replicas * volume: honor configured HTTPS for replica deletes * volume: reject insecure delete authorization forwarding * master: broadcast delete capability changes * volume: align Rust replica routing * http: protect credentialed replica redirects * master: preserve digest compatibility for delete capability * volume: propagate read-only state in short heartbeats * volume: report changed short volume state * http: guard TLS client redirects * master: announce mounted volume read-only state * volume: replace changed identity deltas * master: replace incremental volume layouts in order * master: keep moved volume lookup available * volume: announce read-only mounts |
||
|
Tim Olowandtimolow
|
331c6c3642 |
shell: volume.check.disk — actionable verdict for diverged vacuumed replicas (#11197)
* shell: volume.check.disk — actionable verdict for diverged vacuumed replicas
When -resurrectMissingNeedles is gated off because both replicas have been
vacuumed (compaction revision > 0) — the normal state of any production
cluster — check.disk previously stopped at 'cannot prove they are missing
writes vs vacuumed deletes' and did nothing, leaving a diverged replica with
no repair path. volume.fix.replication does not catch it either: it only
acts when the replica COUNT is below the expected replication, never when
two replicas are both present but hold different live data.
Classify the divergence instead of dead-ending:
liveDivergence() counts live (non-deleted) needles present on one replica's
index but entirely absent from the other, in both directions. Tombstones
are excluded, so vacuum asymmetry (a compacted replica that dropped deleted
entries) is not mistaken for divergence.
reportDivergenceVerdict() turns the count into an operator action:
- one-sided (one replica has all the live data, the other has no unique
live needles) -> print the exact safe repair:
volume.copy -source <complete> -target <lagging> -volumeId <id>
Re-copying the complete replica is safe precisely because the lagging
side holds no unique live data; VolumeCopy's verify-before-destroy gate
independently confirms the source holds the volume before deleting the
target.
- two-sided (split-brain, both sides have unique live data) -> warn and
do NOT emit an auto repair; point to volume.fsck -findMissingChunksInFiler
to confirm the 'missing' needles are orphans before converging.
Report-only: no data is modified and the resurrection safety gate is
untouched. This is what lets a 13-volume diverged cluster be diagnosed and
repaired in minutes instead of by hand-diffing every index.
Observed motivating case: home SeaweedFS 4.45 cluster, 13 010 cross-rack
volumes diverged after failed replicate writes (ReplicatedWrite MaxAttempts=1
fire-and-forget), 11 one-sided + 2 two-sided, all repaired via volume.copy.
* shell: volume.check.disk — address review nits on divergence verdict
- Use pb.NewServerAddressFromDataNode (dialable ip:port, Address with Id
fallback) for the advertised volume.copy -source/-target instead of the
logical node Id, which may not be dialable.
- Make the one-sided verdict tombstone-aware: when the lagging replica has
been vacuumed, absent live needles may be valid deletions whose tombstones
were dropped, so a whole-volume re-copy would resurrect them. The command
is only advertised as safe when the lagging side is proven never-vacuumed
(compaction revision 0 read under -resurrectMissingNeedles); otherwise a
caveat is printed pointing at fsck/needle-level repair.
- Fix reversed copy direction when the source replica is the lagging one
(must copy complete -> lagging in both cases).
- Test: real tombstone (negative size) with the correct 0/0 expectation and
|| assertion; verdict test now covers dialable address, corrected
direction, and caveat on/off.
* shell: volume.check.disk — per-replica revision knowledge, no copy command for vacuumed lagging side
- Track srcRevKnown/tgtRevKnown separately: in unidirectional mode the
target revision IS read, so a proven never-vacuumed target no longer
gets a false resurrection warning (regression: bidi=false, target rev 0).
- A one-sided verdict now only emits the volume.copy command when the
lagging side is proven never-vacuumed; when it is vacuumed (or unproven)
the verdict refuses to print the destructive command and points at
fsck/needle-level repair instead — an appended caveat next to a ready-to-
paste copy command was still inviting the resurrection.
- deletionCaveat now returns the boolean safety decision.
* shell: volume.check.disk — preserve gRPC port, proven two-sided is not split-brain
- Emit the raw ServerAddress string (host:port.grpcPort) instead of
String()/ToHttpAddress(), which drops the custom gRPC port and would
make the suggested volume.copy dial the default port and fail.
- Two-sided divergence with both replicas proven never-vacuumed under
-resurrectMissingNeedles is mutually missed writes, not split-brain:
recommend re-running with -apply (in-place resurrection both
directions) instead of the split-brain no-auto-repair warning.
- Tests: case F (proven two-sided -> -apply, no split-brain warning),
case G (custom gRPC ports preserved in emitted addresses).
---------
Co-authored-by: timolow <tim@timolow.dev>
|
||
|
Chris Lu
|
3225d2b0ce |
rust worker: install rustls CryptoProvider to fix TLS panic (#11194) (#11196)
* rust worker: add install_default_crypto_provider helper lance's aws backend pulls aws-lc-rs and reqwest's rustls-tls pulls ring, so rustls 0.23 cannot auto-select a CryptoProvider and tonic's client TLS panics on first use. Add install_default_crypto_provider, pinning the default to aws-lc-rs, mirroring the Rust volume server's helper of the same name. Includes a regression test that builds a TLS channel and panics without the install in this crate, where both providers link. * rust worker: install the crypto provider at startup Call install_default_crypto_provider before any TLS use, the way the Rust volume server does in its main. Without this a worker started with --tls-ca/--tls-cert/--tls-key panics on the first admin dial (#11194). |
||
|
github-actions[bot]
|
8537d5bc08 | docs: regenerate star history chart | ||
|
Chris Lu
|
70a26cb5d2 |
s3: gate IAM-cache gRPC RPCs behind admin Bearer auth (#11190)
* s3: gate IAM-cache gRPC RPCs behind admin Bearer auth The SeaweedS3IamCacheServer registered on the S3 gateway's internal gRPC port (default 0.0.0.0:18333) accepted PutIdentity/RemoveIdentity/PutPolicy/ DeletePolicy/GetPolicy/ListPolicies/PutGroup/RemoveGroup with no per-RPC authentication. An unauthenticated network peer could call PutIdentity with Actions:[Admin] and write straight into the live accessKeyIdent map that the SigV4 path reads, bypassing S3 authentication entirely. Mirror the filer's IamGrpcServer.checkAdminAuth: require a Bearer token signed with jwt.filer_signing.key (read from the existing s3a.filerGuard) at the top of every IAM-cache RPC. With no key configured the check is a no-op, matching the rest of SeaweedFS's gRPC surface. * credential: attach admin Bearer token to S3 IAM-cache propagation The filer's PropagatingCredentialStore fans IAM mutations out to peer S3 servers over the SeaweedS3IamCache gRPC service. Now that the S3 handlers require a Bearer token signed with jwt.filer_signing.key, attach one to the outgoing propagation context (mirroring shell/iamAdminAuthContext). With no key configured it is a no-op, so deployments that run without the signing key keep working. * credential: mint IAM-cache admin token after master discovery propagateChange attached the admin Bearer token before ListClusterNodes, so master-client retries could run down the (default 10s) token lifetime before the peer S3 fan-out began, leaving peers to reject an expired token and IAM caches stale. Move withIamCacheAdminAuth to after discovery succeeds, immediately before the propagation timeout is derived. * credential: cap IAM-cache propagation timeout below JWT lifetime The propagation fan-out used a fixed 10s timeout. If an operator configures jwt.filer_signing.expires_after_seconds below 10, the admin token can expire while slower S3 peers are still being contacted, leaving their IAM caches stale. Derive the propagation deadline as min(10s, tokenTTL) so it never outlives the token. withIamCacheAdminAuth now returns the token's lifetime (0 = no expiry) for this purpose. |
||
|
Eliah RusinandClaude Opus 5
|
ade4bdf9e6 |
rust volume: stop a tier move whose caller has gone (#11192)
Both tier-move handlers run in a detached tokio::spawn and report progress through a closure that returns (), with the send result discarded. Nothing observes the caller leaving, so an abandoned move uploads or downloads the whole .dat anyway and then commits the transition. Go aborts both. Its progress callback returns `stream.Send`'s error, which surfaces out of the reader in s3_upload.go:99 and the writer in s3_download.go:84 and fails the transfer, so the volume info is never rewritten. The Rust port dropped that by typing the callback as FnMut(i64, f32) with no result. Give the callback Go's signature -- FnMut(i64, f32) -> Result<(), String> -- and abort when the caller's channel is closed. Checked on every part rather than only where progress is reported, since the report is rate-limited to one a second and would miss a caller that left in between. A merely full channel is a slow reader, not a departed one, so only TrySendError::Closed counts as cancellation. Two consequences of aborting mid-transfer that the old code never had to handle: - upload_file now aborts the multipart upload when the transfer fails. An abandoned multipart upload does not show up in an ordinary object listing but still accrues storage charges until a lifecycle rule reaps it, and cancellation makes that a routine path rather than a rare one. - The tier-down handler removes the partial .dat. download_file pre-allocates the destination to the object's full size, so an aborted download leaves a file of the right length and the wrong content -- and this handler refuses to run at all when a local .dat exists, so leaving one wedges every retry on "already on local disk" and a restart would load the sparse file as the volume's data. There is deliberately no check between a finished transfer and the bookkeeping that follows. Once the object is in S3, or the .dat is on disk, that bookkeeping is what makes the state consistent; stopping there would leave an object paid for and referenced by nothing, or a complete local .dat the volume still calls remote. Go does not gate there either -- its callback only runs during the transfer. Claude-Session: https://claude.ai/code/session_0122W3eqt6gmLUMxmRoZdPAb Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
4b41329e12 |
rust volume: stop a VolumeCopy whose caller has gone (#11188)
* rust volume: stop a VolumeCopy whose caller has gone VolumeCopy runs its copy in a detached tokio::spawn and reports progress with the send error discarded, so nothing observes the client leaving. When the caller cancels the RPC -- which weed-admin's batch balance does routinely, starting far more copies than it finishes -- the server streamed the whole volume from the source, wrote it to disk, and mounted it. The destination is then left holding a volume nobody took delivery of: its index cache is never reclaimed, and under replication=000 one volume id ends up on two servers, both writable, which concurrent writes can diverge. Three checks now reach the task: - Every chunk in copy_file_from_source, via the sender's is_closed(). This is the one that matters in practice. The first progress report is 128MB in, so for a smaller volume -- the ordinary balance move -- no send ever happens and its result says nothing; only the closed channel does. The sender is passed for the .idx and .vif copies too, with reporting gated separately, so those phases notice as well. - The throttle sleep, which for a throttled copy runs for seconds at a time, now races the sender's closed() instead of being slept through. - Immediately before mount_volume, and once at the top of the task. Cancellation surfaces as an ordinary Err(Status::cancelled), so it lands in the existing error branch that already removes the partial .dat/.idx/ .vif and the .note. That branch also logs now: the error otherwise went to a channel nobody was reading, leaving the operator with the balancer's "delete that copy, then re-run the move" and no cause. This also clears the stranded read-only sources reported on the issue. They are downstream of the orphan mount, not a separate defect: LiveMoveVolume's cleanup probes the target before undoing the freeze (volume_move.go:95, "the server can finish the copy and mount the target even when the client loses the stream"), and when it finds a mounted copy it cannot attribute, or cannot delete, it deliberately keeps the source readonly rather than risk two writable replicas -- the messages at volume_move.go:110 and :123. With nothing mounted on the target the probe reports clean and the freeze is undone. On Go parity: the progress send result is honoured here too, matching `return false` in volume_grpc_copy.go. But that report is Go's only abort signal, and measured against a 120MiB volume -- above the throttler's activation threshold, below the 128MiB report interval -- a Go destination mounts an abandoned copy as well. The issue's premise that Go aborts holds only above the report interval. The Rust side now stops in both cases; the Go behaviour is worth its own issue. Tests: the integration test runs against both implementations and is green on Go, red on Rust before this change. Its 192MiB fixture is sized for two separate constraints, documented at the fixture: IoBytePerSecond is a no-op below ~100ms of wall clock (64MiB copies in ~110ms on a tmpfs loopback cluster), and the payload must exceed the 128MiB report interval for the Go leg to pass at all. The two Rust unit tests cover what the integration test cannot reach: cancellation detected with no progress report at all, and the cleanup of the partial files plus the .note. Fixes #11186 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0122W3eqt6gmLUMxmRoZdPAb * rust volume: surface VolumeCopy cancellation as Status::cancelled copy_file_from_source returned Result<_, String>, so the per-chunk cancellation path -- the one that matters in practice for volumes below the 128MB report interval -- was wrapped to Status::internal at the call sites. The spawn logging branch then classified it as a generic failure instead of the intended "abandoned by caller", defeating the logging change in the same PR for the case that occurs most often. Return Result<_, Status> from copy_file_from_source: Status::cancelled for caller-gone, Status::internal for the existing errors. Drop the .map_err(|e| Status::internal(e)) at the three call sites. The unit test now asserts the code is Cancelled, not just the message text, so the classification is locked in. * rust volume: close cancellation gaps in VolumeCopy Address two review findings on the same PR: 1. Roll back a mount that races a departing caller. The pre-mount is_closed() check cannot close the window between the check and mount_volume: if the receiver drops in that gap, the volume mounts and the final tx.send(Ok(...)) fails, but its error was discarded (let _ =), so the task returned Ok(()) and the error branch never ran. The destination then held an orphaned mounted replica — the exact defect this PR prevents. Fix: track a mounted flag. The final send now checks its result; on failure it returns Status::cancelled, and the error branch calls store.delete_volume (which unmounts AND removes the files) when mounted is true, instead of only unlinking. 2. Observe cancellation while awaiting the source stream. The per-chunk is_closed() check only runs after stream.message().await returns. A stalled source (slow disk, partition, GC pause) never delivers a chunk, so a caller that has already left cannot preempt the read: the task, the source connection, and the partial files (including the .note) all outlive the caller indefinitely. Fix: race stream.message() against progress_tx.closed() in a tokio::select!, so a departing caller preempts a stalled source. Adds test_volume_copy_after_mount_cancellation_rolls_back_mount to cover the after-mount rollback path. cargo test --release green (497 + 5 + 1 + 28). * rust volume: keep remote data on after-mount rollback, race RPC startup Two review findings on the after-mount rollback added in |
||
|
|
b82cb05d71 |
rust volume: checkpoint the redb index durably every 1000 writes (#11182)
* rust volume: checkpoint the redb index durably every 1000 writes
Every put and delete on a redb-backed volume committed with
Durability::None and nothing ever committed durably, on the theory that
the .idx file is the source of truth. redb, however, keeps an entry in
its transaction tracker for every non-durable commit and cannot recycle
pages that were on disk at the last durable commit until a durable one
happens. With no durable commit for the life of the process, both grew
with every write, and .rdb files could bloat toward double size after a
restart (#11179, the hash-table rehash stacks in the memleak output).
The needle map now counts non-durable commits and reports when a
checkpoint is due; the volume takes it, data first: flush the .dat, then
the map fsyncs the .idx and commits redb durably, recording in the same
transaction how much of the .idx the table reflects. A checkpoint makes
the index durable, so the bytes it points at must be down before it, or
after a power loss the index would reference past the end of the .dat
and the volume would load read-only. A failed .dat flush skips the
checkpoint; it is retried on the next write.
Volume::close() now closes the needle map instead of only syncing it,
and the redb map's close() takes the same checkpoint. Before, a clean
shutdown left the table durable (redb flushes on drop) but the recorded
.idx size stale at its load-time value, so the next load replayed every
entry written since load on top of the counters.
On load, the redb map's counters now come from the whole .idx history,
the way Go's LevelDB map rebuilds them (newest entry first, with a bloom
filter of seen keys), instead of from the table's final state. Both the
reuse and the full-rebuild path use it, so overwritten and deleted bytes
keep counting as garbage across restarts, and the incremental replay of
the .idx tail only touches the table, which makes it idempotent whether
or not the table is ahead of the recorded .idx size.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019x36FiSeyePh77YXao15kK
* rust volume: skip redundant .idx fsync on checkpoint after flush_idx
On the fsync=true write path, flush_idx() already fsyncs the .idx before
maybe_checkpoint_index() runs, so the checkpoint's own sync() fsyncs the
same file a second time for nothing. Thread an idx_already_synced flag
from the volume through maybe_checkpoint_index into checkpoint(sync_idx):
when it is true the checkpoint skips its .idx fsync and only does the
durable redb commit. The delete path and close() still sync (they have
not flushed the .idx beforehand).
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: saturate writes_since_checkpoint to prevent u32 overflow
If checkpoints keep failing (e.g. a persistent .dat flush failure whose
error is not EIO and so does not mark the volume read-only), the counter
increments on every write with no upper bound and wraps at ~4.3 billion.
Use saturating_add so it pins at u32::MAX instead, which keeps
checkpoint_due() true and retries on every subsequent write.
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: only update max_file_key on live entries in idx metric rebuild
metrics_from_idx called maybe_set_max_file_key on every entry including
tombstones, but the live on_put path only calls it for puts and on_delete
never does. A tombstone always has a preceding put for the same key that
already set max_file_key, so the result is the same today; restricting it
to live entries makes the parity with the live path exact and self-evident.
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: advance idx_file_offset only after redb commit succeeds
put() and delete() appended to the .idx file and advanced idx_file_offset
before committing to redb. If the redb commit failed, the offset included
the orphan row that redb doesn't reflect. A later checkpoint would record
that offset as "the table reflects up to here," and the reload would skip
the orphan row entirely — the entry becomes permanently unindexed.
Move the idx_file_offset increment to after the successful redb commit.
The .idx file still has the orphan row (append-only), but idx_file_offset
stays behind it, so the next checkpoint records the smaller offset and
the reload replays the orphan row back into redb.
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: skip index checkpoint on close when .dat sync fails
Volume::close() discarded the .dat sync_all() result and always
checkpointed the redb index. If the .dat sync failed, the checkpoint
made the index durable with entries that may point past the unflushed
.dat tail, and a power loss would leave the volume read-only on reload
(the max_needle_end check fires).
Check the .dat sync result: on success, checkpoint as before; on
failure, call close_without_checkpoint() — sync the .idx and drop the
writer without a durable redb commit. META_IDX_SIZE stays at the last
successful checkpoint, so the reload replays the uncheckpointed tail
(redb still flushes on drop, but without recording idx_size).
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: schedule checkpoints on every index mutation path
maybe_checkpoint_index was only called from do_write_request and
do_delete_request. put_needle_index and write_needle_blob_and_index
also call NeedleMap::put, which increments writes_since_checkpoint,
but neither triggered the checkpoint. Through those paths the counter
could grow past the interval without ever being satisfied, leaving
non-durable redb transaction state until close().
Add maybe_checkpoint_index(false) after the successful nm.put in both
methods. The .dat flush inside maybe_checkpoint_index covers the blob
write in write_needle_blob_and_index; put_needle_index pairs with a
prior write_needle_blob, so the flush covers that too.
Co-Authored-By: Chris Lu <chris.lu@gmail.com>
* rust volume: truncate orphan .idx row on failed redb commit
Commit
|
||
|
Chris Lu
|
d8a40ef750 |
ci: pin actions/setup-python to v7 in star_history workflow (#11191)
The star_history workflow referenced actions/setup-python@v8, which does not exist, causing the workflow to fail at the "Set up job" step. Pin to v7, matching the version used across the other workflows. |
||
|
Chris Lu
|
e6f2386a0f |
admin: redact S3 secret keys for read-only sessions (#11189)
The Admin UI documents its read-only account as view-only and blocks its
write requests, but the authenticated read routes returned object-store
users with plaintext access and secret keys. A read-only admin user could
retrieve another user's live S3 credential pair from GET /api/users and
GET /api/users/{username} and use it directly against the S3 endpoint,
converting view-only access into the victim identity's object-store
authority.
Redact the reusable secret_key in GetUsers, GetUserDetails, and the
rendered users page whenever the requesting session has the read-only
role. The public access_key identifier is retained so identities remain
browsable; only the reusable secret is stripped. Admin and no-auth
sessions are unaffected.
|
||
|
Junker der Provinz
|
78f79a3919 |
master: honour -volume.fileSizeLimitMB on the master's /submit (#11176)
* fix(master): honour -volume.fileSizeLimitMB on the master's /submit - #6748 `weed server -volume.fileSizeLimitMB=2048` still refused anything over 256MB, and the reason is not the one the report assumes: the option does reach the volume server. The master does not use it. Uploads through the master's /submit are buffered by submitForClientHandler, which passed a hardcoded 256MB to needle.ParseUpload, so the master rejected what the volume server it started would have accepted. The limit is now passed in. `weed master` gains its own -fileSizeLimitMB with the same 256 default, so a standalone master behaves exactly as before, and `weed server` and `weed mini` hand it the value their volume server already got. * master.follower: take the same upload limit, and say which flag to match Review found the follower left behind. It serves /submit like the leader and buffers uploads under the same limit, but kept the fixed 256MB, so a cluster raised above that would accept an upload through the leader and refuse the identical one through a follower. Two smaller points from the same review: the master's flag description named only the standalone volume server's spelling, and now names the weed server and weed mini form too; and the under-limit test asserted on the error message alone, so it would have passed had the limit rejected that payload with different wording. It now requires the request to get past parsing. |
||
|
yanglongwei
|
eb717199d0 |
master: delete replica_placement_mismatch labels when volumes leave topology (#11062)
* master: delete replica_placement_mismatch labels when volumes leave topology Fixes #10804. Setting the gauge to 0 left stale Prometheus time series that grew unbounded with volume churn; remove the label set on unregister instead. * master: delete replica_placement_mismatch only after last placement leaves Unconditional DeleteLabelValues on UnRegisterVolumeLayout dropped the series while other data nodes still held the volume, hiding under-replication until the next collect cycle. Delete only when Lookup is empty, and cover the two-copy case in a regression test. |
||
|
Chris Lu
|
3e85d9ec8e |
admin: bind to loopback by default, guard public unauthenticated bind (#11185)
admin: bind to loopback by default, refuse public unauthenticated bind The admin HTTP server (port 23646) defaulted to binding 0.0.0.0 with authentication disabled when -adminPassword was not supplied, exposing the full admin REST API (user creation, credential issuance, bucket deletion, filer deletion) unauthenticated on the network. This is the footgun described in GHSA-m3m8-mrgq-hf9h. Keep the no-auth mode for local dev, but remove the network exposure: - Add -ip flag (default 127.0.0.1) so the server binds loopback only unless the operator explicitly chooses a public address. - Refuse to start when binding a non-loopback address with no -adminPassword and no [https.admin] mTLS. The operator must enable auth or use loopback. - weed mini sets -ip from its existing -ip.bind; the guard does not apply because mini calls startAdminServer directly, not runAdmin. Addresses GHSA-m3m8-mrgq-hf9h. |
||
|
Chris Lu
|
f35e2ccf21 |
s3: warn when a lifecycle change leaves fast-path-stamped objects on their old TTL (#11184)
* s3: warn when a lifecycle change leaves fast-path-stamped objects on their old TTL The per-write TTL fast path (opt-in via s3.bucket.lifecycle.fastpath) stamps a volume TTL at PutObject time that can't be taken back. When an operator lengthens or removes an Expiration.Days rule (or deletes the bucket lifecycle) on a fast-path-enabled bucket, objects already written keep their baked-in TTL and won't be rescued by the change — unlike the default worker-driven path, which re-evaluates the current rules each pass. This is the data-loss direction described in #11183. Surface it: Put/DeleteBucketLifecycle now emit a glog warning and set X-Seaweed-Lifecycle-Fastpath-Warning on the response when the change removes, disables, lengthens, or re-scopes a fast-path-eligible rule. Shortening a rule does not warn (old objects simply expire later, not data loss). Tag-only and overflow-day rules are never on the fast path and never warn. Addresses the warning half of option 2 in #11183. * s3: address review — emit warning after mutation succeeds, fix ID-rename false positive Two issues raised by CodeRabbit, Greptile, and Devin reviews: 1. Failed mutations retained the warning header. The warning was set on the ResponseWriter before storeBucketLifecycleConfiguration / clearStoredBucketLifecycleConfiguration was called; if that failed, the error response carried a warning for a change that was never applied. Now the reason is computed before the mutation but the log and header are emitted only after it succeeds. 2. Rule renames produced false "removed" warnings. fastpathRuleKey used Rule.ID as the sole identity when present, so renaming a rule (same prefix/size/days, different ID) treated the old rule as removed. Replaced with two-pass matching: first by ID, then by fast-path predicates (prefix + size). An ID-only rename with unchanged predicates and days no longer warns. Greedy matching ensures each new rule is consumed by at most one old rule. Added regression tests: ID-only rename (no warn), rename + lengthen (warn), rename + shorten (no warn). |
||
|
|
f99c4a1f14 |
s3: make RenameObject idempotent for a retried request (#11178)
* feat(s3): make RenameObject idempotent for a retried request - #10661 A rename that succeeds but whose response is lost leaves the client with no safe move: retrying returned 404, because the source is already gone, so a retry was indistinguishable from a rename that never happened. The destination now carries what the rename that created it was, under x-seaweedfs-rename-token: the client's token, the source key and the time. A retry that names the same token and the same source and destination is answered 200 without touching anything. The same token sent for a different rename is refused with 409 rather than silently answered, and a token older than 24 hours is treated as unrelated so a key cannot answer for a request indefinitely. Requests without the header behave exactly as before. * s3: answer a reused rename token with 409, not 400 The PR promised Conflict and the code returned Bad Request. 400 tells a client its request was malformed and invites it to give up; this request is well formed and resending it unchanged will not help, because what it collides with is a rename the same token already stands for. The status code is now asserted in a test, since it is the part of this behaviour a client actually acts on. * Update weed/s3api/s3err/s3api_errors.go Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> * s3: fix rename token review notes - ErrIdempotentParameterMismatch returns 409 Conflict, not 400. The comment and TestRenameTokenReuseAnswersConflict both expect 409; the code regressed to 400 in a later commit. - stampRenameToken: clarify that markRenameToken mutates srcEntry in place, so the token reaches the destination via the move regardless of whether the UpdateEntry succeeds. The precondition only guards the pre-move write, not the move itself. - Extract the handler retry branch into retryRenameDecision and add TestRetryRenameDecision, covering the source-still-exists fallthrough that was previously reasoned about but not tested. * s3: IdempotentParameterMismatch returns 400, matching AWS docs The AWS S3 RenameObject API documentation specifies HTTP Status Code: 400 for IdempotencyParameterMismatch. Revert the previous 409 change and align the comment and test with the documented behavior. --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chris.lu@gmail.com> |
||
|
Eliah RusinandClaude Fable 5.1
|
3f9b05946b |
rust volume: bound the redb index cache per volume by --index tier (#11180)
The Rust volume server opens one redb database per volume and built each with redb's defaults, which give every database a 1 GiB page cache (0.9 GiB read cache + 0.1 GiB write buffer). With hundreds of volumes behind one disk the process-wide ceiling was volumes x 1 GiB: memory grew in proportion to the pages traffic touched, never shrank when traffic stopped, and hosts running many instances were OOM-killed under bulk ingest. redb, redbMedium and redbLarge were also treated identically, so the "memory~performance" tiers did nothing. Size the cache per tier instead: 4, 8 and 16 MiB per volume, mirroring the Go server's 3/6/12 MiB LevelDB block cache + write buffer. Thread the budget through RedbNeedleMap::new/load_from_idx so every open path (create, reuse, full rebuild) uses Database::builder().set_cache_size. Fixes #11179 Claude-Session: https://claude.ai/code/session_019x36FiSeyePh77YXao15kK Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Eliah RusinandClaude Fable 5.1
|
adbee9452a |
rust volume: bump redb 3.1.3 -> 4.2.0 (#11181)
No source changes: the API surface the needle map uses (Database create/open/builder, set_cache_size, set_durability, tables, iterators) is unchanged and the on-disk format is still v3, so existing .rdb files open as-is. The 4.0.0 breaking changes (Drop on AccessGuardMut, removal of the Legacy type) do not touch this crate. Relevant to the redb-backed index (#11179): - 4.1.0: optimizes cache usage and memory usage; ~1.5x faster writes. - 4.2.0: Durability::None commits ~2x faster; pages freed by a durable transaction are reused by the very next one; a crash-recovery fix for a crash during repair of an earlier crash. Claude-Session: https://claude.ai/code/session_019x36FiSeyePh77YXao15kK Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Chris Lu
|
8a68337256 |
filer: pack SSE chunks into manifests (#11175)
* filer: pack SSE chunks into manifests * s3: resolve encrypted manifests before reads * s3: scope encrypted manifest resolution to ranges |
||
|
Chris Lu
|
97154802c5 |
docs(star-history): make the chart taller (#11173)
Change the matplotlib figure size from (10, 4) to (10, 6) so the star history chart renders vertically longer in the README. The regenerated note/star_history.svg reflects the new 5:3 aspect ratio (720x432pt) instead of the previous flat 2.5:1 (720x288pt). |
||
|
Chris Lu
|
ff0d5a9adf |
ci(mount-windows): clean up processes and don't let Logs step fail job (#11174)
The "Mount and exercise" step left the weed.exe mini server and the final WinFsp mount running when it exited. The next step's pwsh.exe then failed with STATUS_DLL_INIT_FAILED (0xC0000142), failing a job whose actual test step had passed. The same code passed on both the PR branch and the next master run, so this was a transient launch failure — but it was caused by an unclean environment and made fatal by a diagnostic step. Tear down all weed.exe processes at the end of the test step so subsequent steps launch into a clean environment, and mark the Logs step continue-on-error so a diagnostic step can never fail the job on its own. |
||
|
Chris Luanddevin-ai-integration[bot]
|
811b8b5734 |
make the remote-mount cache wait configurable per mount (#11168)
* add a per-mount cache_wait_ms to the remote storage mount mapping A read of an uncached remote-only object waits on a hardcoded size tier before it can fall back to the origin, so every ranged read of a large remote-only object pays that wait. Carry the wait in the mount mapping so it can be tuned, or set to zero, per mount. * resolve the cache wait of an uncached remote-only read from its mount The wait came only from the object size, so an operator could not trade cache hits for time to first byte. Both read paths now resolve the mount covering the object and let its cache_wait_ms replace the size tiers. * read straight from the remote when a mount waits zero for its cache A mount used as a streaming source pays the cache wait on every ranged read of an object too large to finish caching, and the caching itself is wasted work. A zero wait now skips the cache call, so both read paths go to the origin immediately. * let remote.mount set the cache wait of a mount remote.mount -cacheWait=0 turns a mount into a streaming source, and any other duration trades cache hits against time to first byte. * keep the size based wait for a version-specific read A read pinned to a version cannot fall back to the origin, since the mounted remote only holds the current key, so a mount that opts out of caching would leave it on the 503 retry loop forever. * let the operator allow a remote-only read to dial an internal endpoint The remote-mount read paths in the filer and the S3 gateway always refused an endpoint resolving to a loopback or private host, so a mount backed by an internal S3 could never be read from its origin, only through the local cache. Both now take the allowance the volume server already has, still off by default. * skip the background cache of a mount that waits zero for its cache GetObjectHandler kicks off caching for every remote-only read, so a mount serving as a streaming source kept downloading whole objects even though no read ever waited for them. * cover a zero cache wait end to end The read has to reach a real origin, so the harness also opts the filer and the S3 gateway into dialing the loopback remote it already allows for the volume server. * resolve the S3 cache wait once so the background cache follows it too The background cache that GetObjectHandler starts read the mount on its own, so it skipped a version-specific read that the foreground path still waits for. Both now ask the same resolver. * answer 404 when the origin of a zero-wait read is gone Metadata can outlive the object it points at, and with no cache to fill the read would sit on the 503 retry path forever. The remote backends already report a missing object as ErrRemoteObjectNotFound. * open the origin at write time for a multipart range Every part of a multipart Range is prepared before any is written, so opening eagerly would hold one origin connection per part and leak the ones already opened when a later part fails to open. * reject a cache wait shorter than a millisecond The mapping stores milliseconds, so -cacheWait=500us truncated to zero and silently turned caching off instead of waiting. * restore the doc comment of cacheRemoteObjectForStreamingWithShortTimeout Extracting the wait resolver left its comment on the new function. * stat the origin before committing a multipart range Opening at write time keeps no connection through the preparation, but it also moved a failure past the point where the multipart body picks the response status, so a gone origin truncated a 206 instead of answering 404. One stat up front puts the status back. * stat the origin once per request Every part of a multipart Range is prepared on its own, so the preflight ran once per range instead of once per read. * map Azure and GCS stream not-found to ErrRemoteObjectNotFound ReadFileAsStream on Azure and GCS returned provider-specific not-found errors instead of ErrRemoteObjectNotFound, so a zero-wait read of a deleted object was misclassified as a transient cache failure and retried indefinitely. Map BlobNotFound and ErrObjectNotExist the same way StatFile already does. * Update weed/remote_storage/gcs/gcs_storage_client.go Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Chris Lu
|
f79d83abf4 |
volume: expire TTL volumes whose only traffic is deletes (#11167)
* volume: count a TTL volume's age from its last write, not the .dat mtime A delete appends a tombstone needle and vacuum rewrites the .dat wholesale, so the file's mtime moves without any write ever landing. The loader read lastModifiedTsSeconds back from that mtime, so every restart of a volume taking delete traffic re-armed expired() for another full TTL: an overwrite-heavy collection kept growing until it hit the max-volume cap. Recover the clock from the newest .idx entry that is not a tombstone and read that needle's append timestamp, falling back to the mtime when no write is recoverable. Only TTL volumes pay for the scan. Fixes #11160 * volume: count the .vif destroy time from the last write too ExpireAtSec is what an EC volume is reclaimed on, and it was recomputed as now+TTL every time the .vif was written. A read-only mark, a tier upload or an EC encode therefore handed an already expiring volume another full TTL, the same way the .dat mtime did. Derive it from the volume's last write, falling back to now for a volume that has not taken one yet so a fresh volume is not born expired. * volume: mirror the last-write TTL clock in the Rust volume server Same recovery as the Go loader: scan the .idx backwards for the newest entry that is not a tombstone and take that needle's append timestamp, leaving the clock on the .dat mtime when no write is recoverable. * volume: mirror the last-write destroy time in the Rust volume server Both .vif writers and the EC encode computed ExpireAtSec as now+TTL, the same way Go did, so the destroy time moved every time the sidecar was rewritten. Route all three through the volume's last write. * volume: report the .dat mtime in the Rust heartbeat, like Go does The Rust server reported its TTL clock as ModifiedAtSecond while Go reports the .dat mtime. The shell's quiet-period gates (volume.tier.move, volume.delete_empty) read that field as "last touched", which a delete has to count towards even though the TTL clock deliberately ignores it -- and with the clock now recovered from the last write, the two drift further apart. * volume: take the newest write by timestamp on a vacuumed volume The reverse .idx scan trusted position, which holds only while the .dat is append ordered. Vacuum rewrites it in key order, and since an overwrite keeps its original key, the highest-key survivor is not necessarily the newest write -- the recovered clock could land up to a TTL early and take the volume with data still inside its TTL. A volume that has been vacuumed (CompactionRevision > 0) now takes the maximum append timestamp over a bounded window of write entries instead. An append-ordered volume still answers in one read. * volume: never guess a vacuumed volume's last write, and resolve wrapped offsets Two holes in the reverse scan, both from review: A vacuumed volume's writes are ordered by key, so any of them can hold the newest timestamp. Reading a capped window sampled the highest keys, which could still miss a recently overwritten low-key needle and expire data inside its TTL. The scan now covers every write a vacuumed volume indexes, and a volume too large to scan keeps the .dat mtime rather than report a partial maximum -- late is recoverable, early is not. A .dat past MaxPossibleVolumeSize wraps the offsets in its .idx, so reading a timestamp at the unwrapped offset picks up an unrelated needle. Resolve the entry against the needle header first and retry one volume size in, the way doCheckAndFixVolumeData already does. * volume: drop GitHub issue references from TTL comments |
||
|
Chris Lu
|
567578f08d |
docs(readme): replace star-history.com with self-generated chart (#11171)
* docs(readme): replace star-history.com with self-generated chart The star-history.com SVG is a third-party dependency that can rate limit or go down. Replace it with a GitHub Action that fetches stargazers via the REST API and renders an SVG with matplotlib, committing note/star_history.svg weekly. The README references the committed file directly, so the chart has no runtime dependency on any external service. * ci(star-history): run daily instead of weekly |
||
|
Chris Lu
|
070174726d |
docs(readme): replace rate-limited starchart with star-history (#11170)
starchart.cc is rate-limiting the SVG endpoint, so the Stargazers chart renders blank. Switch to star-history.com, which serves a live SVG for this repo and links to the interactive chart. |