Compare commits

...
Author SHA1 Message Date
Chris LuandClaude Opus 5.5 b89d7e36bf install.sh: offer the vector engine on linux only
Releases build weed-vector for linux amd64/arm64 only, so --component vector
on darwin or windows downloaded a missing asset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK
2026-11-14 11:26:49 -08:00
b067266e6b shell: fix s3.circuitBreaker help examples and error messages (#12657)
* shell: fix s3.circuitBreaker help examples and error messages

The help examples used -type count, but -type accepts Count and MB (as
its flag description says), so both examples failed with "type not
valid, only 'count' and 'bytes' are allowed", an error that names
neither accepted value. Two other errors pointed at flags that do not
exist: -max and -counts, where the flag is -values.

Use -type Count in the examples, name Count and MB in the type error,
and -values in the value errors. TestCircuitBreakerHelpExamplesRun runs
every help example and fails on master for the two -type count lines.

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>

* shell: name the bad action and the -actions flag in s3.circuitBreaker

The unknown-action error pointed at a nonexistent -action flag and echoed
the whole -actions list instead of the action that was rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 11:17:52 -08:00
b3a58506aa filer: report a SQL listing that fails partway through (#12654)
* filer: report a SQL listing that fails partway through

ListDirectoryPrefixedEntries never checked rows.Err() after its rows.Next()
loop. When the result set failed partway, for example on a lost
connection, rows.Next() returned false and the listing returned the
entries read so far with a nil error, so callers saw a shorter directory
instead of an error. Check rows.Err() once the loop has run to the end,
as the scan in abstract_sql_store_scan.go already does. A listing the
callback stopped on purpose is left alone: rows.Err() would return a
canceled request context's error there.

This covers every store built on AbstractSqlStore (mysql, mysql2,
postgres, postgres2, sqlite).

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>

* filer: wait for the canceled SQL listing to release its rows in the test

A fixed sleep let the callback return before database/sql closed the
rows, so a listing that always consulted rows.Err could still pass.
Wait, bounded, until the connection is back in the pool instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 11:14:54 -08:00
anantandanant b42993d24a shell: make TestCircuitBreakerShell compare against the expected configs (#12656)
The test unmarshalled the command's output into both actual and expect,
so it compared the output with itself and the expected JSON in each case
was never checked; it also skipped the first case. Compare every case's
output with its expected config as S3CircuitBreakerConfig messages (the
output includes default fields such as "enabled": false that the
expected JSON leaves out), and restore LoadConfig when the test ends.

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com>
2026-11-14 11:14:14 -08:00
Chris LuandClaude Opus 5.5 fd4603d657 s3 lifecycle: count filer entries a walk reads but skips as progress (#12658)
* dailyrun: count entries read from the filer, skipped ones included

A versioned bucket's pre-pass and a resumed walk read entries without
emitting them, so a walk can read for hours with no visited entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: count listed entries as progress work

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 11:12:33 -08:00
Chris LuandClaude Opus 5.5 53377bcb5d mount: keep a handle dirty when a write lands during its flush (#12652)
A flush uploads the dirty pages without the handle lock, then commits the
entry under it. A write in between grows the entry's size, but its pages
are not in the committed chunks. Clearing dirtyMetadata then made the next
flush skip the commit, leaving a file with the new size and no chunk for
the write, which reads back as zeros.

Count page writes on the handle and clear dirtyMetadata only when none
landed since the flush started uploading.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 11:11:01 -08:00
1167ca8cf5 vacuum: report a request dropped because another vacuum is running (#12648)
* vacuum: report a request dropped because another vacuum is running

The master runs one vacuum at a time. A request that arrives while
another one is running, the automatic vacuum or an earlier request, is
dropped: Topology.Vacuum logs "Vacuum is already running" and returns.
The VacuumVolume RPC and /vol/vacuum then answered as if the vacuum had
run, so volume.vacuum printed nothing and exited 0 with no volume
compacted. Where the automatic vacuum is busy for hours, an explicit
volume.vacuum -volumeId could do nothing every time, and nothing said so.

Topology.Vacuum now returns ErrVacuumRunning for a dropped request. The
RPC returns it and /vol/vacuum answers 409 with it. volume.vacuum names
the request that failed, because a list of volumes or collections is
sent as several requests and a vacuum that starts in between fails only
the later ones. Requests are still dropped, not queued, and the
automatic vacuum ignores the error as before.

The integration tests that need their /vol/vacuum request to run ask
again while the master answers 409.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* docs: note the 409 answer from /vol/vacuum while another vacuum runs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-14 11:07:44 -08:00
Chris LuandClaude Opus 5.5 512f898600 s3 lifecycle: report work-based job progress during a pass (#12644)
* s3 lifecycle: report work-based job progress during a pass

The daily pass sent progress only while starting, so a healthy scan that
ran for many hours could reach the admin's 24-hour no-progress guard.
Worker heartbeats prove the worker is alive, not that this job moves.

While dailyrun.Run is in progress the handler now sends a progress update
every 5 minutes, and only when the pass's counters (entries visited,
replay events, RPC outcomes including errors) moved since the last
delivered update. An idle tick sends nothing, so a stalled pass still
expires. A failed send is retried on the next tick. The reporter is
stopped and joined before the completion is sent, so no progress can
follow the terminal result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: test that progress, not heartbeats, keeps an old job active

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: label progress phase and bucket as the last started

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:55:08 -08:00
Chris LuandClaude Opus 5.5 858f8cbad8 s3 lifecycle: check internal S3 authentication before starting a pass (#12645)
* s3 lifecycle: add a LifecycleReady RPC that only authorizes the caller

LifecycleReady runs the same checkAdminAuth as LifecycleDelete and reads
no bucket or object metadata, so a lifecycle worker can confirm that an S3
front accepts its internal credentials before it starts a pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: check S3 fronts with LifecycleReady before the pass

Execute used to dial the first registered S3 front and start loading
policies and walking buckets, so a missing or mismatched
jwt.filer_signing.key surfaced only as repeated per-bucket delete
failures. It now checks each front in order with LifecycleReady, using
the credentials LifecycleDelete sends, under a 10s bound that also
follows parent cancellation. The first front that passes serves the
pass; fronts that fail are reported in the job's progress and the log.
When none passes, the job fails before any listing with an error that
names the front and says whether this worker has the key set, without
key or token material. Fronts that predate LifecycleReady are checked
with an empty LifecycleDelete, which they authorize and then reject
before any lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:53:07 -08:00
Chris LuandClaude Opus 5.5 c819e3fbea s3 lifecycle: per-pass run results and last full-pass status (#12643)
* s3 lifecycle: collect per-pass bucket results and work counters

WalkBuckets kept only the first bucket error and logged the rest. A
RunStats shared by the walker, the walker dispatcher and the replay
drain now records every bucket's walk result (walked, failed, skipped
on cancellation), entry and object visits, replay events, and
LifecycleDelete outcomes under the dispatch_total outcome labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: persist the last fully successful pass

Shard cursors and walk timestamps advance on partial passes as well, so
neither says when lifecycle last covered every policy. Keep that in its
own record next to the cursors, plus a worker gauge for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: report each pass's result in the job record

The job ended with a fixed success summary, or with only the first error,
and malformed lifecycle policies were V(1) logs. The completion now
carries the pass status (complete, partial, failed, canceled), bucket
totals and failures, invalid policies, entry, object and event counts,
LifecycleDelete outcomes, the last phase and bucket, the elapsed time
and the last full pass, on failed passes as well.

Only a complete pass advances the full-pass record; a malformed policy
makes the pass partial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep a failed job's result in its record

Completions with Success=false dropped Result.Summary and
Result.OutputValues, so a handler could not report what a failed run
had done, which buckets failed, or how far it got. Keep them; the error
message still drives the job's message and run record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* grafana: chart the time since the last full lifecycle pass

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: a replay halted on an unresolved delete is a partial pass

BLOCKED, RETRY_LATER and exhausted transport retries stop a shard's
replay without an error, leaving its due events for a later pass. Count
those shards and report the pass as partial, so it no longer advances
the full-pass record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: a pass with throttled walks is not a full pass

WalkerInterval can skip the walk for walk-only rules (ExpirationDate,
ExpiredDeleteMarker, NewerNoncurrent) with no error. Count those shards
and keep the full-pass record for passes that evaluated every rule; the
job reports full_pass and walks_deferred.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3 lifecycle: keep the saved full-pass time when the save fails

Publishing a timestamp that never reached the filer let the gauge move
back on the next job's reload. Keep the previous value and report the
save error in the job result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: drop a failed attempt's summary when the retry reports none

Retries reuse the job record, and failed completions now keep their
result summary, so a retry that completes without a Result showed the
failed attempt's summary, and a successful one used it as its message.
Clear the summary when a completion carries no Result, as the output
values already are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:50:37 -08:00
Chris LuandClaude Opus 5.5 6ad4b07598 mount: scope and order the final-release writeback fence (#12651)
* mount: fence a dropped handle only when it holds unpublished content

dirtyMetadata is also set by chmod, xattr and fallocate, so a failed
metadata-only close refused every later write of the inode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: fence the inode before its last handle leaves the handle map

A reopen between the map removal and the fence could publish at the
stale kernel EOF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: fence a failed metadata flush only when the handle holds content

A refused synchronous flush or an abandoned writeback metadata flush of a
handle with only chmod, xattr or fallocate changes fenced the inode, so
later writes were refused until FORGET.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: count a size change on an open handle as unpublished content

A truncate or extend through an open handle changes the entry's size and
chunks, but it set only dirtyMetadata. A failed close then skipped the
final-release fence, so a reopen before FORGET could publish against the
untruncated entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: mark copied content dirty before each page of a streaming copy

A copy that failed or was cancelled after earlier pages were accepted left
dirtyContent false, so the final-release fence skipped that content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: mark copied content dirty only after a page is accepted

A copy whose first page is rejected leaves the destination unchanged and
the kernel EOF where it was, so fencing the inode on release only blocked
later writes until FORGET.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: skip the server-side copy while the destination holds unpublished content

A whole-file server copy clears dirtyContent when it commits, so a page a
cancelled streaming copy left behind would lose its release fence. Fall
back to the streaming copy instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: skip the server-side copy while the source holds unpublished content

The filer copies the stored source, so pages still pending on the source
handle would be missing from the destination. The streaming copy reads
them through the dirty pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: check the publication fence before a server-side whole-file copy

The filer copy publishes the destination without going through the
flush-time guard, so a fenced writeback inode could still take new
content before FORGET. Fall back to the streaming copy, whose flush is
guarded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 10:47:33 -08:00
Ping Qiu 65ecb6aa12 fix(mount): prevent negative activity-score shifts (#12631)
fix(mount): bound activity decay shifts across time regressions
2026-11-14 10:46:57 -08:00
Chris LuandClaude Opus 5.5 0a38738076 filer: keep the listing snapshot below later and unfinished event stamps (#12642)
filer: take the listing snapshot as a claim below later and unfinished stamps

A ListEntries snapshot tells the client that every event at or below it is
already in the listing, so the mount drops those events from a directory
build and fences them with the directory floor. The snapshot was a raw
time.Now(), which an event stamp can tie on a coarse clock: on Windows a
mkdir right after a listing of its parent was stamped with the snapshot's
own timestamp, the mount took it as listed, and the new directory stayed
missing from a parent marked cached. A create stamped before the snapshot
but not yet in the store was dropped the same way.

Take the snapshot through the same in-flight claim the idle heartbeat uses:
it stays below every unfinished stamp, and every later stamp lands above it.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 01:17:51 -08:00
10f8b2d5b1 mount: guard peer-truncate gaps, stale append EOF, and terminal writeback refusal (#12630)
* mount: guard append gaps over concurrent content merges

* mount: fence a writeback inode when its async publication is abandoned

A drain that gives up on a non-precondition error left the token clear, so a
reopened writeback handle could publish past the stored EOF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: fence a writeback inode when its async data flush fails

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: fence the inode when its last handle drops uncommitted content

A synchronous release whose final flush fails destroys the handle's dirty
pages, while under writeback the kernel EOF already counts them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 01:13:37 -08:00
b50d4a9cab mount: don't spend the close()/fsync commit deadline on uploads (#12641)
* mount: start the close()/fsync commit deadline after the data upload

Flush, Fsync and the truncate path of SetAttr create the 30 s
metadataFlushTimeout context on entry and pass it to doFlush, so the
clock also runs through FlushData, the upload of the dirty pages. That
upload has its own timeouts and retries. When an assign through a stalled
filer or a slow volume server takes most of the 30 s, CreateEntry then
fails at once with "context deadline exceeded" and close() returns EIO
for data that was already stored. The deadline is meant for the commit
alone (#10102: "a healthy CreateEntry completes in well under a second").

doFlush now takes the commit timeout and starts the deadline once the
data is uploaded. close() still cannot hang on a stuck filer: the commit
keeps its 30 s bound. Release and the flush before a rename pass 0, as
before (no overall bound, only the per-call filer timeouts).

Seen in production on a training cluster: a 30 s stall of one replica's
disk, and a filer that stalled for about 30 s, each failed close() with
EIO although the upload had succeeded.

Tests: close(), fsync() and a truncate below dirty data, each after an
assign slower than the deadline (all three fail before this change), and
a filer that never answers CreateEntry still fails close() at the
deadline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: charge the flush commit deadline only for its filer calls

Starting the 30 s deadline after the data upload is not enough.
flushMetadataToFiler does more than call the filer before it commits: it
waits for the handle's locks, reads the file back and uploads it again
when its chunks are bloated (maybeMergeChunks), and uploads a manifest
once enough chunks have built up (MaybeManifestize). None of that takes
the context. When it outlasts the deadline, CreateEntry again fails at
once with "context deadline exceeded", and close() or fsync() returns an
error for data that was stored.

The deadline becomes a budget, commitBudget, charged only while a filer
call of the commit runs: CreateEntry, and the read of a changed hard
link before it. The attempts of one flush share it, and a flush is not
retried once it is spent, so a stuck filer still fails close() after
30 s. Release, the flush before a rename and the async flush pass no
budget, as before.

Seen in production: the merge of a 1.8 MiB sqlite file with 183 chunks
took 32 s because one chunk's read first tried a volume server that was
down, and the commit failed 32 microseconds after the merge finished.

Tests: a flush whose manifest upload is slower than the deadline commits
(it fails without this change), and the filer calls of one flush share
the budget.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* mount: give the stalled-lookup refusal test the new doFlush argument

TestContentFlushRefusalWithStalledLookup came in from master after this
branch changed doFlush's signature, so the merged package did not build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-13 21:56:44 -08:00
Chris LuandClaude Opus 5.5 e0890391ac mount: keep a refused content create refused when the stored-entry read stalls (#12629)
The read of the stored entry ran in the create's filer callback, so a read
that hit the call deadline turned the refusal into a filer timeout, and
both retry layers resent the refused create. Read it in its own call,
bounded to one filer call.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:19:35 -08:00
Chris LuandClaude Opus 5.5 6eb2fb4842 filer: stop full-chunk reads at the chunk size after a mount truncate (#12632)
* util/http: stop a full-chunk read at the chunk's recorded size

weed mount shrinks a chunk's Size in place on ftruncate but leaves its
needle untouched. A view covering the whole shortened chunk is fetched
without a Range header, so the reader returned every stored byte. The
filer GET then wrote past the declared Content-Length and sent an empty
body, and the stream reader served the stale tail.

ReadUrlAsStream and the cipher path now deliver at most size bytes;
size 0 still reads the whole needle for manifest and log chunks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: limit a plain chunk read to the requested size

A full-chunk read sends no Range header, so a chunk shortened by a mount
truncate streamed its whole needle into the S3 response.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* util/http: still report a failure past the chunk size

The volume server aborts a full-needle response when the needle fails
its checksum, and only at the end. Stopping at the chunk size missed
that, so a damaged copy read as healthy. Read the rest of the body and
return its error so the caller tries another copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:17:46 -08:00
Chris LuandClaude Opus 5.5 63c99183d1 s3vectors: name weed-vector processes servers (#12627)
* s3vectors: call the registered weed-vector processes servers

The gateway's discovery named them workers, the name of the admin
plugin workers. Also drops two comments left without a function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* test/s3vectors: build the Enterprise engine as weed-vector

In an Enterprise checkout, make ./weed-vector builds the distributed
engine, which also serves VectorService, instead of a separate
weed-vector-worker. An open-source checkout builds weed-vector-server
as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* test/s3vectors: distributed mode starts two more weed-vector engines

It started weed-vector-worker, which no target builds now, and waited
for a gateway log line that no longer exists. The Enterprise
weed-vector's engines all join the placement controller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:17:15 -08:00
Chris LuandClaude Opus 5.5 a8436a621a admin: vector servers on the dashboard (#12606)
* admin: vector servers on the dashboard

A Vector servers count in the Cluster card and a Vector Servers table
after S3 Servers, from the engines the dashboard already discovers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* admin: drop the dashboard's vector server kind

Every vector server is a weed-vector engine, so the column said the
same thing on every row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:16:43 -08:00
Chris LuandClaude Opus 5.5 58d25998d0 admin: Cluster → Vector Servers page (#12603)
Lists the weed-vector engines registered with the master, with version
and health, next to the other server lists.


Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:16:00 -08:00
github-actions[bot] d91d58ce47 docs: regenerate star history chart 2026-11-14 02:04:51 +00:00
Chris LuandClaude Opus 5.5 4fc41872e3 mount: keep an open file publishing when a peer unlinks one of its hard link names (#12613)
* mount: record a hard link's looked-up names on its inode

Lookup mapped a second name of a hard-linked file to the existing inode
without adding it to the inode's names, so GetAllPaths missed it and the
final forget left it mapped to a released inode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep an open file publishing when a peer unlinks one of its names

A delete event from another client marked the inode's open handle deleted
even when other names still held the hard-linked file, so every later
fsync reported success without saving metadata. The delete event's entry
carries the link count the filer decided on, so an alias delete now drops
only that name, keeps the handle on a name this mount knows, and has the
next flush read the survivor's stored entry before publishing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test peer unlinks of a hard link name held open

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: leave an unlinked file out of the periodic metadata flush

flushFileMetadata did not check the handle's delete mark. A file
unlinked while open before its deferred create reached the filer was
created again by the next periodic flush, which expects no entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: read a pending link change before a periodic metadata flush

Only flushMetadataToFiler applied linkRefresh. After a peer unlinked one
name, the periodic flush kept publishing over the old link count, so the
filer refused every one until a regular flush ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test the periodic flush after a peer unlinks a hard link name

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 16:14:22 -08:00
Chris LuandClaude Opus 5.5 08f9dcb6de admin: vector buckets link maintenance jobs to the Lance lane (#12633)
The Vector Engines and Maintenance card linked to the general plugin
monitoring page; the vector index jobs run in the Lance lane.


Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 16:14:19 -08:00
Chris LuandClaude Opus 5.5 8a756af348 vector: key shared leases by manifest commit time too (#12626)
On Windows Lance commits local tables by rename and leaves the manifest
ETag empty, so a table dropped and written again at its location reused
the old table's lease for the same version number.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 15:52:52 -08:00
Chris LuandClaude Opus 5.5 68b6a2ed08 mount: model the refused async flush as a replaced file in recovery tests (#12628)
A peer's chunk append now merges into the flush, so these tests no longer
saw the refusal they exercise. A replaced file is still refused.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 13:50:51 -08:00
anantandanant fdd3a692d3 shell: show -collection, not -c, in ec.balance and ec.rebuild help (#12617)
Both usage lines showed [-c EACH_COLLECTION|<collection_name>], but the
flag is -collection; there is no -c. ec.rebuild's own Options section
already lists -collection.

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com>
2026-11-13 13:03:07 -08:00
Chris LuandClaude Opus 5.5 45cd6033a0 s3: make conditional DeleteObject atomic on versioned buckets (#12624)
* s3: re-check a versioned DeleteObject If-Match under the object lock

A versioned or suspended DeleteObject without a versionId routed to the
owner filer after checking If-Match outside any lock, so concurrent
conditional deletes could all pass the check and each add a delete
marker. Keep conditional deletes on the lock path, which re-checks the
precondition under the object write lock, as conditional versioned PUTs
already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: guard a conditional version delete on the owner filer

DeleteObject with a versionId checked If-Match before routing the
delete to the owner filer, so concurrent conditional deletes of one
version could all report success. Carry the If-Match clause into the
owner's transaction as a guard on the version (the null entry for
versionId=null), evaluated under the object's path lock before anything
applies, and map that guard failing to 412 instead of the object-lock
403. An If-Match the filer cannot evaluate stays on the lock path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: carry an If-Match ETag list into the routed delete

A DeleteObject whose If-Match listed several ETags fell back to the
gateway's object write lock, while a single-ETag delete of the same key
ran under the owner filer's path lock. The two locks do not exclude each
other, so a list delete and a single-ETag delete could both report
success. Send every candidate to the filer, whose strong comparison
matches checkDeleteIfMatch, so all conditional deletes with an owner are
decided under the owner's lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: fail a conditional null delete on an owner transport error

The owner may have committed the delete before the error, and the gateway
lock path it fell back to does not exclude routed deletes, so a second
If-Match delete could pass there too. Answer 500 as the specific-version
path does; unconditional deletes still fall back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: fail a conditional null delete on any owner error but the nested refusal

Only the owner's non-empty-folder refusal is known to apply nothing. After
any other owner error the delete may have been partly applied, and a lock-path
re-check does not exclude a concurrent routed delete of the same version, so a
conditional null-version delete answers 500 instead of falling back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: strip a nested conditional null delete on the owner

A conditional versionId=null delete that the owner refuses because other
keys are nested under the entry used to fall back to the gateway lock and
re-check If-Match there. That lock does not exclude routed deletes: once
the last nested key is removed, a concurrent routed delete can pass the
same check on the owner, and the fallback then treats the missing entry as
deleted, so both requests answer 204.

Demote the entry on the owner instead, in a transaction guarded by the
If-Match clauses and by the entry it was read as, with the object-lock
guards on the request. A missing or already demoted entry answers 412.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: add an IF_OBJECT_EXISTS write condition

IF_EXISTS also holds for a plain directory, which is what is left of a key
once the object on it is stripped while other keys stay nested under it.
IF_OBJECT_EXISTS holds only for an entry that carries an object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: require an object for a conditional null delete with If-Match: *

Stripping the null version off an entry other keys are nested under leaves a
plain directory. Once the last nested key is gone, a routed delete with
If-Match: * still found an entry there and removed the empty directory, so a
second conditional delete could also answer 204. The owner now checks that
the entry carries an object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: settle a wildcard null delete on the entry when the filer predates IF_OBJECT_EXISTS

A filer older than IF_OBJECT_EXISTS fails it closed, so with the gateway
upgraded first every If-Match: * delete of a null version answered 412. When
the narrowed guard fails, the gateway now reads the entry on the owner: absent
or a plain directory is a 412, otherwise the file is deleted, or the directory
stripped, under IF_EXISTS and IF_ENTRY_EQUAL on the entry it read. The demote
keeps the caller's If-Match clauses, since the entry guard already requires an
object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:47:22 -08:00
d695d101b5 Fix/s3 abandoned chunk download (#12616)
* filer: cancel a chunk download that no reader waits for

* filer: stop waiting for reader cache budget once cancelled

* filer: keep a cancelled download's re-lookups off its context

The refresh and retry lookups took the download's context. FilerClient
coalesces concurrent lookups of a volume, so a cancelled download leading
one failed the other readers waiting on it. Fetches still stop on cancel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test that cancelling a download leaves its re-lookup running

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: cancel unheld downloads when the reader cache is destroyed

ReaderCache.destroy waited for every downloader, so closing a reader
waited for a stalled prefetch nobody read. Downloads with no reader or
stream on them are now cancelled first; ones still in use keep running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test that destroying the reader cache cancels an unread prefetch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: cancel a destroyed download once its last holder leaves

destroy skipped downloads a reader or stream still held, and once they
were out of the map nothing cancelled them, so destroy waited for a fetch
nobody used. removeConsumedLocked now cancels any consumable download,
whether or not it is still in the map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test a destroyed download cancelled after its last stream leaves

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: release the stream only after destroy dropped the download

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: test a cancelled stream leaving a waiting reader's download running

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:45:34 -08:00
Chris LuandClaude Opus 5.5 813ac74d79 s3tables: keep legacy-subject owners and policy principals for federated STS sessions (#12615)
* s3tables: re-anchor legacy owners for federated STS sessions

A web-identity role session was keyed on its bare OIDC subject until
owners became issuer-scoped. Its sessions now carry the scoped aws:userid
but no asserted issuers, so the legacy subject check never applied and the
session lost the table buckets and namespaces it had created. Read the
configured issuer set for such sessions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: keep token claims from asserting trusted issuers on STS sessions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* iceberg: give the catalog's table handler the local accounts and issuer set

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3tables: test a federated session on a bucket its legacy subject owns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* sts: drop a token-supplied aws:userid when a session has no parent user

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: take a session's aws:userid from its signed parent user

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test that session claims keep only the server-asserted aws:userid

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3tables: match legacy subject patterns in table bucket policy principals

A policy written before issuer scoping can name role sessions by a pattern
over their bare OIDC subjects. Match such a pattern against the caller's
subject claim once that subject resolves to the session's scoped principal
under the single configured issuer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3tables: test a federated session against a legacy subject pattern principal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:43:52 -08:00
Minh Vu 4bc7598a43 admin: escape dynamic text in S3 bucket modals (#12625)
admin: escape dynamic text in bucket modals
2026-11-13 12:42:40 -08:00
Chris LuandClaude Opus 5.5 d685709832 s3api: refuse writes to a stored version's physical path (#12622)
* s3api: refuse a PUT to a stored version's physical path

In a Suspended bucket a PUT to <key>.versions/v_<id> replaced the stored
version's body in place. Look the entry up when the key has the version
file shape and deny the write when it is a version file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: refuse copy, multipart, POST and rename writes to a version file

Apply the same version file check to the CopyObject, CreateMultipartUpload,
CompleteMultipartUpload, POST Object and RenameObject destinations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: read a version file write target on its logical owner

The guard resolved the key as an object, which on a ring reads the
physical path's owner and, in a versioned bucket, may return a nested
version under <key>.versions/v_<id>.versions instead of the stored
version. Look up the physical entry directly, routed by the logical key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: confirm a version file miss on a skipped owner

When the owner was recently unreachable the routed lookup answers from a
replica, whose miss does not show the version is absent. Ask the owner
directly in that case and refuse the write when it cannot answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: confirm every routed version file miss on the owner

Whether the routed lookup skipped the owner can change between the lookup
and a second unreachable check, letting a replica's miss through. Ask the
owner after any miss instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:39:40 -08:00
d1a97f4c3d fix(mount): retain refused async metadata before releasing handle (#12598)
* fix(mount): retain refused async metadata before releasing handle

* mount: sync the failed-metadata directories with util.FsyncDir

Directory fsync fails on Windows, where the mount also builds, and returns
EINVAL on some filesystems. Either kept the retention loop retrying a record
that could never be saved, so the drain never finished.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep failed-metadata records in a per-user directory

The write cache root defaults to the shared temp directory. The first mount
to save a record created failed-metadata there as 0700, so another user's
mount could never save one and its drain retried forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: drop the async conflict probe test

TestContentAsyncConflictKeepsRecoveryRecord drives the same refused drain
with the retry backoff stubbed out; the probe slept through the real backoff
(7 s) and its kept-handle branch could not run. Its peer-entry check moves
into the recovery record test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:34:58 -08:00
5a54009439 filer/etcd: client keepalive and per-request deadlines (#12621)
* filer/etcd: client keepalive and per-request deadlines

An etcd member that goes silent with its connections open (a dead or hung
node, a network partition, a frozen process) keeps the store's requests
balanced onto it waiting forever, because the client has no keepalive and
the calls have no deadline. Writes through the filer stop although the
other members keep quorum.

keepalive_timeout (default 3s, "0s" turns keepalive off) enables gRPC
keepalive with the ping interval fixed at gRPC's 10s minimum. gRPC also
sets TCP_USER_TIMEOUT to it on Linux, so a connection whose sent data goes
unacknowledged that long is closed by the kernel. request_timeout (default
3s) bounds every Get, Put and Delete unless the caller set a deadline;
timeout keeps its meaning as the dial timeout. DeleteFolderChildren gets no
request deadline: its DeleteRange grows with the directory, and keepalive
still ends it if the member goes silent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update weed/filer/etcd/etcd_store.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* Update weed/filer/etcd/etcd_store_test.go

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* Update weed/command/scaffold/filer.toml

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>

* filer/etcd: no request deadline on entry writes

A Put whose client deadline expires may still have been committed. The
filer's upload handler treats the error as a failed write and deletes the
uploaded chunks, which the saved entry may then point at. InsertEntry and
UpdateEntry therefore keep the caller's context: etcd's own request
timeout ends a Put a live member cannot commit, and keepalive ends one on
a member that goes silent. Reads and deletes keep request_timeout; a
timed-out store delete returns before the filer deletes any chunks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: no request deadline on KvPut

The filer wrapper stores a hard-linked file's attributes and chunk list
through KvPut. An HTTP append to a hard-linked file whose KvPut times out
after etcd saved it returns an error, and the upload handler deletes the
appended chunks the saved hard link now points at. KvPut keeps the
caller's context like InsertEntry and UpdateEntry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-13 12:28:48 -08:00
Yip LeoandClaude Opus 5.5 911bfe4014 mount: stop overwriting the mount root's filer entry (#12623)
When a mount is rooted at an existing directory, two code paths replace
that directory's filer entry with one the mount builds itself:

- every mount start re-creates the root with a non-exclusive mkdir,
  which the filer handles as an update of the whole entry;
- creating or removing an entry directly in the root saves the root's
  entry to touch its mtime, and maybeLoadEntry synthesizes that entry
  from the mount options instead of reading the filer.

The filer keeps only crtime and inode on such an update, so the root
loses its mode, owner and extended attributes. When the root is a
bucket it also loses the bucket quota, which silently drops to 0
(unlimited), and the settings S3 keeps on the bucket entry. The explicit
allowEmptyFolders=false that #11370 meant to keep is replaced by true at
every mount start, because the mkdir has wiped it before
ensureBucketAllowEmptyFolders looks.

Create the root only when a lookup says it is missing, and then
exclusively. Leave the root out of the filer-side mtime touch: its
attributes come from the mount options (setRootAttr), so the filer copy
of its mtime was never visible through the mount anyway.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:27:15 -08:00
Chris LuandClaude Opus 5.5 737ef68a11 mount: keep publishing an open file through its surviving hard link (#12609)
* mount: keep flushing an open file whose other hard link survives an unlink

Unlink marks the inode's open handle deleted so no flush recreates the
removed name. When the entry was one of several hard links the inode lives
on under another name, yet the mark stayed and every later fsync and close
returned OK without publishing metadata. Lift the unlink's own mark once
the name is gone and the inode still has a path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: carry a link's stored entry into the file's open handle

Link rewrites the source entry with the new hard link id, count and times,
but an open handle kept its old base and entry. Its next conditional
content flush was refused (EIO), and an unconditional one would have
dropped the hard link id. Move the handle onto the stored entry when only
the link fields and times changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: carry an alias unlink's stored entry into the open handle

Removing one of several hard links lowers the stored link count and
stamps a new ctime, and the handle may still hold the removed name. Look up
the entry at the name the handle flushes through and adopt it, so the next
content flush matches the stored entry, writes the right count, and
attribute saves address the surviving name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test an open handle across link and alias unlink

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: read an alias unlink's lost entry again at the next flush

When the read of the surviving entry right after an alias unlink fails,
the handle kept a base with the old link count and ctime, and every later
flush was refused even after the filer recovered. Leave the hard link id
on the handle and have the next flush read the stored entry and adopt it
before publishing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: publish a queued close before unlinking a name another link keeps

With writeback caching, close only queues the flush. Unlink marked the
handle deleted before waiting for that flush, so the flush skipped its
metadata and the surviving link kept the content from before the close.
When another link keeps the file, wait for the queued flush first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: let a queued close reach the filer before linking the file

With writeback caching a new file is created on the filer by its queued
close, and a listed directory serves the source from the cache, so a link
right after close updated a name the filer did not have yet and failed
with EIO.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: skip a link change no newer than the handle's entry

A link change carried into the handle could replace a base already moved
past it by a later acknowledgment. Use the same clock-domain check as
installAckedEntry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep a link change's ctime in the dirty entry

The flush does not restamp times, so a dirty entry carrying an earlier
write's ctime moved the filer's ctime back from the link or unlink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: flush through the file's current name when the resolved one is gone

doFlush resolves the name before uploading the data. An unlink of that
alias, or a rename, in the meantime sent the publish to a name the file no
longer has.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep the pending link read when its name is already gone

A flush could read the removed alias name, find nothing, and drop the
pending read of the surviving link, leaving the stale base in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test link and alias unlink across queued closes, flush races, versions and ctime

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: pick a flush's name again once its locks are held

An unlink or rename that lands between the name pick and the locks would
otherwise send the flush to the removed name, where the create guard fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test a flush waiting on its lock through an alias unlink

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: check the flush target after every lock acquisition

A flush whose name kept changing published to the name chosen before its
last lock wait. Check after each acquisition and return a retryable error
once the attempts run out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test a flush whose name changes during every lock wait

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:25:50 -08:00
bac2196d65 shell: return flag parse errors from every command (#12618)
* shell: return flag parse errors from every command

48 commands returned nil when their flags failed to parse, so a mistyped
flag printed the flag error but the command still counted as a success:
a piped weed shell run exited 0, the master maintenance loop logged no
error, and admin script jobs reported the line as done.

Return the parse error as s3.bucket.create (#12544) and the other shell
commands already do. TestCommandsReturnFlagParseErrors runs every
registered command with an unknown flag and fails for the 48 on master.

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>

* shell: don't report -h as a failure

With the parse errors returned, -h on those commands now returns
flag.ErrHelp, so the shell printed "error: flag: help requested" and a
piped run exited 1. The flag package has already printed the usage, so
processEachCmd now treats flag.ErrHelp as success for every command.

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>

* shell: keep the -h test beside the flag parse test

TestProcessEachCmdTreatsHelpAsSuccess covers the same change as
TestCommandsReturnFlagParseErrors, so it lives in the same new file and
shell_liner_test.go stays as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Signed-off-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: anant <170962632+antcybersec@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:19:44 -08:00
Chris LuandClaude Opus 5.5 c248ab7865 credential: keep IAM changes made during the legacy-file migration (#12614)
* credential: migrate the legacy IAM file without overwriting newer entries

The migration wrote every identity, service account and group from the
snapshot the load read, over whatever the per-entry files held by then, so
a revoke or delete landing in between was undone for good. It now migrates
only the legacy file's own entries, creates each per-entry file only when
none exists, and waits for the next load when the legacy file changed
after the snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: test changes landing between the IAM load and its migration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: keep a per-entry file deleted after the IAM load deleted

An entry the load read from its own file is already migrated. When that
file is removed before the migration runs, the legacy copy must not
recreate it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* credential: test a deleted per-entry file the legacy file also holds

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:17:45 -08:00
Chris LuandClaude Opus 5.5 12cb707e58 seaweed-volume: copy master.proto with metrics_port (#12605)
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:10:48 -08:00
Chris LuandClaude Opus 5.5 aa901b7ac7 s3tables, iceberg: create missing directories exclusively so a concurrent create keeps its record (#12600)
* s3tables: create a missing directory exclusively in ensureDirectory

ensureDirectory looked the path up and, when it was missing, created it with a
plain CreateEntry. A table create that lost the lookup race then replaced the
directory another create had made, claimed and confirmed since, erasing that
table's record, so a third create could claim the name and win as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* iceberg: create missing metadata parent directories exclusively

saveMetadataBlob created each missing directory on the way to the metadata
directory with a plain CreateEntry after a lookup. A table directory another
create had made and written its record onto since the lookup was replaced by a
bare directory, erasing that record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* iceberg: create missing stage-create marker directories exclusively

The marker path starts at the table bucket directory, which carries the bucket
record; a plain create after a missed lookup replaces a directory another writer
made in between, with whatever it carries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:10:18 -08:00
Chris LuandClaude Opus 5.5 e0beb59ef0 topology: treat a nil ttl as the empty ttl when looking for duplicate volume ids (#12601)
Volume layouts are keyed by ttl.String(), so a nil ttl and the empty ttl
share one layout. ttlEqual told them apart, so a replica reported with a nil
ttl into a layout created with the empty ttl matched its own layout as a
duplicate, and every copy was quarantined.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:09:07 -08:00
Chris LuandClaude Opus 5.5 fe14880f4d shell: tier download replicates the downloaded volume before deleting the remote object (#12599)
* shell: replicate the downloaded volume before deleting its remote object

volume.tier.download downloaded every replica from the remote tier and let
the last one in the master's location list delete the shared object, so a
replica missing from that list kept reading a deleted object.

Download onto one replica, copy it over every other remote-backed replica,
re-read the topology so replicas that show up meanwhile are replaced too,
and only then delete the last remote-backed replica together with the
object before copying the local volume onto it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: test tier download replicates before deleting the remote object

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: copy the downloaded volume onto each replica's own disk type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: test tier download keeps each replica's disk type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: name the hdd disk type when copying a downloaded volume

VolumeCopy treats an empty disk type as the source's, so an HDD replica
copied from an SSD source was placed on SSD or failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: block deletes on the tier download copy source

A readonly-can-delete source took deletes after VolumeCopy read its
index, so the replaced replica lost them and served the files again.
Hard-mark the source readonly while replicas are replaced, then restore
the writable mark or reload the persisted state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: keep the remote object while replicas are unlisted

Tier download deleted the shared object once the listed replicas were
local, even when the replica placement expected more copies than the
topology listed. Keep it in that case so a rerun can finish later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: fail tier download when no replica is listed

Also point the final-replica delete error at volume.fix.replication.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: keep the remote object when a server lists the volume twice

A stale twin on another disk of a replaced server still references the
object but is filtered out with its server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: restore the tier download source when its readonly mark fails

VolumeMarkReadonly blocks local writes and deletes before its second master
notification, so a failed notification returns an error with the source
already frozen. Restore it whenever the status was read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: report a failed tier download source restore

A restore that leaves the source unmounted or frozen was only printed, so the
command still returned success. Join the restore error into the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: wrap the tier download restore error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:07:52 -08:00
Chris LuandClaude Opus 5.5 94c3869057 volume_move: keep the live-move source frozen across volume reloads (#12597)
* volume_move: persist the move's source freeze across reloads

A live move froze the source in memory only, so an unmount/mount or a
volume server restart mid-move reopened it for writes that the move then
deletes with the source. Persist the freeze when the move froze a
writable source, and reopen the target right after the copy since the
copied .vif carries the freeze. An already-readonly source keeps an
in-memory mark, and a failed move still restores the source writable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: live move keeps the source frozen across a reload, on Go and Rust volume servers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: require 409 from the reloaded frozen source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume_move: reopen an interrupted move's source only without a target copy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:05:58 -08:00
Chris LuandClaude Opus 5.5 652648d9c4 mount: merge concurrent content changes from another mount instead of failing or dropping them (#12596)
* mount: merge a refused content flush onto the stored entry

A content flush conditioned on the handle's base failed with EIO whenever a
peer had changed the file since, so two mounts appending to one file could
not both close it cleanly. On a refused condition, reread the entry and
replay this handle's changes since its base onto it: the chunks it added or
dropped, and the attributes, extended keys and inline content it changed.
A local truncation keeps its size and clips the peer's chunks to it;
otherwise the larger size wins. The merged entry is published against the
stored one, up to three times, and installed in the handle so its next
flush keeps the peer's chunks. A replaced or deleted file is still refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: merge a peer's content change into a dirty handle

A subscription event carrying a peer's content change replaced a dirty
handle's entry and discarded its unflushed pages, so the handle's writes
were lost and its close still returned OK. Fold the peer's entry into the
handle with the same merge a refused flush uses, keep the dirty pages, and
take the peer's entry as the new base. A replaced file still invalidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: stop a flush merge once a filer accepts the write

withFilerClient can call the callback on several filers. A refusal from
one left its stored entry behind, so a later filer's success went on to
another merge round.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: refuse a merge that puts inline content next to chunks

A read inside the inline bytes returns them without looking at chunks,
so one side's inline content merged with the other side's chunks hid
data after a successful flush.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: refuse to merge a whole-file rewrite over a peer's chunk change

maybeMergeChunks re-uploads the handle's whole view as unstamped chunks.
Merged after a peer's change, they tie with a peer's own rewrite or bring
back a range the peer truncated, so that flush fails as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: clip peer chunks at the lowest truncate, not the final size

A handle that truncated and then wrote past the old end ends up larger
than its base, so the merge did not see the truncate and kept the peer's
chunks inside the range the truncate cleared. SetAttr now records the
lowest size truncated to over the current base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep a peer's truncate when the handle left the size alone

The merge took the larger of the handle's and the stored size, so a peer
truncate came back as a zero-filled tail when the handle only overwrote
bytes inside it. The stored size now stands unless the handle changed
its size or wrote past it, and rebasing keeps only the growth writes made
after the flush was taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: retry a read view whose merged manifests did not resolve

A failed SetChunks leaves the chunk group answering every read with the
resolution error, and nothing re-ran it while the entry stayed the same.
The handle now marks such a view (on rebase and on SetEntry) and the
next read or lseek resolves the entry's chunks again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep a dirty handle's writes when a peer change cannot merge

The new merge refusals sent a peer event on the same file down the old
path, which discarded the dirty pages and adopted the peer's entry, so
close returned OK without the writes. The handle now keeps its writes and
base, and the flush meets the conflict and fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep an unpublished truncate across a peer's base change

The truncate mark was tied to the base pointer, so a peer event that moved
a dirty handle's base (a merged content change or a metadata-only change)
dropped it, and a later merge on a regrown file kept peer chunks inside
the truncated range. Those base moves now carry the mark; a flush ack or a
discarded handle state still clears it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: wait for the flush before merging a shrink over buffered writes

A write inside the file leaves its bytes in dirty pages without moving
the size. Merging a peer's truncate then adopted the smaller size, and the
chunk uploaded later sat past it, unreadable through the handle. The event
path and the periodic metadata flush now refuse a merge that shrinks the
handle; the close flush uploads the pages first and merges then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: tell a shortened base chunk from a whole-file rewrite

A truncate shortens a base chunk in place, so it gets a new reference
key, and an old chunk can carry no timestamp. The rewrite refusal now
applies only to unstamped chunks whose file id is new to the base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: check the creation time when the base has no inode

An entry without an inode or hard link id matched any replacement at the
same path, so a stale handle could merge into a different file. Without
an inode the merge now needs the same creation time. The filer keeps it
across updates; only a synthesized base has neither and still merges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep a dirty handle's writes when a peer replaces the file

A peer entry with another inode at the handle's path went down the
fallback that destroys the dirty pages and installs the replacement, so
close returned OK without the writes. The handle now keeps its writes
and base like it does for an unmergeable change, and its flush refuses to
update the replacement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: build the content flush tests on freebsd

The invalidation tests, which every platform compiles, now use the
content flush helpers, so the freebsd cross-compile vet failed with
undefined: flushConditionEntry. Nothing in the file is freebsd-specific.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: keep a dirty handle's base when a same-content file replaces it

A replacement with the base's chunks but another inode counted as a
metadata-only change, so the dirty handle adopted it as its base and the
next conditional flush overwrote the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: bring a peer's metadata change into a dirty handle

A metadata-only event moved a dirty handle's base but left its live
entry, so the next flush read the peer's chmod as a local edit and
published the old mode back. Replay the handle's own changes onto the
peer's entry instead, as for a content change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: wait for the flush before merging peer inline content over buffered writes

A buffered write is not a chunk yet, so a merge could install a peer's
inline content that later reads return ahead of the write's chunk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: track a whole-file chunk rewrite on the handle

The merge told a rewrite by its unstamped chunks, which a manifest fold
hides: manifests are unstamped either way. Record the rewrite against the
base it was made from, carried across peer base changes like a pending
truncate, and refuse the merge from that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: refuse a merge that shortens a manifest at the truncate

Clipping a manifest reference leaves its members whole, so peer bytes past
the handle's truncate could reappear once the file grew again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: flush a dirty handle after a peer replaces its file

The replacement test checked only the retained handle state. It now
flushes against a filer holding the replacement, including one with the
base's content, and checks the flush fails and leaves it unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: test an append fsync after a peer's attribute-only change

A second mount's acknowledged chmod, chown, utimes or setxattr leaves
the content alone, so the writer's conditional content flush must still
publish its append and keep that change. On master the refused flush
surfaces as EIO from Fsync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:02:42 -08:00
Chris LuandClaude Opus 5.5 a576398662 weed-vector: advertise its metrics port when registering (#12604)
* master.proto: KeepConnectedRequest carries the node's metrics port

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* weed-vector: advertise --metricsPort when registering with the master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* master.proto: number metrics_port 9

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 22:36:02 -08:00
Chris LuandClaude Opus 5.5 baaa60b24a admin: an arrow icon for vector buckets (#12602)
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 22:11:13 -08:00
Chris LuandClaude Opus 5.5 917cf534db s3api: sign every operation subresource in signature V2 (#12575)
* s3api: sign every operation subresource in signature V2

The V2 canonical resource only kept the subresources inherited from the
original allowlist, so tagging, retention, legal-hold, cors, encryption,
object-lock and the other keys the router dispatches on were left out of
the signature. A V2 signature issued for one request therefore also
verified that request with one of those keys appended, running a
different operation under the signer's identity.

Add every operation-selecting subresource to the signed list. Clients
that already sign tagging, cors, object-lock or replication in V2 now
verify as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: sign every value of a repeated signature V2 subresource

canonicalizedResourceV2 kept only the last value of a repeated query key,
while handlers read the first. A request signed for versionId=A could be
sent as versionId=B&versionId=A and act on version B. Sign every value in
order, as V2 clients do, so a second value no longer verifies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:22:59 -08:00
github-actions[bot] 5ba77af65f docs: regenerate star history chart 2026-11-13 01:20:00 +00:00
c2b2ed769b fix(mount): preserve appended chunks during stale attribute updates (#12593)
* fix(mount): preserve acknowledged chunks during attribute updates

* mount: keep the WFS field block aligned

Adding the publication-lock fields inside the aligned block made gofmt
re-indent 25 unrelated lines. Put them in their own group after
dirIdleEvict, matching the enterprise layout, so syncs do not conflict there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: finish attribute saves through a FUSE interrupt

saveEntryAttributes derived its RPC context from the syscall's cancel
channel, so an interrupt while the conditional UpdateEntry was at the
owner returned EIO for an update the owner then applied, and skipped the
local install and retention. Keep the metadata deadline but drop the
cancellation, like the other metadata mutations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: map attribute CAS ids in place instead of cloning the entry

saveEntryAttributes cloned the whole entry three times per attempt only
to rewrite uid/gid for the filer, and each clone is as large as the
chunk list. Switch the ids in place and restore the exact local values
after the call, keeping one clone. The test now checks the reply carries
the local uid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: compare attribute updates against the filer's own uid/gid

With -map.uid/-map.gid, a filer id that is also a mapped local id reads
back unchanged and maps forward to another filer id. The IF_ENTRY_EQUAL
expected entry then never matched, so chmod, utimes and xattr changes on
such files failed with EAGAIN after three attempts, and an update would
have rewritten the owner. Keep the filer ids read with the entry for the
ids the syscall leaves alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mount): align attribute retention with read lock order

* mount: take the handle lock before entryLock in the attribute snapshot

attributeEntrySnapshot held entryLock while it queued for the shared
handle lock. A Read holding that lock waits on entryLock, the snapshot
waits behind a queued exclusive (an invalidation or a Write), and the
exclusive waits on the Read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mount): complete handle lock order and preserve explicit ID intent

* mount: re-read a refused attribute update from the file's owner filer

The filer checks the IF_ENTRY_EQUAL condition on the owner of the path's
write route key. The retry read any filer, so behind one that had not
applied the owner's latest update, every attempt sent the same stale base
and the syscall failed with EAGAIN. Read from the owner once the lock ring
is known, like lookupFilerEntry, but keep the filer's raw ids.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:28:47 -08:00
d7ae08ecad fix(mount): refuse stale whole-entry content flushes (#12594)
* fix(mount): condition content flushes on the observed entry

* mount: stop retrying a flush the filer refused on its condition

The filer reports a failed CreateEntry condition as FilerError_PRECONDITION_FAILED
in the response, which surfaces as filer_pb.ErrPreconditionFailed rather than a
FailedPrecondition status. The flush retry loop treated it as transient and
replayed the refused flush three more times over 7 s before returning EIO.

The test filer now answers like the real one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: take the handle base from the entry the filer stored

After an acknowledged flush the handle base was the requested entry, but the
filer stores it with a fresh commit_ts_ns (and fills crtime, inode, atime), so
the next IF_ENTRY_EQUAL flush on the same handle never matched: write, fsync,
write, fsync returned EIO, as did close after a periodic metadata flush and the
first flush after an eager create. Use the stored entry from the
acknowledgement's metadata event.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: condition the periodic metadata flush like the close flush

The periodic metadata flush of an open, dirty handle sent the same whole entry
through an unconditional CreateEntry, so a file held open past the flush
interval still overwrote a peer's acknowledged append, and the close flush then
matched the overwritten state. Both paths now publish through one helper that
carries the handle's write condition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: republish a refused flush once when the stored entry is unchanged

A handle's base can differ from the stored entry without any peer write: a
rename while the file is open restamps the entry under its new name, and with
-map.uid a filer id that is also a mapped local id does not map back to itself.
Each made every flush of the handle fail with EIO. On a miss, read the stored
entry and, if it still holds the observed state apart from name, commit stamp,
id encoding and derived size, retry once with that entry as the condition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: take the filer's entry once a write open holds the DLM lock

The lock orders this writer after the previous holder, but the holder's
flush may not have reached the local cache yet, so the handle's base
named the entry from before the wait and the conditional flush refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: share the open-existing path of Create

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: open the file a previous DLM holder created during a deferred create

A deferred create never asks the filer, so a create that waited for the
lock went on to publish over a file the previous holder had just made,
and its flush expecting no entry was refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* master: send the lock ring snapshot to mounts

A mount running -dlm is a lock client like the S3 gateway, and needs the
ring on connect to reach a file's owner filer before the next membership
change is broadcast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: follow the masters' lock ring under -dlm

The lock client only knew the seed filer, so every lock took the forward
hop and the mount had no way to tell which filer owns a file. Subscribe to
the masters named by the filer, like the S3 gateway, and wait briefly for
the first ring before serving.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: read the DLM-ordered entry from the file's owner filer

The filer checks a flush's condition on the owner of the path's write
route key, and another filer may not have applied the previous holder's
flush yet. Look the entry up on that owner once the ring is known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:15:08 -08:00
Chris LuandClaude Opus 5.5 a2b9b37ee5 volume(rust): untrack queued writes before replying (#12595)
* volume(rust): untrack queued writes before replying

The durable write worker sent each write's result and only then dropped
the batch's pending-write trackers, so a caller could see its write
finish while the write was still tracked as pending. Go untracks in the
caller before writeNeedle2 returns. Drop the trackers before sending the
results so the Rust queue path matches.

test_replicate_delete_cancels_a_queued_write asserted the count was zero
right after the fsync write returned and failed about 1 run in 100 under
load (Rust Unit Tests on master c2625dd).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume(rust): untrack a write rejected for a remounted volume before replying

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:11:10 -08:00
Chris LuandClaude Opus 5.5 a2ffdce2ec volume: skip VolumeDelete when its RPC was cancelled before the delete (#12586)
* volume: skip VolumeDelete when its RPC was cancelled before the delete

A caller that cancels (a lost tiering lease, an expired deadline) still
had its admitted delete run in the Go server, including one that waited
behind the copy guard. The Rust server drops a cancelled handler, so
return the context error before the delete starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: wait until the queued delete blocks on the guard before cancelling

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:19:05 -08:00
Chris LuandClaude Opus 5.5 6388dfca9b s3api: verify signed request bodies against the signed payload hash (#12577)
* s3api: verify signed request bodies against the signed payload hash

A SigV4 signature covers the declared x-amz-content-sha256, but only
getRequestDataReader (PutObject, UploadPart) compared the body with it.
Every other handler parsed the body as received, so a signed request,
DeleteObjects for example, could be replayed with a different body.

After a V4 signature verifies, wrap the body so it is handed out only
once all of it has been read and matched against the hash the signature
covered; a decoder that stops reading early never sees unverified bytes.
getRequestDataReader unwraps it and keeps verifying uploads as they
stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: answer XAmzContentSHA256Mismatch for refused DeleteObjects bodies

DeleteObjects answered a body that failed its payload-hash check with
InternalError and CompleteMultipartUpload with MalformedXML. Name the
check that failed instead, as PutObject does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: check Content-MD5 on signed request bodies

Signature V2 signs Content-MD5 rather than a payload hash, and a V4
request sent with UNSIGNED-PAYLOAD may still sign it, but bodies read
outside the upload path never compared it with the body. Check it with
the payload hash once a V2 or V4 signature verifies: BadDigest on a
mismatch, InvalidDigest when it is malformed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: verify presigned uploads against a signed payload hash

A presigned V4 URL can sign X-Amz-Content-Sha256 as a query parameter,
but getRequestDataReader compared upload bodies only with the header of
that name, so the signed hash went unchecked. Verify uploads against the
hash the signature covered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: check Content-MD5 on bodies buffered for their payload hash

IAM and other non-S3 requests signed without x-amz-content-sha256 have
their body buffered and hashed during signature verification. That
buffer counted as already bound, so their Content-MD5 was never checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: check Content-MD5 on directory marker bodies

A PUT of a key ending in "/" stores its body on the directory entry
without going through putToFiler, so its Content-MD5 was never compared
with the body. Answer BadDigest on a mismatch, as putToFiler does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: wrap and unwrap signed bodies through helpers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:17:09 -08:00
Chris LuandClaude Opus 5.5 a3b68aa37c shell: confirm trim survivors still serve the volume before deleting (#12590)
* shell: confirm trim survivors still serve the volume before deleting

volume.fix.replication compared the doomed replica's index with every
survivor and then deleted it unconditionally. A survivor that went away
while the indexes were read still let the delete through, leaving fewer
copies than the check had counted. Re-read each survivor's VolumeStatus
after the index checks and skip the trim if any no longer has the volume.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: test a trim whose survivor goes away after its index read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:32:37 -08:00
Chris LuandClaude Opus 5.5 4cba4eb7bf s3: ignore unsigned object headers on POST Object (#12585)
* s3: ignore unsigned object headers on POST Object

The POST policy signs form fields only, but the upload read tagging,
user metadata, storage class, object lock and the stored content
headers from the request headers as well. Drop them before the form
fields are applied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: test POST Object request headers against signed form fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:28:33 -08:00
Chris LuandClaude Opus 5.5 249f04e63a s3api: NoSuchKey for object lock calls on missing or deleted keys (#12592)
* s3api: answer PutObjectLegalHold on a missing key with NoSuchKey

In a versioned bucket a key with no latest version resolves to
ErrLatestVersionNotFound, which the legal hold handler did not map, so
the request failed with 500 InternalError. PutObjectRetention already
maps it to NoSuchKey.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: keep failed reads distinct when resolving an object lock target

getObjectEntryForUpdate turned every lookup error into a not-found
sentinel, so a failed filer read made PutObjectRetention and
PutObjectLegalHold answer NoSuchKey for an object that exists. Only a
genuine miss becomes the sentinel now, matching getObjectEntry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: answer object lock calls on a delete marker with NoSuchKey

PutObjectLegalHold and PutObjectRetention on a key whose current
version is a delete marker returned 200 and wrote the lock onto the
marker, and the matching GETs read it back. A key whose current version
is a delete marker does not exist, and delete markers take no lock, so
these calls now answer NoSuchKey like the tagging and ACL handlers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: let a lock recorded on a delete marker not block its removal

Delete markers are not WORM-protected, but a legal hold or retention
written onto one (possible before the previous change) made both the
gateway check and the routed delete's filer-side WORM guard refuse to
remove the marker, so the object could not be restored and lifecycle
could not clean the expired marker up.

Lock enforcement now ignores a lock on a delete marker, and the routed
version deletes in DeleteObject and lifecycle drop the WORM guard when
the version they read is a marker. A version id never changes kind, so
the marker read is the one deleted; the null version keeps its guard
because a PUT can replace it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:27:12 -08:00
Chris LuandClaude Opus 5.5 4ee82977af docker: pass entrypoint arguments to weed without splitting them (#12589)
The master, volume, volume-rust, server, mini, filer and s3 cases and the
fallback forwarded the user arguments as an unquoted $@, so the shell split
each one at whitespace and globbed it against /data. An -admin.password with
a space reached weed as two words, and weed ignored every flag after the
stray word. Quote "$@" as the other cases already do; $ARGS stays unquoted
since the script builds it from its own words.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:25:25 -08:00
Chris LuandClaude Opus 5.5 bef1b13b3d replication: keep a local sink path the source recreated after a delete (#12581)
* replication: let the local sink check a delete against the live source

LocalSink removed the destination path for any delete it was handed. A
delete that reaches the sink after the source recreated the path (a peer
filer's event delivered late, or a historical event replayed) removed the
newer file, and no later event brings it back.

DeleteEntryVersion receives the deleted entry. Before removing an existing
destination, the local sink looks up the source path and keeps the
destination when the source holds an entry of the same kind committed
after the deleted version (commit_ts_ns, or mtime for unstamped entries).
A live entry that is the deleted version itself or older means the source
store lags the event, so the delete proceeds. A failed lookup returns the
error so the event is retried instead of deleting a file that may be live.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.sync, filer.replicate: hand the deleted version to the sink

Pure deletes, renames without a native move and moves out of the
watched directory now go through sink.DeleteEntryVersion with the old
entry, so a sink that checks the live source (the local sink) can keep a
path the source recreated after the deleted version. Other sinks fall
back to DeleteEntry unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* replication: order the local sink check by the delete's mutation time

A source filer can hold a version written concurrently with the delete:
newer than the deleted entry but older than the delete itself. The
source's own replay applies the delete over it, so keeping the backup
copy because it outranks the deleted entry left a file the source no
longer has.

The sink now gets the delete's mutation time (MutationTsNs, else TsNs)
and keeps the destination only when the live entry was committed after
it, the order Filer.Replay keeps. That also covers a stamped replacement
of an unstamped entry whose mtime did not move. filer.replicate events
carry no time, so they keep the deleted entry's stamp as the bound, and
mtime only when nothing is stamped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* replication: count an unstamped live entry from its mtime

A source filer that predates commit stamps stores entries with no
CommitTsNs, so comparing the live stamp with the delete's time never
kept a replacement there. An unstamped live entry now counts from its
mtime against the delete's time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* replication: keep a stamped live entry when the deleted one is unstamped and the delete has no time

filer.replicate gets no delete time from the notification queue. When the
deleted entry has no commit stamp either, a stamped live entry was written
after it, so its mtime no longer decides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:23:54 -08:00
Chris LuandClaude Opus 5.5 c2625ddf00 seaweed-volume: read security.toml values as TOML strings (#12582)
The line parser only trimmed quotes, so `key = "abc\n"` gave the Rust
volume server the bytes `abc\n` with a literal backslash while the Go
servers sign with a newline, and every JWT failed. Literal strings kept
their quotes and a trailing comment stayed part of the value.

Decode each value with the TOML parser; text it rejects keeps the old
quote trimming.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:19:18 -08:00
Chris LuandClaude Opus 5.5 b7d6b48921 operation: probe octet-stream uploads for gzip like untyped ones (#12584)
* operation: probe octet-stream uploads for gzip like untyped ones

The 128-byte gzip probe ran only when the caller passed no MimeType, so an
upload labelled application/octet-stream or binary/octet-stream was never
compressed. S3 PutObject and UploadPart pass the request Content-Type (or the
octet-stream that mimeDetect fills in), so every binary S3 upload skipped
compression that the same bytes get through the filer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* operation: check the probed upload keeps the caller's content type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:16:55 -08:00
Chris LuandClaude Opus 5.5 02b9747bd8 wdclient: keep a shared volume lookup alive when its first caller leaves (#12579)
* wdclient: keep a shared volume lookup alive when its first caller leaves

Concurrent misses for the same volumes share one provider lookup, but it
ran on the first caller's context, so that caller's cancellation failed
every request waiting on it. The lookup now runs on a context no single
caller can cancel, and each caller stops waiting when its own context ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* wdclient: wait for the second caller to join before canceling the first

A fixed sleep did not prove the second caller had joined the shared lookup:
a late caller could hit the filled cache and pass without waiting. The test
now waits until the caller reads its own Done channel, which happens only
after DoChan registered it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:15:01 -08:00
Chris LuandClaude Opus 5.5 8581151376 filer: gzip metadata log pieces before uploading them (#12580)
* filer: let the fake-master test helper serve any master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: gzip metadata log pieces before uploading them

Every metadata event carries the old and new entry with their whole chunk
lists, so a file grown by small fsynced appends logs the same chunk records
again and again: 2000 appends of 256 bytes through weed mount wrote 34.8 MB
of log. The uploader only compresses when the first 128 bytes compress, and
a log piece starts with fresh file ids and ETags, so it stored the log as is.

The filer now gzips each piece itself and keeps the compressed bytes when
they save at least 10%. After the same workload the volumes hold 3.05 MB
instead of 35.2 MB, file data included. Readers already honor the chunk's
IsCompressed flag on every path (filer-side decode, log file refs read
straight from volume servers), and the assign request sizes the bytes
actually stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:13:50 -08:00
Chris LuandClaude Opus 5.5 e0f8438284 mount: retry a writeback metadata flush until the filer answers (#12578)
* mount: keep the filer status in the flush create error

flushMetadataToFiler formatted the CreateEntry error with %v, so callers
classifying it saw only the message, which includes the file path. Wrap
it so the gRPC status the filer sent stays reachable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: retry a writeback metadata flush until the filer answers

With -writebackCache, close() returns before the chunk references reach
the filer. The deferred flush gave up after four attempts, about a minute
once each attempt's own filer retries are counted, then released the
handle: a longer filer outage dropped the file even though write() and
close() had succeeded, and its uploaded chunks became orphans.

Keep retrying for as long as the filer is unreachable, as the writeback
chunk uploads already do. A filer that answers with an error still ends
the flush after the bounded attempts, and an unlink of the draining file
still ends it on the next attempt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: retry only an unanswered filer call without a bound

The unbounded phase used util.IsTransientError, which falls back to
matching the error text. A refusal the filer sends in its response
(read-only, name too long, a plain error string) has no gRPC status, and
the flush error carries the file path, so a file named like
"unavailable-transport.log" kept retrying against a filer that answered.

Retry without a bound only when the call failed with a gRPC Unavailable
status: the dial, the connection or the mount's call deadline gave out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 08:57:42 -08:00
Chris LuandClaude Opus 5.5 a6a3bdef8d s3api: deny signed reads of a version file through its physical path (#12574)
* s3api: deny signed reads that name a version file by its physical path

A stored version lives at <key>.versions/v_<id>, and that path is also a
valid object key. Anonymous reads already refuse it; signed GET, HEAD,
copy sources and object subresources did not, so a caller without
s3:GetObjectVersion, or under a Deny on the logical key, could read any
version through it. Apply the same check in the shared entry recheck.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: recheck a version file key before answering a conditional read

A conditional GET or HEAD answered 304 or 412 with the ETag before the
handler's entry recheck, so the physical version path still confirmed a
version's ETag to signed callers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: look up version file keys in the lookup-based policy recheck

Retention, legal hold, tagging writes and deletes only fetched the entry
for the recheck when the bucket had a policy, so without one they still
acted on a version file named by its physical path. Fetch it whenever
the key has the version file shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: refuse a directory-marker delete that names a version file

A trailing slash kept "<key>.versions/v_<id>/" out of the version file
shape check, and the lookup recheck misses the entry for that form, so a
DELETE reached deleteDirectoryMarker and removed a chunkless version or
delete marker without s3:DeleteObjectVersion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 08:21:31 -08:00
Chris LuandClaude Opus 5.5 317d907334 filer: route tests forward to a peer port the test holds (#12576)
The ring tests used 127.0.0.1:18889 as an unreachable peer, but its gRPC
port 28889 is a common filer port: on a shared CI host a live filer there
accepted the forwarded exclusive create, and
TestCreateEntryExclusiveLeavesNonOwner failed. The peer now listens on a
port the test owns and drops every connection.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:10:58 -08:00
github-actions[bot] d51a79335c docs: regenerate star history chart 2026-11-12 00:47:35 +00:00
Chris LuandClaude Opus 5.5 13334a59b9 vector: reuse the Lance session, version leases and opened tables across reads (#12573)
* vector: share one Lance session across reads

Each request opened its table in a session of its own, so every query
loaded and decoded the index again. Get, list and query now open in one
session the server keeps, sized by -index-cache-mb and
-metadata-cache-mb; writes keep a session of their own.

Lance keys some of its cached entries, such as a version's index list,
by location and version alone, and an index dropped and created again at
its location numbers its versions from 1 again: a shared session answered
a query on the new table with the old table's index. CreateIndex now
marks each table with an id in its schema metadata, and the session
starts over when a location turns up holding another table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: share a version lease among the reads of that version

Every get, list and query tagged the version it read and deleted the tag
afterwards: two filer writes and a manifest read per request. Reads of
one version now share one tag, which goes 10 s after the last read took
it; a tag is reused for at most 5 minutes, so a busy version never
reaches the reapers' TTLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: keep opened tables for the next read

A read built the table's store and read its latest manifest each time.
Reads now keep the opened table by location and encryption, up to 1024,
and move it to the latest version, which lists the versions and takes
the manifest from the session. A handle opened in an earlier session is
not reused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: key shared leases by manifest, and take one per version at once

A table dropped and created again within the linger reused, for a
version number the old table had, a lease whose tag the drop had
deleted. Leases are now keyed by the manifest's ETag as well. Reads that
arrive before a version has a lease wait for the first to take it,
rather than each writing a tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: drop kept tables when the shared session starts over

A kept table holds the session it was opened in, so a replaced session
and its caches lived on in the handles of other tables until they were
evicted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: prune shared lease slots only when they double

Every lease lookup scanned all slots under the map's lock, a cost that
grew with the versions read in the last linger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: shrink the lease slot map after a prune

After a burst the map kept its peak allocation, which every later prune
scanned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

* vector: keep a lease slot a read has taken through a prune

A read that took a slot and had not yet locked it could see the slot
pruned, and then pin through it while the next read pinned through a
new slot, writing a second tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UpvafvK8JpagF94MHWqwK

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 16:26:50 -08:00
Chris LuandClaude Opus 5.5 b924ffabf1 admin: calm console restyle with light and dark themes (#12568)
* admin: restyle the UI as a calm console

Grey canvas with white sheets, a docked sidebar that marks the current
page with an indigo tint, ink primary buttons, tinted status pills,
dense tables with tabular numbers and spectrum capacity meters. The
styles are design tokens layered over Bootstrap's own variables, with
light and dark values, so the page templates stay unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: add a light, dark and automatic theme switch

The choice is kept in localStorage and applied before the page paints,
on the sign-in page too; automatic follows the system setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: drop hard-coded colors from page templates

The volume id link, utilization track and plugin form borders used
fixed light-theme colors; sparklines now use the data palette.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: theme the plugin job detail panel background

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep the selected radio dot visible in dark mode

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: cycle the theme from memory when storage is blocked

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep success progress bars green

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep keyboard focus visible on the toolbar togglers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: use contrasting text on danger and success buttons in dark mode

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: lead the dashboard with cluster health and one row of tiles

The header now says in one line whether the cluster is healthy and how
many masters, volume servers and filers report. The two rows of stat
tiles become one: the volume size limit and EC volume count move into
footnotes, the mount client card joins the cluster counts, and the
counts link to their pages instead of sitting in nested cards. The
duplicate version list and the bucket shortcut in the header are gone,
and the section menus become plain View all links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: export the user columns the table actually has

The users table has username, email and access key columns, but the
export waited for six cells per row and so always produced only the
header line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: drop table titles that repeat the page title

List pages titled their only table with the page title again. The
table now sits right under the summary tiles, and the export actions
that were hidden in the table header menu move to the page header as
an Export button, matching the cluster pages. The policies menu linked
to nothing and is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: drop clock and duplicate summary tiles

Buckets, users, policies and service accounts spent a summary tile on
the refresh time, which the page footer already shows (policies gains
that footer). The users page also showed its user count twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: show unset bucket settings as a dash

Rows of No owner, No quota and Not configured made the bucket table
hard to scan. Unset cells now show a quiet dash that keeps the wording
as a tooltip and for screen readers, and the name and creation date no
longer wrap (the full timestamp is in the date's tooltip).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: export buckets through the page's API exporter

admin.js loads after the page, so its copy of exportBucketList replaced
the buckets page's own one. That copy scraped the visible rows, so it
exported only the current page, without CSV quoting, and picked up the
display text of the cells. The page's exporter reads every bucket from
the API with full timestamps and escaped fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: escape quotes in the user export

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: download the bucket export as a Blob

The exporter put the CSV into a data URL with encodeURI, which leaves #
alone, so a bucket owner containing # ended the download there. Use the
shared downloadCSV helper instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: count masters as configured, not reporting

The master list comes from raft membership, which keeps a follower
that is down, so it does not show which masters are reporting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: link the Prometheus metrics endpoint from the sidebar

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: drop the sidebar metrics link

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:32:48 -08:00
Chris LuandClaude Opus 5.5 1618d613a3 s3: resolve a role session's aws:username from its user, not its session name (#12572)
* s3api: record the caller on AssumeRole sessions

The session name is the caller's choice; the session now carries the
caller itself as the user it was issued to, as the web identity, LDAP
and credential paths already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: take a role session's aws:username from its user, not its name

An assumed-role ARN ends in the session name the caller picked, so
naming a session after another user resolved ${aws:username} and
${aws:userid} to that user in bucket and attached policies. Resolve
them from the user the session was issued to instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* iam: take a role session's aws:username from its user, not its name

IsActionAllowed set aws:username to the last segment of an assumed-role
ARN, the caller-chosen session name, for role and bucket policies. Use
the user the session was issued to, and drop a forwarded value when the
session names none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: pass a role session caller's user on to the session it assumes

A role session that calls AssumeRole is named by its session ID, so the
new session recorded that ID as its user. An STS session caller now
passes on its own recorded user, or none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: size the session claims map from the request context

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:32:13 -08:00
Chris LuandClaude Opus 5.5 3f557b73c8 admin: show the right master ports when -port.grpc is not port+10000 (#12570)
* admin: take master HTTP ports from raft member names

The masters page and dashboard turned each raft member's gRPC address
into an HTTP address by subtracting 10000 from the port, so masters with
a custom -port.grpc showed wrong or negative ports. Raft members are
named after their master address, which carries the HTTP port; use it
and keep the old derivation only for names that are not addresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: test raft member address mapping

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:22:29 -08:00
Chris LuandClaude Opus 5.5 a0f95e16bc s3api: fix TestRoutedMultipartFinalize flake from a reused fake filer port (#12571)
* s3api: drop cached gRPC channels when a fake filer takes a port

A test that dials a dead owner leaves its failed channel in the
process-wide gRPC connection cache. When a later test's fake filer is
given the same port, its first RPC fails at once with the cached
connection refused, which is how TestRoutedMultipartFinalize failed on
the 32-bit job. The fake filer helpers now share fakeFilerAddress, which
invalidates any cached channel for the new address.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test a fake filer on a dead owner's reused port

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: drop cached channels for dead test filer addresses too

closedFilerAddress can be handed a port whose cached channel belonged to
a fake filer that just stopped and may still look ready; then the dead
owner's transaction is sent on it instead of failing over. Build its
address with fakeFilerAddress as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:50:16 -08:00
Chris LuandClaude Opus 5.5 6c1eec8703 s3: store the POST Object acl on the uploaded object (#12567)
* s3api: split upload ACL assembly out of preparePutObjectACL

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: store POST Object acl and grant fields on the object

The form acl and x-amz-grant-* fields were authorized but never assembled,
so the object kept no ACL: acl=private on a public-read bucket stayed
anonymously readable and invalid values were accepted. Build them with the
same rules as PutObject, which also covers the Block Public Access check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test POST Object stores its upload ACL

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: take the POST Object ACL from the signed form only

ACL request headers are not covered by the POST policy, so a holder of a
signed form could add one and have it stored with the upload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: reject a POST acl field the policy does not cover

The acl form field now sets the stored object ACL, so like the x-amz-*
fields it needs a policy condition, as AWS requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test unsigned ACL headers and uncovered acl fields on POST

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: evaluate POST upload policies on the canonical ACL

Policy conditions on the grant headers saw the raw form value, so an
equivalent spelling of a denied grantee passed. Use the PutObject request
shape that the PUT path evaluates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: check bucket policy for POST ACLs with auth disabled

PutObject applies the bucket policy to an explicit upload ACL when IAM is
off; POST now does the same before storing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: reject differing acl and x-amz-acl POST fields

Both promote to the canned ACL header, so map order picked the stored ACL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test POST grant canonicalization, auth-off ACL policy, acl conflicts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: check the auth-off POST ACL policy after loading the bucket

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: reject an empty acl or x-amz-acl POST field beside a differing one

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test the auth-off POST ACL policy on a cold bucket cache and empty acl aliases

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: load the bucket before per-key object authorization

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test the POST ACL policy on a cold bucket cache with auth on

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: stop per-key object authorization when the bucket fails to load

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: test the cold-bucket POST ACL allow path and a failed bucket load

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:36:51 -08:00
Chris LuandClaude Opus 5.5 eb159f3181 lance: deregister and re-register need UpdateTable (#12552)
* lance: require UpdateTable to deregister a table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: require UpdateTable to register over an existing table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: test that GetTable alone cannot deregister or re-register a table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: a register that finds a table the caller cannot read needs UpdateTable to bring it back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: decide declare write access by whether the table existed, not by what the caller can read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: test register and declare by a caller that cannot read the table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: register clears the marker on the create branch only after checking it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* lance: keep a register whose table was created when the marker check fails

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:35:00 -08:00
Chris LuandClaude Opus 5.5 4cb52d7705 filer: reopen a leveldb store wedged by a transient write error (#12565)
* filer/leveldb_handle: reopen a database whose journal latched a write error

goleveldb keeps the first journal write error and returns it for every
later write until the database is reopened, so one transient ENOSPC or
EIO fails all writes until a restart. The handle offers the leveldb.DB
methods the stores use; a write that fails on an OS error probes the
storage and, once it accepts writes and no call holds the database,
closes and reopens it and retries the write once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: open the leveldb stores through leveldb_handle

leveldb, leveldb2 and leveldb3 now recover from a transient storage
write error without a restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: leveldb stores return iterator errors from listings and deletes

A listing on a database that cannot be read reported an empty
directory, and DeleteFolderChildren wrote an empty batch, because the
iterator error was never checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/leveldb_handle: test a reopen that fails after the probe passes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/leveldb_handle: probe storage before draining readers

A probe that stalls in Create, Write or Sync held mu exclusively and
blocked every read on the handle. Reopen attempts are now serialized by
their own mutex, and the probe runs before the drain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/leveldb_handle: hold new calls back while a reopen drains

TryLock succeeds only at an instant with no reader, so overlapping
listings could keep a reopen from ever draining while each write spun
for the full drain timeout. The drain now holds new calls back for at
most drainTimeout; a call nested inside a running one waits that out
instead of deadlocking. A drain that times out holds off the next
attempt for busyReopenInterval, so new calls are not held back over and
over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/leveldb_handle: bound the wait for a running reopen

A write that hit the latched error waited on the reopen mutex for as long
as another caller's storage probe ran, so a stalled probe queued every
writer. Callers now wait at most reopenWait and return their error, and
still retry once a concurrent reopen succeeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/leveldb_handle: tests wait for the drain and readers explicitly

The nested-read test polled for a drain that could start and end between
polls and then spun forever; it now syncs on the probe, widens the drain
and fails if the put returns first. The overlapping-readers test waits
until every reader holds an iterator before it writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:23:19 -08:00
Chris LuandClaude Opus 5.5 5150168368 s3: authorize an empty versionId as the current-object action (#12566)
* s3: authorize an empty versionId as the current-object action

GET, HEAD and DELETE treat an empty versionId as the current object, but
the action resolver keyed on the parameter's presence and authorized it
as GetObjectVersion/DeleteObjectVersion, so a Deny on GetObject or
DeleteObject did not apply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: test an empty versionId against a version-only policy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: cover a bare versionId on HEAD

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:21:28 -08:00
Chris LuandClaude Opus 5.5 0c6c6868de volume: refuse a tier upload whose .dat was written during the copy (#12562)
* volume: refuse a tier upload whose .dat grew during the copy

AttachRemoteDatFile only checked that the data backend was still the one
copied, so a needle write acknowledged after the copy read the .dat was
missing from the remote object, and the upload then switched reads to it
and removed the local .dat. Compare the .dat size with the copy under the
same data-file locks; on a mismatch the handler deletes the copy and keeps
the local file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* seaweed-volume: refuse a tier upload whose .dat grew during the copy

Mirror the Go AttachRemoteDatFile check. The tier-up attach took the
write lease only inside each step, so a needle write acknowledged after
the upload read the .dat was missing from the object the volume then
switched to, and the local .dat holding it was removed. Hold one write
lease across the instance, revision and size checks, the remote-file
update, the .vif save and the backend swap, as tier-down does; a size
mismatch aborts, deleting the object and keeping the local .dat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: compare the physical .dat size before a tier attach

DiskFile pads its size past an unaligned tail, which a read-only load
keeps, while the upload copies the physical bytes. Stat the file itself
so such a volume is not refused as written during the copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:19:06 -08:00
dc03c3b1f7 shell: parallel moves and a per-run limit for volumeServer.evacuate (#12561)
* shell: parallel moves and a per-run limit for volumeServer.evacuate

-maxParallelization runs up to N EC shard moves at once (default 1), with the
ec.balance scheduler: never two shards of one volume at a time, free workers
go to the least busy targets. The moves are planned first with the same target
choice as before, then applied through volume_move.Mover.MoveEcShards, which
verifies the target registered a shard before deleting it on the source.

-movesPerExec limits how many volumes and EC shards one run moves (default 0
for unlimited), like -volumesPerExec in volume.balance, so a big server can be
emptied in several runs.

With -skipNonMoveable a shard that fails to move (a read error on a failing
disk, for example) no longer drops the other shards of its volume; they are
moved and the failures are listed at the end. Without it the run stops at the
first failure as before. Losing the admin lock stops the run in either mode.

Refs #12501

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: plan evacuation again after failed moves; fail on a lost lock

A failed shard move kept the target slot it was planned on, so a readable
shard that found no destination in the plan stayed on the server although
the room was never used. Failed moves now give their targets back to the
model and the shards without a destination are planned again, up to
maxEcEvacuationRounds times; the failed shards are not tried again.

With -skipNonMoveable a lost admin lock was printed as a skipped failure and
the run returned success. ec.ErrLockLost is now exported and always fails
the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: volumeServer.evacuate reports -movesPerExec after an applied EC run

The round loop only checked the limit after a failed move, so an applied run
that stopped at -movesPerExec without failures never said to run it again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: volumeServer.evacuate plans again for as long as failed moves leave room

The ten-round cap left readable shards on the server once more than ten
rounds of unreadable shards had taken the free room first, and a rerun
fails the same shards first again. Every round moves or fails each shard it
plans and a failed shard is not planned again, so the rounds end without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: volumeServer.evacuate gives a failed move back every slot it took

When the target already held the shard in another disk type, adding it to
the planned disk type still took a slot, but the undo gave back only the
explicit one. Record the slots the plan took and restore exactly those.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ec: an evacuation move that landed on its target keeps the target slot

A move can fail after the target registered the shard, when only the source
unmount or delete failed. Evacuate then gave the target's slot back and could
plan another shard onto a full server. The runner now checks the target after
a failed move and leaves a shard the target has out of FailedMoves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: volumeServer.evacuate gives back the slot of a failed move onto a shard the target had

A target that held the shard before the plan reports it after any failed
move, so the runner took such a move for one that landed and evacuate kept
the slot the plan had taken. The runner now lists those moves in
LandedMoves, and evacuate undoes the ones whose target had the shard, then
plans again with the room they gave back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-11 10:18:51 -08:00
Chris LuandClaude Opus 5.5 fcfca3fee3 s3, filer: bound the wait for an upload slot (#12559)
* util: add WaitUntil, a cond wait bounded by a context and a timeout

The caller holds the cond lock. Cancellation and the timeout wake the
waiter through a broadcast taken under the same lock, so no wakeup is lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: bound the wait for an upload slot

A PUT that arrived while the concurrent upload limits were exhausted waited
on the condition variable with no bound. The handler reads nothing from the
body while it waits, so a client that gave up and reset its connection went
unnoticed: the socket and goroutine stayed pinned until a slot freed, and a
stalled volume server grew the gateway's fd count by one per aborted PUT.

The wait now ends on InflightUploadDataTimeout with 503 SlowDown, which S3
SDKs retry, and on request cancellation. An admitted upload takes its slot
under the lock, and a finished upload broadcasts, so freed bytes admit every
waiter that fits instead of one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer: bound the wait for an upload slot

The filer's PUT/POST upload limits queued requests the same unbounded way.
Reuse the bounded wait: reject with 429 after InflightUploadDataTimeout, as
the volume server does, and stop on request cancellation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* command: add -inflightUploadDataTimeout for the S3 gateway and the filer

weed s3 -inflightUploadDataTimeout, weed filer -inflightUploadDataTimeout and
-s3.inflightUploadDataTimeout, weed server and weed mini
-filer.inflightUploadDataTimeout and -s3.inflightUploadDataTimeout. The
default of 60s matches the volume server's flag of the same name; 0 keeps
the old unbounded wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: reject a queued upload without reading its body

WriteErrorResponse drains the request body before answering. Behind the
stats recorder the drain has no read deadline, and reading the body sends an
Expect: 100-continue client the go-ahead to upload what is about to be
discarded. Answer the rejection with an empty-body copy of the request so
the drain is skipped; net/http closes the connection when a large unread
body remains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3, filer: release an upload slot under the cond lock

A waiter checks the counters under the cond lock and then waits. A release
that decremented and broadcast without the lock could land between the two,
so the waiter slept with a slot free until the next release or its timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3, filer: close the HTTP/1 connection when rejecting a queued upload

net/http reads up to 256 KiB of an unread request body before sending a
keep-alive reply, so a client that stalls its body held back the 503 or
429 until the connection idle timeout. Connection: close skips that read.
HTTP/2 is left alone: there it would shut down the whole connection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:17:18 -08:00
Chris LuandClaude Opus 5.5 d659228338 shell: s3.clean.uploads frees an open upload under the object write lock (#12558)
* shell: s3.clean.uploads deletes uploads over filer gRPC

Same DeleteEntry the HTTP handler runs, without minting a filer JWT,
and the gRPC path is the one a locked, routed delete needs next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: s3.clean.uploads frees an open upload under the object write lock

The cleaner deleted a stale upload recursively with no lock, so a
CompleteMultipartUpload could pass its upload-exists check after the
part entries and their chunks were gone, and publish an object over
freed chunks. Do what AbortMultipartUpload does: hold the object's
distributed write lock, and free an open upload in an ObjectTransaction
routed to the object's owner filer, conditioned on the object not
carrying the upload; if it does, delete the metadata only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: s3.clean.uploads keeps the object write lock for the whole cleanup

A large upload's recursive delete can outlast one 15 second lease, and
a completion on the distributed-lock path could then take the lock and
commit over chunks being freed. Renew the lock in the background and
skip the delete once it is lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: s3.clean.uploads locks the path a completion commits under

For a trailing-slash key a non-versioned completion writes and locks the
entry inside the directory the key names, not the object path, so the
cleaner's transaction did not serialize with it. Lock and check the
same path the completion uses for the bucket's versioning state; the
route key stays the object's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3api: abort locks the path a completion commits under

A routed abort locked and checked the object path, but for a
trailing-slash key a non-versioned completion commits under the entry
inside the directory the key names. Use the same path the completion
locks for the bucket's versioning state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: end s3.clean.uploads bucket failure line with a newline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cluster: NewBlockingLongLivedLockWithin bounds the wait on a held lock

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: s3.clean.uploads skips an upload whose object lock stays held

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* cluster: share the contended lock filer setup across lock tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: s3.clean.uploads reads the bucket's versioning state per upload

The commit path a routed completion locks depends on versioning for a
trailing-slash key. Reading it once per bucket before the run left a
window as long as the run; read it just before the delete, as the
gateway's abort does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:16:20 -08:00
Chris LuandClaude Opus 5.5 b11af30cb1 mount: remove the local control socket when the mount exits (#12560)
* mount: remove the local control socket when the mount exits

weed mount listened on /tmp/seaweedfs-mount-<hash>.sock and removed it only
when a later mount of the same directory started, so every unmount or
SIGTERM left one socket file behind. Remove it on the normal return path and
in the interrupt hook, unless a later mount of the same directory has
already replaced it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: remove the control socket before a failed mount exits

glog.Fatalf and glog.Exitf call os.Exit, which skips the deferred removal,
so a mount that failed after creating its socket (mount point not
available, fusermount failure) still left it behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* mount: register the socket cleanup hook before mount setup

A SIGTERM that arrived while the mount was still setting up, for example
waiting on the filer to create the mount root, exited through grace without
running the deferred cleanup. Registering the hook right after the listener
is created makes it run last, after the unmount and filesystem hooks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:11:58 -08:00
Chris LuandClaude Opus 5.5 7f18576798 filer/etcd: serve volume.fsck from one etcd revision (#12555)
* filer/etcd: scan entries from one revision

volume.fsck walked etcd one directory at a time, so a file moved from a
directory not yet listed into one already listed was seen under neither
name and its chunks were reported as orphans. The etcd store now
implements filer.EntryScanner: every page is read at the revision of the
first read, and a compacted revision fails the scan instead of falling
back to the walk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: read hard links at the scan revision

Hard-link records resolved during a scan were read at the current
revision, so a link updated meanwhile mixed two instants. Reads made with
the scan's context now use its revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: list etcd among the stores volume.fsck reads from a snapshot

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: skip hard-link records in a scan

Hard-link records live beside the entries under their random ids, so a
root scan could read one as an entry with a path made of random bytes,
which the traversal stream cannot send. Skip hard-link ids and keys whose
directory is not valid UTF-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: recognize a hard-link record by the id it stores

A hard-link id can be valid UTF-8, so the key shape alone let some
records through a root scan as phantom files. The record's value is the
shared entry, whose HardLinkId is the record's own key; no entry key
can equal its own hard-link id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: scan entries under an invalid-UTF-8 directory

The filer sanitizes names but stores directories as given, so a path
written over HTTP can leave entries the listing never reaches. Skipping
them let volume.fsck count their chunks as orphans. Send them under the
sanitized directory instead, the substitution the filer uses for names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer/etcd: fail a scan whose sanitized path names another entry

An entry under an invalid-UTF-8 directory is reported under the sanitized
directory. When that path is also another stored entry at the scan revision,
or another sanitized entry in the same scan, acting on the reported path
(volume.fsck deleting it) would reach the other file, so the scan fails and
names the stored path instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:11:21 -08:00
e23a952878 ec: don't index ev.Shards[0] when the last shard was unmounted (#12538)
* ec: don't index ev.Shards[0] when the last shard was unmounted

* ec: skip a runtime emptied before the scrub merge reads its shards

FindAllEcVolumes releases each location lock before MergeEcRuntimes runs,
so an unmount can leave a merged runtime with no shards and the width
scan indexed Shards[-1].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ec: treat a scrub merge with no shard left as a vanished volume

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ec: read Shards once in ShardSize so a concurrent unmount cannot empty it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ec: read each runtime's Shards once in the scrub merge

The vanished check and the slot construction read Shards separately, so
an unmount between them produced a non-nil view with no shards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ec: anchor the scrub merge on a generation that still holds shards

An unmount can empty the newest runtime while an older one keeps its
shards, which left a merged view anchored on the emptied generation with
no slots and the holder reported as skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Mansur Akhmadi <10739460+mansur-akhmadi@users.noreply.github.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:57:39 -08:00
Chris LuandClaude Opus 5.5 641ab6204b shell: fs.mergeVolumes leaves in-progress multipart parts in place (#12556)
* shell: fs.mergeVolumes leaves in-progress multipart parts in place

CompleteMultipartUpload reads the part entries and later commits the
object with their fids, guarded only by the upload directory. A merge
that moved a part chunk in between deleted the source needle the new
object pointed at. Parts stay where they are until the upload completes
or is cleaned, and their chunks still count as referenced sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: note in fs.mergeVolumes help that upload parts stay in place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: match upload parts when the buckets folder is the root

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: count chunks behind a part's manifest as referenced sources

A skipped part whose chunks on a merge source sit behind a manifest was
not counted, so the merge printed no skip line and warned that nothing
references the source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: keep a part's direct source references when its manifest is unreadable

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: count chunks resolved before an unreadable part manifest

ResolveChunkManifest returns the data chunks of earlier manifests along
with a later manifest's error; record them before returning so the
unreferenced-source warning does not fire for a source they reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: test the partial manifest result without a timing delay

The unreadable manifest is now nested under a readable part manifest, so
the first level resolves fully before the failing lookup runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:55:03 -08:00
Chris LuandClaude Opus 5.5 e9606c8ecc filer.meta.backup: resume an interrupted metadata copy and add -entriesPerSecond (#12549)
* filer_pb: add a depth-first traversal that can resume after a path

TraverseDfs walks a tree in listing order and reports each finished
path, so a caller can save one path and continue from it later. Pages
are read whole before visiting, and the entries of a page are visited
by up to five workers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.meta.backup: resume an interrupted metadata copy

The walk saves the last finished path every few seconds, and a walk
that continues an unfinished one starts after it. A fresh walk clears
the saved path first, and a finished walk clears it with the walk
start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.meta.backup: add -entriesPerSecond to throttle the metadata copy

Each entry the full copy reads waits on a rate limiter, and pages are
listed only after the previous page is copied, so the flag also bounds
how fast the copy lists the source filer. 0 keeps it unlimited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer_pb: report each file of a depth-first walk once all before it are done

A throttled walk only reported progress after a whole listing page, so a
slow copy could go many minutes without a resumable path. Visit entries in
runs that end at a directory and report each file as soon as every entry
before it is done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:08:11 -08:00
Chris LuandClaude Opus 5.5 0ef0260844 volume: fail a vacuum step that finds the volume already compacting (#12553)
* volume: fail a vacuum step that finds a compaction already running

Compact, CompactByIndex and CommitCompact returned success when another
compaction held the volume. A second, uncoordinated vacuum (an expired
admin job still running, a shell volume.vacuum next to the worker) then
went on to commit or clean up the first run's .cpd/.cpx. Return a
"volume N is compacting" error instead, and refuse cleanup while a copy
is in flight, as the Rust volume server already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: check the compaction files before a commit closes the volume

CommitCompact closed the needle map and data file and wrote the .cpc
marker before it found .cpd/.cpx missing. A duplicate commit, such as
the second of two overlapping vacuums, then left the volume registered
with no open files: it dropped out of the heartbeat and looked lost to
volume.list and volume.fix.replication until a restart. Check for the
files first so the refused commit leaves the volume as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: run makeupDiff before a commit closes the volume

CommitCompact closed the needle map and data file before makeupDiff, so
a makeupDiff failure (a compaction revision another commit moved, an I/O
error) returned with them closed: the volume stayed registered but
missing from the heartbeat until a restart. makeupDiff reads the live
files through its own handles while the commit holds the write lock, so
close them only once it has succeeded, as the Rust volume server does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: log a volume the heartbeat skips because its files are closed

A volume left without its needle map or data file vanished from the
heartbeat with no log line, so the master saw a replica disappear while
its files were still on disk. Warn on each skip, as the deleted-.dat
case next to it already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: claim the compaction flag while cleanup removes the temp files

Cleanup only read the flag, so a compaction could start between the
check and the removal and lose its .cpd/.cpx. Claim it for the cleanup
instead, in Go and Rust.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 08:48:58 -08:00
Chris LuandClaude Opus 5.5 a4a68c885a filer.sync: move the watermark over jobs that finished out of order (#12554)
* filer.sync: move the watermark over jobs that finished ahead of an older one

The watermark only took the finishing job's timestamp when it was the
oldest in flight, so jobs that completed before an older one were never
recorded and a quiet stream kept the offset behind finished events.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.sync: keep one finished timestamp per gap behind a running job

A long job kept every later completion in the done heap. Only the newest
one between two running or failed timestamps can become the watermark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.sync: test done-heap compaction across several running and failed jobs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 08:47:30 -08:00
000baf8dfa s3: return an error for GETs canceled before the first byte (#12502)
* s3: answer GETs canceled before the first byte

A stream that fails before writing anything now always gets an S3 error
response, canceled errors included. The idle timeout and a shared lookup
canceled by another request both cancel the request context while the
client is still connected, and returning silently sent that client a
200 with Content-Length: 0. A client that has gone away never sees the
write.

Refs #12500

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: mark inline GET write errors as written; bound the idle-cancel test

An inline object's write error comes after the status line, so the
caller must not write an S3 error on top of it. The idle-cancel test
caps the client wait and gives the response write its own deadline,
keeping only the read side on the 200 ms idle timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: test a GET whose request is canceled before the first byte

Drives GetObjectHandler through a chunk lookup that cancels the request
context, so a handler that returns before answering shows up as the empty
200 again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* s3: check the cancel test reached the chunk lookup

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-11 08:45:20 -08:00
Chris LuandClaude Opus 5.5 ea6e9d670a volume: forward no Content-MD5 for a JPEG the orientation fix rewrote (#12557)
* volume: forward no Content-MD5 for a JPEG the orientation fix rewrote

With -images.fix.orientation the primary stores and forwards the rotated
bytes, but sent replicas the client's digest of the original upload. The
replica checks Content-MD5 against what it receives and rejects the write,
so replicated JPEG uploads with a digest failed. The Rust volume server
already drops the digest once the body is rewritten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: test the forwarded Content-MD5 through PostHandler and a replica

The test now posts each upload to a primary whose fake master names a
real replica volume server, so it covers the JPEG gate, ReplicatedWrite
and the replica's own Content-MD5 check, and reads the stored bytes back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: fail the replica forwarding test instead of hanging when nothing is forwarded

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* volume: compare the primary and replica bytes in the replica forwarding test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 08:44:06 -08:00
1202c5ba08 filer.copy: compute chunk md5 so files get a real ETag (#12497)
* filer.copy: compute chunk md5 so files get a real ETag

Chunks uploaded by filer.copy had no ETag, so every multi-chunk file got
the md5 of the empty string plus -N. An overwritten file kept its old ETag
and a revalidating client got 304 for changed content.

* filer.copy: store chunks in offset order

Chunks are collected in upload completion order, and the entry ETag
hashes the chunk ETags in stored order. Identical files copied with
concurrent chunk uploads got different ETags, and a file with swapped
equal-sized chunks could keep the old one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer.copy: test chunk order and chunk md5 on chunked uploads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 08:43:23 -08:00
b9866bae4f weed/shell: support -noLock flag in volumeServer.evacuate (#12501) (#12516)
* weed/shell: support -noLock flag in volumeServer.evacuate (#12501)

* shell: return flag parse errors from volumeServer.evacuate

A typo such as -noLock=invalid printed the parse error but the command
still succeeded, so a piped script treated the evacuation as done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: volumeServer.evacuate -h prints usage without failing

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:25:42 -08:00
09a63031c1 helm: set replication and disk on buckets created by the chart (#12543)
* helm: set replication and disk on buckets created by the chart

* helm: test bucket replication and disk without ttl

* helm: reject unquoted bucket replication values

* helm: treat a null bucket replication as unset

A blank replication key in a values file failed the render with the
quoted-string error, while blank ttl and disk are skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* helm: require a three-digit bucket replication

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* helm: note that removing bucket settings keeps the stored rule

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* helm: reject bucket replication above 255

fs.configure rejects an xyz value above 255, so the hook Job failed after
the release rendered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:24:55 -08:00
f068fe7f09 fix(shell): fs.log.purge purges every filer, not just one (#12524)
* fix(shell): fs.log.purge purges every filer, not just one

Each filer keeps its own metadata logs in its own store, so purging a
single filer leaves the chunks referenced by the others and the volume
vacuum cannot reclaim anything.

The shell picked that single filer at random (commands.go:208) when no
-filer flag was given, and the admin script worker pinned it to
filers[0] (admin_script_handler.go:574), so the same command purged a
different filer on every run and never the whole cluster.

fs.log.purge now resolves the filer list from the master and purges each
of them, continuing when one is unreachable and reporting the failures
at the end. -filer=<addr>,... restricts the purge to a subset.

Also stops mutating the shared command environment per filer, so a
long-lived shell keeps its original filer.

* fix(shell): bound filer discovery and cover the cluster path

Address review feedback:

- resolveLogPurgeFilers waited on the master with context.Background(): a
  shell whose master connection dropped would hang before purging
  anything, where it used to purge the connected filer. Bound the master
  round trip with a timeout and fall back to the connected filer when the
  master cannot be reached or lists no filer.
- the tests only exercised the single-filer fallback. Rename that test to
  what it actually covers and add one asserting every filer of the list is
  attempted and the shared command environment keeps its original filer.

* shell: reject an fs.log.purge -filer list with no addresses

A comma or whitespace only -filer value fell through to master
discovery and purged every filer instead of the intended subset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: fall back to the shell's filer without a second master lookup

When the master listed no filer, FilerAddress() ran filer discovery again
and could wait out the shell's ConnectTimeout before failing. Also cover
master discovery with the fake discovery master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* shell: test the fs.log.purge filer loop against a fake filer

The previous test looped over the addresses itself, so it passed no
matter what Do did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Martin Morez <martin@morez.org>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:24:04 -08:00
Chris LuandClaude Opus 4.8 fb5ffb815f filer: validate bucket names before the SQL DDL hooks (#12547)
* filer: validate bucket names before the SQL DDL hooks

The SQL store builds CREATE TABLE and DROP TABLE statements by formatting
the bucket name into the query. The DML paths already reject names that
fail VerifyS3BucketName, but the bucket-creation and bucket-deletion
hooks ran the DDL without that check, so a name reaching them through the
filer or WebDAV directory path (which skips the check for dot-prefixed
directories) could inject stacked statements. Apply the same validation
the DML paths use.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* filer: check that a valid bucket runs both DDL statements

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-11-10 21:22:31 -08:00
Chris LuandClaude Opus 5.5 d350a0ad03 weed-db: end startup when password authentication fails (#12550)
* weed-db: end startup when password authentication fails

The failure branches returned the result of writing the error response,
which is nil on success, so the handshake carried on to ReadyForQuery.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* weed-db: test startup with wrong and unknown credentials

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* weed-db: check the auth error code and fail on stray read errors in the startup test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:21:05 -08:00
Chris LuandClaude Opus 5.5 f3150c3495 admin: escape file properties before rendering the modal (#12548)
* admin: escape file properties before rendering the modal

The Properties modal concatenated the entry name, path, MIME type and
extended attributes into HTML, so object metadata written over S3 could
inject markup into the admin page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep every property key when escaping

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:19:53 -08:00
Chris LuandClaude Opus 5.5 8d96985131 security: scoped filer tokens are not admin credentials (#12551)
* security: do not accept a path- or method-scoped filer token as admin

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* security: test that scoped filer tokens fail the admin token check

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 21:18:59 -08:00
2143761319 fix(s3tables): let a table renamed across namespaces keep committing (#12515)
* s3tables: let a table renamed across namespaces keep committing

* s3tables: keep the same-directory exception off a directory another table shares

A record written before metadata locations were confined to their namespace
(6f93f874) and claimed (#12473) can keep its files in another table's
directory, in another namespace. The exception that lets a table renamed
across namespaces keep committing to the directory it left behind accepted
such a record's commits too, where master refused them.

The namespace check now runs first, as on master. Only a location it refuses
is matched against the record's current directory, and the exception holds
only when no other record keeps its files there. Where other records' entries
sit is ignored, so a table created under the renamed table's old name, whose
files go to a directory of its own, does not take the exception away. The
scan runs only on that path, so commits inside the namespace cost what they
did.

Co-Authored-By: Claude <noreply@anthropic.com>

* s3tables: seed the table at the renamed table's old name

A rename may keep the old name taken, so creating a table there can be
refused; seeding the record keeps the test on the update path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2026-11-10 21:15:48 -08:00
Tsic 518009dcd4 filer: apply filerGroup prefix to bucket default collection (#12499)
filer: prefix bucket default collection, assign copy collection explicitly

S3 server-side copy paths (CopyObject/UploadPartCopy) assign volumes with an empty collection and relied on the filer's bucket fallback. The fallback used the raw bucket name, while PutObject uses getCollectionName(bucket) = <filerGroup>_<bucket>, so copies landed in a second, unprefixed collection.

Prefix the filer's bucket default with the filer group, and have the S3 copy path pass that collection explicitly, like PutObject. Direct filer writes keep path-rule precedence, so this only aligns S3 copy with S3 put.

Adds regression tests for the fallback, explicit collection precedence, and the S3 copy collection resolver.
2026-11-10 21:12:49 -08:00
Ryan Leung 7ad8419815 shell: preserve completed uploads for bucket-root objects (#12545)
Signed-off-by: rleungx <rleungx@gmail.com>
2026-11-10 19:39:10 -08:00
github-actions[bot] 9799b82edf docs: regenerate star history chart 2026-11-11 01:16:36 +00:00
Mansur AkhmadiandMansur Akhmadi 9886ac0bcf topology: take dn.Lock when a heartbeat links a new disk type (#12535)
Co-authored-by: Mansur Akhmadi <10739460+mansur-akhmadi@users.noreply.github.com>
2026-11-10 12:17:21 -08:00
Mansur AkhmadiandMansur Akhmadi 9f4b652522 master: make admin lock lease and release atomic (#12532)
Co-authored-by: Mansur Akhmadi <10739460+mansur-akhmadi@users.noreply.github.com>
2026-11-10 12:16:23 -08:00
Saarthak Gupta 017e42633f shell: return flag parse errors from s3.bucket.create (#12544) 2026-11-10 12:15:01 -08:00
Mohd Quamar TyagiandChris Lu 5863e32318 weed/s3api: avoid truncated marker rescan in ListObjects (#12540) (#12541)
* weed/s3api: avoid truncated marker rescan in ListObjects (#12540)

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>

* fix(s3): validate marker directory candidates

Respect prefix filters and internal-directory exclusions when recovering marker candidates. Restrict lookups to direct children, propagate read failures other than not-found, and deduplicate directory marker echoes.

Add regression coverage and model promoted objects consistently in the XML-unsafe prefix fixture.

---------

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-11-10 09:58:45 -08:00
Chris Lu d485d2c9cd filer.sync: shorten retries for confirmed missing chunks (#12546)
* filer.sync: shorten retries for confirmed missing chunks

* filer.sync: clarify missing chunk grace period names
2026-11-10 09:56:17 -08:00
Chris Lu 6c473433c5 vector: bound request buffering before authentication and document installation (#12537)
* s3vectors: bound body buffering before authentication

* docs: include vector engine installation and source builds

* s3vectors: prevent MIME sniffing of JSON error responses

* docs: load vector Cargo configuration when building from source

* docs: note the native protoc requirement for Windows vector builds

* docs: make vector source checkout and working directory explicit

* ci: handle deb822 sources during Ubuntu mirror failover

* ci: bound Ubuntu mirror attempts and reject partial index updates
2026-11-10 04:21:56 -08:00
Chris Lu 8236c8d136 vector: integrate engine health, admin management, and local deployment (#12517)
* vector: expose request metrics and registration-aware health endpoints

* vector: report immutable read cache and storage fetch metrics

* docs: describe vector engine health, metrics, and maintenance

* vector: count canceled requests when their handler is dropped

* vector: account for uncached large immutable storage reads

* vector: reliably release the stalled credential reader in tests

* vector: observe large reads as their streamed body is consumed

* vector: bound heartbeats on established master registration streams

* test: exercise hardened vector maintenance on Go and Rust volumes

* vector: register engines in their configured filer group

* vector: make retried index creation preserve compatible datasets

* vector: allow registration to advertise a separate address

* test: release stalled credential readers without blocking FIFO cleanup

* test: allow hardened vector coverage in a named filer group

* vector: discover engines in the same filer group as their gateway

* vector: fall back when the advertised address is empty

* admin: expose vector readiness, maintenance, and management tools (#12520)

* s3vectors: share vector discovery with trusted management clients

* admin: show vector engine readiness and recent maintenance activity

* admin: suggest S3 Vectors IAM actions from the supported operations

* admin: manage vector indexes, policies, tags, encryption, and query data

* admin: validate SSE-S3 support before creating encrypted vector buckets

* admin: discover vector engines in their registration group

* admin: probe vector engines concurrently with independent time budgets

* admin: verify encryption capability on the remote S3 gateway

* admin: defer discovery until after the HTTP server can start

* admin: distinguish unknown vector health services from failed engines

* admin: count workers that can execute vector index maintenance

* admin: allow read-only users to query vectors and inspect tags

* vector: scope gateway and admin discovery to the configured filer group

* admin: refresh policy state after mutations and serialize policy controls

* vector: drain active RPCs before closing replaced discovery connections

* vector: respect request cancellation while waiting for discovery

* vector: prefer another registered engine after a transport failure

* admin: check all masters before reporting no registered vector engines

* iam: honor attached vector policies without broadening legacy grants

* vector: allow short-lived discovery clients to close their connections

* vector: refresh cached master addresses for each discovery lookup

* iam: enforce vector bucket tag conditions in attached policies

* iam: authorize vector bucket creation with the requested tags

* test: exercise vector IAM grants and tag denies on Go and Rust volumes

* admin: reject authenticated vector mutations without an admin role

* test: run vector IAM assertions before the shared engine is stopped

* vector: add native mini startup, shell tools, and a runnable Compose example (#12522)

* s3vectors: share vector discovery with trusted management clients

* admin: show vector engine readiness and recent maintenance activity

* admin: suggest S3 Vectors IAM actions from the supported operations

* admin: manage vector indexes, policies, tags, encryption, and query data

* admin: validate SSE-S3 support before creating encrypted vector buckets

* admin: discover vector engines in their registration group

* admin: probe vector engines concurrently with independent time budgets

* admin: verify encryption capability on the remote S3 gateway

* admin: defer discovery until after the HTTP server can start

* admin: distinguish unknown vector health services from failed engines

* admin: count workers that can execute vector index maintenance

* admin: allow read-only users to query vectors and inspect tags

* vector: scope gateway and admin discovery to the configured filer group

* admin: refresh policy state after mutations and serialize policy controls

* vector: drain active RPCs before closing replaced discovery connections

* vector: respect request cancellation while waiting for discovery

* vector: prefer another registered engine after a transport failure

* admin: check all masters before reporting no registered vector engines

* iam: honor attached vector policies without broadening legacy grants

* vector: allow short-lived discovery clients to close their connections

* vector: refresh cached master addresses for each discovery lookup

* iam: enforce vector bucket tag conditions in attached policies

* iam: authorize vector bucket creation with the requested tags

* test: exercise vector IAM grants and tag denies on Go and Rust volumes

* admin: reject authenticated vector mutations without an admin role

* test: run vector IAM assertions before the shared engine is stopped

* mini: optionally start and supervise the vector engine

* mini: bootstrap vector buckets with existing Lance catalog helpers

* shell: manage vector buckets, indexes, and vector data

* docs: provide a runnable vector and maintenance worker Compose stack

* shell: scope vector discovery to the session and close its connections

* mini: honor the bind address for vector and metrics listeners

* mini: reserve vector ports before assigning gRPC and compatibility listeners

* mini: wait for vector storage setup and master registration

* mini: create requested vector buckets without requiring the Lance HTTP listener

* shell: refresh master addresses during vector discovery

* mini: resolve sibling vector executables with platform extensions

* vector: preserve uncertain creates and fix mini listener allocation (#12525)

* vector: retain catalog records when index creation has an unknown outcome

* mini: keep chosen listener ports reserved throughout allocation

* mini: resolve hostname binds before starting vector listeners

* vector: fix integration test imports and recover interrupted index creation (#12533)

* shell: register vector operations from the command package

* s3vectors: resume interrupted index initialization safely

* mini: exclude disabled WebDAV from listener allocation

* s3vectors: retain uncertain creates across refused retries

* s3vectors: claim pending indexes before retrying initialization

* s3vectors: compare resolved distance metrics during recovery

* s3vectors: preserve metadata keys when retrying initialization

* s3vectors: verify ownership before initializing new catalog records
2026-11-10 04:21:18 -08:00
github-actions[bot] 3f5b07475d docs: regenerate star history chart 2026-11-10 00:45:43 +00:00
Chris Lu 5abb18e678 helm: manage vector engines and provision vector buckets (#12518)
* helm: deploy vector engines with TLS, probes, and monitoring

* helm: provision vector buckets through the S3 Vectors API

* helm: bracket IPv6 pod addresses for vector listeners

* helm: discover vector engines through the all-in-one master

* helm: accept vector component NetworkPolicy overrides

* helm: bind vector metrics to the pod address on both IP families

* helm: provision vector buckets through embedded and all-in-one gateways

* docs: use the full Helm name for external vector certificates

* helm: render vector instance labels as strings

* helm: allow vector bucket hooks to use restricted security contexts

* helm: honor vector security context enable switches

* helm: prevent duplicate vector monitoring labels

* helm: reject metrics flags with the default pre-metrics vector image

* test: skip vector chart compatibility checks when Helm is unavailable
2026-11-09 16:34:57 -08:00
Chris Lu c1afb1b1cb ci: finish native vector checks and fix Windows installer tests (#12526)
* test: pass installer scripts through stdin on Windows

* ci: allow cold Intel macOS vector builds to finish

* ci: retain completed Rust builds when later platform checks fail
2026-11-09 16:33:50 -08:00
Chris Lu 89d3016d5e vector: publish native macOS and Windows engines and installer support (#12523)
* ci: build, test, and publish vector engines for macOS and Windows

* installer: install native vector archives on macOS and Windows

* ci: run desktop vector checks for shared Rust and protobuf changes

* ci: publish vector releases from a separate job with write permissions

* ci: build tagged vector releases without restoring caches

* ci: propagate Windows bash path conversion failures
2026-11-09 16:14:42 -08:00
Chris Lu ac2857911d grafana: add vector API, engine, cache, and maintenance views (#12519)
* grafana: add vector API, engine, cache, and maintenance panels

* helm: include vector panels with Kubernetes metric selectors

* grafana: preserve cluster and namespace identity when matching engine health

* grafana: retain engine identity when scrapes fail

* grafana: show zero cache hits before the first hit

* grafana: keep cluster and namespace identity in vector cache ratios
2026-11-09 16:00:02 -08:00
dependabot[bot] d9d8942caf build(deps): bump docker/login-action from 4.7.0 to 4.8.0 (#12510)
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.7.0 to 4.8.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v4.7.0...v4.8.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 11:59:14 -08:00
dependabot[bot] f39b1abd03 build(deps): bump github.com/shirou/gopsutil/v4 from 4.26.7 to 4.26.10 (#12507)
Bumps [github.com/shirou/gopsutil/v4](https://github.com/shirou/gopsutil) from 4.26.7 to 4.26.10.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v4.26.7...v4.26.10)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v4
  dependency-version: 4.26.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 11:59:07 -08:00
dependabot[bot] 2373207b6e build(deps): bump sigstore/cosign-installer from 4.1.2 to 4.1.3 in /.github/actions/sign-image (#12512)
build(deps): bump sigstore/cosign-installer

Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](https://github.com/sigstore/cosign-installer/compare/v4.1.2...v4.1.3)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 11:58:55 -08:00
dependabot[bot] 82cd43a884 build(deps): bump helm/kind-action from 1.15.1 to 1.15.2 (#12511)
Bumps [helm/kind-action](https://github.com/helm/kind-action) from 1.15.1 to 1.15.2.
- [Release notes](https://github.com/helm/kind-action/releases)
- [Commits](https://github.com/helm/kind-action/compare/v1.15.1...v1.15.2)

---
updated-dependencies:
- dependency-name: helm/kind-action
  dependency-version: 1.15.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 11:58:40 -08:00
dependabot[bot] d1d081fcf2 build(deps): bump github/codeql-action from 4.39.1 to 4.39.3 (#12509)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.39.1 to 4.39.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.39.1...v4.39.3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.39.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 11:58:24 -08:00
Chris Lu 84cd3c621b s3vectors: integrate bucket metrics, audit events, and request limits (#12514)
* s3vectors: attribute request and traffic metrics to authorized buckets

* s3vectors: emit resource and identity audit events

* s3vectors: apply existing S3 request limits and preserve JSON errors

* s3vectors: select limit resources by the dispatched operation

* s3vectors: count decoded chunked request bodies against byte limits
2026-11-09 11:37:06 -08:00
Chris Lu 620a1c3f39 docs: showcase vector buckets and cluster features (#12513)
* docs: showcase vector buckets and cluster features

* docs: include built-in Iceberg catalog in table bucket overview

* docs: keep feature diagram at cluster level
2026-11-09 08:52:12 -08:00
dependabot[bot] 643784df0d build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.47.5 to 1.47.7 (#12506)
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.47.5 to 1.47.7.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.47.5...v1.47.7)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-version: 1.47.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 08:29:08 -08:00
dependabot[bot] 9bb33d0e36 build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.23.3 to 1.24.0 (#12505)
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore

Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go) from 1.23.3 to 1.24.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.23.3...sdk/azcore/v1.24.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
  dependency-version: 1.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 08:28:40 -08:00
dependabot[bot] a1ba470b32 build(deps): bump github.com/testcontainers/testcontainers-go from 0.44.0 to 0.45.0 (#12504)
build(deps): bump github.com/testcontainers/testcontainers-go

Bumps [github.com/testcontainers/testcontainers-go](https://github.com/testcontainers/testcontainers-go) from 0.44.0 to 0.45.0.
- [Release notes](https://github.com/testcontainers/testcontainers-go/releases)
- [Commits](https://github.com/testcontainers/testcontainers-go/compare/v0.44.0...v0.45.0)

---
updated-dependencies:
- dependency-name: github.com/testcontainers/testcontainers-go
  dependency-version: 0.45.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 08:28:27 -08:00
dependabot[bot] 5b87b7bc95 build(deps): bump github.com/prometheus/procfs from 0.22.0 to 0.23.0 (#12503)
Bumps [github.com/prometheus/procfs](https://github.com/prometheus/procfs) from 0.22.0 to 0.23.0.
- [Release notes](https://github.com/prometheus/procfs/releases)
- [Commits](https://github.com/prometheus/procfs/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/procfs
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-09 08:28:15 -08:00
github-actions[bot] 7506e5f91c docs: regenerate star history chart 2026-11-09 00:35:41 +00:00
Chris LuandClaude Opus 5.5 5c910ad546 topology: list collections without nesting the collection map read lock (#12495)
* topology: list collections without nesting the collection map read lock

Items() already takes the read lock and copies the values. Holding it
around the call made ListCollections take the read lock twice, so a
DeleteCollection waiting for the write lock in between blocked the
second read lock and wedged the master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* topology: test ListCollections against a concurrent collection delete

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:05:41 -08:00
507 changed files with 34314 additions and 5251 deletions

No files matched your search

+1 -1
View File
@@ -15,7 +15,7 @@ runs:
using: composite
steps:
- name: Install cosign
uses: sigstore/cosign-installer@v4.1.2
uses: sigstore/cosign-installer@v4.1.3
- name: Sign
shell: bash
+3 -3
View File
@@ -33,7 +33,7 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4.39.1
uses: github/codeql-action/init@v4.39.3
# Override language selection by uncommenting this and choosing your languages
with:
languages: go
@@ -41,7 +41,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below).
- name: Autobuild
uses: github/codeql-action/autobuild@v4.39.1
uses: github/codeql-action/autobuild@v4.39.3
# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -55,4 +55,4 @@ jobs:
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4.39.1
uses: github/codeql-action/analyze@v4.39.3
+2 -2
View File
@@ -180,14 +180,14 @@ jobs:
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
if: github.event_name != 'pull_request'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -141,7 +141,7 @@ jobs:
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
+6 -6
View File
@@ -270,13 +270,13 @@ jobs:
buildkitd-config: /tmp/buildkitd.toml
- name: Login to Docker Hub
if: needs.setup.outputs.publish == 'true'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
if: needs.setup.outputs.publish == 'true'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -324,7 +324,7 @@ jobs:
fi
- name: Login to GHCR
if: needs.setup.outputs.publish == 'true'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -434,7 +434,7 @@ jobs:
output: trivy-results.sarif
exit-code: '0'
- name: Upload Trivy scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v4.39.1
uses: github/codeql-action/upload-sarif@v4.39.3
if: always()
with:
sarif_file: trivy-results.sarif
@@ -501,12 +501,12 @@ jobs:
ghcr.io/chrislusf/seaweedfs
tags: type=raw,value=${{ github.event_name == 'workflow_dispatch' && github.event.inputs.image_tag || 'latest' }},suffix=${{ steps.config.outputs.tag_suffix }}
- name: Login to Docker Hub
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -58,7 +58,7 @@ jobs:
-
name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
@@ -262,7 +262,7 @@ jobs:
- name: Login to GHCR
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -352,7 +352,7 @@ jobs:
- name: Login to GHCR
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -374,7 +374,7 @@ jobs:
- name: Login to Docker Hub
if: github.event_name != 'workflow_dispatch' || github.event.inputs.variant == 'all' || github.event.inputs.variant == matrix.variant
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
@@ -487,7 +487,7 @@ jobs:
variant: large_disk
steps:
- name: Login to GHCR
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -515,7 +515,7 @@ jobs:
- name: Upload Trivy scan results to GitHub Security
if: always()
uses: github/codeql-action/upload-sarif@v4.39.1
uses: github/codeql-action/upload-sarif@v4.39.3
with:
sarif_file: trivy-results.sarif
category: trivy-${{ matrix.variant }}
@@ -539,13 +539,13 @@ jobs:
latest_tag: latest_large_disk
steps:
- name: Login to Docker Hub
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GHCR
uses: docker/login-action@v4.7.0
uses: docker/login-action@v4.8.0
with:
registry: ghcr.io
username: ${{ secrets.GHCR_USERNAME }}
@@ -96,7 +96,7 @@ jobs:
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Login to Docker Hub
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
+3
View File
@@ -61,6 +61,9 @@ jobs:
restore-keys: |
${{ runner.os }}-buildx-e2e-
- name: Test APT mirror fallback
run: python3 test_apt_install.py
- name: Install dependencies
run: |
sudo rm -f /etc/apt/sources.list.d/azure-cli.list /etc/apt/sources.list.d/microsoft-prod.list
+69 -1
View File
@@ -37,6 +37,13 @@ jobs:
with:
version: v3.18.4
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
- name: Test vector chart resources
run: go test ./k8s/charts/seaweedfs/tests -count=1
- uses: actions/setup-python@v7
with:
python-version: '3.10'
@@ -1321,6 +1328,67 @@ jobs:
echo "$BOOL_FALSE" | grep -q -- '-status Suspended' || { echo "FAIL: bool false versioning did not Suspend the bucket"; exit 1; }
echo "Bucket versioning: YAML bool false suspends consistently with string \"false\""
echo "=== Testing bucket ttl, replication and disk in one fs.configure ==="
FS_CONFIGURE=$(helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=cfgbucket \
--set s3.createBuckets[0].ttl=7d \
--set-string s3.createBuckets[0].replication=001 \
--set s3.createBuckets[0].disk=ssd | grep 'fs.configure -locationPrefix=/buckets/cfgbucket/' || true)
echo "$FS_CONFIGURE" | grep -q -- '-ttl=7d -replication=001 -disk=ssd -apply' || { echo "FAIL: bucket fs.configure missing ttl/replication/disk: $FS_CONFIGURE"; exit 1; }
echo "Bucket fs.configure: ttl, replication and disk render together"
FS_CONFIGURE=$(helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=replbucket \
--set-string s3.createBuckets[0].replication=002 | grep 'fs.configure -locationPrefix=/buckets/replbucket/' || true)
echo "$FS_CONFIGURE" | grep -q -- '/buckets/replbucket/ -replication=002 -apply' || { echo "FAIL: replication-only bucket did not render fs.configure: $FS_CONFIGURE"; exit 1; }
echo "Bucket fs.configure: replication without ttl"
FS_CONFIGURE=$(helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=diskbucket \
--set s3.createBuckets[0].disk=ssd | grep 'fs.configure -locationPrefix=/buckets/diskbucket/' || true)
echo "$FS_CONFIGURE" | grep -q -- '/buckets/diskbucket/ -disk=ssd -apply' || { echo "FAIL: disk-only bucket did not render fs.configure: $FS_CONFIGURE"; exit 1; }
echo "Bucket fs.configure: disk without ttl"
if helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=numbucket \
--set s3.createBuckets[0].replication=0 >/dev/null 2>&1; then
echo "FAIL: a numeric replication should fail to render"; exit 1
fi
echo "Bucket fs.configure: numeric replication is rejected"
if helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=badbucket \
--set-string s3.createBuckets[0].replication=00O >/dev/null 2>&1; then
echo "FAIL: a non-digit replication should fail to render"; exit 1
fi
echo "Bucket fs.configure: non-digit replication is rejected"
if helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=bigbucket \
--set-string s3.createBuckets[0].replication=256 >/dev/null 2>&1; then
echo "FAIL: a replication above 255 should fail to render"; exit 1
fi
FS_CONFIGURE=$(helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=maxbucket \
--set-string s3.createBuckets[0].replication=255 | grep 'fs.configure -locationPrefix=/buckets/maxbucket/' || true)
echo "$FS_CONFIGURE" | grep -q -- '/buckets/maxbucket/ -replication=255 -apply' || { echo "FAIL: replication 255 should render: $FS_CONFIGURE"; exit 1; }
echo "Bucket fs.configure: replication is limited to 000-255"
FS_CONFIGURE=$(helm template test $CHART_DIR \
--set s3.enabled=true \
--set s3.createBuckets[0].name=nullbucket \
--set s3.createBuckets[0].ttl=7d \
--set s3.createBuckets[0].replication=null | grep 'fs.configure -locationPrefix=/buckets/nullbucket/' || true)
echo "$FS_CONFIGURE" | grep -q -- '/buckets/nullbucket/ -ttl=7d -apply' || { echo "FAIL: null replication should be treated as unset: $FS_CONFIGURE"; exit 1; }
echo "Bucket fs.configure: null replication is unset"
echo ""
echo "=== Testing hook Job labels ==="
# The hook Jobs were the only pods in the chart without the standard
@@ -2131,7 +2199,7 @@ jobs:
echo "IMAGE_TAG=$TAG" >> $GITHUB_ENV
- name: Create kind cluster
uses: helm/kind-action@v1.15.1
uses: helm/kind-action@v1.15.2
- name: Verify volume resize hook security contexts
run: |
+135
View File
@@ -0,0 +1,135 @@
name: "rust: vector desktop platforms"
on:
pull_request:
paths:
- 'seaweed-vector/**'
- 'seaweed-worker/**'
- 'seaweed-common/**'
- 'weed/pb/**'
- '.github/workflows/rust-vector-platforms.yml'
- 'install.sh'
- 'test/installer/**'
push:
tags: ['*']
workflow_dispatch:
permissions:
contents: read
concurrency:
group: vector-platforms-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
vector:
permissions:
contents: read
strategy:
fail-fast: false
matrix:
include:
- runner: macos-15-intel
suffix: darwin_amd64
protoc: macos-x86_64
- runner: macos-latest
suffix: darwin_arm64
protoc: macos-aarch_64
- runner: windows-latest
suffix: windows_amd64
protoc: win32
runs-on: ${{ matrix.runner }}
timeout-minutes: 120
env:
CARGO_BUILD_JOBS: '2'
CARGO_PROFILE_TEST_DEBUG: '0'
SEAWEEDFS_COMMIT: ${{ github.sha }}
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v7
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@stable
- uses: actions/setup-python@v6
with:
python-version: '3.13'
- uses: actions/cache/restore@v6
id: cargo-cache
if: ${{ !startsWith(github.ref, 'refs/tags/') }}
with:
path: |
~/.cargo/registry
~/.cargo/git
seaweed-vector/target
key: vector-${{ matrix.suffix }}-${{ hashFiles('seaweed-vector/Cargo.lock') }}
restore-keys: vector-${{ matrix.suffix }}-
- name: Fetch dependencies
run: cargo fetch --locked --manifest-path seaweed-vector/Cargo.toml
- name: Select vendored protoc
env:
PROTOC_PACKAGE: protoc-bin-vendored-${{ matrix.protoc }}
run: |
python - <<'PY'
import os, pathlib, tomllib
lock=tomllib.loads(pathlib.Path('seaweed-vector/Cargo.lock').read_text())
name=os.environ['PROTOC_PACKAGE']
version=next(p['version'] for p in lock['package'] if p['name']==name)
cargo=pathlib.Path(os.environ.get('CARGO_HOME',pathlib.Path.home()/'.cargo'))
executable='protoc.exe' if os.name=='nt' else 'protoc'
path=next(cargo.glob(f'registry/src/*/{name}-{version}/bin/{executable}'))
with open(os.environ['GITHUB_ENV'],'a') as out: out.write(f'PROTOC={path}\n')
PY
- name: Build vector engine
id: cargo-build
run: cargo build --locked --release --manifest-path seaweed-vector/Cargo.toml -p weed-vector-server
- name: Test vector crates
run: cargo test --locked --release --manifest-path seaweed-vector/Cargo.toml --workspace
- name: Test installer
run: |
if [ "$RUNNER_OS" = "Windows" ]; then
BASH_BINARY="$(cygpath -w "$BASH")"
export BASH_BINARY
fi
python -m unittest discover -s test/installer
- name: Save compiled dependencies
if: ${{ always() && !startsWith(github.ref, 'refs/tags/') && steps.cargo-build.outcome == 'success' && steps.cargo-cache.outputs.cache-hit != 'true' }}
uses: actions/cache/save@v6
with:
path: |
~/.cargo/registry
~/.cargo/git
seaweed-vector/target
key: vector-${{ matrix.suffix }}-${{ hashFiles('seaweed-vector/Cargo.lock') }}
- name: Package engine
env:
ASSET_SUFFIX: ${{ matrix.suffix }}
run: |
python - <<'PY'
import hashlib, os, pathlib, tarfile, zipfile
suffix=os.environ['ASSET_SUFFIX']
binary='weed-vector.exe' if os.name=='nt' else 'weed-vector'
source=pathlib.Path('seaweed-vector/target/release')/binary
asset=f'weed-vector_{suffix}'+('.zip' if os.name=='nt' else '.tar.gz')
if os.name=='nt':
with zipfile.ZipFile(asset,'w',zipfile.ZIP_DEFLATED) as archive: archive.write(source,binary)
else:
with tarfile.open(asset,'w:gz') as archive: archive.add(source,arcname=binary)
digest=hashlib.md5(pathlib.Path(asset).read_bytes()).hexdigest()
pathlib.Path(asset+'.md5').write_text(f'{digest} {asset}\n')
PY
- uses: actions/upload-artifact@v7
with:
name: vector-${{ matrix.suffix }}
path: weed-vector_${{ matrix.suffix }}.*
release:
if: startsWith(github.ref, 'refs/tags/')
needs: vector
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/download-artifact@v8
with:
pattern: vector-*
merge-multiple: true
- uses: softprops/action-gh-release@v3
with:
files: weed-vector_*
+3 -8
View File
@@ -87,22 +87,17 @@ jobs:
# volume, filer, gateway, engine) under mutual TLS, read JWTs
# required, and every chunk encrypted by the gateway.
hardened: ["", "1"]
worker: [""]
worker: ["", "1"]
# Worker: the admin server and the Lance maintenance worker join the
# cluster, for the tests that build vector indexes through a job.
include:
- volume: go
hardened: ""
worker: "1"
- volume: rust
hardened: ""
worker: "1"
env:
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_BUILD_JOBS: "2"
CARGO_TARGET_DIR: ${{ github.workspace }}/seaweed-vector/target
VOLUME_SERVER_IMPL: ${{ matrix.volume }}
VECTOR_QUERY_MODE: single
VECTOR_FILER_GROUP: vectors-ci
VECTOR_TLS: ${{ matrix.hardened }}
VECTOR_READ_JWT: ${{ matrix.hardened }}
VECTOR_ENCRYPT_VOLUMES: ${{ matrix.hardened }}
-1
View File
@@ -147,7 +147,6 @@ telemetry/server/telemetry-server
__pycache__/
/test/s3vectors/weed
/test/s3vectors/weed-vector
/test/s3vectors/weed-vector-worker
/test/s3vectors/weed-vector-cli
/test/s3vectors/weed-worker
/test/s3vectors/weed-volume
+41 -3
View File
@@ -17,7 +17,9 @@ SeaweedFS is a simple and highly scalable distributed file system. There are two
1. to store billions of files!
2. to serve the files fast!
One `weed` binary serves an S3 object store, a POSIX file system, and a lakehouse with S3 Tables, all over the same data. Each blob is one disk read away, capacity grows by starting another volume server, and cloud storage can be cached or tiered transparently. Both read and write operations have O(1) complexity and can run at the full speed supported by the underlying hardware.
One SeaweedFS cluster serves an S3 object store, a POSIX file system, a lakehouse with S3 Tables and a built-in Iceberg REST Catalog, and [vector search with S3 Vector Buckets](#vector-search-with-s3-vector-buckets), all on shared storage. Each blob is one disk read away, capacity grows by starting another volume server, and cloud storage can be cached or tiered transparently. Blob reads and writes have O(1) complexity and can run at the full speed supported by the underlying hardware.
![SeaweedFS feature overview: S3 objects, POSIX files, table buckets with an Iceberg REST Catalog and Lance namespace, and S3 Vector Buckets share one cluster of masters, filers, and volume servers, with cloud caching, tiering, and cross-cluster replication.](note/seaweedfs-features.svg)
- [Download Binaries for different platforms](https://github.com/seaweedfs/seaweedfs/releases/latest)
- [Wiki Documentation](https://github.com/seaweedfs/seaweedfs/wiki)
@@ -40,6 +42,7 @@ Table of Contents
* [Scalable](#scalable)
* [The most complete S3 API](#the-most-complete-s3-api)
* [A data warehouse with S3 Tables](#a-data-warehouse-with-s3-tables)
* [Vector search with S3 Vector Buckets](#vector-search-with-s3-vector-buckets)
* [A fast cache for cloud storage](#a-fast-cache-for-cloud-storage)
* [Active-active replication and more](#active-active-replication-and-more)
* [Architecture](#architecture)
@@ -204,12 +207,13 @@ On one laptop, [`weed benchmark`][Benchmarks] writes 1KB files at 15,700 per sec
## The most complete S3 API ##
The S3 gateway implements the object, bucket, S3 Tables, IAM, and STS APIs on one endpoint, so the AWS SDKs and CLI, rclone, restic, Spark, and Trino work unchanged.
The S3 gateway implements the object, bucket, S3 Tables, S3 Vectors, IAM, and STS APIs on one endpoint, so the AWS SDKs and CLI, rclone, restic, Spark, and Trino work unchanged. S3 Vectors uses the `weed-vector` service described [below](#vector-search-with-s3-vector-buckets).
| API | Operations |
| --- | --- |
| S3 bucket and object | 73 |
| S3 Tables | 36 |
| S3 Vectors | 19 |
| IAM | 39 |
| STS | 5 |
@@ -223,7 +227,7 @@ The full operation list is in [Amazon S3 API][AmazonS3API], and [Supported APIs
## A data warehouse with S3 Tables ##
SeaweedFS is a lakehouse in one system. [S3 Table Buckets][S3TableBucket] hold Apache Iceberg tables by default, or [Lance][LanceCatalog] tables for vectors and multimodal data, and the built-in [Iceberg REST Catalog][IcebergCatalog] and Lance namespace serve them directly. There is no Hive Metastore, Glue, or separate catalog service to deploy, secure, and back up.
SeaweedFS is a lakehouse in one system. [S3 Table Buckets][S3TableBucket] hold Apache Iceberg tables by default, or [Lance][LanceCatalog] tables for vectors and multimodal data, and the built-in [Iceberg REST Catalog][IcebergCatalog] and Lance namespace serve them directly. The S3 Tables API and Iceberg REST Catalog share the same namespaces, table metadata, and permissions. There is no Hive Metastore, Glue, or separate catalog service to deploy, secure, and back up.
* Query engines operate on the same tables at the same time: [Spark][SparkIceberg], [Trino][TrinoIceberg], [Dremio][DremioIceberg], [DuckDB][DuckDBIceberg], [Apache Doris][DorisIceberg], [RisingWave][RisingWaveIceberg], ClickHouse, and [LanceDB][LanceDB]. Catalog commits are atomic compare-and-swap, so concurrent writers are safe. [Lakekeeper][Lakekeeper] can front the same storage with STS-vended credentials. [OLake Go][OLake] is a high-performance, open-source EL (Extract–Load) platform that bridges operational databases and open lakehouse storage - writing Apache Iceberg tables or Parquet files on Amazon S3.
* [Automated table maintenance][IcebergMaintenance]: compaction, snapshot expiration, orphan file removal, and manifest rewriting, configured per bucket or table through the S3 Tables maintenance APIs, and the same for [Lance][LanceMaintenance].
@@ -232,6 +236,40 @@ SeaweedFS is a lakehouse in one system. [S3 Table Buckets][S3TableBucket] hold A
`S3_TABLE_BUCKET=warehouse ./weed mini -dir=./data` brings the whole stack up on a laptop.
## Vector search with S3 Vector Buckets ##
Keep embeddings alongside your objects and tables in the same SeaweedFS cluster. S3 Vector Buckets expose the AWS S3 Vectors API for semantic search, retrieval-augmented generation (RAG), and recommendations, using Lance tables on SeaweedFS storage.
* Create vector buckets and indexes; insert, retrieve, list, and delete vectors with the AWS SDKs or CLI pointed at your S3 endpoint.
* Find the nearest vectors with cosine or Euclidean distance, filter by metadata, and return distances and metadata with the results.
* Control access with `s3vectors:*` IAM actions and vector bucket policies; tag buckets and indexes for organization and access control.
Install the separate vector engine; the default installer installs only `weed`:
```bash
curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash -s -- --component vector
```
Release 4.53 provides vector binaries for Linux amd64 and arm64. If your release has no binary for your platform, build from [source](seaweed-vector). From a directory where you want a new checkout:
```bash
git clone https://github.com/seaweedfs/seaweedfs.git
cd seaweedfs
(cd seaweed-vector && cargo build --release -p weed-vector-server)
```
If you already have a checkout, start at its root and skip the clone and `cd seaweedfs` steps.
For Windows source builds, set `PROTOC` to a native `protoc.exe` before running Cargo; the [desktop build workflow](.github/workflows/rust-vector-platforms.yml) shows how to select the vendored executable.
Run the installed `weed-vector` binary alongside the local cluster started with `weed mini` above (or use `seaweed-vector/target/release/weed-vector` after building from source):
```bash
weed-vector --masters=localhost:9333
```
It registers with the master and the S3 gateway discovers it automatically. An AWS S3 Vectors client uses the same S3 endpoint at http://localhost:8333. The vector engine stores its data through the cluster's filers and volume servers, so embeddings share the storage layer's replication and erasure coding. See the [AWS SDK examples](test/s3vectors/s3vectors_test.py) for bucket, index, and query operations.
## A fast cache for cloud storage ##
[Cloud Drive][CloudDrive] mounts a bucket from S3, Google Cloud Storage, Azure, Backblaze B2, Wasabi, Storj, or any S3-compatible store into SeaweedFS and serves it at local speed:
+4
View File
@@ -271,6 +271,10 @@ curl "http://localhost:9333/vol/vacuum?garbageThreshold=0.4"
Vacuuming makes a volume read-only, copies live needles to a new volume, and
swaps it in.
The master runs one vacuum at a time. A request that arrives while the
automatic vacuum or another request is running does nothing and gets
`409 Conflict` with `{"error":"another vacuum is running, try again later"}`.
### Delete a collection
```bash
+8 -10
View File
@@ -3,17 +3,15 @@
# unreachable. See docker/Dockerfile.e2e.
set -e
# Every rewrite starts from the pristine list, so a mirror that just failed does
# not become the pattern the next rewrite has to match.
[ -f /etc/apt/sources.list.orig ] || cp /etc/apt/sources.list /etc/apt/sources.list.orig
for mirror in azure.archive.ubuntu.com archive.ubuntu.com; do
# Any archive host, so this works whether the pristine list came from the
# base image (archive.ubuntu.com) or a CI runner (azure.archive.ubuntu.com).
sed "s|http://[a-z0-9.]*archive\.ubuntu\.com/ubuntu|http://$mirror/ubuntu|g; s|http://security\.ubuntu\.com/ubuntu|http://$mirror/ubuntu|g" \
/etc/apt/sources.list.orig > /etc/apt/sources.list
if apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=15 update && \
DEBIAN_FRONTEND=noninteractive apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=15 install -y \
for apt_source in /etc/apt/sources.list /etc/apt/sources.list.d/*.list /etc/apt/sources.list.d/*.sources; do
[ -f "$apt_source" ] || continue
[ -f "$apt_source.orig" ] || cp "$apt_source" "$apt_source.orig"
sed -E "s|(https?://)[a-z0-9.]*archive\.ubuntu\.com/ubuntu|\1$mirror/ubuntu|g; s|(https?://)security\.ubuntu\.com/ubuntu|\1$mirror/ubuntu|g" \
"$apt_source.orig" > "$apt_source"
done
if timeout --kill-after=5s 120s apt-get -o Acquire::Retries=1 -o Acquire::http::Timeout=15 -o Acquire::https::Timeout=15 -o APT::Update::Error-Mode=any update && \
DEBIAN_FRONTEND=noninteractive timeout --kill-after=5s 180s apt-get -o Acquire::Retries=1 -o Acquire::http::Timeout=15 -o Acquire::https::Timeout=15 install -y \
--no-install-recommends --no-install-suggests "$@"; then
apt-get clean
rm -rf /var/lib/apt/lists/*
+8 -8
View File
@@ -64,7 +64,7 @@ case "$1" in
'master')
ARGS="-mdir=/data -volumeSizeLimitMB=1024"
shift
exec /usr/bin/weed -logtostderr=true master $ARGS $@
exec /usr/bin/weed -logtostderr=true master $ARGS "$@"
;;
'volume')
@@ -73,7 +73,7 @@ case "$1" in
ARGS="-dir=/data"
fi
shift
exec /usr/bin/weed -logtostderr=true volume $ARGS $@
exec /usr/bin/weed -logtostderr=true volume $ARGS "$@"
;;
'volume-rust')
@@ -87,7 +87,7 @@ case "$1" in
echo "Use 'volume' for the Go volume server instead." >&2
exit 1
fi
exec /usr/bin/weed-volume $ARGS $@
exec /usr/bin/weed-volume $ARGS "$@"
;;
'worker-rust')
@@ -119,7 +119,7 @@ case "$1" in
ARGS="-dir=/data -master.volumeSizeLimitMB=1024"
fi
shift
exec /usr/bin/weed -logtostderr=true server $ARGS $@
exec /usr/bin/weed -logtostderr=true server $ARGS "$@"
;;
'mini')
@@ -128,19 +128,19 @@ case "$1" in
ARGS=""
fi
shift
exec /usr/bin/weed -logtostderr=true mini $ARGS $@
exec /usr/bin/weed -logtostderr=true mini $ARGS "$@"
;;
'filer')
ARGS=""
shift
exec /usr/bin/weed -logtostderr=true filer $ARGS $@
exec /usr/bin/weed -logtostderr=true filer $ARGS "$@"
;;
's3')
ARGS="-domainName=$S3_DOMAIN_NAME -key.file=$S3_KEY_FILE -cert.file=$S3_CERT_FILE"
shift
exec /usr/bin/weed -logtostderr=true s3 $ARGS $@
exec /usr/bin/weed -logtostderr=true s3 $ARGS "$@"
;;
'shell')
@@ -150,6 +150,6 @@ case "$1" in
;;
*)
exec /usr/bin/weed $@
exec /usr/bin/weed "$@"
;;
esac
+39
View File
@@ -0,0 +1,39 @@
# Vector buckets locally
Run on Linux containers (amd64 or arm64):
```sh
docker compose up -d
export AWS_ACCESS_KEY_ID=vector-demo
export AWS_SECRET_ACCESS_KEY=vector-demo-secret
export AWS_DEFAULT_REGION=us-east-1
aws --endpoint-url http://localhost:8333 s3vectors create-index \
--vector-bucket-name embeddings --index-name images \
--data-type float32 --dimension 3 --distance-metric cosine
aws --endpoint-url http://localhost:8333 s3vectors put-vectors \
--vector-bucket-name embeddings --index-name images \
--vectors '[{"key":"first","data":{"float32":[1,0,0]}}]'
aws --endpoint-url http://localhost:8333 s3vectors query-vectors \
--vector-bucket-name embeddings --index-name images \
--query-vector '{"float32":[1,0,0]}' --top-k 1
```
The engine registers with the master. Wait for that registration before creating
an index. The Lance worker builds and refreshes ANN indexes as data grows;
queries can scan data before an ANN index is built. The demo's admin UI is at
http://localhost:12535 (user `admin`, password `vector-demo`). Demo credentials
and plaintext cluster connections are for this local example. Production
settings are in the Helm/operator TLS and authentication documentation.
`docker compose down` preserves the named data volume. Add `-v` to discard it.
Override `SEAWEEDFS_IMAGE` to test a newer image. With metrics-enabled engines,
append `--metrics-port=9329 --metrics-ip=0.0.0.0` to the vector command and scrape
`vector:9329/metrics` inside the Compose network.
A native development setup can use `weed mini -vector -vectorBucket=embeddings`
with `weed-vector` beside `weed` or on PATH. `-vector.binary` selects an explicit
binary, and `-vector.metricsPort=9329` enables HTTP readiness and metrics on a
compatible engine. The child inherits the cluster's configuration directory and
stops with mini. Run `weed-worker` separately for ANN maintenance. `weed shell`
provides `s3vectors -operation <Operation> -file request.json` for the same API
operations using trusted cluster credentials.
+34
View File
@@ -0,0 +1,34 @@
services:
seaweedfs:
image: ${SEAWEEDFS_IMAGE:-chrislusf/seaweedfs:4.53}
command: mini -dir=/data -ip=seaweedfs -ip.bind=0.0.0.0 -admin.password=vector-demo -admin.worker.ip=0.0.0.0 -tableBucket=embeddings:LANCE
environment: &environment
WEED_JWT_FILER_SIGNING_KEY: vector-demo-cluster-key
AWS_ACCESS_KEY_ID: vector-demo
AWS_SECRET_ACCESS_KEY: vector-demo-secret
ports:
- "127.0.0.1:8333:8333"
- "127.0.0.1:12535:12535"
volumes:
- data:/data
healthcheck:
test: [CMD-SHELL, "wget -q -O /dev/null http://127.0.0.1:9333/cluster/status"]
interval: 5s
timeout: 3s
retries: 30
vector:
image: ${SEAWEEDFS_IMAGE:-chrislusf/seaweedfs:4.53}
command: vector --listen=vector:8090 --masters=seaweedfs:9333
environment: *environment
depends_on:
seaweedfs:
condition: service_healthy
worker:
image: ${SEAWEEDFS_IMAGE:-chrislusf/seaweedfs:4.53}
command: worker-rust --admin=seaweedfs:12535 --masters=seaweedfs:9333 --namespace=http://seaweedfs:9101
environment: *environment
depends_on:
seaweedfs:
condition: service_healthy
volumes:
data:
+110
View File
@@ -0,0 +1,110 @@
import json
import os
from pathlib import Path
import subprocess
import tempfile
import unittest
class AptInstallTests(unittest.TestCase):
def run_helper(self, sources, fail_action="update", timeout_azure=False):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
apt = root / "apt"
(apt / "sources.list.d").mkdir(parents=True)
(apt / "sources.list").write_text("")
for name, content in sources.items():
(apt / name).write_text(content)
helper = root / "apt-install"
helper.write_text(Path(__file__).with_name("apt-install").read_text()
.replace("/etc/apt", str(apt))
.replace("/var/lib/apt", str(root / "state")))
commands = root / "bin"
commands.mkdir()
fake_apt = commands / "apt-get"
fake_apt.write_text(r"""#!/usr/bin/env python3
import json, os, pathlib, sys
root = pathlib.Path(os.environ["VECTOR_APT_TEST_ROOT"])
apt = root / "apt"
sources = [apt / "sources.list", *apt.glob("sources.list.d/*.list"), *apt.glob("sources.list.d/*.sources")]
content = "".join(p.read_text() for p in sources)
action = next(a for a in sys.argv[1:] if a in ("update", "install", "clean"))
with (root / "calls").open("a") as output:
output.write(json.dumps({"action": action, "sources": content, "args": sys.argv[1:]}) + "\n")
if "azure.archive.ubuntu.com" in content and action == "update" and os.environ["VECTOR_APT_FAIL_ACTION"] == "partial":
sys.exit(100 if "APT::Update::Error-Mode=any" in sys.argv else 0)
if "azure.archive.ubuntu.com" in content and action == os.environ["VECTOR_APT_FAIL_ACTION"]:
sys.exit(100)
""")
fake_apt.chmod(0o755)
fake_timeout = commands / "timeout"
fake_timeout.write_text(r"""#!/usr/bin/env python3
import json, os, pathlib, sys
root = pathlib.Path(os.environ["VECTOR_APT_TEST_ROOT"])
apt = root / "apt"
sources = [apt / "sources.list", *apt.glob("sources.list.d/*.list"), *apt.glob("sources.list.d/*.sources")]
content = "".join(p.read_text() for p in sources)
with (root / "calls").open("a") as output:
output.write(json.dumps({"action": "timeout", "args": sys.argv[1:]}) + "\n")
if os.environ["VECTOR_APT_TIMEOUT_AZURE"] and "azure.archive.ubuntu.com" in content:
sys.exit(124)
os.execvp(sys.argv[3], sys.argv[3:])
""")
fake_timeout.chmod(0o755)
result = subprocess.run(["/bin/sh", str(helper), "fuse"],
env={**os.environ, "PATH": str(commands) + os.pathsep + os.environ["PATH"],
"VECTOR_APT_TEST_ROOT": str(root), "VECTOR_APT_FAIL_ACTION": fail_action,
"VECTOR_APT_TIMEOUT_AZURE": "1" if timeout_azure else ""},
text=True, capture_output=True, timeout=5)
calls = [json.loads(line) for line in (root / "calls").read_text().splitlines()] if (root / "calls").exists() else []
return result, calls
def test_fallback_rewrites_all_source_formats(self):
for protocol in ("http", "https"):
for layout in ("legacy", "deb822", "mixed"):
with self.subTest(protocol=protocol, layout=layout):
sources = {}
if layout in ("legacy", "mixed"):
sources["sources.list"] = f"deb {protocol}://azure.archive.ubuntu.com/ubuntu noble main\n"
if layout in ("deb822", "mixed"):
sources["sources.list.d/ubuntu.sources"] = f"Types: deb\nURIs: {protocol}://azure.archive.ubuntu.com/ubuntu\nSuites: noble\nComponents: main\n"
result, calls = self.run_helper(sources)
self.assertEqual(result.returncode, 0, result.stderr)
installs = [call for call in calls if call["action"] == "install"]
self.assertEqual(len(installs), 1)
self.assertNotIn("azure.archive.ubuntu.com", installs[0]["sources"])
self.assertIn(protocol + "://archive.ubuntu.com", installs[0]["sources"])
def test_install_failure_also_tries_the_next_mirror(self):
result, calls = self.run_helper({"sources.list.d/ubuntu.sources":
"Types: deb\nURIs: http://security.ubuntu.com/ubuntu\nSuites: noble-security\nComponents: main\n"},
fail_action="install")
self.assertEqual(result.returncode, 0, result.stderr)
installs = [call for call in calls if call["action"] == "install"]
self.assertEqual(len(installs), 2)
self.assertIn("azure.archive.ubuntu.com", installs[0]["sources"])
self.assertNotIn("azure.archive.ubuntu.com", installs[1]["sources"])
def test_partial_update_does_not_install_stale_indexes(self):
result, calls = self.run_helper({"sources.list": "deb http://azure.archive.ubuntu.com/ubuntu noble main\n"},
fail_action="partial")
self.assertEqual(result.returncode, 0, result.stderr)
installs = [call for call in calls if call["action"] == "install"]
self.assertEqual(len(installs), 1)
self.assertNotIn("azure.archive.ubuntu.com", installs[0]["sources"])
def test_timeout_advances_to_the_next_mirror(self):
result, calls = self.run_helper({"sources.list": "deb http://azure.archive.ubuntu.com/ubuntu noble main\n"},
fail_action="none", timeout_azure=True)
self.assertEqual(result.returncode, 0, result.stderr)
timeouts = [call for call in calls if call["action"] == "timeout"]
self.assertEqual(len(timeouts), 3)
for call in timeouts:
self.assertEqual(call["args"][0], "--kill-after=5s")
self.assertLessEqual(int(call["args"][1].removesuffix("s")), 180)
installs = [call for call in calls if call["action"] == "install"]
self.assertNotIn("azure.archive.ubuntu.com", installs[0]["sources"])
if __name__ == "__main__":
unittest.main()
+17 -17
View File
@@ -61,7 +61,7 @@ require (
github.com/prometheus/client_golang v1.24.1
github.com/prometheus/client_model v0.6.3
github.com/prometheus/common v0.71.0 // indirect
github.com/prometheus/procfs v0.22.0
github.com/prometheus/procfs v0.23.0
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/seaweedfs/goexif v1.0.3
@@ -84,6 +84,7 @@ require (
github.com/xdg-go/scram v1.2.0
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
go.etcd.io/etcd/api/v3 v3.7.1
go.etcd.io/etcd/client/v3 v3.7.1
go.mongodb.org/mongo-driver v1.17.9
go.opencensus.io v0.24.0 // indirect
@@ -93,7 +94,7 @@ require (
golang.org/x/crypto v0.58.0
golang.org/x/exp v0.0.0-20260813180055-c1d0aacb2297
golang.org/x/image v0.46.0
golang.org/x/net v0.61.0
golang.org/x/net v0.62.0
golang.org/x/oauth2 v0.37.0
golang.org/x/sys v0.49.0
golang.org/x/text v0.43.0 // indirect
@@ -122,7 +123,7 @@ require (
github.com/apache/iceberg-go v0.7.0
github.com/apple/foundationdb/bindings/go v0.0.0-20250911184653-27f7192f47c3
github.com/arangodb/go-driver v1.6.9
github.com/aws/aws-sdk-go-v2 v1.47.5
github.com/aws/aws-sdk-go-v2 v1.47.7
github.com/aws/aws-sdk-go-v2/config v1.33.6
github.com/aws/aws-sdk-go-v2/credentials v1.20.6
github.com/aws/aws-sdk-go-v2/service/s3 v1.114.1
@@ -150,10 +151,10 @@ require (
github.com/redis/go-redis/v9 v9.23.0
github.com/schollz/progressbar/v3 v3.19.1
github.com/seaweedfs/go-fuse/v2 v2.9.4
github.com/shirou/gopsutil/v4 v4.26.7
github.com/shirou/gopsutil/v4 v4.26.10
github.com/tarantool/go-option v1.1.0
github.com/tarantool/go-tarantool/v3 v3.0.2
github.com/testcontainers/testcontainers-go v0.44.0
github.com/testcontainers/testcontainers-go v0.45.0
github.com/tikv/client-go/v2 v2.0.7
github.com/twmb/avro v1.9.0
github.com/xeipuuv/gojsonschema v1.2.0
@@ -174,7 +175,7 @@ require (
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/azure-sdk-for-go/sdk/keyvault/internal v0.7.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/internal v0.1.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Azure/go-ansiterm v0.0.0-20261006220739-8c912ac31dd4 // indirect
github.com/FilenCloudDienste/filen-sdk-go v0.0.39 // indirect
github.com/ProtonMail/gopenpgp/v3 v3.4.1 // indirect
github.com/RoaringBitmap/roaring/v2 v2.26.0 // indirect
@@ -206,7 +207,7 @@ require (
github.com/dave/dst v0.27.2 // indirect
github.com/diskfs/go-diskfs v1.9.4 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-connections v0.8.2 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dromara/dongle v1.0.1 // indirect
github.com/geoarrow/geoarrow-go v0.0.0-20260403143023-f54751c3e3a1 // indirect
@@ -235,11 +236,11 @@ require (
github.com/lib/pq v1.12.3 // indirect
github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
github.com/lpar/calendar v0.2.0 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/magiconair/properties v1.18.12 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.3.0 // indirect
github.com/moby/moby/api v1.55.0 // indirect
github.com/moby/moby/client v0.5.0 // indirect
github.com/moby/go-archive v0.3.3 // indirect
github.com/moby/moby/api v1.56.1 // indirect
github.com/moby/moby/client v0.6.2 // indirect
github.com/moby/patternmatcher v0.6.1 // indirect
github.com/moby/sys/sequential v0.7.0 // indirect
github.com/moby/sys/user v0.4.1 // indirect
@@ -282,7 +283,7 @@ require (
go.opentelemetry.io/proto/otlp v1.11.0 // indirect
go.uber.org/mock v0.5.2 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
go.yaml.in/yaml/v3 v3.0.6 // indirect
golang.org/x/mod v0.42.0 // indirect
gonum.org/v1/gonum v0.17.0 // indirect
)
@@ -295,7 +296,7 @@ require (
cloud.google.com/go/iam v1.14.0 // indirect
cloud.google.com/go/monitoring v1.30.0 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.3
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.24.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.1
github.com/Azure/azure-sdk-for-go/sdk/internal v1.13.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.8.2
@@ -334,7 +335,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/sso v1.38.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.51.6
github.com/aws/smithy-go v1.28.7
github.com/aws/smithy-go v1.28.8
github.com/boltdb/bolt v1.3.1 // indirect
github.com/bradenaw/juniper v0.15.3 // indirect
github.com/buengese/sgzip v0.1.1 // indirect
@@ -352,7 +353,7 @@ require (
github.com/d4l3k/messagediff v1.2.1 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/dropbox/dropbox-sdk-go-unofficial/v6 v6.4.0 // indirect
github.com/ebitengine/purego v0.10.2 // indirect
github.com/ebitengine/purego v0.11.2 // indirect
github.com/elastic/gosigar v0.14.3 // indirect
github.com/emersion/go-message v0.18.2 // indirect
github.com/emersion/go-vcard v0.0.0-20260618161152-d854b7e0e2d3 // indirect
@@ -441,7 +442,7 @@ require (
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pkg/xattr v0.4.12 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6 // indirect
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8 // indirect
github.com/relvacode/iso8601 v1.7.0 // indirect
github.com/rfjakob/eme v1.2.0 // indirect
@@ -479,7 +480,6 @@ require (
github.com/zeebo/blake3 v0.2.4 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.etcd.io/bbolt v1.5.0 // indirect
go.etcd.io/etcd/api/v3 v3.7.1 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.46.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.71.0 // indirect
+34 -33
View File
@@ -34,8 +34,8 @@ filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.3 h1:qgnWTiWs1G7AqcKZegFvkJD1bmXJcaDU3uC0ELSgZ5U=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.23.3/go.mod h1:XtxD48QPInqoj+GMlcRlXHEYsoOxFnP6w2JgIZ9rDS4=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.24.0 h1:5H6YRLGMoC8z5aXpxqcD6FfswI8mwoVNHSecvIoazt4=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.24.0/go.mod h1:Sk63W6X/TElfcE2BknMYg0Q9JuCpebgjnHcbs5md4NM=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.1 h1:u93s+zU2JD62im61Bm5CZIc1ZrOJaIAWEg0WOrMVkEo=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.14.1/go.mod h1:oXtinPO4OLj9d1DOTrqrL1oRwGhcqadvAmrl6wTeGlk=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.4.0 h1:xFaZZ+IubdftrDHnGGwZ6QvQ3KHTtWl2MCK+GMt2vxs=
@@ -54,8 +54,8 @@ github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.7.0 h1:cuiKf1UVyWHu+XSQg
github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.7.0/go.mod h1:9JSyvgXLPAOC4jfhgZg58XeU2FJHsGmxuYSclCFQ4ZY=
github.com/Azure/azure-sdk-for-go/sdk/storage/internal v0.1.0 h1:X/bTwe0qpvmvPPutc+LWo7T4cKMqDP9JHP4k+Co9ZB4=
github.com/Azure/azure-sdk-for-go/sdk/storage/internal v0.1.0/go.mod h1:+kX7ahyFelQ42F3RScM2CFgHJljp0phc/FieKLYVpFI=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/Azure/go-ansiterm v0.0.0-20261006220739-8c912ac31dd4 h1:m2xcjRABtG288vnlV3U/PKh/HnPURxZp50RKNUwBxM8=
github.com/Azure/go-ansiterm v0.0.0-20261006220739-8c912ac31dd4/go.mod h1:MTUE7ZuZO89FAtAwozGg6AWD83pcdefnEq5gF82MB8s=
github.com/Azure/go-ntlmssp v0.1.1 h1:l+FM/EEMb0U9QZE7mKNEDw5Mu3mFiaa2GKOoTSsNDPw=
github.com/Azure/go-ntlmssp v0.1.1/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=
@@ -173,8 +173,8 @@ github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJ
github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
github.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ=
github.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=
github.com/aws/aws-sdk-go-v2 v1.47.5 h1:5yA+A+BiXbDq1ZLbYwPViwhEGDMdR0MEhL8Jh2LnLS4=
github.com/aws/aws-sdk-go-v2 v1.47.5/go.mod h1:iNIkogrRvFwm+1OQIUOP2lPBtlXZboOIUWiCa+pHEMw=
github.com/aws/aws-sdk-go-v2 v1.47.7 h1:jBmtzRtGWVAzbfc4RBWo1dTte0ZZH8p9U42YJIw2+Bw=
github.com/aws/aws-sdk-go-v2 v1.47.7/go.mod h1:I8lFSH4MGrZiw2m7miCrRYHsgpC1+HW8uiVGOiPYPcw=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.20 h1:GPRlPwz40I2B2VrBEASOA3Bi77NyeqejNLkifosX0rs=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.20/go.mod h1:g7PNzKcsOKWb4fkSRBA7BZVAS6Y8IcxzN+nRohhQ1Q8=
github.com/aws/aws-sdk-go-v2/config v1.33.6 h1:MBjkSTLczek/UgiK+EYPIoRTqE7gP8vtW3OFbFo7Nug=
@@ -215,8 +215,8 @@ github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1 h1:orIWdNiLgzrhu/11RcPPKO/S
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.43.1/go.mod h1:skwM/xsbR/1ReUTesv9BhpJp1VjajR7DWQnuVLwiXsQ=
github.com/aws/aws-sdk-go-v2/service/sts v1.51.6 h1:BVjKrtRUtSYKztBCCfE1oAB+d3ViKcW2cKZfserOqMQ=
github.com/aws/aws-sdk-go-v2/service/sts v1.51.6/go.mod h1:eSukFLeLkPMm+1OxDR+F0joauZdyBkF47Ow7MqjZiJw=
github.com/aws/smithy-go v1.28.7 h1:h/n1SHrwVtaDYLz8mR1/Kq4Xh4RaFbFW7bOYckg9mfE=
github.com/aws/smithy-go v1.28.7/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aws/smithy-go v1.28.8 h1:aEmdAq7gVCoGCnvf1NnELlu3I6D/nhqrC321FyFiQkQ=
github.com/aws/smithy-go v1.28.8/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk=
github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg=
github.com/bazelbuild/rules_go v0.46.0 h1:CTefzjN/D3Cdn3rkrM6qMWuQj59OBcuOjyIp3m4hZ7s=
@@ -389,8 +389,8 @@ github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaft
github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.9.8 h1:bIREROb7So6PRlq6KTtdS9MPEjC29OQRkFNlvK2OX8Q=
github.com/docker/docker-credential-helpers v0.9.8/go.mod h1:v1S+hepowrQXITkEfw6o4+BMbGot02wiKpzWhGUZK6c=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc=
github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dromara/dongle v1.0.1 h1:si/7UP/EXxnFVZok1cNos70GiMGxInAYMilHQFP5dJs=
@@ -407,8 +407,8 @@ github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 h1:Oy0F4A
github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3/go.mod h1:YvSRo5mw33fLEx1+DlK6L2VV43tJt5Eyel9n9XBcR+0=
github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/ebitengine/purego v0.10.2 h1:W809HbnvzAxgdm+aOvlSekrM16wGCdT/e76+9tS7gzE=
github.com/ebitengine/purego v0.10.2/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/ebitengine/purego v0.11.2 h1:WWiTx8rD8zYdOwSj8O/NiNbsOvSAgJMTlyV1XlbUSeU=
github.com/ebitengine/purego v0.11.2/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 h1:XBBHcIb256gUJtLmY22n99HaZTz+r2Z51xUPi01m3wg=
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203/go.mod h1:E1jcSv8FaEny+OP/5k9UxZVw9YFWGj7eI4KR/iOBqCg=
github.com/elastic/gosigar v0.14.3 h1:xwkKwPia+hSfg9GqrCUKYdId102m9qTJIIr7egmK/uo=
@@ -827,8 +827,8 @@ github.com/lpar/calendar v0.2.0 h1:A1kxv6sbvBHFUkd2XotanIRqEXQGreQOeuGhkJqIaRA=
github.com/lpar/calendar v0.2.0/go.mod h1:fsVJa4o2NvXYzaCE5RXMadqbZzWEhNCGOUPVHTIZdjc=
github.com/lufia/plan9stats v0.0.0-20260627054121-477a66015f15 h1:YkjVPl/YH5XlJ+/NiwzJtPYXXKRcyjmEUhsDci6YK3c=
github.com/lufia/plan9stats v0.0.0-20260627054121-477a66015f15/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/magiconair/properties v1.18.12 h1:sT9zQpvTB3B4gzrX0tmZNTEaGyg8Zw55MFYRE32Mr9I=
github.com/magiconair/properties v1.18.12/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mailru/easyjson v0.9.2 h1:dX8U45hQsZpxd80nLvDGihsQ/OxlvTkVUXH2r/8cb2M=
github.com/mailru/easyjson v0.9.2/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
@@ -865,20 +865,22 @@ github.com/moby/buildkit v0.31.1 h1:j3p55abBl4kiXXPZgYX+6zWgB2aefqHXoPown12fIzU=
github.com/moby/buildkit v0.31.1/go.mod h1:YM5iNEbNCc6L1Zt3YWFB/aXNLufvf4Rcu0DPlc9HwQg=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.3.0 h1:nos4BtzzUIqB406BgQnWGMI4qib9BZ8XUHU+ucv/n1c=
github.com/moby/go-archive v0.3.0/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
github.com/moby/go-archive v0.3.3 h1:OxxR9paxsluYi+zDUEXTTaIxtkK3viymW+Ka7vRhhME=
github.com/moby/go-archive v0.3.3/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/moby/api v1.55.0 h1:2/sexvQyqIWS8pRSCFddBfpW2qE7vR7FCL+vN8pxwMc=
github.com/moby/moby/api v1.55.0/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.5.0 h1:5XhyPk2fuOWf6RlSFa3MkIIgDZkF25xToXW8Q/BH7cc=
github.com/moby/moby/client v0.5.0/go.mod h1:rcVpF8ncl9vo5gaIBdol6CnbEtSj1uxMvEV/UrykF/s=
github.com/moby/moby/api v1.56.1 h1:PpWkvVPB7Fr/No8w+TfyJ/I6rWZ2YPZhT3JorIg+06c=
github.com/moby/moby/api v1.56.1/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk=
github.com/moby/moby/client v0.6.2 h1:ekSPlMccrZGcw6Scb8T9D39qE0mB+XupU/yS0zNw9N4=
github.com/moby/moby/client v0.6.2/go.mod h1:DGcnO7qmPJrqobx/ggQgQ/AUk+2wgVdQv7uxB0/mEsc=
github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw=
github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs=
github.com/moby/sys/capability v0.4.0 h1:4D4mI6KlNtWMCM1Z/K0i7RV1FkX+DBDHKVJpCndZoHk=
github.com/moby/sys/capability v0.4.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I=
github.com/moby/sys/mount v0.3.5 h1:eS3fsZTjHaBihwjp4/+5Z3jxqLXYsbwxqpVSfFv3M00=
github.com/moby/sys/mount v0.3.5/go.mod h1:WUQDO+/uCiCIkIztx8SrwIDVn2dtMFRBebRhpDFT71M=
github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg=
github.com/moby/sys/mountinfo v0.7.2/go.mod h1:1YOa8w8Ih7uW0wALDUgT1dTTSBrZ+HiBLGws92L2RU4=
github.com/moby/sys/sequential v0.7.0 h1:ASQNGNROJSuOO6LL6bPHbKvuZu6NU8P4ldPWk31zj/8=
@@ -1014,8 +1016,8 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.2.3 h1:NP0eAhjcjImqslEwo/1hq7gpajME0fTLTezBKDqfXqo=
github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6 h1:jL3a8soXdzuTCcRnKhOmtcsVOObdDTFf4O2B403HPRU=
github.com/power-devops/perfstat v0.0.0-20260805114148-88456608a4f6/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/pquerna/cachecontrol v0.2.0 h1:vBXSNuE5MYP9IJ5kjsdo8uq+w41jSPgvba2DEnkRx9k=
github.com/pquerna/cachecontrol v0.2.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI=
github.com/pquerna/ffjson v0.0.0-20190930134022-aa0246cd15f7/go.mod h1:YARuvh7BUWHNhzDq2OM5tzR2RiCcN2D7sapiKyCel/M=
@@ -1039,8 +1041,8 @@ github.com/prometheus/common v0.71.0/go.mod h1:CLJ5H8TEsGX8bl31BdMkfhIZ+QmZ9tBPP
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/prometheus/procfs v0.22.0 h1:6q9+/JL9IKAPbCmBrv9n5O5Ty3NKnciV5X7YGw0oics=
github.com/prometheus/procfs v0.22.0/go.mod h1:CvmFr/GVhIjIvWJZW3tgkODBQMRIf0EyWMQLHCHab58=
github.com/prometheus/procfs v0.23.0 h1:ad+S6FTkzwaMgacKLM5h6bNKExS+DMH0YiUwXItH/o4=
github.com/prometheus/procfs v0.23.0/go.mod h1:3AVXO0FqLafyYFU/V/IQRZlC8Rgqz8Dm/x6pKLK+RIk=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8 h1:Y258uzXU/potCYnQd1r6wlAnoMB68BiCkCcCnKx1SH8=
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8/go.mod h1:bSJjRokAHHOhA+XFxplld8w2R/dXLH7Z3BZ532vhFwU=
github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg=
@@ -1114,8 +1116,8 @@ github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/shibumi/go-pathspec v1.3.0 h1:QUyMZhFo0Md5B8zV8x2tesohbb5kfbpTi9rBnKh5dkI=
github.com/shibumi/go-pathspec v1.3.0/go.mod h1:Xutfslp817l2I1cZvgcfeMQJG5QnU2lh5tVaaMCl3jE=
github.com/shirou/gopsutil/v4 v4.26.7 h1:IXzpHz/dkMRYAhKkOXr1HB6SuzWU3eoyyeWe7g3bNZc=
github.com/shirou/gopsutil/v4 v4.26.7/go.mod h1:5O9FjBiXoTDFatIWjZZosqj4pV0DRtLx598xGbBehzM=
github.com/shirou/gopsutil/v4 v4.26.10 h1:MJN6euesJx53MVEjscYeyRSRcrdIcy3sz5k9nosjldg=
github.com/shirou/gopsutil/v4 v4.26.10/go.mod h1:XpkIrcUMQhwte/dGxu57KtnMMFsbKelI7PP23zweKBk=
github.com/sigstore/sigstore v1.10.8 h1:1Mgkxvkw4AXMfIP1DOjc6kw0GkUgA8pGVpveN/EfOq4=
github.com/sigstore/sigstore v1.10.8/go.mod h1:f9+B/4iaYimvUkySyb2mvc73n3RLqNn24grHZM/ET8M=
github.com/sigstore/sigstore-go v1.2.1 h1:YWP/rDbBaEBvtbkj6xtwsSj38ZCFEhTVVadNOXjVe3A=
@@ -1198,8 +1200,8 @@ github.com/tarantool/go-option v1.1.0 h1:ShoOhNsdL41sRpm4hXCRDjV8H0WzPkd4UnKhLKb
github.com/tarantool/go-option v1.1.0/go.mod h1:hMr9z2JXOWlgdCBpCPSL2nwp8718GKYvNBJ+ZuzJbCo=
github.com/tarantool/go-tarantool/v3 v3.0.2 h1:9ZtHllun80QX7KS9tqd3dXa+QAu2BfqFtX1ZWcNnaW8=
github.com/tarantool/go-tarantool/v3 v3.0.2/go.mod h1:TXxLWhUCgdxXFfelnTSkq+goKRTRj660zxq4/WXPe8k=
github.com/testcontainers/testcontainers-go v0.44.0 h1:/Fwh6HY1mIikhnm9e7HwoxGycx0lzRAE0f5VQpjFxzI=
github.com/testcontainers/testcontainers-go v0.44.0/go.mod h1:IcnwQrYTO86xHXu5bvMaBH7ATlbS3Qn1M1QWW3c66rE=
github.com/testcontainers/testcontainers-go v0.45.0 h1:ss9Stm9KDX6JLgGL4jr2fC5dwpabTbCBsRcGadBPR9k=
github.com/testcontainers/testcontainers-go v0.45.0/go.mod h1:VMJZ7M+qhgfEHWNbtjm596OyEOraULSTDaus/h0hmJI=
github.com/testcontainers/testcontainers-go/modules/compose v0.44.0 h1:8YcW51jhgpkkiRVe10Wj9TCBthJmoNpU2fK5WSf7TQ8=
github.com/testcontainers/testcontainers-go/modules/compose v0.44.0/go.mod h1:0du5YaH5n8Of4bDM/RwIt421Q14kvqrXZ0mB8EjuMoo=
github.com/the42/cartconvert v0.0.0-20131203171324-aae784c392b8 h1:I4DY8wLxJXCrMYzDM6lKCGc3IQwJX0PlTLsd3nQqI3c=
@@ -1421,8 +1423,8 @@ go.uber.org/zap v1.28.0 h1:IZzaP1Fv73/T/pBMLk4VutPl36uNC+OSUh3JLG3FIjo=
go.uber.org/zap v1.28.0/go.mod h1:rDLpOi171uODNm/mxFcuYWxDsqWSAVkFdX4XojSKg/Q=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go.yaml.in/yaml/v3 v3.0.6 h1:zcDfvjXEqTQmpjniUBLc91xYJMB2c4QOFxEF/LCEckM=
go.yaml.in/yaml/v3 v3.0.6/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
gocloud.dev v0.47.0 h1:qN4/0vtutn97f1X5ziisf3WXuaFL9AR0FIRnyJRYzgA=
@@ -1497,8 +1499,8 @@ golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.16.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/net v0.62.0 h1:18ELalfM5+XNRwzM2nYqXsZQ9ByS1uhj2O/U7+5jqRY=
golang.org/x/net v0.62.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.37.0 h1:JUlcxA8oAtauLfiH8FX2/FkAWHAdi0QtGCGc+hofE98=
@@ -1540,7 +1542,6 @@ golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+14 -18
View File
@@ -157,12 +157,20 @@ worker_asset_name() {
echo "weed-worker_${os}_${arch}.tar.gz"
}
# The vector engine behind the S3 Vectors API, built like the worker.
# The vector engine behind the S3 Vectors API, published for linux only like
# the worker.
vector_asset_name() {
local os="$1" arch="$2"
echo "weed-vector_${os}_${arch}.tar.gz"
}
vector_platform_supported() {
case "$1/$2" in
linux/amd64|linux/arm64) return 0 ;;
*) return 1 ;;
esac
}
# Install a single component
install_component() {
local component="$1" os="$2" arch="$3"
@@ -272,15 +280,7 @@ install_component() {
;;
vector)
# Published for linux amd64/arm64 only, like the worker.
case "$os" in
linux) ;;
*) error "The vector engine is not available for ${os}. Supported: linux" ;;
esac
case "$arch" in
amd64|arm64) ;;
*) error "The vector engine is not available for ${arch}. Supported: amd64, arm64" ;;
esac
vector_platform_supported "$os" "$arch" || error "The vector engine is not available for ${os}/${arch}. Supported: linux/amd64, linux/arm64"
asset_name="$(vector_asset_name "$os" "$arch")"
download_url="https://github.com/${REPO}/releases/download/${VERSION}/${asset_name}"
@@ -288,15 +288,11 @@ install_component() {
info "Extracting ${asset_name}..."
tar xzf "${tmpdir}/${asset_name}" -C "$tmpdir"
local vector_bin
vector_bin="$(find "$tmpdir" -name 'weed-vector' -type f | head -1)"
if [ -z "$vector_bin" ]; then
error "Could not find weed-vector binary in archive"
fi
vector_bin="$(find "$tmpdir" -name weed-vector -type f | head -1)"
[ -n "$vector_bin" ] || error "Could not find weed-vector binary in archive"
chmod +x "$vector_bin"
install_binary "$vector_bin" "weed-vector"
install_binary "$vector_bin" weed-vector
VECTOR_INSTALLED=true
ok "Installed weed-vector to ${INSTALL_DIR}/weed-vector"
;;
@@ -364,7 +360,7 @@ main() {
else
install_component "worker-rust" "$os" "$arch"
fi
if [ "$os" != "linux" ] || { [ "$arch" != "amd64" ] && [ "$arch" != "arm64" ]; }; then
if ! vector_platform_supported "$os" "$arch"; then
warn "Skipping the vector engine: no build for ${os}/${arch}"
elif ! asset_exists "https://github.com/${REPO}/releases/download/${VERSION}/$(vector_asset_name "$os" "$arch")"; then
warn "Skipping the vector engine: ${VERSION} does not carry one"
+1 -1
View File
@@ -3,4 +3,4 @@ description: SeaweedFS
name: seaweedfs
appVersion: "4.53"
# Dev note: Trigger a helm chart release by `git tag -a helm-<version>`
version: 4.53.0
version: 4.53.1
+44
View File
@@ -687,3 +687,47 @@ generates a new UUID and the license stops matching. See
[Master data](#master-data-hostpath-vs-a-claim). Check the binding with
`kubectl exec <master-pod> -- curl -s localhost:9333/license/status`
(`cluster_uuid` must equal `license_uuid`).
## Vector buckets
The S3 gateway serves the S3 Vectors API on its existing endpoint. Enable a vector engine and Lance maintenance worker alongside the cluster:
```yaml
s3:
enabled: true
vector:
enabled: true
replicas: 1
worker:
enabled: true
admin:
enabled: true
secret:
adminPassword: change-me
global:
seaweedfs:
securityConfig:
jwtSigning:
filerWrite: true
```
The engine registers its pod address with the masters and stores indexes through the filers and volume servers. Both Go and Rust volume servers use the same setup. The maintenance worker builds and updates ANN indexes; queries can scan data before indexing completes. Enabling filer write signing also requires clients writing directly to the filer HTTP API to supply a signed JWT.
The bundled engine images support Linux amd64 and arm64. Use `vector.imageOverride` to select a compatible engine build, `vector.masters` for an external cluster, and `vector.resources` to set memory and CPU limits. The cache defaults to 512 MiB and is configured with `vector.readCacheMB`.
`global.seaweedfs.enableSecurity` provisions and mounts the vector certificate and shared security configuration. With external certificates, provide the `<fullname>-vector-cert` secret (`my-seaweedfs-vector-cert` for release `my`). `fullname` follows `seaweedfs.fullname`, including `fullnameOverride`, `nameOverride`, and DNS-name truncation. Provide this secret alongside the existing CA and client secrets.
The default `vector.metricsPort: 0` and TCP probes support existing engine images. The default 4.53 image does not accept metrics flags; enabling metrics with it is rejected at render time. For engine builds with `/metrics`, `/health`, and `/ready`, set `vector.imageOverride` to that compatible image and `vector.metricsPort: 9329`; readiness then checks master registration. Enabling `global.seaweedfs.monitoring.enabled` also creates the vector ServiceMonitor. NetworkPolicy includes the configured gRPC and metrics ports.
To provision vector buckets during installation and upgrades, set:
```yaml
vector:
enabled: true
createBuckets:
- embeddings
```
The hook uses the AWS CLI's S3 Vectors API and works with existing SeaweedFS vector releases. It creates missing buckets, preserves existing vector buckets, and fails if the name belongs to an Iceberg bucket. It uses the chart's generated S3 admin credentials, including `s3.credentials.admin.existingSecret`. With a custom `s3.existingConfigSecret` or an external endpoint, set `vector.bucketHook.credentialsSecret` to a Secret containing `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (and optionally `AWS_SESSION_TOKEN`). Key names are configurable. `vector.bucketHook.endpoint` overrides the internal S3 service URL; `caBundleSecret` and `caBundleKey` configure a private HTTPS CA. The hook image and resource limits are configurable under `vector.bucketHook`. For restricted Pod Security, select a non-root-compatible image and configure `vector.bucketHook.podSecurityContext` and `containerSecurityContext`, including `runAsNonRoot` and the image user ID.
@@ -3951,6 +3951,970 @@
],
"title": "Filer Object Size Distribution",
"type": "bargauge"
},
{
"type": "row",
"title": "S3 Vectors",
"collapsed": true,
"id": 95,
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 139
},
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 140
},
"id": 96,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "sum by (type, bucket) (rate(SeaweedFS_s3_request_total{namespace=~\"$NAMESPACE\",type=~\"s3vectors\\\\..*\"}[$__rate_interval]))",
"range": true,
"refId": "A",
"legendFormat": "{{type}} {{bucket}}"
}
],
"title": "Vector API requests by bucket",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "s",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 140
},
"id": 97,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "histogram_quantile(0.95, sum by (le, bucket) (rate(SeaweedFS_s3_request_seconds_bucket{namespace=~\"$NAMESPACE\",type=\"s3vectors.QueryVectors\"}[$__rate_interval])))",
"range": true,
"refId": "A",
"legendFormat": "{{bucket}}"
}
],
"title": "Vector query p95 latency",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 148
},
"id": 98,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "sum by (type, code) (rate(SeaweedFS_s3_request_total{namespace=~\"$NAMESPACE\",type=~\"s3vectors\\\\..*\",code=~\"[45]..\"}[$__rate_interval]))",
"range": true,
"refId": "A",
"legendFormat": "{{type}} {{code}}"
}
],
"title": "Vector API errors",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 148
},
"id": 99,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "last_over_time(SeaweedFS_vector_connected{namespace=~\"$NAMESPACE\"}[5m]) * on (cluster, namespace, instance, job) up{namespace=~\"$NAMESPACE\"}",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
}
],
"title": "Registered vector engines",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "reqps",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 156
},
"id": 100,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "sum by (type, code) (rate(SeaweedFS_vector_request_total{namespace=~\"$NAMESPACE\",code!=\"OK\"}[$__rate_interval]))",
"range": true,
"refId": "A",
"legendFormat": "{{type}} {{code}}"
}
],
"title": "Vector engine request failures",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 156
},
"id": 101,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "sum by (type) (SeaweedFS_vector_in_flight_requests{namespace=~\"$NAMESPACE\"})",
"range": true,
"refId": "A",
"legendFormat": "{{type}}"
}
],
"title": "Vector engine requests in flight",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "percentunit",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 164
},
"id": 102,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "(sum by (cluster, namespace, job, instance) (rate(SeaweedFS_vector_cache_requests_total{namespace=~\"$NAMESPACE\",result=\"hit\"}[$__rate_interval])) or on (cluster, namespace, job, instance) (0 * sum by (cluster, namespace, job, instance) (rate(SeaweedFS_vector_cache_requests_total{namespace=~\"$NAMESPACE\"}[$__rate_interval])))) / sum by (cluster, namespace, job, instance) (rate(SeaweedFS_vector_cache_requests_total{namespace=~\"$NAMESPACE\"}[$__rate_interval]))",
"range": true,
"refId": "A",
"legendFormat": "{{cluster}} {{namespace}} {{instance}}"
}
],
"title": "Vector cache hit ratio",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "bytes",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 164
},
"id": 103,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "SeaweedFS_vector_cache_bytes{namespace=~\"$NAMESPACE\"}",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
}
],
"title": "Vector cache memory",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "Bps",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 172
},
"id": 104,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "rate(SeaweedFS_vector_storage_read_bytes_total{namespace=~\"$NAMESPACE\"}[$__rate_interval])",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
}
],
"title": "Vector storage read bandwidth",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"description": "Vector engine metrics require HTTP metrics to be enabled. Gateway bucket labels require vector bucket attribution.",
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 4,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "ops",
"unitScale": true
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 172
},
"id": 105,
"links": [],
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "8.1.2",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "sum by (instance) (rate(SeaweedFS_worker_lance_vector_indexes_built_total{namespace=~\"$NAMESPACE\"}[$__rate_interval]))",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
}
],
"title": "Vector index maintenance builds",
"type": "timeseries"
}
]
}
],
"refresh": "",
@@ -4039,6 +5003,6 @@
"timezone": "browser",
"title": "SeaweedFS",
"uid": "a24009d7-cbda-4443-a132-1cc1c4677304",
"version": 1,
"version": 2,
"weekStart": ""
}
@@ -0,0 +1,50 @@
{{- include "seaweedfs.compat" . -}}
{{- if and .Values.vector.enabled .Values.global.seaweedfs.enableSecurity (not .Values.certificates.externalCertificates.enabled)}}
apiVersion: cert-manager.io/v1{{ if .Values.global.seaweedfs.certificates.alphacrds }}alpha1{{ end }}
kind: Certificate
metadata:
name: {{ include "seaweedfs.fullname" . }}-vector-cert
namespace: {{ include "seaweedfs.namespace" . | quote }}
labels:
app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
app.kubernetes.io/component: vector
{{- with include "seaweedfs.commonLabels" $ }}{{ . | nindent 4 }}{{ end }}
{{- if .Values.vector.annotations }}
annotations:
{{- toYaml .Values.vector.annotations | nindent 4 }}
{{- end }}
spec:
secretName: {{ include "seaweedfs.fullname" . }}-vector-cert
issuerRef:
name: {{ include "seaweedfs.fullname" . }}-ca-issuer
kind: Issuer
commonName: {{ .Values.certificates.commonName }}
subject:
organizations:
- "SeaweedFS CA"
dnsNames:
- '*.{{ include "seaweedfs.fullname" . }}-vector'
- '*.{{ include "seaweedfs.fullname" . }}-vector.{{ include "seaweedfs.namespace" . }}'
- '*.{{ include "seaweedfs.fullname" . }}-vector.{{ include "seaweedfs.namespace" . }}.svc'
- '*.{{ include "seaweedfs.fullname" . }}-vector.{{ include "seaweedfs.namespace" . }}.svc.cluster.local'
{{- if .Values.certificates.dnsNames }}
{{- range .Values.certificates.dnsNames }}
- {{ . | quote }}
{{- end }}
{{- end }}
{{- if .Values.certificates.ipAddresses }}
ipAddresses:
{{- range .Values.certificates.ipAddresses }}
- {{ . }}
{{- end }}
{{- end }}
privateKey:
algorithm: {{ .Values.certificates.keyAlgorithm }}
size: {{ .Values.certificates.keySize }}
duration: {{ .Values.certificates.duration }}
renewBefore: {{ .Values.certificates.renewBefore }}
{{- end }}
@@ -770,3 +770,15 @@ on install.
{{- $context := merge (dict "Values" (omit .Values "commonLabels")) (omit . "Values") -}}
{{- include (print .Template.BasePath "/master/master-configmap.yaml") $context | sha256sum -}}
{{- end -}}
{{- define "seaweedfs.vector.selectorLabels" -}}
app.kubernetes.io/name: {{ include "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
app.kubernetes.io/component: vector
{{- end -}}
{{- define "seaweedfs.vector.labels" -}}
{{ include "seaweedfs.vector.selectorLabels" . }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
{{- with include "seaweedfs.commonLabels" . }}{{ . | nindent 0 }}{{ end }}
{{- end -}}
@@ -110,6 +110,18 @@
{{- $targets = append $targets (dict "component" "worker" "ports" $ports) }}
{{- end }}
{{- if .Values.vector.enabled }}
{{- $ports := list .Values.vector.port }}
{{- if .Values.vector.metricsPort }}
{{- $ports = append $ports .Values.vector.metricsPort }}
{{- end }}
{{- $targets = append $targets (dict "component" "vector" "ports" $ports) }}
{{- end }}
{{- if and .Values.vector.enabled .Values.vector.createBuckets }}
{{- $targets = append $targets (dict "component" "vector-bucket-hook" "ports" (list)) }}
{{- end }}
{{- if .Values.cosi.enabled }}
{{- /* The provisioner sidecar watches the COSI CRs, so it needs the API
server; nothing needs to reach the driver itself. */}}
@@ -143,7 +155,7 @@
every component the chart can produce rather than the ones enabled here,
so one values file can still carry overrides for a component this release
leaves off. */}}
{{- $known := list "master" "filer" "s3" "sftp" "admin" "worker" "volume" "bucket-hook" "volume-resize-hook" "objectstorage-provisioner" "seaweedfs-all-in-one" }}
{{- $known := list "vector" "vector-bucket-hook" "master" "filer" "s3" "sftp" "admin" "worker" "volume" "bucket-hook" "volume-resize-hook" "objectstorage-provisioner" "seaweedfs-all-in-one" }}
{{- range $vname, $_ := (.Values.volumes | default dict) }}
{{- $known = append $known (trimSuffix "-" (printf "volume-%s" $vname)) }}
{{- end }}
@@ -215,8 +215,11 @@ spec:
{{- end }}
{{- end }}
{{- range $createBuckets }}
{{- if .ttl }}
/bin/echo 'fs.configure -locationPrefix={{ $bucketsFolder }}/{{ .name }}/ -ttl={{ .ttl }} -apply' | /usr/bin/weed shell
{{- if not (or (kindIs "invalid" .replication) (and (kindIs "string" .replication) (or (eq .replication "") (regexMatch "^([01][0-9]{2}|2[0-4][0-9]|25[0-5])$" .replication)))) }}
{{- fail (printf "createBuckets %s: replication must be a quoted three-digit string from 000 to 255 such as \"001\" (use --set-string)" .name) }}
{{- end }}
{{- if or .ttl .replication .disk }}
/bin/echo 'fs.configure -locationPrefix={{ $bucketsFolder }}/{{ .name }}/{{- with .ttl }} -ttl={{ . }}{{- end }}{{- with .replication }} -replication={{ . }}{{- end }}{{- with .disk }} -disk={{ . }}{{- end }} -apply' | /usr/bin/weed shell
{{- end }}
{{- end }}
{{- range $createBuckets }}
@@ -94,6 +94,10 @@ data:
cert = "/usr/local/share/ca-certificates/admin/tls.crt"
key = "/usr/local/share/ca-certificates/admin/tls.key"
[grpc.vector]
cert = "/usr/local/share/ca-certificates/vector/tls.crt"
key = "/usr/local/share/ca-certificates/vector/tls.key"
[grpc.worker]
cert = "/usr/local/share/ca-certificates/worker/tls.crt"
key = "/usr/local/share/ca-certificates/worker/tls.key"
@@ -0,0 +1,167 @@
{{- include "seaweedfs.compat" . -}}
{{- if and .Values.vector.enabled .Values.vector.createBuckets }}
{{- $hook := .Values.vector.bucketHook }}
{{- $component := "s3" }}
{{- $port := .Values.s3.port }}
{{- $enabled := .Values.s3.enabled }}
{{- $auth := .Values.s3.enableAuth }}
{{- $configSecret := .Values.s3.existingConfigSecret }}
{{- if and .Values.allInOne.enabled .Values.allInOne.s3.enabled }}
{{- $component = "all-in-one" }}
{{- $port = .Values.allInOne.s3.port | default .Values.s3.port }}
{{- $enabled = true }}
{{- $auth = or .Values.allInOne.s3.enableAuth .Values.s3.enableAuth .Values.filer.s3.enableAuth }}
{{- $configSecret = or .Values.allInOne.s3.existingConfigSecret .Values.s3.existingConfigSecret .Values.filer.s3.existingConfigSecret }}
{{- else if and (not $enabled) .Values.filer.s3.enabled }}
{{- $component = "filer" }}
{{- $port = .Values.filer.s3.port }}
{{- $enabled = true }}
{{- $auth = .Values.filer.s3.enableAuth }}
{{- $configSecret = .Values.filer.s3.existingConfigSecret }}
{{- end }}
{{- if and (not $enabled) (not $hook.endpoint) }}
{{- fail "vector.createBuckets requires an enabled S3 gateway or vector.bucketHook.endpoint" }}
{{- end }}
{{- range .Values.vector.createBuckets }}
{{- if not (kindIs "string" .) }}{{ fail "vector.createBuckets entries must be bucket names" }}{{ end }}
{{- if not (and (ge (len .) 3) (le (len .) 63) (regexMatch "^[a-z0-9][a-z0-9-]*[a-z0-9]$" .)) }}
{{- fail "vector.createBuckets names must be 3-63 lowercase letters, digits or hyphens" }}
{{- end }}
{{- end }}
{{- $credentials := .Values.s3.credentials | default dict }}
{{- $admin := $credentials.admin | default dict }}
{{- $secret := $hook.credentialsSecret }}
{{- $accessKey := $hook.accessKeyKey }}
{{- $secretKey := $hook.secretKeyKey }}
{{- if and $auth (not $secret) }}
{{- if $configSecret }}{{ fail "vector.bucketHook.credentialsSecret is required with s3.existingConfigSecret" }}{{ end }}
{{- $secret = $admin.existingSecret | default (printf "%s-s3-secret" (include "seaweedfs.fullname" .)) }}
{{- $accessKey = $admin.accessKeyKey | default "admin_access_key_id" }}
{{- $secretKey = $admin.secretKeyKey | default "admin_secret_access_key" }}
{{- end }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ include "seaweedfs.fullname" . }}-vector-bucket-hook
namespace: {{ include "seaweedfs.namespace" . | quote }}
labels:
app.kubernetes.io/name: {{ include "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
app.kubernetes.io/component: vector-bucket-hook
{{- with include "seaweedfs.commonLabels" . }}{{ . | nindent 4 }}{{ end }}
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-weight: "-5"
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
backoffLimit: 2
activeDeadlineSeconds: 360
template:
metadata:
labels:
app.kubernetes.io/name: {{ include "seaweedfs.name" . }}
app.kubernetes.io/instance: {{ .Release.Name | quote }}
app.kubernetes.io/component: vector-bucket-hook
spec:
restartPolicy: Never
automountServiceAccountToken: false
{{- with $hook.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.vector.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.vector.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- include "seaweedfs.imagePullSecrets" . | nindent 6 }}
containers:
- name: create-vector-buckets
image: {{ $hook.image | quote }}
command: ["/bin/sh", "-ec"]
args:
- |
for bucket in {{ range .Values.vector.createBuckets }}{{ . | quote }} {{ end }}; do
attempt=0
while ! aws --endpoint-url "$S3_ENDPOINT" s3vectors get-vector-bucket --vector-bucket-name "$bucket" >/dev/null 2>/tmp/bucket-error; do
if grep -q NotFound /tmp/bucket-error; then
aws --endpoint-url "$S3_ENDPOINT" s3vectors create-vector-bucket --vector-bucket-name "$bucket" ||
aws --endpoint-url "$S3_ENDPOINT" s3vectors get-vector-bucket --vector-bucket-name "$bucket"
break
fi
if grep -q AccessDenied /tmp/bucket-error; then cat /tmp/bucket-error >&2; exit 1; fi
attempt=$((attempt + 1))
if [ "$attempt" -ge 60 ]; then cat /tmp/bucket-error >&2; exit 1; fi
sleep 5
done
done
env:
- name: S3_ENDPOINT
value: {{ $hook.endpoint | default (printf "http://%s.%s:%d" (include "seaweedfs.componentName" (list . $component)) (include "seaweedfs.namespace" .) (int $port)) | quote }}
- name: AWS_DEFAULT_REGION
value: {{ $hook.region | quote }}
- name: AWS_EC2_METADATA_DISABLED
value: "true"
- name: AWS_PAGER
value: ""
- name: AWS_MAX_ATTEMPTS
value: "2"
- name: AWS_ACCESS_KEY_ID
{{- if $secret }}
valueFrom:
secretKeyRef:
name: {{ $secret | quote }}
key: {{ $accessKey | quote }}
{{- else }}
value: seaweedfs
{{- end }}
- name: AWS_SECRET_ACCESS_KEY
{{- if $secret }}
valueFrom:
secretKeyRef:
name: {{ $secret | quote }}
key: {{ $secretKey | quote }}
{{- else }}
value: seaweedfs
{{- end }}
{{- if $hook.credentialsSecret }}
- name: AWS_SESSION_TOKEN
valueFrom:
secretKeyRef:
name: {{ $secret | quote }}
key: {{ $hook.sessionTokenKey | quote }}
optional: true
{{- end }}
{{- if $hook.caBundleSecret }}
- name: AWS_CA_BUNDLE
value: /etc/ssl/vector/ca.crt
{{- end }}
securityContext:
{{- toYaml $hook.containerSecurityContext | nindent 12 }}
{{- with $hook.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
- name: tmp
mountPath: /tmp
{{- if $hook.caBundleSecret }}
- name: ca
mountPath: /etc/ssl/vector
readOnly: true
{{- end }}
volumes:
- name: tmp
emptyDir: {}
{{- if $hook.caBundleSecret }}
- name: ca
secret:
secretName: {{ $hook.caBundleSecret | quote }}
items:
- key: {{ $hook.caBundleKey | quote }}
path: ca.crt
{{- end }}
{{- end }}
@@ -0,0 +1,166 @@
{{- include "seaweedfs.compat" . -}}
{{- if .Values.vector.enabled }}
{{- if and .Values.vector.metricsPort (not .Values.vector.imageOverride) (eq (toString (.Values.image.tag | default .Chart.AppVersion)) "4.53") }}
{{- fail "vector.metricsPort requires vector.imageOverride selecting an engine with metrics support; the default 4.53 image does not support metrics" }}
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "seaweedfs.componentName" (list . "vector") }}
namespace: {{ include "seaweedfs.namespace" . | quote }}
labels:
{{- include "seaweedfs.vector.labels" . | nindent 4 }}
{{- with .Values.vector.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
replicas: {{ .Values.vector.replicas }}
selector:
matchLabels:
{{- include "seaweedfs.vector.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "seaweedfs.vector.selectorLabels" . | nindent 8 }}
{{- with .Values.podLabels }}{{ toYaml . | nindent 8 }}{{ end }}
{{- with .Values.vector.podLabels }}{{ toYaml . | nindent 8 }}{{ end }}
annotations:
{{- with .Values.podAnnotations }}{{ toYaml . | nindent 8 }}{{ end }}
{{- with .Values.vector.podAnnotations }}{{ toYaml . | nindent 8 }}{{ end }}
spec:
{{- include "seaweedfs.imagePullSecrets" . | nindent 6 }}
automountServiceAccountToken: {{ .Values.global.seaweedfs.automountServiceAccountToken }}
{{- with .Values.vector.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.vector.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.vector.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if and .Values.vector.podSecurityContext (or (not (hasKey .Values.vector.podSecurityContext "enabled")) .Values.vector.podSecurityContext.enabled) }}
securityContext:
{{- omit .Values.vector.podSecurityContext "enabled" | toYaml | nindent 8 }}
{{- end }}
containers:
- name: vector
image: {{ .Values.vector.imageOverride | default (include "seaweedfs.image" .) | quote }}
imagePullPolicy: {{ .Values.global.seaweedfs.imagePullPolicy }}
{{- $masters := include "seaweedfs.masterServerArg" . }}
{{- if .Values.allInOne.enabled }}{{ $masters = include "seaweedfs.cluster.masterAddress" . }}{{ end }}
command: ["/bin/sh", "-ec"]
args:
- 'case "$POD_IP" in *:*) POD_IP="[$POD_IP]";; esac; exec /usr/bin/weed-vector --listen="$POD_IP:{{ .Values.vector.port }}" "$@"'
- vector
- {{ printf "--masters=%s" (.Values.vector.masters | default $masters) | quote }}
- {{ printf "--read-cache-mb=%d" (int .Values.vector.readCacheMB) | quote }}
{{- if .Values.filer.filerGroup }}
- {{ printf "--filer-group=%s" .Values.filer.filerGroup | quote }}
{{- end }}
{{- if .Values.vector.nprobes }}
- {{ printf "--nprobes=%d" (int .Values.vector.nprobes) | quote }}
{{- end }}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- --security-toml=/etc/seaweedfs/security.toml
{{- end }}
{{- if .Values.vector.metricsPort }}
- {{ printf "--metrics-port=%d" (int .Values.vector.metricsPort) | quote }}
- --metrics-ip=$(POD_IP)
{{- end }}
{{- range .Values.vector.extraArgs }}
- {{ . | quote }}
{{- end }}
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
{{- $env := dict }}
{{- include "seaweedfs.mergeExtraEnvironmentVars" (dict "global" .Values.global.seaweedfs "component" .Values.vector "target" $env) }}
{{- range $key := keys $env | sortAlpha }}
- name: {{ $key }}
{{- $value := index $env $key }}
{{- if kindIs "map" $value }}
valueFrom:
{{- toYaml $value | nindent 16 }}
{{- else }}
value: {{ tpl (toString $value) $ | quote }}
{{- end }}
{{- end }}
ports:
- name: grpc
containerPort: {{ .Values.vector.port }}
{{- if .Values.vector.metricsPort }}
- name: metrics
containerPort: {{ .Values.vector.metricsPort }}
{{- end }}
startupProbe:
tcpSocket:
port: grpc
periodSeconds: 5
failureThreshold: 60
readinessProbe:
{{- if .Values.vector.metricsPort }}
httpGet:
path: /ready
port: metrics
{{- else }}
tcpSocket:
port: grpc
{{- end }}
periodSeconds: 5
livenessProbe:
{{- if .Values.vector.metricsPort }}
httpGet:
path: /health
port: metrics
{{- else }}
tcpSocket:
port: grpc
{{- end }}
initialDelaySeconds: 10
periodSeconds: 10
{{- if or (not (hasKey .Values.vector.containerSecurityContext "enabled")) .Values.vector.containerSecurityContext.enabled }}
securityContext:
{{- omit .Values.vector.containerSecurityContext "enabled" | toYaml | nindent 12 }}
{{- end }}
{{- with .Values.vector.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
- name: tmp
mountPath: /tmp
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
mountPath: /etc/seaweedfs
readOnly: true
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- range list "ca" "client" "vector" }}
- name: {{ . }}-cert
mountPath: /usr/local/share/ca-certificates/{{ . }}
readOnly: true
{{- end }}
{{- end }}
volumes:
- name: tmp
emptyDir: {}
{{- if include "seaweedfs.securityConfigEnabled" . }}
- name: security-config
configMap:
name: {{ include "seaweedfs.fullname" . }}-security-config
{{- end }}
{{- if .Values.global.seaweedfs.enableSecurity }}
{{- range list "ca" "client" "vector" }}
- name: {{ . }}-cert
secret:
secretName: {{ include "seaweedfs.fullname" $ }}-{{ . }}-cert
{{- end }}
{{- end }}
{{- end }}
@@ -0,0 +1,22 @@
{{- if .Values.vector.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "seaweedfs.componentName" (list . "vector") }}
namespace: {{ include "seaweedfs.namespace" . | quote }}
labels:
{{- include "seaweedfs.vector.labels" . | nindent 4 }}
spec:
clusterIP: None
selector:
{{- include "seaweedfs.vector.selectorLabels" . | nindent 4 }}
ports:
- name: grpc
port: {{ .Values.vector.port }}
targetPort: grpc
{{- if .Values.vector.metricsPort }}
- name: metrics
port: {{ .Values.vector.metricsPort }}
targetPort: metrics
{{- end }}
{{- end }}
@@ -0,0 +1,25 @@
{{- if and .Values.vector.enabled .Values.vector.metricsPort .Values.global.seaweedfs.monitoring.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "seaweedfs.componentName" (list . "vector") }}
namespace: {{ include "seaweedfs.namespace" . | quote }}
labels:
{{- include "seaweedfs.vector.selectorLabels" . | nindent 4 }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
{{- with include "seaweedfs.commonLabels" (dict "context" $ "exclude" $.Values.global.seaweedfs.monitoring.additionalLabels) }}{{ . | nindent 4 }}{{ end }}
{{- with .Values.global.seaweedfs.monitoring.additionalLabels }}{{ toYaml . | nindent 4 }}{{ end }}
{{- with .Values.vector.serviceMonitor.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
endpoints:
- port: metrics
interval: 30s
scrapeTimeout: 5s
selector:
matchLabels:
{{- include "seaweedfs.vector.selectorLabels" . | nindent 6 }}
{{- end }}
@@ -0,0 +1,113 @@
package tests
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
func TestVectorBucketHook(t *testing.T) {
for _, auth := range []bool{false, true} {
t.Run(fmt.Sprint(auth), func(t *testing.T) {
output, documents := render(t, "s3.enabled=true", "vector.enabled=true", "vector.createBuckets[0]=embeddings", "s3.enableAuth="+fmt.Sprint(auth), "networkPolicy.enabled=true")
var found bool
for _, document := range documents {
metadata := document["metadata"].(map[string]interface{})
if document["kind"] == "Job" && strings.HasSuffix(metadata["name"].(string), "-vector-bucket-hook") {
found = true
if metadata["annotations"].(map[string]interface{})["helm.sh/hook"] != "post-install,post-upgrade" {
t.Fatal("missing upgrade hook")
}
}
}
if !found || !strings.Contains(output, "s3vectors get-vector-bucket") || !strings.Contains(output, "s3vectors create-vector-bucket") {
t.Fatal("missing vector bucket provisioning")
}
if auth && !strings.Contains(output, "key: \"admin_access_key_id\"") {
t.Fatal("missing generated admin credential reference")
}
})
}
}
func TestVectorBucketHookRetriesRacesAndRefusesConflicts(t *testing.T) {
_, documents := render(t, "s3.enabled=true", "vector.enabled=true", "vector.createBuckets[0]=embeddings")
var script string
for _, document := range documents {
if document["kind"] != "Job" {
continue
}
metadata := document["metadata"].(map[string]interface{})
if strings.HasSuffix(metadata["name"].(string), "-vector-bucket-hook") {
pod := document["spec"].(map[string]interface{})["template"].(map[string]interface{})["spec"].(map[string]interface{})
container := pod["containers"].([]interface{})[0].(map[string]interface{})
script = container["args"].([]interface{})[0].(string)
}
}
if script == "" {
t.Fatal("missing hook script")
}
for _, mode := range []string{"missing", "existing", "race", "iceberg", "denied"} {
t.Run(mode, func(t *testing.T) {
dir := t.TempDir()
stub := `#!/bin/sh
shift 2
operation=$2
case "$operation" in
get-vector-bucket)
if [ "$BUCKET_MODE" = denied ]; then echo AccessDenied >&2; exit 1; fi
if [ "$BUCKET_MODE" = existing ] || { [ -f "$BUCKET_STATE" ] && [ "$BUCKET_MODE" != iceberg ]; }; then exit 0; fi
echo NotFoundException >&2; exit 1;;
create-vector-bucket)
touch "$BUCKET_STATE"
if [ "$BUCKET_MODE" = race ] || [ "$BUCKET_MODE" = iceberg ]; then exit 1; fi;;
esac
`
if err := os.WriteFile(filepath.Join(dir, "aws"), []byte(stub), 0755); err != nil {
t.Fatal(err)
}
command := exec.Command("sh", "-ec", script)
command.Env = append(os.Environ(), "PATH="+dir+":"+os.Getenv("PATH"), "S3_ENDPOINT=http://unused", "BUCKET_MODE="+mode, "BUCKET_STATE="+filepath.Join(dir, "state"))
output, err := command.CombinedOutput()
wantFailure := mode == "iceberg" || mode == "denied"
if (err != nil) != wantFailure {
t.Fatalf("mode %s: %v %s", mode, err, output)
}
})
}
}
func TestVectorBucketHookRejectsShellNames(t *testing.T) {
if _, err := exec.LookPath("helm"); err != nil {
t.Skip("helm is not installed")
}
command := exec.Command("helm", "template", "test", "..", "--set", "s3.enabled=true", "--set", "vector.enabled=true", "--set", "vector.createBuckets[0]=bucket;touch /tmp/invalid")
output, err := command.CombinedOutput()
if err == nil || !strings.Contains(string(output), "vector.createBuckets names") {
t.Fatalf("invalid bucket accepted: %v %s", err, output)
}
}
func TestVectorBucketHookGatewayModes(t *testing.T) {
for _, mode := range []string{"s3", "filer", "allInOne"} {
settings := []string{"vector.enabled=true", "vector.createBuckets[0]=embeddings"}
component := "s3"
switch mode {
case "s3":
settings = append(settings, "s3.enabled=true", "s3.enableAuth=true")
case "filer":
settings = append(settings, "s3.enabled=false", "filer.s3.enabled=true", "filer.s3.enableAuth=true")
component = "filer"
case "allInOne":
settings = append(settings, "s3.enabled=false", "allInOne.enabled=true", "allInOne.s3.enabled=true", "allInOne.s3.enableAuth=true")
component = "all-in-one"
}
output, _ := render(t, settings...)
if !strings.Contains(output, "http://vector-test-seaweedfs-"+component+".default:8333") || !strings.Contains(output, "key: \"admin_access_key_id\"") {
t.Fatalf("missing endpoint or auth for %s", mode)
}
}
}
+148
View File
@@ -0,0 +1,148 @@
package tests
import (
"fmt"
"io"
"os"
"os/exec"
"strings"
"testing"
"go.yaml.in/yaml/v3"
)
func render(t *testing.T, settings ...string) (string, []map[string]interface{}) {
t.Helper()
if _, err := exec.LookPath("helm"); err != nil {
t.Skip("helm is not installed")
}
args := []string{"template", "vector-test", ".."}
for _, setting := range settings {
args = append(args, "--set", setting)
}
output, err := exec.Command("helm", args...).CombinedOutput()
if err != nil {
t.Fatalf("helm: %v\n%s", err, output)
}
decoder := yaml.NewDecoder(strings.NewReader(string(output)))
var documents []map[string]interface{}
for {
var document map[string]interface{}
if err := decoder.Decode(&document); err == io.EOF {
break
} else if err != nil {
t.Fatal(err)
}
if document != nil {
documents = append(documents, document)
}
}
return string(output), documents
}
func TestVectorDeployment(t *testing.T) {
_, defaults := render(t)
for _, document := range defaults {
if strings.HasSuffix(fmt.Sprint(document["metadata"].(map[string]interface{})["name"]), "-vector") {
t.Fatal("vector resource enabled by default")
}
}
for _, secure := range []bool{false, true} {
t.Run(fmt.Sprintf("security-%v", secure), func(t *testing.T) {
output, documents := render(t, "vector.enabled=true", "vector.metricsPort=9329", "vector.imageOverride=example/metrics-engine:test", "vector.port=8098", "global.seaweedfs.enableSecurity="+fmt.Sprint(secure), "global.seaweedfs.monitoring.enabled=true", "networkPolicy.enabled=true")
for _, want := range []string{"--listen=\"$POD_IP:8098\"", "--metrics-port=9329", "path: /ready", "path: /health", "kind: ServiceMonitor"} {
if !strings.Contains(output, want) {
t.Fatalf("missing %s", want)
}
}
found := false
for _, document := range documents {
metadata := document["metadata"].(map[string]interface{})
if document["kind"] == "Deployment" && strings.HasSuffix(metadata["name"].(string), "-vector") {
found = true
spec := document["spec"].(map[string]interface{})["template"].(map[string]interface{})["spec"].(map[string]interface{})
if spec["nodeSelector"].(map[string]interface{})["kubernetes.io/os"] != "linux" {
t.Fatal("vector engine must use Linux image platforms")
}
}
}
if !found {
t.Fatal("no vector deployment")
}
if secure && (!strings.Contains(output, "[grpc.vector]") || !strings.Contains(output, "vector-cert")) {
t.Fatal("missing vector TLS configuration")
}
})
}
}
func TestVectorExistingImagesAndExternalCertificates(t *testing.T) {
output, _ := render(t, "vector.enabled=true")
start := strings.Index(output, "# Source: seaweedfs/templates/vector/vector-deployment.yaml")
deployment := strings.Split(output[start:], "\n---")[0]
if strings.Contains(deployment, "--metrics-port") || !strings.Contains(deployment, "tcpSocket:") {
t.Fatal("default probes must support existing vector images")
}
output, _ = render(t, "vector.enabled=true", "global.seaweedfs.enableSecurity=true", "certificates.externalCertificates.enabled=true")
if strings.Contains(output, "kind: Certificate") || !strings.Contains(output, "mountPath: /usr/local/share/ca-certificates/vector") {
t.Fatal("external vector certificates are not mounted")
}
}
func TestVectorIPv6Listen(t *testing.T) {
_, docs := render(t, "vector.enabled=true")
var args []interface{}
for _, doc := range docs {
if doc["kind"] != "Deployment" {
continue
}
pod := doc["spec"].(map[string]interface{})["template"].(map[string]interface{})["spec"].(map[string]interface{})
c := pod["containers"].([]interface{})[0].(map[string]interface{})
if c["name"] == "vector" {
args = c["args"].([]interface{})
}
}
for _, ip := range []string{"10.0.0.20", "fd00::20"} {
dir := t.TempDir()
script := strings.ReplaceAll(args[0].(string), "/usr/bin/weed-vector", "printf '%s\\n'")
cmd := exec.Command("sh", "-ec", script, "vector", "--masters=master")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "POD_IP="+ip)
out, err := cmd.CombinedOutput()
address := ip
if strings.Contains(ip, ":") {
address = "[" + ip + "]"
}
if err != nil || !strings.Contains(string(out), "--listen="+address+":8090") || !strings.Contains(string(out), "--masters=master") {
t.Fatalf("%s: %s %v", ip, out, err)
}
}
}
func TestVectorAllInOneMasters(t *testing.T) {
output, _ := render(t, "vector.enabled=true", "allInOne.enabled=true", "master.enabled=false")
if !strings.Contains(output, "--masters=vector-test-seaweedfs-all-in-one.default:9333") {
t.Fatal("vector cannot discover all-in-one master")
}
output, _ = render(t, "vector.enabled=true", "allInOne.enabled=true", "master.enabled=false", "vector.masters=external:9333")
if !strings.Contains(output, "--masters=external:9333") {
t.Fatal("explicit master override lost")
}
}
func TestVectorNetworkPolicyOverrides(t *testing.T) {
output, _ := render(t, "vector.enabled=true", "s3.enabled=true", "vector.createBuckets[0]=embeddings", "networkPolicy.enabled=true", "networkPolicy.components.vector.extraIngress[0].ports[0].port=9329", "networkPolicy.egress.enabled=true", "networkPolicy.components.vector-bucket-hook.extraEgress[0].ports[0].port=443")
if !strings.Contains(output, "port: 9329") || !strings.Contains(output, "port: 443") {
t.Fatal("vector network overrides missing")
}
}
func TestVectorMetricsImageCompatibility(t *testing.T) {
if _, err := exec.LookPath("helm"); err != nil {
t.Skip("helm is not installed")
}
out, err := exec.Command("helm", "template", "vector-test", "..", "--set", "vector.enabled=true", "--set", "vector.metricsPort=9329").CombinedOutput()
if err == nil || !strings.Contains(string(out), "default 4.53 image does not support metrics") {
t.Fatalf("incompatible image accepted: %v %s", err, out)
}
render(t, "vector.enabled=true", "vector.metricsPort=9329", "vector.imageOverride=example/metrics-engine:test")
}
+66
View File
@@ -1083,12 +1083,16 @@ filer:
# You may specify buckets to be created during the install or upgrade process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
# ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
# replication (e.g., 001) and disk (e.g., ssd) apply to writes into the bucket
# removing ttl, replication or disk later keeps the stored rule; set a new value instead
# objectLock enables S3 Object Lock (irreversible, forces versioning)
# versioning: Enabled or Suspended (or bool true/false)
# createBuckets:
# - name: bucket-a
# anonymousRead: true
# ttl: 7d
# replication: "001"
# disk: ssd
# objectLock: true
# versioning: Enabled
# - name: bucket-b
@@ -1165,12 +1169,16 @@ s3:
# You may specify buckets to be created during the install or upgrade process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
# ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
# replication (e.g., 001) and disk (e.g., ssd) apply to writes into the bucket
# removing ttl, replication or disk later keeps the stored rule; set a new value instead
# objectLock enables S3 Object Lock (irreversible, forces versioning)
# versioning: Enabled or Suspended (or bool true/false)
# createBuckets:
# - name: bucket-a
# anonymousRead: true
# ttl: 7d
# replication: "001"
# disk: ssd
# objectLock: true
# versioning: Enabled
# - name: bucket-b
@@ -1733,6 +1741,60 @@ admin:
serviceMonitor:
annotations: {}
vector:
enabled: false
imageOverride: null
replicas: 1
port: 8090
metricsPort: 0
masters: ""
readCacheMB: 512
nprobes: null
extraArgs: []
extraEnvironmentVars: {}
podLabels: {}
podAnnotations: {}
annotations: {}
nodeSelector:
kubernetes.io/os: linux
tolerations: []
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values: [amd64, arm64]
resources: {}
podSecurityContext: {}
containerSecurityContext:
allowPrivilegeEscalation: false
capabilities:
drop: [ALL]
serviceMonitor:
annotations: {}
createBuckets: []
bucketHook:
podSecurityContext:
seccompProfile:
type: RuntimeDefault
containerSecurityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
image: amazon/aws-cli:2.39.6
endpoint: ""
region: us-east-1
credentialsSecret: ""
accessKeyKey: AWS_ACCESS_KEY_ID
secretKeyKey: AWS_SECRET_ACCESS_KEY
sessionTokenKey: AWS_SESSION_TOKEN
caBundleSecret: ""
caBundleKey: tls.crt
resources: {}
worker:
enabled: false
imageOverride: null
@@ -1953,12 +2015,16 @@ allInOne:
# You may specify buckets to be created during the install process.
# Buckets may be exposed publicly by setting `anonymousRead` to `true`
# ttl format: [1-255][m|h|d|w|M|y] (e.g., 7d)
# replication (e.g., 001) and disk (e.g., ssd) apply to writes into the bucket
# removing ttl, replication or disk later keeps the stored rule; set a new value instead
# objectLock enables S3 Object Lock (irreversible, forces versioning)
# versioning: Enabled or Suspended (or bool true/false)
# createBuckets:
# - name: bucket-a
# anonymousRead: true
# ttl: 7d
# replication: "001"
# disk: ssd
# objectLock: true
# versioning: Enabled
# - name: bucket-b
+83
View File
@@ -0,0 +1,83 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1080" height="650" viewBox="0 0 1080 650" role="img" aria-labelledby="title description">
<title id="title">SeaweedFS: one cluster for objects, files, tables, and vectors</title>
<desc id="description">S3 object storage, a POSIX file system, table buckets with a built-in Iceberg REST Catalog and Lance namespace, and S3 Vector Buckets share one SeaweedFS storage cluster. Masters coordinate the cluster, filers manage namespaces and metadata, and volume servers store replicated or erasure-coded blobs. Cloud caching, cloud tiering, and cross-cluster replication extend the cluster.</desc>
<defs>
<marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#75928e"/>
</marker>
</defs>
<rect width="1080" height="650" rx="20" fill="#f5f9f8"/>
<g font-family="Arial, Helvetica, sans-serif">
<text x="40" y="43" fill="#0f766e" font-size="15" font-weight="700" letter-spacing="2">SEAWEEDFS</text>
<text x="40" y="86" fill="#163832" font-size="32" font-weight="700">One cluster. Objects, files, tables, vectors.</text>
<!-- Four ways to use the same storage cluster. -->
<g>
<rect x="40" y="116" width="238" height="184" rx="14" fill="#fff" stroke="#cbded8"/>
<g transform="translate(62 136)" fill="none" stroke="#187b67" stroke-width="2.2" stroke-linejoin="round">
<path d="m0 10 18-9 18 9-18 9z M0 10v21l18 9 18-9V10 M18 19v21"/>
</g>
<text x="62" y="207" fill="#163832" font-size="22" font-weight="700">Object storage</text>
<text x="62" y="237" fill="#48635e" font-size="18">S3 API</text>
<text x="62" y="261" fill="#48635e" font-size="17">Apps, backups, media</text>
</g>
<g>
<rect x="294" y="116" width="238" height="184" rx="14" fill="#fff" stroke="#cbded8"/>
<g transform="translate(316 140)" fill="none" stroke="#2676a3" stroke-width="2.2" stroke-linejoin="round">
<path d="M0 7V2h15l6 6h17v27H0V7z M0 12h38"/>
</g>
<text x="316" y="207" fill="#163832" font-size="22" font-weight="700">File system</text>
<text x="316" y="237" fill="#48635e" font-size="18">POSIX / FUSE</text>
<text x="316" y="261" fill="#48635e" font-size="17">WebDAV, SFTP, HDFS</text>
</g>
<g>
<rect x="548" y="116" width="238" height="184" rx="14" fill="#fff" stroke="#cbded8"/>
<g transform="translate(570 138)" fill="none" stroke="#7a62ab" stroke-width="2.2" stroke-linejoin="round">
<rect x="0" y="0" width="38" height="36" rx="3"/>
<path d="M0 12h38 M0 24h38 M13 12v24 M26 12v24"/>
</g>
<text x="570" y="207" fill="#163832" font-size="22" font-weight="700">Table buckets</text>
<text x="570" y="237" fill="#48635e" font-size="18">S3 Tables</text>
<text x="570" y="261" fill="#48635e" font-size="17">Iceberg REST Catalog</text>
<text x="570" y="285" fill="#48635e" font-size="17">Lance namespace</text>
</g>
<g>
<rect x="802" y="116" width="238" height="184" rx="14" fill="#e4f5ee" stroke="#187b67" stroke-width="1.5"/>
<g transform="translate(824 138)" fill="none" stroke="#187b67" stroke-width="2.2">
<path d="m5 29 13-22 18 22H5z M18 7v30 M5 29l13 8 18-8"/>
<g fill="#e4f5ee">
<circle cx="5" cy="29" r="4"/>
<circle cx="18" cy="7" r="4"/>
<circle cx="36" cy="29" r="4"/>
<circle cx="18" cy="37" r="4"/>
</g>
</g>
<text x="824" y="207" fill="#163832" font-size="22" font-weight="700">Vector buckets</text>
<text x="824" y="237" fill="#266354" font-size="18">S3 Vectors</text>
<text x="824" y="261" fill="#266354" font-size="17">Search, RAG, embeddings</text>
</g>
<!-- A shared connection expresses capabilities, not request routing. -->
<path d="M159 300v28h762v-28 M413 300v28 M667 300v28 M540 328v30" fill="none" stroke="#75928e" stroke-width="2" marker-end="url(#arrow)"/>
<rect x="40" y="362" width="1000" height="174" rx="16" fill="#163e35"/>
<text x="540" y="400" text-anchor="middle" fill="#fff" font-size="25" font-weight="700">One shared storage cluster</text>
<g fill="#fff" font-size="20" font-weight="700" text-anchor="middle">
<text x="213" y="445">Masters</text>
<text x="540" y="445">Filers</text>
<text x="867" y="445">Volume servers</text>
</g>
<path d="M376 425v51 M704 425v51" stroke="#497266"/>
<g fill="#cfe4dc" font-size="17" text-anchor="middle">
<text x="213" y="473">Coordination &amp; placement</text>
<text x="540" y="473">Namespaces &amp; metadata</text>
<text x="867" y="473">Replicated or erasure-coded blobs</text>
</g>
<text x="540" y="511" text-anchor="middle" fill="#cfe4dc" font-size="17">Add volume servers to grow capacity</text>
<path d="M284 536v23 M796 536v23" stroke="#75928e" stroke-width="2" stroke-dasharray="5 5"/>
<rect x="40" y="560" width="488" height="58" rx="12" fill="#fff" stroke="#cbded8"/>
<rect x="552" y="560" width="488" height="58" rx="12" fill="#fff" stroke="#cbded8"/>
<text x="284" y="596" text-anchor="middle" fill="#305c50" font-size="19">Cloud caching &amp; tiering</text>
<text x="796" y="596" text-anchor="middle" fill="#305c50" font-size="19">Cross-cluster replication</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 5.3 KiB

+1084 -1048
View File
File diff suppressed because it is too large. Load diff

Before

Width:  |  Height:  |  Size: 54 KiB

After

Width:  |  Height:  |  Size: 55 KiB

@@ -286,6 +286,7 @@ message WriteCondition {
IF_EXTENDED_TIME_ELAPSED = 8; // fail if present and extended[ext_key] (unix seconds) is in the future
IF_CHUNKS_EQUAL = 9; // fail unless the stored chunk fid multiset equals fids (absent entry = no chunks)
IF_ENTRY_EQUAL = 10; // fail unless the stored entry equals expected_entry
IF_OBJECT_EXISTS = 11; // fail if absent or a plain directory, one holding no object of its own (If-Match: * on a null version)
}
// Clause is one primitive comparison. IF_ETAG_MATCH holds when the current
// entry's ETag equals any value in etags; IF_ETAG_NOT_MATCH holds when it
File diff suppressed because it is too large. Load diff
+17
View File
@@ -6684,6 +6684,19 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "tonic-health"
version = "0.14.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fcfab99db777fba2802f0dfa861d1628d1ae916fb199d29819941f139ae85082"
dependencies = [
"prost",
"tokio",
"tokio-stream",
"tonic",
"tonic-prost",
]
[[package]]
name = "tonic-prost"
version = "0.14.6"
@@ -7217,6 +7230,7 @@ dependencies = [
"arrow-cast",
"arrow-schema",
"async-trait",
"axum",
"bytes",
"clap",
"futures",
@@ -7225,6 +7239,7 @@ dependencies = [
"lance-linalg",
"moka",
"object_store",
"prometheus",
"prost",
"prost-types",
"protoc-bin-vendored",
@@ -7236,10 +7251,12 @@ dependencies = [
"tokio",
"tokio-stream",
"tonic",
"tonic-health",
"tonic-prost",
"tonic-prost-build",
"tracing",
"tracing-subscriber",
"uuid",
"weed-lance-worker",
"weed-vector-core",
]
+40
View File
@@ -0,0 +1,40 @@
# SeaweedFS vector engine
`weed-vector` serves the S3 Vectors data plane through the S3 gateway. Indexes use Lance tables on the cluster's filers and volume servers. Go and Rust volume servers use the same engine and API.
With a local `weed mini` cluster running:
```sh
weed-vector --masters=localhost:9333 --metrics-port=9329
```
Point an AWS S3 Vectors client at the S3 endpoint, normally `http://localhost:8333`. The engine registers with the masters automatically. `--listen` defaults to `127.0.0.1:8090`; use an address reachable from the gateway when running on another host or in a container.
| Setting | Default | Purpose |
| --- | --- | --- |
| `--masters` | Required | Comma-separated master addresses |
| `--listen` | `127.0.0.1:8090` | gRPC listener and registered address |
| `--filer-group` | Empty | Filer group used for table storage |
| `--read-cache-mb` | `512` | Immutable data/index read cache; zero disables it |
| `--nprobes` | Automatic | ANN partitions searched per query |
| `--metrics-port` | `0` | HTTP metrics and health listener; zero disables it |
| `--metrics-ip` | `127.0.0.1` | HTTP listener address |
| `--security-toml` | Discovered | Cluster certificates, signing keys and encryption keys |
The HTTP listener provides `/health`, `/ready`, and `/metrics`. Readiness requires an active registration with the master. The gRPC listener also implements the standard health service for `vector_pb.VectorService` and the whole server. These health statuses follow master registration.
Metrics use the `SeaweedFS_vector_` prefix:
- `connected`: active master registration.
- `request_total{type,code}`, `request_seconds{type}`, and `in_flight_requests{type}`: engine requests and their gRPC outcomes.
- `query_results_total`: vectors returned by queries.
- `cache_requests_total{result}`, `cache_bytes`, and `cache_entries`: immutable read cache activity and occupancy.
- `storage_read_bytes_total` and `storage_read_seconds`: storage fetches for cacheable immutable reads.
The S3 gateway separately publishes `SeaweedFS_s3_` request metrics with `type="s3vectors.<Operation>"`. Scrape both processes to distinguish gateway latency from engine work. Metrics and HTTP probes are unauthenticated; bind them to an internal address or restrict their network access.
For mutual TLS, configure `[grpc.vector]` for the engine's server certificate, `[grpc.client]` for its master/filer connections, and the cluster CA. HTTPS volume reads also use `[https.client]`. The engine follows the same signing and encryption configuration as the other cluster clients.
Run the Lance maintenance worker against the same masters and Lance namespace to build and update ANN indexes. Queries can scan data that has not been indexed yet. See the [maintenance worker documentation](../seaweed-worker/README.md) and [Helm chart](../k8s/charts/seaweedfs/README.md) for deployment settings.
The integration suite at `test/s3vectors` runs the same API and observability checks against both Go and Rust volume servers, including mutual TLS, signed volume reads, and encrypted volume data.
+1 -1
View File
@@ -170,7 +170,7 @@ async fn open_dataset(
storage_options: Default::default(),
managed_versioning: false,
};
return dataset::open_at(&description, storage.wrapper(id).await?).await;
return dataset::open_at(&description, storage.wrapper(id).await?, None).await;
}
let table = dataset::open(client, &parse_id(id), storage).await?;
Ok(table.dataset)
+232 -2
View File
@@ -2,8 +2,8 @@
//! and version tags act as the lease maintenance must respect.
use std::collections::{HashMap, HashSet};
use std::sync::{LazyLock, Mutex};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use std::sync::{Arc, LazyLock, Mutex, Weak};
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
use anyhow::Result;
use lance::Dataset;
@@ -21,6 +21,11 @@ const FOREIGN_TTL: Duration = Duration::from_secs(24 * 3600);
/// Expired-lease scans run at most once per interval per dataset, not per
/// request.
const REAP_INTERVAL: Duration = Duration::from_secs(60);
/// How long a shared lease stays after the last read that took it.
const SHARED_LINGER: Duration = Duration::from_secs(10);
/// A shared lease is handed out for this long after it was taken, then
/// replaced, so a busy version's tag never ages into a reaper's TTL.
const SHARED_FOR: Duration = Duration::from_secs(300);
/// A process-unique owner id embedded in lease tags so a reaper only applies
/// the short TTL to its own leases.
static INSTANCE: LazyLock<String> = LazyLock::new(|| Uuid::new_v4().simple().to_string());
@@ -138,6 +143,106 @@ fn untrack(tag: &str) {
}
}
/// Leases that reads of one version share: the first read tags the version,
/// reads that follow reuse the tag, and the tag goes once no read has taken it
/// for [`SHARED_LINGER`]. Spares a tag write and delete per read.
pub struct SharedLeases {
/// By location, version, manifest ETag and commit time: a table dropped
/// and created again at its location reuses version numbers, not
/// manifests. A rename commit leaves no ETag.
held: Mutex<Held>,
linger: Duration,
share_for: Duration,
}
type LeaseKey = (String, u64, Option<String>, u128);
#[derive(Default)]
struct Held {
slots: HashMap<LeaseKey, Slot>,
/// Slots are pruned once they reach this many, so a lookup does not
/// scan them all.
prune_at: usize,
}
/// Reads of a version that find no lease wait here for the first to take it.
type Slot = Arc<tokio::sync::Mutex<Weak<SharedLease>>>;
impl Default for SharedLeases {
fn default() -> Self {
Self {
held: Mutex::default(),
linger: SHARED_LINGER,
share_for: SHARED_FOR,
}
}
}
/// A lease held for as long as any read, or the linger after the last one,
/// keeps it.
pub struct SharedLease {
_lease: Lease,
taken: Instant,
}
impl SharedLeases {
/// [`try_pin`], reusing a lease another read of this version holds.
pub async fn try_pin(&self, dataset: &Dataset) -> Result<Option<Arc<SharedLease>>> {
let key = (
dataset.uri().to_string(),
dataset.version().version,
dataset.manifest_location().e_tag.clone(),
dataset.manifest().timestamp_nanos,
);
let slot = match self.held.lock() {
Ok(mut held) => {
if held.slots.len() >= held.prune_at {
// A slot another read has taken but not yet locked stays.
held.slots.retain(|_, slot| {
Arc::strong_count(slot) > 1
|| slot
.try_lock()
.map_or(true, |lease| lease.strong_count() > 0)
});
held.prune_at = (2 * held.slots.len()).max(64);
// A burst's allocation would otherwise make every later
// prune scan it.
let capacity = held.prune_at;
held.slots.shrink_to(capacity);
}
held.slots.entry(key).or_default().clone()
}
Err(_) => Slot::default(),
};
let mut slot = slot.lock().await;
let lease = match slot
.upgrade()
.filter(|lease| lease.taken.elapsed() < self.share_for)
{
Some(lease) => lease,
None => {
let Some(lease) = try_pin(dataset).await? else {
return Ok(None);
};
let lease = Arc::new(SharedLease {
_lease: lease,
taken: Instant::now(),
});
*slot = Arc::downgrade(&lease);
lease
}
};
drop(slot);
if let Ok(handle) = tokio::runtime::Handle::try_current() {
let (lingering, linger) = (lease.clone(), self.linger);
handle.spawn(async move {
tokio::time::sleep(linger).await;
drop(lingering);
});
}
Ok(Some(lease))
}
}
/// Drop lease tags whose owner never released them, so an abandoned lease
/// cannot keep a version out of cleanup forever.
async fn reap_expired(dataset: &Dataset, now: u128) {
@@ -324,6 +429,131 @@ mod tests {
b.release().await.unwrap();
}
/// Reads of one version share a tag, another version gets its own, and
/// the tags go once the linger after the last read has passed.
#[tokio::test]
async fn reads_of_a_version_share_one_lease() {
let dir = TempDir::new().unwrap();
let mut dataset =
testing::write_dataset(&dir, &testing::vectors(4, 4), &["a", "b", "c", "d"]).await;
let leases = SharedLeases {
linger: Duration::from_millis(50),
..Default::default()
};
let a = leases.try_pin(&dataset).await.unwrap().unwrap();
let b = leases.try_pin(&dataset).await.unwrap().unwrap();
assert!(Arc::ptr_eq(&a, &b));
assert_eq!(dataset.tags().list().await.unwrap().len(), 1);
testing::append_dataset(&mut dataset, &testing::vectors(1, 4), &["e"]).await;
let c = leases.try_pin(&dataset).await.unwrap().unwrap();
assert!(!Arc::ptr_eq(&a, &c));
assert_eq!(dataset.tags().list().await.unwrap().len(), 2);
drop((a, b, c));
tokio::time::sleep(leases.linger).await;
let mut left = usize::MAX;
for _ in 0..1000 {
// A listing that meets a tag as it is deleted fails; try again.
if let Ok(tags) = dataset.tags().list().await {
left = tags.len();
}
if left == 0 {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert_eq!(left, 0);
}
/// Reads that arrive together, before any lease exists, share the one the
/// first takes.
#[tokio::test]
async fn concurrent_reads_take_one_lease() {
let dir = TempDir::new().unwrap();
let dataset =
testing::write_dataset(&dir, &testing::vectors(4, 4), &["a", "b", "c", "d"]).await;
let leases = SharedLeases::default();
let pins: Vec<_> = futures::future::join_all((0..8).map(|_| leases.try_pin(&dataset)))
.await
.into_iter()
.map(|lease| lease.unwrap().unwrap())
.collect();
assert!(pins.iter().all(|lease| Arc::ptr_eq(lease, &pins[0])));
assert_eq!(dataset.tags().list().await.unwrap().len(), 1);
}
/// A table dropped and written again at its location gets its own lease
/// for a version number the old table had.
#[tokio::test]
async fn a_recreated_table_takes_its_own_lease() {
let dir = TempDir::new().unwrap();
let rows = testing::vectors(4, 4);
let old = testing::write_dataset(&dir, &rows, &["a", "b", "c", "d"]).await;
let leases = SharedLeases::default();
let first = leases.try_pin(&old).await.unwrap().unwrap();
std::fs::remove_dir_all(dir.path()).unwrap();
let new = testing::write_dataset(&dir, &rows, &["e", "f", "g", "h"]).await;
assert_eq!(new.version().version, old.version().version);
let second = leases.try_pin(&new).await.unwrap().unwrap();
assert!(!Arc::ptr_eq(&first, &second));
assert_eq!(new.tags().list().await.unwrap().len(), 1);
}
/// Slots of released leases are pruned as new versions come in.
#[tokio::test]
async fn released_lease_slots_are_pruned() {
let dir = TempDir::new().unwrap();
let mut dataset = testing::write_dataset(&dir, &testing::vectors(1, 4), &["a"]).await;
let leases = SharedLeases {
linger: Duration::ZERO,
..Default::default()
};
for _ in 0..200 {
drop(leases.try_pin(&dataset).await.unwrap());
tokio::task::yield_now().await;
testing::append_dataset(&mut dataset, &testing::vectors(1, 4), &["b"]).await;
}
assert!(leases.held.lock().unwrap().slots.len() <= 128);
}
/// A slot a read has taken but not yet locked survives a prune, so that
/// read and the next share the lease.
#[tokio::test]
async fn a_taken_slot_survives_a_prune() {
let dir = TempDir::new().unwrap();
let dataset = testing::write_dataset(&dir, &testing::vectors(1, 4), &["a"]).await;
let leases = SharedLeases::default();
let key: LeaseKey = ("elsewhere".into(), 1, None, 0);
let taken = {
let mut held = leases.held.lock().unwrap();
held.prune_at = 0;
held.slots.entry(key.clone()).or_default().clone()
};
leases.try_pin(&dataset).await.unwrap().unwrap();
assert!(Arc::ptr_eq(
&leases.held.lock().unwrap().slots[&key],
&taken
));
}
/// A shared lease is not handed out past SHARED_FOR, so a busy version
/// moves to a fresh tag.
#[tokio::test]
async fn a_shared_lease_is_replaced_when_old() {
let dir = TempDir::new().unwrap();
let dataset =
testing::write_dataset(&dir, &testing::vectors(4, 4), &["a", "b", "c", "d"]).await;
let leases = SharedLeases {
share_for: Duration::from_millis(50),
..Default::default()
};
let a = leases.try_pin(&dataset).await.unwrap().unwrap();
tokio::time::sleep(leases.share_for).await;
let b = leases.try_pin(&dataset).await.unwrap().unwrap();
assert!(!Arc::ptr_eq(&a, &b));
}
/// A stale lease tag left behind is reaped by the next pin; a stale-named
/// tag this process still holds is not.
#[tokio::test]
+4
View File
@@ -13,6 +13,9 @@ name = "weed-vector"
path = "src/main.rs"
[dependencies]
axum = "0.8"
prometheus = { version = "0.13", default-features = false }
tonic-health = "0.14"
weed-vector-core = { path = "../core" }
weed-lance-worker = { path = "../../../seaweed-worker/crates/lance" }
sw-client = { path = "../../../seaweed-worker/crates/sw-client" }
@@ -33,6 +36,7 @@ bytes = "1"
clap = { version = "4", features = ["derive", "env"] }
futures = "0.3"
moka = { version = "0.12", features = ["sync"] }
uuid = { version = "1", features = ["v4"] }
object_store = { version = "0.13", features = ["aws"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
+2
View File
@@ -1,5 +1,7 @@
//! The VectorService gRPC server the S3 gateway talks to.
pub mod metrics;
pub mod observed;
pub mod read_cache;
pub mod service;
+85 -8
View File
@@ -5,12 +5,16 @@ use clap::Parser;
use tokio_stream::wrappers::TcpListenerStream;
use tonic::transport::Server;
use weed_lance_worker::cluster::Cluster;
use weed_vector_server::master_client;
use weed_vector_server::pb::vector::vector_service_server::VectorServiceServer;
use weed_vector_server::read_cache::ReadCache;
use weed_vector_server::security::Security;
use weed_vector_server::service::VectorServer;
use weed_vector_server::tls::{self, ServerTls};
use weed_vector_server::{
master_client,
metrics::{self, Metrics},
observed::ObservedService,
};
/// The VectorService the S3 gateway fans vector calls out to.
#[derive(Parser)]
@@ -18,6 +22,9 @@ struct Args {
/// gRPC listen address.
#[arg(long, env = "WEED_VECTOR_LISTEN", default_value = "127.0.0.1:8090")]
listen: String,
/// Address to register when it differs from the listen address.
#[arg(long, env = "WEED_VECTOR_ADVERTISE")]
advertise: Option<String>,
/// Comma-separated master addresses. The engine registers with them, so
/// the gateway finds it, and asks them for the filers it keeps tables
/// through.
@@ -34,6 +41,13 @@ struct Args {
/// Memory for caching reads of write-once table files, in MiB; 0 disables.
#[arg(long, env = "WEED_VECTOR_READ_CACHE_MB", default_value_t = 512)]
read_cache_mb: u64,
/// Memory for indexes loaded by queries, kept for the next query, in MiB.
#[arg(long, env = "WEED_VECTOR_INDEX_CACHE_MB", default_value_t = 1024)]
index_cache_mb: usize,
/// Memory for table metadata (manifests, file footers) kept across
/// queries, in MiB.
#[arg(long, env = "WEED_VECTOR_METADATA_CACHE_MB", default_value_t = 256)]
metadata_cache_mb: usize,
/// Partitions each query probes in a table's vector index. Default: a
/// tenth of the index's partitions, at least 8; more under a filter.
#[arg(long, env = "WEED_VECTOR_NPROBES")]
@@ -45,6 +59,21 @@ struct Args {
/// processes find in -config_dir, ., ~/.seaweedfs and /etc/seaweedfs.
#[arg(long, env = "WEED_VECTOR_SECURITY_TOML")]
security_toml: Option<PathBuf>,
#[arg(
long,
alias = "metricsPort",
env = "WEED_VECTOR_METRICS_PORT",
default_value_t = 0
)]
metrics_port: u16,
#[arg(
long,
alias = "metricsIp",
env = "WEED_VECTOR_METRICS_IP",
default_value = "127.0.0.1"
)]
metrics_ip: String,
/// The directory weed's -config_dir names, searched for security.toml
/// first.
#[arg(long, alias = "config_dir", env = "WEED_VECTOR_CONFIG_DIR")]
@@ -78,7 +107,13 @@ fn normalize_args(args: Vec<String>) -> Vec<String> {
/// A wildcard listen address is not reachable for peers; derive the outbound
/// IP toward the first master instead — the address the master sees anyway.
fn advertised_addr(listen: &str, masters: &[String]) -> String {
fn advertised_addr(listen: &str, masters: &[String], advertise: Option<&str>) -> String {
if let Some(address) = advertise
.map(str::trim)
.filter(|address| !address.is_empty())
{
return address.to_string();
}
let Some((host, _)) = listen.rsplit_once(':') else {
return listen.to_string();
};
@@ -128,11 +163,32 @@ async fn main() -> Result<()> {
.with_context(|| format!("listen on {}", args.listen))?;
let cluster = Cluster::new(&args.masters, &args.filer_group, &security)?;
let reads = cluster.storage(&security).await?;
let (health, health_service) = tonic_health::server::health_reporter();
let metrics = Metrics::new()?.with_health(health);
metrics.set_connected(false).await;
let cache = (args.read_cache_mb > 0)
.then(|| ReadCache::new(args.read_cache_mb << 20).with_metrics(metrics.clone()));
let mut service = VectorServer::new()
.with_nprobes(args.nprobes.map(std::num::NonZeroUsize::get))
.with_lance_caches(args.index_cache_mb << 20, args.metadata_cache_mb << 20)
.with_storage(reads);
if args.read_cache_mb > 0 {
service = service.with_read_cache(ReadCache::new(args.read_cache_mb << 20));
if let Some(cache) = &cache {
service = service.with_read_cache(cache.clone());
}
if args.metrics_port > 0 {
let ip = args
.metrics_ip
.trim_start_matches('[')
.trim_end_matches(']')
.parse::<std::net::IpAddr>()?;
let listener =
tokio::net::TcpListener::bind(std::net::SocketAddr::new(ip, args.metrics_port)).await?;
let metrics = metrics.clone();
tokio::spawn(async move {
if let Err(error) = metrics::serve_listener(metrics, listener, cache).await {
tracing::error!(%error, "vector metrics server stopped");
}
});
}
tracing::info!(
listen = %args.listen,
@@ -148,19 +204,26 @@ async fn main() -> Result<()> {
);
// Registered only now, with storage connected and the listener bound,
// so the gateway never finds an engine that cannot serve yet.
let advertise = advertised_addr(&args.listen, &cluster.masters);
let advertise = advertised_addr(&args.listen, &cluster.masters, args.advertise.as_deref());
tokio::spawn(master_client::keep_connected(
cluster.masters.clone(),
advertise,
args.filer_group.clone(),
args.metrics_port,
cluster.master_tls.clone(),
metrics.clone(),
));
let router = Server::builder()
.add_service(health_service)
// Match the gateway's 1 GiB cap — put batches over the tonic 4 MiB
// default otherwise get rejected before they reach the handler.
.add_service(
VectorServiceServer::new(service)
.max_decoding_message_size(1 << 30)
.max_encoding_message_size(1 << 30),
VectorServiceServer::new(ObservedService {
inner: service,
metrics,
})
.max_decoding_message_size(1 << 30)
.max_encoding_message_size(1 << 30),
);
match server_tls {
Some(tls) => {
@@ -180,6 +243,20 @@ async fn main() -> Result<()> {
mod tests {
use super::*;
#[test]
fn an_empty_advertise_uses_the_listen_address() {
for value in [None, Some(""), Some(" ")] {
assert_eq!(
advertised_addr("127.0.0.1:8090", &[], value),
"127.0.0.1:8090"
);
}
assert_eq!(
advertised_addr("127.0.0.1:8090", &[], Some("vector.example:8090")),
"vector.example:8090"
);
}
#[test]
fn nprobes_must_be_positive() {
let parse = |nprobes: &str| {
@@ -8,20 +8,22 @@ use std::time::Duration;
use anyhow::{Context, Result};
use tokio::time::sleep;
use crate::master_pb::{KeepConnectedRequest, seaweed_client::SeaweedClient};
use crate::master_pb::{
KeepConnectedRequest, ListClusterNodesRequest, seaweed_client::SeaweedClient,
};
use crate::security::GrpcTls;
pub use weed_lance_worker::cluster::{CONNECT_TIMEOUT, connect, grpc_addr};
fn registration(client_addr: &str) -> KeepConnectedRequest {
fn registration(client_addr: &str, filer_group: &str, metrics_port: u16) -> KeepConnectedRequest {
KeepConnectedRequest {
client_type: "vector".into(),
client_address: client_addr.into(),
version: env!("CARGO_PKG_VERSION").into(),
filer_group: String::new(),
filer_group: filer_group.into(),
data_center: String::new(),
rack: String::new(),
leave_lock_ring: false,
..Default::default()
metrics_port: metrics_port.into(),
}
}
@@ -39,12 +41,22 @@ fn http_addr(master: &str) -> String {
/// Retry forever: try each master until a stream opens, hold it until it
/// errors, then rediscover — same shape as wdclient.KeepConnectedToMaster.
pub async fn keep_connected(masters: Vec<String>, client_addr: String, tls: Option<GrpcTls>) {
let client_addr = client_addr.as_str();
pub async fn keep_connected(
masters: Vec<String>,
client_addr: String,
filer_group: String,
metrics_port: u16,
tls: Option<GrpcTls>,
metrics: crate::metrics::Metrics,
) {
let registration = &registration(&client_addr, &filer_group, metrics_port);
let tls = tls.as_ref();
let metrics = &metrics;
loop {
try_all_masters(&masters, |master| async move {
stream_to(&master, client_addr, tls).await
let result = stream_to(&master, registration, tls, Some(&metrics)).await;
metrics.set_connected(false).await;
result
})
.await;
sleep(Duration::from_secs(3)).await;
@@ -82,15 +94,16 @@ where
async fn stream_to(
master: &str,
client_addr: &str,
registration: &KeepConnectedRequest,
tls: Option<&GrpcTls>,
metrics: Option<&crate::metrics::Metrics>,
) -> Result<Option<String>> {
let channel = connect(master, tls).await?;
// A master answers at once, a follower with its leader hint.
let answer_by = tokio::time::Instant::now() + CONNECT_TIMEOUT;
let mut client = SeaweedClient::new(channel);
let (tx, rx) = tokio::sync::mpsc::channel(8);
tx.send(registration(client_addr)).await?;
tx.send(registration.clone()).await?;
let mut stream = tokio::time::timeout_at(
answer_by,
client.keep_connected(tokio_stream::wrappers::ReceiverStream::new(rx)),
@@ -99,6 +112,8 @@ async fn stream_to(
.context("master did not answer keep-connected")??
.into_inner();
let mut answered = false;
let interval = Duration::from_secs(5);
let mut heartbeat = tokio::time::interval_at(tokio::time::Instant::now() + interval, interval);
// The master keeps the node live while the stream is open; resend the
// registration periodically so a reconnect never leaves a stale entry.
loop {
@@ -117,23 +132,72 @@ async fn stream_to(
return Ok(Some(loc.leader));
}
}
if let Some(metrics) = metrics { metrics.set_connected(true).await; }
}
None => anyhow::bail!("master closed the keep-connected stream"),
},
_ = tokio::time::sleep_until(answer_by), if !answered => {
anyhow::bail!("master did not answer keep-connected");
}
_ = sleep(Duration::from_secs(5)) => {
tx.send(registration(client_addr)).await?;
_ = heartbeat.tick() => {
tokio::time::timeout(CONNECT_TIMEOUT, tx.send(registration.clone())).await
.context("registration send stalled")??;
probe_registration(&mut client).await?;
}
}
}
}
async fn probe_registration(client: &mut SeaweedClient<tonic::transport::Channel>) -> Result<()> {
tokio::time::timeout(
CONNECT_TIMEOUT,
client.list_cluster_nodes(ListClusterNodesRequest {
client_type: "vector".into(),
..Default::default()
}),
)
.await
.context("registration heartbeat stalled")??;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn registration_preserves_filer_group() {
assert_eq!(
registration("vector:8090", "tenant-a", 0).filer_group,
"tenant-a"
);
}
#[test]
fn registration_advertises_metrics_port() {
assert_eq!(registration("vector:8090", "", 9327).metrics_port, 9327);
assert_eq!(registration("vector:8090", "", 0).metrics_port, 0);
}
#[tokio::test]
async fn stalled_registration_heartbeat_is_bounded() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let hold = tokio::spawn(async move {
let (_connection, _) = listener.accept().await.unwrap();
std::future::pending::<()>().await;
});
let channel = tonic::transport::Endpoint::from_shared(format!("http://{address}"))
.unwrap()
.connect_lazy();
let mut client = SeaweedClient::new(channel);
let result = tokio::time::timeout(CONNECT_TIMEOUT * 3, probe_registration(&mut client))
.await
.unwrap();
assert!(format!("{:#}", result.unwrap_err()).contains("registration heartbeat stalled"));
hold.abort();
}
/// Runs one pass against scripted masters and returns the dial order.
async fn pass(masters: &[&str], answer: fn(&str) -> Result<Option<String>>) -> Vec<String> {
let masters: Vec<String> = masters.iter().map(|m| m.to_string()).collect();
@@ -183,7 +247,8 @@ mod tests {
#[tokio::test]
async fn dial_to_a_black_holed_master_is_bounded() {
// TEST-NET-1 (RFC 5737) is reserved, so nothing answers the SYN.
let dial = stream_to("192.0.2.1:9333", "127.0.0.1:8080", None);
let registration = registration("127.0.0.1:8080", "", 0);
let dial = stream_to("192.0.2.1:9333", &registration, None, None);
let result = tokio::time::timeout(CONNECT_TIMEOUT * 3, dial).await;
assert!(matches!(result, Ok(Err(_))), "dial was not bounded");
}
@@ -197,7 +262,8 @@ mod tests {
let _ = tokio::io::copy(&mut conn, &mut tokio::io::sink()).await;
});
let master = format!("127.0.0.1:9333.{port}");
let open = stream_to(&master, "127.0.0.1:8080", None);
let registration = registration("127.0.0.1:8080", "", 0);
let open = stream_to(&master, &registration, None, None);
match tokio::time::timeout(CONNECT_TIMEOUT * 3, open).await {
Ok(Err(e)) => assert!(format!("{e:#}").contains("did not answer"), "{e:#}"),
other => panic!("open was not bounded: {other:?}"),
@@ -232,7 +298,8 @@ mod tests {
});
let master = format!("127.0.0.1:9333.{port}");
let tls = credentials();
let open = stream_to(&master, "127.0.0.1:8080", Some(&tls));
let registration = registration("127.0.0.1:8080", "", 0);
let open = stream_to(&master, &registration, Some(&tls), None);
match tokio::time::timeout(CONNECT_TIMEOUT * 3, open).await {
Ok(Err(e)) => assert!(format!("{e:#}").contains("did not complete"), "{e:#}"),
other => panic!("handshake was not bounded: {other:?}"),
@@ -258,16 +325,20 @@ mod tests {
let mut tls = credentials();
tls.ca_path = fifo.display().to_string();
let master = format!("127.0.0.1:9333.{port}");
let open = stream_to(&master, "127.0.0.1:8080", Some(&tls));
let registration = registration("127.0.0.1:8080", "", 0);
let open = stream_to(&master, &registration, Some(&tls), None);
let result = tokio::time::timeout(CONNECT_TIMEOUT * 3, open).await;
// A read+write open of a FIFO does not block, and it lets the
// blocked read finish so the runtime can shut down.
drop(
std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(&fifo),
);
use std::io::Write;
let mut cleanup = std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(&fifo)
.unwrap();
let replacement = dir.path().join("ca.replacement");
std::fs::write(&replacement, b"invalid CA").unwrap();
std::fs::rename(&replacement, &fifo).unwrap();
cleanup.write_all(b"invalid CA").unwrap();
drop(cleanup);
match result {
Ok(Err(e)) => assert!(format!("{e:#}").contains("did not complete"), "{e:#}"),
other => panic!("credential read was not bounded: {other:?}"),
+341
View File
@@ -0,0 +1,341 @@
use std::net::SocketAddr;
use anyhow::Result;
use axum::{Router, http::StatusCode, routing::get};
use prometheus::{
Encoder, Histogram, HistogramOpts, HistogramVec, IntCounter, IntCounterVec, IntGauge,
IntGaugeVec, Opts, Registry, TextEncoder,
};
use tonic_health::{ServingStatus, server::HealthReporter};
use crate::read_cache::ReadCache;
#[derive(Clone)]
pub struct Metrics {
pub registry: Registry,
pub connected: IntGauge,
pub requests: IntCounterVec,
pub request_seconds: HistogramVec,
pub in_flight: IntGaugeVec,
pub query_results: IntCounter,
pub cache_requests: IntCounterVec,
pub storage_bytes: IntCounter,
pub storage_read_seconds: Histogram,
cache_bytes: IntGauge,
cache_entries: IntGauge,
health: Option<HealthReporter>,
}
impl Metrics {
pub fn new() -> Result<Self> {
let registry = Registry::new();
let opts = |name: &str, help: &str| {
Opts::new(name, help)
.namespace("SeaweedFS")
.subsystem("vector")
};
let buckets = prometheus::exponential_buckets(0.0001, 2.0, 24)?;
let histogram =
|name: &str, help: &str| HistogramOpts::from(opts(name, help)).buckets(buckets.clone());
let connected = IntGauge::with_opts(opts(
"connected",
"Whether the engine is registered with the master.",
))?;
let requests = IntCounterVec::new(
opts(
"request_total",
"Vector engine requests by operation and gRPC status.",
),
&["type", "code"],
)?;
let request_seconds = HistogramVec::new(
histogram("request_seconds", "Vector engine request duration."),
&["type"],
)?;
let in_flight = IntGaugeVec::new(
opts("in_flight_requests", "Vector engine requests in progress."),
&["type"],
)?;
let query_results =
IntCounter::with_opts(opts("query_results_total", "Vectors returned by queries."))?;
let cache_requests = IntCounterVec::new(
opts("cache_requests_total", "Immutable reads by cache outcome."),
&["result"],
)?;
let storage_bytes = IntCounter::with_opts(opts(
"storage_read_bytes_total",
"Bytes fetched for immutable cache reads.",
))?;
let storage_read_seconds = Histogram::with_opts(histogram(
"storage_read_seconds",
"Immutable storage fetch duration.",
))?;
let cache_bytes =
IntGauge::with_opts(opts("cache_bytes", "Current cache weight in bytes."))?;
let cache_entries = IntGauge::with_opts(opts("cache_entries", "Current cache entries."))?;
for collector in [
Box::new(connected.clone()) as Box<dyn prometheus::core::Collector>,
Box::new(requests.clone()),
Box::new(request_seconds.clone()),
Box::new(in_flight.clone()),
Box::new(query_results.clone()),
Box::new(cache_requests.clone()),
Box::new(storage_bytes.clone()),
Box::new(storage_read_seconds.clone()),
Box::new(cache_bytes.clone()),
Box::new(cache_entries.clone()),
] {
registry.register(collector)?;
}
Ok(Self {
registry,
connected,
requests,
request_seconds,
in_flight,
query_results,
cache_requests,
storage_bytes,
storage_read_seconds,
cache_bytes,
cache_entries,
health: None,
})
}
pub fn with_health(mut self, health: HealthReporter) -> Self {
self.health = Some(health);
self
}
pub async fn set_connected(&self, connected: bool) {
self.connected.set(i64::from(connected));
if let Some(health) = self.health.clone() {
let status = if connected {
ServingStatus::Serving
} else {
ServingStatus::NotServing
};
health.set_service_status("", status).await;
health
.set_service_status("vector_pb.VectorService", status)
.await;
}
}
pub fn start(&self, operation: &str) -> InFlight {
let gauge = self.in_flight.with_label_values(&[operation]);
gauge.inc();
InFlight {
gauge,
requests: self.requests.clone(),
operation: operation.into(),
completed: false,
_timer: self
.request_seconds
.with_label_values(&[operation])
.start_timer(),
}
}
fn gather(&self, cache: Option<&ReadCache>) -> Result<String> {
if let Some(cache) = cache {
let (entries, bytes) = cache.size();
self.cache_entries.set(entries as i64);
self.cache_bytes.set(bytes as i64);
}
let mut buffer = Vec::new();
TextEncoder::new().encode(&self.registry.gather(), &mut buffer)?;
Ok(String::from_utf8(buffer)?)
}
}
pub struct InFlight {
gauge: IntGauge,
requests: IntCounterVec,
operation: String,
completed: bool,
_timer: prometheus::HistogramTimer,
}
impl InFlight {
pub fn complete(&mut self, error: Option<tonic::Code>) {
let code = error
.map(|c| format!("{c:?}"))
.unwrap_or_else(|| "OK".into());
self.requests
.with_label_values(&[&self.operation, &code])
.inc();
self.completed = true;
}
}
impl Drop for InFlight {
fn drop(&mut self) {
self.gauge.dec();
if !self.completed {
self.requests
.with_label_values(&[&self.operation, "Cancelled"])
.inc();
}
}
}
pub async fn serve(metrics: Metrics, address: SocketAddr, cache: Option<ReadCache>) -> Result<()> {
let listener = tokio::net::TcpListener::bind(address).await?;
serve_listener(metrics, listener, cache).await
}
pub async fn serve_listener(
metrics: Metrics,
listener: tokio::net::TcpListener,
cache: Option<ReadCache>,
) -> Result<()> {
let readiness = metrics.clone();
let app = Router::new()
.route("/health", get(|| async { StatusCode::OK }))
.route(
"/ready",
get(move || {
let metrics = readiness.clone();
async move {
if metrics.connected.get() == 1 {
StatusCode::OK
} else {
StatusCode::SERVICE_UNAVAILABLE
}
}
}),
)
.route(
"/metrics",
get(move || {
let metrics = metrics.clone();
let cache = cache.clone();
async move {
metrics
.gather(cache.as_ref())
.map(|body| ([("content-type", "text/plain; version=0.0.4")], body))
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)
}
}),
);
axum::serve(listener, app).await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn endpoints_follow_registration_and_publish_metrics() {
let metrics = Metrics::new().unwrap();
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
let task = tokio::spawn(serve_listener(metrics.clone(), listener, None));
let client = reqwest::Client::new();
assert_eq!(
client
.get(format!("{endpoint}/health"))
.send()
.await
.unwrap()
.status(),
200
);
assert_eq!(
client
.get(format!("{endpoint}/ready"))
.send()
.await
.unwrap()
.status(),
503
);
metrics.set_connected(true).await;
assert_eq!(
client
.get(format!("{endpoint}/ready"))
.send()
.await
.unwrap()
.status(),
200
);
let in_flight = metrics.start("QueryVectors");
let body = client
.get(format!("{endpoint}/metrics"))
.send()
.await
.unwrap()
.text()
.await
.unwrap();
assert!(body.contains("SeaweedFS_vector_connected 1"));
assert!(body.contains("SeaweedFS_vector_in_flight_requests{type=\"QueryVectors\"} 1"));
drop(in_flight);
assert_eq!(
metrics.in_flight.with_label_values(&["QueryVectors"]).get(),
0
);
metrics.set_connected(false).await;
assert_eq!(
client
.get(format!("{endpoint}/ready"))
.send()
.await
.unwrap()
.status(),
503
);
task.abort();
}
#[tokio::test]
async fn grpc_health_follows_registration() {
use tonic_health::pb::{HealthCheckRequest, health_client::HealthClient};
let (reporter, service) = tonic_health::server::health_reporter();
let metrics = Metrics::new().unwrap().with_health(reporter);
metrics.set_connected(false).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
let task = tokio::spawn(
tonic::transport::Server::builder()
.add_service(service)
.serve_with_incoming(tokio_stream::wrappers::TcpListenerStream::new(listener)),
);
let channel = tonic::transport::Endpoint::from_shared(endpoint)
.unwrap()
.connect()
.await
.unwrap();
let mut client = HealthClient::new(channel);
let request = || HealthCheckRequest {
service: "vector_pb.VectorService".into(),
};
assert_eq!(
client.check(request()).await.unwrap().into_inner().status,
2
);
metrics.set_connected(true).await;
assert_eq!(
client.check(request()).await.unwrap().into_inner().status,
1
);
metrics.set_connected(false).await;
assert_eq!(
client.check(request()).await.unwrap().into_inner().status,
2
);
task.abort();
}
}
impl std::fmt::Debug for Metrics {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Metrics")
.field("connected", &self.connected.get())
.finish()
}
}
@@ -0,0 +1,122 @@
use crate::metrics::Metrics;
use crate::pb::vector::{self as pb, vector_service_server::VectorService};
use crate::service::VectorServer;
use tonic::{Request, Response, Status};
pub struct ObservedService {
pub inner: VectorServer,
pub metrics: Metrics,
}
#[tonic::async_trait]
impl VectorService for ObservedService {
async fn create_index(
&self,
request: Request<pb::CreateIndexRequest>,
) -> Result<Response<pb::CreateIndexResponse>, Status> {
let mut in_flight = self.metrics.start("CreateIndex");
let result = self.inner.create_index(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn drop_index(
&self,
request: Request<pb::DropIndexRequest>,
) -> Result<Response<pb::DropIndexResponse>, Status> {
let mut in_flight = self.metrics.start("DropIndex");
let result = self.inner.drop_index(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn put_vectors(
&self,
request: Request<pb::PutVectorsRequest>,
) -> Result<Response<pb::PutVectorsResponse>, Status> {
let mut in_flight = self.metrics.start("PutVectors");
let result = self.inner.put_vectors(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn get_vectors(
&self,
request: Request<pb::GetVectorsRequest>,
) -> Result<Response<pb::GetVectorsResponse>, Status> {
let mut in_flight = self.metrics.start("GetVectors");
let result = self.inner.get_vectors(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn delete_vectors(
&self,
request: Request<pb::DeleteVectorsRequest>,
) -> Result<Response<pb::DeleteVectorsResponse>, Status> {
let mut in_flight = self.metrics.start("DeleteVectors");
let result = self.inner.delete_vectors(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn list_vectors(
&self,
request: Request<pb::ListVectorsRequest>,
) -> Result<Response<pb::ListVectorsResponse>, Status> {
let mut in_flight = self.metrics.start("ListVectors");
let result = self.inner.list_vectors(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
result
}
async fn query_vectors(
&self,
request: Request<pb::QueryVectorsRequest>,
) -> Result<Response<pb::QueryVectorsResponse>, Status> {
let mut in_flight = self.metrics.start("QueryVectors");
let result = self.inner.query_vectors(request).await;
in_flight.complete(result.as_ref().err().map(|e| e.code()));
if let Ok(response) = &result {
self.metrics
.query_results
.inc_by(response.get_ref().vectors.len() as u64);
}
result
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn failures_are_counted_and_release_in_flight_requests() {
let service = ObservedService {
inner: VectorServer::new(),
metrics: Metrics::new().unwrap(),
};
let response = service
.query_vectors(Request::new(pb::QueryVectorsRequest::default()))
.await;
let code = format!("{:?}", response.unwrap_err().code());
assert_eq!(
service
.metrics
.requests
.with_label_values(&["QueryVectors", &code])
.get(),
1
);
assert_eq!(
service
.metrics
.in_flight
.with_label_values(&["QueryVectors"])
.get(),
0
);
assert_eq!(
service
.metrics
.request_seconds
.with_label_values(&["QueryVectors"])
.get_sample_count(),
1
);
}
}
+189 -2
View File
@@ -9,6 +9,7 @@ use std::fmt;
use std::ops::Range;
use std::sync::Arc;
use crate::metrics::Metrics;
use bytes::Bytes;
use futures::StreamExt;
use futures::stream::BoxStream;
@@ -42,6 +43,7 @@ pub struct ReadCache {
/// Keeps entries apart from those of another [`Self::scoped`] view, for
/// stores that share a prefix but not their bytes.
scope: Arc<str>,
metrics: Option<Metrics>,
}
impl ReadCache {
@@ -54,14 +56,26 @@ impl ReadCache {
})
.build(),
scope: "".into(),
metrics: None,
}
}
pub fn with_metrics(mut self, metrics: Metrics) -> Self {
self.metrics = Some(metrics);
self
}
pub fn size(&self) -> (u64, u64) {
self.cache.run_pending_tasks();
(self.cache.entry_count(), self.cache.weighted_size())
}
/// The same cache, keyed apart under `scope`.
pub fn scoped(&self, scope: &str) -> Self {
Self {
cache: self.cache.clone(),
scope: scope.into(),
metrics: self.metrics.clone(),
}
}
}
@@ -81,6 +95,7 @@ impl WrappingObjectStore for ReadCache {
inner: original,
store: format!("{}|{store_prefix}", self.scope).into(),
cache: self.cache.clone(),
metrics: self.metrics.clone(),
})
}
}
@@ -136,6 +151,7 @@ struct CachedStore {
inner: Arc<dyn ObjectStore>,
store: Arc<str>,
cache: moka::sync::Cache<Key, Entry>,
metrics: Option<Metrics>,
}
impl CachedStore {
@@ -184,14 +200,47 @@ impl ObjectStore for CachedStore {
object: Some(object),
}) = self.cache.get(&key)
{
if let Some(metrics) = &self.metrics {
metrics.cache_requests.with_label_values(&["hit"]).inc();
}
return Ok(whole(bytes, &object));
}
if let Some(metrics) = &self.metrics {
metrics.cache_requests.with_label_values(&["miss"]).inc();
}
let _timer = self
.metrics
.as_ref()
.map(|m| m.storage_read_seconds.start_timer());
let result = self.inner.get_opts(location, options).await?;
if result.meta.size > MAX_WHOLE_OBJECT {
return Ok(result);
let meta = result.meta.clone();
let range = result.range.clone();
let attributes = result.attributes.clone();
let metrics = self.metrics.clone();
let timer = _timer;
let stream = result
.into_stream()
.map(move |chunk| {
let _timer = &timer;
if let (Some(metrics), Ok(bytes)) = (&metrics, &chunk) {
metrics.storage_bytes.inc_by(bytes.len() as u64);
}
chunk
})
.boxed();
return Ok(GetResult {
payload: GetResultPayload::Stream(stream),
meta,
range,
attributes,
});
}
let object = Arc::new((result.meta.clone(), result.attributes.clone()));
let bytes = owned(result.bytes().await?);
if let Some(metrics) = &self.metrics {
metrics.storage_bytes.inc_by(bytes.len() as u64);
}
self.cache.insert(
key,
Entry {
@@ -220,7 +269,21 @@ impl ObjectStore for CachedStore {
.filter(|(_, hit)| hit.is_none())
.map(|(range, _)| range.clone())
.collect();
if let Some(metrics) = &self.metrics {
metrics
.cache_requests
.with_label_values(&["miss"])
.inc_by(missing.len() as u64);
metrics
.cache_requests
.with_label_values(&["hit"])
.inc_by((ranges.len() - missing.len()) as u64);
}
if !missing.is_empty() {
let _timer = self
.metrics
.as_ref()
.map(|m| m.storage_read_seconds.start_timer());
let mut fetched = self.inner.get_ranges(location, &missing).await?.into_iter();
for (range, slot) in ranges.iter().zip(out.iter_mut()) {
if slot.is_some() {
@@ -230,6 +293,9 @@ impl ObjectStore for CachedStore {
break;
};
let bytes = owned(bytes);
if let Some(metrics) = &self.metrics {
metrics.storage_bytes.inc_by(bytes.len() as u64);
}
self.cache.insert(
self.key(location, Some(range.clone())),
Entry {
@@ -287,7 +353,7 @@ mod tests {
use super::*;
use crate::pb::vector as pb;
use crate::pb::vector::vector_service_server::VectorService;
use crate::service::VectorServer;
use crate::service::{VectorServer, schema_for, table_id};
use object_store::ObjectStoreExt;
use object_store::memory::InMemory;
use tonic::Request;
@@ -491,6 +557,16 @@ mod tests {
};
create(&server, &table, 4).await;
create(&server, &table, 4).await;
let mismatch = server
.create_index(Request::new(pb::CreateIndexRequest {
table: Some(table.clone()),
dimension: 3,
..Default::default()
}))
.await
.unwrap_err();
assert_eq!(mismatch.code(), tonic::Code::AlreadyExists);
put(
&server,
&table,
@@ -540,6 +616,117 @@ mod tests {
assert_eq!(query(&server, &table, vec![1.0, 0.0]).await, ["z"]);
}
/// A query loads the index once; the next one finds it in the session
/// every read shares.
#[tokio::test]
async fn queries_reuse_the_loaded_index() {
let bucket = format!("session-{}", uuid_like());
let location = format!("shared-memory://{bucket}/vector/x");
let server = VectorServer::new().with_schemes(&["shared-memory"]);
let table = pb::Table {
id: vec![bucket, "vector".into(), "x".into()],
location: location.clone(),
..Default::default()
};
create(&server, &table, 2).await;
put(
&server,
&table,
(0..300)
.map(|i| vector(&format!("k{i}"), vec![i as f32, 1.0]))
.collect(),
)
.await;
let mut dataset = lance::Dataset::open(&location).await.unwrap();
weed_lance_worker::vector_index::build(&mut dataset, "vector", "idx", 2, false)
.await
.unwrap();
assert_eq!(query(&server, &table, vec![5.0, 1.0]).await[0], "k5");
let session = server.reads.lock().unwrap().session.clone();
let loaded = session.index_cache_stats().await;
assert!(session.approx_num_items() > 0);
assert_eq!(query(&server, &table, vec![7.0, 1.0]).await[0], "k7");
let again = session.index_cache_stats().await;
assert!(server.handles.contains_key(&(location, false)));
assert!(again.hits > loaded.hits, "{loaded:?} then {again:?}");
assert_eq!(again.misses, loaded.misses, "{loaded:?} then {again:?}");
}
/// An index dropped and created again at the same location reaches the
/// version numbers of the old one; the shared session must not answer
/// for them with the old index.
#[tokio::test]
async fn a_recreated_table_does_not_see_the_old_index() {
recreate_after(false).await;
recreate_after(true).await;
}
/// `unmarked`: the first table is written without a table id, as indexes
/// created before ids were are.
async fn recreate_after(unmarked: bool) {
let bucket = format!("recreate-{}", uuid_like());
let location = format!("shared-memory://{bucket}/vector/x");
let server = VectorServer::new().with_schemes(&["shared-memory"]);
let table = pb::Table {
id: vec![bucket, "vector".into(), "x".into()],
location: location.clone(),
..Default::default()
};
let rows = |offset: usize| {
(0..300)
.map(|i| vector(&format!("k{}", i + offset), vec![(i + offset) as f32, 1.0]))
.collect::<Vec<_>>()
};
if unmarked {
let schema = schema_for(2, pb::DistanceMetric::L2, &[]);
let empty = arrow_array::RecordBatch::new_empty(schema.clone());
let reader = arrow_array::RecordBatchIterator::new(vec![Ok(empty)], schema);
lance::Dataset::write(reader, &location, None)
.await
.unwrap();
} else {
create(&server, &table, 2).await;
}
put(&server, &table, rows(0)).await;
let mut dataset = lance::Dataset::open(&location).await.unwrap();
assert_eq!(table_id(&dataset).is_empty(), unmarked);
weed_lance_worker::vector_index::build(&mut dataset, "vector", "idx", 2, false)
.await
.unwrap();
let indexed = dataset.version().version;
assert_eq!(query(&server, &table, vec![5.0, 1.0]).await[0], "k5");
let other = pb::Table {
id: vec![table.id[0].clone(), "vector".into(), "y".into()],
location: location.replace("/vector/x", "/vector/y"),
..Default::default()
};
create(&server, &other, 2).await;
put(&server, &other, rows(0)).await;
assert_eq!(query(&server, &other, vec![5.0, 1.0]).await[0], "k5");
let (store, path) = lance::io::ObjectStore::from_uri(&location).await.unwrap();
store.remove_dir_all(path).await.unwrap();
create(&server, &table, 2).await;
let mut offset = 1000;
while lance::Dataset::open(&location)
.await
.unwrap()
.version()
.version
< indexed
{
put(&server, &table, rows(offset)).await;
offset += 1000;
}
assert_eq!(query(&server, &table, vec![1005.0, 1.0]).await[0], "k1005");
server.handles.run_pending_tasks();
let session = server.reads.lock().unwrap().session.clone();
for (_, (_, dataset)) in server.handles.iter() {
assert!(Arc::ptr_eq(&dataset.session(), &session));
}
}
fn uuid_like() -> String {
format!(
"{:x}",
+233 -17
View File
@@ -9,16 +9,18 @@ use lance::dataset::ProjectionRequest;
use lance::dataset::WriteParams;
use lance::dataset::{MergeInsertBuilder, WhenMatched, WhenNotMatched};
use lance::io::WrappingObjectStore;
use lance::session::Session;
use lance::table::format::Fragment;
use prost_types::{ListValue, Struct, Value, value::Kind};
use std::collections::{BTreeMap, HashMap};
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use tonic::{Request, Response, Status};
use weed_lance_worker::catalog::TableDescription;
use weed_lance_worker::dataset;
use weed_lance_worker::direct_read::{DirectReads, Wrappers};
use weed_lance_worker::jobs::vector_index::is_vector_table;
use weed_vector_core::index;
use weed_vector_core::maintenance::SharedLeases;
use weed_vector_core::query::{self, Query};
use weed_vector_core::table::VectorTable;
@@ -55,7 +57,9 @@ const KEY_COLUMN: &str = "key";
const METADATA_COLUMN: &str = "metadata";
const VECTOR_COLUMN: &str = "vector";
const ROW_ID_COLUMN: &str = "_rowid";
fn schema_for(
const DEFAULT_INDEX_CACHE_BYTES: usize = 1 << 30;
const DEFAULT_METADATA_CACHE_BYTES: usize = 256 << 20;
pub(crate) fn schema_for(
dimension: usize,
metric: pb::DistanceMetric,
non_filterable: &[String],
@@ -95,6 +99,44 @@ fn schema_for(
]))
}
fn matches_index_schema(
dataset: &Dataset,
dimension: usize,
metric: pb::DistanceMetric,
non_filterable: &[String],
) -> bool {
let Ok(layout) = VectorTable::resolve(dataset, Some(VECTOR_COLUMN), Some(KEY_COLUMN)) else {
return false;
};
let schema = Schema::from(dataset.schema());
let Ok(vector) = schema.field_with_name(VECTOR_COLUMN) else {
return false;
};
let Ok(metadata) = schema.field_with_name(METADATA_COLUMN) else {
return false;
};
let stored_keys = metadata
.metadata()
.get("non_filterable_metadata_keys")
.map(|keys| serde_json::from_str::<Vec<String>>(keys))
.transpose();
let Ok(stored_keys) = stored_keys else {
return false;
};
let mut stored_keys = stored_keys.unwrap_or_default();
let mut requested_keys = non_filterable.to_vec();
stored_keys.sort();
requested_keys.sort();
layout.dimension == dimension
&& vector.metadata().get("distance_metric").map(String::as_str)
== Some(metric.as_str_name())
&& stored_keys == requested_keys
&& metadata
.metadata()
.get("ARROW:extension:name")
.is_some_and(|name| name == "lance.json")
}
fn batch_of(vectors: &[pb::Vector], dimension: usize, schema: Arc<Schema>) -> Result<RecordBatch> {
let mut seen = std::collections::HashSet::new();
for vector in vectors {
@@ -406,9 +448,85 @@ fn resolved(table: &pb::Table, schemes: &[&str]) -> Option<TableDescription> {
})
}
/// Marks the table an index was created as, in its schema metadata, which
/// every later version keeps.
const TABLE_ID_KEY: &str = "weed_vector_table_id";
/// The table a dataset was created as; empty for one created before tables
/// were marked.
pub(crate) fn table_id(dataset: &Dataset) -> &str {
dataset
.schema()
.metadata
.get(TABLE_ID_KEY)
.map_or("", String::as_str)
}
/// The session reads share, and the table each location held when this
/// session first read it. Lance keys some of its caches by location and
/// version alone, and an index dropped and created again at its location
/// numbers its versions from 1 again, so a session that sees a location hold
/// another table starts over.
pub(crate) struct Reads {
pub(crate) session: Arc<Session>,
generation: u64,
tables: HashMap<String, String>,
}
/// Opened tables kept for the next read of them.
const MAX_HANDLES: u64 = 1024;
fn handle_key(table: &pb::Table) -> (String, bool) {
(table.location.clone(), table.sse_s3)
}
/// Locations a session remembers before it starts over.
const MAX_READ_TABLES: usize = 100_000;
impl Reads {
fn new(index_bytes: usize, metadata_bytes: usize) -> Self {
Self {
session: Arc::new(Session::new(
index_bytes,
metadata_bytes,
Default::default(),
)),
generation: 0,
tables: HashMap::new(),
}
}
/// Whether a dataset of `table_id` at `location`, opened in this session,
/// may be read. Otherwise the session was replaced and the caller opens
/// the table again.
fn admit(&mut self, location: &str, table_id: &str, (index, metadata): (usize, usize)) -> bool {
match self.tables.get(location) {
Some(seen) if seen == table_id => return true,
None if self.tables.len() < MAX_READ_TABLES => {
self.tables
.insert(location.to_string(), table_id.to_string());
return true;
}
_ => {}
}
let generation = self.generation + 1;
*self = Self::new(index, metadata);
self.generation = generation;
false
}
}
pub struct VectorServer {
storage: Option<DirectReads>,
read_cache: Option<ReadCache>,
/// Lance's index and metadata caches, shared by every request that only
/// reads. Writes open in a session of their own.
pub(crate) reads: Mutex<Reads>,
/// A table opened by an earlier read, by location and encryption, with
/// the generation of the session it was opened in.
pub(crate) handles: moka::sync::Cache<(String, bool), (u64, Dataset)>,
lance_cache_bytes: (usize, usize),
leases: SharedLeases,
nprobes: Option<usize>,
/// URI schemes a table's location may have.
schemes: &'static [&'static str],
@@ -425,6 +543,13 @@ impl VectorServer {
Self {
storage: None,
read_cache: None,
reads: Mutex::new(Reads::new(
DEFAULT_INDEX_CACHE_BYTES,
DEFAULT_METADATA_CACHE_BYTES,
)),
lance_cache_bytes: (DEFAULT_INDEX_CACHE_BYTES, DEFAULT_METADATA_CACHE_BYTES),
handles: moka::sync::Cache::new(MAX_HANDLES),
leases: SharedLeases::default(),
nprobes: None,
schemes: &["s3"],
}
@@ -450,6 +575,13 @@ impl VectorServer {
self
}
/// Bytes of loaded indexes and of table metadata kept across reads.
pub fn with_lance_caches(mut self, index_bytes: usize, metadata_bytes: usize) -> Self {
self.reads = Mutex::new(Reads::new(index_bytes, metadata_bytes));
self.lance_cache_bytes = (index_bytes, metadata_bytes);
self
}
/// Serve repeated reads of write-once table files from `cache`.
pub fn with_read_cache(mut self, cache: ReadCache) -> Self {
self.read_cache = Some(cache);
@@ -491,9 +623,72 @@ impl VectorServer {
}
async fn open(&self, table: &pb::Table) -> Result<Dataset, Status> {
self.open_in(table, None).await
}
/// [`Self::open`] in the shared session, for a request that only reads.
async fn open_to_read(&self, table: &pb::Table) -> Result<Dataset, Status> {
loop {
let (session, generation) = {
let reads = self.reads.lock().unwrap();
(reads.session.clone(), reads.generation)
};
let dataset = match self.handle(table, generation).await? {
Some(dataset) => dataset,
None => {
let dataset = self.open_in(table, Some(session)).await?;
// Under the lock, so no handle of a replaced session
// lands after the replacement cleared them.
let reads = self.reads.lock().unwrap();
if table.version == 0 && reads.generation == generation {
self.handles
.insert(handle_key(table), (generation, dataset.clone()));
}
dataset
}
};
let mut reads = self.reads.lock().unwrap();
if reads.generation != generation {
continue;
}
if reads.admit(&table.location, table_id(&dataset), self.lance_cache_bytes) {
return Ok(dataset);
}
// The handles hold the replaced session and its caches.
self.handles.invalidate_all();
}
}
/// The table from the handle an earlier read of it left in this session,
/// moved to the version asked for: the latest costs a listing of its
/// versions, not a new store and manifest read.
async fn handle(&self, table: &pb::Table, generation: u64) -> Result<Option<Dataset>, Status> {
self.describe(table)?;
let key = handle_key(table);
let Some((at, mut dataset)) = self.handles.get(&key) else {
return Ok(None);
};
if at != generation {
return Ok(None);
}
let moved = match table.version {
0 => dataset.checkout_latest().await.map(|()| dataset),
version => dataset.checkout_version(version).await,
};
moved.map(Some).map_err(|err| {
self.handles.invalidate(&key);
open_status(err.into())
})
}
async fn open_in(
&self,
table: &pb::Table,
session: Option<Arc<Session>>,
) -> Result<Dataset, Status> {
let description = self.describe(table)?;
let wrapper = self.wrapper(table);
let mut dataset = dataset::open_at(&description, wrapper)
let mut dataset = dataset::open_at(&description, wrapper, session)
.await
.map_err(open_status)?;
if table.version != 0 {
@@ -526,6 +721,13 @@ impl VectorService for VectorServer {
// The gateway has already created the table through the catalog.
let description = self.describe(&table)?;
let schema = schema_for(dimension, metric, &req.non_filterable_metadata_keys);
let schema = Arc::new(Schema::new_with_metadata(
schema.fields().clone(),
HashMap::from([(
TABLE_ID_KEY.to_string(),
uuid::Uuid::new_v4().simple().to_string(),
)]),
));
let empty = RecordBatch::new_empty(schema.clone());
let reader = arrow_array::RecordBatchIterator::new(vec![Ok(empty)].into_iter(), schema);
let dataset = Dataset::write(
@@ -536,8 +738,28 @@ impl VectorService for VectorServer {
..Default::default()
}),
)
.await
.map_err(internal)?;
.await;
let dataset = match dataset {
Ok(dataset) => dataset,
Err(error) => match self.open(&table).await {
Ok(existing)
if matches_index_schema(
&existing,
dimension,
metric,
&req.non_filterable_metadata_keys,
) =>
{
existing
}
Ok(_) => {
return Err(Status::already_exists(
"index exists with a different schema",
));
}
Err(_) => return Err(internal(error)),
},
};
Ok(Response::new(pb::CreateIndexResponse {
version: dataset.version().version,
}))
@@ -622,12 +844,10 @@ impl VectorService for VectorServer {
let table = req
.table
.ok_or_else(|| Status::invalid_argument("missing table"))?;
let dataset = self.open(&table).await?;
let dataset = self.open_to_read(&table).await?;
// Lease the resolved version so maintenance cannot reclaim its files
// while this query runs against it.
let _lease = weed_vector_core::maintenance::try_pin(&dataset)
.await
.map_err(internal)?;
let _lease = self.leases.try_pin(&dataset).await.map_err(internal)?;
let resolved = VectorTable::resolve(&dataset, None, Some(KEY_COLUMN)).map_err(invalid)?;
let columns = projection(req.return_data, req.return_metadata);
let batches = query::get(&dataset, &resolved, &req.keys, Some(&columns))
@@ -672,7 +892,7 @@ impl VectorService for VectorServer {
"segment_index must be below segment_count, which is at most {MAX_SEGMENTS}"
)));
}
let dataset = self.open(&table).await?;
let dataset = self.open_to_read(&table).await?;
let offset: u64 = if req.next_token.is_empty() {
0
} else {
@@ -692,9 +912,7 @@ impl VectorService for VectorServer {
// Data and metadata come from a second read of this version, so
// lease it as get_vectors does.
let _lease = if columns.len() > 1 {
weed_vector_core::maintenance::try_pin(&dataset)
.await
.map_err(internal)?
self.leases.try_pin(&dataset).await.map_err(internal)?
} else {
None
};
@@ -739,12 +957,10 @@ impl VectorService for VectorServer {
let table = req
.table
.ok_or_else(|| Status::invalid_argument("missing table"))?;
let dataset = self.open(&table).await?;
let dataset = self.open_to_read(&table).await?;
// Lease the resolved version so maintenance cannot reclaim its files
// while this query runs against it.
let _lease = weed_vector_core::maintenance::try_pin(&dataset)
.await
.map_err(internal)?;
let _lease = self.leases.try_pin(&dataset).await.map_err(internal)?;
let resolved = VectorTable::resolve(&dataset, None, Some(KEY_COLUMN)).map_err(invalid)?;
if req.query_vector.len() != resolved.dimension {
return Err(Status::invalid_argument(format!(
@@ -0,0 +1,111 @@
use std::sync::Arc;
use bytes::Bytes;
use lance::io::WrappingObjectStore;
use object_store::{ObjectStoreExt, memory::InMemory, path::Path};
use weed_vector_server::{metrics::Metrics, read_cache::ReadCache};
#[tokio::test]
async fn immutable_reads_report_hits_misses_bytes_and_weight() {
let metrics = Metrics::new().unwrap();
let cache = ReadCache::new(1024).with_metrics(metrics.clone());
let store = cache.wrap("memory", Arc::new(InMemory::new()));
let path = Path::from("data/example.lance");
store
.put(&path, Bytes::from_static(b"abc").into())
.await
.unwrap();
for _ in 0..2 {
assert_eq!(
store.get(&path).await.unwrap().bytes().await.unwrap(),
"abc"
);
}
assert_eq!(metrics.cache_requests.with_label_values(&["miss"]).get(), 1);
assert_eq!(metrics.cache_requests.with_label_values(&["hit"]).get(), 1);
assert_eq!(metrics.storage_bytes.get(), 3);
assert_eq!(metrics.storage_read_seconds.get_sample_count(), 1);
let (entries, weight) = cache.size();
assert_eq!(entries, 1);
assert!(weight >= 3);
let metrics_endpoint = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}/metrics", metrics_endpoint.local_addr().unwrap());
let task = tokio::spawn(weed_vector_server::metrics::serve_listener(
metrics,
metrics_endpoint,
Some(cache),
));
let body = reqwest::get(endpoint).await.unwrap().text().await.unwrap();
assert!(body.contains("SeaweedFS_vector_cache_entries 1"));
assert!(body.contains("SeaweedFS_vector_cache_bytes "));
task.abort();
}
#[tokio::test]
async fn cancelled_requests_are_counted_once() {
let metrics = Metrics::new().unwrap();
let guard = metrics.start("QueryVectors");
let task = tokio::spawn(async move {
let _guard = guard;
std::future::pending::<()>().await;
});
task.abort();
let _ = task.await;
assert_eq!(
metrics.in_flight.with_label_values(&["QueryVectors"]).get(),
0
);
assert_eq!(
metrics
.requests
.with_label_values(&["QueryVectors", "Cancelled"])
.get(),
1
);
assert_eq!(
metrics
.request_seconds
.with_label_values(&["QueryVectors"])
.get_sample_count(),
1
);
{
let mut guard = metrics.start("QueryVectors");
guard.complete(None);
}
assert_eq!(
metrics
.requests
.with_label_values(&["QueryVectors", "OK"])
.get(),
1
);
assert_eq!(
metrics
.requests
.with_label_values(&["QueryVectors", "Cancelled"])
.get(),
1
);
}
#[tokio::test]
async fn large_immutable_reads_report_bytes_without_caching() {
let metrics = Metrics::new().unwrap();
let cache = ReadCache::new(16 * 1024 * 1024).with_metrics(metrics.clone());
let store = cache.wrap("memory", Arc::new(InMemory::new()));
let path = Path::from("data/large.lance");
let size = 5 * 1024 * 1024;
store
.put(&path, Bytes::from(vec![0; size]).into())
.await
.unwrap();
let result = store.get(&path).await.unwrap();
assert_eq!(metrics.storage_bytes.get(), 0);
assert_eq!(metrics.storage_read_seconds.get_sample_count(), 0);
assert_eq!(result.bytes().await.unwrap().len(), size);
assert_eq!(metrics.storage_bytes.get(), size as u64);
assert_eq!(metrics.cache_requests.with_label_values(&["miss"]).get(), 1);
assert_eq!(metrics.storage_read_seconds.get_sample_count(), 1);
assert_eq!(cache.size(), (0, 0));
}
+4
View File
@@ -230,6 +230,10 @@ message KeepConnectedRequest {
// A draining filer leaves the lock ring but stays a cluster member, so peers
// keep following its metadata until the stream closes.
bool leave_lock_ring = 8;
// Port of this node's Prometheus /metrics listener (-metricsPort), or 0 when
// it is not enabled. Advertised so the admin server can scrape it without
// exposing metrics on the client-facing service port.
uint32 metrics_port = 9;
}
message VolumeLocation {
+43 -1
View File
@@ -1175,7 +1175,7 @@ fn parse_security_config_with_env(path: &str, env: EnvLookup<'_>) -> SecurityCon
if let Some((key, value)) = trimmed.split_once('=') {
let key = key.trim();
let value = value.trim().trim_matches('"');
let value: &str = &toml_value_text(value);
match section {
Section::JwtSigningRead => match key {
"key" => cfg.jwt_read_signing_key = value.as_bytes().to_vec(),
@@ -1265,6 +1265,20 @@ fn parse_security_config_with_env(path: &str, env: EnvLookup<'_>) -> SecurityCon
cfg
}
/// Reads the right-hand side of `key = value` the way TOML (and so Go) does:
/// escapes decoded, literal strings kept as written, a trailing comment
/// dropped. Text TOML rejects keeps the plain quote trimming.
fn toml_value_text(raw: &str) -> String {
match format!("v = {raw}").parse::<toml::Table>() {
Ok(mut table) => match table.remove("v") {
Some(toml::Value::String(text)) => text,
Some(other) => other.to_string(),
None => String::new(),
},
Err(_) => raw.trim().trim_matches('"').to_string(),
}
}
fn resolve_security_config_path(path: &str, env: EnvLookup<'_>) -> Option<PathBuf> {
if !path.is_empty() {
return Some(PathBuf::from(path));
@@ -1868,6 +1882,34 @@ ui = true
assert!(cfg.access_ui);
}
#[test]
fn test_parse_security_config_reads_toml_strings() {
for (encoded, expected) in [
(r#""plain""#, "plain"),
(r#""key\n""#, "key\n"),
(r#""quote\"slash\\tab\t""#, "quote\"slash\\tab\t"),
(r#""key""#, "key"),
("'literal\\key'", "literal\\key"),
(r#""key" # comment"#, "key"),
(r#""YWJj==""#, "YWJj=="),
("unquoted", "unquoted"),
] {
let tmp = tempfile::NamedTempFile::new().unwrap();
std::fs::write(
tmp.path(),
format!(
"[jwt.signing]\nkey = {encoded}\n\n[jwt.signing.read]\nkey = {encoded}\n\
expires_after_seconds = 30 # seconds\n"
),
)
.unwrap();
let cfg = parse_security_config_with_env(tmp.path().to_str().unwrap(), &|_| None);
assert_eq!(cfg.jwt_signing_key, expected.as_bytes(), "{encoded}");
assert_eq!(cfg.jwt_read_signing_key, expected.as_bytes(), "{encoded}");
assert_eq!(cfg.jwt_read_signing_expires, 30, "{encoded}");
}
}
#[test]
fn test_parse_security_config_discovers_current_directory_default() {
let _guard = process_state_lock();
+86 -9
View File
@@ -5052,13 +5052,18 @@ impl VolumeServer for VolumeGrpcService {
// for and referenced by nothing. Go does not gate here either:
// its progress callback only runs during the transfer.
// Update volume info with remote file reference, unless the
// volume was compacted, replaced or removed mid-upload: the
// object would not match what now holds this id.
// volume was compacted, replaced, removed or written to
// mid-upload: the object would not match what now holds this
// id. One write lease spans the checks, the .vif save and the
// backend swap, so no write or commit lands in between.
let abort = {
let store = state.store.read().unwrap();
let abort = match store.find_volume(vid) {
let vol = store.find_volume(vid).map(|(_, v)| v);
let _sync_guard = vol.as_ref().map(|v| v.data_file_sync_lock().lock().unwrap());
let _lease = vol.as_ref().map(|v| v.instance().write_lock());
let abort = match &vol {
None => Some(Status::not_found(format!("volume {} not found", vid))),
Some((_, v))
Some(v)
if !v.is_instance(&instance)
|| v.super_block().compaction_revision != compaction_revision =>
{
@@ -5067,12 +5072,22 @@ impl VolumeServer for VolumeGrpcService {
vid
)))
}
Some(_) => None,
Some(v) => match v.dat_file_size() {
Ok(dat_size) if dat_size == size => None,
Ok(dat_size) => Some(Status::failed_precondition(format!(
"volume {} data file is {} bytes, the copy holds {}: written during the copy",
vid, dat_size, size
))),
Err(e) => Some(Status::internal(format!(
"stat volume {} data file: {}",
vid, e
))),
},
};
if abort.is_none()
&& let Some((_, vol)) = store.find_volume(vid)
&& let Some(vol) = &vol
{
vol.update_remote_files(|files| {
vol.update_remote_files_held(|files| {
files.push(volume_server_pb::RemoteFile {
backend_type: backend_type.clone(),
backend_id: backend_id.clone(),
@@ -5100,7 +5115,7 @@ impl VolumeServer for VolumeGrpcService {
// Switch the data backend from the local .dat to the remote
// object so reads keep working after upload (matches Go's
// LoadRemoteFile).
if let Err(e) = vol.load_remote_dat_file() {
if let Err(e) = vol.load_remote_dat_file_held() {
return Err(Status::internal(format!(
"volume {} failed to load remote file: {}",
vid, e
@@ -8068,7 +8083,7 @@ fn run_vacuum_compact(
.begin_compact_volume(vid, preallocate);
let result = match job {
Ok(Some(job)) => job.run(progress),
Ok(None) => Ok(()), // already compacting
Ok(None) => Err(crate::storage::volume::compacting_error(vid)),
Err(e) => Err(e),
};
@@ -9691,6 +9706,68 @@ mod tests {
let _ = s3.shutdown.send(());
}
// A write acknowledged while the upload runs is not in the object; the
// volume must keep serving the local .dat that holds it.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn test_tier_move_to_remote_aborts_when_written_mid_upload() {
let (service, tmp) = make_local_service_with_volume("", None);
seed_compactable_volume(&service);
let mut s3 = spawn_multipart_s3_server();
register_tier_backend("s3.tier_up_written", s3.endpoint.clone());
let stream = start_parked_tier_move(&service, &mut s3, "s3.tier_up_written", false).await;
let data = b"needle-3".to_vec();
let mut n = Needle {
id: NeedleId(3),
cookie: Cookie(3),
data_size: data.len() as u32,
data: data.clone(),
..Needle::default()
};
service
.state
.store
.read()
.unwrap()
.write_volume_needle(VolumeId(1), &mut n, true)
.expect("the write lands while the upload is parked");
s3.release.add_permits(1);
let terminal = tier_move_error(stream).await;
global_s3_tier_registry()
.write()
.unwrap()
.remove("s3.tier_up_written");
let err = terminal.expect("the tier move must fail once the volume was written");
assert_eq!(err.code(), tonic::Code::FailedPrecondition, "{err:?}");
assert_eq!(
s3.delete_count.load(std::sync::atomic::Ordering::SeqCst),
1,
"the stale object must be deleted"
);
{
let store = service.state.store.read().unwrap();
let (_, vol) = store.find_volume(VolumeId(1)).unwrap();
assert!(!vol.has_remote_file());
assert!(vol.volume_info().files.is_empty());
}
assert!(tmp.path().join("1.dat").exists());
let mut n = Needle {
id: NeedleId(3),
cookie: Cookie(3),
..Needle::default()
};
service
.state
.store
.read()
.unwrap()
.read_volume_needle(VolumeId(1), &mut n)
.expect("the acknowledged write must stay readable");
assert_eq!(n.data, data);
let _ = s3.shutdown.send(());
}
// Go fails here too: unmounting closes the descriptor its copy reads.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn test_tier_move_to_remote_fails_when_volume_is_unmounted_mid_upload() {
+6 -2
View File
@@ -376,9 +376,10 @@ fn process_batch(state: Arc<VolumeServerState>, batch: Vec<WriteRequest>) {
for (needle, fsync, response_tx, pending) in entries {
// A delete reaching a remounted volume cannot cancel a write
// queued for the old one.
if let (Ok(vol), Some(pending)) = (&vol, &pending)
&& !vol.tracks_pending_write(pending)
if let (Ok(vol), Some(tracked)) = (&vol, &pending)
&& !vol.tracks_pending_write(tracked)
{
drop(pending);
let _ = response_tx.send(Err(VolumeError::NotFound));
continue;
}
@@ -396,6 +397,9 @@ fn process_batch(state: Arc<VolumeServerState>, batch: Vec<WriteRequest>) {
})
.collect(),
};
// Untrack before replying, as Go does: a caller that sees its write
// finish must not still find it pending.
drop(pendings);
for (response_tx, result) in senders.into_iter().zip(results) {
// Send result back; ignore error if receiver dropped.
let _ = response_tx.send(result);
+3 -3
View File
@@ -1807,7 +1807,7 @@ impl Store {
{
match self.begin_compact_volume(vid, preallocate)? {
Some(job) => job.run(progress_fn),
None => Ok(()),
None => Err(crate::storage::volume::compacting_error(vid)),
}
}
@@ -3671,8 +3671,8 @@ mod tests {
assert!(store.cleanup_compact_volume(vid).is_err());
let (_, v) = store.find_volume(vid).unwrap();
assert!(v.relocate_index_to(other.path().to_str().unwrap()).is_err());
// Go parity: a commit that finds the volume compacting is a no-op.
store.commit_compact_volume(vid).unwrap();
assert!(store.commit_compact_volume(vid).is_err());
assert!(store.compact_volume(vid, 0, 0, |_| true).is_err());
let (_, v) = store.find_volume(vid).expect("still mounted");
assert!(v.is_compacting());
+58 -12
View File
@@ -1242,6 +1242,10 @@ const RANGE_CHECK_BLOCKS: u64 = 64;
const MIN_RANGE_CHECK_BLOCK_SIZE: u64 = 64 * 1024;
const MAX_RANGE_CHECK_BLOCK_SIZE: u64 = 1024 * 1024;
pub(crate) fn compacting_error(id: VolumeId) -> VolumeError {
VolumeError::Io(io::Error::other(format!("volume {} is compacting", id)))
}
/// Keeps a needle to 64 blocks, each 64 KiB to 1 MiB, as Go's
/// rangeCheckBlockSize.
pub(crate) fn range_check_block_size(data_size: u64) -> u64 {
@@ -1913,10 +1917,7 @@ impl Volume {
}
pub(crate) fn compacting_error(&self) -> VolumeError {
VolumeError::Io(io::Error::other(format!(
"volume {} is compacting",
self.id
)))
compacting_error(self.id)
}
// ---- File naming (matching Go) ----
@@ -2497,9 +2498,15 @@ impl Volume {
Ok(())
}
#[cfg(test)]
pub(crate) fn load_remote_dat_file(&self) -> Result<(), VolumeError> {
let _sync_guard = self.data_file_sync_lock.lock().unwrap();
let _guard = self.data_file_access_control.write_lock();
self.load_remote_dat_file_held()
}
/// `load_remote_dat_file` for callers already holding the write lease.
pub(crate) fn load_remote_dat_file_held(&self) -> Result<(), VolumeError> {
let mut io = self.io.write().unwrap();
self.load_remote_dat_file_io(&mut io)
}
@@ -5843,7 +5850,7 @@ impl Volume {
{
match self.begin_compact_by_index()? {
Some(job) => job.run(progress_fn),
None => Ok(()), // already compacting
None => Err(self.compacting_error()),
}
}
@@ -5948,7 +5955,7 @@ impl Volume {
return Err(VolumeError::NotFound);
}
let Some(_claim) = CompactionClaim::try_claim(&self.is_compacting) else {
return Ok(()); // already compacting, silently skip (matches Go)
return Err(self.compacting_error());
};
// The reload would read the remote object through the compacted .idx.
if self.has_remote_file() {
@@ -5963,6 +5970,8 @@ impl Volume {
}
fn do_commit_compact(&self) -> Result<(), VolumeError> {
// A duplicate commit must fail before it closes the live files.
self.check_compact_files()?;
if let Err(e) = self.makeup_diff() {
warn!("makeup_diff failed: {}", e);
// Match Go: clean up .cpd/.cpx on makeup_diff failure
@@ -6038,8 +6047,13 @@ impl Volume {
/// requires BOTH .cpd/.cpx to be present, so a stale or duplicate commit
/// returns an error without deleting the live .dat/.idx.
fn apply_compact_swap(&self) -> Result<(), VolumeError> {
// Normal commit: both temp files must be present, or a stale/duplicate
// commit could clobber the live .dat/.idx.
self.check_compact_files()?;
self.finish_compact_swap()
}
/// Both temp files of a normal commit must be present, or a stale or
/// duplicate commit could clobber the live .dat/.idx.
fn check_compact_files(&self) -> Result<(), VolumeError> {
if !Path::new(&self.file_name(".cpd")).exists()
|| !Path::new(&self.file_name(".cpx")).exists()
{
@@ -6048,7 +6062,7 @@ impl Volume {
"compact files (.cpd/.cpx) not found",
)));
}
self.finish_compact_swap()
Ok(())
}
/// Renames whichever compaction temp file is still present (.cpd->.dat,
@@ -6148,10 +6162,11 @@ impl Volume {
/// Clean up leftover compaction files (.cpd, .cpx).
pub fn cleanup_compact(&self) -> Result<(), VolumeError> {
// An in-flight copy is still writing .cpd/.cpx.
if self.is_compacting() {
// An in-flight copy is still writing .cpd/.cpx; claim the volume so
// none starts while the files are removed.
let Some(_claim) = CompactionClaim::try_claim(&self.is_compacting) else {
return Err(self.compacting_error());
}
};
// Refuse to unlink .cpd/.cpx while a .cpc marker exists: those temp files
// are the only inputs reconcile can roll forward to, so removing them
// mid-commit would strand a decided swap.
@@ -12929,4 +12944,35 @@ mod tests {
"live .dat unchanged"
);
}
/// A commit that finds no .cpd/.cpx (another vacuum already committed
/// them) must leave the volume loaded and writable.
#[test]
fn test_commit_compact_without_temp_files_keeps_volume_loaded() {
let tmp = TempDir::new().unwrap();
let dir = tmp.path().to_str().unwrap();
let v = make_test_volume(dir);
let needle = |i: u64| Needle {
id: NeedleId(i),
cookie: Cookie(i as u32),
data: format!("data-{}", i).into_bytes(),
data_size: format!("data-{}", i).len() as u32,
..Needle::default()
};
for i in 1..=3u64 {
v.write_needle(&mut needle(i), true, false).unwrap();
}
v.compact_by_index(0, 0, |_| true).unwrap();
v.commit_compact().unwrap();
assert!(v.commit_compact().is_err());
assert!(!Path::new(&v.file_name(".cpc")).exists());
v.write_needle(&mut needle(4), true, false).unwrap();
let mut n1 = Needle {
id: NeedleId(1),
..Needle::default()
};
v.read_needle(&mut n1).unwrap();
assert_eq!(n1.data, b"data-1");
}
}
+11 -4
View File
@@ -15,6 +15,7 @@ use lance::io::{
ObjectStore, ObjectStoreParams, ObjectStoreRegistry, StorageOptionsAccessor,
WrappingObjectStore,
};
use lance::session::Session;
use crate::catalog::{NamespaceClient, TableDescription};
use crate::direct_read::DirectReads;
@@ -74,7 +75,7 @@ pub fn unsigned_options() -> HashMap<String, String> {
pub async fn open(client: &NamespaceClient, id: &[String], storage: &Storage) -> Result<OpenTable> {
let description = client.describe_table(id).await?;
let wrapper = storage.wrapper(&description.location).await?;
let dataset = open_at(&description, wrapper).await?;
let dataset = open_at(&description, wrapper, None).await?;
Ok(OpenTable {
id: id.to_vec(),
location: description.location,
@@ -82,13 +83,19 @@ pub async fn open(client: &NamespaceClient, id: &[String], storage: &Storage) ->
})
}
/// Open a table the caller has already resolved, with the store wrapped.
/// Open a table the caller has already resolved, with the store wrapped, in
/// `session` when given, otherwise in a session of its own.
pub async fn open_at(
description: &TableDescription,
wrapper: Option<Arc<dyn WrappingObjectStore>>,
session: Option<Arc<Session>>,
) -> Result<Dataset> {
DatasetBuilder::from_uri(&description.location)
.with_store_params(store_params(wrapper))
let mut builder =
DatasetBuilder::from_uri(&description.location).with_store_params(store_params(wrapper));
if let Some(session) = session {
builder = builder.with_session(session);
}
builder
.load()
.await
.with_context(|| format!("open lance dataset at {}", description.location))
+59
View File
@@ -0,0 +1,59 @@
import io
import os
from pathlib import Path
import subprocess
import tarfile
import tempfile
import unittest
import zipfile
SOURCE = (Path(__file__).resolve().parents[2] / 'install.sh').read_text().rsplit('\nmain\n', 1)[0]
BASH = os.environ.get('BASH_BINARY', 'bash')
class VectorInstallerTest(unittest.TestCase):
def shell(self, code, **env):
result = subprocess.run([BASH, '-s'], input=(SOURCE + '\n' + code + '\n').encode('utf-8'),
env=dict(os.environ, **env), capture_output=True)
result.stdout = result.stdout.decode('utf-8', errors='replace')
result.stderr = result.stderr.decode('utf-8', errors='replace')
return result
def test_platforms_and_asset_names(self):
for system, arch, supported, suffix in [
('linux', 'amd64', True, 'tar.gz'), ('linux', 'arm64', True, 'tar.gz'),
('darwin', 'amd64', True, 'tar.gz'), ('darwin', 'arm64', True, 'tar.gz'),
('windows', 'amd64', True, 'zip'), ('windows', 'arm64', False, 'zip'),
('freebsd', 'amd64', False, 'tar.gz'),
]:
with self.subTest(system=system, arch=arch):
result = self.shell('vector_platform_supported "$SYSTEM" "$ARCH"; result=$?; '
'vector_asset_name "$SYSTEM" "$ARCH"; exit "$result"', SYSTEM=system, ARCH=arch)
if supported:
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.strip(), f'weed-vector_{system}_{arch}.{suffix}')
else:
self.assertNotEqual(result.returncode, 0)
def test_installs_archive_binary(self):
for system in ['linux', 'darwin', 'windows']:
with self.subTest(system=system), tempfile.TemporaryDirectory() as directory:
directory = Path(directory)
name = 'weed-vector.exe' if system == 'windows' else 'weed-vector'
contents = b'vector engine fixture'
archive = directory / ('vector.zip' if system == 'windows' else 'vector.tar.gz')
if system == 'windows':
with zipfile.ZipFile(archive, 'w') as output:
output.writestr(name, contents)
else:
with tarfile.open(archive, 'w:gz') as output:
info = tarfile.TarInfo(name)
info.size = len(contents)
info.mode = 0o755
output.addfile(info, io.BytesIO(contents))
dest = directory / 'bin'
result = self.shell('download() { cp "$ARCHIVE" "$2"; }; VERSION=test; '
'INSTALL_DIR="$DEST"; install_component vector "$SYSTEM" amd64',
ARCHIVE=archive.as_posix(), DEST=dest.as_posix(), SYSTEM=system)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertEqual((dest / name).read_bytes(), contents)
@@ -44,12 +44,22 @@ func vacuumVolumes(t *testing.T) {
require.NotEmpty(t, defaultConfig.MasterEndpoint, "vacuum needs a master endpoint; set MASTER_ENDPOINT")
endpoint := strings.TrimRight(defaultConfig.MasterEndpoint, "/") + "/vol/vacuum?garbageThreshold=0.001"
httpClient := &http.Client{Timeout: 30 * time.Second}
resp, err := httpClient.Get(endpoint)
require.NoError(t, err, "vacuum request to %s", endpoint)
defer resp.Body.Close()
_, err = io.Copy(io.Discard, resp.Body)
require.NoError(t, err, "reading the vacuum response")
require.Equal(t, http.StatusOK, resp.StatusCode, "vacuum request to %s", endpoint)
// The master runs one vacuum at a time and answers 409 to a request it
// dropped because its automatic vacuum was running. Ask until ours runs.
deadline := time.Now().Add(time.Minute)
for {
resp, err := httpClient.Get(endpoint)
require.NoError(t, err, "vacuum request to %s", endpoint)
_, err = io.Copy(io.Discard, resp.Body)
resp.Body.Close()
require.NoError(t, err, "reading the vacuum response")
if resp.StatusCode == http.StatusConflict && time.Now().Before(deadline) {
time.Sleep(500 * time.Millisecond)
continue
}
require.Equal(t, http.StatusOK, resp.StatusCode, "vacuum request to %s", endpoint)
return
}
}
func requireVersionBody(t *testing.T, client *s3.Client, bucketName, objectKey, versionId string, want []byte, msg string) {
+8 -13
View File
@@ -1,10 +1,7 @@
WEED := ./weed
VEC := ./weed-vector
CLI := ./weed-vector-cli
WORKER_MANIFEST := ../../rust/sw-vector-distributed/Cargo.toml
ifneq ($(wildcard $(WORKER_MANIFEST)),)
WORKER := ./weed-vector-worker
endif
ENGINE_MANIFEST := ../../rust/sw-vector-distributed/Cargo.toml
VOLUME := ./weed-volume
LANCE_WORKER := ./weed-worker
CARGO_PROFILE ?= release
@@ -21,7 +18,11 @@ $(WEED): FORCE
go build -o $(WEED) ../../weed
$(VEC): FORCE
ifneq ($(wildcard $(ENGINE_MANIFEST)),)
cargo build --locked --profile $(CARGO_PROFILE) --manifest-path ../../rust/Cargo.toml -p sw-vector-distributed --bin weed-vector
else
cd ../../seaweed-vector && cargo build --locked --profile $(CARGO_PROFILE) -p weed-vector-server
endif
cp $(CARGO_OUTPUT)/weed-vector $(VEC)
$(CLI): FORCE
@@ -32,19 +33,13 @@ $(LANCE_WORKER): FORCE
cargo build --locked --profile $(CARGO_PROFILE) --manifest-path ../../seaweed-worker/Cargo.toml -p weed-lance-worker --bin weed-worker
cp $(CARGO_OUTPUT)/weed-worker $(LANCE_WORKER)
ifneq ($(WORKER),)
$(WORKER): $(VEC) FORCE
cargo build --locked --profile $(CARGO_PROFILE) --manifest-path ../../rust/Cargo.toml -p sw-vector-distributed --bin weed-vector
cp $(CARGO_OUTPUT)/weed-vector $(WORKER)
endif
$(VOLUME): $(VEC) $(WORKER) FORCE
$(VOLUME): $(VEC) FORCE
cargo build --locked --profile $(CARGO_PROFILE) --manifest-path ../../seaweed-volume/Cargo.toml --bin weed-volume
cp $(CARGO_OUTPUT)/weed-volume $(VOLUME)
FORCE:
build: $(WEED) $(VEC) $(WORKER) $(VOLUME)
build: $(WEED) $(VEC) $(VOLUME)
ifneq ($(VECTOR_WORKER),)
build: $(CLI) $(LANCE_WORKER)
endif
@@ -56,4 +51,4 @@ perf: build $(CLI)
./run_perf.sh
clean:
rm -f $(WEED) $(VEC) $(CLI) $(LANCE_WORKER) ./weed-vector-worker $(VOLUME)
rm -f $(WEED) $(VEC) $(CLI) $(LANCE_WORKER) $(VOLUME)
+30 -21
View File
@@ -123,7 +123,7 @@ def write_tls(directory):
"-out", "outsider.pem", "-days", "2", "-extfile", "ext.cnf")
cert, key, ca = (f'{key} = {json.dumps(str(directory / name), ensure_ascii=False)}\n' for key, name in (("cert", "cert.pem"), ("key", "key.pem"), ("ca", "ca.pem")))
pem = cert + key + ca
grpc = "".join(f"[grpc.{component}]\n{cert}{key}\n" for component in ("master", "volume", "filer", "s3"))
grpc = "".join(f"[grpc.{component}]\n{cert}{key}\n" for component in ("master", "volume", "filer", "s3", "admin", "worker"))
# Clients (the gateway, the engine) accept only servers named localhost;
# the engine serves only clients named localhost.
grpc += f"[grpc.client]\n{cert}{key}allowed_server_commonNames = \"localhost\"\n"
@@ -242,23 +242,22 @@ def run_cluster(param, tmp_path_factory, identities):
# VECTOR_WORKER=1 adds the admin server and the Lance maintenance worker,
# for the tests that build vector indexes through a plugin job.
lance_worker = binary("LANCE_WORKER_BINARY", "weed-worker") if os.environ.get("VECTOR_WORKER") else None
worker = binary("VECTOR_WORKER_BINARY", "weed-vector-worker") if mode == "distributed" else None
volume_binary = binary("RUST_VOLUME_SERVER_BINARY", "weed-volume") if volume == "rust" else weed
processes = {}
with contextlib.ExitStack() as stack:
with contextlib.ExitStack() as reservations:
ports = []
for _ in range(14):
for _ in range(15):
sock = reservations.enter_context(socket.socket())
sock.bind(("127.0.0.1", 0))
ports.append(sock.getsockname()[1])
(master, master_grpc, vol, vol_grpc, filer, filer_grpc, s3, s3_grpc, lance, vector, worker1, worker2,
admin, admin_grpc) = ports
(master, master_grpc, vol, vol_grpc, filer, filer_grpc, s3, s3_grpc, lance, vector, engine1, engine2,
admin, admin_grpc, vector_metrics) = ports
master_addr = f"127.0.0.1:{master}.{master_grpc}"
filer_addr = f"127.0.0.1:{filer}.{filer_grpc}"
# VECTOR_READ_JWT=1 makes volume reads need a JWT; every weed process
# and the workers sign with the key.
# and the engines sign with the key.
read_jwt_key = os.environ.get("VECTOR_READ_JWT") and uuid.uuid4().hex
# The cluster's filer signing key: the gateway's catalog authenticates
# the worker and the CLI by the token they derive from it.
@@ -304,10 +303,11 @@ def run_cluster(param, tmp_path_factory, identities):
# VECTOR_ENCRYPT_VOLUMES=1 turns on the filer's volume encryption, which
# the gateway, the engine and the worker all follow: every chunk they
# store is encrypted, and direct reads must undo it.
filer_group = os.environ.get("VECTOR_FILER_GROUP", "")
encrypt = ["-encryptVolumeData"] if os.environ.get("VECTOR_ENCRYPT_VOLUMES") else []
start("filer", [weed, "filer", f"-master={master_addr}", "-ip=127.0.0.1",
"-ip.bind=127.0.0.1", f"-port={filer}", f"-port.grpc={filer_grpc}",
f"-defaultStoreDir={directory / 'filer'}", *encrypt])
f"-defaultStoreDir={directory / 'filer'}", f"-filerGroup={filer_group}", *encrypt])
wait_for(lambda: listening(filer_grpc), processes, "filer not ready")
endpoint = f"http://127.0.0.1:{s3}"
# weed s3 turns AWS_ACCESS_KEY_ID into an admin identity.
@@ -325,8 +325,12 @@ def run_cluster(param, tmp_path_factory, identities):
# The engine keeps tables on the filers and volume servers itself; it
# has no S3 credentials to be given, and encrypts its chunks for the
# volume servers as the gateway does.
start("vector", [engine, "--masters", master_addr, "--listen", f"127.0.0.1:{vector}"])
start("vector", [engine, "--masters", master_addr, "--listen", f"127.0.0.1:{vector}", "--metrics-port", str(vector_metrics), "--filer-group", filer_group])
wait_for(lambda: listening(vector), processes, "vector engine not ready")
def vector_ready():
with urllib.request.urlopen(f"http://127.0.0.1:{vector_metrics}/ready", timeout=1) as response:
return response.status == 200
wait_for(vector_ready, processes, "vector engine did not register with master")
if os.environ.get("VECTOR_TLS"):
wait_for(lambda: listening(s3_grpc), processes, "s3 gRPC not ready")
for name, port in (("master", master_grpc), ("volume", vol_grpc), ("filer", filer_grpc),
@@ -335,41 +339,46 @@ def run_cluster(param, tmp_path_factory, identities):
probe_grpc_tls("engine", vector, directory, outsider=True)
wait_for(
lambda: f"Cluster: vector node 127.0.0.1:{vector} added to group ''"
lambda: f"Cluster: vector node 127.0.0.1:{vector} added to group '{filer_group}'"
in (directory / "master.log").read_text(),
processes,
"vector engine never registered with the master",
)
if worker:
for i, port in enumerate((worker1, worker2)):
start(f"worker-{i}", [worker, "--masters", master_addr, "--listen", f"127.0.0.1:{port}"])
wait_for(lambda: listening(port), processes, "vector worker not ready")
# Distributed mode needs the Enterprise weed-vector, whose engines all
# join the masters' placement controller.
if mode == "distributed":
for i, port in enumerate((engine1, engine2), 1):
start(f"vector-{i}", [engine, "--masters", master_addr, "--listen", f"127.0.0.1:{port}",
"--filer-group", filer_group, "--state-dir", str(directory / f"vector-{i}"),
"--capacity-bytes", str(1 << 30), "--drain-timeout", "5s"])
wait_for(lambda: listening(port), processes, f"vector-{i} not ready")
wait_for(
lambda: all(
f"S3 Vectors: discovered search worker 127.0.0.1:{port}"
in (directory / "s3.log").read_text()
for port in (worker1, worker2)
),
lambda: all(f"joined at 127.0.0.1:{port}" in (directory / "master.log").read_text()
for port in (engine1, engine2)),
processes,
"gateway did not discover both search workers",
"the engines never joined the placement controller",
)
admin_url = None
if lance_worker:
(directory / "admin").mkdir()
admin_url = f"http://127.0.0.1:{admin}"
start("admin", [weed, "admin", "-ip=127.0.0.1", f"-port={admin}", f"-port.grpc={admin_grpc}",
start("admin", [weed, "admin", f"-filerGroup={filer_group}", "-ip=127.0.0.1", f"-port={admin}", f"-port.grpc={admin_grpc}",
"-port.legacy=0", f"-master={master_addr}", f"-dataDir={directory / 'admin'}"])
wait_for(lambda: http_ready(f"{admin_url}/api/plugin/job-types"), processes, "admin not ready")
tls_args = []
if os.environ.get("VECTOR_TLS"):
tls_args = ["--tls-ca", str(directory / "ca.pem"), "--tls-cert", str(directory / "cert.pem"), "--tls-key", str(directory / "key.pem"), "--tls-server-name", "localhost"]
start("lance-worker", [lance_worker, "--admin", f"127.0.0.1:{admin}.{admin_grpc}",
"--namespace", f"http://127.0.0.1:{lance}", "--id", "vectors-test-worker",
"--masters", master_addr])
"--masters", master_addr, "--filer-group", filer_group, *tls_args])
wait_for(lambda: "lance_vector_index" in
[jt["job_type"] for jt in json.load(urllib.request.urlopen(f"{admin_url}/api/plugin/job-types", timeout=2))],
processes, "the Lance worker never registered with admin")
yield {"endpoint": endpoint, "mode": mode, "processes": processes, "directory": directory, "filer": filer,
"masters": master_addr, "filer_signing_key": filer_signing_key,
"vector_metrics": f"http://127.0.0.1:{vector_metrics}",
"catalog": f"http://127.0.0.1:{lance}", "admin": admin_url, "env": process_env()}
+18
View File
@@ -39,6 +39,24 @@
"actions": [
"Write"
]
},
{
"name": "policy-reader",
"credentials": [
{
"accessKey": "policy-reader",
"secretKey": "policy-reader-secret"
}
],
"policyNames": [
"vector-policy"
]
}
],
"policies": [
{
"name": "vector-policy",
"content": "{\"Version\": \"2012-10-17\", \"Statement\": [{\"Effect\": \"Allow\", \"Action\": [\"s3vectors:GetVectors\", \"s3vectors:QueryVectors\"], \"Resource\": \"arn:aws:s3vectors:*:*:bucket/vector-iam/index/docs\"}, {\"Effect\": \"Allow\", \"Action\": \"s3vectors:CreateVectorBucket\", \"Resource\": \"*\"}, {\"Effect\": \"Allow\", \"Action\": \"s3vectors:TagResource\", \"Resource\": \"arn:aws:s3vectors:*:*:bucket/vector-iam-*\"}, {\"Effect\": \"Deny\", \"Action\": \"s3vectors:QueryVectors\", \"Resource\": \"*\", \"Condition\": {\"StringEquals\": {\"aws:ResourceTag/classification\": \"private\"}}}, {\"Effect\": \"Deny\", \"Action\": \"s3vectors:QueryVectors\", \"Resource\": \"*\", \"Condition\": {\"StringEquals\": {\"s3vectors:VectorBucketTag/environment\": \"production\"}}}, {\"Effect\": \"Deny\", \"Action\": \"s3vectors:CreateVectorBucket\", \"Resource\": \"*\", \"Condition\": {\"StringEquals\": {\"aws:RequestTag/classification\": \"forbidden\"}}}, {\"Effect\": \"Deny\", \"Action\": \"s3vectors:CreateVectorBucket\", \"Resource\": \"*\", \"Condition\": {\"ForAnyValue:StringEquals\": {\"aws:TagKeys\": \"forbidden-key\"}}}]}"
}
]
}
+53 -1
View File
@@ -1,3 +1,4 @@
import urllib.request
"""Run the S3 Vectors API against Go/Rust volumes and single/distributed queries."""
import contextlib
@@ -150,7 +151,7 @@ def run_cli(cluster, *args):
from pathlib import Path
cli = Path(os.environ.get("VECTOR_CLI_BINARY", Path(__file__).resolve().parent / "weed-vector-cli")).resolve()
command = [str(cli), "--namespace", cluster["catalog"], "--masters", cluster["masters"],
"--filer-signing-key", cluster["filer_signing_key"], *args]
"--filer-signing-key", cluster["filer_signing_key"], "--filer-group", os.environ.get("VECTOR_FILER_GROUP", ""), *args]
return json.loads(subprocess.run(command, check=True, capture_output=True, text=True,
cwd=cluster["directory"], env=cluster["env"]).stdout)
@@ -711,6 +712,43 @@ def test_vector_index_job(vectors, cluster):
assert found >= 180, f"recall after retrain {found}/200"
def test_attached_iam_vector_policies(vectors, cluster):
client = boto3.client("s3vectors", endpoint_url=cluster["endpoint"], region_name="us-east-1",
aws_access_key_id="policy-reader", aws_secret_access_key="policy-reader-secret",
config=vectors.meta.config)
bucket = "vector-iam"
vectors.create_vector_bucket(vectorBucketName=bucket)
target = {"vectorBucketName": bucket, "indexName": "docs"}
vectors.create_index(**target, dataType="float32", dimension=3, distanceMetric="cosine")
index_arn = vectors.get_index(**target)["index"]["indexArn"]
bucket_arn = vectors.get_vector_bucket(vectorBucketName=bucket)["vectorBucket"]["vectorBucketArn"]
query = {**target, "queryVector": {"float32": [1, 0, 0]}, "topK": 1}
try:
assert client.query_vectors(**query)["vectors"] == []
assert client.get_vectors(**target, keys=["missing"])["vectors"] == []
expect_error("AccessDeniedException", client.put_vectors, **target,
vectors=[{"key": "blocked", "data": {"float32": [1, 0, 0]}}])
vectors.tag_resource(resourceArn=index_arn, tags={"classification": "private"})
expect_error("AccessDeniedException", client.query_vectors, **query)
vectors.untag_resource(resourceArn=index_arn, tagKeys=["classification"])
assert client.query_vectors(**query)["vectors"] == []
vectors.tag_resource(resourceArn=bucket_arn, tags={"environment": "production"})
expect_error("AccessDeniedException", client.query_vectors, **query)
vectors.untag_resource(resourceArn=bucket_arn, tagKeys=["environment"])
assert client.query_vectors(**query)["vectors"] == []
vectors.put_vector_bucket_policy(vectorBucketName=bucket, policy=json.dumps({"Version": "2012-10-17", "Statement": [
{"Effect": "Deny", "Principal": "*", "Action": "s3vectors:QueryVectors", "Resource": index_arn}]}))
expect_error("AccessDeniedException", client.query_vectors, **query)
expect_error("AccessDeniedException", client.create_vector_bucket,
vectorBucketName="vector-iam-rejected", tags={"classification": "forbidden"})
expect_error("AccessDeniedException", client.create_vector_bucket,
vectorBucketName="vector-iam-key-denied", tags={"forbidden-key": "value"})
finally:
vectors.delete_vector_bucket_policy(vectorBucketName=bucket)
vectors.delete_index(**target)
vectors.delete_vector_bucket(vectorBucketName=bucket)
def test_invalid_credentials(cluster):
client = boto3.client("s3vectors", endpoint_url=cluster["endpoint"], aws_access_key_id="unknown",
aws_secret_access_key="unknown", region_name="us-east-1")
@@ -719,6 +757,18 @@ def test_invalid_credentials(cluster):
assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 403
def test_vector_engine_observability(vectors, index, cluster):
vectors.query_vectors(**index, queryVector={"float32": [1, 0, 0, 0]}, topK=1)
with urllib.request.urlopen(cluster["vector_metrics"] + "/ready", timeout=5) as response:
assert response.status == 200
with urllib.request.urlopen(cluster["vector_metrics"] + "/metrics", timeout=5) as response:
metrics = response.read().decode()
assert "SeaweedFS_vector_connected 1" in metrics
assert any(line.startswith("SeaweedFS_vector_request_total{") and 'type="QueryVectors"' in line
and 'code="OK"' in line for line in metrics.splitlines())
assert "SeaweedFS_vector_request_seconds_bucket" in metrics
def test_query_without_writer(vectors, index, cluster):
from conftest import stop
@@ -753,3 +803,5 @@ def test_engine_without_credentials(open_cluster):
hits = client.query_vectors(**index, queryVector={"float32": [1, 0]}, topK=1)["vectors"]
assert [h["key"] for h in hits] == ["a"]
+15 -5
View File
@@ -983,11 +983,21 @@ func TestTusAbortedPatchKeepsStoredChunks(t *testing.T) {
require.Equal(t, http.StatusNoContent, patchResp.StatusCode)
// A vacuum reclaims whatever the filer deleted, so the file survives this
// only if the chunks the session kept are still stored.
vacuumResp, err := client.Get(fmt.Sprintf("http://127.0.0.1:%s/vol/vacuum?garbageThreshold=0.001", testMasterPort))
require.NoError(t, err)
vacuumResp.Body.Close()
require.Equal(t, http.StatusOK, vacuumResp.StatusCode)
// only if the chunks the session kept are still stored. The master answers
// 409 to a request it dropped because its automatic vacuum was running, so
// ask until ours runs.
vacuumDeadline := time.Now().Add(time.Minute)
for {
vacuumResp, err := client.Get(fmt.Sprintf("http://127.0.0.1:%s/vol/vacuum?garbageThreshold=0.001", testMasterPort))
require.NoError(t, err)
vacuumResp.Body.Close()
if vacuumResp.StatusCode == http.StatusConflict && time.Now().Before(vacuumDeadline) {
time.Sleep(500 * time.Millisecond)
continue
}
require.Equal(t, http.StatusOK, vacuumResp.StatusCode)
break
}
getResp, err := client.Get(cluster.FilerURL() + targetPath)
require.NoError(t, err)
@@ -0,0 +1,137 @@
package volume_server_grpc_test
import (
"context"
"net/http"
"strings"
"testing"
"time"
"github.com/seaweedfs/seaweedfs/test/volume_server/framework"
"github.com/seaweedfs/seaweedfs/test/volume_server/matrix"
"github.com/seaweedfs/seaweedfs/weed/operation/volume_move"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/volume_server_pb"
"github.com/seaweedfs/seaweedfs/weed/storage/needle"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
func moverServerAddress(cluster framework.MultiCluster, index int) pb.ServerAddress {
return pb.ServerAddress(cluster.VolumeAdminAddress(index) + "." + strings.Split(cluster.VolumeGRPCAddress(index), ":")[1])
}
func reloadVolume(t *testing.T, client volume_server_pb.VolumeServerClient, volumeID uint32) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := client.VolumeUnmount(ctx, &volume_server_pb.VolumeUnmountRequest{VolumeId: volumeID}); err != nil {
t.Fatalf("unmount volume %d: %v", volumeID, err)
}
if _, err := client.VolumeMount(ctx, &volume_server_pb.VolumeMountRequest{VolumeId: volumeID}); err != nil {
t.Fatalf("mount volume %d: %v", volumeID, err)
}
}
func uploadStatus(t *testing.T, httpClient *http.Client, volumeURL string, volumeID uint32, key uint64, data string) int {
t.Helper()
resp := framework.UploadBytes(t, httpClient, volumeURL, framework.NewFileID(volumeID, key, 0x5EED0000+uint32(key)), []byte(data))
framework.ReadAllAndClose(t, resp)
return resp.StatusCode
}
func TestLiveMoveFreezeSurvivesSourceReload(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cluster := framework.StartMultiVolumeClusterAuto(t, matrix.P1(), 2)
sourceConn, sourceClient := framework.DialVolumeServer(t, cluster.VolumeGRPCAddress(0))
defer sourceConn.Close()
targetConn, targetClient := framework.DialVolumeServer(t, cluster.VolumeGRPCAddress(1))
defer targetConn.Close()
const volumeID = uint32(977)
framework.AllocateVolume(t, sourceClient, volumeID, "")
httpClient := framework.NewHTTPClient()
if code := uploadStatus(t, httpClient, cluster.VolumeAdminURL(0), volumeID, 1, "before-move"); code != http.StatusCreated {
t.Fatalf("initial upload: status %d", code)
}
var reloadedSourceWrite, targetWrite int
mover := volume_move.NewMover(grpc.WithTransportCredentials(insecure.NewCredentials()))
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
err := mover.LiveMoveVolume(ctx, needle.VolumeId(volumeID), moverServerAddress(cluster, 0), moverServerAddress(cluster, 1), volume_move.VolumeMoveOptions{
IdleTimeout: time.Second,
Progress: func(percent float64, stage string) {
switch percent {
case 20:
// frozen, not yet copied: a reload must not reopen the source
reloadVolume(t, sourceClient, volumeID)
reloadedSourceWrite = uploadStatus(t, httpClient, cluster.VolumeAdminURL(0), volumeID, 2, "lost-after-move")
case 70:
// copied, not yet tailed: the copy must not inherit the freeze
targetWrite = uploadStatus(t, httpClient, cluster.VolumeAdminURL(1), volumeID, 3, "on-target")
}
},
})
if err != nil {
t.Fatalf("LiveMoveVolume: %v", err)
}
if reloadedSourceWrite != http.StatusConflict {
t.Fatalf("reloaded source write: status %d, want %d", reloadedSourceWrite, http.StatusConflict)
}
if targetWrite != http.StatusCreated {
t.Fatalf("copied target refused a write before the tail: status %d", targetWrite)
}
// The writable mark on the target is persisted too.
reloadVolume(t, targetClient, volumeID)
if code := uploadStatus(t, httpClient, cluster.VolumeAdminURL(1), volumeID, 4, "after-reload"); code != http.StatusCreated {
t.Fatalf("moved volume refused a write after a reload: status %d", code)
}
for key, want := range map[uint64]string{1: "before-move", 3: "on-target", 4: "after-reload"} {
resp := framework.ReadBytes(t, httpClient, cluster.VolumeAdminURL(1), framework.NewFileID(volumeID, key, 0x5EED0000+uint32(key)))
body := framework.ReadAllAndClose(t, resp)
if resp.StatusCode != http.StatusOK || string(body) != want {
t.Fatalf("read key %d from target: status %d body %q, want %q", key, resp.StatusCode, body, want)
}
}
}
func TestLiveMoveAbortClearsPersistedFreeze(t *testing.T) {
if testing.Short() {
t.Skip("skipping integration test in short mode")
}
cluster := framework.StartMultiVolumeClusterAuto(t, matrix.P1(), 2)
sourceConn, sourceClient := framework.DialVolumeServer(t, cluster.VolumeGRPCAddress(0))
defer sourceConn.Close()
const volumeID = uint32(978)
framework.AllocateVolume(t, sourceClient, volumeID, "")
httpClient := framework.NewHTTPClient()
if code := uploadStatus(t, httpClient, cluster.VolumeAdminURL(0), volumeID, 1, "before-move"); code != http.StatusCreated {
t.Fatalf("initial upload: status %d", code)
}
mover := volume_move.NewMover(grpc.WithTransportCredentials(insecure.NewCredentials()))
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
err := mover.LiveMoveVolume(ctx, needle.VolumeId(volumeID), moverServerAddress(cluster, 0), moverServerAddress(cluster, 1), volume_move.VolumeMoveOptions{
Progress: func(percent float64, stage string) {
if percent == 20 {
cancel()
}
},
})
if err == nil {
t.Fatalf("LiveMoveVolume succeeded with a cancelled context")
}
reloadVolume(t, sourceClient, volumeID)
if code := uploadStatus(t, httpClient, cluster.VolumeAdminURL(0), volumeID, 2, "after-abort"); code != http.StatusCreated {
t.Fatalf("aborted move left the source readonly after a reload: status %d", code)
}
}
+15 -1
View File
@@ -12,6 +12,7 @@ import (
"github.com/seaweedfs/seaweedfs/weed/iam"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/master_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
)
// Access key status constants
@@ -32,6 +33,7 @@ type AdminData struct {
VolumeServers []VolumeServer `json:"volume_servers"`
FilerNodes []FilerNode `json:"filer_nodes"`
MessageBrokers []MessageBrokerNode `json:"message_brokers"`
VectorNodes []VectorNode `json:"vector_nodes"`
S3Nodes []S3Node `json:"s3_nodes"`
DataCenters []DataCenter `json:"datacenters"`
LastUpdated time.Time `json:"last_updated"`
@@ -250,6 +252,7 @@ func (s *AdminServer) GetAdminData(username string) (AdminData, error) {
FilerNodes: filerNodes,
MessageBrokers: messageBrokers,
S3Nodes: s3Nodes,
VectorNodes: s.getVectorNodesStatus(context.Background()).Engines,
DataCenters: topology.DataCenters,
LastUpdated: topology.UpdatedAt,
TotalEcVolumes: totalEcVolumes,
@@ -288,6 +291,17 @@ func (s *AdminServer) ShowOverview(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, topology)
}
// raftMasterHttpAddress returns a raft member's HTTP address. Members are
// named after their master address, while the raft address only carries the
// gRPC port, which is not always the HTTP port + 10000.
func raftMasterHttpAddress(server *master_pb.RaftListClusterServersResponse_ClusterServers) string {
httpAddress := pb.ServerAddress(server.Id).ToHttpAddress()
if _, port, err := util.ParseHostPort(httpAddress); err == nil && port > 0 && port < 65536 {
return httpAddress
}
return pb.GrpcAddressToServerAddress(server.Address)
}
// getMasterNodesStatus returns the full set of master nodes in the cluster.
// It prefers the authoritative raft membership (RaftListClusterServers) and
// falls back to the currently-connected master if the raft call fails, so the
@@ -311,7 +325,7 @@ func (s *AdminServer) getMasterNodesStatus() []MasterNode {
glog.Warningf("skip master with invalid raft address %q: %v", server.Address, splitErr)
continue
}
httpAddress := pb.GrpcAddressToServerAddress(server.Address)
httpAddress := raftMasterHttpAddress(server)
if httpAddress == "" {
glog.Warningf("skip master with invalid raft address %q", server.Address)
continue
+28
View File
@@ -0,0 +1,28 @@
package dash
import (
"testing"
"github.com/seaweedfs/seaweedfs/weed/pb/master_pb"
)
func TestRaftMasterHttpAddress(t *testing.T) {
tests := []struct {
name, id, address, want string
}{
{"goraft names members by full address", "127.101.0.1:11111.11112", "127.101.0.1:11112", "127.101.0.1:11111"},
{"hashicorp names members by http address", "127.102.0.1:2222", "127.102.0.1:3333", "127.102.0.1:2222"},
{"default ports", "10.0.0.1:9333", "10.0.0.1:19333", "10.0.0.1:9333"},
{"ipv6", "[::1]:2222.3333", "[::1]:3333", "[::1]:2222"},
{"id that is not an address", "master-a", "10.0.0.2:19333", "10.0.0.2:9333"},
{"id without a valid port", "10.0.0.3:0", "10.0.0.3:19334", "10.0.0.3:9334"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := raftMasterHttpAddress(&master_pb.RaftListClusterServersResponse_ClusterServers{Id: tt.id, Address: tt.address})
if got != tt.want {
t.Errorf("raftMasterHttpAddress(%q, %q) = %q, want %q", tt.id, tt.address, got, tt.want)
}
})
}
}
+1 -2
View File
@@ -1371,8 +1371,7 @@ func (s *AdminServer) GetClusterMasters() (*ClusterMastersData, error) {
// Process each raft server
for _, server := range resp.ClusterServers {
// Raft stores gRPC addresses, convert to HTTP address
httpAddress := pb.GrpcAddressToServerAddress(server.Address)
httpAddress := raftMasterHttpAddress(server)
if httpAddress == "" {
continue
}
@@ -28,3 +28,11 @@ func TestListClusterNodesRequestKeepsDefaultFilerGroupEmpty(t *testing.T) {
t.Fatalf("FilerGroup = %q, want empty", req.FilerGroup)
}
}
func TestVectorDiscoveryUsesRegistrationGroup(t *testing.T) {
s := &AdminServer{filerGroup: "tenant-a"}
request := s.listClusterNodesRequest(cluster.VectorType)
if request.FilerGroup != s.filerGroup {
t.Fatalf("vector registration group must match the filer: %+v", request)
}
}
+7 -7
View File
@@ -105,16 +105,16 @@ func (s *AdminServer) GetDashboardTrends() DashboardTrends {
tasks := series(func(s dashSample) float64 { return s.tasks })
workers := series(func(s dashSample) float64 { return s.workers })
// Sparkline colors match the existing cards' border colors.
// Sparkline colors follow the data palette.
return DashboardTrends{
Samples: len(samples),
Volumes: sparklineSVG(series(func(s dashSample) float64 { return s.volumes }), "#1cc88a"), // success
Chunks: sparklineSVG(series(func(s dashSample) float64 { return s.chunks }), "#36b9cc"), // info
DiskUsed: sparklineSVG(series(func(s dashSample) float64 { return s.diskUsed }), "#f6c23e"), // warning
EcShards: sparklineSVG(series(func(s dashSample) float64 { return s.ecShards }), "#5a5c69"), // dark
Tasks: sparklineSVG(tasks, "#36b9cc"),
Volumes: sparklineSVG(series(func(s dashSample) float64 { return s.volumes }), "#5552e0"),
Chunks: sparklineSVG(series(func(s dashSample) float64 { return s.chunks }), "#00897a"),
DiskUsed: sparklineSVG(series(func(s dashSample) float64 { return s.diskUsed }), "#c2541a"),
EcShards: sparklineSVG(series(func(s dashSample) float64 { return s.ecShards }), "#9a45c2"),
Tasks: sparklineSVG(tasks, "#8f6d10"),
TasksValue: trendCount(last(tasks)),
Workers: sparklineSVG(workers, "#4e73df"),
Workers: sparklineSVG(workers, "#5552e0"),
WorkersValue: trendCount(last(workers)),
}
}
+59 -6
View File
@@ -2,11 +2,13 @@ package dash
import (
"context"
"encoding/json"
"errors"
"net/http"
"time"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/s3api/lance"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3_constants"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3tables"
"github.com/seaweedfs/seaweedfs/weed/s3api/s3vectors"
@@ -15,6 +17,7 @@ import (
type S3VectorsBucketsData struct {
Username string `json:"username"`
Buckets []S3VectorsBucketSummary `json:"buckets"`
Operations VectorOperationsData `json:"operations"`
TotalBuckets int `json:"total_buckets"`
Truncated bool `json:"truncated,omitempty"`
LastUpdated time.Time `json:"last_updated"`
@@ -33,6 +36,7 @@ type S3VectorsBucketData struct {
Bucket S3VectorsBucketSummary `json:"bucket"`
Policy string `json:"policy,omitempty"`
Indexes []*s3vectors.Index `json:"indexes"`
Operations VectorOperationsData `json:"operations"`
TotalIndexes int `json:"total_indexes"`
Truncated bool `json:"truncated,omitempty"`
LastUpdated time.Time `json:"last_updated"`
@@ -46,11 +50,15 @@ func (c s3VectorsFilerClient) WithFilerClient(_ bool, fn func(client filer_pb.Se
return c.s.WithFilerClient(fn)
}
// newS3VectorsServer serves the catalog side of S3 Vectors for the admin
// console. It has no vector engine, so it never runs an operation that needs
// one, such as CreateIndex.
func newS3VectorsServer(s *AdminServer) *s3vectors.Server {
server := s3vectors.NewServer(s3VectorsFilerClient{s}, nil, nil)
client := s3VectorsFilerClient{s}
backend := s3vectors.NewVectorDiscovery(s.masterClient.ListCachedMasters, s.grpcDialOption, s.filerGroup)
server := s3vectors.NewServer(client, nil, backend)
catalog := lance.NewServer(client, nil)
server.SetDescribe(func(ctx context.Context, id []string) (string, map[string]string, error) {
request := (&http.Request{Method: http.MethodPost, Header: make(http.Header)}).WithContext(ctx)
return catalog.DescribeTableInternal(request, id)
})
server.SetRegion(s3tables.DefaultRegion)
server.SetAccountID(s3tables.DefaultAccountID)
return server
@@ -103,6 +111,7 @@ func (s *AdminServer) GetS3VectorsBucketsData(ctx context.Context) (S3VectorsBuc
}
req.NextToken = resp.NextToken
}
data.Operations = s.GetVectorOperationsData(ctx)
data.TotalBuckets = len(data.Buckets)
data.LastUpdated = time.Now()
return data, nil
@@ -148,6 +157,7 @@ func (s *AdminServer) GetS3VectorsBucketData(ctx context.Context, bucket string)
}
req.NextToken = resp.NextToken
}
data.Operations = s.GetVectorOperationsData(ctx)
data.TotalIndexes = len(data.Indexes)
return data, nil
}
@@ -166,14 +176,20 @@ func (s *AdminServer) CreateS3VectorsBucket(w http.ResponseWriter, r *http.Reque
return
}
var req struct {
Name string `json:"name"`
Name string `json:"name"`
EncryptionConfiguration map[string]interface{} `json:"encryptionConfiguration,omitempty"`
Tags map[string]string `json:"tags,omitempty"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if req.EncryptionConfiguration["sseType"] == "AES256" && !s.vectorEncryptionConfigured(r.Context()) {
writeJSONError(w, http.StatusBadRequest, "Cannot verify SSE-S3 support on the S3 gateway; configure s3.public_endpoint and an SSE-S3 key on the gateway and vector engines")
return
}
var resp s3vectors.CreateVectorBucketResponse
if err := s.executeS3VectorsOperation(r.Context(), "CreateVectorBucket", &s3vectors.CreateVectorBucketRequest{VectorBucketName: req.Name}, &resp); err != nil {
if err := s.executeS3VectorsOperation(r.Context(), "CreateVectorBucket", &s3vectors.CreateVectorBucketRequest{VectorBucketName: req.Name, EncryptionConfiguration: req.EncryptionConfiguration, Tags: req.Tags}, &resp); err != nil {
writeS3VectorsError(w, err)
return
}
@@ -252,3 +268,40 @@ func writeS3VectorsError(w http.ResponseWriter, err error) {
}
writeJSONError(w, status, e.Message)
}
func (s *AdminServer) ExecuteS3VectorsOperationAPI(w http.ResponseWriter, r *http.Request) {
if !requireSessionCSRFToken(w, r) {
return
}
var req struct {
Operation string `json:"operation"`
Request json.RawMessage `json:"request"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &req); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
switch req.Operation {
case "CreateIndex", "PutVectorBucketPolicy", "DeleteVectorBucketPolicy", "TagResource", "UntagResource", "ListTagsForResource", "PutVectors", "GetVectors", "ListVectors", "DeleteVectors", "QueryVectors":
default:
writeJSONError(w, http.StatusBadRequest, "Unsupported vector operation")
return
}
switch req.Operation {
case "QueryVectors", "GetVectors", "ListVectors", "ListTagsForResource":
default:
if role := RoleFromContext(r.Context()); role != "admin" && (role != "" || UsernameFromContext(r.Context()) != "") {
writeJSONError(w, http.StatusForbidden, "This operation requires admin access")
return
}
}
var resp json.RawMessage
if err := s.executeS3VectorsOperation(r.Context(), req.Operation, req.Request, &resp); err != nil {
writeS3VectorsError(w, err)
return
}
if len(resp) == 0 {
resp = json.RawMessage(`{}`)
}
writeJSON(w, http.StatusOK, resp)
}
+79 -1
View File
@@ -22,7 +22,9 @@ func newS3VectorsTestServer(t *testing.T) (*AdminServer, *s3tables.ManagerClient
server := s3vectors.NewServer(client, nil, nil)
server.SetRegion(s3tables.DefaultRegion)
server.SetAccountID(s3tables.DefaultAccountID)
return &AdminServer{s3VectorsServer: server}, client
gateway := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte(`{"sseS3":true}`)) }))
t.Cleanup(gateway.Close)
return &AdminServer{s3VectorsServer: server, s3PublicEndpoint: gateway.URL}, client
}
// createS3VectorsIndexRecord writes the catalog record CreateIndex leaves
@@ -172,3 +174,79 @@ func TestS3VectorsBucketsStopAtTheListLimit(t *testing.T) {
t.Fatalf("listed %d buckets (truncated %v), want the limit %d and truncated", data.TotalBuckets, data.Truncated, s3TablesAdminListLimit)
}
}
func TestS3VectorsManagementOperations(t *testing.T) {
s, _ := newS3VectorsTestServer(t)
w := s3VectorsRequest(t, s.CreateS3VectorsBucket, http.MethodPost, "/api/s3vectors/buckets", `{"name":"managed","encryptionConfiguration":{"sseType":"AES256"},"tags":{"env":"test"}}`)
if w.Code != http.StatusCreated {
t.Fatal(w.Code, w.Body)
}
summary, err := s.s3VectorsBucketSummary(t.Context(), "managed")
if err != nil || summary.Encryption != "AES256" {
t.Fatal(summary, err)
}
arn := summary.ARN
for _, body := range []string{
`{"operation":"TagResource","request":{"resourceArn":"` + arn + `","tags":{"team":"search"}}}`,
`{"operation":"ListTagsForResource","request":{"resourceArn":"` + arn + `"}}`,
`{"operation":"UntagResource","request":{"resourceArn":"` + arn + `","tagKeys":["team"]}}`,
`{"operation":"PutVectorBucketPolicy","request":{"vectorBucketName":"managed","policy":"{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3vectors:QueryVectors\",\"Resource\":\"*\"}]}"}}`,
`{"operation":"DeleteVectorBucketPolicy","request":{"vectorBucketName":"managed"}}`,
} {
w := s3VectorsRequest(t, s.ExecuteS3VectorsOperationAPI, http.MethodPost, "/api/s3vectors/operations", body)
if w.Code != http.StatusOK {
t.Fatalf("%s: %d %s", body, w.Code, w.Body)
}
}
for _, body := range []string{`{"operation":"Unknown","request":{}}`, `{bad json`} {
if w := s3VectorsRequest(t, s.ExecuteS3VectorsOperationAPI, http.MethodPost, "/api/s3vectors/operations", body); w.Code != http.StatusBadRequest {
t.Fatal(w.Code, w.Body)
}
}
request := httptest.NewRequest(http.MethodPost, "/api/s3vectors/operations", strings.NewReader(`{}`))
request = request.WithContext(WithAuthContext(request.Context(), "admin", "admin", "required-token"))
recorder := httptest.NewRecorder()
s.ExecuteS3VectorsOperationAPI(recorder, request)
if recorder.Code != http.StatusForbidden {
t.Fatal("missing CSRF token accepted", recorder.Code)
}
}
func TestS3VectorsEncryptedCreationRequiresKey(t *testing.T) {
s, _ := newS3VectorsTestServer(t)
s.s3VectorsServer.SetEncryptionChecker(func() bool { return false })
w := s3VectorsRequest(t, s.CreateS3VectorsBucket, http.MethodPost, "/api/s3vectors/buckets", `{"name":"sealed","encryptionConfiguration":{"sseType":"AES256"}}`)
if w.Code != http.StatusBadRequest {
t.Fatalf("encrypted bucket without key: %d %s", w.Code, w.Body)
}
data, err := s.GetS3VectorsBucketsData(t.Context())
if err != nil || len(data.Buckets) != 0 {
t.Fatalf("unusable bucket persisted: %+v %v", data, err)
}
}
func TestS3VectorsReadOnlyOperations(t *testing.T) {
s, _ := newS3VectorsTestServer(t)
s3VectorsRequest(t, s.CreateS3VectorsBucket, http.MethodPost, "/api/s3vectors/buckets", `{"name":"readable"}`)
summary, err := s.s3VectorsBucketSummary(t.Context(), "readable")
if err != nil {
t.Fatal(err)
}
for _, role := range []string{"readonly", ""} {
for _, operation := range []string{"ListTagsForResource", "TagResource", "CreateIndex", "DeleteVectors"} {
body := fmt.Sprintf(`{"operation":%q,"request":{"resourceArn":%q}}`, operation, summary.ARN)
request := httptest.NewRequest(http.MethodPost, "/api/s3vectors/operations", strings.NewReader(body))
request = request.WithContext(WithAuthContext(request.Context(), "viewer", role, "token"))
request.Header.Set("X-CSRF-Token", "token")
w := httptest.NewRecorder()
s.ExecuteS3VectorsOperationAPI(w, request)
want := http.StatusForbidden
if operation == "ListTagsForResource" {
want = http.StatusOK
}
if w.Code != want {
t.Fatalf("%s: %d %s", operation, w.Code, w.Body)
}
}
}
}
+60
View File
@@ -0,0 +1,60 @@
package dash
import (
"context"
"encoding/json"
"io"
"net/http"
"strings"
"time"
"github.com/seaweedfs/seaweedfs/weed/cluster"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/master_pb"
util_http "github.com/seaweedfs/seaweedfs/weed/util/http"
)
func (s *AdminServer) vectorEncryptionConfigured(ctx context.Context) bool {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
endpoints := []string{}
if s.s3PublicEndpoint != "" {
endpoints = append(endpoints, s.s3PublicEndpoint)
} else if s.masterClient != nil {
err := s.masterClient.WithClient(ctx, false, func(client master_pb.SeaweedClient) error {
response, err := client.ListClusterNodes(ctx, s.listClusterNodesRequest(cluster.S3Type))
if err != nil {
return err
}
for _, node := range response.ClusterNodes {
endpoints = append(endpoints, "http://"+pb.ServerAddress(node.Address).ToHttpAddress())
}
return nil
})
if err != nil {
return false
}
}
if len(endpoints) == 0 {
return false
}
for _, endpoint := range endpoints {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimRight(endpoint, "/")+"/status?features", nil)
if err != nil {
return false
}
response, err := util_http.GetGlobalHttpClient().Do(request)
if err != nil {
return false
}
var features struct {
SSES3 bool `json:"sseS3"`
}
err = json.NewDecoder(io.LimitReader(response.Body, 4096)).Decode(&features)
_ = response.Body.Close()
if response.StatusCode != http.StatusOK || err != nil || !features.SSES3 {
return false
}
}
return true
}
+26
View File
@@ -0,0 +1,26 @@
package dash
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestVectorEncryptionChecksRemoteGateway(t *testing.T) {
for _, test := range []struct {
body string
want bool
}{{`{"sseS3":true}`, true}, {`{"sseS3":false}`, false}, {``, false}} {
gateway := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/status" || !r.URL.Query().Has("features") {
t.Errorf("wrong probe: %s", r.URL)
}
_, _ = w.Write([]byte(test.body))
}))
s := &AdminServer{s3PublicEndpoint: gateway.URL}
if got := s.vectorEncryptionConfigured(t.Context()); got != test.want {
t.Errorf("body %q: %v", test.body, got)
}
gateway.Close()
}
}
+107
View File
@@ -0,0 +1,107 @@
package dash
import (
"context"
"sort"
"time"
adminplugin "github.com/seaweedfs/seaweedfs/weed/admin/plugin"
"github.com/seaweedfs/seaweedfs/weed/cluster"
"github.com/seaweedfs/seaweedfs/weed/pb"
"github.com/seaweedfs/seaweedfs/weed/pb/master_pb"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/grpc/status"
)
type VectorNode struct {
Address string `json:"address"`
Version string `json:"version"`
Status string `json:"status"`
}
type VectorOperationsData struct {
Engines []VectorNode `json:"engines"`
DiscoveryError string `json:"discovery_error,omitempty"`
MaintenanceWorkers int `json:"maintenance_workers"`
MaintenanceJobs []adminplugin.TrackedJob `json:"maintenance_jobs"`
}
func (s *AdminServer) getVectorNodesStatus(ctx context.Context) VectorOperationsData {
data := VectorOperationsData{Engines: []VectorNode{}}
if s.masterClient == nil {
data.DiscoveryError = "Master discovery unavailable"
return data
}
var discoveryErr error
for _, master := range s.masterClient.ListCachedMasters() {
discoveryCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
conn, err := pb.GrpcDial(discoveryCtx, master.ToGrpcAddress(), false, s.grpcDialOption)
if err == nil {
response, callErr := master_pb.NewSeaweedClient(conn).ListClusterNodes(discoveryCtx, s.listClusterNodesRequest(cluster.VectorType))
_ = conn.Close()
err = callErr
if err == nil {
for _, node := range response.ClusterNodes {
data.Engines = append(data.Engines, VectorNode{Address: node.Address, Version: node.Version, Status: "Registered"})
}
}
}
cancel()
if err != nil {
discoveryErr = err
}
if len(data.Engines) > 0 {
break
}
}
if len(data.Engines) == 0 && discoveryErr != nil {
data.DiscoveryError = discoveryErr.Error()
return data
}
var checks errgroup.Group
checks.SetLimit(8)
for i := range data.Engines {
checks.Go(func() error {
data.Engines[i].Status = vectorNodeStatus(ctx, data.Engines[i].Address, s.grpcDialOption)
return nil
})
}
_ = checks.Wait()
sort.Slice(data.Engines, func(i, j int) bool { return data.Engines[i].Address < data.Engines[j].Address })
return data
}
func vectorNodeStatus(ctx context.Context, address string, dialOption grpc.DialOption) string {
ctx, cancel := context.WithTimeout(ctx, time.Second)
defer cancel()
conn, err := pb.GrpcDial(ctx, address, false, dialOption)
if err != nil {
return "Unreachable"
}
defer conn.Close()
response, err := grpc_health_v1.NewHealthClient(conn).Check(ctx, &grpc_health_v1.HealthCheckRequest{Service: "vector_pb.VectorService"})
if status.Code(err) == codes.Unimplemented || status.Code(err) == codes.NotFound {
return "Registered (health unavailable)"
}
if err != nil {
return "Unreachable"
}
if response.Status == grpc_health_v1.HealthCheckResponse_SERVING {
return "Ready"
}
return "Not ready"
}
func (s *AdminServer) GetVectorOperationsData(ctx context.Context) VectorOperationsData {
data := s.getVectorNodesStatus(ctx)
for _, worker := range s.GetPluginWorkers() {
if capability := worker.Capabilities["lance_vector_index"]; capability != nil && capability.CanExecute {
data.MaintenanceWorkers++
}
}
data.MaintenanceJobs = s.ListPluginJobs("lance_vector_index", "", 5)
return data
}
+49
View File
@@ -0,0 +1,49 @@
package dash
import (
"context"
"net"
"testing"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/health"
"google.golang.org/grpc/health/grpc_health_v1"
)
func TestVectorNodeStatus(t *testing.T) {
for _, want := range []string{"Ready", "Not ready", "Registered (health unavailable)", "Unknown service"} {
t.Run(want, func(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
server := grpc.NewServer()
if want != "Registered (health unavailable)" {
checker := health.NewServer()
state := grpc_health_v1.HealthCheckResponse_NOT_SERVING
if want == "Ready" {
state = grpc_health_v1.HealthCheckResponse_SERVING
}
if want != "Unknown service" {
checker.SetServingStatus("vector_pb.VectorService", state)
}
grpc_health_v1.RegisterHealthServer(server, checker)
}
go server.Serve(listener)
t.Cleanup(server.Stop)
expected := want
if want == "Unknown service" {
expected = "Registered (health unavailable)"
}
if got := vectorNodeStatus(t.Context(), listener.Addr().String(), grpc.WithTransportCredentials(insecure.NewCredentials())); got != expected {
t.Fatalf("got %s want %s", got, want)
}
})
}
ctx, cancel := context.WithCancel(t.Context())
cancel()
if got := vectorNodeStatus(ctx, "127.0.0.1:1", grpc.WithTransportCredentials(insecure.NewCredentials())); got != "Unreachable" {
t.Fatal(got)
}
}
+2
View File
@@ -131,6 +131,7 @@ func (h *AdminHandlers) registerUIRoutes(r *mux.Router) {
r.HandleFunc("/cluster/masters", h.clusterHandlers.ShowClusterMasters).Methods(http.MethodGet)
r.HandleFunc("/cluster/filers", h.clusterHandlers.ShowClusterFilers).Methods(http.MethodGet)
r.HandleFunc("/cluster/s3", h.clusterHandlers.ShowClusterS3Servers).Methods(http.MethodGet)
r.HandleFunc("/cluster/vector-servers", h.clusterHandlers.ShowClusterVectorServers).Methods(http.MethodGet)
r.HandleFunc("/cluster/volume-servers", h.clusterHandlers.ShowClusterVolumeServers).Methods(http.MethodGet)
r.HandleFunc("/cluster/mount-clients", h.clusterHandlers.ShowMountClients).Methods(http.MethodGet)
@@ -256,6 +257,7 @@ func (h *AdminHandlers) registerAPIRoutes(api *mux.Router, enforceWrite bool) {
s3TablesApi.Handle("/tags", wrapWrite(h.adminServer.UntagS3TablesResource)).Methods(http.MethodDelete)
s3VectorsApi := api.PathPrefix("/s3vectors").Subrouter()
s3VectorsApi.HandleFunc("/operations", h.adminServer.ExecuteS3VectorsOperationAPI).Methods(http.MethodPost)
s3VectorsApi.HandleFunc("/buckets", h.adminServer.ListS3VectorsBucketsAPI).Methods(http.MethodGet)
s3VectorsApi.Handle("/buckets", wrapWrite(h.adminServer.CreateS3VectorsBucket)).Methods(http.MethodPost)
s3VectorsApi.Handle("/buckets", wrapWrite(h.adminServer.DeleteS3VectorsBucket)).Methods(http.MethodDelete)
+10
View File
@@ -388,6 +388,16 @@ func (h *ClusterHandlers) ShowClusterS3Servers(w http.ResponseWriter, r *http.Re
}
}
// ShowClusterVectorServers renders the vector engines registered with the master
func (h *ClusterHandlers) ShowClusterVectorServers(w http.ResponseWriter, r *http.Request) {
data := h.adminServer.GetVectorOperationsData(r.Context())
w.Header().Set("Content-Type", "text/html")
viewCtx := layout.NewViewContext(r, usernameOrDefault(r), dash.CSRFTokenFromContext(r.Context()))
if err := layout.Layout(viewCtx, app.ClusterVectorServers(data)).Render(r.Context(), w); err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to render template: "+err.Error())
}
}
// GetClusterTopology returns the cluster topology as JSON
func (h *ClusterHandlers) GetClusterTopology(w http.ResponseWriter, r *http.Request) {
topology, err := h.adminServer.GetClusterTopology()
+6 -4
View File
@@ -1120,13 +1120,15 @@ func (r *Plugin) trackExecutionCompletion(completed *plugin_pb.JobCompleted) *Tr
if completed.RequestId != "" {
tracked.RequestID = completed.RequestId
}
if completed.Result != nil {
tracked.ResultSummary = completed.Result.Summary
} else {
tracked.ResultSummary = ""
}
if completed.Success {
tracked.State = strings.ToLower(plugin_pb.JobState_JOB_STATE_SUCCEEDED.String())
tracked.Progress = 100
tracked.Stage = "completed"
if completed.Result != nil {
tracked.ResultSummary = completed.Result.Summary
}
tracked.Message = tracked.ResultSummary
if tracked.Message == "" {
tracked.Message = "completed"
@@ -1159,7 +1161,7 @@ func (r *Plugin) trackExecutionCompletion(completed *plugin_pb.JobCompleted) *Tr
detail.Message = clone.Message
detail.ErrorMessage = clone.ErrorMessage
detail.ResultSummary = clone.ResultSummary
if completed.Success && completed.Result != nil {
if completed.Result != nil {
detail.ResultOutputValues = configValueMapToPlain(completed.Result.OutputValues)
} else {
detail.ResultOutputValues = nil
+147
View File
@@ -594,6 +594,95 @@ func TestBuildJobDetailIncludesActivitiesAndRunRecord(t *testing.T) {
}
}
func TestBuildJobDetailKeepsFailedJobResult(t *testing.T) {
t.Parallel()
pluginSvc, err := New(Options{DataDir: t.TempDir()})
if err != nil {
t.Fatalf("New: %v", err)
}
defer pluginSvc.Shutdown()
pluginSvc.trackExecutionStart("req-failed", "worker-f", &plugin_pb.JobSpec{
JobId: "job-failed",
JobType: "s3_lifecycle",
}, 1)
pluginSvc.handleJobCompleted(&plugin_pb.JobCompleted{
RequestId: "req-failed",
JobId: "job-failed",
JobType: "s3_lifecycle",
Success: false,
ErrorMessage: "walk bad-a: filer down",
Result: &plugin_pb.JobResult{
Summary: "pass failed",
OutputValues: map[string]*plugin_pb.ConfigValue{
"buckets_failed": {Kind: &plugin_pb.ConfigValue_Int64Value{Int64Value: 2}},
},
},
CompletedAt: timestamppb.Now(),
})
pluginSvc.Shutdown()
detail, found, err := pluginSvc.BuildJobDetail("job-failed", 100, 5)
if err != nil {
t.Fatalf("BuildJobDetail error: %v", err)
}
if !found || detail == nil || detail.Job == nil {
t.Fatalf("expected job detail")
}
if detail.Job.State != "job_state_failed" || detail.Job.ErrorMessage != "walk bad-a: filer down" {
t.Fatalf("expected the failure to be kept, got state=%q error=%q", detail.Job.State, detail.Job.ErrorMessage)
}
if detail.Job.ResultSummary != "pass failed" {
t.Fatalf("expected the failed job's result summary, got %q", detail.Job.ResultSummary)
}
if detail.Job.ResultOutputValues["buckets_failed"] == nil {
t.Fatalf("expected the failed job's result output values, got %+v", detail.Job.ResultOutputValues)
}
}
func TestRetryWithoutResultDropsFailedAttemptSummary(t *testing.T) {
t.Parallel()
pluginSvc, err := New(Options{DataDir: t.TempDir()})
if err != nil {
t.Fatalf("New: %v", err)
}
defer pluginSvc.Shutdown()
job := &plugin_pb.JobSpec{JobId: "job-retry", JobType: "s3_lifecycle"}
pluginSvc.trackExecutionStart("req-retry", "worker-r", job, 1)
pluginSvc.handleJobCompleted(&plugin_pb.JobCompleted{
RequestId: "req-retry",
JobId: "job-retry",
JobType: "s3_lifecycle",
Success: false,
ErrorMessage: "walk bad-a: filer down",
Result: &plugin_pb.JobResult{Summary: "pass failed"},
CompletedAt: timestamppb.Now(),
})
pluginSvc.trackExecutionStart("req-retry", "worker-r", job, 2)
pluginSvc.handleJobCompleted(&plugin_pb.JobCompleted{
RequestId: "req-retry",
JobId: "job-retry",
JobType: "s3_lifecycle",
Success: true,
CompletedAt: timestamppb.Now(),
})
pluginSvc.Shutdown()
detail, found, err := pluginSvc.BuildJobDetail("job-retry", 100, 5)
if err != nil {
t.Fatalf("BuildJobDetail error: %v", err)
}
if !found || detail == nil || detail.Job == nil {
t.Fatalf("expected job detail")
}
if detail.Job.ResultSummary != "" || detail.Job.Message != "completed" {
t.Fatalf("expected the failed attempt's summary to be dropped, got summary=%q message=%q", detail.Job.ResultSummary, detail.Job.Message)
}
}
func TestBuildJobDetailLoadsFromDiskWhenMemoryCleared(t *testing.T) {
t.Parallel()
@@ -704,6 +793,64 @@ func TestExpireStaleJobsCancelsTheJobOnItsWorker(t *testing.T) {
expectCancelFor(t, session, "req-stale")
}
// A job older than the stale limit stays active while its worker reports
// progress for it; the worker's heartbeat alone does not keep it alive.
func TestExpireStaleJobsKeepsAJobThatReportsProgress(t *testing.T) {
t.Parallel()
pluginSvc, err := New(Options{})
if err != nil {
t.Fatalf("New: %v", err)
}
defer pluginSvc.Shutdown()
const workerID = "worker-1"
session := &streamSession{workerID: workerID, outgoing: make(chan *plugin_pb.AdminToWorkerMessage, 8), done: make(chan struct{})}
pluginSvc.putSession(session)
pluginSvc.handleWorkerMessage(session, &plugin_pb.WorkerToAdminMessage{
Body: &plugin_pb.WorkerToAdminMessage_Hello{Hello: &plugin_pb.WorkerHello{WorkerId: workerID}},
})
pluginSvc.trackExecutionStart("req-working", workerID, &plugin_pb.JobSpec{JobId: "job-working", JobType: "s3_lifecycle"}, 1)
pluginSvc.trackExecutionStart("req-stalled", workerID, &plugin_pb.JobSpec{JobId: "job-stalled", JobType: "s3_lifecycle"}, 1)
startedAt := time.Now().UTC().Add(-defaultStaleActiveJobTimeout - time.Hour)
pluginSvc.jobsMu.Lock()
for _, job := range pluginSvc.jobs {
job.CreatedAt = timeToPtr(startedAt)
job.UpdatedAt = timeToPtr(startedAt)
}
pluginSvc.jobsMu.Unlock()
pluginSvc.handleWorkerMessage(session, &plugin_pb.WorkerToAdminMessage{
Body: &plugin_pb.WorkerToAdminMessage_Heartbeat{Heartbeat: &plugin_pb.WorkerHeartbeat{
WorkerId: workerID,
ExecutionSlotsTotal: 4,
ExecutionSlotsUsed: 2,
RunningWork: []*plugin_pb.RunningWork{
{WorkId: "job-working", Kind: plugin_pb.WorkKind_WORK_KIND_EXECUTION, JobType: "s3_lifecycle"},
{WorkId: "job-stalled", Kind: plugin_pb.WorkKind_WORK_KIND_EXECUTION, JobType: "s3_lifecycle"},
},
}},
})
pluginSvc.handleWorkerMessage(session, &plugin_pb.WorkerToAdminMessage{
Body: &plugin_pb.WorkerToAdminMessage_JobProgressUpdate{JobProgressUpdate: &plugin_pb.JobProgressUpdate{
JobId: "job-working", JobType: "s3_lifecycle", RequestId: "req-working",
State: plugin_pb.JobState_JOB_STATE_RUNNING, Stage: "walk", Message: "entries=10",
}},
})
if n := pluginSvc.expireStaleJobs(time.Now().UTC()); n != 1 {
t.Fatalf("expired %d jobs, want only the stalled one", n)
}
if got, _ := pluginSvc.GetTrackedJob("job-working"); got == nil || got.State != "job_state_running" {
t.Fatalf("expected the job reporting progress to stay running, got %+v", got)
}
if got, _ := pluginSvc.GetTrackedJob("job-stalled"); got == nil || got.State != StateFailed {
t.Fatalf("expected the heartbeat-only job to be failed, got %+v", got)
}
expectCancelFor(t, session, "req-stalled")
}
func expectNoCancel(t *testing.T, session *streamSession) {
t.Helper()
select {
File diff suppressed because it is too large. Load diff
+6 -53
View File
@@ -573,52 +573,6 @@ function refreshBuckets() {
location.reload();
}
// Export bucket list
function exportBucketList() {
// Get table data
const table = document.getElementById('bucketsTable');
if (!table) return;
const rows = Array.from(table.querySelectorAll('tbody tr'));
const data = rows.map(row => {
const cells = row.querySelectorAll('td');
if (cells.length < 5) return null; // Skip empty state row
return {
name: cells[0].textContent.trim(),
owner: cells[1].textContent.trim(),
created: cells[2].textContent.trim(),
logicalSize: cells[3].textContent.trim(),
physicalSize: cells[4].textContent.trim(),
quota: cells[5].textContent.trim()
};
}).filter(item => item !== null);
// Convert to CSV
const csv = [
['Name', 'Owner', 'Created', 'Logical Size', 'Physical Size', 'Quota'].join(','),
...data.map(row => [
row.name,
row.owner,
row.created,
row.logicalSize,
row.physicalSize,
row.quota
].join(','))
].join('\n');
// Download CSV
const blob = new Blob([csv], { type: 'text/csv' });
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `seaweedfs-buckets-${new Date().toISOString().split('T')[0]}.csv`;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
window.URL.revokeObjectURL(url);
}
// showAlert is provided by modal-alerts.js with signature: showAlert(message, type)
// Format date for display
@@ -871,19 +825,18 @@ function exportUsers() {
return;
}
let csvContent = 'Username,Email,Access Key,Status,Created,Last Login\n';
let csvContent = 'Username,Email,Access Key\n';
rows.forEach(row => {
const cells = row.querySelectorAll('td');
if (cells.length >= 6) {
const username = cells[0].textContent.trim();
if (cells.length >= 3) {
const username = (cells[0].querySelector('strong') || cells[0]).textContent.trim();
const email = cells[1].textContent.trim();
const accessKey = cells[2].textContent.trim();
const status = cells[3].textContent.trim();
const created = cells[4].textContent.trim();
const lastLogin = cells[5].textContent.trim();
csvContent += `"${username}","${email}","${accessKey}","${status}","${created}","${lastLogin}"\n`;
csvContent += [username, email, accessKey]
.map(value => `"${value.replace(/"/g, '""')}"`)
.join(',') + '\n';
}
});
+43
View File
@@ -0,0 +1,43 @@
// Applies the saved color theme (auto, light or dark) before the page paints.
(function () {
var storageKey = 'admin-theme';
var choices = ['auto', 'light', 'dark'];
var media = window.matchMedia ? window.matchMedia('(prefers-color-scheme: dark)') : null;
var root = document.documentElement;
function savedChoice() {
try {
var value = localStorage.getItem(storageKey);
return choices.indexOf(value) >= 0 ? value : 'auto';
} catch (e) {
return 'auto';
}
}
function apply(choice) {
var dark = choice === 'dark' || (choice === 'auto' && media && media.matches);
root.setAttribute('data-bs-theme', dark ? 'dark' : 'light');
root.setAttribute('data-theme-choice', choice);
}
var current = savedChoice();
apply(current);
if (media && media.addEventListener) {
media.addEventListener('change', function () {
apply(current);
});
}
document.addEventListener('click', function (event) {
var toggle = event.target.closest && event.target.closest('[data-theme-toggle]');
if (!toggle) {
return;
}
current = choices[(choices.indexOf(current) + 1) % choices.length];
try {
localStorage.setItem(storageKey, current);
} catch (e) {}
apply(current);
});
})();
Binary file not shown.
Binary file not shown.
Binary file not shown.
+138 -201
View File
@@ -7,16 +7,18 @@ import (
templ Admin(data dash.AdminData) {
<div class="d-flex justify-content-between flex-wrap flex-md-nowrap align-items-center pt-3 pb-2 mb-3 border-bottom">
<h1 class="h2">
<i class="bi bi-speedometer2 me-2"></i>Dashboard
</h1>
<div class="btn-toolbar mb-2 mb-md-0">
<div class="btn-group me-2">
<a href={ dash.PUrl(ctx, "/object-store/buckets") } class="btn btn-sm btn-primary">
<i class="bi bi-box me-1"></i>Object Store Buckets
</a>
</div>
<div>
<h1 class="h2">
<i class="bi bi-speedometer2 me-2"></i>Dashboard
</h1>
<p class="mb-0 text-muted">
if len(clusterHealthIssues(data)) == 0 {
<i class="bi bi-circle-fill text-success health-dot me-2"></i>
} else {
<i class="bi bi-circle-fill text-warning health-dot me-2"></i>
}
{ clusterHealthSummary(data) }
</p>
</div>
</div>
@@ -40,18 +42,21 @@ templ Admin(data dash.AdminData) {
<!-- Summary Cards — each carries an inline sparkline of recent history
from the admin's own snapshots (no Prometheus required). -->
<div class="row mb-4">
<div class="col-xl-3 col-md-6 mb-4">
<div class="row mb-2">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-success shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-success text-uppercase mb-1">
Total Volumes
Volumes
</div>
<div class="h5 mb-0 font-weight-bold text-body-emphasis">
<div class="h5 mb-0 font-weight-bold text-gray-800">
{fmt.Sprintf("%d", data.TotalVolumes)}
</div>
<div class="text-xs text-muted">
{fmt.Sprintf("%s limit", formatBytes(int64(data.VolumeSizeLimitMB)*1024*1024))}
</div>
</div>
<div class="col-auto">
<i class="bi bi-database icon-2x text-gray-300"></i>
@@ -62,34 +67,13 @@ templ Admin(data dash.AdminData) {
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="card border-left-info shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-info text-uppercase mb-1" title="Chunks stored in volumes. A file is split into one or more chunks, so this is not the number of files.">
Total Chunks
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{formatNumber(data.TotalChunks)}
</div>
</div>
<div class="col-auto">
<i class="bi bi-boxes icon-2x text-gray-300"></i>
</div>
</div>
@templ.Raw(data.Trends.Chunks)
</div>
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-warning shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-warning text-uppercase mb-1">
Disk Usage
Disk usage
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{formatBytes(data.DiskUsage)}
@@ -109,60 +93,43 @@ templ Admin(data dash.AdminData) {
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="card border-left-primary shadow h-100 py-2">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-info shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-primary text-uppercase mb-1">
Volume Size Limit
<div class="text-xs font-weight-bold text-info text-uppercase mb-1" title="Chunks stored in volumes. A file is split into one or more chunks, so this is not the number of files.">
Chunks
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{fmt.Sprintf("%d MB", data.VolumeSizeLimitMB)}
{formatNumber(data.TotalChunks)}
</div>
</div>
<div class="col-auto">
<i class="bi bi-gear-wide-connected icon-2x text-gray-300"></i>
<i class="bi bi-boxes icon-2x text-gray-300"></i>
</div>
</div>
@templ.Raw(data.Trends.Chunks)
</div>
</div>
</div>
</div>
<!-- Second Row for EC Shards Information -->
<div class="row mb-4">
<div class="col-xl-3 col-md-6 mb-4">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-secondary shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-secondary text-uppercase mb-1">
EC Volumes
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{fmt.Sprintf("%d", data.TotalEcVolumes)}
</div>
</div>
<div class="col-auto">
<i class="bi bi-layers icon-2x text-gray-300"></i>
</div>
</div>
</div>
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="card border-left-dark shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-dark text-uppercase mb-1">
EC Shards
EC shards
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{fmt.Sprintf("%d", data.TotalEcShards)}
</div>
if data.TotalEcVolumes > 0 {
<div class="text-xs text-muted">
{fmt.Sprintf("in %s", countNoun(data.TotalEcVolumes, "EC volume"))}
</div>
}
</div>
<div class="col-auto">
<i class="bi bi-puzzle icon-2x text-gray-300"></i>
@@ -173,13 +140,13 @@ templ Admin(data dash.AdminData) {
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-info shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
<div class="col mr-2">
<div class="text-xs font-weight-bold text-info text-uppercase mb-1">
Active Tasks
Active tasks
</div>
<div class="h5 mb-0 font-weight-bold text-gray-800">
{ data.Trends.TasksValue }
@@ -194,7 +161,7 @@ templ Admin(data dash.AdminData) {
</div>
</div>
<div class="col-xl-3 col-md-6 mb-4">
<div class="col-xl-2 col-md-4 mb-4">
<div class="card border-left-primary shadow h-100 py-2">
<div class="card-body">
<div class="row no-gutters align-items-center">
@@ -254,7 +221,7 @@ templ Admin(data dash.AdminData) {
<td>{fmt.Sprintf("%d", tier.VolumeCount)}</td>
<td>
if tier.IsRemote {
<span class="text-muted">-</span>
<span class="text-muted">–</span>
} else {
{fmt.Sprintf("%d", tier.EcShardCount)}
}
@@ -262,21 +229,21 @@ templ Admin(data dash.AdminData) {
<td>{formatBytes(tier.DataSize)}</td>
<td>
if tier.IsRemote || tier.DiskUsed == 0 {
<span class="text-muted">-</span>
<span class="text-muted">–</span>
} else {
{formatBytes(tier.DiskUsed)}
}
</td>
<td>
if tier.IsRemote {
<span class="text-muted">-</span>
<span class="text-muted">–</span>
} else {
{formatBytes(tier.DiskCapacity)}
}
</td>
<td>
if tier.IsRemote {
<span class="text-muted">-</span>
<span class="text-muted">–</span>
} else {
<div class="d-flex align-items-center">
<div class="progress flex-grow-1" style="height: 20px; min-width: 48px;">
@@ -299,31 +266,6 @@ templ Admin(data dash.AdminData) {
</div>
}
<!-- Mount Clients -->
<div class="row mb-4">
<div class="col-12">
<div class="card shadow mb-4">
<div class="card-header py-3 d-flex justify-content-between align-items-center">
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-plug me-2"></i>Mount Clients
</h6>
<a href={ dash.PUrl(ctx, "/cluster/mount-clients") } class="btn btn-sm btn-outline-primary">
View Details <i class="bi bi-arrow-right ms-1"></i>
</a>
</div>
<div class="card-body">
<div class="d-flex align-items-center">
<i class="bi bi-plug icon-2x text-gray-300 me-3"></i>
<div>
<div class="h4 mb-0 font-weight-bold text-gray-800">{fmt.Sprintf("%d", data.TotalMountClients)}</div>
<div class="text-xs text-muted text-uppercase">Connected FUSE/VFS mount clients</div>
</div>
</div>
</div>
</div>
</div>
</div>
<!-- Master Nodes Status -->
<div class="row mb-4">
<div class="col-lg-4">
@@ -376,79 +318,48 @@ templ Admin(data dash.AdminData) {
</h6>
</div>
<div class="card-body text-center">
<div class="row">
<div class="col">
<div class="card bg-light">
<div class="card-body">
<h5>{fmt.Sprintf("%d", len(data.MasterNodes))}</h5>
<small class="text-muted">Masters</small>
</div>
</div>
</div>
<div class="col">
<div class="card bg-light">
<div class="card-body">
<h5>{fmt.Sprintf("%d", len(data.VolumeServers))}</h5>
<small class="text-muted">Volume Servers</small>
</div>
</div>
</div>
<div class="col">
<div class="card bg-light">
<div class="card-body">
<h5>{fmt.Sprintf("%d", len(data.FilerNodes))}</h5>
<small class="text-muted">Filers</small>
</div>
</div>
</div>
<div class="col">
<div class="card bg-light">
<div class="card-body">
<h5>{fmt.Sprintf("%d", len(data.S3Nodes))}</h5>
<small class="text-muted">S3 Servers</small>
</div>
</div>
</div>
<div class="col">
<div class="card bg-light">
<div class="card-body">
<h5>{fmt.Sprintf("%d", len(data.MessageBrokers))}</h5>
<small class="text-muted">Message Brokers</small>
</div>
</div>
</div>
<div class="col">
<div class="card bg-light">
<div class="card-body">
if data.VersionSummary.Uniform {
<h5 class="text-truncate" title={data.VersionSummary.Version}>{data.VersionSummary.Version}</h5>
<small class="text-muted">Version</small>
} else if data.VersionSummary.Mixed() {
<h5 class="text-warning">
<i class="bi bi-exclamation-triangle-fill"></i>
</h5>
<small class="text-muted">Mixed Versions</small>
} else {
<h5 class="text-truncate">—</h5>
<small class="text-muted">Version</small>
}
</div>
</div>
<div class="row g-0">
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/masters") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.MasterNodes))}</div>
<small class="text-muted">Masters</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/volume-servers") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.VolumeServers))}</div>
<small class="text-muted">Volume servers</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/filers") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.FilerNodes))}</div>
<small class="text-muted">Filers</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/s3") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.S3Nodes))}</div>
<small class="text-muted">S3 servers</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/vector-servers") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.VectorNodes))}</div>
<small class="text-muted">Vector servers</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/cluster/mount-clients") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", data.TotalMountClients)}</div>
<small class="text-muted">Mount clients</small>
</a>
<a class="col cluster-count" href={ dash.PUrl(ctx, "/mq/brokers") }>
<div class="h5 mb-0">{fmt.Sprintf("%d", len(data.MessageBrokers))}</div>
<small class="text-muted">Message brokers</small>
</a>
<div class="col cluster-count">
if data.VersionSummary.Uniform {
<div class="h5 mb-0 text-truncate" title={data.VersionSummary.Version}>{data.VersionSummary.Version}</div>
<small class="text-muted">Version</small>
} else if data.VersionSummary.Mixed() {
<div class="h5 mb-0 text-warning"><i class="bi bi-exclamation-triangle-fill"></i></div>
<small class="text-muted">Mixed versions</small>
} else {
<div class="h5 mb-0">–</div>
<small class="text-muted">Version</small>
}
</div>
</div>
if data.VersionSummary.Mixed() {
<div class="row mt-3">
<div class="col-12 text-start">
<small class="text-muted text-uppercase fw-bold">Node Versions</small>
for _, nt := range data.VersionSummary.NodeTypes {
<div class="d-flex justify-content-between border-bottom py-1">
<span>{nt.Type}</span>
<span>{nt.Spread()}</span>
</div>
}
</div>
</div>
}
</div>
</div>
</div>
@@ -462,16 +373,9 @@ templ Admin(data dash.AdminData) {
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-database me-2"></i>Volume Servers
</h6>
<div class="dropdown no-arrow">
<a class="dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">
<i class="bi bi-three-dots-vertical icon-sm icon-fixed-width text-gray-400"></i>
</a>
<div class="dropdown-menu dropdown-menu-right shadow animated--fade-in">
<div class="dropdown-header">Actions:</div>
<a class="dropdown-item" href={ dash.PUrl(ctx, "/volumes") }>View Details</a>
<a class="dropdown-item" href={ dash.PUrl(ctx, "/cluster") }>Topology View</a>
</div>
</div>
<a href={ dash.PUrl(ctx, "/cluster/volume-servers") } class="btn btn-sm btn-outline-primary">
View all <i class="bi bi-arrow-right ms-1"></i>
</a>
</div>
<div class="card-body">
<div class="table-responsive">
@@ -518,7 +422,7 @@ templ Admin(data dash.AdminData) {
<small class="text-muted">({fmt.Sprintf("%d vol", vs.EcVolumes)})</small>
}
} else {
<span class="text-muted">-</span>
<span class="text-muted">–</span>
}
</td>
<td>
@@ -554,16 +458,9 @@ templ Admin(data dash.AdminData) {
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-folder me-2"></i>Filer Nodes
</h6>
<div class="dropdown no-arrow">
<a class="dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">
<i class="bi bi-three-dots-vertical icon-sm icon-fixed-width text-gray-400"></i>
</a>
<div class="dropdown-menu dropdown-menu-right shadow animated--fade-in">
<div class="dropdown-header">Actions:</div>
<a class="dropdown-item" href={ dash.PUrl(ctx, "/filer") }>File Browser</a>
<a class="dropdown-item" href={ dash.PUrl(ctx, "/cluster") }>Topology View</a>
</div>
</div>
<a href={ dash.PUrl(ctx, "/cluster/filers") } class="btn btn-sm btn-outline-primary">
View all <i class="bi bi-arrow-right ms-1"></i>
</a>
</div>
<div class="card-body">
<div class="table-responsive">
@@ -616,15 +513,9 @@ templ Admin(data dash.AdminData) {
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-cloud me-2"></i>S3 Servers
</h6>
<div class="dropdown no-arrow">
<a class="dropdown-toggle" href="#" role="button" data-bs-toggle="dropdown">
<i class="bi bi-three-dots-vertical icon-sm icon-fixed-width text-gray-400"></i>
</a>
<div class="dropdown-menu dropdown-menu-right shadow animated--fade-in">
<div class="dropdown-header">Actions:</div>
<a class="dropdown-item" href={ dash.PUrl(ctx, "/cluster/s3") }>View Details</a>
</div>
</div>
<a href={ dash.PUrl(ctx, "/cluster/s3") } class="btn btn-sm btn-outline-primary">
View all <i class="bi bi-arrow-right ms-1"></i>
</a>
</div>
<div class="card-body">
<div class="table-responsive">
@@ -662,6 +553,52 @@ templ Admin(data dash.AdminData) {
</div>
</div>
<!-- Vector Servers -->
<div class="row mb-4">
<div class="col-12">
<div class="card shadow mb-4">
<div class="card-header py-3 d-flex flex-row align-items-center justify-content-between">
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-arrow-up-right me-2"></i>Vector Servers
</h6>
<a href={ dash.PUrl(ctx, "/cluster/vector-servers") } class="btn btn-sm btn-outline-primary">
View all <i class="bi bi-arrow-right ms-1"></i>
</a>
</div>
<div class="card-body">
<div class="table-responsive">
<table class="table table-hover" width="100%" cellspacing="0">
<thead>
<tr>
<th>Address</th>
<th>Version</th>
<th>Status</th>
</tr>
</thead>
<tbody>
for _, vectorNode := range data.VectorNodes {
<tr>
<td>{vectorNode.Address}</td>
<td title={vectorNode.Version}>{dash.ShortVersion(vectorNode.Version)}</td>
<td>{vectorNode.Status}</td>
</tr>
}
if len(data.VectorNodes) == 0 {
<tr>
<td colspan="3" class="text-center text-muted py-4">
<i class="bi bi-info-circle me-2"></i>
No vector servers found
</td>
</tr>
}
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
<!-- Last Updated -->
<div class="row">
<div class="col-12">
+65 -1
View File
@@ -23,7 +23,7 @@ func TestAdminDashboardShowsDiskUsage(t *testing.T) {
t.Fatalf("render: %v", err)
}
got := buf.String()
for _, want := range []string{"Disk Usage", "348.0 TB", "of 1000.0 TB (34%)"} {
for _, want := range []string{"Disk usage", "348.0 TB", "of 1000.0 TB (34%)"} {
if !strings.Contains(got, want) {
t.Errorf("dashboard is missing %q", want)
}
@@ -33,9 +33,73 @@ func TestAdminDashboardShowsDiskUsage(t *testing.T) {
}
}
func TestS3BucketModalsEscapeDynamicHTMLText(t *testing.T) {
var buf bytes.Buffer
if err := S3Buckets(dash.S3BucketsData{}).Render(context.Background(), &buf); err != nil {
t.Fatalf("render bucket page: %v", err)
}
for _, want := range []string{
"Bucket Details - ' + escapeHtml(bucketName)",
"Lifecycle - ' + escapeHtml(bucketName)",
"Error loading bucket details: ' + escapeHtml(data.error)",
"Error loading bucket details: ' + escapeHtml(error.message)",
"Error loading lifecycle rules: ' + escapeHtml(data.error || 'unknown error')",
"Error loading lifecycle rules: ' + escapeHtml(error.message)",
} {
if !strings.Contains(buf.String(), want) {
t.Errorf("rendered bucket page is missing escaped text expression %q", want)
}
}
}
func TestCalculatePercent64PetabyteScale(t *testing.T) {
const pib = int64(1) << 50
if got := calculatePercent64(100*pib, 200*pib); got != 50 {
t.Errorf("100 of 200 PiB = %d%%, want 50%%", got)
}
}
func TestClusterHealthSummary(t *testing.T) {
healthy := dash.AdminData{
MasterNodes: []dash.MasterNode{{Address: "m1:9333", IsLeader: true}},
VolumeServers: []dash.VolumeServer{{ID: "v1"}, {ID: "v2"}},
FilerNodes: []dash.FilerNode{{Address: "f1:8888"}},
DiskUsage: 50,
DiskCapacity: 100,
}
if got, want := clusterHealthSummary(healthy), "Healthy. 2 volume servers and 1 filer reporting, 1 master configured."; got != want {
t.Errorf("healthy summary = %q, want %q", got, want)
}
degraded := healthy
degraded.MasterNodes = []dash.MasterNode{{Address: "m1:9333"}}
degraded.DiskUsage = 95
want := "Needs attention: no master leader, disks 95% full. 2 volume servers and 1 filer reporting, 1 master configured."
if got := clusterHealthSummary(degraded); got != want {
t.Errorf("degraded summary = %q, want %q", got, want)
}
if got := clusterHealthIssues(dash.AdminData{}); strings.Join(got, ", ") != "no master reachable, no volume servers" {
t.Errorf("empty cluster issues = %q", got)
}
}
func TestAdminDashboardListsVectorServers(t *testing.T) {
var out bytes.Buffer
data := dash.AdminData{VectorNodes: []dash.VectorNode{{Address: "10.0.0.6:18888", Version: "0.1.0", Status: "Ready"}}}
if err := Admin(data).Render(context.Background(), &out); err != nil {
t.Fatal(err)
}
page := out.String()
if !strings.Contains(page, "Vector servers") || !strings.Contains(page, "<td>10.0.0.6:18888</td>") || strings.Contains(page, "No vector servers found") {
t.Fatal("the dashboard does not list the vector engine")
}
out.Reset()
if err := Admin(dash.AdminData{}).Render(context.Background(), &out); err != nil {
t.Fatal(err)
}
if !strings.Contains(out.String(), "No vector servers found") {
t.Fatal("no empty state without vector servers")
}
}
File diff suppressed because it is too large. Load diff
@@ -45,11 +45,6 @@ templ ClusterBrokers(data dash.ClusterBrokersData) {
<!-- Brokers Table -->
<div class="card shadow mb-4">
<div class="card-header py-3">
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-chat-dots me-2"></i>Message Brokers
</h6>
</div>
<div class="card-body">
if len(data.Brokers) > 0 {
<div class="table-responsive">
+7 -7
View File
@@ -44,7 +44,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</div></div><div class=\"col-auto\"><i class=\"bi bi-chat-dots icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Brokers Table --><div class=\"card shadow mb-4\"><div class=\"card-header py-3\"><h6 class=\"m-0 font-weight-bold text-primary\"><i class=\"bi bi-chat-dots me-2\"></i>Message Brokers</h6></div><div class=\"card-body\">")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</div></div><div class=\"col-auto\"><i class=\"bi bi-chat-dots icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Brokers Table --><div class=\"card shadow mb-4\"><div class=\"card-body\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -61,7 +61,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(broker.Address)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 70, Col: 27}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 65, Col: 27}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
@@ -74,7 +74,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(broker.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 73, Col: 66}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 68, Col: 66}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
@@ -87,7 +87,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var5 string
templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(broker.DataCenter)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 76, Col: 69}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 71, Col: 69}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5))
if templ_7745c5c3_Err != nil {
@@ -100,7 +100,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var6 string
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs(broker.Rack)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 79, Col: 63}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 74, Col: 63}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
if templ_7745c5c3_Err != nil {
@@ -114,7 +114,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var7 string
templ_7745c5c3_Var7, templ_7745c5c3_Err = templ.JoinStringErrs(broker.CreatedAt.Format("2006-01-02 15:04:05"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 83, Col: 60}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 78, Col: 60}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var7))
if templ_7745c5c3_Err != nil {
@@ -148,7 +148,7 @@ func ClusterBrokers(data dash.ClusterBrokersData) templ.Component {
var templ_7745c5c3_Var8 string
templ_7745c5c3_Var8, templ_7745c5c3_Err = templ.JoinStringErrs(data.LastUpdated.Format("2006-01-02 15:04:05"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 108, Col: 67}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_brokers.templ`, Line: 103, Col: 67}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var8))
if templ_7745c5c3_Err != nil {
-5
View File
@@ -45,11 +45,6 @@ templ ClusterFilers(data dash.ClusterFilersData) {
<!-- Filers Table -->
<div class="card shadow mb-4">
<div class="card-header py-3">
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-folder2-open me-2"></i>Filers
</h6>
</div>
<div class="card-body">
if len(data.Filers) > 0 {
<div class="table-responsive">
+9 -9
View File
@@ -44,7 +44,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</div></div><div class=\"col-auto\"><i class=\"bi bi-folder2-open icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Filers Table --><div class=\"card shadow mb-4\"><div class=\"card-header py-3\"><h6 class=\"m-0 font-weight-bold text-primary\"><i class=\"bi bi-folder2-open me-2\"></i>Filers</h6></div><div class=\"card-body\">")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</div></div><div class=\"col-auto\"><i class=\"bi bi-folder2-open icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Filers Table --><div class=\"card shadow mb-4\"><div class=\"card-body\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -61,7 +61,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templruntime.NewAttributeExpression(templ.SafeURL(fmt.Sprintf("//%s", filer.Address))).ResolveURL(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 71, Col: 70}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 66, Col: 70}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var3)
if templ_7745c5c3_Err != nil {
@@ -74,7 +74,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(filer.Address)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 72, Col: 27}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 67, Col: 27}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
@@ -87,7 +87,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var5 string
templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(filer.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 77, Col: 65}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 72, Col: 65}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5))
if templ_7745c5c3_Err != nil {
@@ -100,7 +100,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var6 string
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs(filer.DataCenter)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 80, Col: 68}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 75, Col: 68}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
if templ_7745c5c3_Err != nil {
@@ -113,7 +113,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var7 string
templ_7745c5c3_Var7, templ_7745c5c3_Err = templ.JoinStringErrs(filer.Rack)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 83, Col: 62}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 78, Col: 62}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var7))
if templ_7745c5c3_Err != nil {
@@ -127,7 +127,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var8 string
templ_7745c5c3_Var8, templ_7745c5c3_Err = templ.JoinStringErrs(filer.CreatedAt.Format("2006-01-02 15:04:05"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 87, Col: 59}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 82, Col: 59}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var8))
if templ_7745c5c3_Err != nil {
@@ -146,7 +146,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var9 string
templ_7745c5c3_Var9, templ_7745c5c3_Err = templruntime.NewAttributeExpression(filer.Address).Resolve(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 94, Col: 149}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 89, Col: 149}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var9)
if templ_7745c5c3_Err != nil {
@@ -174,7 +174,7 @@ func ClusterFilers(data dash.ClusterFilersData) templ.Component {
var templ_7745c5c3_Var10 string
templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs(data.LastUpdated.Format("2006-01-02 15:04:05"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 119, Col: 67}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_filers.templ`, Line: 114, Col: 67}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10))
if templ_7745c5c3_Err != nil {
@@ -89,11 +89,6 @@ templ ClusterMasters(data dash.ClusterMastersData) {
<!-- Masters Table -->
<div class="card shadow mb-4">
<div class="card-header py-3">
<h6 class="m-0 font-weight-bold text-primary">
<i class="bi bi-award me-2"></i>Masters
</h6>
</div>
<div class="card-body">
if len(data.Masters) > 0 {
<div class="table-responsive">
+8 -8
View File
@@ -72,7 +72,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
return templ_7745c5c3_Err
}
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "</div></div><div class=\"col-auto\"><i class=\"bi bi-activity icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Masters Table --><div class=\"card shadow mb-4\"><div class=\"card-header py-3\"><h6 class=\"m-0 font-weight-bold text-primary\"><i class=\"bi bi-award me-2\"></i>Masters</h6></div><div class=\"card-body\">")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "</div></div><div class=\"col-auto\"><i class=\"bi bi-activity icon-2x text-gray-300\"></i></div></div></div></div></div></div><!-- Masters Table --><div class=\"card shadow mb-4\"><div class=\"card-body\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
@@ -89,7 +89,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templruntime.NewAttributeExpression(templ.SafeURL(fmt.Sprintf("//%s", master.Address))).ResolveURL(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 113, Col: 71}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 108, Col: 71}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var4)
if templ_7745c5c3_Err != nil {
@@ -102,7 +102,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var5 string
templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(master.Address)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 114, Col: 28}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 109, Col: 28}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5))
if templ_7745c5c3_Err != nil {
@@ -135,7 +135,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var6 string
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs(master.Suffrage)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 132, Col: 30}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 127, Col: 30}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
if templ_7745c5c3_Err != nil {
@@ -158,7 +158,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var7 string
templ_7745c5c3_Var7, templ_7745c5c3_Err = templruntime.NewAttributeExpression(master.Address).Resolve(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 143, Col: 41}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 138, Col: 41}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var7)
if templ_7745c5c3_Err != nil {
@@ -171,7 +171,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var8 string
templ_7745c5c3_Var8, templ_7745c5c3_Err = templruntime.NewAttributeExpression(fmt.Sprintf("%t", master.IsLeader)).Resolve(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 144, Col: 60}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 139, Col: 60}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var8)
if templ_7745c5c3_Err != nil {
@@ -184,7 +184,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var9 string
templ_7745c5c3_Var9, templ_7745c5c3_Err = templruntime.NewAttributeExpression(master.Suffrage).Resolve(ctx)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 145, Col: 43}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 140, Col: 43}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var9)
if templ_7745c5c3_Err != nil {
@@ -212,7 +212,7 @@ func ClusterMasters(data dash.ClusterMastersData) templ.Component {
var templ_7745c5c3_Var10 string
templ_7745c5c3_Var10, templ_7745c5c3_Err = templ.JoinStringErrs(data.LastUpdated.Format("2006-01-02 15:04:05"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 169, Col: 67}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `view/app/cluster_masters.templ`, Line: 164, Col: 67}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var10))
if templ_7745c5c3_Err != nil {
Loaded 100 of 507 files, more files were not shown because too many files have changed in this diff. Show more