Commit Graph
33 Commits
Author SHA1 Message Date
Chris Lu 97a155d14d admin: show capacity per storage tier and stop counting remote-tiered bytes as local disk usage (#10766)
* admin: show capacity per storage tier and stop counting remote-tiered bytes as local disk usage

A remote-tiered volume reports its cloud object's size, so summing volume
sizes inflated the dashboard's used-vs-capacity numbers (the local .dat is
gone after volume.tier.move). Split the accounting: DiskUsage now only
counts bytes on local disks, with the cloud bytes surfaced separately per
server and per remote storage name.

The dashboard gains a Storage Tiers table breaking volumes and EC shards
down by tier (each local disk type plus each remote storage), using the
per-disk-type statfs numbers already in the VolumeList response. The
volumes page badges remote-tiered volumes with their storage name, and
the EC shards page fills in real per-shard sizes instead of hardcoding 0.

* admin: review fixes for the tier capacity display

- A disk that predates disk_total_bytes now contributes its logical
  bytes to the tier's DiskUsed, so a tier mixing old and new volume
  servers doesn't underreport usage; the usage bar always reflects the
  displayed Disk Used value (the DataSize fallback in UsagePercent is
  gone, and the percent math is overflow-safe).
- getTopologyViaGRPC defaults a zero VolumeSizeLimitMb to 30000 MB like
  GetClusterVolumeServers, keeping slot-based capacities consistent.
- The dashboard volume-servers column reads Usage / Capacity to match
  its cell content, and the hdd disk-type default is shared between the
  volumes-page badge and countUniqueDiskTypes.
2026-08-15 12:35:16 -07:00
ruriandChris Lu 340f9951ac admin: make paths relative (#10709)
* admin: make paths relative

* admin: make filer browser link and nav path checks prefix-relative

* admin: add isCurrentPath and currentPathStartsWith helpers

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-08-10 22:31:55 -07:00
Chris Lu 0cf62a921a admin: dashboard counts chunks, not files (#10598)
* admin: count each chunk once in the dashboard total

The dashboard summed file_count from every node's volume list, so a chunk
was counted once per replica and deleted chunks were never subtracted.
Reuse the collection aggregation, which dedupes replicas and EC shard
holders and nets out tombstones.

* admin: the dashboard card counts chunks, so name it that

Volumes store chunks, and a file is split into one or more of them, so
the 'Total Files' card always read far higher than the number of files in
the filer. Rename it to 'Total Chunks' and say so in the tooltip.

* admin: collections pages count chunks once and say so

The collections list and detail pages summed file_count straight off the
topology, so replicas multiplied the count, tombstones stayed in it, and
the detail page ignored EC volumes entirely. Take the numbers from the
shared collection aggregation and label them chunks.

* admin: dedupe replica chunk counts per volume instead of dividing

Dividing each replica's live count by the copy count truncated a chunk
per odd-sized volume, and reported half the count while a volume's
second replica had not checked in yet. Replicas mirror each other's
needles and deletes, so keep the fullest report per volume id.

* admin: fix the collections CSV export column mapping

The exporter read chunks from the EC-volume cell and shifted size and
disk types with it. Read every column the table actually has.
2026-08-06 11:22:06 -07:00
Chris Lu 7df43ad9b5 admin: add connected Mount Clients page and dashboard section (#9968)
* admin: add connected mount clients page and dashboard section

The filer is the authority on who is subscribed to its metadata stream
(FUSE/VFS mounts, S3, peer filers, ...), but its in-memory listener
registry only tracked clientId->epoch and was not exposed.

- Enrich the filer subscriber registry with name/type/address/path/
  connected-time, populated in addClient and cleared in deleteClient so
  it reflects currently-connected clients only.
- Add a ListMetadataSubscribers filer gRPC (optional client-type filter).
- Admin server fans out to every filer, filters to mount types
  ("mount" Go weed mount, "sw-vfs" Rust VFS), and renders a new
  Cluster > Mount Clients page plus a Mount Clients dashboard section.

Read-only; no behavior change to the subscribe hot path.

* admin: address review — parallelize filer fan-out, guard nil map, robust CSV

- GetMountClients now queries filers concurrently, each under a 5s
  timeout, so a slow/unreachable filer can't stall the admin dashboard.
- Defensively initialize fs.subscribers before first write.
- Mount Clients CSV export uses a Blob with quote-escaping instead of a
  data: URI, so special characters in paths export correctly.
2026-06-14 21:44:10 -07:00
Chris Lu d47cc45b1f admin: fold dashboard sparklines into the existing cards (de-dup) (#9964)
admin: fold dashboard sparklines into the existing cards

The trend sparklines added in #9957 lived in a separate "Cluster Trends"
row that duplicated the existing summary cards (Volumes, Files, Disk Used,
EC Shards). Remove that row and instead render each sparkline inside the
matching summary card, so every headline number shows its recent trend
without duplication. The two maintenance metrics that have no existing
card — Active Tasks and Workers — now fill the previously-empty columns of
the EC row (also with sparklines).

DashboardTrends changes from a Cards slice to named per-card sparkline
SVGs (+ current values for the two maintenance cards). Drops the now-unused
trendBytes helper (disk size keeps using the existing formatBytes).
2026-06-14 14:17:43 -07:00
Chris Lu b56d155b31 admin: native at-a-glance trend sparklines on the dashboard (#9957)
* admin: native at-a-glance trend sparklines on the dashboard

Add a "Cluster Trends" row to the admin Dashboard with inline-SVG
sparklines for volumes, EC shards, disk used, files, active maintenance
tasks, and workers.

The data comes entirely from what the admin already holds — the cached
cluster topology and the in-process maintenance queue — sampled into a
small bounded ring buffer on the existing maintenance-metrics ticker
(~15 min of history). No Prometheus/Grafana dependency, no JS chart
library, no extra goroutine: the sparklines are self-contained SVG
rendered server-side via templ.

This gives basic trend visibility out of the box for clusters that don't
run Prometheus, and a quick glance next to the cluster controls; Grafana
remains the place for deep/historical dashboards.

* admin: cap trendBytes unit index to avoid out-of-bounds panic

A value >= 1 ZiB would push exp past the end of the units string and
panic on units[exp]; cap exp at the last unit (EiB).
2026-06-14 13:55:26 -07:00
Chris Lu b2127c86f4 admin: show S3 servers under Cluster (#9847)
* s3: register data center with master on startup

* admin: show S3 servers under Cluster

* admin: add S3 servers to the dashboard
2026-06-07 00:32:20 -07:00
Chris Lu 01b3e4a71c template 2026-05-17 23:12:04 -07:00
Chris Lu 14cd426cf9 templ 2026-04-29 16:08:38 -07:00
Parviz Miriyev e2f96687ff fix(admin): use protocol-relative URLs for component links so HTTPS clusters don't break clicks (#9256)
* fix(admin): use protocol-relative URLs for component links

Hardcoded http:// in admin UI templates breaks browser-initiated clicks
to master / volume / filer / EC shard / Iceberg REST URLs whenever the
target component runs HTTPS-only via security.toml [https.X] sections.
The browser sends plain HTTP to a TLS-only endpoint and gets 400
"client sent an HTTP request to an HTTPS server".

Same root pattern as #9227 (admin's own backend /dir/status fetch);
this PR is the browser-facing equivalent.

Replace fmt.Sprintf("http://%s...") with fmt.Sprintf("//%s...") and the
JS-string '<a href="http://' with '<a href="//' so the browser uses the
same scheme as the page hosting the link. Backwards compatible:
  - HTTPS-only deployments: links now work
  - HTTP-only deployments: identical behavior to before
  - Mixed: edge case, addressed by future per-component public-URL work

Affected templates (9 files), each kept in lockstep with its generated
_templ.go sibling so reviewers don't need to run templ generate:

- weed/admin/view/app/admin.templ
- weed/admin/view/app/cluster_filers.templ
- weed/admin/view/app/cluster_masters.templ (Go templ + JS modal)
- weed/admin/view/app/cluster_volume_servers.templ (Go templ + JS modal)
- weed/admin/view/app/cluster_volumes.templ
- weed/admin/view/app/ec_volume_details.templ
- weed/admin/view/app/volume_details.templ
- weed/admin/view/app/iceberg_catalog.templ
- weed/admin/view/app/s3tables_buckets.templ

17 link constructions total, +32/-32 lines.

* fix(admin): protocol-relative URLs in iceberg + s3tables JS overrides

Per Gemini code review on this PR: the JS scripts in iceberg_catalog
and s3tables_buckets templates overwrite the href attribute of the
"Open Iceberg REST" links after page load, replacing the
protocol-relative URL set by the templ render with a hardcoded
http://<host>:<port>/v1/config.

Apply the same protocol-relative fix to the JS template literals so
they don't undo the templ-side change. Browser uses the page scheme
(http or https) to fill in the protocol.

Mirrored in iceberg_catalog_templ.go and s3tables_buckets_templ.go.

* fix(admin): displayed Iceberg endpoint scheme follows page protocol

Per CodeRabbit review on this PR: the on-page guidance text in iceberg
and s3tables templates still showed a literal `http://` even after the
clickable link was switched to a protocol-relative URL. In HTTPS-only
deployments operators see `http://host:8181/v1` as the suggested
endpoint, copy it, and get a broken connection.

Wrap the scheme in <span id="iceberg-protocol"> (and the s3tables
counterpart) and have the existing inline script set its innerText to
window.location.protocol minus the trailing colon. Same pattern as the
existing dynamic host substitution. Mirrored in *_templ.go so reviewers
do not need templ generate.

SQL/JSON code-block examples (CREATE EXTERNAL TABLE ... ENDPOINT
'http://...', "uri": "http://..." ) are intentionally left as-is —
they are starter snippets users adapt to their environment, not
clickable or copy-paste-into-runtime values. Happy to follow up with
server-side scheme threading if requested.
2026-04-27 23:10:11 -07:00
Chris Lu 46b801aedb fix(admin): list all masters and dedupe EC file counts in dashboard (#9093)
* fix(admin): list all masters and dedupe EC file counts in dashboard

Dashboard -> Master Nodes only ever showed the currently connected master
because getMasterNodesStatus hard-coded a single entry. Replace it with a
RaftListClusterServers call that returns every master in the raft group and
tags the real leader, falling back to the current master only if the raft
call fails.

Buckets -> Object Store Buckets could render 0 objects for a bucket backed
by an EC volume. Every shard holder reports the same whole-volume
file_count (read from the replicated .ecx), so the first-seen value wins;
if that first node had not yet finished loading .ecx it reported 0 and
pinned the aggregate at 0. Take the max across reporting nodes instead.

The dashboard header total_files also dropped after volumes were converted
to erasure coding because getTopologyViaGRPC never folded EC file_count
into topology.TotalFiles. Aggregate it with the same max/sum dedupe.

* fix(admin): address PR review comments

- bound RaftListClusterServers with a 3s timeout so the dashboard endpoint
  cannot hang on a stalled master
- pre-validate raft addresses with net.SplitHostPort before calling
  pb.GrpcAddressToServerAddress, which otherwise glog.Fatalf's on a
  malformed entry and would crash the admin process
- when raft is unreachable, mark the fallback master as not-leader rather
  than claiming leadership the code cannot verify
- warn when summed EC delete_count exceeds file_count while folding into
  topology.TotalFiles, matching collectCollectionStats

* fix(admin): distinguish empty raft response from RPC failure

When RaftListClusterServers returns successfully with no servers, raft is
not initialized (standalone/non-raft cluster), so the single fallback
master is the leader. Only treat the fallback as a non-leader when the
RPC actually failed.

* fix(admin): remove misleading Objects column from S3 buckets page

The bucket "Objects" column displayed needle counts from volume
collection stats, not actual S3 object counts. This is confusing
because a single S3 object can span multiple needles (multipart
uploads, versions) and the count is inaccurate for EC volumes.

Remove the ObjectCount field from S3Bucket, the Objects table column,
the sort-by-objects handler, the detail-view row, and both CSV export
references.

* fix(admin): correct cell indexes in fallback bucket CSV export

After the Objects column was removed, the fallback CSV exporter in
admin.js still used stale cell indexes: cells[1] mapped to Owner
(not Created), cells[2] to Created (not Size), cells[3] to Logical
Size (not Quota). Align all indexes with the current table column
order and include Owner, Logical Size, and Physical Size.
2026-04-15 22:28:54 -07:00
Chris Lu 7c83460b10 adjust template path 2026-03-16 15:29:12 -07:00
Chris Lu e8914ac879 feat(admin): add -urlPrefix flag for subdirectory deployment (#8670)
Allow the admin server to run behind a reverse proxy under a
subdirectory by adding a -urlPrefix flag (e.g. -urlPrefix=/seaweedfs).

Closes #8646
2026-03-16 15:26:02 -07:00
Chris Lu 7b08cf74ed consistent template generation 2026-02-22 13:34:06 -08:00
Chris Lu 5a0204310c Add Iceberg admin UI (#8246)
* Add Iceberg table details view

* Enhance Iceberg catalog browsing UI

* Fix Iceberg UI security and logic issues

- Fix selectSchema() and partitionFieldsFromFullMetadata() to always search for matching IDs instead of checking != 0
- Fix snapshotsFromFullMetadata() to defensive-copy before sorting to prevent mutating caller's slice
- Fix XSS vulnerabilities in s3tables.js: replace innerHTML with textContent/createElement for user-controlled data
- Fix deleteIcebergTable() to redirect to namespace tables list on details page instead of reloading
- Fix data-bs-target in iceberg_namespaces.templ: remove templ.SafeURL for CSS selector
- Add catalogName to delete modal data attributes for proper redirect
- Remove unused hidden inputs from create table form (icebergTableBucketArn, icebergTableNamespace)

* Regenerate templ files for Iceberg UI updates

* Support complex Iceberg type objects in schema

Change Type field from string to json.RawMessage in both IcebergSchemaFieldInfo
and internal icebergSchemaField to properly handle Iceberg spec's complex type
objects (e.g. {"type": "struct", "fields": [...]}). Currently test data
only shows primitive string types, but this change makes the implementation
defensively robust for future complex types by preserving the exact JSON
representation. Add typeToString() helper and update schema extraction
functions to marshal string types as JSON. Update template to convert
json.RawMessage to string for display.

* Regenerate templ files for Type field changes

* templ

* Fix additional Iceberg UI issues from code review

- Fix lazy-load flag that was set before async operation completed, preventing retries
  on error; now sets loaded flag only after successful load and throws error to caller
  for proper error handling and UI updates
- Add zero-time guards for CreatedAt and ModifiedAt fields in table details to avoid
  displaying Go zero-time values; render dash when time is zero
- Add URL path escaping for all catalog/namespace/table names in URLs to prevent
  malformed URLs when names contain special characters like /, ?, or #
- Remove redundant innerHTML clear in loadIcebergNamespaceTables that cleared twice
  before appending the table list
- Fix selectSnapshotForMetrics to remove != 0 guard for consistency with selectSchema
  fix; now always searches for CurrentSnapshotID without zero-value gate
- Enhance typeToString() helper to display '(complex)' for non-primitive JSON types

* Regenerate templ files for Phase 3 updates

* Fix template generation to use correct file paths

Run templ generate from repo root instead of weed/admin directory to ensure
generated _templ.go files have correct absolute paths in error messages
(e.g., 'weed/admin/view/app/iceberg_table_details.templ' instead of
'app/iceberg_table_details.templ'). This ensures both 'make admin-generate'
at repo root and 'make generate' in weed/admin directory produce identical
output with consistent file path references.

* Regenerate template files with correct path references

* Validate S3 Tables names in UI

- Add client-side validation for table bucket and namespace names to surface
  errors for invalid characters (dots/underscores) before submission
- Use HTML validity messages with reportValidity for immediate feedback
- Update namespace helper text to reflect actual constraints (single-level,
  lowercase letters, numbers, and underscores)

* Regenerate templ files for namespace helper text

* Fix Iceberg catalog REST link and actions

* Disallow S3 object access on table buckets

* Validate Iceberg layout for table bucket objects

* Fix REST API link to /v1/config

* merge iceberg page with table bucket page

* Allowed Trino/Iceberg stats files in metadata validation

* fixes

  - Backend/data handling:
      - Normalized Iceberg type display and fallback handling in weed/admin/dash/s3tables_management.go.
      - Fixed snapshot fallback pointer semantics in weed/admin/dash/s3tables_management.go.
      - Added CSRF token generation/propagation/validation for namespace create/delete in:
          - weed/admin/dash/csrf.go
          - weed/admin/dash/auth_middleware.go
          - weed/admin/dash/middleware.go
          - weed/admin/dash/s3tables_management.go
          - weed/admin/view/layout/layout.templ
          - weed/admin/static/js/s3tables.js
  - UI/template fixes:
      - Zero-time guards for CreatedAt fields in:
          - weed/admin/view/app/iceberg_namespaces.templ
          - weed/admin/view/app/iceberg_tables.templ
      - Fixed invalid templ-in-script interpolation and host/port rendering in:
          - weed/admin/view/app/iceberg_catalog.templ
          - weed/admin/view/app/s3tables_buckets.templ
      - Added data-catalog-name consistency on Iceberg delete action in weed/admin/view/app/iceberg_tables.templ.
      - Updated retry wording in weed/admin/static/js/s3tables.js.
      - Regenerated all affected _templ.go files.
  - S3 API/comment follow-ups:
      - Reused cached table-bucket validator in weed/s3api/bucket_paths.go.
      - Added validation-failure debug logging in weed/s3api/s3api_object_handlers_tagging.go.
      - Added multipart path-validation design comment in weed/s3api/s3api_object_handlers_multipart.go.
  - Build tooling:
      - Fixed templ generate working directory issues in weed/admin/Makefile (watch + pattern rule).

* populate data

* test/s3tables: harden populate service checks

* admin: skip table buckets in object-store bucket list

* admin sidebar: move object store to top-level links

* admin iceberg catalog: guard zero times and escape links

* admin forms: add csrf/error handling and client-side name validation

* admin s3tables: fix namespace delete modal redeclaration

* admin: replace native confirm dialogs with modal helpers

* admin modal-alerts: remove noisy confirm usage console log

* reduce logs

* test/s3tables: use partitioned tables in trino and spark populate

* admin file browser: normalize filer ServerAddress for HTTP parsing
2026-02-08 20:06:32 -08:00
Chris Lu 72a8f598f2 Fix Maintenance Task Sorting and Refactor Log Persistence (#8199)
* fix float stepping

* do not auto refresh

* only logs when non 200 status

* fix maintenance task sorting and cleanup redundant handler logic

* Refactor log retrieval to persist to disk and fix slowness

- Move log retrieval to disk-based persistence in GetMaintenanceTaskDetail
- Implement background log fetching on task completion in worker_grpc_server.go
- Implement async background refresh for in-progress tasks
- Completely remove blocking gRPC calls from the UI path to fix 10s timeouts
- Cleanup debug logs and performance profiling code

* Ensure consistent deterministic sorting in config_persistence cleanup

* Replace magic numbers with constants and remove debug logs

- Added descriptive constants for truncation limits and timeouts in admin_server.go and worker_grpc_server.go
- Replaced magic numbers with these constants throughout the codebase
- Verified removal of stdout debug printing
- Ensured consistent truncation logic during log persistence

* Address code review feedback on history truncation and logging logic

- Fix AssignmentHistory double-serialization by copying task in GetMaintenanceTaskDetail
- Fix handleTaskCompletion logging logic (mutually exclusive success/failure logs)
- Remove unused Timeout field from LogRequestContext and sync select timeouts with constants
- Ensure AssignmentHistory is only provided in the top-level field for better JSON structure

* Implement goroutine leak protection and request deduplication

- Add request deduplication in RequestTaskLogs to prevent multiple concurrent fetches for the same task
- Implement safe cleanup in timeout handlers to avoid race conditions in pendingLogRequests map
- Add a 10s cooldown for background log refreshes in GetMaintenanceTaskDetail to prevent spamming
- Ensure all persistent log-fetching goroutines are bounded and efficiently managed

* Fix potential nil pointer panics in maintenance handlers

- Add nil checks for adminServer in ShowTaskDetail, ShowMaintenanceWorkers, and UpdateTaskConfig
- Update getMaintenanceQueueData to return a descriptive error instead of nil when adminServer is uninitialized
- Ensure internal helper methods consistently check for adminServer initialization before use

* Strictly enforce disk-only log reading

- Remove background log fetching from GetMaintenanceTaskDetail to prevent timeouts and network calls during page view
- Remove unused lastLogFetch tracking fields to clean up dead code
- Ensure logs are only updated upon task completion via handleTaskCompletion

* Refactor GetWorkerLogs to read from disk

- Update /api/maintenance/workers/:id/logs endpoint to use configPersistence.LoadTaskExecutionLogs
- Remove synchronous gRPC call RequestTaskLogs to prevent timeouts and bad gateway errors
- Ensure consistent log retrieval behavior across the application (disk-only)

* Fix timestamp parsing in log viewer

- Update task_detail.templ JS to handle both ISO 8601 strings and Unix timestamps
- Fix "Invalid time value" error when displaying logs fetched from disk
- Regenerate templates

* master: fallback to HDD if SSD volumes are full in Assign

* worker: improve EC detection logging and fix skip counters

* worker: add Sync method to TaskLogger interface

* worker: implement Sync and ensure logs are flushed before task completion

* admin: improve task log retrieval with retries and better timeouts

* admin: robust timestamp parsing in task detail view
2026-02-04 08:48:55 -08:00
Chris Lu 1c1f80358f templ 2026-02-03 00:10:43 -08:00
Chris Lu 2bb21ea276 feat: Add Iceberg REST Catalog server and admin UI (#8175)
* feat: Add Iceberg REST Catalog server

Implement Iceberg REST Catalog API on a separate port (default 8181)
that exposes S3 Tables metadata through the Apache Iceberg REST protocol.

- Add new weed/s3api/iceberg package with REST handlers
- Implement /v1/config endpoint returning catalog configuration
- Implement namespace endpoints (list/create/get/head/delete)
- Implement table endpoints (list/create/load/head/delete/update)
- Add -port.iceberg flag to S3 standalone server (s3.go)
- Add -s3.port.iceberg flag to combined server mode (server.go)
- Add -s3.port.iceberg flag to mini cluster mode (mini.go)
- Support prefix-based routing for multiple catalogs

The Iceberg REST server reuses S3 Tables metadata storage under
/table-buckets and enables DuckDB, Spark, and other Iceberg clients
to connect to SeaweedFS as a catalog.

* feat: Add Iceberg Catalog pages to admin UI

Add admin UI pages to browse Iceberg catalogs, namespaces, and tables.

- Add Iceberg Catalog menu item under Object Store navigation
- Create iceberg_catalog.templ showing catalog overview with REST info
- Create iceberg_namespaces.templ listing namespaces in a catalog
- Create iceberg_tables.templ listing tables in a namespace
- Add handlers and routes in admin_handlers.go
- Add Iceberg data provider methods in s3tables_management.go
- Add Iceberg data types in types.go

The Iceberg Catalog pages provide visibility into the same S3 Tables
data through an Iceberg-centric lens, including REST endpoint examples
for DuckDB and PyIceberg.

* test: Add Iceberg catalog integration tests and reorg s3tables tests

- Reorganize existing s3tables tests to test/s3tables/table-buckets/
- Add new test/s3tables/catalog/ for Iceberg REST catalog tests
- Add TestIcebergConfig to verify /v1/config endpoint
- Add TestIcebergNamespaces to verify namespace listing
- Add TestDuckDBIntegration for DuckDB connectivity (requires Docker)
- Update CI workflow to use new test paths

* fix: Generate proper random UUIDs for Iceberg tables

Address code review feedback:
- Replace placeholder UUID with crypto/rand-based UUID v4 generation
- Add detailed TODO comments for handleUpdateTable stub explaining
  the required atomic metadata swap implementation

* fix: Serve Iceberg on localhost listener when binding to different interface

Address code review feedback: properly serve the localhost listener
when the Iceberg server is bound to a non-localhost interface.

* ci: Add Iceberg catalog integration tests to CI

Add new job to run Iceberg catalog tests in CI, along with:
- Iceberg package build verification
- Iceberg unit tests
- Iceberg go vet checks
- Iceberg format checks

* fix: Address code review feedback for Iceberg implementation

- fix: Replace hardcoded account ID with s3_constants.AccountAdminId in buildTableBucketARN()
- fix: Improve UUID generation error handling with deterministic fallback (timestamp + PID + counter)
- fix: Update handleUpdateTable to return HTTP 501 Not Implemented instead of fake success
- fix: Better error handling in handleNamespaceExists to distinguish 404 from 500 errors
- fix: Use relative URL in template instead of hardcoded localhost:8181
- fix: Add HTTP timeout to test's waitForService function to avoid hangs
- fix: Use dynamic ephemeral ports in integration tests to avoid flaky parallel failures
- fix: Add Iceberg port to final port configuration logging in mini.go

* fix: Address critical issues in Iceberg implementation

- fix: Cache table UUIDs to ensure persistence across LoadTable calls
  The UUID now remains stable for the lifetime of the server session.
  TODO: For production, UUIDs should be persisted in S3 Tables metadata.

- fix: Remove redundant URL-encoded namespace parsing
  mux router already decodes %1F to \x1F before passing to handlers.
  Redundant ReplaceAll call could cause bugs with literal %1F in namespace.

* fix: Improve test robustness and reduce code duplication

- fix: Make DuckDB test more robust by failing on unexpected errors
  Instead of silently logging errors, now explicitly check for expected
  conditions (extension not available) and skip the test appropriately.

- fix: Extract username helper method to reduce duplication
  Created getUsername() helper in AdminHandlers to avoid duplicating
  the username retrieval logic across Iceberg page handlers.

* fix: Add mutex protection to table UUID cache

Protects concurrent access to the tableUUIDs map with sync.RWMutex.
Uses read-lock for fast path when UUID already cached, and write-lock
for generating new UUIDs. Includes double-check pattern to handle race
condition between read-unlock and write-lock.

* style: fix go fmt errors

* feat(iceberg): persist table UUID in S3 Tables metadata

* feat(admin): configure Iceberg port in Admin UI and commands

* refactor: address review comments (flags, tests, handlers)

- command/mini: fix tracking of explicit s3.port.iceberg flag
- command/admin: add explicit -iceberg.port flag
- admin/handlers: reuse getUsername helper
- tests: use 127.0.0.1 for ephemeral ports and os.Stat for file size check

* test: check error from FileStat in verify_gc_empty_test
2026-02-02 23:12:13 -08:00
Chris Lu c5b53397c6 templ 2026-01-26 11:38:18 -08:00
Chris Lu 5ba0db7af4 Merge branch 'origin/master' into master
Resolved merge conflicts in:
- weed/admin/static/js/modal-alerts.js: Adopted incoming improvements and HTML support.
- weed/admin/view/app/collection_details.templ: Switched to showAlert info type.
- weed/admin/view/app/file_browser.templ: Used descriptive delete message.
- weed/admin/view/app/maintenance_workers.templ: Used encoding and headers in pauseWorker.
- weed/admin/view/app/object_store_users.templ: Restored accidentally deleted delete functions and used encodeURIComponent.
- weed/admin/view/app/policies.templ: Standardized on showAlert and descriptive confirmations.

Regenerated all templ files.
2026-01-26 11:37:12 -08:00
Yavor Konstantinovandchrislu 1d0471aebb Improve admin urls (#7370)
* Improve Master and Volume URLs in admin dashboard

- Add clickable URL for master node.
- Refactor Volume server URL to use PublicURL if set. 'address' is used
  as fallback.

* Make volume servers show in consistent order

- Sort servers by name to ensure predictable order after each refresh.

* address comment

---------

Co-authored-by: chrislu <chris.lu@gmail.com>
2025-10-24 01:38:01 -07:00
Chris Lu 479e7bc38b go install github.com/a-h/templ/cmd/templ@latest 2025-10-20 22:22:12 -07:00
25bbf4c3d4 Admin UI: Fetch task logs (#7114)
* show task details

* loading tasks

* task UI works

* generic rendering

* rendering the export link

* removing placementConflicts from task parameters

* remove TaskSourceLocation

* remove "Server ID" column

* rendering balance task source

* sources and targets

* fix ec task generation

* move info

* render timeline

* simplified worker id

* simplify

* read task logs from worker

* isValidTaskID

* address comments

* Update weed/worker/tasks/balance/execution.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/erasure_coding/ec_task.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/worker/tasks/task_log_handler.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix shard ids

* plan distributing shard id

* rendering planned shards in task details

* remove Conflicts

* worker logs correctly

* pass in dc and rack

* task logging

* Update weed/admin/maintenance/maintenance_queue.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* display log details

* logs have fields now

* sort field keys

* fix link

* fix collection filtering

* avoid hard coded ec shard counts

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-09 21:47:29 -07:00
9d013ea9b8 Admin UI: include ec shard sizes into volume server info (#7071)
* show ec shards on dashboard, show max in its own column

* master collect shard size info

* master send shard size via VolumeList

* change to more efficient shard sizes slice

* include ec shard sizes into volume server info

* Eliminated Redundant gRPC Calls

* much more efficient

* Efficient Counting: bits.OnesCount32() uses CPU-optimized instructions to count set bits in O(1)

* avoid extra volume list call

* simplify

* preserve existing shard sizes

* avoid hard coded value

* Update weed/storage/erasure_coding/ec_volume_info.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/admin/dash/volume_management.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update ec_volume_info.go

* address comments

* avoid duplicated functions

* Update weed/admin/dash/volume_management.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* simplify

* refactoring

* fix compilation

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-02 02:16:49 -07:00
chrislu 49d43003e1 show volume size limit on dashboard 2025-07-11 19:37:09 -07:00
chrislu 4fcbdc1f61 tweaking dashboard UI 2025-07-11 13:11:39 -07:00
chrislu 3d4a9bdac0 upgrade templ version from v0.3.833 to v0.3.906
// templ: version: v0.3.833
// templ: version: v0.3.906

fix https://github.com/seaweedfs/seaweedfs/issues/6966#issuecomment-3063449163
2025-07-11 13:03:04 -07:00
Chris Lu 51543bbb87 Admin UI: Add message queue to admin UI (#6958)
* add a menu item "Message Queue"

* add a menu item "Message Queue"
  * move the "brokers" link under it.
  * add "topics", "subscribers". Add pages for them.

* refactor

* show topic details

* admin display publisher and subscriber info

* remove publisher and subscribers from the topic row pull down

* collecting more stats from publishers and subscribers

* fix layout

* fix publisher name

* add local listeners for mq broker and agent

* render consumer group offsets

* remove subscribers from left menu

* topic with retention

* support editing topic retention

* show retention when listing topics

* create bucket

* Update s3_buckets_templ.go

* embed the static assets into the binary

fix https://github.com/seaweedfs/seaweedfs/issues/6964
2025-07-11 10:19:27 -07:00
chrislu d8da465cd3 fix link to volume server; display volume space usage 2025-07-04 13:52:28 -07:00
chrislu 2cfe079a1f weed admin: remove system health status 2025-07-02 23:57:36 -07:00
chrislu ebb16f474c remove status fields 2025-07-02 23:17:27 -07:00
chrislu 5c2b2e5513 clean up s3 bucket references 2025-07-01 08:41:53 -07:00
Chris LuandCopilot 1defee3d68 Add admin component (#6928)
* init version

* relocate

* add s3 bucket link

* refactor handlers into weed/admin folder

* fix login logout

* adding favicon

* remove fall back to http get topology

* grpc dial option, disk total capacity

* show filer count

* fix each volume disk usage

* add filers to dashboard

* adding hosts, volumes, collections

* refactor code and menu

* remove "refresh" button

* fix data for collections

* rename cluster hosts into volume servers

* add masters, filers

* reorder

* adding file browser

* create folder and upload files

* add filer version, created at time

* remove mock data

* remove fields

* fix submenu item highlighting

* fix bucket creation

* purge files

* delete multiple

* fix bucket creation

* remove region from buckets

* add object store with buckets and users

* rendering permission

* refactor

* get bucket objects and size

* link to file browser

* add file size and count for collections page

* paginate the volumes

* fix possible SSRF

https://github.com/seaweedfs/seaweedfs/pull/6928/checks?check_run_id=45108469801

* Update weed/command/admin.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/command/admin.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix build

* import

* remove filer CLI option

* remove filer option

* remove CLI options

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-01 01:28:09 -07:00