Commit Graph
8 Commits
Author SHA1 Message Date
Parviz Miriyev f407bdaa36 fix(admin): use TLS-aware HTTP client for /dir/status fetch (#9227)
fetchPublicUrlMap() in weed/admin/dash/cluster_topology.go uses a
dedicated &http.Client{} that doesn't honor security.toml client TLS
configuration, and hardcodes "http://" in the URL. When master is
configured HTTPS-only ([https.master] set), every cluster topology
cache refresh logs:

  NOTICE: http: TLS handshake error from <admin-ip>:<port>: client
  sent an HTTP request to an HTTPS server

The function falls through to glog.V(1).Infof and returns nil, so the
admin UI loses PublicUrl enrichment for data nodes. Cosmetic but noisy.

Switch to util_http.GetGlobalHttpClient() whose Do() calls
NormalizeHttpScheme(), which automatically rewrites http:// to https://
when [https.client] is enabled and presents the configured client cert.
Preserve the 5-second timeout via context.WithTimeout().

Same pattern as weed/admin/handlers/file_browser_handlers.go,
weed/server/master_server.go, weed/shell/command_volume_fsck.go.
2026-04-26 12:25:53 -07:00
Chris Lu 46b801aedb fix(admin): list all masters and dedupe EC file counts in dashboard (#9093)
* fix(admin): list all masters and dedupe EC file counts in dashboard

Dashboard -> Master Nodes only ever showed the currently connected master
because getMasterNodesStatus hard-coded a single entry. Replace it with a
RaftListClusterServers call that returns every master in the raft group and
tags the real leader, falling back to the current master only if the raft
call fails.

Buckets -> Object Store Buckets could render 0 objects for a bucket backed
by an EC volume. Every shard holder reports the same whole-volume
file_count (read from the replicated .ecx), so the first-seen value wins;
if that first node had not yet finished loading .ecx it reported 0 and
pinned the aggregate at 0. Take the max across reporting nodes instead.

The dashboard header total_files also dropped after volumes were converted
to erasure coding because getTopologyViaGRPC never folded EC file_count
into topology.TotalFiles. Aggregate it with the same max/sum dedupe.

* fix(admin): address PR review comments

- bound RaftListClusterServers with a 3s timeout so the dashboard endpoint
  cannot hang on a stalled master
- pre-validate raft addresses with net.SplitHostPort before calling
  pb.GrpcAddressToServerAddress, which otherwise glog.Fatalf's on a
  malformed entry and would crash the admin process
- when raft is unreachable, mark the fallback master as not-leader rather
  than claiming leadership the code cannot verify
- warn when summed EC delete_count exceeds file_count while folding into
  topology.TotalFiles, matching collectCollectionStats

* fix(admin): distinguish empty raft response from RPC failure

When RaftListClusterServers returns successfully with no servers, raft is
not initialized (standalone/non-raft cluster), so the single fallback
master is the leader. Only treat the fallback as a non-leader when the
RPC actually failed.

* fix(admin): remove misleading Objects column from S3 buckets page

The bucket "Objects" column displayed needle counts from volume
collection stats, not actual S3 object counts. This is confusing
because a single S3 object can span multiple needles (multipart
uploads, versions) and the count is inaccurate for EC volumes.

Remove the ObjectCount field from S3Bucket, the Objects table column,
the sort-by-objects handler, the detail-view row, and both CSV export
references.

* fix(admin): correct cell indexes in fallback bucket CSV export

After the Objects column was removed, the fallback CSV exporter in
admin.js still used stale cell indexes: cells[1] mapped to Owner
(not Created), cells[2] to Created (not Size), cells[3] to Logical
Size (not Quota). Align all indexes with the current table column
order and include Owner, Logical Size, and Physical Size.
2026-04-15 22:28:54 -07:00
1f1eac4f08 feat: improve aio support for admin/volume ingress and fix UI links (#8679)
* feat: improve allInOne mode support for admin/volume ingress and fix master UI links

- Add allInOne support to admin ingress template, matching the pattern
  used by filer and s3 ingress templates (or-based enablement with
  ternary service name selection)
- Add allInOne support to volume ingress template, which previously
  required volume.enabled even when the volume server runs within the
  allInOne pod
- Expose admin ports in allInOne deployment and service when
  allInOne.admin.enabled is set
- Add allInOne.admin config section to values.yaml (enabled by default,
  ports inherit from admin.*)
- Fix legacy master UI templates (master.html, masterNewRaft.html) to
  prefer PublicUrl over internal Url when linking to volume server UI.
  The new admin UI already handles this correctly.

* fix: revert admin allInOne changes and fix PublicUrl in admin dashboard

The admin binary (`weed admin`) is a separate process that cannot run
inside `weed server` (allInOne mode). Revert the admin-related allInOne
helm chart changes that caused 503 errors on admin ingress.

Fix bug in cluster_topology.go where VolumeServer.PublicURL was set to
node.Id (internal pod address) instead of the actual public URL. Add
public_url field to DataNodeInfo proto message so the topology gRPC
response carries the public URL set via -volume.publicUrl flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use HTTP /dir/status to populate PublicUrl in admin dashboard

The gRPC DataNodeInfo proto does not include PublicUrl, so the admin dashboard showed internal pod IPs instead of the configured public URL.
Fetch PublicUrl from the master's /dir/status HTTP endpoint and apply it
in both GetClusterTopology and GetClusterVolumeServers code paths.

Also reverts the unnecessary proto field additions from the previous
commit and cleans up a stray blank line in all-in-one-service.yml.

* fix: apply PublicUrl link fix to masterNewRaft.html

Match the same conditional logic already applied to master.html —
prefer PublicUrl when set and different from Url.

* fix: add HTTP timeout and status check to fetchPublicUrlMap

Use a 5s-timeout client instead of http.DefaultClient to prevent
blocking indefinitely when the master is unresponsive. Also check
the HTTP status code before attempting to parse the response body.

* fix: fall back to node address when PublicUrl is empty

Prevents blank links in the admin dashboard when PublicUrl is not
configured, such as in standalone or mixed-version clusters.

* fix: log io.ReadAll error in fetchPublicUrlMap

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-03-18 13:20:55 -07:00
Chris Lu 24556ebdcc Refine Bucket Size Metrics: Logical and Physical Size (#7943)
* refactor: implement logical size calculation with replication factor using dedicated helper

* ui: update bucket list to show logical/physical size
2026-01-02 18:28:00 -08:00
9d013ea9b8 Admin UI: include ec shard sizes into volume server info (#7071)
* show ec shards on dashboard, show max in its own column

* master collect shard size info

* master send shard size via VolumeList

* change to more efficient shard sizes slice

* include ec shard sizes into volume server info

* Eliminated Redundant gRPC Calls

* much more efficient

* Efficient Counting: bits.OnesCount32() uses CPU-optimized instructions to count set bits in O(1)

* avoid extra volume list call

* simplify

* preserve existing shard sizes

* avoid hard coded value

* Update weed/storage/erasure_coding/ec_volume_info.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/admin/dash/volume_management.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update ec_volume_info.go

* address comments

* avoid duplicated functions

* Update weed/admin/dash/volume_management.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* simplify

* refactoring

* fix compilation

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-02 02:16:49 -07:00
Chris Lu 69553e5ba6 convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
chrislu 64c5dde2f3 support multiple masters
fix https://github.com/seaweedfs/seaweedfs/issues/6988
2025-07-15 10:51:07 -07:00
chrislu e85fbd29a1 refactor 2025-07-04 13:33:58 -07:00