Files
seaweedfs/weed/admin/handlers/file_browser_handlers.go
T
Chris LuandClaude Opus 5.5 2ec0623073 filer UI, admin: upload a folder (#12371)
* admin: place uploaded files by their path inside an uploaded folder

The multipart file name keeps only the base name, so the upload API takes
an optional relativePaths field listing each file's path under the target
directory. Paths that are empty, absolute or contain . or .. components
are rejected per file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: upload a folder from the file browser

The upload modal gets a folder picker, and a folder dropped on the listing
is walked into, so its files upload with their subfolder paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer UI: upload a folder

An Upload Folder button picks a whole folder, and a folder dropped on the
listing is walked into. Each file is posted to its subfolder under the
current directory, which the filer creates on the first upload into it.
Progress rows are keyed by that path, so equal names in different
subfolders no longer share a row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: reject an empty relativePaths field instead of ignoring it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: upload a large selection in batches below the multipart part limit

Go's multipart parser rejects a form of more than 1000 parts, so a folder
of about a thousand files failed as a whole. The upload modal now sends
500 files per request and sums the results; progress covers all batches,
and the button stays disabled until the last one finishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: escape file names in upload error messages

A failed upload reports the client-supplied file or folder path, and the
toast renders its message as HTML.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: list failed uploads in the modal and reload the listing when it closes

A partial upload logged its failures only to the console before the page
reloaded, and a batch that failed after earlier ones succeeded left the
listing stale. The failures now stay in the open modal, and closing it
reloads the listing once any file was uploaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: cancel the upload when its modal closes

Closing the modal, including with its Cancel button, left the batch loop
running, so the remaining batches were still written. The batch in flight
is now aborted, which also stops the loop, and the listing reloads since
the aborted request may already have stored files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: count upload progress by files when the selection has no bytes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: keep the newest upload selection when a folder drop finishes late

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: cap the upload preview at a fixed number of rows

A folder of many thousands of files rendered one preview row per file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer UI: upload a few files at a time

Every selected file started its own request at once, so a large folder
opened thousands of requests together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: regenerate the gzipped admin.js

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: reload the listing after a batch fails without a result

A batch that fails mid-request may still have stored some of its files,
so closing the modal reloads the listing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: stop reading a folder drop once a newer selection replaces it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: skip the read error of a folder drop a newer selection replaced

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* filer UI: share the four-upload limit across selections

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: regenerate the gzipped admin.js

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* admin: set up the file browser handlers once

admin.js already runs setupFileManagerEventHandlers on DOMContentLoaded,
so the file browser page registered every listener twice and each drop
was read twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 11:35:32 -08:00

922 lines
29 KiB
Go

package handlers
import (
"bufio"
"compress/gzip"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"mime"
"mime/multipart"
"net/http"
"os"
"path"
"slices"
"strconv"
"strings"
"time"
"github.com/seaweedfs/seaweedfs/weed/admin/dash"
"github.com/seaweedfs/seaweedfs/weed/admin/view/app"
"github.com/seaweedfs/seaweedfs/weed/admin/view/layout"
"github.com/seaweedfs/seaweedfs/weed/filer"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
"github.com/seaweedfs/seaweedfs/weed/util/http/client"
"google.golang.org/protobuf/proto"
)
type FileBrowserHandlers struct {
adminServer *dash.AdminServer
httpClient *client.HTTPClient
}
func NewFileBrowserHandlers(adminServer *dash.AdminServer) *FileBrowserHandlers {
// Create HTTP client with TLS support from https.client configuration
// The client is created without a timeout - each operation will set its own timeout
// If TLS is enabled but misconfigured, fail fast to alert the operator immediately
// rather than silently falling back to HTTP and causing confusing runtime errors
httpClient, err := client.NewHttpClient(client.Client)
if err != nil {
glog.Fatalf("Failed to create HTTPS client for file browser: %v", err)
}
return &FileBrowserHandlers{
adminServer: adminServer,
httpClient: httpClient,
}
}
// ShowFileBrowser renders the file browser page
func (h *FileBrowserHandlers) ShowFileBrowser(w http.ResponseWriter, r *http.Request) {
// Get path from query parameter, default to root
dirPath := path.Clean(util.CleanWindowsPath(defaultQuery(r.URL.Query().Get("path"), "/")))
if !strings.HasPrefix(dirPath, "/") {
dirPath = "/" + dirPath
}
if dash.IsReadOnlyRole(dash.RoleFromContext(r.Context())) && restrictedBrowserPath(dirPath) {
http.Redirect(w, r, dash.P(r.Context(), "/admin?error=Insufficient permissions"), http.StatusSeeOther)
return
}
// Get pagination parameters
lastFileName := r.URL.Query().Get("lastFileName")
pageSize, err := strconv.Atoi(defaultQuery(r.URL.Query().Get("limit"), "20"))
if err != nil || pageSize < 1 {
pageSize = 20
}
if pageSize > 200 {
pageSize = 200
}
// Get file browser data with cursor-based pagination
browserData, err := h.adminServer.GetFileBrowser(dirPath, "", lastFileName, pageSize)
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to get file browser data: "+err.Error())
return
}
// Set username
username := dash.UsernameFromContext(r.Context())
if username == "" {
username = "admin"
}
browserData.Username = username
// Render HTML template
w.Header().Set("Content-Type", "text/html")
browserComponent := app.FileBrowser(*browserData)
viewCtx := layout.NewViewContext(r, username, dash.CSRFTokenFromContext(r.Context()))
layoutComponent := layout.Layout(viewCtx, browserComponent)
err = layoutComponent.Render(r.Context(), w)
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to render template: "+err.Error())
return
}
}
// DeleteFile handles file deletion API requests
func (h *FileBrowserHandlers) DeleteFile(w http.ResponseWriter, r *http.Request) {
var request struct {
Path string `json:"path" binding:"required"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &request); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if strings.TrimSpace(request.Path) == "" {
writeJSONError(w, http.StatusBadRequest, "path is required")
return
}
// Delete file via filer
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
return filer_pb.DoRemove(context.Background(), client, path.Dir(request.Path), path.Base(request.Path), true, true, false, false, nil)
})
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to delete file: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"message": "File deleted successfully"})
}
// DeleteMultipleFiles handles multiple file deletion API requests
func (h *FileBrowserHandlers) DeleteMultipleFiles(w http.ResponseWriter, r *http.Request) {
var request struct {
Paths []string `json:"paths" binding:"required"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &request); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if len(request.Paths) == 0 {
writeJSONError(w, http.StatusBadRequest, "No paths provided")
return
}
for _, p := range request.Paths {
if strings.TrimSpace(p) == "" {
writeJSONError(w, http.StatusBadRequest, "path is required")
return
}
}
var deletedCount int
var failedCount int
var errors []string
// Delete each file/folder
for _, p := range request.Paths {
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
return filer_pb.DoRemove(context.Background(), client, path.Dir(p), path.Base(p), true, true, false, false, nil)
})
if err != nil {
failedCount++
errors = append(errors, fmt.Sprintf("%s: %v", p, err))
} else {
deletedCount++
}
}
// Prepare response
response := map[string]interface{}{
"deleted": deletedCount,
"failed": failedCount,
"total": len(request.Paths),
}
if len(errors) > 0 {
response["errors"] = errors
}
if deletedCount > 0 {
if failedCount == 0 {
response["message"] = fmt.Sprintf("Successfully deleted %d item(s)", deletedCount)
} else {
response["message"] = fmt.Sprintf("Deleted %d item(s), failed to delete %d item(s)", deletedCount, failedCount)
}
writeJSON(w, http.StatusOK, response)
} else {
response["message"] = "Failed to delete all selected items"
writeJSON(w, http.StatusInternalServerError, response)
}
}
// CreateFolder handles folder creation requests
func (h *FileBrowserHandlers) CreateFolder(w http.ResponseWriter, r *http.Request) {
var request struct {
Path string `json:"path" binding:"required"`
FolderName string `json:"folder_name" binding:"required"`
}
if err := decodeJSONBody(newJSONMaxReader(w, r), &request); err != nil {
writeJSONError(w, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if strings.TrimSpace(request.Path) == "" {
writeJSONError(w, http.StatusBadRequest, "path is required")
return
}
// Clean and validate folder name
folderName := strings.TrimSpace(request.FolderName)
if folderName == "" || strings.Contains(folderName, "/") || strings.Contains(folderName, "\\") {
writeJSONError(w, http.StatusBadRequest, "Invalid folder name")
return
}
// Create full path for new folder
base := "/" + strings.TrimPrefix(request.Path, "/")
fullPath := path.Join(base, folderName)
// Create folder via filer
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
_, err := client.CreateEntry(context.Background(), &filer_pb.CreateEntryRequest{
Directory: path.Dir(fullPath),
Entry: &filer_pb.Entry{
Name: path.Base(fullPath),
IsDirectory: true,
Attributes: &filer_pb.FuseAttributes{
FileMode: uint32(0o755 | os.ModeDir), // Directory mode
Uid: filer_pb.OS_UID,
Gid: filer_pb.OS_GID,
Crtime: time.Now().Unix(),
Mtime: time.Now().Unix(),
TtlSec: 0,
},
},
})
return err
})
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to create folder: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]interface{}{"message": "Folder created successfully"})
}
// UploadFile handles file upload requests
func (h *FileBrowserHandlers) UploadFile(w http.ResponseWriter, r *http.Request) {
maxSize := h.adminServer.GetUploadMaxSize()
if maxSize > 0 {
r.Body = http.MaxBytesReader(w, r.Body, maxSize+uploadFormOverhead)
}
// Parts beyond 32MB spill to temporary files instead of memory
err := r.ParseMultipartForm(32 << 20)
if err != nil {
var maxBytesErr *http.MaxBytesError
if errors.As(err, &maxBytesErr) {
writeJSONError(w, http.StatusRequestEntityTooLarge, uploadTooLargeMessage(maxSize))
return
}
writeJSONError(w, http.StatusBadRequest, "Failed to parse multipart form: "+err.Error())
return
}
// The router hands us a copy of the request, so the server cannot remove the spooled parts itself
defer r.MultipartForm.RemoveAll()
currentPath := r.FormValue("path")
if currentPath == "" {
currentPath = "/"
}
// Get uploaded files (supports multiple files)
files := r.MultipartForm.File["files"]
if len(files) == 0 {
writeJSONError(w, http.StatusBadRequest, "No files uploaded")
return
}
if maxSize > 0 {
var totalSize int64
for _, fileHeader := range files {
totalSize += fileHeader.Size
}
if totalSize > maxSize {
writeJSONError(w, http.StatusRequestEntityTooLarge, uploadTooLargeMessage(maxSize))
return
}
}
// A folder upload names each file by its path inside the folder; the multipart
// file name keeps only the base name.
var relativePaths []string
if v, ok := r.MultipartForm.Value["relativePaths"]; ok {
if len(v) != 1 || json.Unmarshal([]byte(v[0]), &relativePaths) != nil || len(relativePaths) != len(files) {
writeJSONError(w, http.StatusBadRequest, "relativePaths must list one path per uploaded file")
return
}
}
var uploadResults []map[string]interface{}
var failedUploads []string
// Process each uploaded file
for i, fileHeader := range files {
// Validate file name
fileName := fileHeader.Filename
if relativePaths != nil {
fileName = relativePaths[i]
}
if fileName == "" {
failedUploads = append(failedUploads, "invalid filename")
continue
}
// Normalize Windows-style backslashes to forward slashes
fileName = util.CleanWindowsPath(fileName)
if relativePaths != nil && !isRelativeUploadPath(fileName) {
failedUploads = append(failedUploads, fmt.Sprintf("%s: invalid relative path", fileName))
continue
}
// Create full path for the file using path.Join for URL path semantics
// path.Join handles double slashes and is not OS-specific like filepath.Join
fullPath := path.Join(currentPath, fileName)
if !strings.HasPrefix(fullPath, "/") {
fullPath = "/" + fullPath
}
// Upload file to filer
err = h.uploadFileToFiler(r.Context(), fullPath, fileHeader)
if err != nil {
failedUploads = append(failedUploads, fmt.Sprintf("%s: %v", fileName, err))
} else {
uploadResults = append(uploadResults, map[string]interface{}{
"name": fileName,
"size": fileHeader.Size,
"path": fullPath,
})
}
}
// Prepare response
response := map[string]interface{}{
"uploaded": len(uploadResults),
"failed": len(failedUploads),
"files": uploadResults,
}
if len(failedUploads) > 0 {
response["errors"] = failedUploads
}
if len(uploadResults) > 0 {
if len(failedUploads) == 0 {
response["message"] = fmt.Sprintf("Successfully uploaded %d file(s)", len(uploadResults))
} else {
response["message"] = fmt.Sprintf("Uploaded %d file(s), %d failed", len(uploadResults), len(failedUploads))
}
writeJSON(w, http.StatusOK, response)
} else {
response["message"] = "All file uploads failed"
writeJSON(w, http.StatusInternalServerError, response)
}
}
// isRelativeUploadPath reports whether name stays inside the upload directory.
func isRelativeUploadPath(name string) bool {
for _, part := range strings.Split(name, "/") {
if part == "" || part == "." || part == ".." {
return false
}
}
return true
}
// uploadFormOverhead allows for the multipart headers and the path field on
// top of the file bytes an upload may carry. Go's multipart parser accepts at
// most 1000 parts, whose headers take about half of this.
const uploadFormOverhead = 1 << 20
func uploadTooLargeMessage(maxSize int64) string {
return fmt.Sprintf("Total upload size exceeds the %d MB limit", maxSize>>20)
}
// uploadFileToFiler uploads a file to the cluster via filer gRPC + volume
// HTTP. This works whether or not the filer is running with -disableHttp=true,
// since the bytes never traverse the filer's HTTP listener. The multipart
// file is streamed through the chunked uploader, so the admin process never
// buffers the entire payload in memory. The caller passes the request
// context so a client disconnect cancels the in-flight chunk uploads instead
// of letting them run to completion against the volume servers.
func (h *FileBrowserHandlers) uploadFileToFiler(ctx context.Context, filePath string, fileHeader *multipart.FileHeader) error {
file, err := fileHeader.Open()
if err != nil {
return fmt.Errorf("failed to open file: %w", err)
}
defer file.Close()
return h.uploadFileGrpc(ctx, filePath, fileHeader.Filename, fileHeader.Header.Get("Content-Type"), file)
}
// validateAndCleanFilePath validates and cleans the file path to prevent path traversal
func (h *FileBrowserHandlers) validateAndCleanFilePath(filePath string) (string, error) {
if filePath == "" {
return "", fmt.Errorf("file path cannot be empty")
}
// Normalize Windows-style backslashes to forward slashes
filePath = util.CleanWindowsPath(filePath)
// Clean the path to remove any .. or . components
// Use path.Clean (not filepath.Clean) since this is a URL path
cleanPath := path.Clean(filePath)
// Ensure the path starts with /
if !strings.HasPrefix(cleanPath, "/") {
cleanPath = "/" + cleanPath
}
// Reject parent-directory components; names merely containing ".." are valid
if slices.Contains(strings.Split(cleanPath, "/"), "..") {
return "", fmt.Errorf("path traversal not allowed")
}
return cleanPath, nil
}
// restrictedBrowserPath reports whether a cleaned path lies inside the filer's
// /etc tree, which holds the credential store and cluster configuration.
func restrictedBrowserPath(cleanPath string) bool {
return cleanPath == strings.TrimSuffix(filer.DirectoryEtcRoot, "/") ||
strings.HasPrefix(cleanPath, filer.DirectoryEtcRoot)
}
// cleanReadPath validates the request's path parameter; read-only sessions are
// denied the /etc tree so the file browser cannot serve stored credentials.
func (h *FileBrowserHandlers) cleanReadPath(w http.ResponseWriter, r *http.Request) (string, bool) {
cleanPath, err := h.validateAndCleanFilePath(r.URL.Query().Get("path"))
if err != nil {
writeJSONError(w, http.StatusBadRequest, err.Error())
return "", false
}
if dash.IsReadOnlyRole(dash.RoleFromContext(r.Context())) && restrictedBrowserPath(cleanPath) {
writeJSONError(w, http.StatusForbidden, "Access to this path requires admin access")
return "", false
}
return cleanPath, true
}
// exportExcludedPrefixes lists the subtrees a metadata export skips; read-only
// sessions also lose /etc so stored credentials stay out of the dump.
func exportExcludedPrefixes(r *http.Request) []string {
prefixes := []string{filer.SystemLogDir}
if dash.IsReadOnlyRole(dash.RoleFromContext(r.Context())) {
prefixes = append(prefixes, strings.TrimSuffix(filer.DirectoryEtcRoot, "/"))
}
return prefixes
}
// fetchFileContent fetches file content via the filer gRPC service. It is
// used for the "view as text" path, so the maxBytes cap matches the 1 MB
// limit the caller already applies before invoking us.
func (h *FileBrowserHandlers) fetchFileContent(filePath string, timeout time.Duration) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
return h.fetchFileContentGrpc(ctx, filePath, 0)
}
// DownloadFile streams a file straight from the volume servers via the filer
// gRPC service, so the admin file browser keeps working even when the filer
// is started with -disableHttp=true.
func (h *FileBrowserHandlers) DownloadFile(w http.ResponseWriter, r *http.Request) {
filePath, ok := h.cleanReadPath(w, r)
if !ok {
return
}
inline := r.URL.Query().Get("inline") == "true"
tracker := &responseWriteTracker{ResponseWriter: w}
if err := h.downloadFileGrpc(r.Context(), filePath, tracker, inline); err != nil {
// Once bytes have been written we can't switch to a JSON error body
// without corrupting the partial response — log and stop. Before any
// write the response is still uncommitted, so a 502 with details is
// safe.
if tracker.committed {
glog.Errorf("Error streaming file download: %v", err)
return
}
writeJSONError(w, http.StatusBadGateway, "Failed to fetch file: "+err.Error())
}
}
// responseWriteTracker wraps http.ResponseWriter to record whether the
// response has been committed (status line + headers sent). DownloadFile
// uses this instead of probing Header() so future header-setting code
// reorganization can't silently break the "did we already send bytes?"
// detection.
type responseWriteTracker struct {
http.ResponseWriter
committed bool
}
func (t *responseWriteTracker) WriteHeader(code int) {
t.committed = true
t.ResponseWriter.WriteHeader(code)
}
func (t *responseWriteTracker) Write(p []byte) (int, error) {
t.committed = true
return t.ResponseWriter.Write(p)
}
// ViewFile handles file viewing requests (for text files, images, etc.)
func (h *FileBrowserHandlers) ViewFile(w http.ResponseWriter, r *http.Request) {
filePath, ok := h.cleanReadPath(w, r)
if !ok {
return
}
// Get file metadata first
var fileEntry dash.FileEntry
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
resp, err := client.LookupDirectoryEntry(context.Background(), &filer_pb.LookupDirectoryEntryRequest{
Directory: path.Dir(filePath),
Name: path.Base(filePath),
})
if err != nil {
return err
}
entry := resp.Entry
if entry == nil {
return fmt.Errorf("file not found")
}
// Convert to FileEntry
var modTime time.Time
if entry.Attributes != nil && entry.Attributes.Mtime > 0 {
modTime = time.Unix(entry.Attributes.Mtime, 0)
}
var size int64
if entry.Attributes != nil {
size = int64(entry.Attributes.FileSize)
}
mime := dash.ResolveEntryMime(entry)
fileEntry = dash.FileEntry{
Name: entry.Name,
FullPath: filePath,
IsDirectory: entry.IsDirectory,
Size: size,
ModTime: modTime,
Mime: mime,
}
return nil
})
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to get file metadata: "+err.Error())
return
}
// Check if file is viewable as text
var content string
var viewable bool
var reason string
// First check if it's a known text type or if we should check content
isKnownTextType := strings.HasPrefix(fileEntry.Mime, "text/") ||
fileEntry.Mime == "application/json" ||
fileEntry.Mime == "application/javascript" ||
fileEntry.Mime == "application/xml"
// For unknown types, check if it might be text by content
if !isKnownTextType && fileEntry.Mime == "application/octet-stream" {
isKnownTextType = h.isLikelyTextFile(filePath, 512)
if isKnownTextType {
// Update MIME type for better display
fileEntry.Mime = "text/plain"
}
}
if isKnownTextType {
// Limit text file size for viewing (max 1MB)
if fileEntry.Size > 1024*1024 {
viewable = false
reason = "File too large for viewing (>1MB)"
} else {
// Fetch file content from filer
var err error
content, err = h.fetchFileContent(filePath, 30*time.Second)
if err != nil {
reason = err.Error()
}
viewable = (err == nil)
}
} else {
// Not a text file, but might be viewable as image or PDF
if strings.HasPrefix(fileEntry.Mime, "image/") || fileEntry.Mime == "application/pdf" {
viewable = true
} else {
viewable = false
reason = "File type not supported for viewing"
}
}
writeJSON(w, http.StatusOK, map[string]interface{}{
"file": fileEntry,
"content": content,
"viewable": viewable,
"reason": reason,
})
}
// GetFileProperties handles file properties requests
func (h *FileBrowserHandlers) GetFileProperties(w http.ResponseWriter, r *http.Request) {
filePath, ok := h.cleanReadPath(w, r)
if !ok {
return
}
// Get detailed file information from filer
var properties map[string]interface{}
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
resp, err := client.LookupDirectoryEntry(context.Background(), &filer_pb.LookupDirectoryEntryRequest{
Directory: path.Dir(filePath),
Name: path.Base(filePath),
})
if err != nil {
return err
}
entry := resp.Entry
if entry == nil {
return fmt.Errorf("file not found")
}
properties = make(map[string]interface{})
properties["name"] = entry.Name
properties["full_path"] = filePath
properties["is_directory"] = entry.IsDirectory
if entry.Attributes != nil {
properties["size"] = entry.Attributes.FileSize
properties["size_formatted"] = h.formatBytes(int64(entry.Attributes.FileSize))
if entry.Attributes.Mtime > 0 {
modTime := time.Unix(entry.Attributes.Mtime, 0)
properties["modified_time"] = modTime.Format("2006-01-02 15:04:05")
properties["modified_timestamp"] = entry.Attributes.Mtime
}
if entry.Attributes.Crtime > 0 {
createTime := time.Unix(entry.Attributes.Crtime, 0)
properties["created_time"] = createTime.Format("2006-01-02 15:04:05")
properties["created_timestamp"] = entry.Attributes.Crtime
}
properties["file_mode"] = dash.FormatFileMode(entry.Attributes.FileMode)
properties["file_mode_formatted"] = dash.FormatFileMode(entry.Attributes.FileMode)
properties["file_mode_octal"] = fmt.Sprintf("%o", entry.Attributes.FileMode)
properties["uid"] = entry.Attributes.Uid
properties["gid"] = entry.Attributes.Gid
properties["ttl_seconds"] = entry.Attributes.TtlSec
if entry.Attributes.TtlSec > 0 {
properties["ttl_formatted"] = fmt.Sprintf("%d seconds", entry.Attributes.TtlSec)
}
}
// Get extended attributes
if entry.Extended != nil {
extended := make(map[string]string)
for key, value := range entry.Extended {
extended[key] = string(value)
}
properties["extended"] = extended
}
// Get chunk information for files
if !entry.IsDirectory && len(entry.Chunks) > 0 {
chunks := make([]map[string]interface{}, 0, len(entry.Chunks))
for _, chunk := range entry.Chunks {
chunkInfo := map[string]interface{}{
"file_id": chunk.FileId,
"offset": chunk.Offset,
"size": chunk.Size,
"modified_ts": chunk.ModifiedTsNs,
"e_tag": chunk.ETag,
"source_fid": chunk.SourceFileId,
}
chunks = append(chunks, chunkInfo)
}
properties["chunks"] = chunks
properties["chunk_count"] = len(entry.Chunks)
}
if !entry.IsDirectory {
properties["mime_type"] = dash.ResolveEntryMime(entry)
}
return nil
})
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to get file properties: "+err.Error())
return
}
if isDir, _ := properties["is_directory"].(bool); !isDir {
if objectURL := h.adminServer.GetS3ObjectURL(filePath); objectURL != "" {
properties["object_url"] = objectURL
}
}
writeJSON(w, http.StatusOK, properties)
}
// ExportMetadata streams a file or folder's metadata as a gzipped, length-prefixed
// FullEntry stream — the format weed shell fs.meta.load reads. Directories are
// walked recursively via the filer BFS metadata stream.
func (h *FileBrowserHandlers) ExportMetadata(w http.ResponseWriter, r *http.Request) {
cleanPath, ok := h.cleanReadPath(w, r)
if !ok {
return
}
excludedPrefixes := exportExcludedPrefixes(r)
tracker := &responseWriteTracker{ResponseWriter: w}
err := h.adminServer.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {
stream, err := client.TraverseBfsMetadata(r.Context(), &filer_pb.TraverseBfsMetadataRequest{
Directory: cleanPath,
ExcludedPrefixes: excludedPrefixes,
})
if err != nil {
return err
}
// Read the first entry before sending headers so a bad path returns a clean error.
first, err := stream.Recv()
if err != nil {
return err
}
downloadName := path.Base(cleanPath)
if downloadName == "/" || downloadName == "." || downloadName == "" {
downloadName = "root"
}
tracker.Header().Set("Content-Type", "application/gzip")
tracker.Header().Set("Content-Disposition", mime.FormatMediaType("attachment", map[string]string{"filename": downloadName + ".meta.gz"}))
tracker.WriteHeader(http.StatusOK)
bw := bufio.NewWriter(tracker)
gw := gzip.NewWriter(bw)
sizeBuf := make([]byte, 4)
writeEntry := func(resp *filer_pb.TraverseBfsMetadataResponse) error {
b, err := proto.Marshal(&filer_pb.FullEntry{Dir: resp.Directory, Entry: resp.Entry})
if err != nil {
return err
}
util.Uint32toBytes(sizeBuf, uint32(len(b)))
if _, err := gw.Write(sizeBuf); err != nil {
return err
}
_, err = gw.Write(b)
return err
}
if err := writeEntry(first); err != nil {
return err
}
for {
resp, recvErr := stream.Recv()
if recvErr == io.EOF {
break
}
if recvErr != nil {
return recvErr
}
if err := writeEntry(resp); err != nil {
return err
}
}
if err := gw.Close(); err != nil {
return err
}
return bw.Flush()
})
if err != nil {
if tracker.committed {
glog.Errorf("Error exporting metadata for %s: %v", cleanPath, err)
return
}
writeJSONError(w, http.StatusInternalServerError, "Failed to export metadata: "+err.Error())
}
}
// Helper function to format bytes
func (h *FileBrowserHandlers) formatBytes(bytes int64) string {
const unit = 1024
if bytes < unit {
return fmt.Sprintf("%d B", bytes)
}
div, exp := int64(unit), 0
for n := bytes / unit; n >= unit; n /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %cB", float64(bytes)/float64(div), "KMGTPE"[exp])
}
// Helper function to check if a file is likely a text file by checking content
func (h *FileBrowserHandlers) isLikelyTextFile(filePath string, maxCheckSize int64) bool {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
content, err := h.fetchFileContentGrpc(ctx, filePath, int(maxCheckSize))
if err != nil {
return false
}
if len(content) == 0 {
return true
}
return h.isPrintableText([]byte(content))
}
// Helper function to check if content is printable text
func (h *FileBrowserHandlers) isPrintableText(data []byte) bool {
if len(data) == 0 {
return true
}
// Count printable characters
printable := 0
for _, b := range data {
if b >= 32 && b <= 126 || b == 9 || b == 10 || b == 13 {
// Printable ASCII, tab, newline, carriage return
printable++
} else if b >= 128 {
// Potential UTF-8 character
printable++
}
}
// If more than 95% of characters are printable, consider it text
return float64(printable)/float64(len(data)) > 0.95
}
// Helper function for min
func min(a, b int64) int64 {
if a < b {
return a
}
return b
}
// maxListFoldersEntries caps how many subfolder names ListFolders will
// collect for a single request, so a directory with an unusually large
// number of children can't turn one autocomplete keystroke into an
// unbounded, slow full-directory walk.
const maxListFoldersEntries = 2000
// maxListFoldersScanned caps how many entries ListFolders will page through
// looking for those subfolders. The folder cap alone doesn't bound the work:
// a bucket holding nothing but flat object keys has no subfolders to count,
// so the walk runs to the end of the directory - a million keys is a million
// entries read, 200 per round trip, behind a single keystroke.
const maxListFoldersScanned = 10000
// ListFolders returns, as JSON, the names of the subdirectories directly
// under the given path. It exists to back progressive autocomplete (e.g. the
// policy editor's Resource ARN field building up "bucket/folder/subfolder"
// one path segment at a time) rather than to be a general directory listing
// API, so it's restricted to paths under /buckets. The optional prefix is the
// segment the caller is still typing, and narrows the listing to it.
func (h *FileBrowserHandlers) ListFolders(w http.ResponseWriter, r *http.Request) {
dirPath := defaultQuery(r.URL.Query().Get("path"), "/buckets")
// Clean before the scope check: CleanWindowsPath only rewrites backslashes,
// so "/buckets/../etc" would otherwise satisfy the prefix test below.
dirPath = path.Clean(util.CleanWindowsPath(dirPath))
if dirPath != "/buckets" && !strings.HasPrefix(dirPath, "/buckets/") {
writeJSONError(w, http.StatusBadRequest, "path must be under /buckets")
return
}
prefix := r.URL.Query().Get("prefix")
folders := []string{}
lastFileName := ""
scanned := 0
for {
browserData, err := h.adminServer.GetFileBrowser(dirPath, prefix, lastFileName, 200)
if err != nil {
writeJSONError(w, http.StatusInternalServerError, "Failed to list directory: "+err.Error())
return
}
scanned += len(browserData.Entries)
for _, entry := range browserData.Entries {
if entry.IsDirectory {
folders = append(folders, entry.Name)
if len(folders) >= maxListFoldersEntries {
break
}
}
}
if len(browserData.Entries) == 0 || !browserData.HasNextPage ||
len(folders) >= maxListFoldersEntries || scanned >= maxListFoldersScanned {
break
}
lastFileName = browserData.Entries[len(browserData.Entries)-1].Name
}
writeJSON(w, http.StatusOK, map[string]interface{}{"folders": folders})
}