Files
seaweedfs/weed/mount/wfs_save.go
T
Chris LuandDevin 3abf8b2572 filer,pb,mount: keep per-chunk cipher keys out of logs (#12085)
* filer: format ChunkView without its cipher key

readChunkSliceAt logs a failed chunk fetch with %+v, which printed the
per-chunk AES-256-GCM key into error-level logs on every failed read.
Give ChunkView a String() that omits CipherKey so %v and %+v stay safe.

* pb: scrub cipher keys when an entry must be logged whole

ScrubCipherKeys clones an Entry with chunk CipherKey fields cleared, so
a warning that needs the full entry still cannot leak the keys.

* log entry identifiers instead of whole metadata messages

Several log lines formatted whole protobuf messages — SubscribeMetadataResponse,
EventNotification, Entry-carrying requests — whose chunks include cipher_key
under -encryptVolumeData, so the keys escaped into logs (some at default
verbosity). Log the directory, entry name and event timestamp instead; where a
chunk list was printed, print file ids via FileIdsString.

* log queue: unwrap SubscribeMetadataResponse and never print raw protos

The fallback printed any message with %+v, so a SubscribeMetadataResponse
carrying entries would still put cipher keys in the log. Unwrap its event
notifications (batched events included) and reduce unknown messages to
their type name.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* remote sync logs: name the event's real path

Rename targets logged under resp.Directory while the entry lives under
NewParentPath; route them through MetadataEventTargetDirectory. Sync
failure logs printed only the directory, so same-dir failures shared an
identifier — log the source and destination paths instead.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* log queue: log the entry's destination path on moves

A cross-directory rename printed "file" -> "file" under the source
path, hiding where the entry landed. Use NewParentPath so the log
shows the destination directory next to the new name.

* admin: install the go.mod-pinned templ, not @latest

install-templ fetched templ@latest, which now emits
templruntime.NewAttributeExpression - a symbol absent from the
runtime pinned in go.mod - so every fresh 'make' regenerated
uncompilable _templ.go files. Install the go.mod version and
reinstall when the binary on PATH reports a different one.

* admin: drop stray Makefile change belonging to the templ pin PR

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* notification: join the move destination path in log queue output

A NewParentPath of "/" printed //name for moves into the root;
path.Join keeps a single leading slash.

* log queue: skip a nil event notification instead of logging an empty move

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* pb: tolerate nil chunk entries while scrubbing cipher keys for logs

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* subscription and backup failure logs name the event's file

The process/marshal failure logs printed only the event directory and
timestamp, so a failed sync left operators guessing which file failed.
Use MetadataEventSourceFullPath/MetadataEventTargetFullPath so each
failure names the source and target paths without printing the entries.

* replication: identify entry-less events without printing the proto

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* s3: scrub cipher keys from the bucket-metadata debug dump

buildBucketMetadata marshals the whole filer entry to JSON for a V(3)
log line; a chunked entry would emit each chunk's cipher_key base64.
Marshal the scrubbed copy instead.

---------

Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 11:52:56 -07:00

124 lines
4.4 KiB
Go

package mount
import (
"context"
"fmt"
"syscall"
"time"
"github.com/seaweedfs/go-fuse/v2/fuse"
"google.golang.org/protobuf/proto"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
)
func (wfs *WFS) saveEntry(path util.FullPath, entry *filer_pb.Entry) (code fuse.Status) {
parentDir, _ := path.DirAndName()
wfs.mapPbIdFromLocalToFiler(entry)
defer wfs.mapPbIdFromFilerToLocal(entry)
request := &filer_pb.UpdateEntryRequest{
Directory: parentDir,
Entry: entry,
Signatures: []int32{wfs.signature},
}
glog.V(1).Infof("save entry %s", path)
var resp *filer_pb.UpdateEntryResponse
err := retryMetadataFlushIf(context.Background(), func() error {
var callErr error
resp, callErr = wfs.streamUpdateEntry(context.Background(), request)
return callErr
}, isRetryableFilerError, func(nextAttempt, totalAttempts int, backoff time.Duration, err error) {
glog.Warningf("saveEntry %s: retrying UpdateEntry (attempt %d/%d) after %v: %v",
path, nextAttempt, totalAttempts, backoff, err)
})
if err != nil {
// Wrap with %w so grpcErrorToFuseStatus can still unwrap the gRPC status
// (e.g. codes.Canceled → ETIMEDOUT). Using %v would stringify the error and
// status.FromError would fall through to the default EIO.
err = fmt.Errorf("UpdateEntry dir %s: %w", path, err)
fuseStatus := grpcErrorToFuseStatus(err)
if fuseStatus == fuse.EIO {
glog.Errorf("saveEntry failed for %s: %v (returning EIO)", path, err)
} else {
glog.V(1).Infof("saveEntry failed for %s: %v (returning %v)", path, err, fuseStatus)
}
return fuseStatus
}
// The mutation is acknowledged; bring any open handle for this path up
// to the acknowledged state — a handle opened while this save was in
// flight holds an older entry, and advancing its version alone would
// fence out the events carrying what it lacks. A no-change update
// returns no event but still carries the log position it confirmed.
ackVersion := ackVersionTsNs(resp)
if inode, found := wfs.inodeToPath.GetInode(path); found {
if fh, fhFound := wfs.fhMap.FindFileHandle(inode); fhFound {
ackedEntry := proto.Clone(entry).(*filer_pb.Entry)
wfs.mapPbIdFromFilerToLocal(ackedEntry)
fh.installAckedEntry(ackedEntry, ackVersion, resp.GetLogSignature())
}
}
event := resp.GetMetadataEvent()
if event == nil {
event = metadataUpdateEvent(parentDir, entry)
if event != nil {
event.TsNs = ackVersion
}
}
if applyErr := wfs.applyLocalMetadataEvent(context.Background(), event); applyErr != nil {
glog.Warningf("saveEntry %s: best-effort metadata apply failed: %v", path, applyErr)
wfs.inodeToPath.InvalidateChildrenCache(util.FullPath(parentDir))
}
return fuse.OK
}
func (wfs *WFS) mapPbIdFromFilerToLocal(entry *filer_pb.Entry) {
if entry.Attributes == nil {
return
}
entry.Attributes.Uid, entry.Attributes.Gid = wfs.option.UidGidMapper.FilerToLocal(entry.Attributes.Uid, entry.Attributes.Gid)
}
func (wfs *WFS) mapPbIdFromLocalToFiler(entry *filer_pb.Entry) {
if entry.Attributes == nil {
return
}
entry.Attributes.Uid, entry.Attributes.Gid = wfs.option.UidGidMapper.LocalToFiler(entry.Attributes.Uid, entry.Attributes.Gid)
}
// sanitizeFuseName scrubs a name arriving from the kernel before it is placed
// in a proto string field. Linux (and macOS) pass raw bytes for filenames;
// apps like GNOME Trash produce partial files whose names contain binary
// payloads. Proto3 `string` fields require valid UTF-8, so an unsanitized
// name causes gRPC to fail the whole AssignVolume / CreateEntry / DeleteEntry
// RPC with "grpc: error while marshaling: string field contains invalid
// UTF-8", which surfaces to userspace as EIO. Sanitizing at every FUSE
// boundary keeps filer RPCs marshalable and prevents a single ill-named file
// from poisoning the shared gRPC channel for every other in-flight request.
//
// Delegates to util.SanitizeUTF8Name so the replacement character is chosen
// in exactly one place across the codebase.
func sanitizeFuseName(name string) string {
return util.SanitizeUTF8Name(name)
}
func checkName(name string) (string, fuse.Status) {
name = sanitizeFuseName(name)
// The Linux FUSE kernel module enforces NAME_MAX=255 at the VFS layer.
// Return ENAMETOOLONG early to avoid creating entries that cannot be
// looked up via normal syscalls (stat, chmod, etc.).
if len(name) > 255 {
return name, fuse.Status(syscall.ENAMETOOLONG)
}
return name, fuse.OK
}