Files
seaweedfs/weed/mount/weedfs_attr.go
T
Chris Lu bf31f404bc test: add pjdfstest POSIX compliance suite (#9013)
* test: add pjdfstest POSIX compliance suite

Adds a script and CI workflow that runs the upstream pjdfstest POSIX
compliance test suite against a SeaweedFS FUSE mount. The script starts
a self-contained `weed mini` server, mounts the filesystem with
`weed mount`, builds pjdfstest from source, and runs it under prove(1).

* fix: address review feedback on pjdfstest setup

- Use github.ref instead of github.head_ref in concurrency group so
  push events get a stable group key
- Add explicit timeout check after filer readiness polling loop
- Refresh pjdfstest checkout when PJDFSTEST_REPO or PJDFSTEST_REF are
  overridden instead of silently reusing stale sources

* test: add Docker-based pjdfstest for faster iteration

Adds a docker-compose setup that reuses the existing e2e image pattern:
- master, volume, filer services from chrislusf/seaweedfs:e2e
- mount service extended with pjdfstest baked in (Dockerfile extends e2e)
- Tests run via `docker compose exec mount /run.sh`
- CI workflow gains a parallel `pjdfstest (docker)` job

This avoids building Go from scratch on each iteration — just rebuild the
e2e image once and iterate on the compose stack.

* fix: address second round of review feedback

- Use mktemp for WORK_DIR so each run starts with a clean filer state
- Pin PJDFSTEST_REF to immutable commit (03eb257) instead of master
- Use cp -r instead of cp -a to avoid preserving ownership during setup

* fix: address CI failure and third round of review feedback

- Fix docker job: fall back to plain docker build when buildx cache
  export is not supported (default docker driver in some CI runners)
- Use /healthz endpoint for filer healthcheck in docker-compose
- Copy logs to a fixed path (/tmp/seaweedfs-pjdfstest-logs/) for
  reliable CI artifact upload when WORK_DIR is a mktemp path

* fix(mount): improve POSIX compliance for FUSE mount

Address several POSIX compliance gaps surfaced by the pjdfstest suite:

1. Filename length limit: reduce from 4096 to 255 bytes (NAME_MAX),
   returning ENAMETOOLONG for longer names.

2. SUID/SGID clearing on write: clear setuid/setgid bits when a
   non-root user writes to a file (POSIX requirement).

3. SUID/SGID clearing on chown: clear setuid/setgid bits when file
   ownership changes by a non-root user.

4. Sticky bit enforcement: add checkStickyBit helper and enforce it
   in Unlink, Rmdir, and Rename — only file owner, directory owner,
   or root may delete entries in sticky directories.

5. ctime (inode change time) tracking: add ctime field to the
   FuseAttributes protobuf message and filer.Attr struct. Update
   ctime on all metadata-modifying operations (SetAttr, Write/flush,
   Link, Create, Mkdir, Mknod, Symlink, Truncate). Fall back to
   mtime for backward compatibility when ctime is 0.

* fix: add -T flag to docker compose exec for CI

Disable TTY allocation in the pjdfstest docker job since GitHub
Actions runners have no interactive TTY.

* fix(mount): update parent directory mtime/ctime on entry changes

POSIX requires that a directory's st_mtime and st_ctime be updated
whenever entries are created or removed within it. Add
touchDirMtimeCtime() helper and call it after:
- mkdir, rmdir
- create (including deferred creates), mknod, unlink
- symlink, link
- rename (both source and destination directories)

This fixes pjdfstest failures in mkdir/00, mkfifo/00, mknod/00,
mknod/11, open/00, symlink/00, link/00, and rmdir/00.

* fix(mount): enforce sticky bit on destination directory during rename

POSIX requires sticky-bit enforcement on both source and destination
directories during rename. When the destination directory has the
sticky bit set and a target entry already exists, only the file owner,
directory owner, or root may replace it.

* fix(mount): add in-memory atime tracking for POSIX compliance

Track atime separately from mtime using a bounded in-memory map
(capped at 8192 entries with random eviction). atime is not persisted
to the filer — it's only kept in mount memory to satisfy POSIX stat
requirements for utimensat and related syscalls.

This fixes utimensat/00, utimensat/02, utimensat/04, utimensat/05,
and utimensat/09 pjdfstest failures where atime was incorrectly
aliased to mtime.

* fix(mount): restore long filename support, fix permission checks

- Restore 4096-byte filename limit (was incorrectly reduced to 255).
  SeaweedFS stores names as protobuf strings with no ext4-style
  constraint — the 255 limit is not applicable.

- Fix AcquireHandle permission check to map filer uid/gid to local
  space before calling hasAccess, matching the pattern used in Access().

- Fix hasAccess fallback when supplementary group lookup fails: fall
  through to "other" permissions instead of requiring both group AND
  other to match, which was overly restrictive for non-existent UIDs.

* fix(mount): fix permission checks and enforce NAME_MAX=255

- Fix AcquireHandle to map uid/gid from filer-space to local-space
  before calling hasAccess, consistent with the Access handler.

- Fix hasAccess fallback when supplementary group lookup fails: use
  "other" permissions only instead of requiring both group AND other.

- Enforce NAME_MAX=255 with a comment explaining the Linux FUSE kernel
  module's VFS-layer limit. Files >255 bytes can be created via direct
  FUSE protocol calls but can't be stat'd/chmod'd via normal syscalls.

- Don't call touchDirMtimeCtime for deferred creates to avoid
  invalidating the just-cached entry via filer metadata events.

* ci: mark pjdfstest steps as continue-on-error

The pjdfstest suite has known failures (Linux FUSE NAME_MAX=255
limitation, hard link nlink/ctime tracking, nanosecond precision)
that cannot be fixed in the mount layer. Mark the test steps as
continue-on-error so the CI job reports results without blocking.

* ci: increase pjdfstest bare metal timeout to 90 minutes

* fix: use full commit hash for PJDFSTEST_REF in run.sh

Short hashes cannot be resolved by git fetch --depth 1 on shallow
clones. Use the full 40-char SHA.

* test: add pjdfstest known failures skip list

Add known_failures.txt listing 33 test files that cannot pass due to:
- Linux FUSE kernel NAME_MAX=255 (26 files)
- Hard link nlink/ctime tracking requiring filer changes (3 files)
- Parent dir mtime on deferred create (1 file)
- Directory rename permission edge case (1 file)
- rmdir after hard link unlink (1 file)
- Nanosecond timestamp precision (1 file)

Both run.sh and run_inside_container.sh now skip these tests when
running the full suite. Any failure in a non-skipped test will cause
CI to fail, catching regressions immediately.

Remove continue-on-error from CI steps since the skip list handles
known failures.

Result: 204 test files, 8380 tests, all passing.

* ci: remove bare metal pjdfstest job, keep Docker only

The bare metal job consistently gets stuck past its timeout due to
weed processes not exiting cleanly. The Docker job covers the same
tests reliably and runs faster.
2026-04-10 09:52:16 -07:00

350 lines
9.3 KiB
Go

package mount
import (
"os"
"syscall"
"time"
"github.com/seaweedfs/go-fuse/v2/fuse"
"github.com/seaweedfs/seaweedfs/weed/filer"
"github.com/seaweedfs/seaweedfs/weed/glog"
"github.com/seaweedfs/seaweedfs/weed/pb/filer_pb"
"github.com/seaweedfs/seaweedfs/weed/util"
)
func (wfs *WFS) GetAttr(cancel <-chan struct{}, input *fuse.GetAttrIn, out *fuse.AttrOut) (code fuse.Status) {
glog.V(4).Infof("GetAttr %v", input.NodeId)
if input.NodeId == 1 {
wfs.setRootAttr(out)
return fuse.OK
}
inode := input.NodeId
_, _, entry, status := wfs.maybeReadEntry(inode)
if status == fuse.OK {
out.AttrValid = 1
wfs.setAttrByPbEntry(&out.Attr, inode, entry, true)
wfs.applyInMemoryAtime(&out.Attr, inode)
return status
} else {
if fh, found := wfs.fhMap.FindFileHandle(inode); found {
out.AttrValid = 1
// Use shared lock to prevent race with Write operations
fhActiveLock := wfs.fhLockTable.AcquireLock("GetAttr", fh.fh, util.SharedLock)
wfs.setAttrByPbEntry(&out.Attr, inode, fh.entry.GetEntry(), true)
wfs.fhLockTable.ReleaseLock(fh.fh, fhActiveLock)
wfs.applyInMemoryAtime(&out.Attr, inode)
out.Nlink = 0
return fuse.OK
}
}
return status
}
func (wfs *WFS) SetAttr(cancel <-chan struct{}, input *fuse.SetAttrIn, out *fuse.AttrOut) (code fuse.Status) {
// Check quota including uncommitted writes for real-time enforcement
if wfs.IsOverQuotaWithUncommitted() {
return fuse.Status(syscall.ENOSPC)
}
path, fh, entry, status := wfs.maybeReadEntry(input.NodeId)
if status != fuse.OK || entry == nil {
return status
}
if fh != nil {
fh.entryLock.Lock()
defer fh.entryLock.Unlock()
}
wormEnforced, wormEnabled := wfs.wormEnforcedForEntry(path, entry)
if wormEnforced {
return fuse.EPERM
}
if size, ok := input.GetSize(); ok {
glog.V(4).Infof("%v setattr set size=%v chunks=%d", path, size, len(entry.GetChunks()))
if size < filer.FileSize(entry) {
// fmt.Printf("truncate %v \n", fullPath)
var chunks []*filer_pb.FileChunk
var truncatedChunks []*filer_pb.FileChunk
for _, chunk := range entry.GetChunks() {
int64Size := int64(chunk.Size)
if chunk.Offset+int64Size > int64(size) {
// this chunk is truncated
int64Size = int64(size) - chunk.Offset
if int64Size > 0 {
chunks = append(chunks, chunk)
glog.V(4).Infof("truncated chunk %+v from %d to %d\n", chunk.GetFileIdString(), chunk.Size, int64Size)
chunk.Size = uint64(int64Size)
} else {
glog.V(4).Infof("truncated whole chunk %+v\n", chunk.GetFileIdString())
truncatedChunks = append(truncatedChunks, chunk)
}
} else {
chunks = append(chunks, chunk)
}
}
// set the new chunks and reset entry cache
entry.Chunks = chunks
if fh != nil {
fh.entryChunkGroup.SetChunks(chunks)
}
}
entry.Attributes.Mtime = time.Now().Unix()
entry.Attributes.FileSize = size
}
if mode, ok := input.GetMode(); ok {
// commit the file to worm when it is set to readonly at the first time
if entry.WormEnforcedAtTsNs == 0 && wormEnabled && !hasWritePermission(mode) {
entry.WormEnforcedAtTsNs = time.Now().UnixNano()
}
// glog.V(4).Infof("setAttr mode %o", mode)
entry.Attributes.FileMode = chmod(entry.Attributes.FileMode, mode)
if input.NodeId == 1 {
wfs.option.MountMode = os.FileMode(chmod(uint32(wfs.option.MountMode), mode))
}
}
ownerChanged := false
if uid, ok := input.GetUID(); ok {
entry.Attributes.Uid = uid
ownerChanged = true
if input.NodeId == 1 {
wfs.option.MountUid = uid
}
}
if gid, ok := input.GetGID(); ok {
entry.Attributes.Gid = gid
ownerChanged = true
if input.NodeId == 1 {
wfs.option.MountGid = gid
}
}
// POSIX: clear SUID/SGID bits when ownership changes (unless caller is root).
if ownerChanged && input.Uid != 0 {
entry.Attributes.FileMode &^= 0o6000
}
if atime, ok := input.GetATime(); ok {
wfs.setAtime(input.NodeId, atime.Unix())
}
if mtime, ok := input.GetMTime(); ok {
entry.Attributes.Mtime = mtime.Unix()
}
// POSIX: update ctime on any metadata change.
entry.Attributes.Ctime = time.Now().Unix()
out.AttrValid = 1
size, includeSize := input.GetSize()
if includeSize {
out.Attr.Size = size
}
wfs.setAttrByPbEntry(&out.Attr, input.NodeId, entry, !includeSize)
wfs.applyInMemoryAtime(&out.Attr, input.NodeId)
if fh != nil {
fh.dirtyMetadata = true
return fuse.OK
}
return wfs.saveEntry(path, entry)
}
func (wfs *WFS) setRootAttr(out *fuse.AttrOut) {
now := uint64(time.Now().Unix())
out.AttrValid = 119
out.Ino = 1
setBlksize(&out.Attr, blockSize)
out.Uid = wfs.option.MountUid
out.Gid = wfs.option.MountGid
out.Mtime = now
out.Ctime = now
out.Atime = now
out.Mode = toSyscallType(os.ModeDir) | uint32(wfs.option.MountMode)
out.Nlink = 1
}
func (wfs *WFS) setAttrByPbEntry(out *fuse.Attr, inode uint64, entry *filer_pb.Entry, calculateSize bool) {
out.Ino = inode
setBlksize(out, blockSize)
if entry == nil {
return
}
if entry.Attributes != nil && entry.Attributes.Inode != 0 {
out.Ino = entry.Attributes.Inode
}
if calculateSize {
out.Size = filer.FileSize(entry)
}
if entry.FileMode()&os.ModeSymlink != 0 {
out.Size = uint64(len(entry.Attributes.SymlinkTarget))
}
out.Blocks = (out.Size + blockSize - 1) / blockSize
out.Mtime = uint64(entry.Attributes.Mtime)
if entry.Attributes.Ctime != 0 {
out.Ctime = uint64(entry.Attributes.Ctime)
} else {
out.Ctime = uint64(entry.Attributes.Mtime)
}
out.Atime = uint64(entry.Attributes.Mtime)
// In-memory atime overlay is applied by the caller via applyInMemoryAtime.
out.Mode = toSyscallMode(os.FileMode(entry.Attributes.FileMode))
if entry.HardLinkCounter > 0 {
out.Nlink = uint32(entry.HardLinkCounter)
} else {
out.Nlink = 1
}
out.Uid = entry.Attributes.Uid
out.Gid = entry.Attributes.Gid
out.Rdev = entry.Attributes.Rdev
}
func (wfs *WFS) setAttrByFilerEntry(out *fuse.Attr, inode uint64, entry *filer.Entry) {
out.Ino = inode
out.Size = entry.FileSize
if entry.Mode&os.ModeSymlink != 0 {
out.Size = uint64(len(entry.SymlinkTarget))
}
out.Blocks = (out.Size + blockSize - 1) / blockSize
setBlksize(out, blockSize)
out.Atime = uint64(entry.Attr.Mtime.Unix())
out.Mtime = uint64(entry.Attr.Mtime.Unix())
if !entry.Attr.Ctime.IsZero() {
out.Ctime = uint64(entry.Attr.Ctime.Unix())
} else {
out.Ctime = uint64(entry.Attr.Mtime.Unix())
}
out.Mode = toSyscallMode(entry.Attr.Mode)
if entry.HardLinkCounter > 0 {
out.Nlink = uint32(entry.HardLinkCounter)
} else {
out.Nlink = 1
}
out.Uid = entry.Attr.Uid
out.Gid = entry.Attr.Gid
out.Rdev = entry.Attr.Rdev
}
func (wfs *WFS) outputPbEntry(out *fuse.EntryOut, inode uint64, entry *filer_pb.Entry) {
out.NodeId = inode
out.Generation = 1
out.EntryValid = 1
out.AttrValid = 1
wfs.setAttrByPbEntry(&out.Attr, inode, entry, true)
}
func (wfs *WFS) outputFilerEntry(out *fuse.EntryOut, inode uint64, entry *filer.Entry) {
out.NodeId = inode
out.Generation = 1
out.EntryValid = 1
out.AttrValid = 1
wfs.setAttrByFilerEntry(&out.Attr, inode, entry)
}
// touchDirMtimeCtime updates a directory's mtime and ctime on the filer.
// POSIX requires this when entries are created or removed in the directory.
func (wfs *WFS) touchDirMtimeCtime(dirPath util.FullPath) {
dirEntry, code := wfs.maybeLoadEntry(dirPath)
if code != fuse.OK || dirEntry == nil || dirEntry.Attributes == nil {
return
}
now := time.Now().Unix()
dirEntry.Attributes.Mtime = now
dirEntry.Attributes.Ctime = now
wfs.saveEntry(dirPath, dirEntry)
}
const atimeMapMaxSize = 8192
// setAtime stores an in-memory atime for an inode. The map is bounded;
// when full, a random entry is evicted.
func (wfs *WFS) setAtime(inode uint64, t int64) {
wfs.atimeMu.Lock()
defer wfs.atimeMu.Unlock()
if len(wfs.atimeMap) >= atimeMapMaxSize {
// evict one random entry
for k := range wfs.atimeMap {
delete(wfs.atimeMap, k)
break
}
}
wfs.atimeMap[inode] = t
}
// applyInMemoryAtime overlays the in-memory atime onto a fuse.Attr if present.
func (wfs *WFS) applyInMemoryAtime(out *fuse.Attr, inode uint64) {
wfs.atimeMu.Lock()
if t, ok := wfs.atimeMap[inode]; ok {
out.Atime = uint64(t)
}
wfs.atimeMu.Unlock()
}
func chmod(existing uint32, mode uint32) uint32 {
return existing&^07777 | mode&07777
}
const ownerWrite = 0o200
const groupWrite = 0o020
const otherWrite = 0o002
func hasWritePermission(mode uint32) bool {
return (mode&ownerWrite != 0) || (mode&groupWrite != 0) || (mode&otherWrite != 0)
}
func toSyscallMode(mode os.FileMode) uint32 {
return toSyscallType(mode) | uint32(mode)
}
func toSyscallType(mode os.FileMode) uint32 {
switch mode & os.ModeType {
case os.ModeDir:
return syscall.S_IFDIR
case os.ModeSymlink:
return syscall.S_IFLNK
case os.ModeNamedPipe:
return syscall.S_IFIFO
case os.ModeSocket:
return syscall.S_IFSOCK
case os.ModeDevice:
return syscall.S_IFBLK
case os.ModeCharDevice:
return syscall.S_IFCHR
default:
return syscall.S_IFREG
}
}
func toOsFileType(mode uint32) os.FileMode {
switch mode & (syscall.S_IFMT & 0xffff) {
case syscall.S_IFDIR:
return os.ModeDir
case syscall.S_IFLNK:
return os.ModeSymlink
case syscall.S_IFIFO:
return os.ModeNamedPipe
case syscall.S_IFSOCK:
return os.ModeSocket
case syscall.S_IFBLK:
return os.ModeDevice
case syscall.S_IFCHR:
return os.ModeCharDevice
default:
return 0
}
}
func toOsFileMode(mode uint32) os.FileMode {
return toOsFileType(mode) | os.FileMode(mode&07777)
}