mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-10-24 15:00:43 +02:00
NFS-Server: add wiki page for experimental NFSv3 server
Documents the weed nfs command, its configuration surface (safer defaults, allowedClients, readOnly, volumeServerAccess), architecture notes on deterministic filehandles + inode secondary index + metadata subscription, client mount syntax for Linux and macOS, and the current limitations (no shared locks, 64 MiB buffered write cap, no portmapper registration). Also links the new page into the sidebar next to SFTP Server and WebDAV.
Chris Lu committed
1 parent
ace91dc135
commit
e6e0bbdca9
2 files changed
+238
No files matched your search
+236
@@ -0,0 +1,236 @@
|
||||
# NFS Server (experimental)
|
||||
|
||||
SeaweedFS ships an experimental NFSv3 server that exports a filer subtree
|
||||
over the NFS wire protocol. Unlike running `weed mount` and re-exporting
|
||||
the FUSE mount with an external NFS server, the NFS frontend talks to the
|
||||
filer directly. Filehandles are deterministic, metadata operations go
|
||||
through filer RPCs, and chunk reads go to volume servers using the same
|
||||
direct-access rules as the FUSE mount.
|
||||
|
||||
The code lives under [`weed/server/nfs/`](https://github.com/seaweedfs/seaweedfs/tree/master/weed/server/nfs).
|
||||
Status: experimental, safe for evaluation and development, not yet
|
||||
recommended for production workloads that need shared locking or
|
||||
whole-file rewrites larger than 64 MiB.
|
||||
|
||||
## Overview
|
||||
|
||||
The NFS server provides:
|
||||
|
||||
- **NFSv3 wire protocol** served directly from SeaweedFS
|
||||
- **Deterministic filehandles** that survive filer restarts and
|
||||
multi-head failover by encoding the export ID, the filer-assigned
|
||||
inode, and a per-inode generation
|
||||
- **Filer-backed metadata operations** — lookup, readdir, create,
|
||||
mkdir, rename, remove, symlink, readlink, and the hardlink RPC
|
||||
- **Direct volume-server data path** for chunk reads (with optional
|
||||
`publicUrl` or `filerProxy` routing)
|
||||
- **Buffered large-file writes** up to 64 MiB backed by
|
||||
`AssignVolume` and volume-server chunk uploads
|
||||
- **Export access controls** via CIDR or hostname allowlist
|
||||
- **Metadata subscription-based cache invalidation** for each NFS head
|
||||
- **Read-only export mode** for safe serving of an existing subtree
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Starting the NFS Server
|
||||
|
||||
```bash
|
||||
weed nfs \
|
||||
-filer=localhost:8888 \
|
||||
-port=2049 \
|
||||
-filer.path=/nfs
|
||||
```
|
||||
|
||||
`weed nfs` is a standalone binary today. The `filer.path` subtree must
|
||||
already exist in the filer namespace — the server does not create the
|
||||
export root on its own. Create the directory first (for example, by
|
||||
uploading a placeholder file under it with the filer HTTP API) if it
|
||||
does not already exist.
|
||||
|
||||
### Mounting from a Client
|
||||
|
||||
Linux:
|
||||
|
||||
```bash
|
||||
sudo mount -t nfs -o nfsvers=3,proto=tcp,port=2049,mountport=2049 \
|
||||
127.0.0.1:/nfs /mnt/seaweedfs-nfs
|
||||
```
|
||||
|
||||
macOS:
|
||||
|
||||
```bash
|
||||
sudo mount -t nfs \
|
||||
-o nfsvers=3,tcp,port=2049,mountport=2049,noresvport \
|
||||
127.0.0.1:/nfs /Volumes/seaweedfs-nfs
|
||||
```
|
||||
|
||||
Both `port=` and `mountport=` have to be set because the NFS server
|
||||
does not register with the system portmapper — the kernel client would
|
||||
otherwise look on port 111 for the MOUNT program.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Command-Line Options
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `-filer` | `localhost:8888` | Filer server address (ip:port) |
|
||||
| `-ip.bind` | `127.0.0.1` | IP address to bind to. Defaults to loopback so the experimental server is not reachable from other hosts unless you override it. |
|
||||
| `-port` | `2049` | NFS server listen port |
|
||||
| `-filer.path` | (required) | Remote filer path to export. No default — operators must opt in to exporting a namespace subtree. |
|
||||
| `-readOnly` | `false` | Export the filer path as read-only |
|
||||
| `-allowedClients` | (empty) | Comma-separated client IPs, hostnames, or CIDR blocks allowed to connect |
|
||||
| `-volumeServerAccess` | `direct` | How the NFS server reaches volume servers: `direct`, `publicUrl`, or `filerProxy` |
|
||||
|
||||
### Safer Defaults
|
||||
|
||||
Because export ACLs and full POSIX lock support are not yet available,
|
||||
the server ships with defensive defaults:
|
||||
|
||||
- `-ip.bind` defaults to `127.0.0.1`. You must set it explicitly (for
|
||||
example `-ip.bind=0.0.0.0` or a specific interface IP) to expose the
|
||||
experimental server to the network.
|
||||
- `-filer.path` has no default. `weed nfs` refuses to start without it,
|
||||
to prevent an accidental full-namespace export. Setting
|
||||
`-filer.path=/` still works, but the server logs a warning because
|
||||
exporting the entire filer namespace with no ACLs in place is almost
|
||||
never what you want.
|
||||
|
||||
### Client Allowlist
|
||||
|
||||
```bash
|
||||
weed nfs \
|
||||
-filer=localhost:8888 \
|
||||
-filer.path=/nfs \
|
||||
-allowedClients=10.0.0.0/8,203.0.113.42,nfs-client.example.com
|
||||
```
|
||||
|
||||
Entries may be:
|
||||
|
||||
- an IPv4 or IPv6 literal — matched against the client's remote
|
||||
address at connection time
|
||||
- a CIDR block — matched against the client's remote address
|
||||
- a hostname — resolved to IPs at startup and those IPs are then
|
||||
matched. Resolution happens once; the server does not re-resolve
|
||||
later.
|
||||
|
||||
An unmatched client gets a TCP-level close before the MOUNT RPC is
|
||||
served.
|
||||
|
||||
### Read-Only Export
|
||||
|
||||
```bash
|
||||
weed nfs -filer=localhost:8888 -filer.path=/nfs -readOnly
|
||||
```
|
||||
|
||||
In read-only mode every mutating RPC returns `NFS3ERR_ROFS`. The
|
||||
server still uses the deterministic filehandle format and metadata
|
||||
subscription invalidation, so it is safe to run a read-only replica
|
||||
next to a read-write head.
|
||||
|
||||
### Volume Server Access Modes
|
||||
|
||||
The NFS server reads chunks directly from volume servers by default
|
||||
(`-volumeServerAccess=direct`), matching the `weed mount` default. Use
|
||||
`publicUrl` to route through the volume servers' public URLs, or
|
||||
`filerProxy` to route chunk reads through the filer host when volume
|
||||
servers are not reachable from the NFS host.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Deterministic Filehandles
|
||||
|
||||
Every filer entry in this PR now carries a filer-assigned inode
|
||||
(Snowflake-style, unique per filer and stable across restarts). The
|
||||
NFS server encodes its filehandles as:
|
||||
|
||||
```
|
||||
exportID || filehandleKind || inode || generation
|
||||
```
|
||||
|
||||
- `exportID` is a CRC32 of the export root path, so a handle minted by
|
||||
one export cannot resolve inside another.
|
||||
- `filehandleKind` distinguishes files, directories, and symlinks.
|
||||
- `inode` is the filer-assigned inode, so the handle stays valid
|
||||
across rename / move as long as the entry exists.
|
||||
- `generation` is the inode secondary index generation counter.
|
||||
Handles from a deleted-and-recreated path decode as stale and the
|
||||
server returns `NFS3ERR_STALE` instead of handing the client the
|
||||
wrong object.
|
||||
|
||||
### Inode Secondary Index
|
||||
|
||||
The filer writes an inode secondary index record next to every entry
|
||||
it creates. The record maps inode -> current paths (multi-path for
|
||||
hardlinks) and carries a generation counter that bumps on
|
||||
delete/recreate. Handle resolution in the NFS server goes through this
|
||||
index so it never has to walk the namespace.
|
||||
|
||||
Secondary index updates are best-effort: if the KV write fails, the
|
||||
primary store mutation is still visible and the index is rebuilt on
|
||||
the next write to the same inode.
|
||||
|
||||
### Metadata Subscription
|
||||
|
||||
Each NFS head subscribes to filer metadata change events via the
|
||||
filer's `SubscribeMetadata` stream starting from the export root.
|
||||
Chunk entries cached in the shared reader cache get invalidated as
|
||||
soon as the filer confirms a rename or delete, so a write on one head
|
||||
becomes visible to a reader on another head without waiting for a
|
||||
cache TTL.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Experimental.** The on-wire protocol and filehandle codec are
|
||||
stable inside a single server binary, but the server has not yet
|
||||
been stress-tested as a production NFS endpoint.
|
||||
- **Shared locks are not implemented.** The upstream
|
||||
`github.com/willscott/go-nfs` backend exposes only local billy
|
||||
`File.Lock` / `Unlock` hooks. It does not implement the NLM or
|
||||
NFSv4 lock state RPCs, so cluster-wide locks, lock reclaim, and
|
||||
grace periods are not available yet. The server advertises no
|
||||
lock capability and returns `ErrNotSupported` for NLM requests.
|
||||
- **Write size cap of 64 MiB per file.** Writes to an existing file
|
||||
are buffered in the NFS server memory up to `maxBufferedWriteSize`
|
||||
(64 MiB). Larger whole-file rewrites return `NFS3ERR_NOTSUPP`. A
|
||||
streaming WRITE path is tracked as follow-up work.
|
||||
- **Single-chunk persistence for buffered writes.** Buffered writes
|
||||
larger than `maxInlineWriteSize` (4 MiB) are flushed as a single
|
||||
chunk rather than split on SeaweedFS chunk boundaries. A future
|
||||
refactor will route writes through the filer's standard chunking
|
||||
logic.
|
||||
- **No portmapper registration.** Clients must pass `port=` and
|
||||
`mountport=` explicitly because the server does not register with
|
||||
the system rpcbind.
|
||||
|
||||
## Testing
|
||||
|
||||
End-to-end integration tests live in
|
||||
[`test/nfs`](https://github.com/seaweedfs/seaweedfs/tree/master/test/nfs).
|
||||
Each test boots its own master + volume + filer + nfs subprocess
|
||||
cluster on loopback and drives it through the NFSv3 RPC protocol via
|
||||
`github.com/willscott/go-nfs-client`. No kernel NFS mount or
|
||||
privileged port is required, which means the suite runs in CI on
|
||||
GitHub Actions directly without host network configuration.
|
||||
|
||||
```bash
|
||||
cd test/nfs
|
||||
make test
|
||||
```
|
||||
|
||||
Covered scenarios include read/write round-trip, directory operations,
|
||||
nested mkdir, rename content preservation, overwrite with explicit
|
||||
truncate, large (3 MiB) binary files, arbitrary byte payloads, empty
|
||||
files, symlinks, readdir ordering, remove-missing error propagation,
|
||||
FSINFO limit sanity, sequential append writes, and readdir after
|
||||
remove.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[FUSE Mount]] — the original filesystem-in-userspace mount that
|
||||
shares the same filer-backed data path
|
||||
- [[WebDAV]] — a similar gateway over a different protocol
|
||||
- [[SFTP Server]] — SSH/SFTP gateway
|
||||
- [[Filer Commands and Operations]] — filer operations reference
|
||||
- [NFS design notes](https://github.com/seaweedfs/seaweedfs/blob/master/weed/server/nfs/DESIGN.md) and
|
||||
[development plan](https://github.com/seaweedfs/seaweedfs/blob/master/weed/server/nfs/DEV_PLAN.md) in the repo
|
||||
+2
@@ -64,6 +64,8 @@
|
||||
|
||||
### [[SFTP Server]]
|
||||
|
||||
### [[NFS Server]]
|
||||
|
||||
### Cloud Drive
|
||||
* [[Cloud Drive Benefits]]
|
||||
* [[Cloud Drive Architecture]]
|
||||
|
||||
Reference in new issue
Block a user