mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 13:30:46 +02:00
copy
chrislusf
@@ -0,0 +1,172 @@
|
||||
# File Operations Quick Reference
|
||||
|
||||
This page provides a quick reference for common file operations in SeaweedFS using the HTTP API.
|
||||
|
||||
## Basic File Operations
|
||||
|
||||
### Upload a File
|
||||
|
||||
**Small files (direct upload):**
|
||||
```bash
|
||||
# Upload with PUT
|
||||
curl -X PUT "http://localhost:8888/path/to/file.txt" -d "file content"
|
||||
|
||||
# Upload with POST (multipart)
|
||||
curl -X POST "http://localhost:8888/path/to/file.txt" -F "file=@localfile.txt"
|
||||
```
|
||||
|
||||
### Download a File
|
||||
|
||||
```bash
|
||||
# Get file content
|
||||
curl "http://localhost:8888/path/to/file.txt"
|
||||
|
||||
# Download to local file
|
||||
curl "http://localhost:8888/path/to/file.txt" -o localfile.txt
|
||||
```
|
||||
|
||||
### Delete a File
|
||||
|
||||
```bash
|
||||
# Delete single file
|
||||
curl -X DELETE "http://localhost:8888/path/to/file.txt"
|
||||
|
||||
# Delete directory (recursive)
|
||||
curl -X DELETE "http://localhost:8888/path/to/directory/?recursive=true"
|
||||
```
|
||||
|
||||
## Advanced File Operations
|
||||
|
||||
### Move/Rename Files
|
||||
|
||||
```bash
|
||||
# Move file to new location
|
||||
curl -X POST "http://localhost:8888/new/path/file.txt?mv.from=/old/path/file.txt"
|
||||
|
||||
# Rename file in same directory
|
||||
curl -X POST "http://localhost:8888/path/to/newname.txt?mv.from=/path/to/oldname.txt"
|
||||
```
|
||||
|
||||
### Copy Files ⭐ **New Feature**
|
||||
|
||||
```bash
|
||||
# Copy file (preserves original)
|
||||
curl -X POST "http://localhost:8888/backup/file.txt?cp.from=/original/file.txt"
|
||||
|
||||
# Copy with automatic name resolution
|
||||
curl -X POST "http://localhost:8888/backup/?cp.from=/original/file.txt"
|
||||
# Result: /backup/file.txt
|
||||
```
|
||||
|
||||
### List Directory Contents
|
||||
|
||||
```bash
|
||||
# List directory
|
||||
curl -H "Accept: application/json" "http://localhost:8888/path/to/directory/?pretty=y"
|
||||
|
||||
# List with pagination
|
||||
curl -H "Accept: application/json" "http://localhost:8888/path/?limit=10&lastFileName=somefile.txt"
|
||||
```
|
||||
|
||||
### Create Directory
|
||||
|
||||
```bash
|
||||
# Create empty directory
|
||||
curl -X POST "http://localhost:8888/path/to/new/directory/"
|
||||
```
|
||||
|
||||
## File Metadata Operations
|
||||
|
||||
### Get File Information
|
||||
|
||||
```bash
|
||||
# Get file metadata
|
||||
curl -I "http://localhost:8888/path/to/file.txt"
|
||||
```
|
||||
|
||||
### File Attributes and Tagging
|
||||
|
||||
```bash
|
||||
# Set custom attributes
|
||||
curl -X PUT "http://localhost:8888/path/to/file.txt" \
|
||||
-H "Seaweed-Custom-Attribute: value" \
|
||||
-F "file=@localfile.txt"
|
||||
|
||||
# Set TTL (time to live)
|
||||
curl -X POST "http://localhost:8888/path/to/file.txt?ttl=3600" \
|
||||
-F "file=@localfile.txt"
|
||||
```
|
||||
|
||||
## Response Codes
|
||||
|
||||
| HTTP Code | Operation | Meaning |
|
||||
| --------- | --------- | ------- |
|
||||
| 200 OK | GET | File retrieved successfully |
|
||||
| 201 Created | POST/PUT | File uploaded successfully |
|
||||
| 204 No Content | DELETE, COPY, MOVE | Operation completed successfully |
|
||||
| 400 Bad Request | Any | Invalid parameters or unsupported operation |
|
||||
| 404 Not Found | GET, DELETE | File or directory not found |
|
||||
| 409 Conflict | POST/PUT | File already exists (when using exclusive creation) |
|
||||
|
||||
## Operation Comparison
|
||||
|
||||
| Operation | Source File | Use Case | Performance |
|
||||
| --------- | ----------- | -------- | ----------- |
|
||||
| **Upload** | External → SeaweedFS | Add new files | Network dependent |
|
||||
| **Download** | SeaweedFS → External | Retrieve files | Network dependent |
|
||||
| **Move** (`mv.from`) | Deleted | Rename/relocate | Very fast (metadata only) |
|
||||
| **Copy** (`cp.from`) | Preserved | Backup/duplicate | Depends on file size |
|
||||
| **Delete** | Deleted | Remove files | Fast |
|
||||
|
||||
## Best Practices
|
||||
|
||||
### When to Use Copy vs Move
|
||||
|
||||
**Use Copy (`cp.from`) when:**
|
||||
- Creating backups before modifications
|
||||
- Duplicating configuration templates
|
||||
- Staging files for testing
|
||||
- Need to preserve original file
|
||||
|
||||
**Use Move (`mv.from`) when:**
|
||||
- Renaming files
|
||||
- Reorganizing file structure
|
||||
- Moving files between directories
|
||||
- Don't need original file
|
||||
|
||||
### Performance Tips
|
||||
|
||||
1. **Batch operations**: Group multiple operations when possible
|
||||
2. **Small files**: Use direct PUT for files < 1MB
|
||||
3. **Large files**: POST with multipart automatically chunks files
|
||||
4. **Copy operations**: Server-side copy is much faster than download + upload
|
||||
5. **Directory operations**: Plan directory structure to minimize reorganization
|
||||
|
||||
### Error Handling
|
||||
|
||||
```bash
|
||||
# Check operation status
|
||||
if curl -X POST "http://localhost:8888/dest?cp.from=/src" -w "%{http_code}" -o /dev/null -s | grep -q "204"; then
|
||||
echo "Copy successful"
|
||||
else
|
||||
echo "Copy failed"
|
||||
fi
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Filer Server API](Filer-Server-API.md) - Complete API reference
|
||||
- [Filer Commands and Operations](Filer-Commands-and-Operations.md) - Command-line tools
|
||||
- [Getting Started](Getting-Started.md) - Installation and setup
|
||||
- [FAQ](FAQ.md) - Common questions and answers
|
||||
|
||||
## Examples Repository
|
||||
|
||||
For more complex examples and use cases, see:
|
||||
- [SeaweedFS Examples](https://github.com/seaweedfs/seaweedfs/tree/master/examples) (if available)
|
||||
- Community-contributed scripts and tools
|
||||
- Language-specific client libraries
|
||||
|
||||
---
|
||||
|
||||
💡 **Tip**: Use the `?pretty=y` parameter with JSON responses to get formatted output for easier reading during development and testing.
|
||||
@@ -23,6 +23,36 @@ The above `weed copy` command is very efficient. It will contact the master serv
|
||||
|
||||
This put very little loads on filer and the master server. Data is only transmitted between the local machine and the volume server.
|
||||
|
||||
## Copy files within Filer
|
||||
|
||||
SeaweedFS also supports copying files within the filer using the HTTP API. This is useful for creating backups, duplicates, or templates without downloading and re-uploading files.
|
||||
|
||||
### HTTP API Copy
|
||||
|
||||
```bash
|
||||
# Copy a file to a new location
|
||||
curl -X POST 'http://localhost:8888/path/to/destination?cp.from=/path/to/source'
|
||||
|
||||
# Copy with automatic name resolution
|
||||
curl -X POST 'http://localhost:8888/backup/?cp.from=/important/config.json'
|
||||
# Creates: /backup/config.json
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- **Efficient**: Server-side copy without client data transfer
|
||||
- **Independent chunks**: Creates new chunk copies (not shared references)
|
||||
- **Atomic operation**: Either succeeds completely or fails with no partial state
|
||||
- **Preserves metadata**: File attributes, timestamps, and permissions maintained
|
||||
|
||||
**Comparison of copy methods:**
|
||||
|
||||
| Method | Use Case | Data Transfer | Performance |
|
||||
| ------ | -------- | ------------- | ----------- |
|
||||
| `weed filer.copy` | Local files → Filer | Client → Volume Server | Good for initial uploads |
|
||||
| `cp.from` HTTP API | File → File within Filer | Volume Server → Volume Server | Excellent for server-side copies |
|
||||
|
||||
For more details, see the [Filer Server API documentation](Filer-Server-API.md#copy-files).
|
||||
|
||||
## Register a file on Filer
|
||||
|
||||
As mentioned above, the (path, fileId, fileSize) can be registered on filer with this gRPC call.
|
||||
|
||||
+123
@@ -237,6 +237,129 @@ Notice that the tag names follow http header key convention, with the first char
|
||||
| ---- | -- | -- |
|
||||
| mv.from | move from one file or directory to another location | Required field |
|
||||
|
||||
### Copy files
|
||||
|
||||
SeaweedFS supports efficient file copying using the `cp.from` parameter. This operation creates a complete copy of a file while preserving the original file.
|
||||
|
||||
#### Basic Usage
|
||||
```bash
|
||||
# Copy a file to the same directory with a new name
|
||||
> curl -X POST 'http://localhost:8888/documents/report_backup.pdf?cp.from=/documents/report.pdf'
|
||||
|
||||
# Copy a file to a different directory
|
||||
> curl -X POST 'http://localhost:8888/backup/important.txt?cp.from=/projects/important.txt'
|
||||
|
||||
# Copy with automatic name resolution (uses source filename)
|
||||
> curl -X POST 'http://localhost:8888/backup/?cp.from=/projects/important.txt'
|
||||
# Creates: /backup/important.txt
|
||||
```
|
||||
|
||||
#### Parameters
|
||||
| POST Parameter | Description | Default |
|
||||
| ---- | -- | -- |
|
||||
| cp.from | Source file path to copy from. Must be a valid file path. | Required field |
|
||||
|
||||
#### How Copy Works
|
||||
|
||||
**Small Files (< chunk size):**
|
||||
- Content is stored directly in the filer metadata
|
||||
- Copy operation duplicates the content bytes
|
||||
- Very fast, only metadata operation required
|
||||
|
||||
**Large Files (chunked):**
|
||||
- File data is stored as chunks on volume servers
|
||||
- Copy operation reads data from source chunks and writes to new chunks
|
||||
- Creates independent chunk copies (not shared references)
|
||||
- Preserves file integrity and allows independent deletion
|
||||
|
||||
#### Examples
|
||||
|
||||
**Copy a configuration file:**
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8888/config/app.conf.backup?cp.from=/config/app.conf'
|
||||
```
|
||||
|
||||
**Copy a large media file:**
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8888/media/backup/video.mp4?cp.from=/media/original/video.mp4'
|
||||
```
|
||||
|
||||
**Copy with path resolution:**
|
||||
```bash
|
||||
# If destination ends with /, uses source filename
|
||||
curl -X POST 'http://localhost:8888/backup/?cp.from=/important/data.json'
|
||||
# Result: /backup/data.json
|
||||
```
|
||||
|
||||
#### Response Codes
|
||||
|
||||
| HTTP Code | Description |
|
||||
| ---- | -- |
|
||||
| 204 No Content | Copy operation completed successfully |
|
||||
| 400 Bad Request | Invalid source path, missing cp.from parameter, or directory copy attempt |
|
||||
| 404 Not Found | Source file does not exist |
|
||||
| 500 Internal Server Error | Volume server error or chunk copy failure |
|
||||
|
||||
#### Performance Characteristics
|
||||
|
||||
- **Small files**: Near-instant (metadata only)
|
||||
- **Large files**: Proportional to file size (requires data transfer)
|
||||
- **Network efficient**: Direct volume-to-volume transfer when possible
|
||||
- **Atomic operation**: Either completes fully or fails with no partial state
|
||||
|
||||
#### Limitations
|
||||
|
||||
- **Directory copying**: Not supported (returns 400 error)
|
||||
- **Cross-cluster copying**: Limited to same SeaweedFS cluster
|
||||
- **Concurrent access**: Source file should not be modified during copy
|
||||
|
||||
#### Error Examples
|
||||
|
||||
**Attempting to copy a directory:**
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8888/new_folder/?cp.from=/existing_folder/'
|
||||
# Returns: 400 Bad Request - "directory copying not yet supported"
|
||||
```
|
||||
|
||||
**Source file not found:**
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8888/copy.txt?cp.from=/nonexistent.txt'
|
||||
# Returns: 400 Bad Request - "failed to get src entry"
|
||||
```
|
||||
|
||||
**Missing cp.from parameter:**
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8888/copy.txt'
|
||||
# Returns: Normal file upload behavior (not a copy operation)
|
||||
```
|
||||
|
||||
#### Comparison with Move Operation
|
||||
|
||||
| Operation | Source File | Use Case | Speed |
|
||||
| ---- | ---- | ---- | ---- |
|
||||
| `mv.from` | Deleted | Rename/relocate files | Very fast (metadata only) |
|
||||
| `cp.from` | Preserved | Backup/duplicate files | Depends on file size |
|
||||
|
||||
#### Best Practices
|
||||
|
||||
1. **Backup workflows**: Use copy for creating backups before modifications
|
||||
2. **Template files**: Copy configuration templates to create new instances
|
||||
3. **Data migration**: Copy files before cross-cluster transfers
|
||||
4. **Testing**: Copy production files to staging environments
|
||||
|
||||
#### Shell Command Equivalent
|
||||
|
||||
The SeaweedFS copy operation is similar to:
|
||||
```bash
|
||||
# SeaweedFS copy
|
||||
curl -X POST 'http://localhost:8888/dst?cp.from=/src'
|
||||
|
||||
# Unix equivalent
|
||||
cp /src /dst
|
||||
```
|
||||
|
||||
Note: Directory copying is not currently supported. Only individual files can be copied.
|
||||
|
||||
### Create an empty folder
|
||||
Folders usually are created automatically when uploading a file. To create an empty file, you can use this:
|
||||
```
|
||||
|
||||
+1
@@ -24,6 +24,7 @@
|
||||
### Filer
|
||||
* [[Filer Setup]]
|
||||
* [[Directories and Files]]
|
||||
* [[File Operations Quick Reference]]
|
||||
* [[Data Structure for Large Files]]
|
||||
* [[Filer Data Encryption]]
|
||||
* [[Filer Commands and Operations]]
|
||||
|
||||
Reference in New Issue
Block a user