Updated Hadoop Compatible File System (markdown)

Chris Lu
2018-12-11 15:10:06 -08:00
parent 308a54e9f1
commit 27b5aea48d
+51 -4
@@ -8,13 +8,13 @@ $cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/client
$ mvn install
$cd $GOPATH/src/github.com/chrislusf/seaweedfs/other/java/hdfs
$ mvn package
$ ls -al target/seaweedfs-hadoop-client-1.0.1.jar
$ ls -al target/seaweedfs-hadoop-client-1.0.2.jar
```
Or you can download the latest version from MavenCentral
https://mvnrepository.com/artifact/com.github.chrislusf/seaweedfs-hadoop-client
https://oss.sonatype.org/content/repositories/public/com/github/chrislusf/seaweedfs-hadoop-client/1.0.1/seaweedfs-hadoop-client-1.0.1.jar
https://oss.sonatype.org/content/repositories/public/com/github/chrislusf/seaweedfs-hadoop-client/1.0.2/seaweedfs-hadoop-client-1.0.2.jar
# Test SeaweedFS on Hadoop
@@ -29,7 +29,7 @@ $ cd ${HADOOP_HOME}
$ echo "<configuration></configuration>" > etc/hadoop/mapred-site.xml
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop-client-1.0.1.jar \
-libjars ./seaweedfs-hadoop-client-1.0.2.jar \
-ls /
```
@@ -59,7 +59,7 @@ $ bin/hadoop classpath
# Copy SeaweedFS HDFS client jar to one of the folders
$ cd ${HADOOP_HOME}
$ cp ./seaweedfs-hadoop-client-1.0.1.jar share/hadoop/common/lib/
$ cp ./seaweedfs-hadoop-client-1.0.2.jar share/hadoop/common/lib/
```
Now you can do this:
@@ -83,6 +83,53 @@ If HBase is used, create a folder and configure the HBase root directory in `etc
</property>
</configuration>
```
# Supported Operations
```
bin/hdfs dfs -appendToFile README.txt /weedfs/weedfs.txt
bin/hdfs dfs -cat /weedfs/weedfs.txt
bin/hdfs dfs -rm -r /uber
bin/hdfs dfs -chown -R chris:chris /weedfs
bin/hdfs dfs -chmod -R 755 /weedfs
bin/hdfs dfs -copyFromLocal README.txt /weedfs/README.txt.2
bin/hdfs dfs -copyToLocal /weedfs/README.txt.2 .
bin/hdfs dfs -count /weedfs/README.txt.2
bin/hdfs dfs -cp /weedfs/README.txt.2 /weedfs/README.txt.3
bin/hdfs dfs -du -h /weedfs
bin/hdfs dfs -find /weedfs -name "*.txt" -print
bin/hdfs dfs -get /weedfs/weedfs.txt
bin/hdfs dfs -getfacl /weedfs
bin/hdfs dfs -getmerge -nl /weedfs w.txt
bin/hdfs dfs -ls /
bin/hdfs dfs -mkdir /tmp
bin/hdfs dfs -mkdir -p /tmp/x/y
bin/hdfs dfs -moveFromLocal README.txt.2 /tmp/x/
bin/hdfs dfs -mv /tmp/x/y/README.txt.2 /tmp/x/y/README.txt.3
bin/hdfs dfs -mv /tmp/x /tmp/z
bin/hdfs dfs -put README.txt /tmp/z/y/
bin/hdfs dfs -rm /tmp/z/y/*
bin/hdfs dfs -rmdir /tmp/z/y
bin/hdfs dfs -stat /weedfs
bin/hdfs dfs -tail /weedfs/weedfs.txt
bin/hdfs dfs -test -f /weedfs/weedfs.txt
bin/hdfs dfs -text /weedfs/weedfs.txt
bin/hdfs dfs -touchz /weedfs/weedfs.txtx
```
## Operations Plan to Support
```
getfattr
setfacl
setfattr
truncate
```
## Operations not planned to Support
```
createSnapshot
deleteSnapshot
renameSnapshot
setrep
```
# Notes
## No native shared libraries