Github-Mirror/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-09-19 04:50:54 +02:00
Code Issues Packages Projects Releases Wiki Activity

java clients 4.38

github-actions[bot]
2026-07-08 06:44:08 +00:00
parent 8ee7f826d6
commit 1e5efbeec3
6 changed files with 22 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Hadoop-Benchmark.md
+1 -1
@@ -26,7 +26,7 @@ Then get the seaweedfs hadoop client jar.
```
cd share/hadoop/common/lib/
wget https://oss.sonatype.org/service/local/repositories/releases/content/com/seaweedfs/seaweedfs-hadoop3-client/4.00/seaweedfs-hadoop3-client-4.00.jar
wget https://oss.sonatype.org/service/local/repositories/releases/content/com/seaweedfs/seaweedfs-hadoop3-client/4.38/seaweedfs-hadoop3-client-4.38.jar
```
# TestDFSIO Benchmark
Hadoop-Compatible-File-System.md
+6 -6
@@ -10,7 +10,7 @@ $ mvn install
# build for hadoop3
$cd $GOPATH/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3
$ mvn package
$ ls -al target/seaweedfs-hadoop3-client-4.00.jar
$ ls -al target/seaweedfs-hadoop3-client-4.38.jar
```
Maven
@@ -18,20 +18,20 @@ Maven
<dependency>
<groupId>com.seaweedfs</groupId>
<artifactId>seaweedfs-hadoop3-client</artifactId>
<version>4.00</version>
<version>4.38</version>
</dependency>
```
Or you can download the latest version from MavenCentral
* https://central.sonatype.com/artifact/com.seaweedfs/seaweedfs-hadoop3-client
* [seaweedfs-hadoop3-client-4.00.jar](https://repo1.maven.org/maven2/com/seaweedfs/seaweedfs-hadoop3-client/4.00/seaweedfs-hadoop3-client-4.00.jar)
* [seaweedfs-hadoop3-client-4.38.jar](https://repo1.maven.org/maven2/com/seaweedfs/seaweedfs-hadoop3-client/4.38/seaweedfs-hadoop3-client-4.38.jar)
# Test SeaweedFS on Hadoop
Suppose you are getting a new Hadoop installation. Here are the minimum steps to get SeaweedFS to run.
You would need to start a weed filer first, build the seaweedfs-hadoop3-client-4.00.jar, and do the following:
You would need to start a weed filer first, build the seaweedfs-hadoop3-client-4.38.jar, and do the following:
```
# optionally adjust hadoop memory allocation
@@ -44,7 +44,7 @@ $ echo "<configuration></configuration>" > etc/hadoop/mapred-site.xml
# on hadoop3
$ bin/hdfs dfs -Dfs.defaultFS=seaweedfs://localhost:8888 \
-Dfs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem \
-libjars ./seaweedfs-hadoop3-client-4.00.jar \
-libjars ./seaweedfs-hadoop3-client-4.38.jar \
-ls /
```
@@ -120,7 +120,7 @@ $ bin/hadoop classpath
# Copy SeaweedFS HDFS client jar to one of the folders
$ cd ${HADOOP_HOME}
# for hadoop3
$ cp ./seaweedfs-hadoop3-client-4.00.jar share/hadoop/common/lib/
$ cp ./seaweedfs-hadoop3-client-4.38.jar share/hadoop/common/lib/
```
Now you can do this:
Run-Presto-on-SeaweedFS.md
+5 -5
@@ -5,10 +5,10 @@ The installation steps are divided into 2 steps:
* https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+Administration
### Configure Hive Metastore to support SeaweedFS
1. Copy the seaweedfs-hadoop3-client-4.00.jar to hive lib directory,for example:
1. Copy the seaweedfs-hadoop3-client-4.38.jar to hive lib directory,for example:
```
cp seaweedfs-hadoop3-client-4.00.jar /opt/hadoop/share/hadoop/common/lib/
cp seaweedfs-hadoop3-client-4.00.jar /opt/hive-metastore/lib/
cp seaweedfs-hadoop3-client-4.38.jar /opt/hadoop/share/hadoop/common/lib/
cp seaweedfs-hadoop3-client-4.38.jar /opt/hive-metastore/lib/
```
2. Modify core-site.xml
modify core-site.xml to support SeaweedFS, 30888 is the filer port
@@ -50,9 +50,9 @@ metastore.thrift.port is the access port exposed by the Hive Metadata service it
Follow instructions for installation of Presto:
* https://prestosql.io/docs/current/installation/deployment.html
### Configure Presto to support SeaweedFS
1. Copy the seaweedfs-hadoop3-client-4.00.jar to Presto directory,for example:
1. Copy the seaweedfs-hadoop3-client-4.38.jar to Presto directory,for example:
```
cp seaweedfs-hadoop3-client-4.00.jar /opt/presto-server-347/plugin/hive-hadoop3/
cp seaweedfs-hadoop3-client-4.38.jar /opt/presto-server-347/plugin/hive-hadoop3/
```
2. Modify core-site.xml
SeaweedFS-Java-Client.md
+2 -2
@@ -13,7 +13,7 @@ $ mvn install
Gradle
```gradle
implementation 'com.seaweedfs:seaweedfs-client:4.00'
implementation 'com.seaweedfs:seaweedfs-client:4.38'
```
Maven
@@ -21,7 +21,7 @@ Maven
<dependency>
<groupId>com.seaweedfs</groupId>
<artifactId>seaweedfs-client</artifactId>
<version>4.00</version>
<version>4.38</version>
</dependency>
```
Or you can download the latest version from MavenCentral
run-HBase-on-SeaweedFS.md
+1 -1
@@ -1,7 +1,7 @@
# Installation for HBase
Two steps to run HBase on SeaweedFS
1. Copy the seaweedfs-hadoop3-client-4.00.jar to `${HBASE_HOME}/lib`
1. Copy the seaweedfs-hadoop3-client-4.38.jar to `${HBASE_HOME}/lib`
1. And add the following 2 properties in `${HBASE_HOME}/conf/hbase-site.xml`
```
run-Spark-on-SeaweedFS.md
+7 -7
@@ -11,12 +11,12 @@ To make these files visible to Spark, set HADOOP_CONF_DIR in $SPARK_HOME/conf/sp
## installation not inheriting from Hadoop cluster configuration
Copy the seaweedfs-hadoop3-client-4.00.jar to all executor machines.
Copy the seaweedfs-hadoop3-client-4.38.jar to all executor machines.
Add the following to spark/conf/spark-defaults.conf on every node running Spark
```
spark.driver.extraClassPath=/path/to/seaweedfs-hadoop3-client-4.00.jar
spark.executor.extraClassPath=/path/to/seaweedfs-hadoop3-client-4.00.jar
spark.driver.extraClassPath=/path/to/seaweedfs-hadoop3-client-4.38.jar
spark.executor.extraClassPath=/path/to/seaweedfs-hadoop3-client-4.38.jar
```
And modify the configuration at runtime:
@@ -37,8 +37,8 @@ And modify the configuration at runtime:
1. change the spark-defaults.conf
```
spark.driver.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.00.jar
spark.executor.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.00.jar
spark.driver.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.38.jar
spark.executor.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.38.jar
spark.hadoop.fs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem
```
@@ -81,8 +81,8 @@ spark.history.fs.cleaner.enabled=true
spark.history.fs.logDirectory=seaweedfs://localhost:8888/spark2-history/
spark.eventLog.dir=seaweedfs://localhost:8888/spark2-history/
spark.driver.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.00.jar
spark.executor.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.00.jar
spark.driver.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.38.jar
spark.executor.extraClassPath=/Users/chris/go/src/github.com/seaweedfs/seaweedfs/other/java/hdfs3/target/seaweedfs-hadoop3-client-4.38.jar
spark.hadoop.fs.seaweedfs.impl=seaweed.hdfs.SeaweedFileSystem
spark.hadoop.fs.defaultFS=seaweedfs://localhost:8888
```