diff --git a/Hadoop-Benchmark.md b/Hadoop-Benchmark.md index 31c3687..4015a89 100644 --- a/Hadoop-Benchmark.md +++ b/Hadoop-Benchmark.md @@ -33,6 +33,8 @@ wget https://oss.sonatype.org/service/local/repositories/releases/content/com/gi The TestDFSIO benchmark is used for measuring I/O (read/write) performance. +**However, the generated data to write is all zero. SeaweedFS automatically compress this kind of data. So this is not scientific for now.** + ## TestDFSIO write tests Start the TestDFSIO write tests: @@ -40,15 +42,15 @@ Start the TestDFSIO write tests: bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0-tests.jar TestDFSIO -write -nrFiles 8 -fileSize 1GB -resFile /tmp/TestDFSIOwrite.txt ... -20/07/16 13:45:25 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write -20/07/16 13:45:25 INFO fs.TestDFSIO: Date & time: Thu Jul 16 13:45:25 PDT 2020 -20/07/16 13:45:25 INFO fs.TestDFSIO: Number of files: 8 -20/07/16 13:45:25 INFO fs.TestDFSIO: Total MBytes processed: 8192 -20/07/16 13:45:25 INFO fs.TestDFSIO: Throughput mb/sec: 37.96 -20/07/16 13:45:25 INFO fs.TestDFSIO: Average IO rate mb/sec: 38.03 -20/07/16 13:45:25 INFO fs.TestDFSIO: IO rate std deviation: 1.68 -20/07/16 13:45:25 INFO fs.TestDFSIO: Test exec time sec: 218.24 -20/07/16 13:45:25 INFO fs.TestDFSIO: +20/07/17 15:57:08 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write +20/07/17 15:57:08 INFO fs.TestDFSIO: Date & time: Fri Jul 17 15:57:08 PDT 2020 +20/07/17 15:57:08 INFO fs.TestDFSIO: Number of files: 8 +20/07/17 15:57:08 INFO fs.TestDFSIO: Total MBytes processed: 8192 +20/07/17 15:57:08 INFO fs.TestDFSIO: Throughput mb/sec: 379.29 +20/07/17 15:57:08 INFO fs.TestDFSIO: Average IO rate mb/sec: 389.95 +20/07/17 15:57:08 INFO fs.TestDFSIO: IO rate std deviation: 74.91 +20/07/17 15:57:08 INFO fs.TestDFSIO: Test exec time sec: 24.05 +20/07/17 15:57:08 INFO fs.TestDFSIO: ``` ## TestDFSIO read tests @@ -60,13 +62,13 @@ bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.0 ... -20/07/16 13:46:57 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read -20/07/16 13:46:57 INFO fs.TestDFSIO: Date & time: Thu Jul 16 13:46:57 PDT 2020 -20/07/16 13:46:57 INFO fs.TestDFSIO: Number of files: 8 -20/07/16 13:46:57 INFO fs.TestDFSIO: Total MBytes processed: 8192 -20/07/16 13:46:57 INFO fs.TestDFSIO: Throughput mb/sec: 406.01 -20/07/16 13:46:57 INFO fs.TestDFSIO: Average IO rate mb/sec: 406.71 -20/07/16 13:46:57 INFO fs.TestDFSIO: IO rate std deviation: 16.61 -20/07/16 13:46:57 INFO fs.TestDFSIO: Test exec time sec: 21.68 -20/07/16 13:46:57 INFO fs.TestDFSIO: +20/07/17 15:59:38 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read +20/07/17 15:59:38 INFO fs.TestDFSIO: Date & time: Fri Jul 17 15:59:38 PDT 2020 +20/07/17 15:59:38 INFO fs.TestDFSIO: Number of files: 8 +20/07/17 15:59:38 INFO fs.TestDFSIO: Total MBytes processed: 8192 +20/07/17 15:59:38 INFO fs.TestDFSIO: Throughput mb/sec: 393.26 +20/07/17 15:59:38 INFO fs.TestDFSIO: Average IO rate mb/sec: 393.72 +20/07/17 15:59:38 INFO fs.TestDFSIO: IO rate std deviation: 13.33 +20/07/17 15:59:38 INFO fs.TestDFSIO: Test exec time sec: 22.76 +20/07/17 15:59:38 INFO fs.TestDFSIO: ``` \ No newline at end of file