HiBench Testing

HiBench is a Hadoop benchmark suite. It is good for testing the correctness and health of a Hadoop installation.

• Download HiBench from github:

git clone https://github.com/intel-hadoop/HiBench.git

Getting Started

Prerequisites

• Setup HiBench

  Make sure these things are installed: maven. Then, locate into HiBench/common/hibench and run mvn process-sources to get dependencies.

• Setup Hadood

  Before you run any workload in the package, please verify the Hadoop framework is running correctly. All the workloads have been tested with Cloudera Distribution of Hadoop 5(cdh5.1.0) and Hadoop version 1.0.4 and 2.2.0

• Setup Hive (for hivebench)

  Please make sure you have properly set up Hive in your cluster if you want to test hivebench. Or the benchmark willuse the default release fetched by maven.

Configure for the all workloads

You need to set some global environment variables in the bin/hibench-config.sh file located in the root dir.

export JAVA_HOME=/usr/lib/jvm/java
export HADOOP_HOME=/opt/mapr/hadoop/hadoop-2.5.1/
export HADOOP_EXECUTABLE=hadoop
export HADOOP_CONF_DIR=/opt/mapr/hadoop/hadoop-2.5.1/etc/hadoop/
export HADOOP_EXAMPLES_JAR=/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1-mapr-1501.jar
export MAPRED_EXECUTABLE=mapred
#Set the varaible below only in YARN mode
export HADOOP_JOBCLIENT_TESTS_JAR=/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.1-mapr-1501-tests.jar

These variables are for use with MapR. (see MapR: Landing Page