Bright:Testing Hadoop

Bright's Hadoop Tests

Automatic tests:

cd /cm/local/apps/cluster-tools/hadoop/
./cm-hadoop-tests.sh <instance> # instance is hdfs1 in our case

End User test runs:

• Firstly, you must give the user access to the Hadoop instance:

 [bright70->user[fred]]% set hadoophdfsaccess apache220; commit

• Then the user can run the tests like this:

[fred@bright70 ~]$ module add hadoop/Apache220/Apache/2.2.0
[fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 1 5
...
Job Finished in 19.732 seconds
Estimated value of Pi is 4.00000000000000000000