Difference between revisions of "Bright:Testing Hadoop"
Jump to navigation
Jump to search
| Line 18: | Line 18: | ||
[fred@bright70 ~]$ module load zookeeper/hdfs1/Hortonworks/3.4.6.2.2.0.0-2041 | [fred@bright70 ~]$ module load zookeeper/hdfs1/Hortonworks/3.4.6.2.2.0.0-2041 | ||
[fred@bright70 ~]$ module load hbase/hdfs1/Hortonworks/0.98.4.2.2.0.0-2041-hadoop2 | [fred@bright70 ~]$ module load hbase/hdfs1/Hortonworks/0.98.4.2.2.0.0-2041-hadoop2 | ||
| − | [fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/ | + | [fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hhadoop-mapreduce-examples-2.6.0.2.2.0.0-2041.jar pi 1 5 |
... | ... | ||
Job Finished in 19.732 seconds | Job Finished in 19.732 seconds | ||
Estimated value of Pi is 4.00000000000000000000 | Estimated value of Pi is 4.00000000000000000000 | ||
| + | [fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hhadoop-mapreduce-examples-2.6.0.2.2.0.0-2041.jar pi <numMappers> <numSamplesPerMapper> | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Revision as of 14:48, 6 June 2015
Bright's Hadoop Tests
Automatic tests:
cd /cm/local/apps/cluster-tools/hadoop/
./cm-hadoop-tests.sh <instance> # instance is boston-hdfs in our caseEnd User test runs:
- Firstly, you must give the user access to the Hadoop instance:
[bright70->user[fred]]% set hadoophdfsaccess boston-hdfs; commit- Then the user can run the tests like this:
[fred@bright70 ~]$ module load hadoop/hdfs1/Hortonworks/2.6.0.2.2.0.0-2041
[fred@bright70 ~]$ module load zookeeper/hdfs1/Hortonworks/3.4.6.2.2.0.0-2041
[fred@bright70 ~]$ module load hbase/hdfs1/Hortonworks/0.98.4.2.2.0.0-2041-hadoop2
[fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hhadoop-mapreduce-examples-2.6.0.2.2.0.0-2041.jar pi 1 5
...
Job Finished in 19.732 seconds
Estimated value of Pi is 4.00000000000000000000
[fred@bright70 ~]$ hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hhadoop-mapreduce-examples-2.6.0.2.2.0.0-2041.jar pi <numMappers> <numSamplesPerMapper>