Friday, November 15, 2013

hadoop 2.2.0 installation resource

General steps:

http://milinda.pathirage.org/hadoop/yarn/2013/09/29/how-to-setup-multi-node-hadoop-20xyarn-cluster.html#references<br />

It refers to:
http://www.javacodegeeks.com/2013/06/setting-up-apache-hadoop-multi-node-cluster.html
http://raseshmori.wordpress.com/2012/10/14/install-hadoop-nextgen-yarn-multi-node-cluster/

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce.shuffle</value>
  </property>

should be:
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>

How to build 64bit native library or simply download from:
http://shaurong.blogspot.com/2013/11/hadoop-220-centos-64-x64.html

It has a detailed process on how to build from hadoop source. This is the only reliable/working steps I have found so far.
Hadoop build is already moved to maven but the hadoop document still shows old steps with ant (of course it won't work).
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html

Really sucks.

Another error you may have:
$ hadoop fs -ls
ls: `.': No such file or directory

How to solve:
hadoop fs -mkdir $USER

No comments: