Why 'mapred-site.xml' is not included in the latest Hadoop 2.2.0?

apache hadoop mapreduce bigdata
pcdhan picture pcdhan · Nov 23, 2013 · Viewed 9.6k times · Source

Latest build of Hadoop provides mapred-site.xml.template Do we need to create a new mapred-site.xml file using this? Any link on documentation or explanation related to Hadoop 2.2.0 will be much appreciated.

Answer

Vishal picture Vishal · Nov 24, 2013

I believe it's still required. For our basic Hadoop 2.2.0 2-node cluster setup that we have working I did the following from the setup documentation.

"

From the base of the Hadoop installation, edit the etc/hadoop/mapred-site.xml file. A new configuration option for Hadoop 2 is the capability to specify a framework name for MapReduce, setting the mapreduce.framework.name property. In this install we will use the value of "yarn" to tell MapReduce that it will run as a YARN application. First, copy the template file to the mapred-site.xml.

cp mapred-site.xml.template mapred-site.xml

Next, copy the following into Hadoop etc/hadoop/mapred-site.xml file and remove the original empty tags. 

 <configuration> 
 <property>   
 <name>mapreduce.framework.name</name>  
 <value>yarn</value>  
 </property>   
 </configuration>

"

Wrt documentation, I found this the most useful. Also, etc/hosts configs for cluster setup and other cluster related configs were a bit hard to figure out.

Related questions

Checksum Exception when reading from or copying to hdfs in apache hadoop

I am trying to implement a parallelized algorithm using Apache hadoop, however I am facing some issues when trying to transfer a file from the local file system to hdfs. A checksum exception is being thrown when trying to read …

Read More
What is the maximum container(s) in a single-node cluster (hadoop)?

I am new in hadoop and i am not yet familiar to its configuration. I just want to ask the maximum container per node. I am using a single node cluster (6GB ram laptop) and below is my mapred and …

Read More
hadoop.mapred vs hadoop.mapreduce?

Why are there two separate packages map-reduce package in Apache's hadoop package tree: org.apache.hadoop.mapred http://javasourcecode.org/html/open-source/hadoop/hadoop-1.0.3/org/apache/hadoop/mapred/ org.apache.hadoop.mapreduce http://javasourcecode.org/html/open-source/hadoop/hadoop-1.0.3/org/…

Read More