Hadoop Configuration Files

we can view all the configuration files under the path /etc/hadoop folder as shown below

core-site.xml

mapred-site.xml

hdfs-site.xml

yarn-site.xml

hadoop-env.sh

.bashrc

 

core-site.xml: It is one of the important configuration files which is required for run -time environment settings of a Hadoop cluster.

It informs Hadoop daemons where the NAMENODE runs in the cluster. It also informs the Name Node as to which IP and ports it should bind.

mapred-site.xml: It is one of the important configuration files which is required for runtime environment settings of a Hadoop. 

It contains the configuration settings for MapReduce. In this file, we specify a framework name for MapReduce, by setting the MapReduce.framework.name.

hdfs-site.xml: It is one of the important configuration files which is required for runtime environment settings of a Hadoop. 

1. To configure block replication factor 

2.To configure/specify NAMENODE metadata location 

3.To configure/specify DATANODE data storage location 

yarn-site.xml: This file contains the configuration settings related to YARN. For example, it contains settings for Node Manager, Resource Manager, Containers, and Application Master.

 

hadoop-env.sh: It specifies the environment variables that affect the JDK used by Hadoop Daemon (bin/Hadoop).

We know that the Hadoop framework is written in Java and uses JRE so one of the environment variables in Hadoop Daemons is $Java_Home in Hadoop-env.sh.

 .bashrc :  

 

Master : 

• It is used to determine the master Nodes in Hadoop cluster. It will inform about the location of SECONDARY NAMENODE to Hadoop Daemon. 
• The Mater File on Slave node is blank.

Slave : 

• It is used to determine the slave Nodes in Hadoop cluster.
• The Slave file at Master Node contains a list of hosts, one per line.
• The Slave file at Slave server contains IP address of Slave nodes.