Exp 1: Installation of HADOOP in STANDALONE MODE.

Local Mode or Stand alone Mode :

• The default mode run by the Hadoop is the stand alone mode.
• This mode is majorly used in case of debugging where HDFS will not be used.
• In this mode both the input and the output can be used as local file system.
• No Custom configuration required for mapred-site.xml, hdfs-site.xml and core-site.xml.
• This mode is the fastest modes in Hadoop as the local file system is used for both the input and the output.

Installation of HADOOP in STANDALONE MODE:

Objective:

Standalone mode is the default mode of operation of Hadoop and it runs on a single node ( a node is your machine).

Software Requirements:
• Oracle Virtual Box 5.x
• Ubuntu Desktop OS 18.x(64bit)
• Hadoop-3.1.0
• OpenJdk version-8 Hardware Requirements:
• Minimum RAM required: 4GB (Suggested: 8GB)
• Minimum Free Disk Space: 25GB

• Minimum Processor i3 or above

Analysis:

By default, Hadoop is configured to run in a non-distributed or standalone mode, as a single Java process. There are no daemons running and everything runs in a single JVM instance. HDFS is not used. We don't have to do anything as far as configuration is concerned, except the JAVA_HOME. Just download the tar file and unzip it

Installation Procedure:

Java Installation

1. sudo apt install openjdk-8-jdk

(java home : /usr/lib/jvm/Java-8-openjdk-amd64)

/* Check the path of java */

bdsa@bdsa-VirtualBox:~$ readlink -f /usr/bin/javac
/usr/lib/jvm/java-8-openjdk-amd64

Installation of HADOOP

Download the Hadoop file from hadoop. apache.org

wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz

/* file extraction */

2. tar -zxvf hadoop-3.3.4.tar.gz

/* creation of hadoop home directory */

3. sudo mkdir /usr/lib/hadoop3

/* change ownership to hadoop3 */

4. sudo chown <username> /usr/lib/hadoop3

Note: Enter the given username  <username> in it 

for example:rk@rk-VirtualBox:~$ pwd

/home/rk

rk@rk-VirtualBox:~$ sudo chown rk /usr/lib/hadoop3

/* Move extracted file to hadoop home directory */

5. sudo mv hadoop-3.3.4/* /usr/lib/hadoop3

6. cd /usr/lib/hadoop3

cd /home/username (change directory to ubuntu home) 

pwd (present working directory)

/* Running Hadoop in standalone Mode from Hadoop Home Directory */

7.Set the path of java 

bdasa@bdasa-VirtualBox:/usr/lib/hadoop3$ readlink -f /usr/bin/javac

bdasa@bdasa-VirtualBox:/usr/lib/hadoop3$cd etc/

bdasa@bdasa-VirtualBox:/usr/lib/hadoop3/etc$ ls

hadoop

bdasa@bdasa-VirtualBox:/usr/lib/hadoop3/etc$ cd hadoop/

bdasa@bdasa-VirtualBox:/usr/lib/hadoop3/etc/hadoop$ ls

capacity-scheduler.xml            kms-log4j.properties
configuration.xsl                 kms-site.xml
container-executor.cfg            log4j.properties
core-site.xml                     mapred-env.cmd
hadoop-env.cmd                    mapred-env.sh
hadoop-env.sh                     mapred-queues.xml.template
hadoop-metrics2.properties        mapred-site.xml
hadoop-policy.xml                 shellprofile.d
hadoop-user-functions.sh.example  ssl-client.xml.example
hdfs-rbf-site.xml                 ssl-server.xml.example
hdfs-site.xml                     user_ec_policies.xml.template
httpfs-env.sh                     workers
httpfs-log4j.properties           yarn-env.cmd
httpfs-site.xml                   yarn-env.sh
kms-acls.xml                      yarnservice-log4j.properties
kms-env.sh                        yarn-site.xml
bdasa@bdasa-VirtualBox:/usr/lib/hadoop3/etc/hadoop$ nano hadoop-env.sh 

8. bin/hadoop

 

Limitations:
Standalone mode is the default mode of operation of Hadoop and it runs on a single node ( a node is your machine). HDFS and YARN doesn't run on standalone mode.

Conclusion:
Standalone Mode is the default operation of Hadoop Eco System where the hadoop services will run in the Single JVM. As in this experiment basic Java installation and extraction of the Hadoop files are sufficient to run the Hadoop services.