Interview Questions.

Question 1. What Is Yarn?

Answer :

Apache YARN, which stands for 'Yet another Resource Negotiator', is Hadoop cluster resource control device.

YARN presents APIs for requesting and working with Hadoop's cluster assets. These APIs are generally used by components of Hadoop's dispensed frameworks consisting of MapReduce, Spark, and Tez etc. Which can be building on pinnacle of YARN. User applications normally do now not use the YARN APIs without delay. Instead, they use higher level APIs provided by the framework (MapReduce, Spark, and many others.) which cover the useful resource control info from the person.

Question 2. What Are The Key Components Of Yarn?

Answer :

The simple concept of YARN is to break up the functionality of resource management and activity scheduling/tracking into separate daemons.

YARN includes the following one-of-a-kind components:

Resource Manager - The Resource Manager is a worldwide component or daemon, one according to cluster, which manages the requests to and assets throughout the nodes of the cluster.

Node Manager - Node Manger runs on each node of the cluster and is chargeable for launching and monitoring containers and reporting the repute returned to the Resource Manager.

Application Master is a consistent with-utility component this is liable for negotiating resource requirements for the useful resource supervisor and running with Node Managers to execute and reveal the obligations.

Container is YARN framework is a UNIX technique jogging at the node that executes an utility-particular process with a restrained set of resources (Memory, CPU, and many others.).

SQL Server 2008 Interview Questions
Question three. What Is Resource Manager In Yarn?

Answer :

The YARN Resource Manager is a international aspect or daemon, one per cluster, which manages the requests to and assets across the nodes of the cluster.

The Resource Manager has two foremost additives - Scheduler and Applications Manager.

Scheduler - The scheduler is liable for allocating resources to and beginning applications primarily based at the summary belief of resource packing containers having a limited set of resources.

Application Manager - The Applications Manager is accountable for accepting process-submissions, negotiating the primary container for executing the software unique Application Master and gives the provider for restarting the Application Master field on failure.

Question four. What Are The Scheduling Policies Available In Yarn?

Answer :

YARN scheduler is accountable for scheduling resources to person applications based on a described scheduling policy. YARN offers 3 scheduling alternatives - FIFO scheduler, Capacity scheduler and Fair scheduler.

FIFO Scheduler - FIFO scheduler puts utility requests in queue and runs them inside the order of submission.

Capacity Scheduler - Capacity scheduler has a separate dedicated queue for smaller jobs and begins them as soon as they're submitted.

Fair Scheduler - Fair scheduler dynamically balances and allocates sources between all of the running jobs.

SQL Server 2008 Tutorial
Question 5. How Do You Setup Resource Manager To Use Capacity Scheduler?

Answer :

You can configure the Resource Manager to use Capacity Scheduler via setting the cost of property 'yarn.Resourcemanager.Scheduler.Magnificence' to 'org.Apache.Hadoop.Yarn.Server.Resourcemanager.Scheduler.Potential.CapacityScheduler' within the report 'conf/yarn-web site.Xml'.

MySQL Interview Questions
Question 6. How Do You Setup Resource Manager To Use Fair Scheduler?

Answer :

You can configure the Resource Manager to apply FairScheduler with the aid of putting the fee of belongings 'yarn.Resourcemanager.Scheduler.Elegance' to 'org.Apache.Hadoop.Yarn.Server.Resourcemanager.Scheduler.Honest.FairScheduler' within the report 'conf/yarn-web page.Xml'.

Question 7. How Do You Setup Ha For Resource Manager?

Answer :

Resource Manager is chargeable for scheduling programs and tracking sources in a cluster. Prior to Hadoop 2.Four, the Resource Manager does no longer have choice to be setup for HA and is a unmarried factor of failure in a YARN cluster.

Since Hadoop 2.4, YARN Resource Manager may be setup for excessive availability. High availability of Resource Manager is enabled via use of Active/Standby structure. At any point of time, one Resource Manager is lively and one or more of Resource Managers are within the standby mode. In case the active Resource Manager fails, one of the standby Resource Managers transitions to an active mode.

MySQL Tutorial Hadoop Interview Questions
Question eight. What Are The Core Changes In Hadoop 2.X?

Answer :

Many adjustments, in particular unmarried point of failure and Decentralize Job Tracker energy to facts-nodes is the primary changes. Entire process tracker structure modified.

Some of the primary distinction among Hadoop 1.X and 2.X given beneath:

Single factor of failure – Rectified.
Nodes dilemma (4000- to limitless) – Rectified.
Job Tracker bottleneck – Rectified.
Map-lessen slots are changed static to dynamic.
High availability – Available.
Support each Interactive, graph iterative algorithms (1.X not assist).
Allows other applications additionally to combine with HDFS.
Question nine. What Is The Difference Between Mapreduce 1 And Mapreduce 2/yarn?

Answer :

In MapReduce 1, Hadoop centralized all duties to the Job Tracker. It allocates sources and scheduling the jobs throughout the cluster. In YARN, de-centralized this to ease the paintings pressure at the Job Tracker. Resource Manager obligation allocate assets to the unique nodes and Node supervisor schedule the jobs on the software Master. YARN lets in parallel execution and Application Master managing and execute the process. This method can ease many Job Tracker troubles and improves to scale up potential and optimize the activity overall performance. Additionally YARN can permits to create more than one programs to scale up on the dispensed surroundings.

Apache Hive Interview Questions
Question 10. How Hadoop Determined The Distance Between Two Nodes?

Answer :

Hadoop admin write a script called Topology script to decide the rack region of nodes. It is trigger to understand the gap of the nodes to duplicate the information. Configure this script in middle-web page.Xml

<Property>

<name>topology.Script.File.Call</name>

<value>middle/rack-consciousness.Sh</value>

</property>

within the rack-recognition.Sh you ought to write script wherein the nodes located.

Hadoop Tutorial
Question eleven. Mistakenly User Deleted A File, How Hadoop Remote From Its File System? Can U Roll Back It?

Answer :

HDFS first renames its document call and region it in /trash directory for a configurable quantity of time. In this situation block might freed, however not record. After this time, Namenode deletes the file from HDFS name-area and make report freed. It’s configurable as fs.Trash.C program languageperiod in middle-website online.Xml. By default its cost is 1, you may set to zero to delete report with out storing in trash.

Apache Pig Interview Questions
Question 12. What Is Difference Between Hadoop Namenode Federation, Nfs And Journal Node?

Answer :

HDFS federation can separate the namespace and garage to enhance the scalability and isolation.

SQL Server 2008 Interview Questions
Question 13. Yarn Is Replacement Of Mapreduce?

Answer :

YARN is general idea, it help MapReduce, but it’s not replacement of MapReduce. You can development many programs with the assist of YARN. Spark, drill and lots of extra programs paintings at the top of YARN.

Apache Hive Tutorial
Question 14. What Are The Core Concepts/techniques In Yarn?

Answer :

Resource supervisor: As equal to the Job Tracker

Node manager: As equal to the Task Tracker.

Application manager: As equivalent to Jobs. Everything is utility in YARN. When client submit process (software),

Containers: As equal to slots.

Yarn child: If you submit the application, dynamically Application grasp release Yarn baby to do Map and Reduce duties.

If utility manager failed, no longer a hassle, useful resource supervisor automatically begin new software assignment.

Question 15. Steps To Upgrade Hadoop 1.X To Hadoop 2.X?

Answer :

To improve 1.X to 2.X dont upgrade without delay. Simple download regionally then cast off old documents in 1.X documents. Up-gradation take greater time.

Share folder there. Its important.. Proportion.. Hadoop .. Mapreduce .. Lib.
Stop all methods.
Delete vintage meta information information… from work/hadoop2data
Copy and rename first 1.X facts into paintings/hadoop2.X
Don’t layout NN whilst up gradation.
Hadoop namenode -improve // It will take a lot of time.
Don’t near previous terminal open new terminal.
Hadoop namenode -rollback.
Hadoop Administration Interview Questions
Question 16. What Is Apache Hadoop Yarn?

Answer :

YARN is a powerful and efficient function rolled out as part of Hadoop 2.0.YARN is a huge scale dispensed machine for strolling huge statistics packages.

Apache Pig Tutorial
Question 17. Is Yarn A Replacement Of Hadoop Mapreduce?

Answer :

YARN is not a substitute of Hadoop however it is a extra effective and efficient technology that supports MapReduce and is also called Hadoop 2.0 or MapReduce 2.

Apache Flume Interview Questions
Question 18. What Are The Additional Benefits Yarn Brings In To Hadoop?

Answer :

Effective utilization of the assets as a couple of packages can be run in YARN all sharing a commonplace useful resource. In Hadoop MapReduce there are seperate slots for Map and Reduce responsibilities whereas in YARN there is no fixed slot. The equal field can be used for Map and Reduce tasks main to better utilization.

YARN is backward like minded so all the existing MapReduce jobs.
Using YARN, one may even run programs that are not based at the MapReduce model.
MySQL Interview Questions
Question 19. How Can Native Libraries Be Included In Yarn Jobs?

Answer :

There are  methods to consist of local libraries in YARN jobs:-

By putting the -Djava.Library.Route at the command line however in this example there are probabilities that the local libraries won't be loaded efficiently and there is opportunity of mistakes.
The higher choice to encompass native libraries is to the set the LD_LIBRARY_PATH within the .Bashrc document.
Apache Flume Tutorial
Question 20. Explain The Differences Between Hadoop 1.X And Hadoop 2.X?

Answer :

In Hadoop 1.X, MapReduce is accountable for each processing and cluster management while in Hadoop 2.X processing is looked after via different processing models and YARN is liable for cluster control.
Hadoop 2.X scales better while as compared to Hadoop 1.X with near 10000 nodes in line with cluster.
Hadoop 1.X has single factor of failure hassle and each time the Namenode fails it has to be recovered manually. However, in case of Hadoop 2.X StandBy Namenode overcomes the SPOF trouble and whenever the Namenode fails it's far configured for automated recovery.
Hadoop 1.X works at the idea of slots whereas Hadoop 2.X works on the idea of containers and also can run commonplace obligations.
Apache Impala Interview Questions
Question 21. What Are The Core Changes In Hadoop 2.Zero?

Answer :

Hadoop 2.X offers an upgrade to Hadoop 1.X in terms of useful resource control, scheduling and the way wherein execution happens. In Hadoop 2.X the cluster resource management abilties paintings in isolation from the MapReduce specific programming logic. This allows Hadoop to proportion assets dynamically between multiple parallel processing frameworks like Impala and the center MapReduce thing. Hadoop 2.X Hadoop 2.X lets in viable and high-quality grained resource configuration leading to green and better cluster usage in order that the software can scale to method larger number of jobs.

Question 22. Differentiate Between Nfs, Hadoop Namenode And Journal Node?

Answer :

HDFS is a write as soon as document machine so a consumer can't update the documents when they exist both they are able to study or write to it. However, underneath certain scenarios within the organization surroundings like file uploading, document downloading, document surfing or facts streaming –it isn't always feasible to acquire all this the use of the standard HDFS. This is in which a dispensed file machine protocol Network File System (NFS) is used. NFS permits get entry to to files on remote machines simply much like how neighborhood document system is accessed by using programs.
Namenode is the heart of the HDFS record machine that maintains the metadata and tracks wherein the file records is kept across the Hadoop cluster.
StandBy Nodes and Active Nodes speak with a collection of light weight nodes to maintain their state synchronized. These are called Journal Nodes.
Apache Impala Tutorial
Question 23. What Are The Modules That Constitute The Apache Hadoop 2.0 Framework?

Answer :

Hadoop 2.Zero contains 4 critical modules of which 3 are inherited from Hadoop 1.0 and a brand new module YARN is delivered to it.

Hadoop Common – This module includes all of the simple utilities and libraries that required by using other modules.

HDFS- Hadoop Distributed file gadget that stores big volumes of records on commodity machines across the cluster.

MapReduce- Java based programming version for data processing.

YARN- This is a new module brought in Hadoop 2.Zero for cluster aid management and process scheduling.

MongoDB Interview Questions
Question 24. How Is The Distance Between Two Nodes Defined In Hadoop?

Answer :

Measuring bandwidth is difficult in Hadoop so community is denoted as a tree in Hadoop. The distance among  nodes within the tree plays a essential function in forming a Hadoop cluster and is defined with the aid of the network topology and java interface DNS Switch Mapping. The distance is identical to the sum of the distance to the nearest commonplace ancestor of each the nodes. The technique get Distance(Node node1, Node node2) is used to calculate the distance among two nodes with the belief that the distance from a node to its discern node is always 1.

Hadoop Interview Questions