Interview Questions.

Q1. Does Apache Flume Support Third-birthday celebration Plugins?

Yes, Flume has 100% plugin-based totally architecture, it can load and ships facts from outside sources to external destination which one by one from Flume. SO that most of the massive information analysis use this device for streaming information.

Q2. What Is An Agent?

A system that hosts flume additives such as resources, channels and sinks, and hence has the potential to obtain, store and forward occasions to their vacation spot.

Q3. Which Is The Reliable Channel In Flume To Ensure That There Is No Data Loss?

FILE Channel is the maximum dependable channel many of the 3 channels JDBC, FILE and MEMORY.

Q4. Is It Possible To Leverage Real Time Analysis On The Big Data Collected By Flume Directly? If Yes, Then Explain How?

Data from Flume may be extracted, trformed and loaded in actual-time into Apache Solr servers usingMorphlineSolrSink.

Q5. Can Flume Can Distribute Data To Multiple Destinations?

Yes. It aid multiplexing drift. The occasion flows from one supply to more than one channel and a couple of destionations, It is acheived by defining a drift multiplexer.

Q6. What Is Apache Flume?

Apache Flume is a dispensed, reliable, and to be had machine for effectively gathering, aggregating and moving large quantities of log information from many special sources to a centralized facts source. Review this Flume use case to learn the way Mozilla collects and Analyse the Logs the use of Flume and Hive.

Flume is a framework for populating Hadoop with information. Agents are populated in the course of ones IT infrastructure – interior net servers, software servers and mobile devices, for example – to acquire data and integrate it into Hadoop.

Q7. What Is Flume?

Flume is a disbursed, dependable, and available service for efficiently amassing, aggregating, and shifting large amounts of log records. It has a easy and flexible structure primarily based on streaming statistics flows. It is robust and fault tolerant with tunable reliability mechanisms and plenty of fail over and recuperation mechanisms. It uses a easy extensible information model that permits for online analytic software.

Q8. What Are The Complicated Steps In Flume Configurations?

Flume can processing streaming information. So if started as soon as, there's no stop/end to the procedure. Asynchronously it can flows information from source to HDFS through agent. First of all agent need to understand man or woman additives how they may be linked to load records. So configuration is cause to load streaming data. As an example consumerkey, consumersecret accessToken and accessTokenSecret are key thing to down load records from twitter.

Q9. What Are The Data Extraction Tools In Hadoop?

Sqoop can be used to trfer facts between RDBMS and HDFS. Flume can be used to extract the streaming information from social media, web log and so on and shop it on HDFS.

Q10. Explain What Are The Tools Used In Big Data?

Tools utilized in Big Data includes

Hadoop

Hive

Pig

Flume

Mahout

Sqoop

Q11. Explain About The Replication And Multiplexing Selectors In Flume?

Channel Selectors are used to handle more than one channels. Based on the Flume header cost, an event may be written just to a unmarried channel or to more than one channels. If a channel selector is not particular to the supply then by way of default it is the Replicating selector. Using the replicating selector, the identical occasion is written to all of the channels in the source’s channels list. Multiplexing channel selector is used whilst the software has to send specific activities to unique channels.

Q12. Explain About The Different Channel Types In Flume. Which Channel Type Is Faster?

The 3 special built in channel types to be had in Flume are:

MEMORY Channel – Events are read from the supply into memory and passed to the sink.

JDBC Channel – JDBC Channel stores the occasions in an embedded Derby database.

FILE Channel –File Channel writes the contents to a file at the record device after studying the occasion from a source. The document is deleted best after the contents are efficaciously added to the sink.

MEMORY Channel is the fastest channel among the 3 however has the danger of records loss. The channel which you select absolutely relies upon on the character of the massive statistics application and the value of each event.

Q13. What Is Sink Processors?

Sinc processors is mechanism by using which you can create a fail-over undertaking and cargo balancing.

Q14. Does Apache Flume Provide Support For Third Party Plug-ins?

Most of the facts analysts use Apache Flume has plug-in based structure as it may load facts from external resources and trfer it to external locations.

Q15. What Is A Channel?

It shops activities,events are introduced to the channel via sources working in the agent.An occasion stays within the channel until a sink removes it for in addition trport.

Q16. Why We Are Using Flume?

Most regularly Hadoop developer use this too to get statistics from social media web sites. Its evolved by Cloudera for aggregating and moving very huge quantity if records. The number one use is to acquire log files from one of a kind resources and asynchronously persist in the hadoop cluster.

Q17. Differentiate Between Filesink And Filerollsink?

The principal difference between HDFS FileSink and FileRollSink is that HDFS File Sink writes the activities into the Hadoop Distributed File System (HDFS) while File Roll Sink stores the events into the neighborhood document gadget.

Q18. How Can Flume Be Used With Hbase?

Apache Flume can be used with HBase using one of the two HBase links:

HBaseSink (org.Apache.Flume.Sink.Hbase.HBaseSink) supports secure HBase clusters and also the novel HBase IPC that become introduced inside the model HBase zero.9@

AsyncHBaseSink (org.Apache.Flume.Sink.Hbase.AsyncHBaseSink) has higher performance than HBase sink as it can without problems make non-blocking off calls to HBase.

Working of the HBaseSink:

In HBaseSink, a Flume Event is converted into HBase Increments or Puts. Serializer implements the HBaseEventSerializer that is then instantiated while the sink begins. For each event, sink calls the initialize technique inside the serializer which then trlates the Flume Event into HBase increments and puts to be sent to HBase cluster.

Working of the AsyncHBaseSink:

AsyncHBaseSink implements the AsyncHBaseEventSerializer. The initialize method is called best once by using the sink while it starts. Sink invokes the setEvent method and then makes calls to the getIncrements and getActions techniques just just like HBase sink. When the sink stops, the cleanUp method is referred to as by way of the serializer.

 

Q19. What Is Apache Spark?

Spark is a quick, easy-to-use and flexible data processing framework. It has a complicated execution engine helping cyclic facts  flow and in-reminiscence computing. Spark can run on Hadoop, standalone or inside the cloud and is capable of getting access to diverse statistics resources including HDFS, HBase, Cassandra and others.

Q20. What Is Flumeng?

A real time loader for streaming your information into Hadoop. It stores information in HDFS and HBase. You’ll want to get started with FlumeNG, which improves at the authentic flume.

Q21. Does Flume Provide a hundred% Reliability To The Data Flow?

Yes, Apache Flume offers end to cease reliability due to its tractional method in statistics go with the flow.

Q22. What Are The Similarities And Differences Between Apache Flume And Apache Kafka?

Flume pushes messages to their vacation spot thru its Sinks.With Kafka you need to consume messages from Kafka Broker using a Kafka Consumer API.

Q23. What Is Flume Event?

A unit of facts with set of string attribute known as Flume occasion. The external supply like web-server ship events to the supply. Internally Flume has built in capability to understand the source format.

Each log report is recollect as an occasion. Each event has header and fee sectors, which has header information and appropriate cost that assign to articular header.

Q24. How Multi-hop Agent Can Be Setup In Flume?

Avro RPC Bridge mechanism is used to setup Multi-hop agent in Apache Flume.

Q25. What Are Interceptors?

Interceptors are used to filter out the activities among source and channel, channel and sink. These channels can clear out un-necessary or focused log files. Depends on requirements you can use n quantity of interceptors.

Q26. Can You Explain About Configuration Files?

The agent configuration is stored in nearby configuration document. It  accommodates of each dealers supply,  sink and channel facts.

Q27. Explain Reliability And Failure Handling In Apache Flume?

Flume NG makes use of channel-primarily based tractions to assure dependable message shipping. When a message moves from one agent to any other, two tractions are commenced, one at the agent that can provide the occasion and the other at the agent that gets the event. In order for the sending agent to devote it’s traction, it should get hold of fulfillment indication from the receiving agent.

The receiving agent best returns a achievement indication if it’s very own traction commits well first. This guarantees assured shipping semantics between the hops that the float makes. Figure  beneath indicates a series diagram that illustrates the relative scope and length of the tractions running inside the two interacting dealers.

Q28. Tell Any Two Features Of Flume?

Fume collects statistics successfully, mixture and movements big amount of log facts from many one of a kind resources to centralized information keep.

Flume isn't restrained to log information aggregation and it can trport huge amount of occasion data which includes but now not restrained to community site visitors information, social-media generated statistics , e mail message na quite much any facts garage.

Q29. What Are Flume Core Components ?

Cource, Channels and sink are core additives in Apache Flume. When Flume supply receives occasion from externalsource, it stores the event in a single or more than one channels. Flume channel is quickly keep and keep the occasion till’s consumed via the Flume sink. It act as Flume repository. Flume Sink eliminates the occasion from channel and positioned into an external repository like HDFS or Move to the following flume.

Q30. Why Flume.?

Flume is not restrained to acquire logs from allotted structures, but it's far able to appearing other use cases which include

Collecting readings from array of sensors

Collecting impressions from custom apps for an ad community

Collecting readings from community gadgets in an effort to screen their overall performance.

Flume is centered to preserve the reliability, scalability, manageability and extensibility whilst it serves maximum number of clients with better QoS