Describe the use of Combiner in Hadoop?

1.	Describe the use of Combiner in Hadoop?
Answer» Combiner also known as Mini-Reducer acts as an optional step between MAP and Reduce. Briefly we can that, it helps to take the output from Map function. It then summarizes that output using the same key and then it passes the final summarized RECORDS as input to the Reducer. When we make use of MAPREDUCE JOB on a large dataset. Then large chunk of data is generated by the Mapper which when passed to the reducer for further processing can cause congestion in the network. In ORDER to deal with kind of congestion, Combiner is used by Hadoop Framework as an intermediate between Mapper and Reducer to reduce network congestion.

Answer»

Combiner also known as Mini-Reducer acts as an optional step between MAP and Reduce. Briefly we can that, it helps to take the output from Map function. It then summarizes that output using the same key and then it passes the final summarized RECORDS as input to the Reducer.

When we make use of MAPREDUCE JOB on a large dataset. Then large chunk of data is generated by the Mapper which when passed to the reducer for further processing can cause congestion in the network. In ORDER to deal with kind of congestion, Combiner is used by Hadoop Framework as an intermediate between Mapper and Reducer to reduce network congestion.

Discussion

No Comment Found

Related InterviewSolutions

What do you mean by trigger in SQL?
What do you mean by SQL injection?
What do you mean by alias in SQL?
What are the differences between IN and BETWEEN operators?
What do you mean by SciPy?
Explain pass, continue and break statements in Python?
How can you differentiate between append() and extend() in Python?
Explain Decorator in Python?
How memory can be managed in Python?
How can you differentiate between “is” and “==” operators in Python?