how to sort word count by value in hadoop?

hadoop mapreduce bigdata partitioner
user1585111 picture user1585111 · Aug 23, 2013 · Viewed 23.8k times · Source

hi i wanted to learn how to sort the word count by value in hadoop.i know hadoop takes of sorting keys, but not by values.

i know to sort the values we must have a partitioner,groupingcomparator and a sortcomparator

but i am bit confused in applying these concepts together to sort the word count by value.

do we need another map reduce job to achieve the same or else a combiner to count the occurrences and then sort here and emit the same to reducer?

can any one explain how to sort word count example by values?

Answer

Rags picture Rags · Aug 23, 2013

You need to have a second mapreduce job. Unless you conclude on the the totals counts (which the first MR job does) how can you think of sorting by value (the counts of the words)? Logically not possible.

Related questions

Hive ParseException - cannot recognize input near 'end' 'string'

I am getting the following error when trying to create a Hive table from an existing DynamoDB table: NoViableAltException(88@[]) at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:9123) at org.apache.hadoop.hive.ql.parse.HiveParser.…

Read More
Number of reducers in hadoop

I was learning hadoop, I found number of reducers very confusing : 1) Number of reducers is same as number of partitions. 2) Number of reducers is 0.95 or 1.75 multiplied by (no. of nodes) * (no. of maximum containers per node). 3) Number of reducers is …

Read More
MySQL Cluster vs. Hadoop for handling big data

I want to know the advantages/disadvantages of using a MySQL Cluster and using the Hadoop framework. What is the better solution. I would like to read your opinion. I think the advantages of using a MySQL Cluster are: high …

Read More