counting duplicates in a sorted sequence using command line tools

bash command-line sorting count duplicates
letronje picture letronje · Jul 7, 2009 · Viewed 67.1k times · Source

I have a command (cmd1) that greps through a log file to filter out a set of numbers. The numbers are in random order, so I use sort -gr to get a reverse sorted list of numbers. There may be duplicates within this sorted list. I need to find the count for each unique number in that list.

For e.g. if the output of cmd1 is:

100 
100 
100 
99 
99 
26 
25 
24 
24

I need another command that I can pipe the above output to, so that, I get:

100     3
99      2
26      1
25      1
24      2

Answer

Stephen Paul Lesniewski picture Stephen Paul Lesniewski · Jul 7, 2009

how about;

$ echo "100 100 100 99 99 26 25 24 24" \
    | tr " " "\n" \
    | sort \
    | uniq -c \
    | sort -k2nr \
    | awk '{printf("%s\t%s\n",$2,$1)}END{print}'

The result is :

100 3
99  2
26  1
25  1
24  2

Related questions

How do I execute any command editing its file (argument) "in place" using bash?

I have a file temp.txt, that I want to sort with the sort command in bash. I want the sorted results to replace the original file. This doesn't work for example (I get an empty file): sortx temp.txt &…

Read More
How do I set a variable to the output of a command in Bash?

I have a pretty simple script that is something like the following: #!/bin/bash VAR1="$1" MOREF='sudo run command against $VAR1 | grep name | cut -c7-' echo $MOREF When I run this script from the command line and pass it …

Read More
How do I parse command line arguments in Bash?

Say, I have a script that gets called with this line: ./myscript -vfd ./foo/bar/someFile -o /fizz/someOtherFile or this one: ./myscript -v -f -d -o /fizz/someOtherFile ./foo/bar/someFile What's the accepted way of parsing this such …

Read More