How to sort,uniq and display line that appear more than X times

bash sorting uniq
Andrew Kennen picture Andrew Kennen · Nov 22, 2013 · Viewed 9.8k times · Source

I have a file like this:

80.13.178.2
80.13.178.2
80.13.178.2
80.13.178.2
80.13.178.1
80.13.178.3
80.13.178.3
80.13.178.3
80.13.178.4
80.13.178.4
80.13.178.7

I need to display unique entries for repeated line (similar to uniq -d) but only entries that occur more than just twice (twice being an example so flexibility to define the lower limit.)

Output for this example should be like this when looking for entries with three or more occurrences: 

80.13.178.2
80.13.178.3

Answer

iruvar picture iruvar · Nov 22, 2013

Feed the output from uniq -cd to awk

sort test.file | uniq -cd | awk -v limit=2 '$1 > limit{print $2}'

Related questions

Bash output the line with highest value

my question is pretty much like this one but with one difference; i want the output the line that has highest score on the 3rd tab. my data is like: 1.gui Qxx 16 2.gui Qxy 23 3.guT QWS 11 and i want to …

Read More
Bash- is it possible to use -uniq for only one column of a line?

1.gui Qxx 16 2.gu Qxy 23 3.guT QWS 18 4.gui Qxr 21 i want to sort a file depending a value in the 3rd column, so i use: sort -rnk3 myfile 2.gu Qxy 23 4.gui Qxr 21 3.guT QWS 18 1.gui Qxx 16 now i have to output …

Read More
How to sort an array in Bash

I have an array in Bash, for example: array=(a c b f 3 5) I need to sort the array. Not just displaying the content in a sorted way, but to get a new array with the sorted elements. The new …

Read More