How can I find repeated words in a file using grep/egrep?

regex bash unix grep word-boundary
Mouse picture Mouse · Oct 28, 2015 · Viewed 11.6k times · Source

I need to find repeated words in a file using egrep (or grep -e) in unix (bash)

I tried:

egrep "(\<[a-zA-Z]+\>) \1" file.txt

and

egrep "(\b[a-zA-Z]+\b) \1" file.txt

but for some reason these consider things to be repeats that aren't! for example, it thinks the string "word words" meets the criteria despite the word boundary condition \> or \b.

Answer

rici picture rici · Oct 28, 2015

\1 matches whatever string was matched by the first capture. That is not the same as matching the same pattern as was matched by the first capture. So the fact that the first capture matched on a word boundary is no longer relevant, even though the \b is inside the capture parentheses.

If you want the second instance to also be on a word boundary, you need to say so:

egrep "(\b[a-zA-Z]+) \1\b" file.txt

That is no different from:

egrep "\b([a-zA-Z]+) \1\b" file.txt

The space in the pattern forces a word boundary, so I removed the redundant \bs. If you wanted to be more explicit, you could put them in:

egrep "\<([a-zA-Z]+)\> \<\1\>" file.txt

Related questions

grep --ignore-case --only

grep fails when using both --ignore-case and --only-match options. Example: $ echo "abc" | grep -io abc abc $ echo "ABC" | grep -io abc $ But $ echo "abc" | grep -i abc abc $ echo "ABC" | grep -i abc ABC According to man page: -o, --only-matching …

Read More
linux find regex

I'm having trouble using the regex of the find command. Probably something I don't understand about escaping on the command line. Why are these not the same? find -regex '.*[1234567890]' find -regex '.*[[:digit:]]' Bash, Ubuntu

Read More
Find and kill a process in one line using bash and regex

I often need to kill a process during programming. The way I do it now is: [~]$ ps aux | grep 'python csp_build.py' user 5124 1.0 0.3 214588 13852 pts/4 Sl+ 11:19 0:00 python csp_build.py user 5373 0.0 0.0 8096 960 pts/6 S+ 11:20 0:00 grep python csp_build.py [~]$ kill 5124 How …

Read More