Print lines in one file matching patterns in another file

unix sed awk grep extract
Jon picture Jon · Jan 27, 2014 · Viewed 36.5k times · Source

I have a file with more than 40.000 lines (file1) and I want to extract the lines matching patterns in file2 (about 6000 lines). I use grep like this, but it is very slow: grep -f file2 file1 > out

Is there a faster way to do this using awk or sed?

Here's some extracts from my files:

File1:
scitn003869.2| scign003869 CGCATGTGTGCATGTATTATCGTATCCCTTG
scitn007747.1| scign007747  CACGCAGACGCAGTGGAGCATTCCAGGTCACAA
scitn003155.1| scign003155  TAAAAATCGTTAGCACTCGCTTGGTACACTAAC
scitn018252.1| scign018252  CGTGTGTGTGCATATGTGTGCATGCGTG
scitn004671.2| scign004671  TCCTCAGGTTTTGAAAGGCAGGGTAAGTGCT


File2:
scign000003
scign000004
scign000005
scign004671
scign000013

`

Answer

glenn jackman picture glenn jackman · Jan 27, 2014

Try grep -Fwf file2 file1 > out

The -F option specifies plain string matching, so should be faster without having to engage the regex engine.

Related questions

How to remove leading whitespace from each line in a file

I have a file that looks something like this: for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (i = 0; i < 100; i++) for (…

Read More
Bash tool to get nth line from a file

Is there a "canonical" way of doing that? I've been using head -n | tail -1 which does the trick, but I've been wondering if there's a Bash tool that specifically extracts a line (or a range of lines) from a …

Read More
How to select lines between two marker patterns which may occur multiple times with awk/sed

Using awk or sed how can I select lines which are occurring between two different marker patterns? There may be multiple sections marked with these patterns. For example: Suppose the file contains: abc def1 ghi1 jkl1 mno abc def2 ghi2 …

Read More