bash: /bin/ls: Argument list too long

list ls fastq
LookIntoEast picture LookIntoEast · Aug 11, 2011 · Viewed 24k times · Source

I need to make a list of a large number of files (40,000 files) like below:

ERR001268_1_100.fastq  ERR001268_2_156.fastq  ERR001753_2_78.fastq
ERR001268_1_101.fastq  ERR001268_2_157.fastq  ERR001753_2_79.fastq
ERR001268_1_102.fastq  ERR001268_2_158.fastq  ERR001753_2_7.fastq
ERR001268_1_103.fastq  ERR001268_2_159.fastq  ERR001753_2_80.fastq

my command is: ls ERR*_1_*.fastq |sed 's/\.fastq//g'|sort -n > masterlist However error is: bash: /bin/ls: Argument list too long

However can I solve this problem? Any other way to make list like this by perl/python?

thx

Answer

Jim Lewis picture Jim Lewis · Aug 11, 2011

You should be able to replace ls ERR*_1_*.fastq with find . -name "ERR*_1_*.fastq".
This way, you can avoid having the wildcard expand into a huge argument list.

(The find output will include a leading "./", e.g. ./ERR001268_1_100.fastq . If that's undesirable, you can get rid of it with another sed command later in the pipeline.)

Related questions

How can I list (ls) the 5 last modified files in a directory?

I know ls -t will list all files by modified time. But how can I limit these results to only the last n files?

Read More
Finding the index of an item in a list

Given a list ["foo", "bar", "baz"] and an item in the list "bar", how do I get its index (1) in Python?

Read More
How do I get the number of elements in a list?

Consider the following: items = [] items.append("apple") items.append("orange") items.append("banana") # FAKE METHOD: items.amount() # Should return 3 How do I get the number of elements in the list items?

Read More