How to use > in an xargs command?

Jesse Shieh picture Jesse Shieh · May 10, 2009 · Viewed 60.2k times · Source

I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something like this

ls -1 | xargs -I{} "grep ABC '{}' > '{}'.out"

but, as far as I know, xargs doesn't like the double-quotes. If I remove the double-quotes, however, then the command redirects the output of the entire command to a single file called '{}'.out instead of to a series of individual files.

Does anyone know of a way to do this using xargs? I just used this grep scenario as an example to illustrate my problem with xargs so any solutions that don't use xargs aren't as applicable for me.

Answer

lhunath picture lhunath · May 10, 2009

Do not make the mistake of doing this:

sh -c "grep ABC {} > {}.out"

This will break under a lot of conditions, including funky filenames and is impossible to quote right. Your {} must always be a single completely separate argument to the command to avoid code injection bugs. What you need to do, is this:

xargs -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}

Applies to xargs as well as find.

By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).

Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs

Use find for everything that needs recursion and a simple loop with a glob for everything else:

find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;

or non-recursive:

for file in *; do grep "$file" > "$file.out"; done

Notice the proper use of quotes.