xargs split at newlines not spaces

fakedrake picture fakedrake · Apr 17, 2014 · Viewed 22.4k times · Source

here is my problem in short

$ echo 'for i in $@; do echo arg: $i; done; echo DONE' > /tmp/test.sh
$ echo "ac\nbc\ncc\n" | xargs bash /tmp/test.sh 
arg: ac
arg: bc
arg: cc
DONE

Which is what i expect, but

$ echo "ac s\nbc s\ncc s\n" | xargs -d \n bash /tmp/test.sh
arg: ac
arg: s
arg: bc
arg: s
arg: cc
arg: s
DONE

Shouldn't the output be?

arg: ac s
arg: bc s
arg: cc s
DONE

How do I get the 2nd output with xargs?

Answer

mklement0 picture mklement0 · Apr 17, 2014

Try:

printf %b 'ac s\nbc s\ncc s\n' | xargs -d '\n' bash /tmp/test.sh

You neglected to quote the \n passed to -d, which means that just n rather than \n was passed to xargs as the delimiter - the shell "ate" the \ (when the shell parses an unquoted string, \ functions as an escape character; if an ordinary character follows the \ - n in this case - only that ordinary character is used).

Also heed @glenn jackman's advice to double-quote the $@ inside the script (or omit the in "$@" part altogether).

Also: xargs -d is a GNU extension, which, for instance, won't work on FreeBSD/macOS. To make it work there, see @glenn jackman's xargs -0-based solution.


Note that I'm using printf rather than echo to ensure that the \n instances in the string are interpreted as newlines in all Bourne-like shells:
In bash and ksh[1], echo defaults to NOT interpreting \-based escape sequences (you have to use -e to achieve that) - unlike in zsh and strictly POSIX-compliant shells such as dash.
Therefore, printf is the more portable choice.

[1] According to the manual, ksh's echo builtin exhibits the same behavior as the host platform's external echo utility; while this may vary across platforms, the Linux and BSD/macOS implementations do not interpret \ escape sequences by default.