GNU parallel not working at all

parallel-processing gnu xargs gnu-parallel
WaelJ picture WaelJ · May 8, 2013 · Viewed 20.2k times · Source

I have been trying to use GNU parallel for some time, but I have never been able to get it to function at all!

For example, running (in a non-empty directory!):

ls | parallel echo            # Outputs single new line
ls | parallel echo echo echo  # Outputs three new lines.
ls | parallel echo {}         # /bin/bash: {}: command not found
ls | parallel echo '{}'       # /bin/bash: {}: command not found
ls | parallel 'echo {}'       # Outputs: {}
ls | parallel -IMM 'echo MM'  # Outputs: MM

It seems that it is simply executing each argument as a command, which makes no sense.

I have tried bash, zsh, tcsh, csh, and sh, to no avail.

Answer

WaelJ picture WaelJ · May 8, 2013

As I was about to complete writing this question, I ran parallel --version to report the version, only to find:

WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.

It is not clear to me why that flag is set by default. Needless to say, using --gnu worked!

Thought I would post this to save someone hours of frustration and confusion.

EDIT: To fix this permanently (in Ubuntu at least), delete the --tollef flag in /etc/parallel/config

Related questions

How do I parallelize a simple Python loop?

This is probably a trivial question, but how do I parallelize the following loop in python? # setup output lists output1 = list() output2 = list() output3 = list() for j in range(0, 10): # calc individual parameter value parameter = j * offset # call the calculation out1, …

Read More
How to wait for all threads to finish, using ExecutorService?

I need to execute some amount of tasks 4 at a time, something like this: ExecutorService taskExecutor = Executors.newFixedThreadPool(4); while(...) { taskExecutor.execute(new MyTask()); } //...wait for completion somehow How can I get notified once all of them are complete? For now …

Read More
How do you run multiple programs in parallel from a bash script?

I am trying to write a .sh file that runs many programs simultaneously I tried this prog1 prog2 But that runs prog1 then waits until prog1 ends and then starts prog2... So how can I run them in parallel?

Read More