Parallelizing a while loop with arrays read from a file in bash

Question

I have a while loop in Bash handled like this:

while IFS=$'\t' read -r -a line;
do
    myprogram ${line[0]} ${line[1]} ${line[0]}_vs_${line[1]}.result;
done < fileinput

It reads from a file with this structure, for reference:

foo   bar
baz   foobar

and so on (tab-delimited).

I would like to parallelize this loop (since the entries are a lot and processing can be slow) using GNU parallel, however the examples are not clear on how I would assign each line to the array, like I do here.

What would be a possible solution (alternatives to GNU parallel work as well)?

Answer 1

From https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Use-a-table-as-input:

"""
Content of table_file.tsv:

foo<TAB>bar
baz <TAB> quux

To run:

cmd -o bar -i foo
cmd -o quux -i baz

you can run:

parallel -a table_file.tsv --colsep '\t' cmd -o {2} -i {1}

"""

So in your case it will be:

cat fileinput | parallel --colsep '\t' myprogram {1} {2} {1}_vs_{2}.result