How to pipe input to a Bash while loop and preserve variables after loop ends

Wakan Tanka picture Wakan Tanka · Oct 24, 2013 · Viewed 90.5k times · Source

Bash allows to use: cat <(echo "$FILECONTENT")

Bash also allow to use: while read i; do echo $i; done </etc/passwd

to combine previous two this can be used: echo $FILECONTENT | while read i; do echo $i; done

The problem with last one is that it creates sub-shell and after the while loop ends variable i cannot be accessed any more.

My question is:

How to achieve something like this: while read i; do echo $i; done <(echo "$FILECONTENT") or in other words: How can I be sure that i survives while loop?

Please note that I am aware of enclosing while statement into {} but this does not solves the problem (imagine that you want use the while loop in function and return i variable)

Answer

Jonathan Leffler picture Jonathan Leffler · Oct 24, 2013

The correct notation for Process Substitution is:

while read i; do echo $i; done < <(echo "$FILECONTENT")

The last value of i assigned in the loop is then available when the loop terminates. An alternative is:

echo $FILECONTENT | 
{
while read i; do echo $i; done
...do other things using $i here...
}

The braces are an I/O grouping operation and do not themselves create a subshell. In this context, they are part of a pipeline and are therefore run as a subshell, but it is because of the |, not the { ... }. You mention this in the question. AFAIK, you can do a return from within these inside a function.


Bash also provides the shopt builtin and one of its many options is:

lastpipe

If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.

Thus, using something like this in a script makes the modfied sum available after the loop:

FILECONTENT="12 Name
13 Number
14 Information"
shopt -s lastpipe   # Comment this out to see the alternative behaviour
sum=0
echo "$FILECONTENT" |
while read number name; do ((sum+=$number)); done
echo $sum

Doing this at the command line usually runs foul of 'job control is not active' (that is, at the command line, job control is active). Testing this without using a script failed.

Also, as noted by Gareth Rees in his answer, you can sometimes use a here string:

while read i; do echo $i; done <<< "$FILECONTENT"

This doesn't require shopt; you may be able to save a process using it.