I found some ways to pass external shell variables to an awk
script, but I'm confused about '
and "
.
First, I tried with a shell script:
$ v=123test
$ echo $v
123test
$ echo "$v"
123test
Then tried awk:
$ awk 'BEGIN{print "'$v'"}'
$ 123test
$ awk 'BEGIN{print '"$v"'}'
$ 123
Why is the difference?
Lastly I tried this:
$ awk 'BEGIN{print " '$v' "}'
$ 123test
$ awk 'BEGIN{print ' "$v" '}'
awk: cmd. line:1: BEGIN{print
awk: cmd. line:1: ^ unexpected newline or end of string
I'm confused about this.
Getting shell variables into
awk
may be done in several ways. Some are better than others. This should cover most of them. If you have a comment, please leave below. v1.5
-v
(The best way, most portable)Use the -v
option: (P.S. use a space after -v
or it will be less portable. E.g., awk -v var=
not awk -vvar=
)
variable="line one\nline two"
awk -v var="$variable" 'BEGIN {print var}'
line one
line two
This should be compatible with most awk
, and the variable is available in the BEGIN
block as well:
If you have multiple variables:
awk -v a="$var1" -v b="$var2" 'BEGIN {print a,b}'
Warning. As Ed Morton writes, escape sequences will be interpreted so \t
becomes a real tab
and not \t
if that is what you search for. Can be solved by using ENVIRON[]
or access it via ARGV[]
PS If you like three vertical bar as separator |||
, it can't be escaped, so use -F"[|][|][|]"
Example on getting data from a program/function inn to
awk
(here date is used)
awk -v time="$(date +"%F %H:%M" -d '-1 minute')" 'BEGIN {print time}'
Here we get the variable after the awk
code. This will work fine as long as you do not need the variable in the BEGIN
block:
variable="line one\nline two"
echo "input data" | awk '{print var}' var="${variable}"
or
awk '{print var}' var="${variable}" file
awk '{print a,b,$0}' a="$var1" b="$var2" file
FS
for each file.awk 'some code' FS=',' file1.txt FS=';' file2.ext
BEGIN
block:echo "input data" | awk 'BEGIN {print var}' var="${variable}"
Variable can also be added to awk
using a here-string from shells that support them (including Bash):
awk '{print $0}' <<< "$variable"
test
This is the same as:
printf '%s' "$variable" | awk '{print $0}'
P.S. this treats the variable as a file input.
ENVIRON
inputAs TrueY writes, you can use the ENVIRON
to print Environment Variables.
Setting a variable before running AWK, you can print it out like this:
X=MyVar
awk 'BEGIN{print ENVIRON["X"],ENVIRON["SHELL"]}'
MyVar /bin/bash
ARGV
inputAs Steven Penny writes, you can use ARGV
to get the data into awk:
v="my data"
awk 'BEGIN {print ARGV[1]}' "$v"
my data
To get the data into the code itself, not just the BEGIN:
v="my data"
echo "test" | awk 'BEGIN{var=ARGV[1];ARGV[1]=""} {print var, $0}' "$v"
my data test
You can use a variable within the awk
code, but it's messy and hard to read, and as Charles Duffy
points out, this version may also be a victim of code injection. If someone adds bad stuff to the variable, it will be executed as part of the awk
code.
This works by extracting the variable within the code, so it becomes a part of it.
If you want to make an awk
that changes dynamically with use of variables, you can do it this way, but DO NOT use it for normal variables.
variable="line one\nline two"
awk 'BEGIN {print "'"$variable"'"}'
line one
line two
Here is an example of code injection:
variable='line one\nline two" ; for (i=1;i<=1000;++i) print i"'
awk 'BEGIN {print "'"$variable"'"}'
line one
line two
1
2
3
.
.
1000
You can add lots of commands to awk
this way. Even make it crash with non valid commands.
It's always good to double quote variable "$variable"
If not, multiple lines will be added as a long single line.
Example:
var="Line one
This is line two"
echo $var
Line one This is line two
echo "$var"
Line one
This is line two
Other errors you can get without double quote:
variable="line one\nline two"
awk -v var=$variable 'BEGIN {print var}'
awk: cmd. line:1: one\nline
awk: cmd. line:1: ^ backslash not last character on line
awk: cmd. line:1: one\nline
awk: cmd. line:1: ^ syntax error
And with single quote, it does not expand the value of the variable:
awk -v var='$variable' 'BEGIN {print var}'
$variable