How to use multiple arguments for awk with a shebang (i.e. #!)?

shell unix awk gawk shebang
Hans-Peter Störr picture Hans-Peter Störr · Nov 29, 2010 · Viewed 38.4k times · Source

I'd like to execute an gawk script with --re-interval using a shebang. The "naive" approach of 

#!/usr/bin/gawk --re-interval -f
... awk script goes here

does not work, since gawk is called with the first argument "--re-interval -f" (not splitted around the whitespace), which it does not understand. Is there a workaround for that?

Of course you can either not call gawk directly but wrap it into a shell script that splits the first argument, or make a shell script that then calls gawk and put the script into another file, but I was wondering if there was some way to do this within one file.

The behaviour of shebang lines differs from system to system - at least in Cygwin it does not split the arguments by whitespaces. I just care about how to do it on a system that behaves like that; the script is not meant to be portable.

Answer

Jörg W Mittag picture Jörg W Mittag · Nov 29, 2010

The shebang line has never been specified as part of POSIX, SUS, LSB or any other specification. AFAIK, it hasn't even been properly documented.

There is a rough consensus about what it does: take everything between the ! and the \n and exec it. The assumption is that everything between the ! and the \n is a full absolute path to the interpreter. There is no consensus about what happens if it contains whitespace.

  1. Some operating systems simply treat the entire thing as the path. After all, in most operating systems, whitespace or dashes are legal in a path.
  2. Some operating systems split at whitespace and treat the first part as the path to the interpreter and the rest as individual arguments.
  3. Some operating systems split at the first whitespace and treat the front part as the path to the interpeter and the rest as a single argument (which is what you are seeing).
  4. Some even don't support shebang lines at all.

Thankfully, 1. and 4. seem to have died out, but 3. is pretty widespread, so you simply cannot rely on being able to pass more than one argument.

And since the location of commands is also not specified in POSIX or SUS, you generally use up that single argument by passing the executable's name to env so that it can determine the executable's location; e.g.:

#!/usr/bin/env gawk

[Obviously, this still assumes a particular path for env, but there are only very few systems where it lives in /bin, so this is generally safe. The location of env is a lot more standardized than the location of gawk or even worse something like python or ruby or spidermonkey.]

Which means that you cannot actually use any arguments at all.

Related questions

Bash tool to get nth line from a file

Is there a "canonical" way of doing that? I've been using head -n | tail -1 which does the trick, but I've been wondering if there's a Bash tool that specifically extracts a line (or a range of lines) from a …

Read More
Trim leading and trailing spaces from a string in awk

I'm trying to remove leading and trailing space in 2nd column of the below input.txt: Name, Order   Trim, working cat,cat1 I have used the below awk to remove leading and trailing space in 2nd column but it is …

Read More
How to select lines between two marker patterns which may occur multiple times with awk/sed

Using awk or sed how can I select lines which are occurring between two different marker patterns? There may be multiple sections marked with these patterns. For example: Suppose the file contains: abc def1 ghi1 jkl1 mno abc def2 ghi2 …

Read More