Assign string containing null-character (\0) to a variable in Bash

bash null-character
antiplex picture antiplex · Jul 4, 2011 · Viewed 35.4k times · Source

While trying to process a list of file-/foldernames correctly (see my other questions) through the use of a NULL-character as a delimiter I stumbled over a strange behaviour of Bash that I don't understand:

When assigning a string containing one or more NULL-character to a variable, the NULL-characters are lost / ignored / not stored.

For example,

echo -ne "n\0m\0k" | od -c   # -> 0000000   n  \0   m  \0   k

But:

VAR1=`echo -ne "n\0m\0k"`
echo -ne "$VAR1" | od -c   # -> 0000000   n   m   k

This means that I would need to write that string to a file (for example, in /tmp) and read it back from there if piping directly is not desired or feasible.

When executing these scripts in Z shell (zsh) the strings containing \0 are preserved in both cases, but sadly I can't assume that zsh is present in the systems running my script while Bash should be.

How can strings containing \0 chars be stored or handled efficiently without losing any (meta-) characters?

Answer

jeff picture jeff · Jul 4, 2011

In Bash, you can't store the NULL-character in a variable.

You may, however, store a plain hex dump of the data (and later reverse this operation again) by using the xxd command.

VAR1=`echo -ne "n\0m\0k" | xxd -p | tr -d '\n'`
echo -ne "$VAR1" | xxd -r -p | od -c   # -> 0000000    n  \0   m  \0   k

Related questions

Reading null delimited strings through a Bash loop

I want to iterate through a list of files without caring about what characters the filenames might contain, so I use a list delimited by null characters. The code will explain things better. # Set IFS to the null character to …

Read More
How to concatenate string variables in Bash

In PHP, strings are concatenated together as follows: $foo = "Hello"; $foo .= " World"; Here, $foo becomes "Hello World". How is this accomplished in Bash?

Read More
How can I check if a directory exists in a Bash shell script?

What command can be used to check if a directory exists or not, within a Bash shell script?

Read More