Identifying and removing null characters in UNIX

unix shell null special-characters
dogbane picture dogbane · Mar 8, 2010 · Viewed 126.2k times · Source

I have a text file containing unwanted null characters (ASCII NUL, \0). When I try to view it in vi I see ^@ symbols, interleaved in normal text. How can I:

  1. Identify which lines in the file contain null characters? I have tried grepping for \0 and \x0, but this did not work.

  2. Remove the null characters? Running strings on the file cleaned it up, but I'm just wondering if this is the best way?

Answer

Pointy picture Pointy · Mar 8, 2010

I’d use tr:

tr < file-with-nulls -d '\000' > file-without-nulls

If you are wondering if input redirection in the middle of the command arguments works, it does. Most shells will recognize and deal with I/O redirection (<, >, …) anywhere in the command line, actually.

Related questions

How can I check if a directory exists in a Bash shell script?

What command can be used to check if a directory exists or not, within a Bash shell script?

Read More
In the shell, what does " 2>&1 " mean?

In a Unix shell, if I want to combine stderr and stdout into the stdout stream for further manipulation, I can append the following on the end of my command: 2>&1 So, if I want to use head on …

Read More
How to read a file into a variable in shell?

I want to read a file and save it in variable, but I need to keep the variable and not just print out the file. How can I do this? I have written this script but it isn't quite what …

Read More