'Embedded nul in string' error when importing csv with fread

r data.table fread
datahappy picture datahappy · Mar 25, 2014 · Viewed 25.9k times · Source

I have a large file (3.5G) that I'm trying to import using data.table::fread.

It was originally created from an rpt file that was opened as text and saved as a CSV.

This has worked fine with smaller files (of the same type of data-same columns and all. This one is just for a longer timeframe and wider reach).

When I try and run 

mydata <- fread("mycsv.csv")

I get the error:

Error in fread("mycsv.csv") : embedded nul in string: 'y\0e\0a\0r\0'

What does this mean?

Answer

Robert Krzyzanowski picture Robert Krzyzanowski · Mar 25, 2014

We can remove the null terminators on the command line using something like:

sed 's/\\0//g' mycsv.csv > mycsv.csv

Or as suggested by @marbel, fread allows you to pass the sed call inside the text. Such as:

fread("sed 's/\\0//g' mycsv.csv")

Related questions

fread in R imports a large .csv file as a data frame with one row

I'm importing a large .csv file into R (about 0.5 million rows), so I've been trying to use fread() from the data.table package as a faster alternative to read.table() and read.csv(). However, fread() returns a data frame with …

Read More
using colClasses in fread

I don't know how to choose specific columns using the colClasses option in fread. I tried to use NULL in several ways but nothing worked. Here's a minimal example. I just want columns 1 and 3. dt <- data.table(a=1:5,…

Read More
fread - skip lines starting with certain character - "#"

I am using the fread function in R for reading files to data.tables objects. However, when reading the file I'd like to skip lines that start with #, is that possible? I could not find any mention to that in …

Read More