How do I remove NUL characters from a CSV using Python

python-2.7 csv nul
lsward picture lsward · Aug 4, 2016 · Viewed 10.3k times · Source

I've noticed that several CSVs that come my way have random 'NUL' values placed through out the file. I noticed this because when I import the file into a database using a SSIS package I built those files throw a no column delimiter found error. I'm thinking about writing a python script to clean these files up, but I can't find a solution to this problem in Python. How would I use Python to remove these NUL characters?

I would include a picture, but I don't have enough reputation to include one.

Ex. "123456","Brown, Jim","","?NUL","",False,"8/16/2014 12:00:00 AM",""NUL,""InboNULund"

Answer

lsward picture lsward · Aug 15, 2016

I don't know how I didn't find this answer in my search, but this solution worked. It's weird that it worked because I tried the string replace method and it didn't seem to catch all of them but I think the through answer provided by @JohnMachin in this post really laid the ground work for me solving the problem. He provides a comprehensive way to investigate the problem and I suggest taking a look at it if you are having a similar issue. Python CSV error: line contains NULL byte

Related questions

Writing a Python list of lists to a csv file

I have a long list of lists of the following form --- a = [[1.2,'abc',3],[1.2,'werew',4],........,[1.4,'qew',2]] i.e. the values in the list are of different types -- float,int, strings.How do I write it into a csv file …

Read More
Skip the headers when editing a csv file using Python

I am using below referred code to edit a csv using Python. Functions called in the code form upper part of the code. Problem: I want the below referred code to start editing the csv from 2nd row, I want …

Read More
Reading a huge .csv file

I'm currently trying to read data from .csv files in Python 2.7 with up to 1 million rows, and 200 columns (files range from 100mb to 1.6gb). I can do this (very slowly) for the files with under 300,000 rows, but once I go …

Read More