Loading a date in Numpy genfromtxt

python numpy genfromtxt
Mark Morrisson picture Mark Morrisson · Jan 7, 2016 · Viewed 7.1k times · Source

I'm trying to import a simple CSV file with Numpy genfromtxt but can't manage to convert the data of first column to dates.

Here is my code:

import numpy as np
from datetime import datetime

str2date = lambda x: datetime.strptime(x, '%Y-%m-%d %H:%M:%S')

data = np.genfromtxt('C:\\\\data.csv',dtype=None,names=True, delimiter=',', converters = {0: str2date})

I get the following error in str2date:

TypeError: must be str, not bytes

The problem is there are many columns, so I'd prefer avoiding the specification of all the column types (which are basically numerical).

Answer

N. Wouda picture N. Wouda · Jan 7, 2016

The problem is that the argument passed to str2date is of this form b'%Y-%m-%d %H:%M:%S'. These are bytes, which rightfully cannot be parsed to a datetime object. The solution to that problem is quite simple though, as you should decode your byte string to a UTF-8 string:

str2date = lambda x: datetime.strptime(x.decode("utf-8"), '%Y-%m-%d %H:%M:%S')

Related questions

How do I read CSV data into a record array in NumPy?

I wonder if there is a direct way to import the contents of a CSV file into a record array, much in the way that R's read.table(), read.delim(), and read.csv() family imports data to R's data frame? …

Read More
Reading data into numpy array from text file

I have a file with some metadata, and then some actual data consisting of 2 columns with headings. Do I need to separate the two types of data before using genfromtxt in numpy? Or can I somehow split the data maybe? …

Read More
Read in all csv files from a directory using Python

I hope this is not trivial but I am wondering the following: If I have a specific folder with n csv files, how could I iteratively read all of them, one at a time, and perform some calculations on their …

Read More