Datetime in pandas dataframe will not subtract from each other

Graham Streich picture Graham Streich · Jun 17, 2017 · Viewed 15.1k times · Source

I am trying to find the difference in times between two columns in a pandas dataframe both in datetime format.

Below is some of the data in my dataframe and the code I have been using. I have triple checked that these two columns dtypes are datetime64.

My data:

date_updated                  date_scored 
2016-03-30 08:00:00.000       2016-03-30 08:00:57.416  
2016-04-07 23:50:00.000       2016-04-07 23:50:12.036 

My code:

data['date_updated'] = pd.to_datetime(data['date_updated'], 
format='%Y-%m-%d %H:%M:%S')
data['date_scored'] = pd.to_datetime(data['date_scored'], 
format='%Y-%m-%d %H:%M:%S')
data['Diff'] =  data['date_updated'] - data['date_scored']

The error message I receive:

TypeError: data type "datetime" not understood

Any help would be appreciated, thanks!

My work around solution:

for i in raw_data[:10]:
scored = i.date_scored
scored_date =  pd.to_datetime(scored, format='%Y-%m-%d %H:%M:%S')
if type(scored_date) == "NoneType":
    pass
elif scored_date.year >= 2016:
    extracted = i.date_extracted
    extracted =  pd.to_datetime(extracted, format='%Y-%m-%d %H:%M:%S')
    bank = i.bank.name
    diff = scored - extracted
    datum = [str(bank), str(extracted), str(scored), str(diff)]
    data.append(datum)
else:
    pass

Answer

ac2001 picture ac2001 · Sep 7, 2017

I encountered the same error using the above syntax (worked on another machine though):

data['Diff'] =  data['date_updated'] - data['date_scored']

It worked on my new machine with:

data['Diff'] =  data['date_updated'].subtract(data['date_scored'])