Python pandas .isnull() does not work on NaT in object dtype

python datetime pandas isnull
ragesz picture ragesz · Sep 30, 2015 · Viewed 10.3k times · Source

I have this series:

ser=pd.Series([11,22,33,np.nan,np.datetime64('nat')],name='my_series')

The series looks like this:

0     11
1     22
2     33
3    NaN
4    NaN
Name: my_series, dtype: object

But I get only one True for NULL values:

ser.isnull()

0    False
1    False
2    False
3     True
4    False
Name: my_series, dtype: bool

Is it a bug or how can I count correctly the NULL values in a pandas series? This does not help:

ser=ser.replace('NaN',np.nan)

Thanks!

Answer

nck picture nck · Apr 9, 2019

I ran into a similar issue this morning but in an str series! This worked for me and with your sample data as well:

pd.isna(ser)

Related questions

Convert Pandas Column to DateTime

I have one field in a pandas DataFrame that was imported as string format. It should be a datetime variable. How do I convert it to a datetime column and then filter based on date. Example: DataFrame Name: raw_data …

Read More
Converting between datetime, Timestamp and datetime64

How do I convert a numpy.datetime64 object to a datetime.datetime (or Timestamp)? In the following code, I create a datetime, timestamp and datetime64 objects. import datetime import numpy as np import pandas as pd dt = datetime.datetime(2012, 5, 1) # A …

Read More
Keep only date part when using pandas.to_datetime

I use pandas.to_datetime to parse the dates in my data. Pandas by default represents the dates with datetime64[ns] even though the dates are all daily only. I wonder whether there is an elegant/clever way to convert …

Read More