Python, Pandas : Return only those rows which have missing values

python pandas missing-data
user2487726 picture user2487726 · May 26, 2015 · Viewed 62.4k times · Source

While working in Pandas in Python...

I'm working with a dataset that contains some missing values, and I'd like to return a dataframe which contains only those rows which have missing data. Is there a nice way to do this?

(My current method to do this is an inefficient "look to see what index isn't in the dataframe without the missing values, then make a df out of those indices.")

Answer

metersk picture metersk · May 26, 2015

You can use any axis=1 to check for least one True per row, then filter with boolean indexing:

null_data = df[df.isnull().any(axis=1)]

Related questions

How to lowercase a pandas dataframe string column if it has missing values?

The following code does not work. import pandas as pd import numpy as np df=pd.DataFrame(['ONE','Two', np.nan],columns=['x']) xLower = df["x"].map(lambda x: x.lower()) How should I tweak it to get xLower = ['one',…

Read More
Best way to count the number of rows with missing values in a pandas DataFrame

I currently came up with some work arounds to count the number of missing values in a pandas DataFrame. Those are quite ugly and I am wondering if there is a better way to do it. Let's create an example …

Read More
Can't drop NAN with dropna in pandas

I import pandas as pd and run the code below and get the following result Code: traindataset = pd.read_csv('/Users/train.csv') print traindataset.dtypes print traindataset.shape print traindataset.iloc[25,3] traindataset.dropna(how='any') print traindataset.iloc[25,3] …

Read More