Count number of non-NaN entries in every column of Dataframe

python pandas dataframe count nan
cryp picture cryp · Apr 30, 2015 · Viewed 72.8k times · Source

I have a really big DataFrame and I was wondering if there was short (one or two liner) way to get the a count of non-NaN entries in a DataFrame. I don't want to do this one column at a time as I have close to 1000 columns. 

df1 = pd.DataFrame([(1,2,None),(None,4,None),(5,None,7),(5,None,None)], 
                    columns=['a','b','d'], index = ['A', 'B','C','D'])

    a   b   d
A   1   2 NaN
B NaN   4 NaN
C   5 NaN   7
D   5 NaN NaN

Output:

a: 3
b: 2
d: 1

Answer

Alex Riley picture Alex Riley · Apr 30, 2015

The count() method returns the number of non-NaN values in each column:

>>> df1.count()
a    3
b    2
d    1
dtype: int64

Similarly, count(axis=1) returns the number of non-NaN values in each row.

Related questions

How to select rows from a DataFrame based on column values

How can I select rows from a DataFrame based on values in some column in Pandas? In SQL, I would use: SELECT * FROM table WHERE colume_name = some_value I tried to look at Pandas' documentation, but I did not …

Read More
Renaming columns in pandas

I have a DataFrame using pandas and column labels that I need to edit to replace the original column labels. I'd like to change the column names in a DataFrame A where the original column names are: ['$a', '$…

Read More
How to iterate over rows in a DataFrame in Pandas

I have a DataFrame from Pandas: import pandas as pd inp = [{'c1':10, 'c2':100}, {'c1':11,'c2':110}, {'c1':12,'c2':120}] df = pd.DataFrame(inp) print df Output: c1 c2 0 10 100 1 11 110 2 12 120 Now I want to iterate over the rows …

Read More