I have a data frame with some columns with empty lists and others with lists of strings:
donation_orgs donation_context
0 [] []
1 [the research of Dr. ...] [In lieu of flowers , memorial donations ...]
I'm trying to return a data set without any of the rows where there are empty lists.
I've tried just checking for null values:
dfnotnull = df[df.donation_orgs != []]
dfnotnull
and
dfnotnull = df[df.notnull().any(axis=1)]
pd.options.display.max_rows=500
dfnotnull
And I've tried looping through and checking for values that exist, but I think the lists aren't returning Null or None like I thought they would:
dfnotnull = pd.DataFrame(columns=('donation_orgs', 'donation_context'))
for i in range(0,len(df)):
if df['donation_orgs'].iloc(i):
dfnotnull.loc[i] = df.iloc[i]
All three of the above methods simply return every row in the original data frame.=
To avoid converting to str
and actually use the list
s, you can do this:
df[df['donation_orgs'].map(lambda d: len(d)) > 0]
It maps the donation_orgs
column to the length of the lists of each row and keeps only the ones that have at least one element, filtering out empty lists.
It returns
Out[1]:
donation_context donation_orgs
1 [In lieu of flowers , memorial donations] [the research of Dr.]
as expected.