Remove rows not .isin('X')

python filtering pandas
DrewH picture DrewH · Dec 27, 2012 · Viewed 94.8k times · Source

Sorry just getting into Pandas, this seems like it should be a very straight forward question. How can I use the isin('X') to remove rows that are in the list X? In R I would write !which(a %in% b).

Answer

Jonny Brooks picture Jonny Brooks · Feb 9, 2017

You have many options. Collating some of the answers above and the accepted answer from this post you can do:
1. df[-df["column"].isin(["value"])]
2. df[~df["column"].isin(["value"])]
3. df[df["column"].isin(["value"]) == False]
4. df[np.logical_not(df["column"].isin(["value"]))]

Note: for option 4 for you'll need to import numpy as np

Update: You can also use the .query method for this too. This allows for method chaining:
5. df.query("column not in @values").
where values is a list of the values that you don't want to include.

Related questions

Truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

Having issue filtering my result dataframe with an or condition. I want my result df to extract all column var values that are above 0.25 and below -0.25. This logic below gives me an ambiguous truth value however it work when …

Read More
Filtering Pandas DataFrames on dates

I have a Pandas DataFrame with a 'date' column. Now I need to filter out all rows in the DataFrame that have dates outside of the next two months. Essentially, I only need to retain the rows that are within …

Read More
Detect and exclude outliers in Pandas data frame

I have a pandas data frame with few columns. Now I know that certain rows are outliers based on a certain column value. For instance column 'Vol' has all values around 12xx and one value is 4000 (outlier). Now I would …

Read More