How to filter a pandas DatetimeIndex by day of week and hour in the day

motam79 picture motam79 · Jan 31, 2018 · Viewed 8.3k times · Source

I have a pandas DatetimeIndex and I would like to filter the index by the criterion that the day of the week and hour of the day matches a list. For example, I have of list of tuples indicating valid (day of week, hour, minute) for each TimeStamp:

[(4, 6), (5, 7)]

The final index should only contain date times that are Friday(day_of_week = 4) hour 6 or Saturday(day_of_week = 5) hour 7.

Lets say the input data frame is like:

2016-04-02 06:30:00  1
2016-04-02 06:45:00  2
2016-04-02 07:00:00  3
2016-04-02 07:15:00  4
2016-04-03 07:30:00  5
2016-04-03 07:45:00  6
2016-04-03 08:00:00  7

After the filter, it should be like:

2016-04-02 06:30:00  1
2016-04-02 06:45:00  2
2016-04-03 07:30:00  5

Because I only keep indices whose day of week and hour of the day in the list [(4, 6), (5, 7)]

Answer

Thomas picture Thomas · Jan 31, 2018

You should add a column day_of_week and a column hour, then you can filer on this columns.

For example :

df["day_of_week"] = df["date"].dayofweek()
df["hour"] = df["date"].hour()

pd.concat([
    df.loc[df["day_of_week"].isin(x[0]) & df["hour"].isin(x[1])]
    for x in [(4, 6), (5, 7)]
])

Note that I iterate over all your conditions, then I concatenate all the resulting dataframe.