Pandas TimeSeries resample produces NaNs
I am resampling a Pandas TimeSeries. The timeseries consist of binary values (it is a categorical variable) with no missing values, but after resampling NaNs appear. How is this possible?
I can't post any example data here since it is sensitive info, but I create and resample the series as follows:
series = pd.Series(data, ts)
series_rs = series.resample('60T', how='mean')
Answer
upsampling converts to a regular time interval, so if there are no samples you get NaN.
You can fill missing values backward by fill_method='bfill' or for forward - fill_method='ffill' or fill_method='pad'.
import pandas as pd
ts = pd.date_range('1/1/2015', periods=10, freq='100T')
data = range(10)
series = pd.Series(data, ts)
print series
#2015-01-01 00:00:00 0
#2015-01-01 01:40:00 1
#2015-01-01 03:20:00 2
#2015-01-01 05:00:00 3
#2015-01-01 06:40:00 4
#2015-01-01 08:20:00 5
#2015-01-01 10:00:00 6
#2015-01-01 11:40:00 7
#2015-01-01 13:20:00 8
#2015-01-01 15:00:00 9
#Freq: 100T, dtype: int64
series_rs = series.resample('60T', how='mean')
print series_rs
#2015-01-01 00:00:00 0
#2015-01-01 01:00:00 1
#2015-01-01 02:00:00 NaN
#2015-01-01 03:00:00 2
#2015-01-01 04:00:00 NaN
#2015-01-01 05:00:00 3
#2015-01-01 06:00:00 4
#2015-01-01 07:00:00 NaN
#2015-01-01 08:00:00 5
#2015-01-01 09:00:00 NaN
#2015-01-01 10:00:00 6
#2015-01-01 11:00:00 7
#2015-01-01 12:00:00 NaN
#2015-01-01 13:00:00 8
#2015-01-01 14:00:00 NaN
#2015-01-01 15:00:00 9
#Freq: 60T, dtype: float64
series_rs = series.resample('60T', how='mean', fill_method='bfill')
print series_rs
#2015-01-01 00:00:00 0
#2015-01-01 01:00:00 1
#2015-01-01 02:00:00 2
#2015-01-01 03:00:00 2
#2015-01-01 04:00:00 3
#2015-01-01 05:00:00 3
#2015-01-01 06:00:00 4
#2015-01-01 07:00:00 5
#2015-01-01 08:00:00 5
#2015-01-01 09:00:00 6
#2015-01-01 10:00:00 6
#2015-01-01 11:00:00 7
#2015-01-01 12:00:00 8
#2015-01-01 13:00:00 8
#2015-01-01 14:00:00 9
#2015-01-01 15:00:00 9
#Freq: 60T, dtype: float64