how to implement walk forward testing in sklearn?

python scikit-learn time-series cross-validation
PhilChang picture PhilChang · Aug 11, 2015 · Viewed 8.3k times · Source

In sklearn, GridSearchCV can take a pipeline as a parameter to find the best estimator through cross validation. However, the usual cross validation is like this:enter image description here

to cross validate a time series data, the training and testing data are often splitted like this:enter image description here

That is to say, the testing data should be always ahead of training data.

My thought is:

  1. Write my own version class of k-fold and passing it to GridSearchCV so I can enjoy the convenience of pipeline. The problem is that it seems difficult to let GridSearchCV to use an specified indices of training and testing data.

  2. Write a new class GridSearchWalkForwardTest which is similar to GridSearchCV, I am studying the source code grid_search.py and find it is a little complicated.

Any suggestion is welcome.

Answer

Matthijs Brouns picture Matthijs Brouns · Apr 10, 2017

I think you could use a Time Series Split either instead of your own implementation or as a basis for implementing a CV method which is exactly as you describe it.

After digging around a bit, it seems like someone added a max_train_size to the TimeSeriesSplit in this PR which seems like it does what you want.

Related questions

How to predict time series in scikit-learn?

Scikit-learn utilizes a very convenient approach based on fit and predict methods. I have time-series data in the format suited for fit and predict. For example I have the following Xs: [[1.0, 2.3, 4.5], [6.7, 2.7, 1.2], ..., [3.2, 4.7, 1.1]] and the corresponding ys: [[1.0], [2.3], ..., [7.7]] These data have the following …

Read More
Time series forecasting with scikit learn

I am a complete newbie to SVM-based forecasting and so looking for some guidance here. I am trying to set-up a python code for forecasting a time-series, using SVM libraries of scikit-learn. My data contains X values at 30 minute interval …

Read More
How to normalize an array in NumPy?

I would like to have the norm of one NumPy array. More specifically, I am looking for an equivalent version of this function def normalize(v): norm = np.linalg.norm(v) if norm == 0: return v return v / norm Is there …

Read More