predict_proba for a cross-validated model

Question

I would like to predict the probability from Logistic Regression model with cross-validation. I know you can get the cross-validation scores, but is it possible to return the values from predict_proba instead of the scores?

# imports
from sklearn.linear_model import LogisticRegression
from sklearn.cross_validation import (StratifiedKFold, cross_val_score,
                                      train_test_split)
from sklearn import datasets

# setup data
iris = datasets.load_iris()
X = iris.data
y = iris.target

# setup model
cv = StratifiedKFold(y, 10)
logreg = LogisticRegression()

# cross-validation scores
scores = cross_val_score(logreg, X, y, cv=cv)

# predict probabilities
Xtrain, Xtest, ytrain, ytest = train_test_split(X, y)
logreg.fit(Xtrain, ytrain)
proba = logreg.predict_proba(Xtest)

Answer 1

This is now implemented as part of scikit-learn version 0.18. You can pass a 'method' string parameter to the cross_val_predict method. Documentation is here.

Example:

proba = cross_val_predict(logreg, X, y, cv=cv, method='predict_proba')

Also note that this is part of the new sklearn.model_selection package so you will need this import:

from sklearn.model_selection import cross_val_predict