R Metric RMSE not applicable for classification models

paveltr picture paveltr · Apr 16, 2016 · Viewed 8.6k times · Source

I am trying to investigate my model with R with xgboost. Training model in general works well, but with caret it is some problem with metric.

I tried to set a factor for a class column, bit there is still no result.

My data

ID  var1var2TARGET
1   5   0   1
2   4   3   1
3   4   2   0
4   3   1   0
5   2   4   1
6   1   2   1
7   5   3   1
8   4   1   0
9   4   1   0
10  2   4   1
11  5   5   1

For this i do

train <- read.csv()
train.y <- train$TARGET
train$TARGET <- NULL
train$ID <- NULL
train.y <- lapply(train.y, factor)

Then I prepare model parameters

xgb_grid_1 = expand.grid(
  nrounds = 1000,
  eta = c(0.01, 0.001, 0.0001),
  max_depth = c(2, 4, 6, 8, 10),
  gamma = 1
)

# pack the training control parameters
xgb_trcontrol_1 = trainControl(
  method = "cv",
  number = 5,
  verboseIter = TRUE,
  returnData = FALSE,
  returnResamp = "all",                                                        # save losses across all models
  classProbs = TRUE,                                                           # set to TRUE for AUC to be computed
  summaryFunction = twoClassSummary,
  allowParallel = TRUE
)

And after all of that, I call train function

xgb_train_1 = train(
  x = train,
  y = train.y,
  trControl = xgb_trcontrol_1,
  tuneGrid = xgb_grid_1,
  method = "xgbTree"
)

It gives me

Error in train.default(x = train, y = train.y, trControl = xgb_trcontrol_1,  : 
  Metric RMSE not applicable for classification models

Why could it be?

Answer

Llu&#237;s Ramon picture Lluís Ramon · Apr 16, 2016

You should try to change train.y <- lapply(train.y, factor) to train.y <- factor(train.y, labels = c("yes", "no")).

caret usually complains if labels are 0 or 1, so try to change it.