neuralnet prediction returns the same values for all predictions
I'm trying to build a neural net with the neuralnet package and I'm having some trouble with it. I've been successful with the nnet package but no luck with the neuralnet one. I have read the whole documentation package and can't find the solution, or maybe I'm not able to spot it.
The training command I'm using is
nn<-neuralnet(V15 ~ V1 + V2 + V3 + V4 + V5 + V6 + V7 + V8 + V9 + V10 + V11 + V12 + V13 + V14,data=test.matrix,lifesign="full",lifesign.step=100,hidden=8)
and for prediction
result<- compute(nn,data.matrix)$net.result
The training takes a whole lot longer than the nnet training. I have tried using the same algorithm as nnet (backpropagation instead of resilent backpropagation) and nothing, changed the activation function too (and the linear.output=F) and pretty much everything else, and the result didn't improved. Predicted values are all the same. I don't understand why the nnet works for me, while the neuralnet one doesn't.
I could really use some help, my (lack of) understanding of both things (neural nets and R) it's probably the cause, but can't find why.
My dataset is from UCI. I want to use the neural network for a binary classification. A sample of the data would be:
25,Private,226802,11th,7,Never-married,Machine-op-inspct,Own-child,Black,Male,0,0,40,United-States,<=50K.
38,Private,89814,HS-grad,9,Married-civ-spouse,Farming-fishing,Husband,White,Male,0,0,50,United-States,<=50K.
28,Local-gov,336951,Assoc-acdm,12,Married-civ-spouse,Protective-serv,Husband,White,Male,0,0,40,United-States,>50K.
44,Private,160323,Some-college,10,Married-civ-spouse,Machine-op-inspct,Husband,Black,Male,7688,0,40,United-States,>50K.
18,?,103497,Some-college,10,Never-married,NA,Own-child,White,Female,0,0,30,United-States,<=50K.
34,Private,198693,10th,6,Never-married,Other-service,Not-in-family,White,Male,0,0,30,United-States,<=50K.
29,?,227026,HS-grad,9,Never-married,?,Unmarried,Black,Male,0,0,40,United-States,<=50K.
63,Self-emp-not-inc,104626,Prof-school,15,Married-civ-spouse,Prof-specialty,Husband,White,Male,3103,0,32,United-States,>50K.
24,Private,369667,Some-college,10,Never-married,Other-service,Unmarried,White,Female,0,0,40,United-States,<=50K.
55,Private,104996,7th-8th,4,Married-civ-spouse,Craft-repair,Husband,White,Male,0,0,10,United-States,<=50K.
65,Private,184454,HS-grad,9,Married-civ-spouse,Machine-op-inspct,Husband,White,Male,6418,0,40,United-States,>50K.
36,Federal-gov,212465,Bachelors,13,Married-civ-spouse,Adm-clerical,Husband,White,Male,0,0,40,United-States,<=50K.
26,Private,82091,HS-grad,9,Never-married,Adm-clerical,Not-in-family,White,Female,0,0,39,United-States,<=50K.
Converted into a matrix, with the factors as numerical values:
V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15
39 7 77516 10 13 5 1 2 5 2 2174 0 40 39 0
50 6 83311 10 13 3 4 1 5 2 0 0 13 39 0
38 4 215646 12 9 1 6 2 5 2 0 0 40 39 0
53 4 234721 2 7 3 6 1 3 2 0 0 40 39 0
28 4 338409 10 13 3 10 6 3 1 0 0 40 5 0
37 4 284582 13 14 3 4 6 5 1 0 0 40 39 0
49 4 160187 7 5 4 8 2 3 1 0 0 16 23 0
52 6 209642 12 9 3 4 1 5 2 0 0 45 39 1
31 4 45781 13 14 5 10 2 5 1 14084 0 50 39 1
42 4 159449 10 13 3 4 1 5 2 5178 0 40 39 1
37 4 280464 16 10 3 4 1 3 2 0 0 80 39 1
30 7 141297 10 13 3 10 1 2 2 0 0 40 19 1
23 4 122272 10 13 5 1 4 5 1 0 0 30 39 0
Summary of the predicted values:
V1
Min. :0.2446871
1st Qu.:0.2446871
Median :0.2446871
Mean :0.2451587
3rd Qu.:0.2446871
Max. :1.0000000
Value of the Wilcoxon-Mann-Whitney test (area under the curve) shows that the prediction performance is virtualy the same as a random.
performance(predneural,"auc")@y.values
[1] 0.5013319126
Answer
The first reason to consider when you get weird results with neural networks is normalization. Your data must be normalized, otherwise, yes, the training will result in skewed NN which will produce the same outcome all the time, it is a common symptom.
Looking at your data set, there are values >>1 which means they are all treated by NN essentially the same. The reason for it is that the traditionally used response functions are (almost) constant outside some range around 0.
Always normalize your data before feeding it into a neural network.