Keras Extremely High Loss

Question

I'm trying to predict price by characteristics. I chose a pretty simple model, but it works very strange. Loss function is extremely high and I can't see where the problem is.

Here is my model:

# define base model
def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(62, input_dim = 62, kernel_initializer='normal', activation='relu'))
    model.add(Dense(31, kernel_initializer='normal', activation='relu'))
    model.add(Dense(15, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal'))
    # Compile model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model

That's how I prepare the data: (One-Hot and I split all data to train and test)

df = encode_onehot(dataframe, cols=['Shape', 'Cut', 'Color', 'Clarity', 'Polish', 'Symmetry', 'Culet', '\tFluorescence'])

dataset = df.values
X = dataset[1:,4:66]
Y = dataset[1:,2]

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.25, random_state=42)

Finally, training:

baseline_model().fit(X_train, y_train, epochs=10, batch_size=64)
scores = baseline_model().evaluate(X_test, y_test, verbose=0)
print(baseline_model().summary())

And results are very sad:

Epoch 1/10
149767/149767 [==============================] - 4s - loss: 104759338.0333     
Epoch 2/10
149767/149767 [==============================] - 4s - loss: 104594236.9627     
Epoch 3/10
149767/149767 [==============================] - 4s - loss: 104556662.2948     

And it doesn't get better.

What am I doing wrong?

Answer 1

As @Yu-Yang said you are using mean squared error as loss function. I had this same problem before where the loss value will be very large, on changing the loss function to mean_squared_logarithmic_error, I got the desired result.

model %>% compile(
optimizer = optimizer_rmsprop(lr=0.0001),
loss = loss_mean_squared_logarithmic_error,
metrics = c("accuracy")
)

The loss value changed to

Epoch 1/10
326981/326981 [==============================] - 17s - loss: 0.0048 - acc: 0.9896

Hope this finds useful !