I am getting a
Classification metrics can't handle a mix of multilabel-indicator and multiclass targets
error when I try to use confusion matrix.
I am doing my first deep learning project. I am new to it. I am using the mnist dataset provided by keras. I have trained and tested my model successfully.
However, when I try to use the scikit learn confusion matrix I get the error stated above. I have searched for an answer and while there are answers on this error, none of them worked for me. From what I found online it probably has something to do with the loss function (I use the categorical_crossentropy
in my code). I tried changing it to sparse_categorical_crossentropy
but that just gave me the
Error when checking target: expected dense_2 to have shape (1,) but got array with shape (10,)
when I run the fit()
function on the model.
This is the code. (I have left out the imports for the sake of brevity)
model = Sequential()
model.add(Dense(512, activation='relu', input_shape=(28 * 28,)))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics=['accuracy'])
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images.reshape((60000, 28 * 28))
train_images = train_images.astype('float32') / 255
test_images = test_images.reshape((10000, 28 * 28))
test_images = test_images.astype('float32') / 255
train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)
model.fit(train_images, train_labels, epochs=10, batch_size=128)
rounded_predictions = model.predict_classes(test_images, batch_size=128, verbose=0)
cm = confusion_matrix(test_labels, rounded_predictions)
How can i fix this?
Confusion matrix needs both labels & predictions as single-digits, not as one-hot encoded vectors; although you have done this with your predictions using model.predict_classes()
, i.e.
rounded_predictions = model.predict_classes(test_images, batch_size=128, verbose=0)
rounded_predictions[1]
# 2
your test_labels
are still one-hot encoded:
test_labels[1]
# array([0., 0., 1., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)
So, you should convert them too to single-digit ones, as follows:
import numpy as np
rounded_labels=np.argmax(test_labels, axis=1)
rounded_labels[1]
# 2
After which, the confusion matrix should come up OK:
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(rounded_labels, rounded_predictions)
cm
# result:
array([[ 971, 0, 0, 2, 1, 0, 2, 1, 3, 0],
[ 0, 1121, 2, 1, 0, 1, 3, 0, 7, 0],
[ 5, 4, 990, 7, 5, 3, 2, 7, 9, 0],
[ 0, 0, 0, 992, 0, 2, 0, 7, 7, 2],
[ 2, 0, 2, 0, 956, 0, 3, 3, 2, 14],
[ 3, 0, 0, 10, 1, 872, 3, 0, 1, 2],
[ 5, 3, 1, 1, 9, 10, 926, 0, 3, 0],
[ 0, 7, 10, 1, 0, 2, 0, 997, 1, 10],
[ 5, 0, 3, 7, 5, 7, 3, 4, 937, 3],
[ 5, 5, 0, 9, 10, 3, 0, 8, 3, 966]])