How to unpack pkl file?

Question

I have a pkl file from MNIST dataset, which consists of handwritten digit images.

I'd like to take a look at each of those digit images, so I need to unpack the pkl file, except I can't find out how.

Is there a way to unpack/unzip pkl file?

Answer 1

Generally

Your pkl file is, in fact, a serialized pickle file, which means it has been dumped using Python's pickle module.

To un-pickle the data you can:

import pickle


with open('serialized.pkl', 'rb') as f:
    data = pickle.load(f)

For the MNIST data set

Note gzip is only needed if the file is compressed:

import gzip
import pickle


with gzip.open('mnist.pkl.gz', 'rb') as f:
    train_set, valid_set, test_set = pickle.load(f)

Where each set can be further divided (i.e. for the training set):

train_x, train_y = train_set

Those would be the inputs (digits) and outputs (labels) of your sets.

If you want to display the digits:

import matplotlib.cm as cm
import matplotlib.pyplot as plt


plt.imshow(train_x[0].reshape((28, 28)), cmap=cm.Greys_r)
plt.show()

The other alternative would be to look at the original data:

http://yann.lecun.com/exdb/mnist/

But that will be harder, as you'll need to create a program to read the binary data in those files. So I recommend you to use Python, and load the data with pickle. As you've seen, it's very easy. ;-)