What is the difference between steps and epochs in TensorFlow?

machine-learning tensorflow
Yang picture Yang · Jul 13, 2016 · Viewed 87.9k times · Source

In most of the models, there is a steps parameter indicating the number of steps to run over data. But yet I see in most practical usage, we also execute the fit function N epochs.

What is the difference between running 1000 steps with 1 epoch and running 100 steps with 10 epoch? Which one is better in practice? Any logic changes between consecutive epochs? Data shuffling?

Answer

MarvMind picture MarvMind · Jun 7, 2017

A training step is one gradient update. In one step batch_size many examples are processed.

An epoch consists of one full cycle through the training data. This is usually many steps. As an example, if you have 2,000 images and use a batch size of 10 an epoch consists of 2,000 images / (10 images / step) = 200 steps.

If you choose our training image randomly (and independent) in each step, you normally do not call it epoch. [This is where my answer differs from the previous one. Also see my comment.]

Related questions

Tensorflow: how to save/restore a model?

After you train a model in Tensorflow: How do you save the trained model? How do you later restore this saved model?

Read More
What is logits, softmax and softmax_cross_entropy_with_logits?

I was going through the tensorflow API docs here. In the tensorflow documentation, they used a keyword called logits. What is it? In a lot of methods in the API docs it is written like tf.nn.softmax(logits, name=…

Read More
TensorFlow, "'module' object has no attribute 'placeholder'"

I've been trying to use tensorflow for two days now installing and reinstalling it over and over again in python2.7 and 3.4. No matter what I do, I get this error message when trying to use tensorflow.placeholder() It's very boilerplate …

Read More