What exactly is timestep in an LSTM Model?

Steven Wang picture Steven Wang · Jan 17, 2019 · Viewed 14k times · Source

I am a newbie to LSTM and RNN as a whole, I've been racking my brain to understand what exactly is a timestep. I would really appreciate an intuitive explanation to this

Answer

Ivaylo Strandjev picture Ivaylo Strandjev · Jan 17, 2019

Let's start with a great image from Chris Olah's blog (a highly recommended read btw):

enter image description here

In a recurrent neural network you have multiple repetitions of the same cell. The way inference goes is - you take some input (x0), pass it through the cell to get some output1(depicted with black arrow to the right on the picture), then pass output1 as input(possibly adding some more input components - x1 on the image) to the same cell, producing new output output2, pass that again as input to the same cell(again with possibly additional input component x2), producing output3 and so on.

A time step is a single occurrence of the cell - e.g. on the first time step you produce output1, h0, on the second time step you produce output2 and so on.