I am a newbie to LSTM and RNN as a whole, I've been racking my brain to understand what exactly is a timestep. I would really appreciate an intuitive explanation to this
Let's start with a great image from Chris Olah's blog (a highly recommended read btw):
In a recurrent neural network you have multiple repetitions of the same cell. The way inference goes is - you take some input (x0), pass it through the cell to get some output1(depicted with black arrow to the right on the picture), then pass output1 as input(possibly adding some more input components - x1 on the image) to the same cell, producing new output output2, pass that again as input to the same cell(again with possibly additional input component x2), producing output3 and so on.
A time step is a single occurrence of the cell - e.g. on the first time step you produce output1, h0, on the second time step you produce output2 and so on.