Getting timestamp of each frame in a video

Question

I have recorded several videos from the front cam of my tablet with an Android 5.2 application I have written. I have stored the start timestamp in milliseconds (Unix time) for each video.

Unfortunately each video has a different framerate (ranging from 20 to 30). With OpenCV I'm able to get the framerate for each video:

import cv2
video = cv2.VideoCapture(videoFile)
fps = video.get(cv2.CAP_PROP_FPS)

This works well and theoretically I could just add 1000/fps (due to milliseconds) for each frame in the video. But this assumes that the framerate remains stable throughout the whole recording. I don't know if this is the case.

Is there a possibility in Python to get the timestamp (in milliseconds) of each frame in the video independent of the framerate?

Answer 1

You want cv2.CAP_PROP_POS_MSEC. See all the different capture properties here.

Edit: Actually, as Dan Mašek pointed out to me, when you grab that property, it looks like OpenCV is exactly doing that calculation (at least assuming you're using FFMPEG):

case CV_FFMPEG_CAP_PROP_POS_MSEC:
    return 1000.0*(double)frame_number/get_fps();

So it seems you're always going to rely on a constant frame rate assumption. However, even assuming a constant frame rate, it's important that you multiply by the frame number and not just keep adding 1000/fps. Errors will build up when you're repeatedly adding floats which, over a long video, can make a big difference. For example:

import cv2

cap = cv2.VideoCapture('vancouver2.mp4')
fps = cap.get(cv2.CAP_PROP_FPS)

timestamps = [cap.get(cv2.CAP_PROP_POS_MSEC)]
calc_timestamps = [0.0]

while(cap.isOpened()):
    frame_exists, curr_frame = cap.read()
    if frame_exists:
        timestamps.append(cap.get(cv2.CAP_PROP_POS_MSEC))
        calc_timestamps.append(calc_timestamps[-1] + 1000/fps)
    else:
        break

cap.release()

for i, (ts, cts) in enumerate(zip(timestamps, calc_timestamps)):
    print('Frame %d difference:'%i, abs(ts - cts))

Frame 0 difference: 0.0
Frame 1 difference: 0.0
Frame 2 difference: 0.0
Frame 3 difference: 1.4210854715202004e-14
Frame 4 difference: 0.011111111111091532
Frame 5 difference: 0.011111111111091532
Frame 6 difference: 0.011111111111091532
Frame 7 difference: 0.011111111111119953
Frame 8 difference: 0.022222222222183063
Frame 9 difference: 0.022222222222183063
...
Frame 294 difference: 0.8111111111411446

This is of course in milliseconds, so maybe it doesn't seem that big. But here I'm almost 1ms off in the calculation, and this is just for an 11-second video. And anyways, using this property is just easier.