How to read mp4 video to be processed by scikit-image?

gaggio picture gaggio · Apr 18, 2015 · Viewed 44.1k times · Source

I would like to apply a scikit-image function (specifically the template matching function match_template) to the frames of a mp4 video, h264 encoding. It's important for my application to track the time of each frame, but I know the framerate so I can easily calculate from the frame number.

Please note that I'm running on low resources, and I would like to keep dependencies as slim as possible: numpy is needed anyway, and since I'm planning to use scikit-image, I would avoid importing (and compiling) openCV just to read the video.

I see at the bottom of this page that scikit-image can seamleassly process video stored as a numpy array, obtaining that would thus be ideal.

Answer

head7 picture head7 · Apr 20, 2015

Imageio python package should do what you want. Here is a python snippet using this package:

import pylab
import imageio
filename = '/tmp/file.mp4'
vid = imageio.get_reader(filename,  'ffmpeg')
nums = [10, 287]
for num in nums:
    image = vid.get_data(num)
    fig = pylab.figure()
    fig.suptitle('image #{}'.format(num), fontsize=20)
    pylab.imshow(image)
pylab.show()

enter image description here enter image description here

You can also directly iterate over the images in the file (see the documentation ):

for i, im in enumerate(vid):
    print('Mean of frame %i is %1.1f' % (i, im.mean()))

To install imageio you can use pip:

pip install imageio

An other solution would be to use moviepy (which use a similar code to read video), but I think imageio is lighter and does the job.


response to first comment

In order to check if the nominal frame rate is the same over the whole file, you can count the number of frame in the iterator:

count = 0
try:
    for _ in vid:
        count += 1
except RuntimeError:
    print('something went wront in iterating, maybee wrong fps number')
finally:
    print('number of frames counted {}, number of frames in metada {}'.format(count, vid.get_meta_data()['nframes']))


In [10]: something went wront in iterating, maybee wrong fps number
         number of frames counted 454, number of frames in metada 461

In order to display the timestamp of each frame:

try:
    for num, image in enumerate(vid.iter_data()):
        if num % int(vid._meta['fps']):
            continue
        else:
            fig = pylab.figure()
            pylab.imshow(image)
            timestamp = float(num)/ vid.get_meta_data()['fps']
            print(timestamp)
            fig.suptitle('image #{}, timestamp={}'.format(num, timestamp), fontsize=20)
            pylab.show()
except RuntimeError:
    print('something went wrong')