MATLAB - capturing video streams (MJPEG, rtsp, mpeg)

Alexey picture Alexey · Dec 29, 2011 · Viewed 9.8k times · Source

Has anyone worked with capturing video streams from IP cameras in MATLAB? For example to grab frames in MATLAB from rtsp://10.10.10.10:554/live.sdp (rtsp stream) or from http://x.x.x.x/axis-cgi/mjpg/video.cgi (mjpeg stream). MATLAB's Image Acquisition Toolbox does not currently support this. I found 2 options: 1) using mmread. However http stream reading is not supported under 64-bit MATLAB or 2) to write my own C++ function that grabs frames (I use OpenCV library) and then compile it into MATLAB MEX function. Any suggestions are appreciated.

Answer

Alexey picture Alexey · Dec 18, 2012

This is the answer I got from MATLAB support:

Unfortunately, you are correct that currently the Image Acquisition Toolbox does not support IP cameras. Regarding workarounds: 1. If mmread works for you, perhaps it is feasible for you to install a 32-bit MATLAB on your 64-bit machine. 2. Writing your own MEX driver should be a possible option. 3. IMREAD is able to obtain frames from IP cameras. It may be possible to utilize this capability and build a function that constructs the video stream. Although frame rate may be an issue.

I suggest implementing your own Matlab mex function to grab video frames. Here are some pointers to do so:

  1. OpenCV library is used to capture video streams from network cameras, see OpenCV with Network Cameras. Each IP camera may have a different API for accessing video streams (i.e. URL address). For example, http://10.10.10.10/axis-cgi/mjpg/video.cgi?resolution=800x600&.mjpg.
  2. Below is a link to collection and development kit of matlab mex functions for OpenCV library (thanks to Kota Yamaguchi): https://github.com/kyamagu/mexopencv. This library makes it easy to convert between OpenCV data types and mxArray. Here's an example:

    #include "mexopencv.hpp"
    void mexFunction( int nlhs, mxArray *plhs[],
                      int nrhs, const mxArray *prhs[] )
        {
        // Check arguments
        if (nlhs!=1 || nrhs!=1)
            mexErrMsgIdAndTxt("myfunc:invalidArgs", "Wrong number of arguments");
    
        // Convert MxArray to cv::Mat
        cv::Mat mat = MxArray(prhs[0]).toMat();
    
        // Do whatever you want
    
        // Convert cv::Mat back to mxArray*
        plhs[0] = MxArray(mat);
    }
    
  3. The application can be made asynchronous by using threads, where producer thread grabs frames from the camera and puts it into a circular buffer. Consumer thread, on the other hand, retrieves frames from the buffer and converts them into mxArray (matrix) output. See How to implement a circular buffer of cv::Mat objects (OpenCV)?. Circular buffer will need to be made thread safe, see Thread safe implementation of circular buffer.