MATLAB - capturing video streams (MJPEG, rtsp, mpeg)

Question

Has anyone worked with capturing video streams from IP cameras in MATLAB? For example to grab frames in MATLAB from rtsp://10.10.10.10:554/live.sdp (rtsp stream) or from http://x.x.x.x/axis-cgi/mjpg/video.cgi (mjpeg stream). MATLAB's Image Acquisition Toolbox does not currently support this. I found 2 options: 1) using mmread. However http stream reading is not supported under 64-bit MATLAB or 2) to write my own C++ function that grabs frames (I use OpenCV library) and then compile it into MATLAB MEX function. Any suggestions are appreciated.

Answer 1

This is the answer I got from MATLAB support:

Unfortunately, you are correct that currently the Image Acquisition Toolbox does not support IP cameras. Regarding workarounds: 1. If mmread works for you, perhaps it is feasible for you to install a 32-bit MATLAB on your 64-bit machine. 2. Writing your own MEX driver should be a possible option. 3. IMREAD is able to obtain frames from IP cameras. It may be possible to utilize this capability and build a function that constructs the video stream. Although frame rate may be an issue.

I suggest implementing your own Matlab mex function to grab video frames. Here are some pointers to do so:

1. OpenCV library is used to capture video streams from network cameras, see OpenCV with Network Cameras. Each IP camera may have a different API for accessing video streams (i.e. URL address). For example, http://10.10.10.10/axis-cgi/mjpg/video.cgi?resolution=800x600&.mjpg.

2. Below is a link to collection and development kit of matlab mex functions for OpenCV library (thanks to Kota Yamaguchi): https://github.com/kyamagu/mexopencv. This library makes it easy to convert between OpenCV data types and mxArray. Here's an example:

   #include "mexopencv.hpp"
   void mexFunction( int nlhs, mxArray *plhs[],
                     int nrhs, const mxArray *prhs[] )
       {
       // Check arguments
       if (nlhs!=1 || nrhs!=1)
           mexErrMsgIdAndTxt("myfunc:invalidArgs", "Wrong number of arguments");
   
       // Convert MxArray to cv::Mat
       cv::Mat mat = MxArray(prhs[0]).toMat();
   
       // Do whatever you want
   
       // Convert cv::Mat back to mxArray*
       plhs[0] = MxArray(mat);
   }
   

3. The application can be made asynchronous by using threads, where producer thread grabs frames from the camera and puts it into a circular buffer. Consumer thread, on the other hand, retrieves frames from the buffer and converts them into mxArray (matrix) output. See How to implement a circular buffer of cv::Mat objects (OpenCV)?. Circular buffer will need to be made thread safe, see Thread safe implementation of circular buffer.