How to parse mjpeg http stream from ip camera?

Question

Given below is the code written for getting live stream from an IP Camera.

from cv2 import *
from cv2 import cv
import urllib
import numpy as np
k=0
capture=cv.CaptureFromFile("http://IPADDRESS of the camera/axis-cgi/mjpg/video.cgi")
namedWindow("Display",1)

while True:
    frame=cv.QueryFrame(capture)
    if frame is None:
        print 'Cam not found'
        break
    else:
        cv.ShowImage("Display", frame)
    if k==0x1b:
        print 'Esc. Exiting'
        break

On running the code the output that I am getting is:

Cam not found

Where am I going wrong? Also, why is frame None here? Is there some problem with the conversion?

Answer 1

import cv2
import urllib 
import numpy as np

stream = urllib.urlopen('http://localhost:8080/frame.mjpg')
bytes = ''
while True:
    bytes += stream.read(1024)
    a = bytes.find('\xff\xd8')
    b = bytes.find('\xff\xd9')
    if a != -1 and b != -1:
        jpg = bytes[a:b+2]
        bytes = bytes[b+2:]
        i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)
        cv2.imshow('i', i)
        if cv2.waitKey(1) == 27:
            exit(0)   

edit (explanation)

I just saw that you mention that you have c++ code that is working, if that is the case your camera may work in python as well. The code above manually parses the mjpeg stream without relying on opencv, since in some of my projects the url will not be opened by opencv no matter what I did(c++,python).

Mjpeg over http is multipart/x-mixed-replace with boundary frame info and jpeg data is just sent in binary. So you don't really need to care about http protocol headers. All jpeg frames start with marker 0xff 0xd8 and end with 0xff 0xd9. So the code above extracts such frames from the http stream and decodes them one by one. like below.

...(http)
0xff 0xd8      --|
[jpeg data]      |--this part is extracted and decoded
0xff 0xd9      --|
...(http)
0xff 0xd8      --|
[jpeg data]      |--this part is extracted and decoded
0xff 0xd9      --|
...(http)

edit 2 (reading from mjpg file)

Regarding your question of saving the file, yes the file can be directly saved and reopened using the same method with very small modification. For example you would do curl http://IPCAM > output.mjpg and then change the line stream=urllib.urlopen('http://localhost:8080/frame.mjpg')so that the code becomes this

import cv2
import urllib 
import numpy as np

stream = open('output.mjpg', 'rb')
bytes = ''
while True:
    bytes += stream.read(1024)
    a = bytes.find('\xff\xd8')
    b = bytes.find('\xff\xd9')
    if a != -1 and b != -1:
        jpg = bytes[a:b+2]
        bytes = bytes[b+2:]
        i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)
        cv2.imshow('i', i)
        if cv2.waitKey(1) == 27:
            exit(0)   

Of course you are saving a lot of redundant http headers, which you might want to strip away. Or if you have extra cpu power, maybe just encode to h264 first. But if the camera is adding some meta data to http header frames such as channel, timestamp, etc. Then it may be useful to keep them.

edit 3 (tkinter interfacing)

import cv2
import urllib 
import numpy as np
import Tkinter
from PIL import Image, ImageTk
import threading

root = Tkinter.Tk()
image_label = Tkinter.Label(root)  
image_label.pack()

def cvloop():    
    stream=open('output.mjpg', 'rb')
    bytes = ''
    while True:
        bytes += stream.read(1024)
        a = bytes.find('\xff\xd8')
        b = bytes.find('\xff\xd9')
        if a != -1 and b != -1:
            jpg = bytes[a:b+2]
            bytes = bytes[b+2:]
            i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)            
            tki = ImageTk.PhotoImage(Image.fromarray(cv2.cvtColor(i, cv2.COLOR_BGR2RGB)))
            image_label.configure(image=tki)                
            image_label._backbuffer_ = tki  #avoid flicker caused by premature gc
            cv2.imshow('i', i)
        if cv2.waitKey(1) == 27:
            exit(0)  

thread = threading.Thread(target=cvloop)
thread.start()
root.mainloop()