Opencv - Haar cascade - Face tracking is very slow

Question

I have developed a project to tracking face through camera using OpenCV library. I used haar cascade with haarcascade_frontalface_alt.xml to detect face.

My problem is if image capture from webcame doesn't contain any faces, process to detect faces is very slow so images from camera, which are showed continuosly to user, are delayed.

My source code:

void camera() 
{
    String face_cascade_name = "haarcascade_frontalface_alt.xml";
    String eye_cascade_name = "haarcascade_eye_tree_eyeglasses.xml";
    CascadeClassifier face_cascade;
    CascadeClassifier eyes_cascade;
    String window_name = "Capture - Face detection";
    VideoCapture cap(0);

    if (!face_cascade.load(face_cascade_name))
        printf("--(!)Error loading\n");

    if (!eyes_cascade.load(eye_cascade_name))
        printf("--(!)Error loading\n");

    if (!cap.isOpened()) 
    {
        cerr << "Capture Device ID " << 0 << "cannot be opened." << endl;
    } 
    else 
    {
        Mat frame;
        vector<Rect> faces;
        vector<Rect> eyes;
        Mat original;
        Mat frame_gray;
        Mat face;
        Mat processedFace;

        for (;;) 
        {
            cap.read(frame);
            original = frame.clone();    
            cvtColor(original, frame_gray, CV_BGR2GRAY);
            equalizeHist(frame_gray, frame_gray);
            face_cascade.detectMultiScale(frame_gray, faces, 2, 0,
                    0 | CASCADE_SCALE_IMAGE, Size(200, 200));

            if (faces.size() > 0)
                rectangle(original, faces[0], Scalar(0, 0, 255), 2, 8, 0);

            namedWindow(window_name, CV_WINDOW_AUTOSIZE);
            imshow(window_name, original);
        }

        if (waitKey(30) == 27)
            break;
    }
}

Answer 1

Haar classifier is relatively slow by nature. Furthermore, there is not much of optimization you can do to the algorithm itself because detectMultiScale is parallelized in OpenCV.

The only note about your code: do you really get some faces ever detected with minSize which equals to Size(200, 200)? Though surely, the bigger the minSize - the better the performance is.

Try scaling the image before detecting anything:

const int scale = 3;
cv::Mat resized_frame_gray( cvRound( frame_gray.rows / scale ), cvRound( frame_gray.cols / scale ), CV_8UC1 );
cv::resize( frame_gray, resized_frame_gray, resized_frame_gray.size() );
face_cascade.detectMultiScale(resized_frame_gray, faces, 1.1, 3, 0 | CASCADE_SCALE_IMAGE, Size(20, 20));

(don't forget to change minSize to more reasonable value and to convert detected face locations to real scale)

Image size reducing for 2, 3, 5 times is a great performance relief for any image processing algorithm, especially when it comes to some costly stuff like detection.

As it was mentioned before, if resizing won't do the trick, try fetching some other bottlenecks using a profiler.

And you can also switch to LBP classifier which is comparably faster though less accurate.

Hope it will help.