OpenCV - Image Stitching

Question

I am using following code to stitch to input images. For an unknown reason the output result is crap! It seems that the homography matrix is wrong (or is affected wrongly) because the transformed image is like an "exploited star"! I have commented the part that I guess is the source of the problem but I cannot realize it. Any help or point is appriciated!

Have a nice day, Ali

void Stitch2Image(IplImage *mImage1, IplImage *mImage2) 
{ 

    // Convert input images to gray 
    IplImage* gray1 = cvCreateImage(cvSize(mImage1->width, mImage1->height), 8, 1); 

    cvCvtColor(mImage1, gray1, CV_BGR2GRAY); 
    IplImage* gray2 = cvCreateImage(cvSize(mImage2->width, mImage2->height), 8, 1); 

    cvCvtColor(mImage2, gray2, CV_BGR2GRAY); 
    // Convert gray images to Mat 
    Mat img1(gray1); 
    Mat img2(gray2); 
    // Detect FAST keypoints and BRIEF features in the first image 
    FastFeatureDetector detector(50); 
    BriefDescriptorExtractor descriptorExtractor; 
    BruteForceMatcher<L1<uchar> > descriptorMatcher; 
    vector<KeyPoint> keypoints1; 
    detector.detect( img1, keypoints1 ); 
    Mat descriptors1; 
    descriptorExtractor.compute( img1, keypoints1, descriptors1 );

/* Detect FAST keypoints and BRIEF features in the second image*/


    vector<KeyPoint> keypoints2; 
    detector.detect( img1, keypoints2 ); 
    Mat descriptors2; 
    descriptorExtractor.compute( img2, keypoints2, descriptors2 ); 
    vector<DMatch> matches; 
    descriptorMatcher.match(descriptors1, descriptors2, matches); 
    if (matches.size()==0) 
            return; 
    vector<Point2f> points1, points2; 
    for(size_t q = 0; q < matches.size(); q++) 
    { 
            points1.push_back(keypoints1[matches[q].queryIdx].pt); 
            points2.push_back(keypoints2[matches[q].trainIdx].pt); 
    } 
    // Create the result image 
    result = cvCreateImage(cvSize(mImage2->width * 2, mImage2->height), 8, 3); 
    cvZero(result); 

   // Copy the second image in the result image 

    cvSetImageROI(result, cvRect(mImage2->width, 0, mImage2->width, mImage2->height)); 
    cvCopy(mImage2, result); 
    cvResetImageROI(result); 

  // Create warp image 
    IplImage* warpImage = cvCloneImage(result); 
    cvZero(warpImage); 

  /************************** Is there anything wrong here!? *******************/ 
   // Find homography matrix 
    Mat H = findHomography(Mat(points1), Mat(points2), 8, 3.0); 
    CvMat HH = H; // Is this line converted correctly? 
   // Transform warp image 
    cvWarpPerspective(mImage1, warpImage, &HH); 
  // Blend 
    blend(result, warpImage);
  /*******************************************************************************/ 

    cvReleaseImage(&gray1); 
    cvReleaseImage(&gray2); 
    cvReleaseImage(&warpImage); 
}

Answer 1

This is what I would suggest you to try, in this order:

1) Use CV_RANSAC option for homography. Refer http://opencv.willowgarage.com/documentation/cpp/calib3d_camera_calibration_and_3d_reconstruction.html

2) Try other descriptors, particularly SIFT or SURF which ship with OpenCV. For some images FAST or BRIEF descriptors are not discriminating enough. EDIT (Aug '12): The ORB descriptors, which are based on BRIEF, are quite good and fast!

3) Try to look at the Homography matrix (step through in debug mode or print it) and see if it is consistent.

4) If above does not give you a clue, try to look at the matches that are formed. Is it matching one point in one image with a number of points in the other image? If so the problem again should be with the descriptors or the detector.

My hunch is that it is the descriptors (so 1) or 2) should fix it).