Libraries to get depth map from stereo images?
I'm going to set up some devices to perform real-time 3D motion tracking. My first idea to do so is to use a pair of cameras to take stereo image and calculate the depth map to get the 3D data I needed.
Are there any good open source libraries (C/C++) available and is fast enough for real-time(~12-24fps)? I've found 3D Reconstruction using Stereo Vision and EStereo in SourceForge, have anyone tried them?
Or any algorithm suggestion that I can implement?
Answer
Before you start, consider the fact that stereo vision is not just a software problem. The calibration of your pair of camera's is critically important to the accuracy of your depth map. For best results you want two identical camera's pointed exactly parallel, with exactly the same lenses on both cameras, with the CCD's of the camera's lying on one plane in 3D space. This is very difficult to achieve with just any random pair of camera's. Especially if they come from the cheaper segment.
If all you're interested in is using the 3D depth map for 3D motion tracking then there are precalibrated stereo camera's you can buy that typically come with software that allows you to extract the 3D map directly. I've had good experience with the Bumblebee camera from Point Grey Research but probably there are other good systems as well. This will allow you to concentrate on the 3D tracking. Otherwise just getting a usable depth map from a camera pair will eat up a lot of your time.