OpenCV Transform using Chessboard
I have only just started experimenting with OpenCV a little bit. I have a setup of an LCD with a static position, and I'd like to extract what is being displayed on the screen from the image. I've seen the chessboard pattern used for calibrating a camera, but it seems like that is used to undistort the image, which isn't totally what I want to do.
I was thinking I'd display the chessboard on the LCD and then figure out the transformations needed to convert the image of the LCD into the ideal view of the chessboard directly overhead and cropped. Then I would store the transformations, change what the LCD is displaying, take a picture, perform the same transformations, and get the ideal view of what was now being displayed.
I'm wondering if that sounds like a good idea? Is there a simpler way to achieve what I'm trying to do? And any tips on the functions I should be using to figure out the transformations, perform them, store them (maybe just keep the transform matrices in memory or write them to file), etc?
Answer
I'm not sure I understood correctly everything you are trying to do, but bear with me.
Some cameras have lenses that cause a little distortion to the image, and for this purpose OpenCV offers methods to aid in the camera calibration process.
Practically speaking, if you want to write an application that will automatically correct the distortion in the image, first, you need to discover what are the magical values that need to be used to undo this effect. These values come from a proper calibration procedure.
The chessboard image is used together with an application to calibrate the camera. So, after you have an image of the chessboard taken by the camera device, pass this image to the calibration app. The app will identify the corners of the squares and compute the values of the distortion and return the magical values you need to use to counter the distortion effect. At this point, you are interested in 2 variables returned by calibrateCamera(): they are cameraMatrix and distCoeffs. Print them, and write the data on a piece of paper.
At the end, the system you are developing needs to have a function/method to undistort the image, where these 2 variables will be hard coded inside the function, followed by a call to cv::undistort() (if you are using the C++ API of OpenCV):
cv::Mat undistorted;
cv::undistort(image, undistorted, cameraMatrix, distCoeffs);
and that's it.
Detecting rotation automatically might be a bit tricky, but the first thing to do is find the coordinates of the object you are interested in. But if the camera is in a fixed position, this is going to be easy.
For more info on perspective change and rotation with OpenCV, I suggest taking a look at these other questions:
Executing cv::warpPerspective for a fake deskewing on a set of cv::Point
Affine Transform, Simple Rotation and Scaling or something else entirely?
Rotate cv::Mat using cv::warpAffine offsets destination image