image coordinate to world coordinate opencv
I calibrated my mono camera using opencv. Now I know the camera intrinsic matrix and distortion coefs [K1, K2, P1 ,P2,K3 ,K4, K5, K6] of my camera. Assuming that camera is place in [x, y, z] with [Roll, Pitch, Yaw] rotations. how can I get each pixel in world coordinate when the camera is looking on the floor [z=0].
Answer
I suggest you start by studying the pinhole camera model, which models the process through which a point in the 3D world is mapped to the image plane using the camera intrinsic parameters. As you'll see, this process is not one-to-one, and thus it usually cannot be inverted (image to 3D), unless you have depth information (which you have, since you said the points are located at z=0). This particular case is mentioned on slide 27 of this presentation. Previous lectures explain in details the image formation process, and can be used as a first reference to actually determine the transformation from image to world coordinates. Szeliski's book and this PDF are also great resources.