How to calculate OpenCV camera projectionMatrix in python

Question

I am trying to calculate projection_matrix using OpenCV 2.4 in Python 2.7 for my camera (I am using ps eye). I need it for cv2.triangulatePoints(). I already did the calibration using cv2.calibrateCamera() (using calibrate.py from OpenCV examples) so I have rms, camera_matrix, dist_coefs, rvecs and tvecs.

But I have a problem actually calculating projection_matrix from these parameters (I did not find any Python examples online).

PS: Do I have to calibrate every ps eye camera ? I have 3 and I would like to track object in 3D space.

Answer 1

If you have only one camera, projection matrix should be equal to camera_matrix. There is only one complication.

The cv2.triangulatePoints is defined to work with 2 views from 2 different cameras.

Documentation also states that

The function reconstructs 3-dimensional points (in homogeneous coordinates) by using their observations with a stereo camera. Projections matrices can be obtained from stereoRectify().

So yes, you have to calibrate each camera and calibrate every pair of cameras in order to retrieve each camera matrix and the rotation matrix and the translation vector from one camera to the "main camera".

For a given couple of cameras, with K1 and K2 the camera matrices, it is true that

The projection matrix of the main camera (the camera is the world reference system) is

P1 = K1*[I | z]

where I is the identy matrix and z is a 0,0,0 vector in the fourth column. You could think something like

  1 0 0 0
  0 1 0 0 
  0 0 1 0

If R is the rotation matrix between the 2 cameras and t the distance between the two cameras, the second projection matrix is

P2 = K2*[R | t]

In python, if you can not obtain the matrices from stereoRectify, one method to do it manually is

import numpy as np
P = np.concatenate((np.dot(K,R),np.dot(K,t)), axis = 1)