5

I wish to convert a MATLAB stereoParameters structure to intrinsics and extrinsics matrices to use in OpenCV's stereoRectify.

If I understood http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html and http://mathworks.com/help/vision/ref/stereoparameters-class.html , stereoParameters.CameraParameters1 and stereoParameters.CameraParameters2 store the intrinsic matrices and the other members of stereoParameters the extrinsic ones.

I think I got this mapping

Intrinsics:

  • cameraMatrix1 = stereoParameters.CameraParameters1.IntrinsicMatrix'
  • cameraMatrix2 = stereoParameters.CameraParameters2.IntrinsicMatrix'
  • distCoeffs1 = [stereoParameters.CameraParameters1.RadialDistortion(1:2), stereoParameters.CameraParameters1.TangentialDistortion, stereoParameters.CameraParameters1.RadialDistortion(3)]
  • distCoeffs2 = [stereoParameters.CameraParameters2.RadialDistortion(1:2), stereoParameters.CameraParameters2.TangentialDistortion, stereoParameters.CameraParameters2.RadialDistortion(3)]

Extrinsics:

  • R = stereoParameters.RotationOfCamera2'
  • T = stereoParameters.TranslationOfCamera2'

Is that correct, so far?

Still, I can't see how to get

  • R1 (3x3)
  • R2 (3x3)
  • P1 (3x4)
  • P2 (3x4)
  • Q (4x4)

matrices from the rest of stereoParameters.

Is there an existing converter I can use, and if not, what are the formulas?

Gnubie
  • 2,587
  • 4
  • 25
  • 38
  • I found a formula for Q at http://stackoverflow.com/a/28317841 . – Gnubie Oct 10 '17 at 11:18
  • I want to do the opposite and convert from OpenCV to Matlab yml files. Have you found a conversion? Anything could be helpful at this point. – god_kane Sep 13 '18 at 16:14

3 Answers3

2

As you already found out, both camera matrices need to be transposed due to a different notation in MATLAB

MATLAB Camera Intrinsic Matrix

and OpenCV

OpenCV Camera Intrinsic Matrix

The same applies for the rotation matrix and the translation vector between the cameras: stereoParams.RotationOfCamera2 and stereoParams.TranslationOfCamera2 need to be transposed in order to obtain OpenCV's R matrix and T vector.

(Quick validation: R should be close to an identity matrix if the cameras are almost parallel and the first element of T should match your baseline between the cameras in millimeters.)

OpenCV's distortion coefficient vector is composed of MATLAB's two tangential distortion coefficients followed by the two radial distortion coefficients.

That said, I was able to compute correct R1, R2, P1, P2 and Q using (R1, R2, P1, P2, Q, leftROI, rightROI) = cv2.stereoRectify(leftCamMatrix, leftDistCoeffs, rightCamMatrix, rightDistCoeffs, imageSize, R, T, None, None, None, None, None, cv2.CALIB_ZERO_DISPARITY, 0)

Note that for data type reasons, the disparity values obtained using OpenCV's stereo matcher need to be divided by 16 and the coordinates in the 3d point cloud returned by cv2.reprojectImageTo3D need to be divided by 64 to obtain metric values.

(Quick validation: when grabbing the coordinates of the same object in the rectified left and right image, the y-coordinates should be almost equal and you should be able to compute the object distance in meters with f*B/(x_right-x_left)/1000 with f being a combined focal length of a virtual camera in Q and B the baseline in millimeters.)

F1iX
  • 823
  • 6
  • 12
  • Thanks for this, but can you elaborate on the divide by 16 and divide by 64 elements of this comment? This is the only part I don't understand. Thanks! – kevinkayaks Aug 03 '18 at 03:36
  • 1
    I think the order of Tangential and Radial should be reversed in your answer. Check [this](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) – Rakshit Kothari Mar 15 '19 at 10:28
  • @RakshitKothari what do you mean? The order in the function's parameters is R, T just like above? – F1iX Mar 15 '19 at 15:06
  • 2
    OpenCV's distortion coefficient vector is composed of MATLAB's two tangential distortion coefficients followed by the two radial distortion coefficients. I think you need to reverse that statement to two radial coeffs followed by two tangential coeffs. – Rakshit Kothari Mar 15 '19 at 19:46
  • Sorry for comment this answer, but I try your answer and It work exactly fine except the distortion coefficients. What @RakshitKothari says is right, Distortion vector OpenCV started by two radial coefficients followed by two tangential coefficients. You may wanna edit that section so future reader like me didn't make a mistake – Albert H M Jul 01 '21 at 11:13
2

You can use the stereoRectify function in OpenCV to obtain R1, R2, P1, P2, Q given cameraMatrix1, cameraMatrix2, distCoeffs1, distCoeffs2, R & T.

In C++ it would be

cv::Mat R1, R2, P1, P2, Q; cv::Rect validRoi[2]; cv::stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imSize, R, T, R1, R2, P1, P2, Q, CV_CALIB_ZERO_DISPARITY, 0, imSize, &validRoi[0], &validRoi[1]);

One important thing to note is that the matrices cameraMatrix1, cameraMatrix2 and R need to be transposed when copying them from their MATLAB counterparts.

(I put this in bold as it cost 2 days to figure out why my rectification wasn't working when I converted it from MATLAB to C++ OpenCV)

souljaboy764
  • 31
  • 4
0

https://stackoverflow.com/a/28317841 gives the formula for the Q matrix:

Q formula

Tx is from matrix T. cx, cy and cx' are from the camera matrices. f is some sensible combination of their x and y focal lengths.

Still dunno how to get P1, P2, R1 and R2 though. Anybody?

Gnubie
  • 2,587
  • 4
  • 25
  • 38