How to use the camera_tr_rig.yaml file in stereo calibration? about camera_calibration HOT 5 CLOSED

Regarding how to rotate and translate a 3D point from pattern to image

The comment in the file that you quoted briefly describes the general process already: "Each pose gives the image_tr_pattern transformation (i.e., pattern to image with right-multiplication). Quaternions are written as used by the Eigen library."

Typically, one would use a Quaternion implementation from some math library (such as for example the Eigen library mentioned in the comment) and compute a rotation matrix R from it. Given a 3D point p in the pattern coordinate system, it can then be transformed to the image by computing: R * p + t, where t is the translation from the image_tr_pattern pose.

Now, the comment in this file is unfortunately referencing the incorrect transformation, as hinted at here (sorry about that):

The poses in this file are actually the camera_tr_rig poses, as stated by the filename, rather than the image_tr_pattern poses, as stated by the comment. There should also be a file rig_tr_global.yaml that contains the rig_tr_global poses. These poses can be combined to get the camera_tr_global poses. For example, if the poses are converted to 4x4 homogeneous transformation matrices, then these can be multiplied:

camera_tr_global = camera_tr_rig * rig_tr_global

If you have individual R matrices and t vectors, these can also be combined:

R1 * (R2 * p + t2) + t1 ==
R1 * R2 * p + R1 * t2 + t1 ==
(R1 * R2) * p + (R1 * t2 + t1)

So, the combined transformation should have the rotation matrix computed as R1 * R2 and the translation vector computed as R1 * t2 + t1 (if I wrote it down correctly here).

The bundle-adjusted pattern points should be given in the "global" space. So, they can be rotated and translated into the image space by applying the camera_tr_global pose to them as described above.

Regarding the reprojection errors

The reprojection errors are influenced by many factors, such as for example the camera clarity and many more, so there is not necessarily something that can be done to achieve the same level of error magnitudes for one camera as for another camera. Perhaps some tuning of the detector and calibration parameters could improve them.

from camera_calibration.

The wording in the pose file comment should now be fixed in commit 3217358.

from camera_calibration.

If I understand it correctly, I may get the relative rotation and translation from camera1 to camera0 after the following steps.

1. Get the rotation matrices and translation vectors from camera_tr_rig.yaml.

// Convert a (normalized) quaternion to a rotation matrix.
Eigen::Matrix3d quaternion2rmat(const double qx, const double qy, const double qz, const double qw) {
  Eigen::Quaterniond q;
  q.x() = qx;
  q.y() = qy;
  q.z() = qz;
  q.w() = qw;
  return q.toRotationMatrix();
}

// Get the rotation matrices from the quaternions.
Eigen::Matrix3d R0 = quaternion2rmat(qx0, qy0, qz0, qw0);
Eigen::Matrix3d R1 = quaternion2rmat(qx1, qy1, qz1, qw1);

// Get the translation vectors.
Eigen::Vector3d t0(tx0, ty0, tz0);
Eigen::Vector3d t1(tx1, ty1, tz1);

2. So, given a 3D point p in the "rig" space, it can be transformed into p0 (in the camera0 space) and p1 (in the camera1 space) by:

p0 = R0 * p + t0
p1 = R1 * p + t1

3. The rotation R and translation t from camera1 to camera0 is given by:

p1 = R1 * p + t1
   = R1 * [R0.inv() * (p0 - t0)] + t1
   = R1 * R0.inv() * p0 - R1 * R0.inv() * t0 + t1

Eigen::Matrix3d R = R1 * R0.inv();
Eigen::Vector3d t = -R1 * R0.inv() * t0 + t1;

However, the resulted translation vector t seems wrong, what's the problem?

from camera_calibration.

I didn't spot a mistake in what you wrote. In case the wrong aspect about the result is only its scaling, then perhaps the dimensions given for the pattern grid might be off, since the final scaling depends on them. Otherwise, not sure what it is.

from camera_calibration.

Hi @puzzlepaint,

I may spot where the mistake comes from. The pattern was displayed on a computer display instead of an A4 paper to take images. However, when I ran your camera_calibration program to detect the features, I just used the default .yaml file of the pattern which describes the pattern in A4's physical size.

I replaced square_length_in_meters with the actual physical size on the display then I got the right calibrated translation vector.

Thanks a lot!

from camera_calibration.