Matching-based refinement rendering issues about camera_calibration HOT 6 CLOSED

I quickly verified on the sample dataset that the non-CUDA version of the intensity based refinement generally does seem to work in the current state of the code. The following image shows a typical result from there, where the red point is the final result, the white point is the result after intensity-based refinement, and the gray point is the initial prediction:

It has been a while that I worked on this, but this matches with what I remember as the typical difference between the two methods. Nevertheless, the difference might be larger on other datasets. The main possible reason that I can think of at the moment is the following:

The implementation of the intensity-based refinement does not optimize the whole homography that it uses, instead it optimizes for a translation only. Perhaps the predicted homography is inaccurate in your dataset, such that optimizing the translation is not sufficient? The symmetry-based refinement, on the other hand, optimizes the whole homography, so it can "fix" an inaccurate initial guess of the homography.

from camera_calibration.

I am attempting to render the same pattern (8 star segments) as the one provided in your sample dataset.

from camera_calibration.

I realized, in my custom implementation, that I forgot to account for the fact that PatternIntensityAt requires coordinates from [-0.5, 0.5], with pixel center convention. After adding the necessary transform, I am able to render the pattern correctly.

from camera_calibration.

Thanks for the update - can this issue be closed, or are the results still unstable after the fix?

from camera_calibration.

Actually I did have more questions about the results.

I've un-commented all the debug code here and here and all the rendered and transformed patterns work fine.

I've added statements to track loss and while the intensity-based refinement appears to be converging (loss reduces significantly with each iteration, eventually "converges", in my example from 41181.1 to 3393.69), the final results differ wildly from the symmetry-based method.

I've included the features refined from both methods below, where the red signifies results from the intensity based matching, and green signifies results from symmetry.

As same predictions and transformations are used between both methods, do you have any insight into why the output from the methods would differ by such a huge degree?

Steps I have tried:

1. Changing anti-aliasing sample #
2. Changing window_half_extent values

Thank you so much for your time!

from camera_calibration.

You're right - the matching based refinement was working well, but as it's only optimizing for the translation shift, and the original homography was off, it predicts a feature that isn't accurate.

In my pipeline I feed the homography predicted from the symmetry based refinement into the matching based refinement and the results look great. Thank you so much, I really appreciate the time and effort you took to help me with this.

from camera_calibration.