Question on The Physical method - camera calibration and fundamental matrix, or given essential matrix about stereo HOT 5 OPEN

I'm not sure I understand what the question is - are you saying you want to artificially construct the front view from two angled views?

from stereo.

Im sorry for not mention it in clear way.

My question is that now i have a convergence stereo camera and the object that i would like to capture is putting flat on the table, which mean the depth return from the object should be flat( or constant) but the depth return in gradient/varient increasing depth ( due to keystoneof the toe in stereo camera i would believe).

Mayb you help me how can i get constant depth return from flat surface instead of increasing/decreasing depth from the item.

from stereo.

I apologise for the late response. I think I would need to see your setup to fully understand but if the camera you get depth with respect to is angled from your object, then of course you'll naturally get a gradient of depth. Can you define a point between your cameras, that is perpendicular to the plane of your object, and transform the object's depth points in the camera to the reference frame of this point? That should achieve the end you want. Does that make send?

from stereo.

Hi, its okay for late response because i still in need your help haha!

Attached image is quite similar to my setup, and like you say the depth is respect to angled from my object.

Can you help me on how to define the plane between camera the is perpendicular to my object??

I would wan to set up my camera in converging but the result using code to adjust it become parallel (non gradient as the object is flat )

This is my result as can see from the color map the depth is not consistent (gradient due to angle ) but i would like to adjust this

from stereo.

Well, if your origin is at camera 1 - call this C1 - and you have a transform T that is the translation and rotation of camera 2 with respect to camera 1, then apply half of that transform to get the midpoint. You'd then be looking at that particular plane head-on. More generally, draw a perpendicular ray from your surface to the line in between your two cameras, and then where those two lines intersect, use that.

Remember that if T is an SE3 - which if you are using my code, it should be - then you need to take the log to move from the group space to the algebra space, half that 6-vector to get the half rotation and half translation, and then take the exponential again and apply this as a left-update to the identity transform from the origin.

from stereo.