[feature-request] also get the 6doF camera odometry about record3d HOT 8 CLOSED

that is awesome, thank you!

from record3d.

I'm not sure if it's possible to have 640x480 px TrueDepth camera stream running via AVCaptureSession (which is what Record3D does now) and also having an ARKit ARWorldTrackingConfiguration session running at the same time.

Got it. I did not know this. Thanks a lot for the information.

from record3d.

The next version of Record3D (1.7.2) will stream the ARKit world pose (quaternion + world position). It should be released by the end of the next week.

from record3d.

Please update Record3D to version 1.7.2, update the record3d to version 1.3.1; camera pose data for LiDAR are available.

For Python, see (read the comment)

And for C++, see

from record3d.

Hi @marek-simonik
I am running this on an iPhone 12 mini, which doesn't have LiDAR. So, it does report xyz to always be 0, 0, 0.

I am wondering why the requirement for LiDAR -- actually if you do LiDAR based, it will be more inferior than setting a WorldTracking session which does sensor fusion among camera, lidar and intertial sensors.

Wouldn't it be better and more widely supported (on the non-Pro models too) if you created a ARWorldTrackingConfiguration() session and then get the current ARFrame's frame.camera.transform?

from record3d.

Record3D currently uses ARKit for access to LiDAR data (hence the availability of estimated pose data), but FaceID streaming does not use ARKit; the last time I checked, the ARKit API did not provide 640x480 px resolution option for FaceID depth maps.

from record3d.

I agree that streaming of phone's 6dof odometry is completely independent of FaceID depth maps.

Record3D currently uses ARKit for access to LiDAR data (hence the availability of estimated pose data)

What I am saying is, the estimated 6dof pose is available in ARKit even when there is no LiDAR. It fuses information from the back-camera in low-fps and the IMU sensor. That estimated 6dof pose wont correspond to the FaceID depth maps exactly, but we can compute a fixed transform from the ARKit's ARWorldTrackingConfiguration's frame.camera.transform and the FaceID depth transform.

from record3d.

I'm not sure if it's possible to have 640x480 px TrueDepth camera stream running via AVCaptureSession (which is what Record3D does now) and also having an ARKit ARWorldTrackingConfiguration session running at the same time.

If I'm not mistaken, then ARKit does internally use the AVCaptureSession too and since iOS doesn't allow more than 1 AVCaptureSession running at the same time, the TrueDepth camera's AVCaptureSession would automatically stop as soon as I would start an ARKit session.

I currently cannot think of a way how to get 640x480 px TrueDepth depth map stream and ARKit's pose estimate (estimated using the rear cameras with IMU sensor fusion, as you say) on all devices with the TrueDepth camera — am I missing something obvious?

from record3d.