A large drifting with Realsense D435i about vins-fusion HOT 17 OPEN

Hello 👋🏻 ! Many issues here ....
First consider switching on the sdk 2.25 and ros corresponding 2.2.8 because they fix many problem on the imu part. Then use the last production firmware 11.2.6.250

-Then do not use their linear interpolation implementation : because as you will be able to see it is buggy ! It is a mixed interpolation between fast and slow stream so as you will be able to see time stamp are messed up !

• Then consider using imu_tools to compute Allan parameters of your device : you will have higher value than those provided by the config file in the github repository of vins fusion : impressive
  -Then I will recommend you to use global time flag in the ros node of RealSense to get time of capture instead of time of reception message and better do not use ros node provided by realsense but one called rs4se which makes a good implementation for visual inertial system
• use Kalibr to make the calibration between cameras and camera/imu
• take into account that vins convention seems to wait for a z up body frame in the camera to body projection matrix
• vins is waiting for imu data that is corrected for misalignment and bias, the realsense documentation do state that the operation is performed in the firmware after performing the es-imu-calibration script. But as I could have seen, there is a bug in the correction, so you will have to disable this correction by setting the option “enable motion correction “ and implement it by yourself in the realsense node
  !
  Hopefully it can help anyone

from vins-fusion.

I have just run vins successfully with a realsense D435i camera. Stereo + IMU config is fine.
The success relies heavily on versions.

As a suggestion, you may try:
ubuntu 18.04 + ROS melodic (I think 16.04+ROS Kinetic shoud also work)
Linux kernel 4.15.0-72-generic (5.0 didn't work for me, be sure to reinstall librealsense2-dkms after changing kernel)
librealsense2 2.19.0-0~realsense0.615
librealsense2-dev 2.19.0-0~realsense0.615 (2.19.2 didn't work for me)
librealsense2-dkms 1.3.10-0ubuntu1
realsense2_camera 2.2.3 (just git checkout 2.2.3 and catkin_make)
a realsense launch file at https://github.com/mhc994/VINS-Fusion/blob/wz_fly/config/realsense_d435i/vins_infra.launch (I disabled useless depth and color topics)

good luck

from vins-fusion.

感觉遇到的问题跟你一样

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

But on my computer， VINS cant provide satisfactory result.
The path is very different from the real scene

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

I using dual-camera and Calibrate them by kalibr.

My cameras are 120degree angle.

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

I tried to calibrate my cameras by different types. such as mei & pinhole

from vins-fusion.

@quxichao Please check the firmware version you used. The confing is for SDK2.19. The version of Realsense SDK higher than 2.19 exists some issue with its IMU. You can directly using the config file with SDK 2.19.

If using dual-camera only without IMU。 Will this problem arise?

You can try it yourself. If your usage is right, this problem should not happen.

Config file as follows, can you give me some tips and advice?:

from vins-fusion.

@quxichao @HKVINS89757 The same problem, has anyone solved it? I tried switching SDK to version 2.18.1, but it was worse.
我换了SDK版本后, 还是没有解决这个问题

from vins-fusion.

@quxichao @HKVINS89757 The same problem, has anyone solved it? I tried switching SDK to version 2.18.1, but it was worse.

说中文

from vins-fusion.

@pjrambo Which Intel firmware version are you guys using on D435i?

from vins-fusion.

I have same problem with realsense D435i (Mono Camera + IMU).
SDK version is 2.23.0

Here is a link to rosbag and configuration file:
https://drive.google.com/open?id=1W9tkiHEgJiWnozo_7JZIxDuND5SF3lx1

from vins-fusion.

Same problem with 2.25 SDK. Could anyone give a solution?

from vins-fusion.

Hi @mhc994
Im trying d435i and succefully make VINS working with 1 imu and 1 camera but the stereo is not working.
Im using SDK 2.36 and rs4se to grab the data.
The calibration made with kalibr is good with very low errors, but still I have awful result when using stereo.
I think that one issue can be the calibration of the IMU that is different from yours, my calib is the right one.
As you can see the IMU position is rotated:

Do you experienced the same issue with the different versions you used?

Im going to try your setup but I have problems to start my pc with linux kernel 4.15, can you tell me what type of problem you had with kernel 5?

thanks

from vins-fusion.

Hi @mhc994
Im trying d435i and succefully make VINS working with 1 imu and 1 camera but the stereo is not working.
Im using SDK 2.36 and rs4se to grab the data.
The calibration made with kalibr is good with very low errors, but still I have awful result when using stereo.
I think that one issue can be the calibration of the IMU that is different from yours, my calib is the right one.
As you can see the IMU position is rotated:

Do you experienced the same issue with the different versions you used?

Im going to try your setup but I have problems to start my pc with linux kernel 4.15, can you tell me what type of problem you had with kernel 5?

thanks

As @mhc994 confirms it, and as i said in my previous post VINS convention waits for IMU with Z up so apply the corresponding rotation to your Kalibr matrix parameter.

from vins-fusion.